Making State Explicit for Imperative Big Data Processing
- Submitting institution
-
Imperial College of Science, Technology and Medicine
- Unit of assessment
- 11 - Computer Science and Informatics
- Output identifier
- 2270
- Type
- E - Conference contribution
- DOI
-
-
- Title of conference / published proceedings
- Proceedings of USENIX ATC ’14: 2014 USENIX Annual Technical Conference
- First page
- 49
- Volume
- -
- Issue
- -
- ISSN
- -
- Open access status
- -
- Month of publication
- June
- Year of publication
- 2014
- URL
-
-
- Supplementary information
-
-
- Request cross-referral to
- -
- Output has been delayed by COVID-19
- No
- COVID-19 affected output statement
- -
- Forensic science
- No
- Criminology
- No
- Interdisciplinary
- No
- Number of additional authors
-
3
- Research group(s)
-
-
- Citation count
- -
- Proposed double-weighted
- No
- Reserve for an output with double weighting
- No
- Additional information
- The paper was the first to question the predominant opinion that data in distributed dataflow systems must be immutable. Ideas on mutable state have now been adopted by all major open-source distributed dataflow systems (Apache Flink, Apache Spark), and companies including LinkedIn, evidenced by a CIDR'15 paper (http://cidrdb.org/cidr2015/Papers/CIDR15_Paper25u.pdf). The work also resulted in follow-on funding from BP (£300K) on supporting exploratory big data processing (subsequently published at SIGMOD'16; https://doi.org/10.1145/2882903.2882962). Ideas from stateful processing also led to faculty awards by VMware ($75K) and Google ($85K), and a keynote invitation at ACM DEBS'16 (https://2016.debs.org/keynote-speakers.html#pietzuch). USENIX ATC'14 acceptance rate: 14.9%/241.
- Author contribution statement
- -
- Non-English
- No
- English abstract
- -
