Making state explicit for imperative big data processing
- Submitting institution
-
The University of Kent
- Unit of assessment
- 11 - Computer Science and Informatics
- Output identifier
- 9352
- Type
- E - Conference contribution
- DOI
-
-
- Title of conference / published proceedings
- Proceedings of the 2014 USENIX Conference on USENIX Annual Technical Conference
- First page
- 49
- Volume
- -
- Issue
- -
- ISSN
- -
- Open access status
- -
- Month of publication
- June
- Year of publication
- 2014
- URL
-
https://kar.kent.ac.uk/49288/
- Supplementary information
-
-
- Request cross-referral to
- -
- Output has been delayed by COVID-19
- No
- COVID-19 affected output statement
- -
- Forensic science
- No
- Criminology
- No
- Interdisciplinary
- No
- Number of additional authors
-
3
- Research group(s)
-
-
- Citation count
- -
- Proposed double-weighted
- No
- Reserve for an output with double weighting
- No
- Additional information
- Imperative languages are commonly used to implement machine learning algorithms but are difficult to scale; whereas, data parallel processing frameworks are highly scalable but are based on complex programming models. This paper is significant because we propose an execution model, Stateful Dataflow Graphs (SDGs), that makes it easy to translate imperative programmes to distributed and fault tolerant representations. We prove SDGs can outperform existing data parallel processing frameworks on various popular algorithms, including logistic regression and collaborative filtering.
- Author contribution statement
- -
- Non-English
- No
- English abstract
- -
