Crossbow: Scaling Deep Learning with Small Batch Sizes on Multi-GPU Servers
- Submitting institution
-
University of Edinburgh
- Unit of assessment
- 11 - Computer Science and Informatics
- Output identifier
- 156250739
- Type
- D - Journal article
- DOI
-
10.14778/3342263.3342276
- Title of journal
- Proceedings of the VLDB Endowment (PVLDB)
- Article number
- -
- First page
- 1399
- Volume
- 12
- Issue
- 11
- ISSN
- 2150-8097
- Open access status
- Technical exception
- Month of publication
- July
- Year of publication
- 2019
- URL
-
-
- Supplementary information
-
-
- Request cross-referral to
- -
- Output has been delayed by COVID-19
- No
- COVID-19 affected output statement
- -
- Forensic science
- No
- Criminology
- No
- Interdisciplinary
- No
- Number of additional authors
-
5
- Research group(s)
-
A - Computer Systems
- Citation count
- 4
- Proposed double-weighted
- No
- Reserve for an output with double weighting
- No
- Additional information
- This paper introduces a novel deep learning system: CrossBow. CrossBow is the first of its kind that can fully utilise a multi-GPU server in training AI models with accuracy-friendly small batch sizes. It offers the missed system support for scalable small-batch training, which has been requested by leading AI practitioners (Jeff Dean, David Patterson, Cliff Young, IEEE Micro’18). The significance of CrossBow has led to an invited submission to a prestigious journal: ACM SIGOPS Operating System Review. The practical values of CrossBow further attract fundings from Huawei and Alibaba (700K GBP in total) to expand its design.
- Author contribution statement
- -
- Non-English
- No
- English abstract
- -