Elastic Coordination for Scalable Machine Learning

Project Period: 2019-03-01 – 2024-02-29
Externally Funded
Acronym
ScaleML
Principal Investigator
Dan-Adrian Alistarh
Department(s)
Alistarh Group
Grant Number
805223
Funding Organisation
EC/H2020

21 Publications

2021 | Journal Article | IST-REx-ID: 8286 | OA
Dynamic averaging load balancing on cycles
D.-A. Alistarh, G. Nadiradze, A. Sabour, Algorithmica (2021).
View | Files available | DOI | arXiv
 
2019 | Conference Paper | IST-REx-ID: 7201 | OA
SparCML: High-performance sparse communication for machine learning
C. Renggli, S. Ashkboos, M. Aghagolzadeh, D.-A. Alistarh, T. Hoefler, in:, International Conference for High Performance Computing, Networking, Storage and Analysis, SC, ACM, 2019.
View | DOI | Download Preprint (ext.) | arXiv
 
2019 | Conference Paper | IST-REx-ID: 7542 | OA
Powerset convolutional neural networks
C. Wendler, D.-A. Alistarh, M. Püschel, in:, Neural Information Processing Systems Foundation, 2019, pp. 927–938.
View | Download Published Version (ext.) | arXiv
 
2020 | Conference Paper | IST-REx-ID: 7636
Non-blocking interpolation search trees with doubly-logarithmic running time
T.A. Brown, A. Prokopec, D.-A. Alistarh, in:, Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, ACM, 2020, pp. 276–291.
View | DOI
 
2021 | Conference Paper | IST-REx-ID: 10854
Input-dynamic distributed algorithms for communication networks
K.-T. Foerster, J. Korhonen, A. Paz, J. Rybicki, S. Schmid, in:, Abstract Proceedings of the 2021 ACM SIGMETRICS / International Conference on Measurement and Modeling of Computer Systems, Association for Computing Machinery, 2021, pp. 71–72.
View | Files available | DOI
 
2021 | Journal Article | IST-REx-ID: 10855 | OA
Input-dynamic distributed algorithms for communication networks
K.-T. Foerster, J. Korhonen, A. Paz, J. Rybicki, S. Schmid, Proceedings of the ACM on Measurement and Analysis of Computing Systems 5 (2021) 1–33.
View | Files available | DOI | Download Preprint (ext.) | arXiv
 
2022 | Conference Paper | IST-REx-ID: 11180 | OA
Multi-queues can be state-of-the-art priority schedulers
A. Postnikova, N. Koval, G. Nadiradze, D.-A. Alistarh, in:, Proceedings of the 27th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, Association for Computing Machinery, 2022, pp. 353–367.
View | DOI | Download Preprint (ext.) | arXiv
 
2022 | Conference Paper | IST-REx-ID: 11183 | OA
Beyond distributed subgraph detection: Induced subgraphs, multicolored problems and graph parameters
A. Nikabadi, J. Korhonen, in:, Q. Bramas, V. Gramoli, A. Milani (Eds.), 25th International Conference on Principles of Distributed Systems, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2022.
View | Files available | DOI
 
2022 | Conference Paper | IST-REx-ID: 11184 | OA
Fast graphical population protocols
D.-A. Alistarh, R. Gelashvili, J. Rybicki, in:, Q. Bramas, V. Gramoli, A. Milani (Eds.), 25th International Conference on Principles of Distributed Systems, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2022.
View | Files available | DOI | arXiv
 
2019 | Conference Paper | IST-REx-ID: 6673 | OA
Efficiency guarantees for parallel incremental algorithms under relaxed schedulers
D.-A. Alistarh, G. Nadiradze, N. Koval, in:, 31st ACM Symposium on Parallelism in Algorithms and Architectures, ACM Press, 2019, pp. 145–154.
View | Files available | DOI | Download Preprint (ext.) | arXiv
 
2020 | Conference Paper | IST-REx-ID: 8722 | OA
Taming unbalanced training workloads in deep learning with partial collective operations
S. Li, T.B.-N. Tal Ben-Nun, S.D. Girolamo, D.-A. Alistarh, T. Hoefler, in:, Proceedings of the 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, Association for Computing Machinery, 2020, pp. 45–61.
View | DOI | Download Preprint (ext.) | arXiv
 
2021 | Journal Article | IST-REx-ID: 8723 | OA
Breaking (global) barriers in parallel stochastic optimization with wait-avoiding group averaging
S. Li, T.B.-N. Tal Ben-Nun, G. Nadiradze, S.D. Girolamo, N. Dryden, D.-A. Alistarh, T. Hoefler, IEEE Transactions on Parallel and Distributed Systems 32 (2021).
View | DOI | Download Preprint (ext.) | arXiv
 
2020 | Conference Paper | IST-REx-ID: 8724 | OA
On the sample complexity of adversarial multi-source PAC learning
N.H. Konstantinov, E. Frantar, D.-A. Alistarh, C. Lampert, in:, Proceedings of the 37th International Conference on Machine Learning, ML Research Press, 2020, pp. 5416–5425.
View | Files available | arXiv
 
2020 | Conference Paper | IST-REx-ID: 8725 | OA
The splay-list: A distribution-adaptive concurrent skip-list
V. Aksenov, D.-A. Alistarh, A. Drozdova, A. Mohtashami, in:, 34th International Symposium on Distributed Computing, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2020, p. 3:1-3:18.
View | Files available | DOI | arXiv
 
2021 | Conference Paper | IST-REx-ID: 10217 | OA
Lower bounds for shared-memory leader election under bounded write contention
D.-A. Alistarh, R. Gelashvili, G. Nadiradze, in:, 35th International Symposium on Distributed Computing, Schloss Dagstuhl - Leibniz Zentrum für Informatik, 2021.
View | Files available | DOI
 
2021 | Conference Paper | IST-REx-ID: 10219 | OA
Brief announcement: Sinkless orientation is hard also in the supported LOCAL model
J. Korhonen, A. Paz, J. Rybicki, S. Schmid, J. Suomela, in:, 35th International Symposium on Distributed Computing, Schloss Dagstuhl - Leibniz Zentrum für Informatik, 2021.
View | Files available | DOI | arXiv
 
2021 | Thesis | IST-REx-ID: 10429 | OA
On achieving scalability through relaxation
G. Nadiradze, On Achieving Scalability through Relaxation, IST Austria, 2021.
View | Files available | DOI
 
2021 | Conference Paper | IST-REx-ID: 10432 | OA
Elastic consistency: A practical consistency model for distributed stochastic gradient descent
G. Nadiradze, I. Markov, B. Chatterjee, V. Kungurtsev, D.-A. Alistarh, in:, Proceedings of the AAAI Conference on Artificial Intelligence, 2021, pp. 9037–9045.
View | Files available | Download Published Version (ext.) | arXiv
 
2021 | Conference Paper | IST-REx-ID: 10435 | OA
Asynchronous decentralized SGD with quantized and local updates
G. Nadiradze, A. Sabour, P. Davies, S. Li, D.-A. Alistarh, in:, 35th Conference on Neural Information Processing Systems, NeurIPS, 2021.
View | Files available | Download Published Version (ext.) | arXiv
 
2020 | Conference Paper | IST-REx-ID: 9631 | OA
Scalable belief propagation via relaxed scheduling
V. Aksenov, D.-A. Alistarh, J. Korhonen, in:, Advances in Neural Information Processing Systems, Curran Associates, 2020, pp. 22361–22372.
View | Download Published Version (ext.) | arXiv
 
2020 | Conference Paper | IST-REx-ID: 9632 | OA
WoodFisher: Efficient second-order approximation for neural network compression
S.P. Singh, D.-A. Alistarh, in:, Advances in Neural Information Processing Systems, Curran Associates, 2020, pp. 18098–18109.
View | Download Published Version (ext.) | arXiv