Scalable belief propagation via relaxed scheduling

Aksenov V, Alistarh D-A, Korhonen J. 2020. Scalable belief propagation via relaxed scheduling. Advances in Neural Information Processing Systems. NeurIPS: Conference on Neural Information Processing Systems vol. 33, 22361–22372.

Conference Paper | Published | English

Scopus indexed
Department
Abstract
The ability to leverage large-scale hardware parallelism has been one of the key enablers of the accelerated recent progress in machine learning. Consequently, there has been considerable effort invested into developing efficient parallel variants of classic machine learning algorithms. However, despite the wealth of knowledge on parallelization, some classic machine learning algorithms often prove hard to parallelize efficiently while maintaining convergence. In this paper, we focus on efficient parallel algorithms for the key machine learning task of inference on graphical models, in particular on the fundamental belief propagation algorithm. We address the challenge of efficiently parallelizing this classic paradigm by showing how to leverage scalable relaxed schedulers in this context. We present an extensive empirical study, showing that our approach outperforms previous parallel belief propagation implementations both in terms of scalability and in terms of wall-clock convergence time, on a range of practical applications.
Publishing Year
Date Published
2020-12-06
Proceedings Title
Advances in Neural Information Processing Systems
Acknowledgement
We thank Marco Mondelli for discussions related to LDPC decoding, and Giorgi Nadiradze for discussions on analysis of relaxed schedulers. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 805223 ScaleML).
Volume
33
Page
22361-22372
Conference
NeurIPS: Conference on Neural Information Processing Systems
Conference Location
Vancouver, Canada
Conference Date
2020-12-06 – 2020-12-12
ISSN
IST-REx-ID

Cite this

Aksenov V, Alistarh D-A, Korhonen J. Scalable belief propagation via relaxed scheduling. In: Advances in Neural Information Processing Systems. Vol 33. Curran Associates; 2020:22361-22372.
Aksenov, V., Alistarh, D.-A., & Korhonen, J. (2020). Scalable belief propagation via relaxed scheduling. In Advances in Neural Information Processing Systems (Vol. 33, pp. 22361–22372). Vancouver, Canada: Curran Associates.
Aksenov, Vitaly, Dan-Adrian Alistarh, and Janne Korhonen. “Scalable Belief Propagation via Relaxed Scheduling.” In Advances in Neural Information Processing Systems, 33:22361–72. Curran Associates, 2020.
V. Aksenov, D.-A. Alistarh, and J. Korhonen, “Scalable belief propagation via relaxed scheduling,” in Advances in Neural Information Processing Systems, Vancouver, Canada, 2020, vol. 33, pp. 22361–22372.
Aksenov V, Alistarh D-A, Korhonen J. 2020. Scalable belief propagation via relaxed scheduling. Advances in Neural Information Processing Systems. NeurIPS: Conference on Neural Information Processing Systems vol. 33, 22361–22372.
Aksenov, Vitaly, et al. “Scalable Belief Propagation via Relaxed Scheduling.” Advances in Neural Information Processing Systems, vol. 33, Curran Associates, 2020, pp. 22361–72.
All files available under the following license(s):
Copyright Statement:
This Item is protected by copyright and/or related rights. [...]

Link(s) to Main File(s)
Access Level
OA Open Access

Export

Marked Publications

Open Data IST Research Explorer

Sources

arXiv 2002.11505

Search this title in

Google Scholar
ISBN Search