Optimal combination of linear and spectral estimators for generalized linear models

Mondelli M, Thrampoulidis C, Venkataramanan R. 2021. Optimal combination of linear and spectral estimators for generalized linear models. Foundations of Computational Mathematics.

Download
OA 2021_Springer_Mondelli.pdf 2.31 MB

Journal Article | Published | English

Scopus indexed
Author
Mondelli, MarcoIST Austria ; Thrampoulidis, Christos; Venkataramanan, Ramji
Department
Abstract
We study the problem of recovering an unknown signal π‘₯π‘₯ given measurements obtained from a generalized linear model with a Gaussian sensing matrix. Two popular solutions are based on a linear estimator π‘₯π‘₯^L and a spectral estimator π‘₯π‘₯^s. The former is a data-dependent linear combination of the columns of the measurement matrix, and its analysis is quite simple. The latter is the principal eigenvector of a data-dependent matrix, and a recent line of work has studied its performance. In this paper, we show how to optimally combine π‘₯π‘₯^L and π‘₯π‘₯^s. At the heart of our analysis is the exact characterization of the empirical joint distribution of (π‘₯π‘₯,π‘₯π‘₯^L,π‘₯π‘₯^s) in the high-dimensional limit. This allows us to compute the Bayes-optimal combination of π‘₯π‘₯^L and π‘₯π‘₯^s, given the limiting distribution of the signal π‘₯π‘₯. When the distribution of the signal is Gaussian, then the Bayes-optimal combination has the form πœƒπ‘₯π‘₯^L+π‘₯π‘₯^s and we derive the optimal combination coefficient. In order to establish the limiting distribution of (π‘₯π‘₯,π‘₯π‘₯^L,π‘₯π‘₯^s), we design and analyze an approximate message passing algorithm whose iterates give π‘₯π‘₯^L and approach π‘₯π‘₯^s. Numerical simulations demonstrate the improvement of the proposed combination with respect to the two methods considered separately.
Publishing Year
Date Published
2021-08-17
Journal Title
Foundations of Computational Mathematics
Acknowledgement
M. Mondelli would like to thank Andrea Montanari for helpful discussions. All the authors would like to thank the anonymous reviewers for their helpful comments.
IST-REx-ID

Cite this

Mondelli M, Thrampoulidis C, Venkataramanan R. Optimal combination of linear and spectral estimators for generalized linear models. Foundations of Computational Mathematics. 2021. doi:10.1007/s10208-021-09531-x
Mondelli, M., Thrampoulidis, C., & Venkataramanan, R. (2021). Optimal combination of linear and spectral estimators for generalized linear models. Foundations of Computational Mathematics. Springer. https://doi.org/10.1007/s10208-021-09531-x
Mondelli, Marco, Christos Thrampoulidis, and Ramji Venkataramanan. β€œOptimal Combination of Linear and Spectral Estimators for Generalized Linear Models.” Foundations of Computational Mathematics. Springer, 2021. https://doi.org/10.1007/s10208-021-09531-x.
M. Mondelli, C. Thrampoulidis, and R. Venkataramanan, β€œOptimal combination of linear and spectral estimators for generalized linear models,” Foundations of Computational Mathematics. Springer, 2021.
Mondelli M, Thrampoulidis C, Venkataramanan R. 2021. Optimal combination of linear and spectral estimators for generalized linear models. Foundations of Computational Mathematics.
Mondelli, Marco, et al. β€œOptimal Combination of Linear and Spectral Estimators for Generalized Linear Models.” Foundations of Computational Mathematics, Springer, 2021, doi:10.1007/s10208-021-09531-x.
All files available under the following license(s):
Creative Commons Attribution 4.0 International Public License (CC-BY 4.0):
Main File(s)
Access Level
OA Open Access
Date Uploaded
2021-12-13
MD5 Checksum
9ea12dd8045a0678000a3a59295221cb


Link(s) to Main File(s)
Access Level
OA Open Access

Export

Marked Publications

Open Data IST Research Explorer

Sources

arXiv 2008.03326v3

Search this title in

Google Scholar