Learning a priori constrained weighted majority votes

A. Bellet, A. Habrard, E. Morvant, M. Sebban, Machine Learning 97 (2014) 129–154.


Journal Article | Published | English
Author
; ; ;
Department
Abstract
Weighted majority votes allow one to combine the output of several classifiers or voters. MinCq is a recent algorithm for optimizing the weight of each voter based on the minimization of a theoretical bound over the risk of the vote with elegant PAC-Bayesian generalization guarantees. However, while it has demonstrated good performance when combining weak classifiers, MinCq cannot make use of the useful a priori knowledge that one may have when using a mixture of weak and strong voters. In this paper, we propose P-MinCq, an extension of MinCq that can incorporate such knowledge in the form of a constraint over the distribution of the weights, along with general proofs of convergence that stand in the sample compression setting for data-dependent voters. The approach is applied to a vote of k-NN classifiers with a specific modeling of the voters' performance. P-MinCq significantly outperforms the classic k-NN classifier, a symmetric NN and MinCq using the same voters. We show that it is also competitive with LMNN, a popular metric learning algorithm, and that combining both approaches further reduces the error.
Publishing Year
Date Published
2014-10-01
Journal Title
Machine Learning
Acknowledgement
This work was funded by the French project SoLSTiCe ANR-13-BS02-01 of the ANR.
Volume
97
Issue
1-2
Page
129 - 154
IST-REx-ID

Cite this

Bellet A, Habrard A, Morvant E, Sebban M. Learning a priori constrained weighted majority votes. Machine Learning. 2014;97(1-2):129-154. doi:10.1007/s10994-014-5462-z
Bellet, A., Habrard, A., Morvant, E., & Sebban, M. (2014). Learning a priori constrained weighted majority votes. Machine Learning, 97(1–2), 129–154. https://doi.org/10.1007/s10994-014-5462-z
Bellet, Aurélien, Amaury Habrard, Emilie Morvant, and Marc Sebban. “Learning a Priori Constrained Weighted Majority Votes.” Machine Learning 97, no. 1–2 (2014): 129–54. https://doi.org/10.1007/s10994-014-5462-z.
A. Bellet, A. Habrard, E. Morvant, and M. Sebban, “Learning a priori constrained weighted majority votes,” Machine Learning, vol. 97, no. 1–2, pp. 129–154, 2014.
Bellet A, Habrard A, Morvant E, Sebban M. 2014. Learning a priori constrained weighted majority votes. Machine Learning. 97(1–2), 129–154.
Bellet, Aurélien, et al. “Learning a Priori Constrained Weighted Majority Votes.” Machine Learning, vol. 97, no. 1–2, Springer, 2014, pp. 129–54, doi:10.1007/s10994-014-5462-z.

Link(s) to Main File(s)
Access Level
OA Open Access

Export

Marked Publications

Open Data IST Research Explorer

Search this title in

Google Scholar