Function norms for neural networks

A. Rannen-Triki, M. Berman, V. Kolmogorov, M.B. Blaschko, in:, Proceedings of the 2019 International Conference on Computer Vision Workshop, IEEE, 2019, pp. 748–752.

Download
No fulltext has been uploaded. References only!

Conference Paper | Published | English

Scopus indexed
Author
Rannen-Triki, Amal; Berman, Maxim; Kolmogorov, VladimirIST Austria; Blaschko, Matthew B.
Department
Abstract
Deep neural networks (DNNs) have become increasingly important due to their excellent empirical performance on a wide range of problems. However, regularization is generally achieved by indirect means, largely due to the complex set of functions defined by a network and the difficulty in measuring function complexity. There exists no method in the literature for additive regularization based on a norm of the function, as is classically considered in statistical learning theory. In this work, we study the tractability of function norms for deep neural networks with ReLU activations. We provide, to the best of our knowledge, the first proof in the literature of the NP-hardness of computing function norms of DNNs of 3 or more layers. We also highlight a fundamental difference between shallow and deep networks. In the light on these results, we propose a new regularization strategy based on approximate function norms, and show its efficiency on a segmentation task with a DNN.
Publishing Year
Date Published
2019-10-01
Proceedings Title
Proceedings of the 2019 International Conference on Computer Vision Workshop
Article Number
748-752
Conference
ICCVW: International Conference on Computer Vision Workshop
Conference Location
Seoul, South Korea
Conference Date
2019-10-27 – 2019-10-28
IST-REx-ID

Cite this

Rannen-Triki A, Berman M, Kolmogorov V, Blaschko MB. Function norms for neural networks. In: Proceedings of the 2019 International Conference on Computer Vision Workshop. IEEE; 2019:748-752. doi:10.1109/ICCVW.2019.00097
Rannen-Triki, A., Berman, M., Kolmogorov, V., & Blaschko, M. B. (2019). Function norms for neural networks. In Proceedings of the 2019 International Conference on Computer Vision Workshop (pp. 748–752). Seoul, South Korea: IEEE. https://doi.org/10.1109/ICCVW.2019.00097
Rannen-Triki, Amal, Maxim Berman, Vladimir Kolmogorov, and Matthew B. Blaschko. “Function Norms for Neural Networks.” In Proceedings of the 2019 International Conference on Computer Vision Workshop, 748–52. IEEE, 2019. https://doi.org/10.1109/ICCVW.2019.00097.
A. Rannen-Triki, M. Berman, V. Kolmogorov, and M. B. Blaschko, “Function norms for neural networks,” in Proceedings of the 2019 International Conference on Computer Vision Workshop, Seoul, South Korea, 2019, pp. 748–752.
Rannen-Triki A, Berman M, Kolmogorov V, Blaschko MB. 2019. Function norms for neural networks. Proceedings of the 2019 International Conference on Computer Vision Workshop. ICCVW: International Conference on Computer Vision Workshop 748–752.
Rannen-Triki, Amal, et al. “Function Norms for Neural Networks.” Proceedings of the 2019 International Conference on Computer Vision Workshop, IEEE, 2019, pp. 748–52, doi:10.1109/ICCVW.2019.00097.

Export

Marked Publications

Open Data IST Research Explorer

Search this title in

Google Scholar
ISBN Search