---
res:
bibo_abstract:
- "Fitting a function by using linear combinations of a large number N of `simple'
components is one of the most fruitful ideas in statistical learning. This idea
lies at the core of a variety of methods, from two-layer neural networks to kernel
regression, to boosting. In general, the resulting risk minimization problem is
non-convex and is solved by gradient descent or its variants. Unfortunately, little
is known about global convergence properties of these approaches.\r\nHere we consider
the problem of learning a concave function f on a compact convex domain Ω⊆ℝd,
using linear combinations of `bump-like' components (neurons). The parameters
to be fitted are the centers of N bumps, and the resulting empirical risk minimization
problem is highly non-convex. We prove that, in the limit in which the number
of neurons diverges, the evolution of gradient descent converges to a Wasserstein
gradient flow in the space of probability distributions over Ω. Further, when
the bump width δ tends to 0, this gradient flow has a limit which is a viscous
porous medium equation. Remarkably, the cost function optimized by this gradient
flow exhibits a special property known as displacement convexity, which implies
exponential convergence rates for N→∞, δ→0. Surprisingly, this asymptotic theory
appears to capture well the behavior for moderate values of δ,N. Explaining this
phenomenon, and understanding the dependence on δ,N in a quantitative manner remains
an outstanding challenge.@eng"
bibo_authorlist:
- foaf_Person:
foaf_givenName: Adel
foaf_name: Javanmard, Adel
foaf_surname: Javanmard
- foaf_Person:
foaf_givenName: Marco
foaf_name: Mondelli, Marco
foaf_surname: Mondelli
foaf_workInfoHomepage: http://www.librecat.org/personId=27EB676C-8706-11E9-9510-7717E6697425
orcid: 0000-0002-3242-7020
- foaf_Person:
foaf_givenName: Andrea
foaf_name: Montanari, Andrea
foaf_surname: Montanari
dct_date: 2019^xs_gYear
dct_language: eng
dct_publisher: ArXiv@
dct_title: Analysis of a two-layer neural network via displacement convexity@
...