Seminars and Colloquia Schedule

Dynamic Stability in Stochastic Gradient Descent

CDSNS Colloquium
Friday, May 24, 2024 - 15:30 for
Skiles 254
Dennis ChemnitzFU Berlin

Streaming via Zoom: />

Most modern machine learning applications are based on overparameterized neural networks trained by variants of stochastic gradient descent. To explain the performance of these networks from a theoretical perspective (in particular the so-called "implicit bias"), it is necessary to understand the random dynamics of the optimization algorithms. Mathematically this amounts to the study of random dynamical systems with manifolds of equilibria. In this talk, I will give a brief introduction to machine learning theory and explain how almost-sure Lyapunov exponents and moment Lyapunov exponents can be used to characterize the set of possible limit points for stochastic gradient descent.