Statistical trajectory predictions for complex algorithms with random data
- Series
- Stochastics Seminar
- Time
- Thursday, October 31, 2024 - 15:30 for 1 hour (actually 50 minutes)
- Location
- Skiles 006
- Speaker
- Ashwin Pananjady – Georgia Tech – ashwinpm@gatech.edu
Iterative algorithms are the workhorses of modern statistical signal processing and machine learning. While the choice of an algorithm and its hyperparameters determines both the speed and fidelity of the learning pipeline, it is common for this choice to be made heuristically, either by expensive trial-and-error or by comparing upper bounds on convergence rates of various candidate algorithms. Motivated by these issues, I will present a toolbox for deriving “state evolutions” for a wide variety of algorithms with random data. These are non-asymptotic, near-exact predictions of the statistical behavior of the algorithm, which apply even when the underlying optimization problem is nonconvex or the algorithm is randomly initialized. We will showcase these predictions on deterministic and stochastic variants of complex algorithms employed in some canonical statistical models.