Deep Generative Models in the Diffusion Limit
- Series
- Stochastics Seminar
- Time
- Thursday, September 19, 2019 - 15:05 for 1 hour (actually 50 minutes)
- Location
- Skiles 006
- Speaker
- Maxim Raginsky – ECE Department, University of Illinois at Urbana-Champaign
In deep generative models, the latent variable is generated by a time-inhomogeneous Markov chain, where at each time step we pass the current state through a parametric nonlinear map, such as a feedforward neural net, and add a small independent Gaussian perturbation. In this talk, based on joint work with Belinda Tzen, I will discuss the diffusion limit of such models, where we increase the number of layers while sending the step size and the noise variance to zero. The resulting object is described by a stochastic differential equation in the sense of Ito. I will first show that sampling in such generative models can be phrased as a stochastic control problem (revisiting the classic results of Föllmer and Dai Pra) and then build on this formulation to quantify the expressive power of these models. Specifically, I will prove that one can efficiently sample from a wide class of terminal target distributions by choosing the drift of the latent diffusion from the class of multilayer feedforward neural nets, with the accuracy of sampling measured by the Kullback-Leibler divergence to the target distribution.