Georgia Institute of Technology College of Sciences

This talk will report some of our progress in showing how dynamics can be a useful mathematical tool for machine learning. Three demonstrations will be given, namely, how dynamics help design (and analyze) optimization algorithms, how dynamics help quantitatively understand nontrivial observations in deep learning practices, and how deep learning can in turn help dynamics (or more broadly put, AI for sciences). More precisely, in part 1 (dynamics for algorithm): I will talk about how to add momentum to gradient descent on a class of manifolds known as Lie groups. The treatment will be based on geometric mechanics and an interplay between continuous and discrete time dynamics. It will lead to accelerated optimization. Part 2 (dynamics for understanding deep learning) will be devoted to better understanding the nontrivial effects of large learning rates. I will describe how large learning rates could deterministically lead to chaotic escapes from local minima, which is an alternative mechanism to commonly known noisy escapes due to stochastic gradients. I will also mention another example, on an implicit regularization effect of large learning rates which is to favor flatter minimizers.  Part 3 (AI for sciences) will be on data-driven prediction of mechanical dynamics, for which I will demonstrate one strong benefit of having physics hard-wired into deep learning models; more precisely, how to make symplectic predictions, and how that generically improves the accuracy of long-time predictions.

Let X_{N,n} be an iid product of N real Gaussian matrices of size n x n. In this talk, I will explain some recent joint work with G. Paouris 
(arXiv:2005.08899) about a non-asymptotic analysis of the singular values of X_{N,n} . I will begin by giving some intuition and motivation for studying such matrix products. Then, I will explain two new results. The first gives a rate of convergence for the global distribution of singular values of X_{N,n} to the so-called Triangle Law in the limit where N,n tend to infinity. The second is a kind of quantitative version of the multiplicative ergodic theorem, giving estimates at finite but large N on the distance between the joint distribution of all Lyapunov exponents of X_{N,n} and appropriately normalized independent Gaussians in the near-ergodic regime (N >> n).

Breathers are temporally periodic and spatially localized solutions of evolutionary PDEs. They are known to exist for integrable PDEs such as the sine-Gordon equation, but are believed to be rare for general nonlinear PDEs. When the spatial dimension is equal to one, exchanging the roles of time and space variables (in the so-called spatial dynamics framework), breathers can be interpreted as homoclinic solutions to steady solutions and thus arising from the intersections of the stable and unstable manifolds of the steady states. In this talk, we shall study small breathers of the nonlinear Klein-Gordon equation generated in an unfolding bifurcation as a pair of eigenvalues collide at the original when a parameter (temporal frequency) varies. Due to the presence of the oscillatory modes, generally the finite dimensional stable and unstable manifolds do not intersect in the infinite dimensional phase space, but with an exponentially small splitting (relative to the amplitude of the breather) in this singular perturbation problem of multiple time scales. This splitting leads to the transversal intersection of the center-stable and center-unstable manifolds which produces small amplitude generalized breathers with exponentially small tails. Due to the exponential small splitting, classical perturbative techniques cannot be applied. We will explain how to obtain an asymptotic formula for the distance between the stable and unstable manifold of the steady solutions. This is a joint work of O. Gomide, M. Guardia, T. Seara, and C. Zeng. 

In this talk we present the formation of steady waves in two-dimensional fluids under a current with mean velocity $c$ flowing over a periodic bottom. Using a formulation based on the Dirichlet-Neumann operator, we establish the unique continuation of a steady solution from the trivial solution for a flat bottom, with the exception of a sequence of velocities $c_{k}$.  Furthermore, we prove that at least two steady solutions for a near-flat bottom persist close to a non-degenerate $S^1$-orbit of steady waves for a flat bottom. As a consequence, we obtain the persistence of at least two steady waves close to a non-degenerate $S^1$-orbit of Stokes waves bifurcating from the velocities $c_{k}$ for a flat bottom. This is a joint work with W. Craig.