Georgia Institute of Technology School of Mathematics | Georgia Institute of Technology | Atlanta, GA

We construct integrators to be used in Hamiltonian (or Hybrid) Monte Carlo sampling. The new integrators are easily implementable and, for a given computational budget, may deliver five times as many accepted proposals as standard leapfrog/Verlet without impairing in any way the quality of the samples. They are based on a suitable modification of the   processing technique first introduced by J.C. Butcher. The idea of modified processing may also be useful for other purposes, like the construction of high-order splitting integrators with positive coefficients.

Joint work with Mari Paz Calvo, Fernando Casas, and Jesús M. Sanz-Serna

Many scientific problems involve invariant structures, and learning functions that rely on a much lower dimensional set of features than the data itself.   Incorporating these invariances into a parametric model can significantly reduce the model complexity, and lead to a vast reduction in the number of labeled examples required to estimate the parameters.  We display this benefit in two settings.  The first setting concerns ReLU networks, and the size of networks and number of points required to learn certain functions and classification regions.  Here, we assume that the target function has built in invariances, namely that it only depends on the projection onto a very low dimensional, function defined manifold (with dimension possibly significantly smaller than even the intrinsic dimension of the data).  We use this manifold variant of a single or multi index model to establish network complexity and ERM rates that beat even the intrinsic dimension of the data.  We should note that a corollary of this result is developing intrinsic rates for a manifold plus noise data model without needing to assume the distribution of the noise decays exponentially, and we also discuss implications in two-sample testing and statistical distances.  The second setting for building invariances concerns linearized optimal transport (LOT), and using it to build supervised classifiers on distributions.  Here, we construct invariances to families of group actions (e.g., shifts and scalings of a fixed distribution), and show that LOT can learn a classifier on group orbits using a simple linear separator.   We demonstrate the benefit of this on MNIST by constructing robust classifiers with only a small number of labeled examples.  This talk covers joint work with Timo Klock, Xiuyuan Cheng, and Caroline Moosmueller.
 

In quantum mechanics and the analysis of Markov processes, Monte Carlo methods are needed to identify low-lying eigenfunctions of dynamical generators. The standard Monte Carlo approaches for identifying eigenfunctions, however, can be inaccurate or slow to converge. What limits the efficiency of the currently available spectral estimation methods and what is needed to build more efficient methods for the future? Through numerical analysis and computational examples, we begin to answer these questions. We present the first-ever convergence proof and error bounds for the variational approach to conformational dynamics (VAC), the dominant method for estimating eigenfunctions used in biochemistry. Additionally, we analyze and optimize variational Monte Carlo (VMC), which combines Monte Carlo with neural networks to accurately identify low-lying eigenstates of quantum systems.

Neural network-based machine learning methods, inlcuding the most notably deep learning have achieved extraordinary successes in numerious  fields. In spite of the rapid development of learning algorithms based on neural networks, their mathematical analysis are far from understood. In particular, it has been a big mystery that neural network-based machine learning methods work extremely well for solving high dimensional problems.

In this talk, I will demonstrate the power of  neural network methods for solving two classes of high dimensional problems: statistical sampling and PDEs. In the first part of the talk, I will present a universal approximation theorem of deep neural networks for representing high dimensional probability distributions. In the second part of the talk, I will discuss a generalization error bound of the Deep Ritz Method for solving high dimensional elliptic problems. For both problems,  our theoretical results show that neural networks-based methods  can overcome the curse of dimensionality.