Georgia Institute of Technology School of Mathematics | Georgia Institute of Technology | Atlanta, GA

In this talk, we consider the problem of minimizing multi-modal loss functions with many local optima. Since the local gradient points to the direction of the steepest slope in an infinitesimal neighborhood, an optimizer guided by the local gradient is often trapped in a local minimum. To address this issue, we develop a novel nonlocal gradient to skip small local minima by capturing major structures of the loss's landscape in black-box optimization. The nonlocal gradient is defined by a directional Gaussian smoothing (DGS) approach. The key idea is to conducts 1D long-range exploration with a large smoothing radius along orthogonal directions, each of which defines a nonlocal directional derivative as a 1D integral. Such long-range exploration enables the nonlocal gradient to skip small local minima. We use the Gauss-Hermite quadrature rule to approximate the d 1D integrals to obtain an accurate estimator. We also provide theoretical analysis on the convergence of the method on nonconvex landscape. In this work, we investigate the scenario where the objective function is composed of a convex function, perturbed by a highly oscillating, deterministic noise. We provide a convergence theory under which the iterates converge to a tightened neighborhood of the solution, whose size is characterized by the noise frequency. We complement our theoretical analysis with numerical experiments to illustrate the performance of this approach.

Gaussian processes (GPs) are widely employed as versatile modeling and predictive tools in spatial statistics, functional data analysis, computer modeling and diverse applications of machine learning. They have been widely studied over Euclidean spaces, where they are specified using covariance functions or covariograms for modelling complex dependencies. There is a growing literature on GPs over Riemannian manifolds in order to develop richer and more flexible inferential frameworks. While GPs have been extensively studied for asymptotic inference on Euclidean spaces using positive definite covariograms, such results are relatively sparse on Riemannian manifolds. We undertake analogous developments for GPs constructed over compact Riemannian manifolds. Building upon the recently introduced Matérn covariograms on a compact Riemannian manifold, we employ formal notions and conditions for the equivalence of two Matérn Gaussian random measures on compact manifolds to derive the microergodic parameters and formally establish the consistency of their maximum likelihood estimates as well as asymptotic optimality of the best linear unbiased predictor.

Many fluid flows display at a wide range of space and time scales. Turbulent and multiphase flows can include small eddies or particles, but likewise large advected features. This challenge makes some degree of multi-scale modeling or homogenization necessary. Such models are restricted, though: they should be numerically accurate, physically consistent, computationally expedient, and more. I present two tools crafted for this purpose. First, the fast macroscopic forcing method (Fast MFM), which is based on an elliptic pruning procedure that localizes solution operators and sparse matrix-vector sampling. We recover eddy-diffusivity operators with a convergence that beats the best spectral approximation (from the SVD), attenuating the cost of, for example, targeted RANS closures. I also present a moment-based method for closing multiphase flow equations. Buttressed by a recurrent neural network, it is numerically stable and achieves state-of-the-art accuracy. I close with a discussion of conducting these simulations near exascale. Our simulations scale ideally on the entirety of ORNL Summit's GPUs, though the HPC landscape continues to shift.

In this talk, we propose a Neural Oracle Search(NOS) model in Automatic Speech Recognition(ASR) to select the most likely hypothesis using a sequence of acoustic representations and multiple hypotheses as input. The model provides a sequence level score for each audio-hypothesis pair that is obtained by integrating information from multiple sources, such as the input acoustic representations, N-best hypotheses, additional 1st-pass statistics, and unpaired textual information through an external language model. These scores are then used to map the search problem of identifying the most likely hypothesis to a sequence classification problem. The definition of the proposed model is broad enough to allow its use as an alternative to beam search in the 1st-pass or as a 2nd-pass, rescoring step. This model achieves up to 12% relative reductions in Word Error Rate (WER) across several languages over state-of-the-art baselines with relatively few additional parameters. In addition, we investigate the use of the NOS model on a 1st-pass multilingual model and show that similar to the 1st-pass model, the NOS model can be made multilingual.