Georgia Institute of Technology School of Mathematics | Georgia Institute of Technology | Atlanta, GA

https://gatech.zoom.us/j/95197085752?pwd=WmtJUVdvM1l6aUJBbHNJWTVKcVdmdz09

Abstract:  The interaction of machine learning and dynamics can lead to both new methodology for dynamics, and deepened understanding and/or efficacious algorithms for machine learning. This talk will give examples in both directions. Specifically, I will first discuss data-driven learning and prediction of mechanical dynamics, for which I will demonstrate one strong benefit of having physics hard-wired into deep learning models; more precisely, how to make symplectic predictions, and how that probably improves the accuracy of long-time predictions. Then I will discuss how dynamics can be used to better understand the implicit biases of large learning rates in the training of machine learning models. They could lead to quantitative escapes from local minima via chaos, which is an alternative mechanism to commonly known noisy escapes due to stochastic gradients. I will also report how large learning rates bias toward flatter minimizers, which arguably generalize better.

N/A

In this talk, we propose a Neural Oracle Search(NOS) model in Automatic Speech Recognition(ASR) to select the most likely hypothesis using a sequence of acoustic representations and multiple hypotheses as input. The model provides a sequence level score for each audio-hypothesis pair that is obtained by integrating information from multiple sources, such as the input acoustic representations, N-best hypotheses, additional 1st-pass statistics, and unpaired textual information through an external language model. These scores are then used to map the search problem of identifying the most likely hypothesis to a sequence classification problem. The definition of the proposed model is broad enough to allow its use as an alternative to beam search in the 1st-pass or as a 2nd-pass, rescoring step. This model achieves up to 12% relative reductions in Word Error Rate (WER) across several languages over state-of-the-art baselines with relatively few additional parameters. In addition, we investigate the use of the NOS model on a 1st-pass multilingual model and show that similar to the 1st-pass model, the NOS model can be made multilingual.

In this talk, I will present a simple proof of the matrix Spencer conjecture up to poly-logarithmic rank: given symmetric d by d matrices A_1,...,A_n each with operator norm at most 1 and rank at most n/\log^3 n, one can efficiently find \pm 1 signs x_1,... ,x_n such that their signed sum has spectral norm \|\sum_{i=1}^n x_i A_i\|_op= O(\sqrt{n}). This result also implies a (\log n - 3 \log \log n) qubit lower bound for quantum random access codes encoding n classical bits with advantage >> 1/\sqrt{n}. Our proof uses the recent refinement of the non-commutative Khintchine inequality in [Bandeira, Boedihardjo, van Handel, 2022] for random matrices with correlated Gaussian entries.