Georgia Institute of Technology College of Sciences

In the past decade, the revival of deep neural networks has led to dramatic success in numerous applications ranging from computer vision to natural language processing to scientific discovery and beyond. Nevertheless, the practice of deep networks has been shrouded with mystery as our theoretical understanding of the success of deep learning remains elusive.

In this talk, we will exploit low-dimensional modeling to help understand and improve deep learning performance. We will first provide a geometric analysis for understanding neural collapse, an intriguing empirical phenomenon that persists across different neural network architectures and a variety of standard datasets. We will utilize our understanding of neural collapse to improve training efficiency. We will then exploit principled methods for dealing with sparsity and sparse corruptions to address the challenges of overfitting for modern deep networks in the presence of training data corruptions. We will introduce a principled approach for robustly training deep networks with noisy labels and robustly recovering natural images by deep image prior.

Given an n-by-d matrix A and a vector of size n, the p-norm problem asks for a vector x that minimizes the following

\sum_{i=1}^n (a_i^T x - b_i)^p,

where a_i is the i’th row of A. The study of p=2 and p=1 cases dates back to Legendre, Gauss, and Laplace. Other cases of p have also been used in statistics and machine learning as robust estimators for decades. In this talk, we present the following improvements in the running time of solving p-norm regression problems.

For the case of 1

For 1

The talk is based on a joint work with Richard Peng and Santosh Vempala.

There are many interesting classes of polynomials in real algebraic geometry that are of modern interest. A polynomial is nonnegative if it only takes nonnegative values on R^n. A univariate polynomial is real-rooted if all of its complex roots are real, and a hyperbolic polynomial is a multivariate generalization of a real-rooted polynomial. We will discuss connections between these two classes of polynomials. In particular, we will discuss recent ideas of Saunderson giving new ways of proving that a polynomial is nonnegative beyond showing that it is sum-of-squares.

Teams link: https://teams.microsoft.com/l/meetup-join/19%3a3a9d7f9d1fca4f5b991b4029b09c69a1%40thread.tacv2/1646885419648?context=%7b%22Tid%22%3a%22482198bb-ae7b-4b25-8b7a-6d7f32faa083%22%2c%22Oid%22%3a%2206706002-23ff-4989-8721-b078835bae91%22%7d

Classical matrix concentration inequalities are sharp up to a logarithmic factor. This logarithmic factor is necessary in the commutative case but unnecessary in many classical noncommutative cases. We will present some matrix concentration results that are sharp in many cases, where we overcome this logarithmic factor by using an easily computable quantity that captures noncommutativity. Joint work with Afonso Bandeira and Ramon van Handel.

Due to privacy, access to real data is often restricted. Data that are not completely real but resemble certain properties of real data become natural substitutes. Data of this type are called synthetic data. I will talk about the extent to which synthetic data may resemble real data under privacy and computational complexity restrictions. Joint work with Thomas Strohmer and Roman Vershynin.

The link to the online talk:  https://bluejeans.com/405947238/3475