Georgia Institute of Technology College of Sciences

High-dimensional inference problems such as sparse PCA and planted clique often exhibit statistical-vs-computational tradeoffs whereby there is no known polynomial-time algorithm matching the performance of the optimal estimator. I will discuss an emerging framework -- based on the so-called low-degree likelihood ratio -- for precisely predicting these tradeoffs and giving rigorous evidence for computational hardness in the conjectured hard regime. This method was originally proposed in a sequence of works on the sum-of-squares hierarchy, and the key idea is to study whether or not there exists a low-degree polynomial that succeeds at a given statistical task.

In the second part of the talk, I will give an application to the algorithmic problem of finding an approximate ground state of the SK (Sherrington-Kirkpatrick) spin glass model. I will explain two variants of this problem: "optimization" and "certification." While optimization can be solved in polynomial time [Montanari'18], we give rigorous evidence (in the low-degree framework) that certification cannot be. This result reveals a fundamental discrepancy between two classes of algorithms: local search succeeds while convex relaxations fail.

Based on joint work with Afonso Bandeira and Tim Kunisky (https://arxiv.org/abs/1902.07324 and https://arxiv.org/abs/1907.11636).

In deep generative models, the latent variable is generated by a time-inhomogeneous Markov chain, where at each time step we pass the current state through a parametric nonlinear map, such as a feedforward neural net, and add a small independent Gaussian perturbation. In this talk, based on joint work with Belinda Tzen, I will discuss the diffusion limit of such models, where we increase the number of layers while sending the step size and the noise variance to zero. The resulting object is described by a stochastic differential equation in the sense of Ito. I will first show that sampling in such generative models can be phrased as a stochastic control problem (revisiting the classic results of Föllmer and Dai Pra) and then build on this formulation to quantify the expressive power of these models. Specifically, I will prove that one can efficiently sample from a wide class of terminal target distributions by choosing the drift of the latent diffusion from the class of multilayer feedforward neural nets, with the accuracy of sampling measured by the Kullback-Leibler divergence to the target distribution.

We study the effect of sparsity on the appearance of outliers in the semi-circular law. As a corollary of our main results, we show that, for the Erdos-Renyi random graph with parameter p, the second largest eigenvalue is (asymptotically almost surely) detached from the bulk of the spectrum if and only if pn

A random array indexed by the paths of an infinitely-branching rooted tree of finite depth is hierarchically exchangeable if its joint distribution is invariant under rearrangements that preserve the tree structure underlying the index set. Austin and Panchenko (2014) prove that such arrays have de Finetti-type representations, and moreover, that an array indexed by a finite collection of such trees has an Aldous-Hoover-type representation.

Motivated by problems in Bayesian nonparametrics and probabilistic programming discussed in Staton et al. (2018), we generalize hierarchical exchangeability to a new kind of partial exchangeability for random arrays which we call DAG-exchangeability. In our setting a random array is indexed by N^{|V|} for some DAG G=(V,E), and its exchangeability structure is governed by the edge set E. We prove a representation theorem for such arrays which generalizes the Aldous-Hoover representation theorem, and for which the Austin-Panchenko representation is a special case.