Georgia Institute of Technology College of Sciences

This talk will be concerned with nonasymptotic lower bounds for the estimation of principal subspaces. I will start by reviewing some previous methods, including the local asymptotic minimax theorem and the Grassmann approach. Then I will present a new approach based on a van Trees inequality (i.e. a Bayesian version of the Cramér-Rao inequality) tailored for invariant statistical models. As applications, I will provide nonasymptotic lower bounds for principal component analysis and the matrix denoising problem, two examples that are invariant with respect to the orthogonal group. These lower bounds are characterized by doubly substochastic matrices whose entries are bounded by the different Fisher information directions, confirming recent upper bounds in the context of the empirical covariance operator.

Seminar link: https://bluejeans.com/129119189

In the past several decades, scale invariant stochastic processes have been used in a wide range of applications including internet traffic modeling and hydrology.  However, by comparison to univariate scale invariance, far less attention has been paid to characteristically multivariate models that display aspects of scaling behavior the limit theory arguably suggests is most natural.
 
In this talk, I will introduce a new scale invariance model called operator fractional Lévy motion and discuss some of its interesting features, as well as some aspects of wavelet-based estimation of its scaling exponents. This is related to joint work with Gustavo Didier (Tulane University), Herwig Wendt (CNRS, IRIT Univ. of Toulouse) and Patrice Abry (CNRS, ENS-Lyon).

For a certain scaling of the initialization of stochastic gradient descent (SGD), wide neural networks (NN) have been shown to be well approximated by reproducing kernel Hilbert space (RKHS)  methods. Recent empirical work showed that, for some classification tasks, RKHS methods can replace NNs without a large loss in performance. On the other hand, two-layers NNs are known to encode richer smoothness classes than RKHS and we know of special examples for which SGD-trained NN provably outperform RKHS. This is true also in the wide network limit, for a different scaling of the initialization.

How can we reconcile the above claims? For which tasks do NNs outperform RKHS? If feature vectors are nearly isotropic, RKHS methods suffer from the curse of dimensionality, while NNs can overcome it by learning the best low-dimensional representation. Here we show that this curse of dimensionality becomes milder if the feature vectors display the same low-dimensional structure as the target function, and we precisely characterize this tradeoff. Building on these results, we present a model that can capture in a unified framework both behaviors observed in earlier work. We hypothesize that such a latent low-dimensional structure is present in image classification. We test numerically this hypothesis by showing that specific perturbations of the training distribution degrade the performances of RKHS methods much more significantly than NNs.