Georgia Institute of Technology College of Sciences

We develop an extension of the Stein-Malliavin calculus which allows to measure the Wasserstein distance between the probability distributions of $ (X, Y)$ and $(Z,Y)$, where $X,Y$ are arbitrary random vectors and $ Z\sim N(0, \sigma ^{2})$ is independent of $Y$. In particular, this method allows to quantify the asymptotic independence between sequences of random variables and vectors. We will discuss some particular applications of this method to various limit theorems.

This dissertation develops algorithms and theory for optimization, sampling, and generative modeling on manifolds, with Lie groups as a central object of study. Lie groups are manifolds with additional group structure; when endowed with a left-invariant Riemannian metric they become Riemannian manifolds, a setting that plays a central role throughout this work.

We first consider optimization on the Stiefel manifold $\mathrm{St}(n,d)$, the set of $n\times d$ matrices with orthonormal columns. By deriving a variational principle on this manifold, we construct the Momentum Stiefel Optimizer, a momentum-based algorithm that exactly preserves the orthogonality constraint at every iteration. The optimizer is applied to suitably-orthogonal attention in transformers and to optimal transport problems, achieving consistent improvements over existing methods.

We then establish quantitative convergence guarantees for momentum optimizers on Lie groups equipped with a left-invariant metric. Using the left-trivialization technique, which maps the curved dynamics to a flat Euclidean space for the momentum variable, we prove the first explicit convergence rates for both Heavy-Ball and Nesterov Accelerated methods on compact Lie groups, with rates that match Euclidean theory in terms of the smoothness and strong-convexity constants.

Next, we develop gauge-equivariant accelerated methods for optimization over the Grassmannian $\mathrm{Gr}(n,d)$, the set of $d$-dimensional subspaces of $\mathbb{R}^n$. Because each subspace has infinitely many orthonormal representatives related by an $\mathrm{O}(d)$ rotation, a naive lift of Stiefel algorithms to the Grassmannian is not gauge-equivariant. We introduce a gauge-fixing pipeline that converts any Stiefel optimizer into a gauge-equivariant Grassmann algorithm, yielding Grassmann Anderson Acceleration and Grassmann NAG, validated on density functional theory and low-rank matrix completion.

We then turn to sampling on Lie groups. By adding tractable noise to the left-trivialized momentum dynamics, we construct the first kinetic (momentum) Langevin Monte Carlo sampler on Lie groups with rigorous nonasymptotic convergence guarantees. The sampler preserves the group structure exactly at every step. Exponential convergence in $W_2$ distance is proved under only compactness of the Lie group and geodesic smoothness of the potential, without any convexity or isoperimetric assumption.

Finally, we address score-based generative modeling on general Riemannian manifolds. The standard denoising score matching framework requires a tractable forward-process transition kernel, which is unavailable on general manifolds because the heat kernel is intractable. We propose splitting diffusion, which lifts the dynamics to the tangent bundle and alternates closed-form stochastic momentum updates in the Euclidean tangent space with deterministic geodesic transport. The resulting transition kernel is closed-form, enabling denoising score matching training on general Riemannian manifolds requiring only exponential map as oracle.

Let $(S,\mathcal{A})$ be a measurable space and let $\mathcal{P}$ be a family of probability distributions on it. Given a Banach space $E$, a mapping $\theta:\mathcal{P}\to E$, and a smooth functional $f:E\to\mathbb{R}$, we consider the problem of estimating $f(\theta(P))$ from i.i.d. observations $X_1,\ldots,X_n\sim P$, where $P\in\mathcal{P}$. We write $f\in C^s(E)$ for a functional of Hölder smoothness $s=m+\rho$, where $m\ge 0$ is an integer and $\rho\in(0,1]$. Our aim is to construct estimators of $f(\theta(P))$ and to study their dependence on the sample size $n$, the smoothness $s$, and the dimension or complexity of the parameter $\theta(P)$.

When $\hat\theta_n$ is a $\sqrt{n}$-consistent base estimator of $\theta(P)$, the plug-in estimator $f(\hat\theta_n)$ is asymptotically efficient in classical low-dimensional models, but in high-dimensional and infinite-dimensional settings its bias is often too large to attain the rate $n^{-1/2}$. Existing bias reduction methods, based on iterated bootstrap or on linear aggregation of plug-in estimators, rely on concentration inequalities for $f(\hat\theta_n)$ that are available only for a limited class of models.

In this dissertation we study a class of estimators $T_f(X_1,\ldots,X_n)$ obtained from a Taylor expansion of $f$ about $\hat\theta_n$, of order determined by $s$, together with a sample split. Their analysis uses only bounds on the moments of the linear and higher order terms of $\hat\theta_n-\theta(P)$, rather than concentration inequalities. For functionals of smoothness $s\ge 1$, we derive upper bounds on the $L_p$-errors of $T_f$ whose dependence on $n$, on $s$, and on the dimension or complexity of the parameter matches the minimax lower bounds we obtain. We also give conditions under which these estimators are asymptotically normal and asymptotically efficient.

We develop these results in three settings: models composed of a large number of independent low-dimensional components, high-dimensional exponential families, and functionals of covariance operators in infinite-dimensional subgaussian models, where the complexity of the model is measured by the effective rank of the covariance operator.