Georgia Institute of Technology College of Sciences

This dissertation develops algorithms and theory for optimization, sampling, and generative modeling on manifolds, with Lie groups as a central object of study. Lie groups are manifolds with additional group structure; when endowed with a left-invariant Riemannian metric they become Riemannian manifolds, a setting that plays a central role throughout this work.

We first consider optimization on the Stiefel manifold $\mathrm{St}(n,d)$, the set of $n\times d$ matrices with orthonormal columns. By deriving a variational principle on this manifold, we construct the Momentum Stiefel Optimizer, a momentum-based algorithm that exactly preserves the orthogonality constraint at every iteration. The optimizer is applied to suitably-orthogonal attention in transformers and to optimal transport problems, achieving consistent improvements over existing methods.

We then establish quantitative convergence guarantees for momentum optimizers on Lie groups equipped with a left-invariant metric. Using the left-trivialization technique, which maps the curved dynamics to a flat Euclidean space for the momentum variable, we prove the first explicit convergence rates for both Heavy-Ball and Nesterov Accelerated methods on compact Lie groups, with rates that match Euclidean theory in terms of the smoothness and strong-convexity constants.

Next, we develop gauge-equivariant accelerated methods for optimization over the Grassmannian $\mathrm{Gr}(n,d)$, the set of $d$-dimensional subspaces of $\mathbb{R}^n$. Because each subspace has infinitely many orthonormal representatives related by an $\mathrm{O}(d)$ rotation, a naive lift of Stiefel algorithms to the Grassmannian is not gauge-equivariant. We introduce a gauge-fixing pipeline that converts any Stiefel optimizer into a gauge-equivariant Grassmann algorithm, yielding Grassmann Anderson Acceleration and Grassmann NAG, validated on density functional theory and low-rank matrix completion.

We then turn to sampling on Lie groups. By adding tractable noise to the left-trivialized momentum dynamics, we construct the first kinetic (momentum) Langevin Monte Carlo sampler on Lie groups with rigorous nonasymptotic convergence guarantees. The sampler preserves the group structure exactly at every step. Exponential convergence in $W_2$ distance is proved under only compactness of the Lie group and geodesic smoothness of the potential, without any convexity or isoperimetric assumption.

Finally, we address score-based generative modeling on general Riemannian manifolds. The standard denoising score matching framework requires a tractable forward-process transition kernel, which is unavailable on general manifolds because the heat kernel is intractable. We propose splitting diffusion, which lifts the dynamics to the tangent bundle and alternates closed-form stochastic momentum updates in the Euclidean tangent space with deterministic geodesic transport. The resulting transition kernel is closed-form, enabling denoising score matching training on general Riemannian manifolds requiring only exponential map as oracle.

Let $(S,\mathcal{A})$ be a measurable space and let $\mathcal{P}$ be a family of probability distributions on it. Given a Banach space $E$, a mapping $\theta:\mathcal{P}\to E$, and a smooth functional $f:E\to\mathbb{R}$, we consider the problem of estimating $f(\theta(P))$ from i.i.d. observations $X_1,\ldots,X_n\sim P$, where $P\in\mathcal{P}$. We write $f\in C^s(E)$ for a functional of Hölder smoothness $s=m+\rho$, where $m\ge 0$ is an integer and $\rho\in(0,1]$. Our aim is to construct estimators of $f(\theta(P))$ and to study their dependence on the sample size $n$, the smoothness $s$, and the dimension or complexity of the parameter $\theta(P)$.

When $\hat\theta_n$ is a $\sqrt{n}$-consistent base estimator of $\theta(P)$, the plug-in estimator $f(\hat\theta_n)$ is asymptotically efficient in classical low-dimensional models, but in high-dimensional and infinite-dimensional settings its bias is often too large to attain the rate $n^{-1/2}$. Existing bias reduction methods, based on iterated bootstrap or on linear aggregation of plug-in estimators, rely on concentration inequalities for $f(\hat\theta_n)$ that are available only for a limited class of models.

In this dissertation we study a class of estimators $T_f(X_1,\ldots,X_n)$ obtained from a Taylor expansion of $f$ about $\hat\theta_n$, of order determined by $s$, together with a sample split. Their analysis uses only bounds on the moments of the linear and higher order terms of $\hat\theta_n-\theta(P)$, rather than concentration inequalities. For functionals of smoothness $s\ge 1$, we derive upper bounds on the $L_p$-errors of $T_f$ whose dependence on $n$, on $s$, and on the dimension or complexity of the parameter matches the minimax lower bounds we obtain. We also give conditions under which these estimators are asymptotically normal and asymptotically efficient.

We develop these results in three settings: models composed of a large number of independent low-dimensional components, high-dimensional exponential families, and functionals of covariance operators in infinite-dimensional subgaussian models, where the complexity of the model is measured by the effective rank of the covariance operator.

In this dissertation, we use methods of probabilistic combinatorics to work towards a conjecture of Alon and Kim about coloring the edges of k-uniform t-simple hypergraphs. A hypergraph is k-uniform if every edge contains exactly k vertices, and t-simple if any two edges intersect in at most t vertices. The chromatic index of a hypergraph H is the smallest integer N such that one can properly color the edges of H with N colors.

In 1997, Alon and Kim conjectured that if H is a k-uniform t-simple hypergraph with maximum degree D sufficiently large, then the chromatic index \chi'(H) is upper bounded by (t-1+1/t+\epsilon)D. Using probabilistic techniques and a nibble coloring method, we prove a general coloring theorem stating that a k-uniform t-simple hypergraph H with large maximum degree D has chromatic index at most (b+\epsilon)kD, where b is a particular parameter derived from local structural information about H.

We use structural techniques to prove sharp upper bounds on b in the 3-uniform 2-simple, and 3-uniform 3-simple cases. In particular, we deduce as a corollary that for sufficiently large D, every 3-uniform 2-simple and 3-simple hypergraph of maximum degree at most D has chromatic index at most 2.358D and 2.679D, respectively. We also prove that for sufficiently large D, every 3-uniform 2-simple hypergraph has fractional chromatic index at most 2D.

We first investigate reproducing pairs in Hilbert spaces, with a focus on the discrete case. Reproducing pairs generalize frames and consist of two sequences $\Psi$ and $\Phi$, along with a bounded invertible operator $S_{\Psi,\Phi}$. The work examines sequences that are overcomplete by one element—that is, they become exact upon removal of a single element. A central result shows that if such a sequence admits a reproducing partner, the resulting exact subsequence must form a Schauder basis. This implies that systems like the Gaussian Gabor system at critical density, which lacks a Schauder basis, cannot have a reproducing partner. The result is further generalized to sequences overcomplete by finitely many elements.

Next, we introduce exponential reproducing pairs, where the sequences are weighted exponentials. The associated operator $S_{g\gamma}$ acts as a multiplication operator, and necessary and sufficient conditions are established for when a pair $(g, \gamma)$ forms an exponential reproducing pair.

Lastly, by extending a 2012 result of Heil and Yoon, we develop a two-dimensional theory for weighted exponential systems. It characterizes when weighted double exponential systems are minimal and complete, and provides necessary and sufficient conditions for exactness of arbitrary weighted systems.

Zoom Link: https://gatech.zoom.us/j/93221716846