Georgia Institute of Technology College of Sciences

Many emerging applications of artificial intelligence, from analyzing scientific literature to developing complex software, rely on language models to process increasingly large amounts of information. Yet the ability to accept a long input, or context, does not guarantee that a model can use it effectively. In this talk, I will investigate how language models use long contexts through mathematical models of transformers, the architecture underlying many language models.

I will first study how language models learn from examples supplied in their input without changing their parameters, an ability known as in-context learning. A longer context can contain more examples, but how rapidly can a model benefit from this growing body of evidence? Using mean-field analysis, we establish quantitative guarantees for learning from finitely many examples.

The benefit of additional information, however, also depends on whether a model can focus on useful evidence amid irrelevant material. I will therefore study temperature scaling, a practical strategy that adjusts how selectively a model weighs different parts of its input. Drawing on extreme-value theory, we identify a critical scaling with context length at which only several selected pieces of information remain influential.

I will conclude by connecting these results to my broader research on mathematical machine learning and highlighting how applied mathematics can inform the design of reliable learning algorithms.

Ptychographic phase retrieval is an ill-posed nonlinear inverse problem typically formulated as large-scale nonconvex optimization. I will present stochastic multigrid methods, beginning with the known-probe case as a foundation for blind ptychography. The approach uses quadratic surrogate models that majorize the data-misfit objective, together with compatible coarse-grid models that accelerate surrogate minimization. We extend this construction to blind ptychography, where the object and probe must be recovered jointly. Geometric-mean updates coordinate changes to the object and probe within each scan, while intensity-weighted minibatch synthesis combines the resulting local updates across scan positions. I will discuss the mathematical analysis and present results using synthetic data and experimental X-ray measurements.

 We develop adaptive Chebyshev and Fourier-Chebyshev spectral collocation methods for the nonlinear prescribed mean curvature equations used to model the height of a fluid interface.  The non-linearity is treated with a Newton’s method.  Various geometries and symmetries are considered, and the resulting methods are used to solve the free boundary problems that arise from the equilibrium of a floating drop.

Finding a path between two probability distributions is a recurring problem in optimal transport, generative modeling, population dynamics, and control. A broad class of these problems can be formulated through an action functional defined over curves of probability distributions. The corresponding first-order optimality conditions lead to a coupled Hamilton–Jacobi and continuity system, which can be written as a boundary-value Hamiltonian flow in density space.

Computing these paths from samples becomes challenging in moderate and high dimensions. In this talk, I will discuss three approaches for learning and optimizing them: Parametric Density Path Optimization (PDPO), Hamiltonian Rectification (HR), and Neural Multiple Shooting (NMS) with a focus on NMS. 

The central idea of NMS is to reformulate the boundary-value problem in density space as a multiple-shooting problem in particle space. Since the particle dynamics are determined by the prescribed Hamiltonian system once the momentum is specified, the unknowns reduce to momentum maps that initialize a sequence of Hamiltonian initial-value problems. The resulting hybrid method combines neural approximation with classical multiple shooting: neural networks determine the momentum at the beginning of each shooting segment, while an ODE solver propagates the corresponding trajectories according to the prescribed dynamics.

I will illustrate the method through examples involving obstacle avoidance, interacting particle systems, double-integrator dynamics with smooth drag, and unicycle dynamics with obstacles and mean-field interactions.