Georgia Institute of Technology College of Sciences

Large language models exhibit a remarkable ability to learn and adapt from context without updating their parameters. In this talk, I will present our recent work on both understanding how foundation models extract task information from context and designing contexts that enable continual improvement during inference. I will first discuss a geometric analysis on in-context learning, providing new insights into how task representations emerge and evolve across layers. I will then discuss how these insights motivate a broader paradigm of inference-time learning, in which context is actively constructed rather than passively consumed. Building on iterative refinement frameworks such as AlphaEvolve, we view context as an evolving memory that stores hypotheses, intermediate solutions, and feedback. Drawing inspiration from optimization and sequential Monte Carlo, we develop principled approaches for designing and updating context over time. Overall, understanding how models read context and how we can systematically write and evolve context may provide a foundation for the next generation of adaptive AI systems.

In cosmology, a basic explanation of the observed concentration of mass in singular structures is provided by the Zeldovich approximation, which takes the form of free-streaming flow for perturbations of a uniform Einstein-de Sitter universe in co-moving coordinates. The adhesion model suppresses multi-streaming by introducing viscosity. We study mass flow in this model by analysis of Lagrangian advection in the zero-viscosity limit. Under mild conditions, we show that a unique limiting Lagrangian semi-flow exists. Limiting particle paths stick together after collision and are characterized uniquely by a differential inclusion. The absolutely continuous part of the mass measure satisfies a Monge-Ampère equation related to convexification of the free-streaming velocity potential.

The use of Monge-Ampère equations and optimal transport theory for the reconstruction of inverse Lagrangian maps in cosmology was introduced in work of Brenier and Frisch et al (2003). We show that the singular part of the mass measure can differ from the Alexandrov solution to the Monge-Ampère equation, however, when flows along singular structures merge, as shown by analysis of a 2D Riemann problem. In a neighborhood of merging singular structures in our examples, we show that reconstruction yielding a monotone Lagrangian map cannot be exact a.e., even off of the singularities themselves.

Deep neural network–based surrogate models have recently gained traction for solving fluid-flow partial differential equations, but their reliance on global interpolation often demands large, computationally expensive architectures and extensive training data. Neural networks with local converging inputs (NNLCI) offer a contrasting strategy. By restricting attention to the local domain of dependence and using converging coarse-grid solutions as inputs, NNLCI dramatically reduces computational cost and data requirements while achieving strong generalization. In this work, we extend the NNLCI framework to the three‑dimensional Stokes equations and introduce a new subdomain data generation methodology specifically tailored for NNLCI, enabling high‑fidelity prediction while completely eliminating the need to compute fine‑grid numerical solutions on the full domain at any stage of the computing process. This innovation eliminates the most computationally intensive component of 3D simulations at its root.

We introduce in-context operator learning on probability measure spaces for optimal transport (OT). The goal is to learn a single solution operator that maps a pair of distributions to the OT map, using only few-shot samples from each distribution as a prompt and without gradient updates at inference. We parameterize the solution operator and develop scaling-law theory in two regimes. In the nonparametric setting, when tasks concentrate on a low-intrinsic-dimension manifold of source– target pairs, we establish generalization bounds that quantify how in-context accuracy scales with prompt size, intrinsic task dimension, and model capacity. In the parametric setting (e.g., Gaussian families), we give an explicit architecture that recovers the exact OT map in context and provide finite-sample excess-risk bounds. Our numerical experiments on synthetic transports and generative modeling benchmarks validate the framework.