Georgia Institute of Technology College of Sciences

Neural networks have become powerful tools for solving Partial Differential Equations (PDEs), with wide-ranging applications in engineering, physics, and biology. In this talk, we explore the performance of deep neural networks in solving PDEs, focusing on two primary sources of error: approximation error, and generalization error. The approximation error captures the gap between the exact PDE solution and the neural network’s hypothesis space. Generalization error arises from the challenges of learning from finite samples. We begin by analyzing the approximation capabilities of deep neural networks, particularly under Sobolev norms, and discuss strategies to overcome the curse of dimensionality. We then present generalization error bounds, offering insight into when and why deep networks can outperform shallow ones in solving PDEs.

Data assimilation techniques are crucial for correcting trajectories when modeling complex dynamical systems. The Latent Ensemble Score Filter (Latent-EnSF), our recently developed data assimilation method, has shown great promise in high-dimensional and nonlinear data assimilation problems with sparse observations. However, this method faces the challenge of high computational cost due to the expensive forward simulation. In this talk, we present Latent Dynamics EnSF (LD-EnSF), a novel methodology that evolves the neural dynamics in a low-dimensional latent space and significantly accelerates the data assimilation process.

To achieve this, we introduce a novel variant of Latent Dynamics Networks (LDNets) to effectively capture the system's dynamics within a low-dimensional latent space. Additionally, we propose a new method for encoding sparse observations into the latent space using recurrent neural networks. We demonstrate the robustness, accuracy, and efficiency of the proposed methods and their limitations for complex dynamical systems with highly sparse (in both space and time) and noisy observations, including shallow water wave propagation for tsunami modeling, FourCastNet in numerical weather prediction, and Kolmogorov flow that exhibits chaotic and turbulent phenomena.

Recent advances in deep learning have introduced numerous innovative frameworks for solving high-dimensional mean-field games (MFGs). However, these methods are often limited to solving single-instance MFGs and require extensive computational time for each instance, presenting challenges for practical applications.

In this talk, I will present our recent work on a novel framework for learning the MFG solution operator. Our model takes MFG instances as input and directly outputs their solutions in a single forward pass, significantly improving computational efficiency. Our method offers two key advantages: (1) it is discretization-free, making it particularly effective for high-dimensional MFGs, and (2) it can be trained without requiring supervised labels, thereby reducing the computational burden of preparing training datasets common in existing operator learning methods. If time permits, I will also explore connections between this framework and in-context learning, highlighting its broader implications and potential for further advancements.

Equation learning has been a holy grail of scientific research for decades. Only recently has the capability of learning equations directly from data become a computationally feasible task, due to the availability of high-resolution data and fast algorithms capable of surpassing the inherent combinatorial complexity of most model classes. Weak form equation learning has arisen as an advantageous framework for efficiently selecting models from data with noise and nonsmoothness, qualities inherent to observed data. By viewing the dynamics through the guise of test functions, the weak form affords a flexible representation of the governing equations that naturally incorporates these data maladies. More generally, the weak form has been shown to reveal alternative dynamical descriptions, such as coarse-grained and reduced-order models, opening the door to hierarchical model discovery. In this talk I will give a broad overview of historical advances in weak form equation learning and parameter inference, from the 1950s to WSINDy and more recent algorithms. I will then give an outlook for future research directions in this field, in light of now-known computational limitations and recently demonstrated successes, both theoretical and applied, with applications to molecular dynamics, plasma physics, cell biology, and weather forecasting.