Seminars and Colloquia by Series

Physics-guided interpretable data-driven simulations

Series
Applied and Computational Mathematics Seminar
Time
Monday, September 18, 2023 - 14:00 for 1 hour (actually 50 minutes)
Location
https://gatech.zoom.us/j/98355006347
Speaker
Youngsoo ChoiLawrence Livermore National Laboratory

Please Note: This is a virtual seminar. Speaker Bio: Youngsoo is a computational math scientist in Center for Applied Scientific Computing (CASC) under Computing directorate at LLNL. His research focuses on developing efficient reduced order models for various physical simulations for time-sensitive decision-making multi-query problems, such as inverse problems, design optimization, and uncertainty quantification. His expertise includes various scientific computing disciplines. Together with his team and collaborators, he has developed powerful model order reduction techniques, such as machine learning-based nonlinear manifold, space–time reduced order models, and latent space dynamics identification methods for nonlinear dynamical systems. He has also developed the component-wise reduced order model optimization algorithm, which enables fast and accurate computational modeling tools for lattice-structure design. He is currently leading data-driven physical simulation team at LLNL, with whom he developed the open source codes, libROM (i.e., https://www.librom.net), LaghosROM (i.e., https://github.com/CEED/Laghos/tree/rom/rom), LaSDI (i.e., https://github.com/LLNL/LaSDI), gLaSDI (i.e., https://github.com/LLNL/gLaSDI), and GPLaSDI (i.e., https://github.com/LLNL/GPLaSDI). He earned his undergraduate degree in Civil and Environmental Engineering from Cornell University and his Ph.D. degree in Computational and Mathematical Engineering from Stanford University. He was a postdoctoral scholar at Sandia National Laboratories and Stanford University prior to joining LLNL in 2017.

A computationally expensive physical simulation is a huge bottleneck to advance in science and technology. Fortunately, many data-driven approaches have emerged to accelerate those simulations, thanks to the recent advancements in machine learning (ML) and artificial intelligence. For example, a well-trained 2D convolutional deep neural network can predict the solution of the complex Richtmyer–Meshkov instability problem with a speed-up of 100,000x [1]. However, the traditional black-box ML models do not incorporate existing governing equations, which embed underlying physics, such as conservation of mass, momentum, and energy. Therefore, the black-box ML models often violate important physics laws, which greatly concern physicists, and require big data to compensate for the missing physics information. Additionally, it comes with other disadvantages, such as non-structure-preserving, computationally expensive training phase, non-interpretability, and vulnerability in extrapolation. To resolve these issues, we can bring physics into the data-driven framework. Physics can be incorporated into different stages of data-driven modeling, i.e., the sampling stage and model-building stage. Physics-informed greedy sampling procedure minimizes the number of required training data for a target accuracy [2]. Physics-guided data-driven model better preserves the physical structure and is more robust in extrapolation than traditional black-box ML models. Numerical results, e.g., hydrodynamics [3,4], particle transport [5], plasma physics, and 3D printing, will be shown to demonstrate the performance of the data-driven approaches. The benefits of the data-driven approaches will also be illustrated in multi-query decision-making applications, such as design optimization [6,7].

 

Reference
[1] Jekel, Charles F., Dane M. Sterbentz, Sylvie Aubry, Youngsoo Choi, Daniel A. White, and Jonathan L. Belof. “Using Conservation Laws to Infer Deep Learning Model Accuracy of Richtmyer-meshkov Instabilities.” arXiv preprint arXiv:2208.11477 (2022).
[2] He, Xiaolong, Youngsoo Choi, William D. Fries, Jon Belof, and Jiun-Shyan Chen. “gLaSDI: Parametric Physics-informed Greedy Latent Space Dynamics Identification.” arXiv preprint arXiv:2204.12005 (2022).
[3] Copeland, Dylan Matthew, Siu Wun Cheung, Kevin Huynh, and Youngsoo Choi. “Reduced order models for Lagrangian hydrodynamics.” Computer Methods in Applied Mechanics and Engineering 388 (2022): 114259.
[4] Kim, Youngkyu, Youngsoo Choi, David Widemann, and Tarek Zohdi. “A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder.” Journal of Computational Physics 451 (2022): 110841.
[5] Choi, Youngsoo, Peter Brown, William Arrighi, Robert Anderson, and Kevin Huynh. “Space–time reduced order model for large-scale linear dynamical systems with application to Boltzmann transport problems.” Journal of Computational Physics 424 (2021): 109845.
[6] McBane, Sean, and Youngsoo Choi. “Component-wise reduced order model lattice-type structure design.” Computer methods in applied mechanics and engineering 381 (2021): 113813.
[7] Choi, Youngsoo, Gabriele Boncoraglio, Spenser Anderson, David Amsallem, and Charbel Farhat. “Gradient-based constrained optimization using a database of linear reduced-order models.” Journal of Computational Physics 423 (2020): 109787.

 

Recent Advances in Finite Element Methods for Solving Poisson-Nernst-Planck Ion Channel Models

Series
Applied and Computational Mathematics Seminar
Time
Monday, August 28, 2023 - 14:00 for 1 hour (actually 50 minutes)
Location
Skiles 005 and https://gatech.zoom.us/j/98355006347 (to be confirmed)
Speaker
Dexuan XieUniversity of Wisconsin-Milwaukee
Ion channels are a class of proteins embedded in biological membranes, acting as biological devices or 'valves' for cells and playing a critical role in controlling various biological functions. To compute macroscopic ion channel kinetics, such as Gibbs free energy, electric currents, transport fluxes, membrane potential, and electrochemical potential, Poisson-Nernst-Planck ion channel (PNPIC) models have been developed as systems of nonlinear partial differential equations. However, they are difficult to solve numerically due to solution singularities, exponential nonlinearities, multiple physical domain issues, and the requirement of ionic concentration positivity. In this talk, I will present the recent progress we made in the development of finite element methods for solving PNPIC models. Specifically, I will introduce our improved PNPIC models and describe the mathematical and numerical techniques we utilized to develop efficient finite element iterative methods. Additionally, I will introduce the related software packages we developed for a voltage-dependent anion-channel protein and a mixture solution of multiple ionic species. Finally, I will present numerical results to demonstrate the fast convergence of our iterative methods and the high performance of our software package. This work was partially supported by the National Science Foundation through award number DMS-2153376 and the Simons Foundation through research award 711776.

Two Phases of Scaling Laws for Nearest Neighbor Classifiers

Series
Applied and Computational Mathematics Seminar
Time
Thursday, May 25, 2023 - 10:30 for 1 hour (actually 50 minutes)
Location
https://gatech.zoom.us/j/98355006347
Speaker
Jingzhao ZhangTsinghua University

Please Note: Special time & day. Remote only.

A scaling law refers to the observation that the test performance of a model improves as the number of training data increases. A fast scaling law implies that one can solve machine learning problems by simply boosting the data and the model sizes. Yet, in many cases, the benefit of adding more data can be negligible. In this work, we study the rate of scaling laws of nearest neighbor classifiers. We show that a scaling law can have two phases: in the first phase, the generalization error depends polynomially on the data dimension and decreases fast; whereas in the second phase, the error depends exponentially on the data dimension and decreases slowly. Our analysis highlights the complexity of the data distribution in determining the generalization error. When the data distributes benignly, our result suggests that nearest neighbor classifier can achieve a generalization error that depends polynomially, instead of exponentially, on the data dimension.

Uncovering the Law of Data Separation in Deep Learning

Series
Applied and Computational Mathematics Seminar
Time
Monday, April 17, 2023 - 14:00 for 1 hour (actually 50 minutes)
Location
Skiles 005 and https://gatech.zoom.us/j/98355006347
Speaker
Prof. Weijie SuUniversity of Pennsylvania (Wharton)

Please Note: The speaker will present in person.

In this talk, we will investigate the emergence of geometric patterns in well-trained deep learning models by making use of a layer-peeled model and the law of equi-separation. The former is a nonconvex optimization program that models the last-layer features and weights. We use the model to shed light on the neural collapse phenomenon of Papyan, Han, and Donoho, and to predict a hitherto-unknown phenomenon that we term minority collapse in imbalanced training.
 
The law of equi-separation is a pervasive empirical phenomenon that describes how data are separated according to their class membership from the bottom to the top layer in a well-trained neural network. We will show that, through extensive computational experiments, neural networks improve data separation through layers in a simple exponential manner. This law leads to roughly equal ratios of separation that a single layer is able to improve, thereby showing that all layers are created equal. We will conclude the talk by discussing the implications of this law on the interpretation, robustness, and generalization of deep learning, as well as on the inadequacy of some existing approaches toward demystifying deep learning.
 

Mathematical Foundations of Graph-Based Bayesian Semi-Supervised Learning

Series
Applied and Computational Mathematics Seminar
Time
Monday, April 10, 2023 - 14:00 for 1 hour (actually 50 minutes)
Location
Skiles 005 and https://gatech.zoom.us/j/98355006347
Speaker
Prof. Daniel Sanz-AlonsoU Chicago

Please Note: Speaker will present in person

Semi-supervised learning refers to the problem of recovering an input-output map using many unlabeled examples and a few labeled ones. In this talk I will survey several mathematical questions arising from the Bayesian formulation of graph-based semi-supervised learning. These questions include the modeling of prior distributions for functions on graphs, the derivation of continuum limits for the posterior, the design of scalable posterior sampling algorithms, and the contraction of the posterior in the large data limit.

New gradient sliding results on convex optimization with smoothness structure

Series
Applied and Computational Mathematics Seminar
Time
Monday, April 3, 2023 - 14:00 for 1 hour (actually 50 minutes)
Location
Skiles 005 and https://gatech.zoom.us/j/98355006347
Speaker
Yuyuan OuyangClemson University

In this talk, we present new gradient sliding results for constrained convex optimization with applications in image reconstruction and decentralized distributed optimization. Specifically, we will study classes of large-scale problems that minimizes a convex objective function over feasible set with linear constraints. We will show that by exploring the gradient sliding technique, the number of gradient evaluations of the objective function can be reduced by exploring the smoothness structure. Our results could lead to new decentralized algorithms for multi-agent optimization with graph topology invariant gradient/sampling complexity and new ADMM algorithms for solving total variation image reconstruction problems with accelerated gradient complexity.

 

Application of NNLCIs to the scattering of electromagnetic waves around curved PECs

Series
Applied and Computational Mathematics Seminar
Time
Monday, March 27, 2023 - 14:00 for 1 hour (actually 50 minutes)
Location
Skiles 005 and https://gatech.zoom.us/j/98355006347
Speaker
Hwi LeeGeorgia Tech Math

In this talk, we demonstrate the application of Neural Networks with Locally Converging Inputs (NNLCI) to simulate the scattering of electromagnetic waves around two-dimensional perfect electric conductors (PEC). The NNLCIs are designed to output high-fidelity numerical solutions from local patches of two coarse grid numerical solutions obtained by a convergent numerical scheme. Once trained, the NNLCIs can play the role of a computational cost-saving tool for repetitive computations with varying parameters. To generate the inputs to our NNLCI, we design on uniform rectangular grids a second-order accurate finite difference scheme that can handle curved PEC boundaries systematically. More specifically, our numerical scheme is based on the Back and Forth Error Compensation and Correction method together with the construction of ghost points via a level set framework, PDE-based extension technique, and what we term guest values. We illustrate the performance of our NNLCI subject to variations in incident waves as well as PEC boundary geometries.

The Surprising Robustness and Computational Efficiency of Weak Form System Identification

Series
Applied and Computational Mathematics Seminar
Time
Monday, March 13, 2023 - 14:00 for 1 hour (actually 50 minutes)
Location
Skiles 005 (ZOOM)
Speaker
David BortzUniversity of Colorado, Boulder

Recent advances in data-driven modeling approaches have proven highly successful in a wide range of fields in science and engineering. In this talk, I will briefly discuss several ubiquitous challenges with the conventional model development / discretization / parameter inference / model revision loop that our methodology attempts to address. I will present our weak form methodology which has proven to have surprising performance properties. In particular, I will describe our equation learning (WSINDy) and parameter estimation (WENDy) algorithms.  Lastly, I will discuss applications to several benchmark problems illustrating how our approach addresses several of the above issues and offers advantages in terms of computational efficiency, noise robustness, and modest data needs (in an online learning context).

Optimal Transport for Averaged Control

Series
Applied and Computational Mathematics Seminar
Time
Monday, March 6, 2023 - 14:00 for 1 hour (actually 50 minutes)
Location
Skile 005 and https://gatech.zoom.us/j/98355006347
Speaker
Dr. Daniel Owusu AduUGA

We study the problem of designing a robust parameter-independent feedback control input that steers, with minimum energy, the average of a linear system submitted to parameter perturbations where the states are initialized and finalized according to a given initial and final distribution. We formulate this problem as an optimal transport problem, where the transport cost of an initial and final state is the minimum energy of the ensemble of linear systems that have started from the initial state and the average of the ensemble of states at the final time is the final state. The by-product of this formulation is that using tools from optimal transport, we are able to design a robust parameter-independent feedback control with minimum energy for the ensemble of uncertain linear systems. This relies on the existence of a transport map which we characterize as the gradient of a convex function.

Generalization and sampling from the dynamics perspective

Series
Applied and Computational Mathematics Seminar
Time
Monday, February 27, 2023 - 14:00 for 1 hour (actually 50 minutes)
Location
Skiles 005 and https://gatech.zoom.us/j/98355006347
Speaker
Prof. Nisha ChandramoorthyGT CSE

Please Note: Speaker will present in person

In this talk, we obtain new computational insights into two classical areas of statistics: generalization and sampling. In the first part, we study generalization: the performance of a learning algorithm on unseen data. We define a notion of generalization for non-converging training with local descent approaches via the stability of loss statistics. This notion yields generalization bounds in a similar manner to classical algorithmic stability. Then, we show that more information from the training dynamics provides clues to generalization performance.   

In the second part, we discuss a new method for constructing transport maps. Transport maps are transformations between the sample space of a source (which is generally easy to sample) and a target (typically non-Gaussian) probability distribution. The new construction arises from an infinite-dimensional generalization of a Newton method to find the zero of a "score operator". We define such a score operator that gives the difference of the score -- gradient of logarithm of density -- of a transported distribution from the target score. The new construction is iterative, enjoys fast convergence under smoothness assumptions, and does not make a parametric ansatz on the transport map.

Pages