Computational challenges in operational data assimilation: problems and solutions

Series
Time
Friday, October 21, 2022 - 11:00am for 1 hour (actually 50 minutes)
Location
Online
Speaker
Ivo Pasmans – University of Reading, National Center for Earth Observation
Organizer
Jorge Gonzalez

https://gatech.zoom.us/j/95197085752?pwd=WmtJUVdvM1l6aUJBbHNJWTVKcVdmdz09

Operational weather and ocean forecasting proceeds as a sequence of time intervals. During each interval numerical models produce a forecast, observations are collected and a comparison between the two is made. This comparison is used, in a process called data assimilation (DA), to construct observation-informed initial conditions for the forecast in the next time interval. Many DA algorithms are in use, but they all share the need to solve a high-dimensional (>1010) system of linear equations. Constructing and solving this system in the limited amount of time available between the reception of the observations and the start of the next time interval is highly non-trivial for three reasons. 1) As the numerical models are computationally demanding, it is generally impossible to construct the full linear system. 2) Its high dimensionality makes it impossible to store the system as a matrix in memory. Consequently, it is not possible to directly invert it. 3) The operational time-constraints strongly limit the number of iterations that can be used by iterative linear solvers. By adapting DA algorithms to use parallelization, it is possible to leverage the computational power of superclusters to construct a high-rank approximation to the linear system and solve it using less then ~20 iterations. In this talk, I will first introduce the two most popular families of DA algorithms: Kalman filters and variational DA. After this, I will discuss some of the adaptations that have been developed to enable parallelization. Among these are ensemble Kalman filters, domain localization, the EVIL (Ensemble Variational Integrated Localized) and saddle point algorithms.