Georgia Institute of Technology College of Sciences

How to achieve the optimal control for general stochastic nonlinear is notoriously difficult, which becomes even more difficult by involving learning and exploration for unknown dynamics in reinforcement learning setting. In this talk, I will present our recent work on exploiting the power of representation in RL to bypass these difficulties. Specifically, we designed practical algorithms for extracting useful representations, with the goal of improving statistical and computational efficiency in exploration vs. exploitation tradeoff and empirical performance in RL. We provide rigorous theoretical analysis of our algorithm, and demonstrate the practical superior performance over the existing state-of-the-art empirical algorithms on several benchmarks. 

Recent advancements in operator-type neural networks, such as Fourier Neural Operator (FNO) and Deep Operator Network (DeepONet), have shown promising results in approximating the solutions of spatial-temporal Partial Differential Equations (PDEs). However, these neural networks often entail considerable training expenses, and may not always achieve the desired accuracy required in many scientific and engineering disciplines. In this paper, we propose a new operator learning framework to address these issues. The proposed paradigm leverages the traditional wisdom from numerical PDE theory and techniques to refine the pipeline of existing operator neural networks. Specifically, the proposed architecture initiates the training for a single or a few epochs for the operator-type neural networks in consideration, concluding with the freezing of the model parameters. The latter are then fed into an error correction scheme: a single parametrized linear spectral layer trained with a convex loss function defined through a reliable functional-type a posteriori error estimator.This design allows the operator neural networks to effectively tackle low-frequency errors, while the added linear layer addresses high-frequency errors. Numerical experiments on a commonly used benchmark of 2D Navier-Stokes equations demonstrate improvements in both computational time and accuracy, compared to existing FNO variants and traditional numerical approaches.

In this talk, we consider a finite-horizon optimal control problem of stochastic reaction-diffusion equations. First, we apply the spike variation method which relies on introducing the first and second order adjoint state. We give a novel characterization of the second order adjoint state as the solution to a backward SPDE. Using this representation, we prove the maximum principle for controlled SPDEs.

In the second part, we present a numerical algorithm that allows the efficient approximation of optimal controls in the case of stochastic reaction-diffusion equations with additive noise by first reducing the problem to controls of feedback form and then approximating the feedback function using finitely based approximations. Numerical experiments using artificial neural networks as well as radial basis function networks illustrate the performance of our algorithm.

This talk is based on joint work with Wilhelm Stannat and Alexander Vogler. Talk will also be streamed: https://gatech.zoom.us/j/93808617657?pwd=ME44NWUxbk1NRkhUMzRsK3c0ZGtvQT09

Each point x in Gr(r, n) corresponds to an r × n matrix Ax which gives rise to a matroid Mx on its columns. Gel’fand, Goresky, MacPherson, and Serganova showed that the sets {y ∈ Gr(r, n)|My = Mx} form a stratification of Gr(r, n) with many beautiful properties. However, results of Mnëv and Sturmfels show that these strata can be quite complicated, and in particular may have arbitrary singularities. We study the ideals Ix of matroid varieties, the Zariski closures of these strata. We construct several classes of examples based on theorems from projective geometry and describe how the Grassmann-Cayley algebra may be used to derive non-trivial elements of Ix geometrically when the combinatorics of the matroid is sufficiently rich.