Georgia Institute of Technology College of Sciences

Part I (SAM as an Optimal Relaxation of Bayes) Dr. Thomas Moellenhoff

Sharpness-aware minimization (SAM) and related adversarial deep-learning methods can drastically improve generalization, but their underlying mechanisms are not yet fully understood. In this talk, I will show how SAM can be interpreted as optimizing a relaxation of the Bayes objective where the expected negative-loss is replaced by the optimal convex lower bound, obtained by using the so-called Fenchel biconjugate. The connection enables a new Adam-like extension of SAM to automatically obtain reasonable uncertainty estimates, while sometimes also improving its accuracy.

Part II (Lie Group updates for Learning Distributions on Machine Learning Parameters) Dr. Eren Mehmet Kıral

I will talk about our recent paper https://arxiv.org/abs/2303.04397 with Thomas Möllenhoff and Emtiyaz Khan, and other related results. Bayesian Learning learns a distribution over the model parameters, allowing for different descriptions of the same data. This is (contrary to classical learning which "bets-it-all" on a single set of parameters in describing a given dataset and making predictions. We focus on classes of distributions which have a transitive Lie group action on them given by pushforwards of an action on the parameter space. I will also specialize to a few concrete Lie groups and show distinct learning behavior.