- Series
- Applied and Computational Mathematics Seminar
- Time
- Monday, March 28, 2022 - 2:00pm for 1 hour (actually 50 minutes)
- Location
- https://gatech.zoom.us/j/96551543941 (note: Zoom, not Bluejeans)
- Speaker
- Prof. Bao Wang – University of Utah – bwang@math.utah.edu – http://www.math.utah.edu/~bwang/index.html
- Organizer
- Molei Tao
We will present a new class of continuous-depth deep neural networks that were motivated by the ODE limit of the classical momentum method, named heavy-ball neural ODEs (HBNODEs). HBNODEs enjoy two properties that imply practical advantages over NODEs: (i) The adjoint state of an HBNODE also satisfies an HBNODE, accelerating both forward and backward ODE solvers, thus significantly accelerate learning and improve the utility of the trained models. (ii) The spectrum of HBNODEs is well structured, enabling effective learning of long-term dependencies from complex sequential data.
Second, we will extend HBNODE to graph learning leveraging diffusion on graphs, resulting in new algorithms for deep graph learning. The new algorithms are more accurate than existing deep graph learning algorithms and more scalable to deep architectures, and also suitable for learning at low labeling rate regimes. Moreover, we will present a fast multipole method-based efficient attention mechanism for modeling graph nodes interactions.
Third, if time permits, we will discuss proximal algorithms for accelerating learning continuous-depth neural networks.