Variational Formulations and Distributed Convex Optimization Methods for Modern Data Science Applications

Tuesday, April 13, 2021 - 3:00pm to 3:55pm

Event Calendar Category

LIDS Seminar Series

Speaker Name

Todd Coleman

Affiliation

University of California San Diego

Zoom meeting id

913 8477 9474

Join Zoom meeting

https://mit.zoom.us/j/91384779474

Abstract

The need to reason about uncertainty in large, complex, and multi-modal datasets has become increasingly common. The ability to transform samples from one distribution P to another distribution Q enables the solution to many problems in data science and machine learning. Performing provably good transformations, in general, still comprises computational difficulties, especially in high dimensions. Here, we consider the problem of computing such "measure transport maps" efficiently and at scale. Under the mild assumptions that P need not be known but can be sampled from, that the density of Q is known up to a proportionality constant, and that Q is log-concave, we provide a convex optimization formulation.  This approach, involving sequentially solving Wasserstein regularized relative entropy minimization problems, is inspired by variational formulations in non-equilibrium thermodynamics and allows for sequential construction of transport maps with exponential convergence in relative entropy.   We provide an empirical risk minimization formulation using maps parametrized by polynomial chaos along with a distributed and convergent algorithm that has been efficiently implemented in GPUs and Amazon Web Services architectures.   We provide examples of this framework, within the context of Bayesian inference, active learning for brain-machine interfaces, reinforcement learning, and generative modeling.  

 

Biography

Todd P. Coleman received B.S. degrees in electrical engineering (summa cum laude), as well as computer engineering (summa cum laude) from the University of Michigan. He received M.S. and Ph.D. degrees from MIT in electrical engineering (minor in mathematics) and did postdoctoral studies at MIT in neuroscience. He is currently a Professor in the Department of Bioengineering at UCSD, where he directs the Neural Interaction Laboratory.  Dr. Coleman’s research is very multi-disciplinary, using tools from applied probability, physiology, and bioelectronics.  His research spans from developing fundamental information theory and machine learning techniques to partnering with clinicians and solving important healthcare challenges.  He has been selected as a National Academy of Engineering Gilbreth Lecturer, as a TEDMED speaker, and as a Fellow of the American Institute for Medical and Biological Engineering.