Mutual Information in Deep RL: Toward a Single Tractable Reward Function for General Intelligence

Monday, April 5, 2021 - 11:30am to 12:30pm

Event Calendar Category

CSAIL

Speaker Name

Shane Gu

Affiliation

Google Brain

Zoom meeting id

643989

Join Zoom meeting

https://mit.zoom.us/j/94172410878

Abstract

What is intelligence? How to measure it? Why robotics over games?: I will discuss fundamental questions for a journey toward a (form of) general intelligence. While appealing proposals have been made before (Legg & Hutter 2005, Chollet 2019) on measuring intelligence, in this talk I will propose an arguably more limited but tractable measure based on mutual information (MI) maximization, or empowerment, from multiple perspectives. I will present (1) how a model-based approach can enable efficient unsupervised learning on real robots (DADS 2020), (2) how unsupervised MI-based skill discovery can be better studied through viewing it as representation learning for goal-conditioned RL (Variational GCRL 2021), and (3) how MI between reward and policy parameters can estimate learning difficulty of RL environments (PIC 2021).

 

Biography

Shane holds PhD in Machine Learning from the University of Cambridge in the UK and the Max Planck Institute for Intelligent Systems in Tübingen Germany, supervised by Richard E. Turner, Zoubin Ghahramani, and Bernhard Schölkopf, and also mentored by Sergey Levine/Ilya Sutskever at UC Berkeley/Google Brain and Timothy Lillicrap at DeepMind as a student researcher. Shane holds B.ASc. in Engineering Science from the University of Toronto, where he completed his thesis with Geoffrey Hinton in distributed training of neural networks using evolutionary algorithms, and worked with Steve Mann, to develop real-time HDR capture for wearable cameras. Shane also volunteered as a Scientist at CDL, a tech-startup incubator in Canada, and as a visiting scholar at Stanford University and University of Tokyo. Shane's academic work received Best Paper Award at CoRL 2019, Google Focused Research Award, Cambridge-Tübingen PhD Fellowship, and NSERC Scholarship, and was featured in Google Research Blogpost and MIT Technology Review.

Shane is a Japan-born Chinese Canadian, and he speaks, reads, and writes in three languages. Having lived in Japan, China, Canada, the US, the UK, and Germany, he goes under multiple names: Shane Gu, Shixiang Gu, 顾世翔, 顧世翔(ぐう せいしょう).