Monday, November 29, 2021 - 4:00pm to 5:00pm
Event Calendar Category
LIDS Seminar Series
Speaker Name
Doina Precup
Affiliation
McGill University
Zoom meeting id
93266970951
Join Zoom meeting
https://mit.zoom.us/j/93266970951
Abstract
Reinforcement Learning allows intelligent agents to learn by interacting with their environment over time and receiving rewards. Hierarchical Reinforcement Learning (HRL) approaches promise to provide more efficient solutions to sequential decision making problems, both in terms of statistical as well as computational efficiency. While this has been demonstrated empirically over time in a variety of tasks, theoretical results quantifying the benefits of such methods are still few and far between. In this talk, I will discuss the theoretical underpinnings of HRL in the framework of Markov and semi-Markov Decision Processes. I will describe the kind of structure in a Markov decision process which gives rise to efficient HRL methods. Specifically, we formalize the intuition that HRL can exploit repeating sub-structures. We show that, under reasonable assumptions, such algorithms can achieve statistical efficiency, as established through a finite-time regret bound, as well as near-optimal and computationally efficient planning, using hierarchical models.
Biography
Doina Precup splits her time between McGill University, where she co-directs the Reasoning and Learning Lab in the School of Computer Science, and DeepMind Montreal, where she has led the research team since its formation in October 2017. Her research interests are in the areas of reinforcement learning, deep learning, time series analysis, and diverse applications of machine learning in health care, automated control, and other fields. Her origial paper on temporal anstraction in reinforcement learning received the Classic Paper award from the Artificial Intelligence journal in 2019. Dr. Precup became a senior member of the Association for the Advancement of Artificial Intelligence in 2015, Canada Research Chair in Machine Learning in 2016, Senior Fellow of the Canadian Institute for Advanced Research in 2017, and received a Canada CIFAR AI (CCAI) Chair in 2018. Dr. Precup is also involved in activities supporting the organization of Mila (the Quebec AI Institute) and the wider Montreal and Quebec AI ecosystem, and she has co-founded the AI4Good summer lab, whose goal is to improve gender diversity in machine learning.