Hierarchical Reinforcement Learning: Framework ad Recent Results

Monday, November 29, 2021 - 4:00pm to 5:00pm

Event Calendar Category

LIDS Seminar Series

Speaker Name

Doina Precup


McGill University

Zoom meeting id


Join Zoom meeting



Reinforcement Learning allows intelligent agents to learn by interacting with their environment over time and receiving rewards. Hierarchical Reinforcement Learning (HRL) approaches promise to provide more efficient solutions to sequential decision making problems, both in terms of statistical as well as computational efficiency. While this has been demonstrated empirically over time in a variety of tasks, theoretical results quantifying the benefits of such methods are still few and far between. In this talk, I will discuss the theoretical underpinnings of HRL in the framework of Markov and semi-Markov Decision Processes. I will describe the kind of structure in a Markov decision process which gives rise to efficient HRL methods. Specifically, we formalize the intuition that HRL can exploit repeating sub-structures. We show that, under reasonable assumptions, such algorithms can achieve statistical efficiency, as established through a finite-time regret bound, as well as near-optimal and computationally efficient planning, using hierarchical models.


Doina Precup splits her time between McGill University, where she co-directs the Reasoning and Learning Lab in the School of Computer Science, and DeepMind Montreal, where she has led the research team since its formation in October 2017. Her research interests are in the areas of reinforcement learning, deep learning, time series analysis, and diverse applications of machine learning in health care, automated control, and other fields. Her origial paper on temporal anstraction in reinforcement learning received the Classic Paper award from the Artificial Intelligence journal in 2019. Dr. Precup became a senior member of the Association for the Advancement of Artificial Intelligence in 2015, Canada Research Chair in Machine Learning in 2016, Senior Fellow of the Canadian Institute for Advanced Research in 2017, and received a Canada CIFAR AI (CCAI) Chair in 2018. Dr. Precup is also involved in activities supporting the organization of Mila (the Quebec AI Institute) and the wider Montreal and Quebec AI ecosystem, and she has co-founded the AI4Good summer lab, whose goal is to improve gender diversity in machine learning.