Monday, February 5, 2024 - 4:00pm
Event Calendar Category
LIDS Seminar Series
Speaker Name
Urbashi Mitra
Affiliation
USC
Building and Room Number
32-155
Abstract
Many agent-based systems interacting with their environments are well-modeled by Markov Decision Processes (MDPs). MDPs also form the ``model ‘’ framework for many learning strategies such as reinforcement learning. We focus on policy design to optimize some arbitrary objective function. We develop a novel on-line/off-line Q-learning methodology which is based on the creation multiple, distinct, synthetically created and structurally related Markovian environments in parallel. We dub these synthetic systems, ``digital cousins’’ to distinguish them from the concept of a digital twin which is supposed to be identical to the system of interest. Each cousin has its own learning strategy to optimize a policy; the policies of the cousins and the original system are fused to create the final policy. We use policy optimization for wireless networks as a motivating example for the methodology. The approach is theoretically analyzed, including showing the convergence of key statistics and Q-functions. The improvement in learning rate is roughly proportional to the number of digital cousins, due in part to the fact that the cousins run at different time scales. Partial ordering results on the relative informativeness of different cousins are also provided. Numerical results across several network models show that the proposed algorithm can achieve up to 55% less average policy error with up to 50% less runtime complexity than the state-of-the-art A-learning algorithms. Numerical results validate assumptions made in the theoretical analysis.
Biography
Urbashi Mitra received the B.S. and the M.S. degrees from the University of California at Berkeley and her Ph.D. from Princeton University. Dr. Mitra is currently the Gordon S. Marshall Professor in Engineering at the University of Southern California with appointments in Electrical & Computer Engineering and Computer Science. She was the inaugural Editor-in-Chief for the IEEE Transactions on Molecular, Biological and Multi-scale Communications. She has been a member of the IEEE Information Theory Society's Board of Governors (2002-2007, 2012-2017), the IEEE Signal Processing Society’s Technical Committee on Signal Processing for Communications and Networks (2012-2016), the IEEE Signal Processing Society’s Awards Board (2017-2018), the Chair/Vice-Chair of the IEEE Communication Theory Technical Committee (2017-2020), and the Vice-Chair of the IEEE Signal Processing for Communications and Networking Technical Committee (2024-). Dr. Mitra is a Fellow of the IEEE. She is the recipient of: the 2021 USC Viterbi School of Engineering Senior Research Award, the 2017 IEEE Women in Communications Engineering Technical Achievement Award, a 2015 UK Royal Academy of Engineering Distinguished Visiting Professorship, a 2015 US Fulbright Scholar Award, a 2015-2016 UK Leverhulme Trust Visiting Professorship, IEEE Communications and Signal Processing Societies Distinguished Lecturer, 2012 Globecom Signal Processing for Communications Symposium Best Paper Award, 2012 US National Academy of Engineering Lillian Gilbreth Lectureship, the 2009 DCOSS Applications & Systems Best Paper Award, 2001 Okawa Foundation Award, 2000 Ohio State University’s College of Engineering Lumley Award for Research, and a 1996 National Science Foundation CAREER Award. Her research interests are in wireless communications, structured statistical methods, communication and sensor networks, biological communication systems, detection and estimation and the interface of communication, sensing and control.