Stable Optimal Control and Semicontractive Dynamic Programming

Tuesday, May 16, 2017 - 4:00pm

Speaker Name

Dimitri Bertsekas

Affiliation

Massachusetts Institute of Technology

Building and Room Number

32-141

Abstract

We consider discrete-time infinite horizon deterministic optimal control problems with nonnegative cost, and a destination that is cost-free and absorbing. The classical linear-quadratic regulator problem is a special case. The analysis aims to connect classical issues of stability and optimization, and is based on the ideas of semicontractive models, where some but not all policies are well-behaved/contractive. Our assumptions are very general, and allow the possibility that the optimal policy may not be stabilizing the system, e.g., may not reach the destination either asymptotically or in a finite number of steps. We introduce a new unifying notion of stability based on perturbation of the cost per stage, and we consider the properties of two distinct cost functions: $J^*$, the overall optimal, and $\hat J$, the optimal over just the stable policies. We show that both $J^*$ and $\hat J$ are solutions of Bellman's equation, and the standard value iteration algorithm may be attracted to either one depending on the initial condition. We also discuss a new perturbation-based policy iteration algorithm to find $\hat J$ and a nearly optimal stable policy, as a substitute for ordinary policy iteration, which may not work in general.

Biography

Dimitri P. Bertsekas undergraduate studies were in engineering at the National Technical University of Athens, Greece. He obtained his MS in electrical engineering at the George Washington University, Wash. DC in 1969, and his Ph.D. in system science in 1971 at the Massachusetts Institute of Technology.

Dr. Bertsekas has held faculty positions with the Engineering-Economic Systems Dept., Stanford University (1971-1974) and the Electrical Engineering Dept. of the University of Illinois, Urbana (1974-1979). Since 1979 he has been teaching at the Electrical Engineering and Computer Science Department of the Massachusetts Institute of Technology (M.I.T.), where he is currently McAfee Professor of Engineering. He has held editorial positions in several journals. His research at M.I.T. spans several fields, including optimization, control, large-scale computation, and data communication networks, and is closely tied to his teaching and book authoring activities. He has written numerous research papers, and sixteen books and research monographs, several of which are used as textbooks in MIT classes.

Professor Bertsekas was awarded the INFORMS 1997 Prize for Research Excellence in the Interface Between Operations Research and Computer Science for his book "Neuro-Dynamic Programming" (co-authored with John Tsitsiklis), the 2000 Greek National Award for Operations Research, the 2001 ACC John R. Ragazzini Education Award, the 2009 INFORMS Expository Writing Award, the 2014 ACC Richard E. Bellman Control Heritage Award for "contributions to the foundations of deterministic and stochastic optimization-based methods in systems and control," the 2014 Khachiyan Prize for Life-Time Accomplishments in Optimization, and the SIAM/MOS 2015 George B. Dantzig Prize. In 2001, he was elected to the United States National Academy of Engineering for "pioneering contributions to fundamental research, practice and education of optimization/control theory, and especially its application to data communication networks."

Dr. Bertsekas' recent books are "Introduction to Probability: 2nd Edition" (2008), "Dynamic Programming and Optimal Control," Vol. I, (2017), and Vol. II: (2012), "Abstract Dynamic Programming" (2013), "Convex Optimization Theory" (2009), and "Convex Optimization Algorithms" (2015), all published by Athena Scientific.

Reception information

Reception to follow at 5 PM.