Scaling Value-Aligned Learning to Robotics

Before learning robots can be deployed in the real world, it is critical to be able to assure that their goals and behaviors will be aligned with the values of human users. While great progress has been made in the use of imitation learning to teach increasingly complex behaviors to robots, the value alignment problem has largely been ignored. The practical usefulness of such algorithms will remain limited without methods that can provide strong, finite-sample guarantees that scale to realistic robotics problems.

Toward this goal, I will first introduce a series of algorithms that challenge common, but poor, statistical assumptions commonly made in imitation learning. This work culminates in an approach that enables value-aligned imitation learning to scale to high-dimensional control problems for the first time. Second, I will discuss a body of work that leverages historically under-utilized data modalities -- ranging from human gaze and facial expressions, to contact and force sensing -- to further improve the characterization of human values and environmental uncertainty, thereby minimizing risk. Taken together, these algorithms represent a significant step toward enabling value-aligned robot learning in the real world, using only modest amounts of data.