LIDS Research Highlights

Reinforcement Learning with Multi-Fidelity Simulators

Simulators play a key role as testbeds for robotics control algorithms, but deciding when to use them versus collecting real-world data is often treated as an art. We have developed a framework for efficient reinforcement learning (RL) in a scenario where multiple simulators or a target task, each with varying levels of fidelity, are available. Our framework is designed to limit the number of samples used in each successively higher-fidelity/cost simulator by allowing a learning agent to choose to run trajectories at the lowest level simulator that will still provide it with information.

Online Prediction and Learning for Data with Changepoints using Gaussian Processes

Gaussian Processes (GPs) have become an increasingly popular framework for nonlinear regression and learning in a variety of online applications such as control, reinforcement learning, and machine learning. Most GP algorithms assume a static world, and will perform poorly if the generating process switches at discrete instants in time (changepoints). However, many applications of interest, such as controlling an aircraft or predicting the stock market, may include such changepoints.

Focused Active Inference

Probabilistic graphical models compactly represent the structure of stochastic processes, and exploiting the sparsity of that structure often leads to computationally efficient inference algorithms, which process realized observations to update a belief about latent states that are not observable.

Camera Control for Learning Nonlinear Target Dynamics via Bayesian Non-Parametric Dirichlet-Process Gaussian-Process (DPGP) Models

Mobile sensors are often deployed to cooperatively perform surveillance of an environment and track objects of interest: for example, intrusion detection throughout a secure facility, and anomaly detection in manufacturing plants. In many such cases it is infeasible to provide accurate sensor coverage of the entire environment, and thus an environment model and associated controller that accounts for sensor field-of-view (FoV) geometry is necessary to determine sensor configurations that minimize uncertainty.

Approximate Decentralized Bayesian Inference

Recent trends in the growth of datasets, and the methods by which they are collected, have led to increasing interest in the parallelization of machine learning algorithms. Parallelization results in reductions in both the memory usage and computation time of learning, and allows data to be collected and processed by a network of learning agents rather than by a single central agent.

Distributed Asynchronous Methods for Minimizing Sums of Non-Separable Functions

Recently, there has been a flurry of research activity for study of non-separable network optimization prob- lems. The problem formulation is very general and captures many important problems, such as parameter estimation in sensor networks and pattern recognition in machine learning settings. Due to the versatility of the problem formulation, fast distributed algorithms for this problem can provide drastic improvement in the system wide performance for many applications. The main challenge is to develop a distributed algorithm, which is necessary in many large scale applications.