Fast Learning Guarantees for Weakly Supervised Learning

Wednesday, October 7, 2020 - 4:00pm to 4:30pm

Event Calendar Category

LIDS & Stats Tea

Speaker Name

Joshua Robinson



Zoom meeting id

934 5386 7137

Join Zoom meeting


We study generalization properties of weakly supervised learning. That is, learning where only a few true labels are present for a task of interest but many more “weak” labels are available. In particular, we show that embeddings trained using weak labels only can be fine-tuned on the downstream task of interest at the fast learning rate of O(1/n) where n denotes the number of labeled data points for the downstream task. This acceleration sheds light on the sample efficiency of pre-trained embeddings and can happen even if by itself true labeled data on the task of interest admits only the slower O(1/ \sqrt{n}) rate. The amount of acceleration depends continuously on the number of weak labels available, and on the relation between the two tasks. Our theoretical results are reflected empirically and illustrate how pre-training with weak labels improves sample efficiency.


Josh is a PhD student working with Suvrit Sra and Stefanie Jegelka. His research interests are broadly in the analysis and design of sample efficient learning algorithms. Recent work has focused on learning with little or no supervision.