Experiment-of-experiments Designs for Causal Inference with Network Interference

Wednesday, February 14, 2018 - 4:30pm to 5:00pm

Event Calendar Category

LIDS & Stats Tea

Speaker Name

Jean Pouget Abadie

Affiliation

Harvard

Building and Room Number

LIDS Lounge

Abstract

Tech companies rely on online experiments to understand the impact of changes to their ecosystem or product. The classic assumption, on which these experiments depend, of no interference amongst users—that is, the fact that the outcome of a user does not depend on the treatment assigned to other users—is rarely tenable. Experiment-of-experiments (EoE) designs are powerful tools to quantify and mitigate interference on these experimentation platforms. We suggest an EoE design to test non-parametrically whether interference is present in the ecosystem. The proposed EoE design is illustrated on a live experiment on 5% of the LinkedIn graph. Furthermore, if interference is detected above a certain threshold, cluster-based randomized designs are common mitigation tools, leaving open the question of which clustering is most appropriate. We show that, when it is reasonable to make a monotonicity assumption, the suggested EoE design can be modified to determine which of two clusterings is optimal. This assumption is shown to hold in reserve price experiments in positional ad auctions. We validate this approach on a Yahoo! Search auction dataset.​​​

Biography

Jean is a fourth-year Ph.D. candidate in Computer Science at Harvard University, advised by Edoardo Airoldi and Salil Vadhan, and a 2017-2018 Siebel scholar. Before Harvard, he was an undergraduate at Ecole Polytechnique, Paris. His recent research interests focus on causal inference and experimental design, particularly when network interference is present. He most recently completed an internship with Vahab Mirrokni with the Algorithms research group at Google NYC.