LIDS Tea Talk: Chenyu Zhang

“Personalized Collaborative Learning with Affinity-Based Variance Reduction”

Multi-agent learning faces a fundamental tension: leveraging distributed collaboration without sacrificing personalization for diverse agents. We introduce personalized collaborative learning (PCL), a learning setup that embraces this tension, aiming for full personalization while adapting to varying heterogeneity—gaining collaborative speedup when agents are similar, without performance degradation when they are different.

Our method robustly handles both environment and objective heterogeneity, attaining sample complexity (O(t^{-1}\max{n^{-1},\delta})) for strongly monotonic problems, where (t) is the number of samples per agent, (n) the number of agents, and (\delta\in[0,1]) reflects their affinity. This rate automatically interpolates between the linear speedup of federated learning in homogeneous settings and the baseline of independent learning in heterogeneous ones, without requiring prior system knowledge. Our analysis further shows that even among maximally heterogeneous agents, some can still achieve significant speedup, unveiling new insights into personalization and collaboration in the high heterogeneity regime.