Seminar: Problem-Parameter-Free Federated Learning (Accepted as an Oral Presentation at ICLR 2025, with an acceptance rate of 1.82%)

https://openreview.net/forum?id=ZuazHmXTns

Abstract: 

Federated learning (FL) has garnered significant attention from academia and industry in recent years due to its advantages in data privacy, scalability, and communication efficiency. However, current FL algorithms face a critical limitation: their performance heavily depends on meticulously tuned hyperparameters, particularly the learning rate or stepsize. This manual tuning process is challenging in federated settings due to data heterogeneity and limited accessibility of local datasets. Consequently, the reliance on problem-specific parameters hinders the widespread adoption of FL and may compromise its performance in dynamic or diverse environments. To address this issue, we introduce PAdaMFed, a novel algorithm for nonconvex FL that carefully combines adaptive stepsize and momentum techniques. PAdaMFed offers two key advantages. First, it operates autonomously without requiring problem-specific parameters. Second, it effectively handles data heterogeneity and partial participation without assuming known bounds on heterogeneity.

Biography:
Xuanyu Cao received the B.S. degree in electrical engineering from Shanghai Jiao Tong University in 2013, and the M.S. and Ph.D. degrees in electrical engineering from the University of Maryland, College Park, in 2016 and 2017, respectively. From 2017 to 2021, he was successively a Postdoctoral Research Associate with the Department of Electrical Engineering at Princeton University and the Coordinated Science Lab at the University of Illinois, Urbana-Champaign. He was an Assistant Professor with the Department of Electronic and Computer Engineering at the Hong Kong University of Science and Technology (HKUST), Clear Water Bay, Hong Kong, from 2021 to 2024. Since 2025, he has been an Assistant Professor with the School of Electrical Engineering and Computer Science at Washington State University. His research interests include distributed and online optimization, communication-efficient distributed learning, federated learning, and network economics. He is an Editor for IEEE Transactions on Wireless Communications and IEEE Transactions on Vehicular Technology, a TPC Member for ACM MobiHoc from 2022 to 2024, and the Lead Guest Editor for the special issue on “Communication-Efficient Distributed Learning over Networks” in IEEE Journal on Selected Areas in Communications.