NSFC General Program (PI: Dr. Bo Han, Department of Computer Science, Hong Kong Baptist University)
Project Award Information
Award Number: NSFC GP 62376235 Title: The Research on Trustworthy Federated Learning in Imperfect Environments Principal Investigator (PI): Dr. Bo Han, Department of Computer Science, Hong Kong Baptist University
Project Summary
Federated Learning (FL) is a new machine learning paradigm, which aims to keep client data localized, but boosts the performance of local models by synergizing other models via the central server. Due to the protection of data privacy, FL has been widely applied to many daily scenarios, such as the cross-device setting (e.g., mobile devices) and the cross-silo setting (e.g., banks or clinics). The design of classical FL algorithms is based on the implicit assumption of perfect environments, such as fully supervised data, cooperative clients, and homogeneous models. However, such assumption hinders the deployment and application of FL algorithms in real-world imperfect environments, such as weakly supervised data, competitive clients, and heterogeneous models. Thus, this project proposes “trustworthy federated learning (TFL)” paradigm, which aims to reliably enable FL algorithms in imperfect environments. The key philosophy of TFL paradigm focuses on two folds. First, local clients should be autonomous rather than dependent. Second, the role of central servers should shift from pure computing to monitoring plus computing. Specifically, from data perspective, we propose to design the excerpt-auxiliary protocol to handle federated learning with weakly
supervised data. From client perspective, we propose to design the composite mean embedding discrepancy test for tackling federated learning with competitive clients. From server perspective, we propose to develop the adaptive group model aggregating framework to enable federated learning with heterogeneous models. Meanwhile, this project preliminarily discusses how to provide the theoretical guarantees for TFL in imperfect environments, and how to integrate the proposed learning algorithms into the TFL prototype system. This project also aims to apply the proposed TFL algorithms and system to more real-world problems.
Research Publications
The following papers focus on trustworthy federated learning from data perspective:
virtual homogeneity learning against data heterogeneity in federated learning (ICML'22) feature distillation against data heterogeneity in federated learning (NeurIPS'23) federated learning with bilateral curation for
partially class-disjoint data (NeurIPS'23) robust training of federated models with extremely label deficiency (ICLR'24) enhancing one-shot federated learning through data and ensemble co-boosting (ICLR'24)
The following papers focus on trustworthy federated learning from client perspective:
combating exacerbated heterogeneity for robust models in federated learning (ICLR'23) federated noisy client learning (TNNLS'23) measuring and improving client update in federated learning (ICLR'24) accurate forgetting for heterogeneous federated continual learning (ICLR'24) federated learning with extremely noisy-labeled clients via negative distillation (AAAI'24)
The following papers focus on trustworthy federated learning from server perspective:
Software
virtual homogeneity learning against data heterogeneity in federated learning, [code] feature distillation against data heterogeneity in federated learning, [code] federated learning with bilateral curation for partially class-disjoint data, [code] robust training of federated models with extremely label deficiency, [code] enhancing one-shot federated learning through data and ensemble co-boosting, [code] combating exacerbated heterogeneity for robust models in federated learning, [code] federated noisy client learning, [code] measuring and improving client update in federated learning, [code] accurate forgetting for heterogeneous federated continual learning, [code] federated learning with extremely noisy-labeled clients via negative distillation, [code] server-client collaborative distillation for federated reinforcement learning, [code]
Education
UG Course: COMP3057 (2023 Autumn, 2024 Autumn, 2025 Autumn) PG Course: COMP7250 (2023 Spring, 2024 Spring, 2025 Spring), COMP7160 (2023 Autumn, 2024 Autumn, 2025 Autumn) Undergraduate Research Programme (UGRP): Yixiao Zheng, Fengfei Yu, Jiayu Wang, Yanyao Huang, Guanhong Jiang, Abhinav Hang Rai Summer Undergraduate Research Fellowship: Yixiao Zheng, Fengfei Yu
Collaborators
University: University of Maryland College Park, The University of Sydney, Hong Kong University of Science and Technology, The Chinese University of Hong Kong, University of Macau, University of Auckland, Tsinghua University, Shanghai Jiao Tong University, University of Science and Technology of China, Xiamen University Institute: Shanghai AI Laboratory, A*STAR Institute of High Performance Computing Industry: Microsoft Research, Baidu Research
Acknowlewdgement
This material is based upon work supported by the National Natural Science Foundation (NSFC) under Grant No. 62376235. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Natural Science Foundation (NSFC).
|