Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

Cyclic Distillation-guided Channel Decoupling

Cyclic Distillation-guided Channel Decoupling is a federated learning framework designed to personalize neural network models across heterogeneous client devices by partitioning model parameters at the channel level and harmonizing them through cyclical knowledge transfer. Unlike traditional layer-wise personalization strategies, this approach decouples individual feature channels within network layers into globally shared channels that capture general representations and local private channels that retain client-specific knowledge. To facilitate effective collaboration and prevent representation drift between the shared and private components during training rounds, a cyclic distillation mechanism consistently regularizes feature representations between the local personalized models and the global model. This integration enables distributed systems to effectively handle diverse forms of client data heterogeneity, such as feature distribution skew, label imbalance, and concept shift, while maintaining strong local performance and broad generalization ability.

1 item

CD2-pFed: Cyclic Distillation-guided Channel Decoupling for Model Personalization in Federated Learning

CD2-pFed: Cyclic Distillation-guided Channel Decoupling for Model Personalization in Federated Learning

Yiqing Shen, Yuyin Zhou, Lequan Yu

OrganizationsShanghai Jiao Tong UniversityUniversity of California, Santa CruzUniversity of Hong Kong

Why you should read this

Proposes a personalized federated learning framework that decouples model parameters at the channel dimension rather than across layers, pairing this design with cyclic knowledge distillation to address diverse non-IID data distributions across natural and medical image benchmarks.

Federated learning (FL) is a distributed learning paradigm that enables multiple clients to collaboratively learn a shared global model. Despite the recent progress, it remains challenging to deal with heterogeneous data clients, as the discrepant data distributions usually prevent the global model from delivering good generalization ability on each participating client. In this paper, we propose CD²-pFed, a novel Cyclic Distillation-guided Channel Decoupling framework, to personalize the global model in FL, under various settings of data heterogeneity. Different from previous works which establish layer-wise personalization to overcome the non-IID data across different clients, we make the first attempt at channel-wise assignment for model personalization, referred to as channel decoupling. To further facilitate the collaboration between private and shared weights, we propose a novel cyclic distillation scheme to impose a consistent regularization between the local and global model representations during the federation. Guided by the cyclical distillation, our channel decoupling framework can deliver more accurate and generalized results for different kinds of heterogeneity, such as feature skew, label distribution skew, and concept shift. Comprehensive experiments on four benchmarks, including natural image and medical image analysis tasks, demonstrate the consistent effectiveness of our method on both local and external validations.

Added

2026-09-26