효율적인 클라이언트 관리를 통한 연합학습 성능 향상
Performance Enhancement of Federated Learning through Effective Client Management
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
연합학습의 non-IID 데이터셋 환경에서는 학습 딜레이 및 클라이언트 간의 데이터셋 불균형으로 인한 학습 성능 저하 등의 문제점이 있다. 이러한 성능 저하를 방지하기 위해, 적절한 클라이언트를 선택하는 것은 학습에서 중요한 부분 중 하나이다. 이 논문에서는 연합학습 모델에 가상큐를 기반으로 한 UCB 알고리즘을 적용하여 클라이언트 선택을 진행하는 방식을 사용하고 있다. 또한 기존의 UCB 알고리즘 선택 방식과 다르게학습 진행 상황에 따라 클라이언트 선택을 조절하는 알고리즘을 제안한다. 먼저 학습 진행 상황을 클라이언트 선택 과정에 반영하여 모델의 학습시간을 줄일 수 있었다. 추가적으로 모델의 성능 향상을 위해 UCB의 클라이언트 탐색을 동적으로 바꾸는 방식을 제안한다. 학습 중반 이후의학습이 안정화되면, 클라이언트의 탐색을 조절하여 모델의 성능 개선을 이루었다. 논문의 실험 결과는 해당 알고리즘을 활용하였을 때 학습시간단축, 모델의 정확도 향상 등의 이점이 있음을 보여준다.
Abstract
In a federated learning with non-IID datasets, training delays and poor performance due to unequalness in datasets among clientsare prevalent. To prevent such performance degradation, selecting appropriate clients is a crucial part of training. This paper appliesa UCB algorithm based on a virtual queue for client selection in federated learning models. Unlike the traditional UCB algorithm, theproposed client selection method reflects the training progress of federated learning. It can reduce the proposed model’s training time. Additionally, to further improve the performance of the model, the exploration of clients through UCB is dynamically adjusted. Oncethe training is stabilized in the mid-phase, the client exploration is involved to extract additional performance gain. The experimentalresults demonstrate the benefit of this algorithm such as a reduced training time and an improved model accuracy
참고문헌 (19)
- K. Wei, "Federated Learning with Differential Privacy : Algorithms and Performance Analysis", IEEE Transactions on Information Forensics and Security, 15(6), pp.3454~3469 (2020)
- F. Sattler, "Robust and Communication-Efficient Federated Learning From Non-i. i. d. Data", IEEE Transactions on Neural Networks and Learning Systems, 31(9), pp.3400~3413 (2019)
- X. Shen, "Verifiable Privacy-Preserving Federated Learning Under Multiple Encrypted Keys", IEEE Internet of Things Journal, 11(3), pp.3430~2445 (2024)
- H. B. McMahan, "Communication-Efficient Learning of Deep Networks from Decentralized Data", Proceedings of the 20th International Conference on Artificial Intelligence and Statistics(AISTATS), pp.1223~1234 (2016)
- H. Wang, "Optimizing Federated Learning on Non-IID Data with Reinforcement Learning", Proceedings of IEEE INFOCOM 2020-IEEE Conference on Computer Communications (2020)
- Y. Zhao, "Federated Learning with Non-IID Data", Proceedings of the NeurIPS Workshop on Federated Learning (2018)
- W. Xia, "Multi-Armed Bandit-Based Client Scheduling for Federated Learning", IEEE Transactions on Wireless Communications, 19(11), pp.7108~7123 (2020)
- T. T. Vu, "Straggler Effect Mitigation for Federated Learning in Cell-Free Massive MIMO", Proceedings of ICC 2021-IEEE International Conference on Communications (2021)
- Y. Chen, "Asynchronous Online Federated Learning for Edge Devices with Non-IID Data", Proceedings of IEEE International Conference on Big Data(Big Data) (2020)
- N. Yoshida, "MAB-Based Client Selection for Federated Learning with Uncertain Resources in Mobile Networks", Proceedings of IEEE Globecom Workshops(GC Wkshps) (2020)
- H. Zhu, "Online Client Selection for Asynchronous Federated Learning with Fairness Consideration", IEEE Transactions on Wireless Communications, 22(4), pp.1234~1245 (2023)
- H. Zhu, "Client Selection with Staleness Compensation in Asynchronous Federated Learning", IEEE Transactions on Vehicular Technology (2023)
- J. Tong, "From Learning to Analytics : Improving Model Efficacy with Goal-Directed Client Selection", IEEE Transactions on Mobile Computing (2024)
- C. Shi, "Reward Teaching for Federated Multiarmed Bandits", IEEE Transactions on Signal Processing (2023)
- Z. Zhou, "Towards Efficient and Stable K-Asynchronous Federated Learning with Unbounded Stale Gradients on Non-IID Data", IEEE Transactions on Parallel and Distributed Systems, 33(12), pp.3291~3305 (2022)
- C. Wang, "A Buffered Semi-Asynchronous Mechanism with MAB for Efficient Federated Learning", Proceedings of the International Conference on High Performance Big Data and Intelligent Systems(HDIS) (2022)
- A. Fallah, "Perso nalized Federated Learning with Theoretical Guarantees : A Model-Agnostic Meta-Learning Approach", Proceedings of the 34th Conference on Neural Information Processing Systems(NeurIPS) (2020)
- G. Co hen, "EMNIST : Extending MNIST to Handwritten Letters", Proceedings of the 2017 International Joint Conference on Neural Networks(IJCNN) (2017)