로컬 모델 검증을 통한 가중치 부여 연합학습 프레임워크 설계
Design of weighted federated learning framework based on local model validation
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
본 논문에서는 학습에 참여하는 각 디바이스의 모델들로부터 성능검증에 따라 가중치를 두어글로벌 모델을 업데이트하는 VW-FedAVG(Validation based Weighted FedAVG)를 두 가지 방식으로제안 한다. 첫 번째 방식은 서버 검증(Server side Validation) 구조로 글로벌 모델을 업데이트 하기전에 각 로컬 클라이언트 모델을 하나의 전체 검증 데이터셋을 통해 검증하도록 설계 했다. 두번째는 클라이언트 검증(Client side Validation) 구조로 검증 데이터셋을 각 클라이언트에 고르게분배하여 검증을 한 후 글로벌 모델을 업데이트 하는 방식으로 설계 했다. 전체 실험에 적용한데이터셋은 MNIST, CIFAR-10으로 이미지 분류에 대해 IID, Non-IID 분포에서 기존 연구 대비 더높은 정확도를 얻을 수 있었다.
Abstract
In this paper, we proposed VW-FedAVG(Validation based Weighted FedAVG) which updates the global model by weighting according to performance verification from the models of each device participating in the training. The first method is designed to validate each local client model through validation dataset before updating the global model with a server side validation structure. The second is a client-side validation structure, which is designed in such a way that the validation data set is evenly distributed to each client and the global model is after validation. MNIST, CIFAR-10 is used, and the IID, Non-IID distribution for image classification obtained higher accuracy than previous studies.
참고문헌 (14)
- Ding, Chuntao, "Resource-aware feature extraction in mobile edge computing", IEEE Transactions on Mobile Computing, 21(1), pp.321~331 (2020)[DOI]
- Ren, H, "Querying in internet of things with privacy preserving: Challenges, solutions and opportunities", IEEE Network, 32(6), pp.144~151 (2018)[DOI]
- Garcia, David, "Analyzing gender inequality through large-scale Facebook advertising data", Proceedings of the National Academy of Sciences, 115(27), pp.6958~6963 (2018)[DOI]
- McMahan, Brendan, "Federated learning:Collaborative machine learning without centralized training data", Google Research Blog, ~ (2017)
- H. Brendan McMahan, "Communication-efficient learning of deep networks from decentralized data", ~ (2016)
- 홍만수, "연합학습 모델에서 이상 클라이언트 제거", 한국지능시스템학회 논문지, 30(6), pp.488~493 (2020), 한국지능시스템학회[DOI]
- Yang, Q, "Federated machine learning: Concept and applications", ACM Transactions on Intelligent Systems and Technology, 10(2), pp.1~19 (2019)[DOI]
- Karimireddy SP, "Scaffold: Stochastic controlled averaging for federated learning", InInternational Conference on Machine Learning, pp.5132~5143 (2020)
- Krizhevsky, Alex, "Learning multiple layers of features from tiny images", 7~ (2009)
- "IEEE signal processing magazine", 2012, 141~142, 10.1109[DOI]
- Wang H, "Federated learning with matched averaging", ~ (2020)
- SSahu, Anit Kumar, "On the convergence of federated optimization in heterogeneous networks", ~ (2018)
- Simonyan, Karen, "Very deep convolutional networks for large-scale image recognition", ~ (2014)
- He, Kaiming, "Deep residual learning for image recognition", Proceedings of the IEEE conference on computer vision and pattern recognition, pp.770~778 (2016)