산업용 사물 인터넷을 위한 프라이버시 보존 연합학습 기반 심층 강화학습 모델
Federated Deep Reinforcement Learning Based on Privacy Preserving for Industrial Internet of Things
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
최근 사물 인터넷을 활용한 산업 현장에서 수집되는 빅데이터를 활용해 복잡한 문제들을 해결하기 위하여 심층 강화학습 기술을 적용한 다양한 연구들이 이루어지고 있다. 심층 강화학습은 강화 학습의 시행 착오 알고리즘과 보상의 누적값을 이용해 자체 데이터를 생성하여 학습하고 신경망 구조와 파라미터 결정을 빠르게 탐색한다. 그러나 종래 방법은 학습 데이터의 크기가 커질수록 메모리 사용량과 탐색 시간이 기하급수적으로 높아지며 정확도가 떨어진다. 본 연구에서는 메타 학습을 적용한 연합학습 기반의 심층 강화학습 모델을 활용하여 55.9%만큼 보안성을 개선함으로써 프라이버시 침해 문제를 해결하고, 종래 최적화 기반 메타 학습 모델 대비 5.5% 향상된 97.8%의 분류 정확도를 달성하면서 평균 28.9%의 지연시간을 단축하였다.
Abstract
Recently, various studies using deep reinforcement learning (deep RL) technology have been conducted to solve complex problems using big data collected at industrial internet of things. Deep RL uses reinforcement learning's trial-and-error algorithms and cumulative compensation functions to generate and learn its own data and quickly explore neural network structures and parameter decisions. However, studies so far have shown that the larger the size of the learning data is, the higher are the memory usage and search time, and the lower is the accuracy. In this study, model-agnostic learning for efficient federated deep RL was utilized to solve privacy invasion by increasing robustness as 55.9% and achieve 97.8% accuracy, an improvement of 5.5% compared with the comparative optimization-based meta learning models, and to reduce the delay time by 28.9% on average.
참고문헌 (29)
- Sun-jin Lee, "Machine learning-based jamming attack classification and effective defense technique", Computers & Security(ELSEVIER) (2023)
- Badnava, Babak, "A new potential-based reward shaping for reinforcement learning agent", 2023 IEEE 13th Annual Computing and Communication Workshop and Conference(CCWC) (2023), IEEE
- Yang, Yulong, "Reinforcement Learning with Reward Shaping and Hybrid Exploration in Sparse Reward Scenes", 2023 IEEE 6th International Conference on Industrial Cyber-Physical Systems(ICPS) (2023), IEEE
- Lu, Qin, "Gaussian process temporal-difference learning with scalability and worst-case performance guarantees", ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP) (2021), IEEE
- Y. Hu, "Reward Shaping Based Federated Reinforcement Learning", IEEE Access, 9, pp.67259~67267 (2021)
- Z. A. El Houda, "Cost-efficient Federated Reinforcement Learning-Based Network Routing for Wireless Networks", 2022 IEEE Future Networks World Forum(FNWF), pp.243~248 (2022)
- T. Gafni, "Restless Multi-Armed Bandits under Exogenous Global Markov Process", ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP), pp.5218~5222 (2022)
- Abbas, Momin, "Sharp-MAML : Sharpness-Aware Model-Agnostic Meta Learning", International Conference on Machine Learning (2022)
- Leibo, Joel Z., "Scalable evaluation of multi-agent reinforcement learning with melting pot", International conference on machine learning (2021), PMLR
- Finn, Chelsea, "Model-agnostic meta-learning for fast adaptation of deep networks", International conference on machine learning (2017), PMLR
- Yan Duan, "Fast Reinforcement Learning via Slow Reinforcement Learning" (2016)
- Nichol, Alex, "On first-order meta-learning algorithms" (2018)
- T. Yang, "Dynamic Spectrum Sharing Based on Federated Learning and Multi-Agent Actor-Critic Reinforcement Learning", 2023 International Wireless Communications and Mobile Computing(IWCMC), pp.947~952 (2023)
- W. Shin, "FedVar : Federated Learning Algorithm with Weight Variation in Clients", 2022 37th International Technical Conference on Circuits, 2022, 1~4
- J. K. Catapang, "Optimizing Speed and Accuracy Trade-off in Machine Learning Models via Stochastic Gradient Descent Approximation", 2022 9th International Conference on Soft Computing & Machine Intelligence(ISCMI), pp.124~128 (2022)
- McMahan, B., "Communication-efficient learning of deep networks from decentralized data", Artificial intelligence and statistics (2017), PMLR
- W. Yang, "A Practical Cross-Device Federated Learning Framework over 5G Networks", IEEE Wireless Communications, 29(6), pp.128~134 (2022)
- Richard S. Sutton, "Reinforcement Learning, Second Edition : An Introduction" (2018), MIT Press
- T. Deng, "Extrapolation Method for Solving Fuzzy Volterra Integral Equations in Two Dimensions", 2022 7th International Conference on Computational Intelligence and Applications(ICCIA), pp.22~26 (2022)
- G. Putro Dirgantoro, "Smoothing Weight Distance to Solve Euclidean Distance Measurement Problems in K-Nearest Neighbor Algorithm", 2021 IEEE 5th International Conference on Information Technology, Information Systems and Electrical Engineering(ICITISEE), pp.294~298 (2021)
- K. Beshara-Flynn, "Effects of Signal and Array Parameters on MSE and CRB in DOA Estimation", 2022 IEEE 13th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON), pp.0373~0379 (2022)
- Krizhevsky, Alex, "Convolutional deep belief networks on cifar-10", 40(7), pp.1~9 (2010)
- R. Ning, "Cross-Subject EEG Emotion Recognition Using Domain Adaptive Few-Shot Learning Networks", 2021 IEEE International Conference on Bioinformatics and Biomedicine(BIBM), pp.1468~1472 (2021)
- "Statistics How To" (2023)
- "Github" (2023)[URL]
- L. Qin, "Topology Identification Method of Low-voltage Distribution Network Based on Improved Pearson Correlation Coefficient Method", 2021 IEEE 2nd China International Youth Conference on Electrical Engineering(CIYCEE), pp.1~6 (2021)
- Li, Jiangeng, "Generative Adversarial Imitation Learning from Human Behavior with Reward Shaping", 2022 34th Chinese Control and Decision Conference(CCDC) (2022), IEEE