Rate Adaptation with Q-Learning in CSMA/CA Wireless Networks
2
조회
0
다운로드
5
인용
0
참고문헌
키워드
초록
In this study, we propose a reinforcement learning agent to control the data transmission rates of nodes in carriersensing multiple access with collision avoidance (CSMA/CA)-based wireless networks. We design areinforcement learning (RL) agent, based on Q-learning. The agent learns the environment using the timeoutevents of packets, which are locally available in data sending nodes. The agent selects actions to control thedata transmission rates of nodes that adjust the modulation and coding scheme (MCS) levels of the data packetsto utilize the available bandwidth in dynamically changing channel conditions effectively. We use the ns3-gymframework to simulate RL and investigate the effects of the parameters of Q-learning on the performance ofthe RL agent. The simulation results indicate that the proposed RL agent adequately adjusts the MCS levelsaccording to the changes in the network, and achieves a high throughput comparable to those of the existingdata transmission rate adaptation schemes such as Minstrel.
Abstract
In this study, we propose a reinforcement learning agent to control the data transmission rates of nodes in carriersensing multiple access with collision avoidance (CSMA/CA)-based wireless networks. We design areinforcement learning (RL) agent, based on Q-learning. The agent learns the environment using the timeoutevents of packets, which are locally available in data sending nodes. The agent selects actions to control thedata transmission rates of nodes that adjust the modulation and coding scheme (MCS) levels of the data packetsto utilize the available bandwidth in dynamically changing channel conditions effectively. We use the ns3-gymframework to simulate RL and investigate the effects of the parameters of Q-learning on the performance ofthe RL agent. The simulation results indicate that the proposed RL agent adequately adjusts the MCS levelsaccording to the changes in the network, and achieves a high throughput comparable to those of the existingdata transmission rate adaptation schemes such as Minstrel.
참고문헌 (21)
- 김희찬, "Document Summarization Model Based on General Context in RNN", JIPS(Journal of Information Processing Systems), 15(6), pp.1378~1391 (2019), 한국정보처리학회[DOI]
- M. J. J. Ghrabat, "An effective image retrieval based on optimized genetic algorithm utilized a novel SVM-based convolutional neural network classifier", Human-centric Computing and Information Sciences, 9, pp.31~ (2019)
- R. S. Sutton, "Reinforcement Learning: An Introduction" (2018), MIT Press
- M. A. Rahman, "An efficient transmission mode selection based on reinforcement learning for cooperative cognitive radio networks", Human-centric Computing and Information Sciences, 6(2) (2016)
- Y. Bengio, "Learning Deep Architectures for AI" (2009), Now Publishers Inc
- C. Zhang, "Deep learning in mobile and wireless networking : a survey", IEEE Communications Surveys & Tutorials, 21(3), pp.2224~2287 (2019)
- V. Mnih, "Human-level control through deep reinforcement learning", Nature, 518, pp.529~533 (2015)
- M. G. Bellemare, "The arcade learning environment : an evaluation platform for general agents", Journal of Artificial Intelligence Research, 47, pp.253~279 (2013)
- C. J. Watkins, "Q-learning", Machine Learning, 8, pp.279~292 (1992)
- Y. Sun, "A trust-aware task allocation method using deep q-learning for uncertain mobile crowdsourcing", Human-centric Computing and Information Sciences, 9(25) (2019)
- G. A. Preethi, "Seamless Mobility of Heterogeneous Networks Based on Markov Decision Process", JIPS(Journal of Information Processing Systems), 11(4), pp.616~629 (2015), 한국정보처리학회[DOI]
- "OpenAI Gym"[URL]
- P. Gawlowicz, "Ns-3 meets OpenAI Gym: the playground for machine learning in networking research", Proceedings of the 22nd International ACM Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems, pp.113~120 (2019)
- "ns-3 is a discrete-event network simulator"[URL]
- "IEEE Standard for Telecommunications and Information Exchange Between Systems – LAN", MAN Specific Requirements - Part 11: Wireless Medium Access Control (MAC) and physical layer (PHY) specifications:High Speed Physical Layer in the 5 GHz band, 2003
- "Rate Adaptation for 802.11 Wireless Networks: Minstrel" (2010)[URL]
- J. Kim, "CARA : collision-aware rate adaptation for IEEE 802. 11 WLANs", Proceedings of the 25th IEEE International Conference on Computer Communications (INFOCOM) (2006)
- "IEEE Standard for Information technology – Telecommunications and information exchange between systems Local and metropolitan area networks – Specific requirements - Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications", IEEE Std 802.11-2016, ~ (2016)
- S. Cho, "SINR-based MCS level adaptation in CSMA", CA wireless networks to embrace IoT devices, 2017, pp.9, 236~
- D. Xia, "Evaluation of the Minstrel rate adaptation algorithm in IEEE 802. 11g WLANs", Proceedings of 2013 IEEE International Conference on Communications (ICC), pp.2223~2228 (2013)
- A. Geron, "Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems" (2017), O'Reilly Media