A3C 기반의 강화학습을 사용한 DASH 시스템
A DASH System Using the A3C-based Deep Reinforcement Learning
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
The simple procedural segment selection algorithm commonly used in Dynamic Adaptive Streaming over HTTP (DASH) reveals severe weakness to provide high-quality streaming services in the integrated mobile networks of various wired and wireless links. A major issue could be how to properly cope with dynamically changing underlying network conditions. The key to meet it should be to make the segment selection algorithm much more adaptive to fluctuation of network traffics. This paper presents a system architecture that replaces the existing procedural segment selection algorithm with a deep reinforcement learning algorithm based on the Asynchronous Advantage Actor-Critic (A3C). The distributed A3C-based deep learning server is designed and implemented to allow multiple clients in different network conditions to stream videos simultaneously, collect learning data quickly, and learn asynchronously, resulting in greatly improved learning speed as the number of video clients increases. The performance analysis shows that the proposed algorithm outperforms both the conventional DASH algorithm and the Deep Q-Network algorithm in terms of the user’s quality of experience and the speed of deep learning.
Abstract
The simple procedural segment selection algorithm commonly used in Dynamic Adaptive Streaming over HTTP (DASH) reveals severe weakness to provide high-quality streaming services in the integrated mobile networks of various wired and wireless links. A major issue could be how to properly cope with dynamically changing underlying network conditions. The key to meet it should be to make the segment selection algorithm much more adaptive to fluctuation of network traffics. This paper presents a system architecture that replaces the existing procedural segment selection algorithm with a deep reinforcement learning algorithm based on the Asynchronous Advantage Actor-Critic (A3C). The distributed A3C-based deep learning server is designed and implemented to allow multiple clients in different network conditions to stream videos simultaneously, collect learning data quickly, and learn asynchronously, resulting in greatly improved learning speed as the number of video clients increases. The performance analysis shows that the proposed algorithm outperforms both the conventional DASH algorithm and the Deep Q-Network algorithm in terms of the user’s quality of experience and the speed of deep learning.
참고문헌 (10)
- J. Kua, "A Survey of Rate Adaptation Techniques for Dynamic Adaptive Streaming Over HTTP", IEEE Communications Surveys and Tutorials, 19(3), pp.1842~1866 (2017)
- K. Miller, "Adaptation Algorithm for Adaptive Streaming over HTTP", 2012 IEEE 19th International Packet Video Workshop, pp.173~178 (2012)
- M. Seufert, "A Survey on Quality of Experience of HTTP Adaptive Streaming", IEEE Communications Surveys &Tutorials, 17(1), pp.469~492 (2015)
- 김이슬, "QoE 향상을 위한 Deep Q-Network 기반의 지능형 비디오 스트리밍 메커니즘", 멀티미디어학회논문지, 21(2), pp.188~198 (2018), 한국멀티미디어학회[DOI]
- 김이슬, "DQN 기반 비디오 스트리밍 서비스에서 세그먼트크기가 품질 선택에 미치는 영향", 멀티미디어학회논문지, 21(10), pp.1182~1194 (2018), 한국멀티미디어학회[DOI]
- V. Mnih, "Asynchronous Methods for Deep Reinforcement Learning", Proceedings of The 33rdInternational Conference on Machine Learning, 48, pp.1928~1937 (2016)
- T. Haarnoja, "Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor", Proceedings of the 35th International Conference on Machine Learning, 80, pp.1861~1870 (2018)
- P. Juluri, "QoE Management in DASH Systems Using the Segment Aware Rate Adaptation Algorithm", Proceeding of IEEE, 2016, 129~136
- V. Mnih, "Playing Atari with Deep Reinforcement Learning", NIPS Deep Learning Workshop 2013 (2013)
- T. R. Henderson, "Network Simulations with the ns-3 Simulator", Proceeding of Association for Comput ing Machinery Conference on Special Interest Group on Data Communication, pp.527~ (2008)