Dynamic Action Space Handling Method for Reinforcement Learning Models

Abstract

Recently, extensive studies have been conducted to apply deep learning to reinforcement learning to solve thestate-space problem. If the state-space problem was solved, reinforcement learning would become applicablein various fields. For example, users can utilize dance-tutorial systems to learn how to dance by watching andimitating a virtual instructor. The instructor can perform the optimal dance to the music, to which reinforcementlearning is applied. In this study, we propose a method of reinforcement learning in which the action space isdynamically adjusted. Because actions that are not performed or are unlikely to be optimal are not learned, andthe state space is not allocated, the learning time can be shortened, and the state space can be reduced. In anexperiment, the proposed method shows results similar to those of traditional Q-learning even when the statespace of the proposed method is reduced to approximately 0.33% of that of Q-learning. Consequently, theproposed method reduces the cost and time required for learning. Traditional Q-learning requires 6 million statespaces for learning 100,000 times. In contrast, the proposed method requires only 20,000 state spaces. A higherwinning rate can be achieved in a shorter period of time by retrieving 20,000 state spaces instead of 6 million.