CNN 기반 기보학습 및 강화학습을 이용한 인공지능 게임 에이전트
An Artificial Intelligence Game Agent Using CNN Based Records Learning and Reinforcement Learning
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
본 논문에서는 인공지능 오델로 게임 에이전트를 구현하기 위해 실제 프로기사들의 기보를 CNN으로 학습시키고 이를 상태의 형세 판단을 위한 근거로 삼아 최소최대탐색을 이용해 현 상태에서 최적의 수를 찾는 의사결정구조를 사용하고 이를발전시키고자 강화학습 이론을 이용한 자가대국 학습방법을 제안하여 적용하였다. 본 논문에서 제안하는 구현 방법은 기보학습의 성능 평가 차원에서 가치평가를 위한 네트워크로서 기존의 ANN을 사용한 방법과 대국을 통한 방법으로 비교하였으며, 대국 결과 흑일 때 69.7%, 백일 때 72.1%의 승률을 나타내었다. 또한 본 논문에서 제안하는 강화학습 적용 결과 네크워크의 성능을 강화학습을 적용하지 않은 ANN 및 CNN 가치평가 네트워크 기반 에이전트와 비교한 결과 각각 100%, 78% 승률을 나타내어 성능이 개선됨을 확인할 수 있었다.
Abstract
This paper proposes a CNN architecture as value function network of an artificial intelligence Othello game agentand its learning scheme using reinforcement learning algorithm. We propose an approach to construct the valuefunction network by using CNN to learn the records of professional players’ real game and an approach to enhance thenetwork parameter by learning from self-play using reinforcement learning algorithm. The performance of valuefunction network CNN was compared with existing ANN by letting two agents using each network to play gameseach other. As a result, the winning rate of the CNN agent was 69.7% and 72.1% as black and white, respectively. Inaddition, as a result of applying the reinforcement learning, the performance of the agent was improved by showing100% and 78% winning rate, respectively, compared with the network-based agent without the reinforcement learning.
참고문헌 (15)
- 전영진, "ANN 기반 기보학습 및 Minimax 탐색 알고리즘을 이용한 오델로 게임 플레이어의 구현", 전기학회논문지, 67(12), pp.1657~1664 (2018), 대한전기학회
- D. Silver, "Mastering the game of Go with deep neural networks and tree search", Nature, 529, pp.484~489 (2016)
- D. Silver, "Mastering the game of Go with-out human knowledge", Nature, 550, pp.354~359 (2017)
- Murray Campbell, "Deep Blue", Artificial Intelligence, 134(1-2), pp.57~83 (2002), Elsevier BV[DOI]
- M. Buro, "LOGISTELLO-A Strong Learning Othello Program" (1997), NEC Research Institute
- Paul S. Rosenbloom, "A world-championship-level Othello program", Artificial Intelligence, 19(3), pp.279~320 (1982), Elsevier BV[DOI]
- Kai-Fu Lee, "The development of a world class Othello program", Artificial Intelligence, 43(1), pp.21~36 (1990), Elsevier BV[DOI]
- J. Schaeffer, "Chips Challenging Champions: Games, Computers and Artificial Intelligence", pp.135~ (2002), North Holland
- Gunawan, "Evolutionary Neural Network for Othello Game", Procedia - Social and Behavioral Sciences, 57, pp.419~425 (2012), Elsevier BV[DOI]
- Pawel Liskowski, "Learning to Play Othello With Deep Neural Networks", IEEE Transactions on Games, 10(4), pp.354~364 (2018), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
- Reinforcement learning and its application to othello, "N. J. van Eck", Econometric Institute, ~, pp.Technical Report EI 2005-47 (2005)
- M. van der Ree, "Reinforcement learning in the game of Othello: Learning against a fixed opponent and learning from self-play", 2013 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), pp.108~115 (2013)
- R. S. Sutton, "Reinforcement Learning: An Introduction" (1998), MIT Press
- R Hahnloser, "Digital selection and analogue amplification coesist in a cortex-inspired silicon circuit", Nature, 405, pp.947~951 (2000)
- Y. J. Jeon, "Implementation of an artificial intelligence game agent using deep neural network and reinforcement learning", master’s, pp.2019