Unity ML Agents를 활용한 트랙 경로 탐색 시뮬레이션 설계
Design of track path-finding simulation using Unity ML Agents
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
본 연구에서는 강화학습 기술을 이용하여, 시뮬레이션이나 게임 환경 내에서 개체의 경로 탐색을 위한 시뮬레이션을 개발하는 것을 목표로 한다. 본 연구에서는 주어진 트랙 위에 생성된 임의위치의 장애물을 회피하고, 아이템을 획득할 수 있는 경로를 자동으로 탐색할 수 있도록 시뮬레이션 내 개체를 학습시킨 점이 주된 특징이다. 해당 시뮬레이션을 구현하기 위해 유니티 게임 엔진에서 제공하는 ML 에이전트 (Machine Learning Agents)를 사용하였고, PPO(Proximal Policy Optimization)에 기반을 둔 학습 정책을 수립하여 강화학습 환경을 구성한다. 본 논문에서 제안한강화학습 기반의 시뮬레이션을 통해, 개체가 학습을 진행할수록 장애물을 회피하고, 아이템을 획득할 수 있는 경로를 탐색해 트랙 위를 움직이고 있다는 점을 시뮬레이션 결과와 학습 결과 그래프를 분석하여 확인할 수 있다.
Abstract
This paper aims to design a simulation for path-finding of objects in a simulation or game environment using reinforcement learning techniques. The main feature of this study is that the objects in the simulation are trained to avoid obstacles at random locations generated on a given track and to automatically explore path to get items. To implement the simulation, ML Agents provided by Unity Game Engine were used, and a learning policy based on PPO (Proximal Policy Optimization) was established to form a reinforcement learning environment. Through the reinforcement learning-based simulation designed in this study, we were able to confirm that the object moves on the track by avoiding obstacles and exploring path to acquire items as it learns, by analyzing the simulation results and learning result graph.
참고문헌 (16)
- H. -B. Choi, "Design and Implementation of Reinforcement Learning Environment Using Unity 3D-based ML-Agents Toolkit", Proceedings of the Korea Information Processing Society Conference, pp.548~551 (2019)
- Vinyals, O., "Grandmaster level in StarCraft II using multi-agent reinforcement learning", Nature, 575, pp.350~354 (2019)
- "Ml-agents github page"[URL]
- "Unity Machine Learning Agents"[URL]
- Jin-Woong Kim, "Design of Action Game Using Three-Dimensional Map and Interactions between In-Game Objects", Journal of The Korea Society of Computer and Information, 27(12), pp.85~92 (2022)
- Lee Young-Joon, "Development of Rhythm Game using Unity Engine", Proceedings of KIIT Conference, pp.530~533 (2022)
- Bae Jae Hwan, "Design and Development for Unity3D Game Engine using the Shooting Game", Journal of The Korean Society for Computer Game, 29(1), pp.93~100 (2016)
- Gang In Lee, "Implementation of Metaverse Virtual World using Unity Game Engine", Journal of the Semiconductor & Display Technology, 22(2), pp.120~127 (2023)
- Z. Wang, "Driver Behavior Modeling Using Game Engine and Real Vehicle : A Learning-Based Approach", IEEE Transactions on Intelligent Vehicles, 5(4), pp.738~749 (2020)[DOI]
- S. Jo, "Drone Obstacle Avoidance Algorithm using Camera-based Reinforcement Learning", Journal of the Korea Computer Graphics Society, 27(5), pp.63~71 (2021), Korea Computer Graphics Society
- Č. Livada, "Advanced Mechanisms of Perception in the Digital Hide and Seek Game Based on Deep Learning", 2022 International Conference on Smart Systems and Technologies (SST), pp.135~140 (2022)
- D. Kim, "Performance Analysis of Target Tracking AI based on Unity ML-Agents", The Journal of Korean Institute of Information Technology, 19(12), pp.19~26 (2021)
- Sung Gwan Park, "Autonomous Flying of Drone Based on PPO Reinforcement Learning Algorithm", Journal of Institute of Control, Robotics and Systems, 26(11), pp.955~963 (2020)
- "Parameters In ML-Agents Toolkit"[URL]
- Y. -H. Liang, "A Study about the Usefulness of Reinforcement Learning in Business Simulation Games using PPO Algorithm", Journal of Korea Game Society, 19(6), pp.61~70 (2019), Korea Academic Society of Games