강화학습과 Motion VAE를 이용한 자동 장애물 충돌 회피 시스템 구현
An Auto Obstacle Collision Avoidance System using Reinforcement Learning and Motion VAE
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
컴퓨터 애니메이션 및 로보틱스 분야에서 장애물을 회피하면서 목적지에 도착하는 것은 어려운 과제이다. 특히 이동경로의 계획과 적절한 동작의 계획을 동시에 수행하는 것은 기존에 많이 다루어지지 않은 연구분야이다. 최근 연구자들은데이터 기반 생성 모델인 VAE(Variational Auto-Encoder)를 활용하여 캐릭터 모션을 생성하는 문제를 활발하게 연구해왔다. 본 연구에서는 VAE 생성 모델을 동작 공간으로 확장하고, 잠재 공간에서 강화학습을 적용한 MVAE 모델을 활용한다[1]. 이 접근 방식을 통해 학습된 정책을 사용하면 에이전트는 고정된 장애물과 움직이는 장애물을 모두 피하면서, 자연스러운움직임으로 목적지에 도달할 수 있다. 캐릭터는 무작위 방향으로 이동하는 장애물을 견고하게 회피할 수 있었고, 기존연구보다 성능이 뛰어나고 학습 시간도 단축됨을 실험적으로 보였다.
Abstract
In the fields of computer animation and robotics, reaching a destination while avoiding obstacles has always been a difficult task. Moreover, generating appropriate motions while planning a route is even more challenging. Recently, academic circles are actively conducting research to generate character motions by modifying and utilizing VAE (Variational Auto-Encoder), a data-based generation model. Based on this, in this study, the latent space of the MVAE model is learned using a reinforcement learning method[1]. With the policy learned in this way, the character can arrive its destination while avoiding both static and dynamic obstacles with natural motions. The character can easily avoid obstacles moving in random directions, and it is experimentally shown that the performance is improved, and the learning time is greatly reduced compared to existing approach.
참고문헌 (20)
- Hung Yu Ling, "Character Controllers Using Motion VAEs", ACM Trans. Graph, 39(4), pp.12~ (2020)[DOI]
- Xue Bin Peng, "DeepLoco: Dynamic Locomotion Skills Using Hierarchical Deep Reinforcement Learning", ACM Trans. Graph, 36(4), pp.16~ (2017)[DOI]
- Jaedong Lee, "Crowd Simulation by Deep Reinforcement Learning", Proceedings of Motion, Interaction and Games, pp.7~ (2018)
- V. Lumelsky, "Incorporating range sensing in the robot navigation function", IEEE Transactions on Systems Man and Cybernetics, 20, pp.1058~1068 (1990)
- V. Lumelsky, "Autonomous Robots Vehicles", pp.1058~1068 (1990), Springer
- O. Khatib, "Real-time obstacle avoidance for manipulators and mobile robots", International Journal of Robotics Research, 5(1), pp.90~98 (1995)
- J. Borenstein, "The vector field histogram-fast obstacle avoidance for mobile robots", IEEE Transaction on Robotics and Automation, 7(3), pp.278~288 (1991)
- Volodymyr Mnih, "Human-level control through deep reinforcement learning", Nature, 518, pp.529~533 (2015)
- David Silver, "Mastering the Game of Go with Deep Neural Networks and Tree Search", Nature, 529(7587), pp.484~489 (2016)
- Motion fields for interactive character locomotion, "LEE Y.", ACM SIGGRAPH Asia, ~ (2010)
- LEVINE S., "Continuous character control with lowdimensional embeddings", ACM Transactions on Graphics(TOG), 31(4), pp.1~ (2012)
- Robust task based control policies for physics-based characters, "COROS S.", ACM SIGGRAPH Asia, 1~9 (2009)
- PENG X. B., "Dynamic terrain traversal skills using reinforcement learning", ACM Transactions on Graphics(TOG), 34(4), pp.1~11 (2015)
- DUAN Y., "Benchmarking deep reinforcement learning for continuous control", International conference on machine learning, pp.1329~1338 (2016), PMLR
- LEE S., "Scalable muscleactuated human simulation and control", ACM Transactions On Graphics(TOG), 38(4), pp.1~13 (2019)
- "PyTorch Implementations of Reinforcement Learning Algorithms" (2018)[URL]
- Xue Bin Peng, "DeepMimic: Example-Guided Deep Reinforcement Learning of Physics Based Character Skills", ACM Trans. Graph, 37(4), pp.18~ (2018)[DOI]
- He Zhang, "Mode Adaptive Neural Networks for Quadruped Motion Control", ACM Trans. Graph, 37(4), pp.11~ (2018)[DOI]
- Taesoo Kwon, "Adaptive Tracking of a Single-Rigid-Body Character in Various Environments", SIGGRAPH Asia 2023 Conference Papers (SA Conference Papers ’23), pp.11~ (2023), ACM