심층 강화학습을 이용한 휠-다리 로봇의 3차원 장애물극복 고속 모션 계획 방법
Fast Motion Planning of Wheel-legged Robot for Crossing 3D Obstacles using Deep Reinforcement Learning
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
In this study, a fast motion planning method for the swing motion of a 6x6 wheel-legged robot to traverse large obstacles and gaps is proposed. The motion planning method presented in the previous paper, which was based on trajectory optimization, took up to tens of seconds and was limited to two-dimensional, structured vertical obstacles and trenches. A deep neural network based on one-dimensional Convolutional Neural Network (CNN) is introduced to generate keyframes, which are then used to represent smooth reference commands for the six leg angles along the robot’s path. The network is initially trained using the behavioral cloning method with a dataset gathered from previous simulation results of the trajectory optimization. Its performance is then improved through reinforcement learning, using a one-step REINFORCE algorithm. The trained model has increased the speed of motion planning by up to 820 times and improved the success rates of obstacle crossing under harsh conditions, such as low friction and high roughness.
Abstract
In this study, a fast motion planning method for the swing motion of a 6x6 wheel-legged robot to traverse large obstacles and gaps is proposed. The motion planning method presented in the previous paper, which was based on trajectory optimization, took up to tens of seconds and was limited to two-dimensional, structured vertical obstacles and trenches. A deep neural network based on one-dimensional Convolutional Neural Network (CNN) is introduced to generate keyframes, which are then used to represent smooth reference commands for the six leg angles along the robot’s path. The network is initially trained using the behavioral cloning method with a dataset gathered from previous simulation results of the trajectory optimization. Its performance is then improved through reinforcement learning, using a one-step REINFORCE algorithm. The trained model has increased the speed of motion planning by up to 820 times and improved the success rates of obstacle crossing under harsh conditions, such as low friction and high roughness.
참고문헌 (13)
- 정순규, "휠-다리 로봇의 장애물극복 모션 계획 및 제어 방법", 로봇학회 논문지, 17(4), pp.500~507 (2022), 한국로봇학회[DOI]
- Xue Bin Peng, "DeepLoco: Dynamic locomotion skills using hierarchical deep reinforcement learning", ACM Transactions on Graphics, 36(4), pp.1~13 (2017), Association for Computing Machinery (ACM)[DOI]
- Vassilios Tsounis, "DeepGait: Planning and Control of Quadrupedal Gaits Using Deep Reinforcement Learning", IEEE Robotics and Automation Letters, 5(2), pp.3699~3706 (2020), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
- N. Rudin, "Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning", Robotics, pp.2021~ (1978)[DOI]
- Kai Arulkumaran, "Deep Reinforcement Learning: A Brief Survey", IEEE Signal Processing Magazine, 34(6), pp.26~38 (2017), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
- M. Bain, "A framework for behavioural cloning", Machine Intelligence, 15 (1995)
- J. D. Chang, "Mitigating covariate shift in imitation learning via offline data with partial coverage", Machine Learning (2021)[DOI]
- Jan Peters, "Reinforcement learning of motor skills with policy gradients", Neural Networks, 21(4), pp.682~697 (2008), Elsevier BV[DOI]
- Mohammad Sadegh Jazayeri, "Utilizing B-Spline Curves and Neural Networks for Vehicle Trajectory Prediction in an Inverse Reinforcement Learning Framework", Journal of Sensor and Actuator Networks, 11(1), pp.14~ (2022), MDPI AG[DOI]
- Mehmet Hasanzade, "A Dynamically Feasible Fast Replanning Strategy with Deep Reinforcement Learning", Journal of Intelligent & Robotic Systems, 101(1) (2020), Springer Science and Business Media LLC[DOI]
- Emanuel Todorov, "MuJoCo: A physics engine for model-based control", 2012 IEEE, 2012, ~[DOI]
- "pybind11 — Seamless operability between C++11 and Python"[URL]
- "Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms"[URL]