백스터 로봇의 시각기반 로봇 팔 조작 딥러닝을 위한 강화학습 알고리즘 구현
Implementation of End-to-End Training of Deep Visuomotor Policies for Manipulation of a Robotic Arm of Baxter Research Robot
2
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
Reinforcement learning has been applied to various problems in robotics. However, it was still hard to train complex robotic manipulation tasks since there is a few models which can be applicable to general tasks. Such general models require a lot of training episodes. In these reasons, deep neural networks which have shown to be good function approximators have not been actively used for robot manipulation task. Recently, some of these challenges are solved by a set of methods, such as Guided Policy Search, which guide or limit search directions while training of a deep neural network based policy model. These frameworks are already applied to a humanoid robot, PR2. However, in robotics, it is not trivial to adjust existing algorithms designed for one robot to another robot. In this paper, we present our implementation of Guided Policy Search to the robotic arms of the Baxter Research Robot. To meet the goals and needs of the project, we build on an existing implementation of Baxter Agent class for the Guided Policy Search algorithm code using the built-in Python interface. This work is expected to play an important role in popularizing robot manipulation reinforcement learning methods on cost-effective robot platforms.
Abstract
Reinforcement learning has been applied to various problems in robotics. However, it was still hard to train complex robotic manipulation tasks since there is a few models which can be applicable to general tasks. Such general models require a lot of training episodes. In these reasons, deep neural networks which have shown to be good function approximators have not been actively used for robot manipulation task. Recently, some of these challenges are solved by a set of methods, such as Guided Policy Search, which guide or limit search directions while training of a deep neural network based policy model. These frameworks are already applied to a humanoid robot, PR2. However, in robotics, it is not trivial to adjust existing algorithms designed for one robot to another robot. In this paper, we present our implementation of Guided Policy Search to the robotic arms of the Baxter Research Robot. To meet the goals and needs of the project, we build on an existing implementation of Baxter Agent class for the Guided Policy Search algorithm code using the built-in Python interface. This work is expected to play an important role in popularizing robot manipulation reinforcement learning methods on cost-effective robot platforms.
참고문헌 (16)
- D. Silver, "Mastering the game of go with deep neural networks and tree search", Nature, 529(7587), pp.484~489 (2016)
- D. Silver, "Mastering the game of go without human knowledge", Nature, 550(7676), pp.354~359 (2017)
- K. Lee, "Deep reinforcement learning in continuous action spaces: a case study in the game of simulated curling", 35th International Conference on Machine Learning (ICML), pp.2937~2946 (2018)
- K. Arulkumaran, "Deep reinforcement learning : A brief survey", IEEE Signal Processing Magazine, 34(6), pp.26~38 (2017)
- J. Kober, "Learning Motor Skills", pp.9~67 (2014), Springer
- V. Mnih, "Human-level control through deep reinforcement learning", Nature, 518(7540), pp.529~533 (2015)
- T. P. Lillicrap, "Continuous control with deep reinforcement learning", ~ (2015)
- S. Levine, "Guided policy search", 30th International Conference on Machine Learning (ICML), pp.1~9 (2013)
- "Guided policy search"[URL]
- P. Henderson, "Deep reinforcement learning that matters", ~ (2017)
- C. Finn, "Deep spatial autoencoders for visuomotor learning", 2016 IEEE International Conference on Robotics and Automation (ICRA), pp.512~519 (2016)
- Y. Tsurumine, "Deep dynamic policy programming for robot control with raw images", 2017 IEEE, 2017, 1545~1550
- Y. Chebotar, "Combining model-based and model-free updates for trajectory-centric reinforcement learning", 34th International Conference on Machine Learning (ICML), pp.703~711 (2017)
- S. Levine, "End-to-end training of deep visuomotor policies", Journal of Machine Learning Research(JMLR), 17(39), pp.1~40 (2016)
- Bregman alternating direction method of multipliers, "H. Wang", Advances in Neural Information Processing Systems (NIPS), 2816~2824 (2014)
- Guided policy search via approximate mirror descent, "W. Montgomery", Advances in Neural Information Processing Systems (NIPS), 4008~4016 (2016)