시연에 의해 유도된 탐험을 통한 시각 기반의 물체 조작
Visual Object Manipulation Based on Exploration Guided by Demonstration
2
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
A reward function suitable for a task is required to manipulate objects through reinforcement learning. However, it is difficult to design the reward function if the ample information of the objects cannot be obtained. In this study, a demonstration-based object manipulation algorithm called stochastic exploration guided by demonstration (SEGD) is proposed to solve the design problem of the reward function. SEGD is a reinforcement learning algorithm in which a sparse reward explorer (SRE) and an interpolated policy using demonstration (IPD) are added to soft actor-critic (SAC). SRE ensures the training of the critic of SAC by collecting prior data and IPD limits the exploration space by making SEGD’s action similar to the expert’s action. Through these two algorithms, the SEGD can learn only with the sparse reward of the task without designing the reward function. In order to verify the SEGD, experiments were conducted for three tasks. SEGD showed its effectiveness by showing success rates of more than 96.5% in these experiments.
Abstract
A reward function suitable for a task is required to manipulate objects through reinforcement learning. However, it is difficult to design the reward function if the ample information of the objects cannot be obtained. In this study, a demonstration-based object manipulation algorithm called stochastic exploration guided by demonstration (SEGD) is proposed to solve the design problem of the reward function. SEGD is a reinforcement learning algorithm in which a sparse reward explorer (SRE) and an interpolated policy using demonstration (IPD) are added to soft actor-critic (SAC). SRE ensures the training of the critic of SAC by collecting prior data and IPD limits the exploration space by making SEGD’s action similar to the expert’s action. Through these two algorithms, the SEGD can learn only with the sparse reward of the task without designing the reward function. In order to verify the SEGD, experiments were conducted for three tasks. SEGD showed its effectiveness by showing success rates of more than 96.5% in these experiments.
참고문헌 (14)
- S. Gu, "Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates", 2017 IEEE International Conference on Robotics and Automation (ICRA) (2017)
- A. Y. Ng, "Algorithms for inverse reinforcement learning", 17th International Conference on Machine Learning (ICML) (2000)
- G. Schoettler, "Deep reinforcement learning for industrial insertion tasks with visual inputs and natural rewards", International Conference on Intelligent Robots and Systems (2019)
- X. B. Peng, "Deepmimic:Example-guided deep reinforcement learning of physics-based character skills", ACM SIGGRAPH (2018)
- "Learning complex dexterous manipulation with deep reinforcement learning and demonstrations" (2018)[URL]
- "Learning from demonstrations for real world reinforcement learning" (2017)[URL]
- A. Nair, "Overcoming exploration in reinforcement learning with demonstrations", 2018 IEEE International Conference on Robotics and Automation (ICRA) (2017)
- "EndtoEnd Robotic Reinforcement Learning without Reward Engineering" (2019)[URL]
- "Unsupervised perceptual rewards for imitation learning" (2017)[URL]
- "AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videos" (2019)[URL]
- "Soft Actor-Critic:Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor, Machine Learning Research" (2018)[URL]
- S. Schaal, "Computational approaches to motor learning by imitation", Philosophical Transactions:Biological Sciences (2003)[DOI]
- "Understanding disentangling in beta-vae" (2017)[URL]
- "Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction, Advances in Neural Information Processing Systems 32 (NeurIPS 2019)" (2019)[URL]