정리정돈을 위한 Q-learning 기반의 작업계획기
Tidy-up Task Planner based on Q-learning
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
As the use of robots in service area increases, research has been conducted to replace human tasks in daily life with robots. Among them, this study focuses on the tidy-up task on a desk using a robot arm. The order in which tidy-up motions are carried out has a great impact on the success rate of the task. Therefore, in this study, a neural network-based method for determining the priority of the tidy-up motions from the input image is proposed. Reinforcement learning, which shows good performance in the sequential decision-making process, is used to train such a task planner. The training process is conducted in a virtual tidy-up environment that is configured the same as the actual tidy-up environment. To transfer the learning results in the virtual environment to the actual environment, the input image is preprocessed into a segmented image. In addition, the use of a neural network that excludes unnecessary tidy-up motions from the priority during the tidy-up operation increases the success rate of the task planner. Experiments were conducted in the real world to verify the proposed task planning method.
Abstract
As the use of robots in service area increases, research has been conducted to replace human tasks in daily life with robots. Among them, this study focuses on the tidy-up task on a desk using a robot arm. The order in which tidy-up motions are carried out has a great impact on the success rate of the task. Therefore, in this study, a neural network-based method for determining the priority of the tidy-up motions from the input image is proposed. Reinforcement learning, which shows good performance in the sequential decision-making process, is used to train such a task planner. The training process is conducted in a virtual tidy-up environment that is configured the same as the actual tidy-up environment. To transfer the learning results in the virtual environment to the actual environment, the input image is preprocessed into a segmented image. In addition, the use of a neural network that excludes unnecessary tidy-up motions from the priority during the tidy-up operation increases the success rate of the task planner. Experiments were conducted in the real world to verify the proposed task planning method.
참고문헌 (20)
- Mehmet R. Dogar, "A Planning Framework for Non-Prehensile Manipulation under Clutter and Uncertainty", Autonomous Robots, 33(3), pp.217~236 (2012), Springer Science and Business Media LLC[DOI]
- M. Fox, "PDDL2.1: An Extension to PDDL for Expressing Temporal Planning Domains", Journal of Artificial Intelligence Research, 20, pp.61~124 (2003), AI Access Foundation[DOI]
- S. Srivastava, "Combined task and motion planning through an extensible planner-independent interface layer", 2014 IEEE International Conference on Robotics and Automation(ICRA), pp.639~646 (2014)
- R. Munos, "From bandits to Monte-Carlo Tree Search : The optimistic principle applied to optimization and planning", Foundations and Trends R in Machine Learning, 7(1), pp.1~129 (2014)
- Yann Labbe, "Monte-Carlo Tree Search for Efficient Visually Guided Rearrangement Planning", IEEE Robotics and Automation Letters, 5(2), pp.3715~3722 (2020), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
- "Transfer from simulation to real world through learning deep inverse dynamics model" (2016)[URL]
- Liang-Chieh Chen, "DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs", IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4), pp.834~848 (2018), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
- T. Schaul, "Universal value function approximators", The 32nd International Conference on Machine Learning, 1312(1320), pp.1312~1320 (2015)
- "Rainbow : Combining improvements in deep reinforcement learning" (2017)[URL]
- Volodymyr Mnih, "Human-level control through deep reinforcement learning", Nature, 518(7540), pp.529~533 (2015), Springer Science and Business Media LLC[DOI]
- R. Dearden, "Bayesian Q-learning", Innovative Applications of Artificial Intelligence Conference, pp.761~768 (1998)
- H. Hasselt, "Double Q-learning", Advances in Neural Information Processing Systems 23 (NIPS 2010), pp.2613~2621 (2010)
- "Prioritized experience replay" (2015)[URL]
- Z. Wang, "Dueling network architectures for deep reinforcement learning", The 33rd International Conference on Machine Learning, pp.1995~2003 (2016)
- "Noisy networks for exploration" (2017)[URL]
- Richard S. Sutton, "Learning to predict by the methods of temporal differences", Machine Learning, 3(1), pp.9~44 (1988), Springer Science and Business Media LLC[DOI]
- "A distributional perspective on reinforcement learning" (2017)[URL]
- Xiaojin Zhu, "Introduction to Semi-Supervised Learning", Synthesis Lectures on Artificial Intelligence and Machine Learning, 3(1), pp.1~130 (2009), Morgan & Claypool Publishers LLC[DOI]
- D. M. Allen, "Mean square error of prediction as a criterion for selecting variables", Technometrics, 13(3), pp.469~475 (1971)
- Igor V. Tetko, "Neural network studies. 1. Comparison of overfitting and overtraining", Journal of Chemical Information and Computer Sciences, 35(5), pp.826~833 (1995), American Chemical Society (ACS)[DOI]