노이즈 환경에서 효과적인 로봇 강화 학습의 정책 탐색 방법
Effective Policy Search Method for Robot Reinforcement Learning with Noisy Reward
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Robots are widely used in industries and services. Traditional robots have been used to perform repetitive tasks in a fixed environment, and it is very difficult to solve a problem in which the physical interaction of the surrounding environment or other objects is complicated with the existing control method. Reinforcement learning has been actively studied as a method of machine learning to solve such problems, and provides answers to problems that robots have not solved in the conventional way. Studies on the learning of all physical robots are commonly affected by noise. Complex noises, such as control errors of robots, limitations in performance of measurement equipment, and complexity of physical interactions with surrounding environments and objects, can act as factors that degrade learning. A learning method that works well in a virtual environment may not very effective in a real robot. Therefore, this paper proposes a weighted sum method and a linear regression method as an effective and accurate learning method in a noisy environment. In addition, the bottle flipping was trained on a robot and compared with the existing learning method, the validity of the proposed method was verified.
Abstract
Robots are widely used in industries and services. Traditional robots have been used to perform repetitive tasks in a fixed environment, and it is very difficult to solve a problem in which the physical interaction of the surrounding environment or other objects is complicated with the existing control method. Reinforcement learning has been actively studied as a method of machine learning to solve such problems, and provides answers to problems that robots have not solved in the conventional way. Studies on the learning of all physical robots are commonly affected by noise. Complex noises, such as control errors of robots, limitations in performance of measurement equipment, and complexity of physical interactions with surrounding environments and objects, can act as factors that degrade learning. A learning method that works well in a virtual environment may not very effective in a real robot. Therefore, this paper proposes a weighted sum method and a linear regression method as an effective and accurate learning method in a noisy environment. In addition, the bottle flipping was trained on a robot and compared with the existing learning method, the validity of the proposed method was verified.
참고문헌 (16)
- N. Kohl, "Policy Gradient Reinforcement Learning for Fast Quadrupedal Locomotion", 2004 IEEE International Conference on Robotics & Automation, pp.2619~2624 (2004)
- M. T. Rosenstein, "Robot Weightlifting By Direct Policy Search", 2001 International Joint Conference on Artificial Intelligence, pp.839~844 (2001)
- Jens Kober, "Policy search for motor primitives in robotics", Machine Learning, 84(1-2), pp.171~203 (2010), Springer Science and Business Media LLC[DOI]
- P. Kormushev, "Learning the skill of archery by a humanoid robot iCub", 2010 10th IEEE-RAS International Conference on Humanoid Robots, pp.417~423 (2010)
- 강동훈, "실시간 장애물 회피 자동 조작을 위한 차량 동역학 기반의 강화학습 전략", 로봇학회 논문지, 12(3), pp.297~305 (2017), 한국로봇학회
- R. S. Sutton, "Reinforcement Learning: An Introduction", pp.1~18 (2014), The MIT Press
- Marc Peter Deisenroth, "A Survey on Policy Search for Robotics", Foundations and Trends in Robotics, 2(1-2), pp.1~142 (2011), Now Publishers[DOI]
- 양영하, "보틀플리핑의 로봇 강화학습을 위한 효과적인 보상 함수의 설계", 로봇학회 논문지, 14(2), pp.081~086 (2019), 한국로봇학회[DOI]
- P. Abbeel, "Using Inaccurate Models in Reinforcement Learning", 23rd International Conference on Machine Learning, 1(8), pp.1~8 (2006)
- M. J. Mataric, "Reward Functions for Accelerated Learning", Eleventh International Conference, pp.181~189 (1994)
- H. Hachiya, "Reward-Weighted Regression with Sample Reuse for Direct Policy Search in Reinforcement Learning", Neural Computation, 23(11), pp.2798~2832 (2011)[DOI]
- B. C. da Silva, "Learning parameterized motor skills on a humanoid robot", 2014 IEEE International Conference on Robotics and Automation (ICRA), pp.5239~5244 (2014)
- S. H. Lee, "Designing an efficient reward function for robot reinforcement learning of the water bottle flipping task", M.S, pp.2018
- J. Kober, "Learning Motor Primitives for Robotics", 2009 IEEE International Conference on Robotics and Automation, pp.2112~2118 (2009)
- J. Wang, "Reinforcement Learning with Perturbed Rewards", AAAI Technical Track : Machine Learning (2020)[DOI]
- K. Främling, "Reinforcement Learning in a Noisy Environment : Light-Seeking Robot", WSEAS Transactions on Systems, 3(2), pp.714~719 (2004)