Q-Learning을 사용한 로봇팔의 SMCSPO 게인 튜닝
Gain Tuning for SMCSPO of Robot Arm with Q-Learning
2
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
Sliding mode control (SMC) is a robust control method to control a robot arm with nonlinear properties. A high switching gain of SMC causes chattering problems, although the SMC allows the adequate control performance by giving high switching gain, without the exact robot model containing nonlinear and uncertainty terms. In order to solve this problem, SMC with sliding perturbation observer (SMCSPO) has been researched, where the method can reduce the chattering by compensating the perturbation, which is estimated by the observer, and then choosing a lower switching control gain of SMC. However, optimal gain tuning is necessary to get a better tracking performance and reducing a chattering. This paper proposes a method that the Q-learning automatically tunes the control gains of SMCSPO with an iterative operation. In this tuning method, the rewards of reinforcement learning (RL) are set minus tracking errors of states, and the action of RL is a change of control gain to maximize rewards whenever the iteration number of movements increases. The simple motion test for a 7-DOF robot arm was simulated in MATLAB program to prove this RL tuning algorithm. The simulation showed that this method can automatically tune the control gains for SMCSPO.
Abstract
Sliding mode control (SMC) is a robust control method to control a robot arm with nonlinear properties. A high switching gain of SMC causes chattering problems, although the SMC allows the adequate control performance by giving high switching gain, without the exact robot model containing nonlinear and uncertainty terms. In order to solve this problem, SMC with sliding perturbation observer (SMCSPO) has been researched, where the method can reduce the chattering by compensating the perturbation, which is estimated by the observer, and then choosing a lower switching control gain of SMC. However, optimal gain tuning is necessary to get a better tracking performance and reducing a chattering. This paper proposes a method that the Q-learning automatically tunes the control gains of SMCSPO with an iterative operation. In this tuning method, the rewards of reinforcement learning (RL) are set minus tracking errors of states, and the action of RL is a change of control gain to maximize rewards whenever the iteration number of movements increases. The simple motion test for a 7-DOF robot arm was simulated in MATLAB program to prove this RL tuning algorithm. The simulation showed that this method can automatically tune the control gains for SMCSPO.
참고문헌 (21)
- 한승훈, "동적 신경회로망을 이용한 비선형 크레인 시스템의 위치제어", 전기학회논문지ABCD, 56(5), pp.966~972 (2007), 대한전기학회
- K. D. Young, "A control engineer’s guide to sliding mode control", IEEE Transactions on Control Systems Technology, 7(3), pp.328~342 (1999)[DOI]
- V. Utkin, "Chattering Problem in Sliding Mode Control Systems", International Workshop on Variable Structure Systems, 2006. VSS’06, pp.346~350 (2006)
- J. T. Moura, "Sliding Mode Control with Sliding Perturbation Observer", ASME. J. Dyn. Sys., Meas., Control, 119(4), pp.657~665 (1997)
- 정민규, "SMCSPO 기반 조립용 산업 매니퓰레이터의 강인한 제어 연구", 제어.로봇.시스템학회 논문지, 24(6), pp.552~560 (2018), 제어·로봇·시스템학회[DOI]
- 차금강, "원격해체 작업을 위한 유압 시스템의 SPO 기반 반력 추정 및 힘 반향 양방향 원격제어", 로봇학회 논문지, 12(1), pp.1~10 (2017), 한국로봇학회
- T. -C. Kuo, "Sliding mode control with self-tuning law for uncertain nonlinear systems", ISA Transactions, 47(2), pp.171~178 (2008)[DOI]
- Min Cheol Lee, "Sliding Mode Controller with Sliding Perturbation Observer Based on Gain Optimization using Genetic Algorithm", Journal of Mechanical Science and Technology, 18(4), pp.630~639 (2004), 대한기계학회
- C. Szepesvári, "Algorithms for reinforcement learning", Synthesis Lectures on Artificial Intelligence and Machine Learning, 4(1), pp.1~103 (2010)[DOI]
- Y. Yu, "A Grasping CNN with Image Segmentation for Mobile Manipulating Robot", 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp.1688~1692 (2019)
- H. H. Kim, "Development of Reinforcement Learning Assembly Algorithm Based on Estimated Reaction Force Using Sliding Perturbation Observer", International Conference on Control, Automation and System, pp.1018~1021 (2020)
- Y. Ansari, "A Multiagent Reinforcement Learning approach for inverse kinematics of high dimensional manipulators with precision positioning", 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob), pp.457~463 (2016)
- F. L. Lewis, "Reinforcement Learning and Feedback Control : Using Natural Decision Methods to Design Optimal Adaptive Controllers", IEEE Control Systems Magazine, 32(6), pp.76~105 (2012)[DOI]
- W. J. Shipman, "Reinforcement Learning and Deep Neural Networks for PI Controller Tuning", IFAC-Papers OnLine, 52(14), pp.111~116 (2019)[DOI]
- C. J. Watkins, "Q-learning", Machine Learning, 8, pp.279~292 (1992)
- J. M. Hollerbach, "A Recursive Lagrangian Formulation of Maniputator Dynamics and a Comparative Study of Dynamics Formulation Complexity", IEEE Transactions on Systems, Man, and Cybernetics, 10(11), pp.730~736 (1980)[DOI]
- J. -J. Slotine, "On Sliding Observers for Non-Linear Systems", ASME Journal of Dynamic Systems, Measurement, and Control, 109(3), pp.245~252 (1987)[DOI]
- E. Rodrigues Gomes, "Dynamic analysis of multiagent Q-learning with ε-greedy exploration", Proceedings of the 26th Annual International Conference on Machine Learning (ICML ’09). Association for Computing Machinery, pp.369~376 (2009)
- D. Silver, "Deterministic Policy Gradient Algorithms", 31st International Conference on Machine Learning, Proceedings of Machine Learning Research, 32(1), pp.387~395 (2014)
- T. Hester, "Deep q-learning from demonstrations", arXiv:1704.03732, ~ (2018)
- W. Jouini, "Upper Confidence Bound Based Decision Making Strategies and Dynamic Spectrum Access", 2010 IEEE International Conference on Communications, pp.1~5 (2010)