Reinforcement learning-based control with application to the once-through steam generator system
2
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
A reinforcement learning framework is proposed for the control problem of outlet steam pressure of the once-through steam generator(OTSG) in this paper. The double-layer controller using Proximal Policy Optimization(PPO) algorithm is applied in the control structure of the OTSG. The PPO algorithm can train the neural networks continuously according to the process of interaction with the environment and then the trained controller can realize better control for the OTSG. Meanwhile, reinforcement learning has the characteristic of difficult application in real-world objects, this paper proposes an innovative pretraining method to solve this problem. The difficulty in the application of reinforcement learning lies in training. The optimal strategy of each step is summed up through trial and error, and the training cost is very high. In this paper, the LSTM model is adopted as the training environment for pretraining, which saves training time and improves efficiency. The experimental results show that this method can realize the self-adjustment of control parameters under various working conditions, and the control effect has the advantages of small overshoot, fast stabilization speed, and strong adaptive ability.
Abstract
A reinforcement learning framework is proposed for the control problem of outlet steam pressure of the once-through steam generator(OTSG) in this paper. The double-layer controller using Proximal Policy Optimization(PPO) algorithm is applied in the control structure of the OTSG. The PPO algorithm can train the neural networks continuously according to the process of interaction with the environment and then the trained controller can realize better control for the OTSG. Meanwhile, reinforcement learning has the characteristic of difficult application in real-world objects, this paper proposes an innovative pretraining method to solve this problem. The difficulty in the application of reinforcement learning lies in training. The optimal strategy of each step is summed up through trial and error, and the training cost is very high. In this paper, the LSTM model is adopted as the training environment for pretraining, which saves training time and improves efficiency. The experimental results show that this method can realize the self-adjustment of control parameters under various working conditions, and the control effect has the advantages of small overshoot, fast stabilization speed, and strong adaptive ability.
참고문헌 (29)
- H. Yao, "Study on the systematic thermal-hydraulic characteristics of helical coil once-through steam generator", Ann. Nucl. Energy, 154, pp.108096~ (2021)
- G. Zhao, "Integral control strategy between the casing oncethrough steam generator and the turbine", Energy Conserv. Technol, 220, pp.162~166 (2020)
- Y. Zhang, "Dynamic modeling ,simulation and control of helical coiled once-through steam generator", Appl. Sci. Technol, 313, pp.71~77 (2020)
- S. Cheng, "Research of pressure control based on artificial immune control of once -through steam generator", Nucl. Power Eng, 36, pp.62~65 (2015)
- Z. Chen, "Study on application of T-S fuzzy neural method in once-through steam generator feedwater control", Nucl. Power Eng, 33, pp.20~23 (2012)
- X. Hu, "Research on control strategy of once-through steam generator for integrated reactor", J. Shanghai Univ. Electr. Power, 37, pp.115~120 (2021)
- R. S. Sutton, "Reinforcement learning is direct adaptive optimal control", IEEE Control Syst. Mag, 12, pp.19~22 (1992)
- C. J. C. H. Watkins, "Q-learn", Mach. Learn, 8, pp.279~292 (1992)
- T.P. Lillicrap, "Continuous Control with Deep Reinforcement Learning", IN201847005934A, 1~16 (2018)
- X. Wang, "Solving job scheduling problems in a resource preemption environment with multi-agent reinforcement learning", Robot. Comput. Integrated Manuf, 77, pp.102324~ (2022)
- X. Deng, "Towards optimal HVAC control in non-stationary building environments combining active change detection and deep reinforcement learning", Build. Environ, 211, pp.108680, 1~108680.16 (2022)
- X. Qiu, "Attitude control of a moving MassA-ctuated UAV based on deep reinforcement learning", J. Aero. Eng, 35, pp.4021133.1~4021133.12 (2022)
- R. B. Grando, "Double critic deep reinforcement learning for mapless 3D navigation of unmanned aerial vehicles", J. Intell. Rob. Syst, 104, pp.29~43 (2022)
- R. Zhang, "A reinforcement learning method for human-robot collaboration in assembly tasks", Robot. Comput. Integrated Manuf, 73, pp.1~10 (2022)
- J. K. Park, "Providing support to operators for monitoring safety functions using reinforcement learning", Prog. Nucl. Energy, 118, pp.103123~ (2022)
- T. Nishida, "Data transformation and normalization", Rinsho Byori the Japanese Journal of Clinical Pathology, 58, pp.990~997 (2010)
- M. S. David, "Comparison of word embeddings in text classification based on RNN and CNN", IOP Conf. Ser. Mater. Sci. Eng, 1187, pp.247~255 (2021)
- Q. Ye, "A power load prediction method of associated industry chain production resumption based on multitask LSTM", Energy Rep, 8, pp.239~249 (2022)
- A. Zeng, "Stock recommendation system based on deep bidirectional LSTM", Comput. Sci, 46, pp.84~89 (2019)
- J. Ren, "Stock forecasting system based on elstm-l model", Stat. Decis, 35, pp.160~164 (2019)
- I. Papatsouma, "Approximating symmetric distributions via sampling and coefficient of variation", Commun. Stat, 49, pp.61~77 (2020)
- V. Mnih, "Playing atari with deep reinforcement learning", CoRR abs, 2013, pp.1~9
- T.P. Lillicrap, "Continuous Control with Deep Reinforcement Learning", Computer ence, pp.1~16 (2015)
- R. J. Williams, "Simple statistical gradient-following algorithms for connectionist reinforcement learning", Mach. Learn, 8, pp.229~256 (1992)
- J. Schulman, "Trust region policy optimization", Int. Conf. Mach. Learn, 3, pp.244~259 (2016)
- P. Hamalainen, "PPO-CMA: proximal policy optimization with covariance matrix adaptation", IEEE 30th Int. Workshop on Mach. Learn. Signal Proc, pp.1~6 (2020)
- J. Baxter, "Infinite-horizon policy-gradient estimation", J. Artif. Intell. Res, 15, pp.319~350 (2019)
- D. Yan, "Bench marking deep reinforcement learning for continuous control", Int. Conf. Mach. Learn, 3, pp.2001~2014 (2016)
- Y. Wu, "Reinforcement learning in dual-arm trajectory planning for a free-floating space robot", Aero. Sci. Technol, 98, pp.105657~ (2020)