Possibilities of reinforcement learning for nuclear power plants: Evidence on current applications and beyond

키워드

참고문헌 (177)

1. Vivianne H. M. Visschers, "Climate change benefits and energy supply benefits as determinants of acceptance of nuclear power stations : Investigating an explanatory model", Energy Policy, 39(6), pp.3621~3629 (2011)
2. Sheng Zhou, "Nuclear energy development in China : a study of opportunities and challenges", Energy, 35(11), pp.4282~4288 (2010)
3. Barry W Brook, "Why nuclear energy is sustainable and has to be part of the energy mix", Sustain. Mater. Technol, 1, pp.8~16 (2014)
4. James H. Rust, "Nuclear power plant engineering" (1979)
5. Ronald Allen Knief, "Nuclear energy technology: theory and practice of commercial nuclear power" (1981)
6. José M. Arias, "An Advanced Course in Modern Nuclear Physics, Vol. 581" (2008), Springer
7. The Report On The Development of China’s Nuclear Energy 2021, "Mingrong Li Tingke Zhang", EG and G Idaho, Inc, ~ (2021)
8. Steffen Schlömer, "Climate Change 2014: Mitigation of Climate Change: Contribution of Working Group III to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change", pp.1329~1356 (2014), Cambridge University Press
9. Henry P. Birmingham, "A design philosophy for man-machine control systems", Proc. IRE, 42(12), pp.1748~1758 (1954)
10. D. T. McRuer, "The man-machine system concept", Proc. IRE, 50(5), pp.1117~1123 (1962)
11. Jean-Michel Hoc, "From human–machine interaction to human–machine cooperation", Ergonomics, 43(7), pp.833~843 (2000)
12. Hang Wu, "PID controllers: Design and tuning methods", 2014 9th IEEE Conference on Industrial Electronics and Applications, pp.808~813 (2014), IEEE
13. Rames C. Panda, "Introduction to PID Controllers: Theory, Tuning and Application to Frontier Areas" (2012), BoD–Books on Demand
14. MA Ebrahim, "Optimization of proportional-integral-differential controller for wind power plant using particle swarm optimization technique", Int. J. Electr. Power Eng, 6(1), pp.32~37 (2012)
15. Lezhen Shi, "An improved nonlinear proportionalintegral-differential controller combined with fractional operator and symbolic adaptation algorithm", Trans. Inst. Meas. Control, 42(5), pp.927~941 (2020)
16. Stuart Bennett, "Development of the PID controller", IEEE Control Syst. Mag, 13(6), pp.58~62 (1993)
17. Kit-Sang Tang, "An optimal fuzzy PID controller", IEEE Trans. Ind. Electron, 48(4), pp.757~765 (2001)
18. Pritesh Shah, "Review of fractional PID controller", Mechatronics, 38, pp.29~41 (2016)
19. Kelvin T. Erickson, "Programmable logic controllers", IEEE Potentials, 15(1), pp.14~17 (1996)
20. William Bolton, "Programmable Logic Controllers" (2015), Newnes
21. Ephrem Ryan Alphonsus, "A review on the applications of programmable logic controllers(PLCs)", Renew. Sustain. Energy Rev, 60, pp.1185~1205 (2016)
22. Gary A. Dunning, "Introduction to Programmable Logic Controllers" (2005), Cengage Learning
23. Eric Monmasson, "FPGA design methodology for industrial control systems—A review", IEEE Trans. Ind. Electron, 54(4), pp.1824~1842 (2007)
24. Ian Kuon, "FPGA architecture : Survey and challenges", Found. Trends® Electron. Des. Autom, 2(2), pp.135~253 (2008)
25. Andrew Canis, "LegUp: high-level synthesis for FPGA-based processor", accelerator systems, SIGDA International Symposium on Field Programmable Gate Arrays, pp.2011[DOI]
26. Teng Lei, "Design of high pressure water jet decontamination device with pressure and flow synchronous control function", Nucl. Power Eng, 41(3), pp.153~157 (2020)
27. Jonghyun Kim, "Autonomous Nuclear Power Plants with Artificial Intelligence, Vol. 94" (2023), Springer Nature
28. Jürgen Schmidhuber, "Deep learning in neural networks : An overview", Neural Netw, 61, pp.85~117 (2015)
29. Yann LeCun, "Deep learning", Nature, 521(7553), pp.436~444 (2015)
30. Li Deng, "Deep learning : methods and applications", Found. Trends® Signal Process, 7(3–4), pp.197~387 (2014)
31. Pramila P. Shinde, "A review of machine learning and deep learning applications", 2018 Fourth International Conference on Computing Communication Control and Automation (ICCUBEA), pp.1~6 (2018), IEEE
32. Ajeet Ram Pathak, "Application of deep learning for object detection", Procedia Comput. Sci, 132, pp.1706~1717 (2018)
33. Zongmei Gao, "Deep learning application in plant stress imaging : a review", AgriEngineering, 2(3), pp.29~ (2020)
34. Samir Khan, "A review on the application of deep learning in system health management", Mech. Syst. Signal Process, 107, pp.241~265 (2018)
35. Richard S. Sutton, "Reinforcement Learning: An Introduction" (2018), MIT Press
36. Leslie Pack Kaelbling, "Reinforcement learning : A survey", J. Artif. Intell. Res, 4, pp.237~285 (1996)
38. James Ladyman, "What is a complex system?", Eur. J. Philos. Sci, 3(1), pp.33~67 (2013)
39. Takayuki Kanda, "Development and evaluation of interactive humanoid robots", Proc. IEEE, 92(11), pp.1839~1850 (2004)
40. Bradley T. Werner, "Complexity in natural landform patterns", Science, 284(5411), pp.102~104 (1999)
41. Klaus Mainzer, "Thinking in complexity: The complex dynamics of matter, mind, and mankind", Comput. Phys, 9(4), pp.398~ (1995)
42. Robert S. MacKay, "Nonlinearity in complexity science", Nonlinearity, 21(12), pp.T273~ (2008)
43. Alicia Juarrero, "Dynamics in action : Intentional behavior as a complex system", Emergence, 2(2), pp.24~57 (2000)
44. Sven Bertelsen, "Construction as a complex system", Proceedings for the 11th Annual Conference of the International Group for Lean Construction, pp.11~23 (2003)
45. Steven D. Gribble, "Robustness in complex systems", Proceedings Eighth Workshop on Hot Topics in Operating Systems, pp.21~26 (2001), IEEE
46. Herbert A. Simon, "Facets of Systems Science", pp.457~476 (1991), Springer
47. Volodymyr Mnih, "Human-level control through deep reinforcement learning", Nature, 518(7540), pp.529~533 (2015)
48. David Silver, "Mastering the game of go without human knowledge", Nature, 550(7676), pp.354~359 (2017)
50. Sergey Levine, "End-to-end training of deep visuomotor policies", J. Mach. Learn. Res, 17(1), pp.1334~1373 (2016)
51. Oriol Vinyals, "Grandmaster level in StarCraft II using multi-agent reinforcement learning", Nature, 575(7782), pp.350~354 (2019)
53. Jens Kober, "Reinforcement learning in robotics : A survey", Int. J. Robot. Res, 32(11), pp.1238~1274 (2013)
54. Petar Kormushev, "Reinforcement learning in robotics : Applications and real-world challenges", Robotics, 2(3), pp.122~148 (2013)
55. Athanasios S. Polydoros, "Survey of model-based reinforcement learning : Applications on robotics", J. Intell. Robot. Syst, 86(2), pp.153~173 (2017)
56. Wenshuai Zhao, "Sim-to-real transfer in deep reinforcement learning for robotics: a survey", 2020 IEEE Symposium Series on Computational Intelligence (SSCI), pp.737~744 (2020), IEEE
57. Zhong-Qiu Zhao, "Object detection with deep learning : A review", IEEE Trans. Neural Netw. Learn. Syst, 30(11), pp.3212~3232 (2019)
58. Md Tahmid Hasan Fuad, "Recent advances in deep learning techniques for face recognition", IEEE Access, 9, pp.99112~99142 (2021)
59. Dinggang Shen, "Deep learning in medical image analysis", Annu. Rev. Biomed. Eng, 19, pp.221~ (2017)
60. Geert Litjens, "A survey on deep learning in medical image analysis", Med. Image Anal, 42, pp.60~88 (2017)
61. Afan Galih Salman, "Weather forecasting using deep learning techniques", 2015 International Conference on Advanced Computer Science and Information Systems (ICACSIS), pp.281~285 (2015), IEEE
62. Tom Young, "Recent trends in deep learning based natural language processing", IEEE Comput. Intell. Mag, 13(3), pp.55~75 (2018)
63. Yushi Chen, "Deep learningbased classification of hyperspectral data", IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens, 7(6), pp.2094~2107 (2014)
64. Zidong Zhang, "Deep reinforcement learning for power system applications : An overview", CSEE J. Power Energy Syst, 6(1), pp.213~225 (2019)
65. Nguyen Cong Luong, "Applications of deep reinforcement learning in communications and networking : A survey", IEEE Commun. Surv. Tutor, 21(4), pp.3133~3174 (2019)
66. Hao-nan Wang, "Deep reinforcement learning : a survey", Front. Inf. Technol. Electron. Eng, 21(12), pp.1726~1744 (2020)
67. Amirhosein Mosavi, "Comprehensive review of deep reinforcement learning methods and applications in economics", Mathematics, 8(10), pp.1640~ (2020)
68. Ammar Haydari, "Deep reinforcement learning for intelligent transportation systems: A survey", IEEE Trans. Intell. Transp. Syst (2020)
69. Christopher John Cornish Hellaby Watkins, "Learning from delayed rewards", ~ (1989)
71. Leemon Baird, "Residual algorithms : Reinforcement learning with function approximation", Machine Learning Proceedings 1995, pp.30~37 (1995), Elsevier
72. John Tsitsiklis, "Analysis of temporal-diffference learning with function approximation", Adv. Neural Inf. Process. Syst, 9 (1996)
73. Long-Ji Lin, "Reinforcement Learning for Robots using Neural Networks", pp.1992
74. Hado Hasselt, "Double Q-learning", Adv. Neural Inf. Process. Syst., 23 (2010)
75. Hado Van Hasselt, "Deep reinforcement learning with double q-learning", Proceedings of the AAAI Conference on Artificial Intelligence, 30 (2016)
76. Richard S Sutton, "Policy gradient methods for reinforcement learning with function approximation", Adv. Neural Inf. Process. Syst., 12 (1999)
79. John Schulman, "Trust region policy optimization", International Conference on Machine Learning, pp.1889~1897 (2015), PMLR
81. Volodymyr Mnih, "Asynchronous methods for deep reinforcement learning", International Conference on Machine Learning, pp.1928~1937 (2016), PMLR
82. Tuomas Haarnoja, "Soft actor-critic:Off-policy maximum entropy deep reinforcement learning with a stochastic actor", International Conference on Machine Learning, pp.1861~1870 (2018), PMLR
84. Scott Fujimoto, "Off-policy deep reinforcement learning without exploration", International Conference on Machine Learning (2019)
85. Aviral Kumar, "Conservative qlearning for offline reinforcement learning", Adv. Neural Inf. Process. Syst, 33, pp.1179~1191 (2020)
86. Jiafei Lyu, "Mildly conservative Q-learning for offline reinforcement learning", Thirty-Sixth Conference on Neural Information Processing Systems (2022)
87. Scott Fujimoto, "A minimalist approach to offline reinforcement learning", Advances in Neural Information Processing Systems (2021)
88. Aviral Kumar, "Sergey Levine, Stabilizing off-policy Qlearning via bootstrapping error reduction", Neural Information Processing Systems (2019)
89. Jiafei Lyu, "State advantage weighting for offline RL", 3rd Offline RL Workshop: Offline RL As a ’’Launchpad’’ (2022)
91. Tianhe Yu, "MOPO: Model-based offline policy optimization", Advances in Neural Information Processing Systems (2020)
92. Tianhe Yu, "COMBO: Conservative offline model-based policy optimization", Neural Information Processing Systems (2021)
97. Suyang Zhou, "Campbell Booth, Combined heat and power system intelligent economic dispatch: A deep reinforcement learning approach", Int. J. Electr. Power Energy Syst, 120, pp.106016~ (2020)
98. Esmat Samadi, "Decentralized multi-agent based energy management of microgrid using reinforcement learning", Int. J. Electr. Power Energy Syst, 122, pp.106211~ (2020)
99. Hussain Kazmi, "Multi-agent reinforcement learning for modeling and control of thermostatically controlled loads", Appl. Energy, 238, pp.1022~1035 (2019)
100. Tianshu Wei, "Deep reinforcement learning for building HVAC control", Proceedings of the 54th Annual Design Automation Conference 2017, pp.1~6 (2017)
101. Eunsung Oh, "Reinforcement-learning-based energy storage system operation strategies to manage wind power forecast uncertainty", IEEE Access, 8, pp.20965~20976 (2020)
102. Chun Wei, "Reinforcement-learning-based intelligent maximum power point tracking control for wind energy conversion systems", IEEE Trans. Ind. Electron, 62(10), pp.6360~6370 (2015)
103. Huifeng Zhang, "Two-step wind power prediction approach with improved complementary ensemble empirical mode decomposition and reinforcement learning", IEEE Syst. J (2021)
104. Yinliang Xu, "Multiagent-based reinforcement learning for optimal reactive power dispatch", IEEE Trans. Syst. Man Cybern. C, 42(6), pp.1742~1751 (2012)
105. John G. Vlachogiannis, "Reinforcement learning for reactive power control", IEEE Trans. Power Syst, 19(3), pp.1317~1325 (2004)
106. Tao Yu, "Stochastic optimal relaxed automatic generation control in non-Markov environment based on multi-step q learning", IEEE Trans. Power Syst, 26(3), pp.1272~1282 (2011)
107. Fatheme Daneshfar, "Load–frequency control : a GA-based multi-agent reinforcement learning", IET Gener. Transm. Distrib, 4(1), pp.13~26 (2010)
108. T. P. Imthias Ahamed, "A reinforcement learning approach to automatic generation control", Electr. Power Syst. Res, 63(1), pp.9~26 (2002)
109. Tao Yu, "R(𝜆)imitation learning for automatic generation control of interconnected power grids", Automatica, 48(9), pp.2130~2136 (2012)
110. Yujian Ye, "Deep reinforcement learning for strategic bidding in electricity markets", IEEE Trans. Smart Grid, 11(2), pp.1343~1355 (2019)
111. Sina Zarrabian, "Reinforcement learning approach for congestion management and cascading failure prevention with experimental application", Electr. Power Syst. Res, 141, pp.179~190 (2016)
112. Xiaorui Liu, "Reinforcement learning for cyberphysical security assessment of power systems", 2019 IEEE Milan PowerTech, pp.1~6 (2019), IEEE
113. Yin Cheng, "ThermalNet : A deep reinforcement learning-based combustion optimization system for coal-fired boiler", Eng. Appl. Artif. Intell, 74, pp.303~311 (2018)
114. Jigao Fu, "Control strategy for denitrification efficiency of coal-fired power plant based on deep reinforcement learning", IEEE Access, 8, pp.65127~65136 (2020)
116. Volker Stephan, "A new control scheme for combustion processes using reinforcement learning based on neural networks", Int. J. Comput. Intell. Appl, 1(02), pp.121~136 (2001)
117. OpenAI : Marcin Andrychowicz, "Learning dexterous in-hand manipulation", Int. J. Robot. Res, 39(1), pp.3~20 (2020)
119. Bianca Sangiovanni, "Deep reinforcement learning for collision avoidance of robotic manipulators", 2018 European Control Conference (ECC), pp.2063~2068 (2018), IEEE
120. Steven A Harp, "SEPIA. a simulator for electric power industry agents", IEEE Control Syst. Mag, 20(4), pp.53~69 (2000)
121. Morteza Rahimiyan, "An adaptive 𝑄-learning algorithm developed for agent-based computational modeling of electricity market", IEEE Trans. Syst. Man Cybern. C, 40(5), pp.547~556 (2010)
122. Vishnuteja Nanduri, "A reinforcement learning model to assess market power under auction-based energy pricing", IEEE Trans. Power Syst, 22(1), pp.85~95 (2007)
123. Byung-Gook Kim, "Dynamic pricing and energy consumption scheduling with reinforcement learning", IEEE Trans. Smart Grid, 7(5), pp.2187~2198 (2015)
124. Thilo Krause, "A comparison of Nash equilibria analysis and agentbased modelling for power markets", Int. J. Electr. Power Energy Syst, 28(9), pp.599~607 (2006)
126. Aviral Kumar, "Conservative Qlearning for offline reinforcement learning", Advances in Neural Information Processing Systems (2020)
127. Xinyue Chen, "BAIL: Best-action imitation learning for batch deep reinforcement learning", Advances in Neural Information Processing Systems (2020)
128. Ilya Kostrikov, "Offline reinforcement learning with implicit Q-learning", International Conference on Learning Representations (2022)
130. Yue Wu, "Uncertainty weighted actor-critic for offline reinforcement learning", ICML (2021)
131. Zhe Dong, "Multilayer perception based reinforcement learning supervisory control of energy systems with application to a nuclear steam supply system", Appl. Energy, 259, pp.114193~ (2020)
132. Koroush Shirvan Forgeta, "Physics-informed reinforcement learning optimization of nuclear assembly design", ~ (2021)
133. Xiangyi Chen, "Deep reinforcement learning control of a boiling water reactor", IEEE Trans. Nucl. Sci, 69(8), pp.1820~1832 (2022)
135. Tianhao Zhang, "Multi-objective optimization of thermal power and outlet steam temperature for a nuclear steam supply system with deep reinforcement learning", SSRN 4490266, ~
136. JaeKwan Park, "Providing support to operators for monitoring safety functions using reinforcement learning", Prog. Nucl. Energy, 118, pp.103123~ (2020)
137. Daeil Lee, "Development of autonomous operation agent for normal and emergency situations in nuclear power plants", 2021 5th International Conference on System Reliability and Safety (ICSRS), pp.240~247 (2021), IEEE
138. Jing Li, "The application of deep reinforcement learning in coordinated control of nuclear reactors", Journal of Physics: Conference Series, 2113, pp.012030~ (2021), IOP Publishin
139. Jae Min Kim, "Strategy to coordinate actions through a plant parameter prediction model during startup operation of a nuclear power plant", Nucl. Eng. Technol, 55(3), pp.839~849 (2023)
140. Junyong Bae, "Deep reinforcement learning for a multi-objective operation in a nuclear power plant", Nucl. Eng. Technol (2023)
141. Jae Min Kim, "Framework of two-level operation module for autonomous system of nuclear power plants during startup and shutdown operation", Transactions of the Korean Nuclear Society Autumn Meeting (2019)
142. Daeil Lee, "Algorithm for autonomous power-increase operation using deep reinforcement learning and a rule-based system", IEEE Access, 8, pp.196727~196746 (2020)
143. JaeKwan Park, "Control automation in the heat-up mode of a nuclear power plant using reinforcement learning", Prog. Nucl. Energy, 145, pp.104107~ (2022)
144. Jonas Degrave, "Magnetic control of tokamak plasmas through deep reinforcement learning", Nature, 602(7897), pp.414~419 (2022)
145. ATHENA Code Manual. Volume 1. Code Structure, System Models, and Solution Methods, "K. E. Carlson", EG and G Idaho, Inc, ~ (1986)
146. Deail Lee, "Autonomous algorithm for start-up operation of nuclear power plants by using LSTM", International Conference on Applied Human Factors and Ergonomics, pp.465~475 (2018), Springer
147. Seung Jun Lee, "Development of automated operating procedure system using fuzzy colored petri nets for nuclear power plants", Ann. Nucl. Energy, 31(8), pp.849~869 (2004)
148. Yochan Kima, "Envisioning human-automation interactions for responding emergency situations of NPPs: a viewpoint from human-computer interaction", Proc. Trans. Korean Nucl. Soc. Autumn Meeting (2018)
149. Ar Ryum Kim, "Study on the identification of main drivers affecting the performance of human operators during low power and shutdown operation", Ann. Nucl. Energy, 92, pp.447~455 (2016)
150. Thomas M Moerland, "Model-based reinforcement learning : A survey", Found. Trends® Mach. Learn, 16(1), pp.1~118 (2023)
151. Michael Janner, "When to trust your model: Model-based policy optimization", Adv. Neural Inf. Process. Syst, 32 (2019)
152. Marvin Zhang, "Solar: Deep structured representations for model-based reinforcement learning", International Conference on Machine Learning, pp.7444~7453 (2019), PMLR
155. Rasmus E Andersen, "Self-learning processes in smart factories : Deep reinforcement learning for process control of robot brine injection", Procedia Manuf, 38, pp.171~177 (2019)
157. Daeil Lee, "Comparison of deep reinforcement learning and PID controllers for automatic cold shutdown operation", Energies, 15(8), pp.2834~ (2022)
159. Jiafei Lyu, "Efficient continuous control with double actors and regularized critics", AAAI Conference on Artificial Intelligence (2021)
160. Daeil Lee, "Anomaly recovery algorithm based on robust AI concept for nuclear power plants" (2023)
161. Daeil Lee, "Concept of robust AI with meta-learning for accident diagnosis" (2022)
164. Josiah P. Hanna, "Grounded action transformation for sim-to-real reinforcement learning", Mach. Learn, 110, pp.2469~2499 (2021)
165. Wenshuai Zhao, "Sim-to-real transfer in deep reinforcement learning for robotics: a survey", 2020 IEEE Symposium Series on Computational Intelligence (SSCI), pp.737~744 (2020)
166. Han Hu, "A sim-to-real pipeline for deep reinforcement learning for autonomous robot navigation in cluttered rough terrain", IEEE Robot. Autom. Lett, 6, pp.6569~6576 (2021)
167. Xue Bin Peng, "Sim-to-real transfer of robotic control with dynamics randomization", 2018 IEEE International Conference on Robotics and Automation (ICRA), pp.3803~3810 (2018), IEEE
168. Stephen James, "Sim-to-real via sim-to-sim: Data-efficient robotic grasping via randomized-tocanonical adaptation networks", Proceedings of the IEEE, 2019, 12627~12637
169. Karol Arndt, "Meta reinforcement learning for sim-to-real domain adaptation", 2020 IEEE International Conference on Robotics and Automation (ICRA), pp.2725~2731 (2020)
171. Caihua Shan, "Reinforcement learning enhanced explainer for graph neural networks", NeurIPS (2021)
172. Francis Maes, "Policy search in a space of simple closed-form formulas: Towards interpretability of reinforcement learning", Discovery Science (2012)
173. Nicklas Hansen, "Generalization in reinforcement learning by soft data augmentation", 2021 IEEE International Conference on Robotics and Automation (ICRA), pp.13611~13617 (2021)
175. Roberta Raileanu, "Automatic data augmentation for generalization in reinforcement learning", NeurIPS (2021)

관련 논문AI추천