본문 바로가기
KERD
논문KCI 등재

Method for collision avoidance based on deep reinforcement learning with path-speed control for an autonomous ship

이혜원|유동훈|노명일|전도현
대한조선학회 (The Society Of Naval Architects Of Korea)|2024.12|International Journal of Naval Architecture and Ocean Engineering|제16권 |pp. 1-19|DOI 10.1016/j.ijnaoe.2023.100579

0

조회

0

다운로드

3

인용

0

참고문헌

키워드

Deep reinforcement learningCollision riskAutonomous shipCollision avoidanceCOLREGs
주제분류조선공학

초록

In this paper, we propose a collision avoidance method based on deep reinforcement learning (DRL) that simultaneously controls the path and speed of a ship. The DRL is actively applied in machine control and artificial intelligence. To verify the proposed method, we applied it to the Imazu problem. It provides benchmark scenarios for collision avoidance. In particular, we compared and analyzed the collision avoidance performance according to the level of learning and various parameters to ensure that the proposed method displays optimal avoidance performance. The results indicated that the proposed method can determine a safe avoidance path for a given situation. Finally, to compare the performance of the proposed method, we compared the collision avoidance method based on the path–speed control of the OS proposed in this study with the collision avoidance method that controls only the path of the OS (Chun et al., 2021). We observed that the proposed method failed in 6 out of 20 scenarios of the Imazu problem when only the path of the OS was controlled. However, it succeeded in collision avoidance in all the 20 scenarios when both path and speed were controlled simultaneously.

Abstract

In this paper, we propose a collision avoidance method based on deep reinforcement learning (DRL) that simultaneously controls the path and speed of a ship. The DRL is actively applied in machine control and artificial intelligence. To verify the proposed method, we applied it to the Imazu problem. It provides benchmark scenarios for collision avoidance. In particular, we compared and analyzed the collision avoidance performance according to the level of learning and various parameters to ensure that the proposed method displays optimal avoidance performance. The results indicated that the proposed method can determine a safe avoidance path for a given situation. Finally, to compare the performance of the proposed method, we compared the collision avoidance method based on the path–speed control of the OS proposed in this study with the collision avoidance method that controls only the path of the OS (Chun et al., 2021). We observed that the proposed method failed in 6 out of 20 scenarios of the Imazu problem when only the path of the OS was controlled. However, it succeeded in collision avoidance in all the 20 scenarios when both path and speed were controlled simultaneously.

참고문헌 (43)

  1. Bae, J. H., "Heave reduction of payload through crane control based on deep reinforcement learning using dual offshore cranes", J. Comput. Des. Eng, 10(1), pp.414~424 (2022)
  2. Bhopale, P., "Reinforcement learning based obstacle avoidance for autonomous underwater vehicle", J. Mar. Sci. Appl, 18, pp.228~238 (2019)
  4. Cheng, Y., "Concise deep reinforcement learning obstacle avoidance for underactuated unmanned marine vessels", Neurocomputing, 272, pp.63~73 (2018)
  5. Chun, D. H., "Deep reinforcement learningbased collision avoidance for an autonomous ship", Ocean. Eng, 234, pp.109216~ (2021)
  6. Chun, D. H., "Automation of crane control for block lifting based on deep reinforcement learning", J. Comput. Des. Eng, 9(4), pp.1430~1448 (2022)
  7. Cui, Y., "Reinforcement learning ship autopilot: sample efficient and model predictive control-based approach", IEEE International Conference on Intelligent Robots and Systems, pp.2868~2875 (2019)
  8. Everett, M., "Motion planning among dynamic, decisionmaking agents with deep reinforcement learning", IEEE International Conference on Intelligent Robots and Systems, pp.3052~3059 (2018), Institute of Electrical and Electronics Engineers Inc
  9. Standard Equations of Motion for Submarine Simulations, "Gertler, M.", NSRDC, 2510~ (1967)
  10. Guan, W., "Generalized behavior decision-making model for ship collision avoidance via reinforcement learning method", J. Mar. Sci. Eng, 12(2), pp.273~ (2023)
  11. Guan, W., "Generalized behavior decisionmaking model for ship collision avoidance via reinforcement learning method", J. Mar. Sci. Eng, 11(2), pp.273~ (2023)
  12. Ha, J., "Quantitative calculation method of the collision risk for collision avoidance in ship navigation using the CPA and ship domain", J. Comput. Des. Eng, 8(3), pp.894~909 (2021)
  13. He, Y., "Quantitative analysis of COLREGs rules and seamanship for autonomous collision avoidance at open sea", Ocean. Eng, 140, pp.281~291 (2017)
  14. Huang, Y., "Generalized velocity obstacle algorithm for preventing ship collisions at Sea", Ocean. Eng, 173(1), pp.142~156 (2019)
  15. Imazu, H., "Research on Collision Avoidance Manoeuvre", Ph.D, pp.1984
  16. Kijima, K., "Prediction method of ship manoeuvrability in deep and shallow waters, MARSIM and ICSM 90", ~ (1990)
  17. Kim, S. Y., "Prediction of Maneuvering Performance for KVLCC1 & KVLCC2 Based on PMM Data", Proceeding of SIMMAN 2008 (2008)
  18. Kim, J. H., "Collision avoidance based on predictive probability using Kalman filter", Int. J. Nav. Archit. Ocean Eng, 14, pp.100438~ (2022)
  19. Kim, Y. S., "Application of reinforcement learning based on curriculum learning for the pipe auto-routing of ships", J. Comput. Des. Eng, 10(1), pp.318~328 (2023)
  20. Kong, W. R., "Hierarchical reinforcement learning from competitive self-play for dual-aircraft formation air combat", J. Compu. Des. Eng, 10(2), pp.830~859 (2023)
  21. Lefevre, S., "A learning-based framework for velocity control in autonomous driving", IEEE Trans. Autom. Sci. Eng, 13(1), pp.32~42 (2016)
  22. Li, L., "A path planning strategy unified with a COLREGS collision avoidance function based on deep reinforcement learning and artificial potential field", Appl. Ocean Res, 113, pp.102759~ (2021)
  23. Long, P., "Towards optimally decentralized multi-robot collision avoidance via deep reinforcement learning", IEEE Int. Conf. Robot. Autom (2018)
  24. Oh, S. H., "Distributional reinforcement learning with the independent learners for flexible job shop scheduling problem with high variability", J. Comput. Des. Eng, 9(4), pp.1157~1174 (2022)
  25. Sawada, R., "Automatic ship collision avoidance using deep reinforcement learning with LSTM in continuous action spaces", J. Mar. Sci. Technol, 26, pp.509~524 (2021)
  26. Schulman, J., "Proximal Policy Optimization Algorithms", ~ (2017)
  27. Seo, C., "Ship collision avoidance route planning using CRI-based A* algorithm", Int. J. Nav. Archit. Ocean Eng, 15, pp.100551~ (2023)
  28. Shen, H., "Automatic collision avoidance of multiple ships based on deep Q-learning", Appl. Ocean Res, 86, pp.268~288 (2019)
  29. Song, L., "Interference intention classification of moving obstacles used for USV collision avoidance", Int. J. Nav. Archit. Ocean Eng, 14, pp.100459~ (2022)
  30. Sutton, R. S., "Reinforcement Learning : an Introduction" (2018), MIT Press
  31. Wang, C., "Collision avoidance for autonomous ship using deep reinforcement learning and prior-knowledge-based approximate representation", Front. Mar. Sci, 9, pp.1084763~ (2023)
  32. Wang, J., "An energy-efficient hierarchical algorithm of dynamic obstacle avoidance for unmanned surface vehicle", Int. J. Nav. Archit. Ocean Eng, 15, pp.100528~ (2023)
  33. Wu, C., "Deep reinforcement learning with dynamic window aapproach based collision avoidance path planning for maritime autonomous surface ships", Ocean. Eng, 284, pp.115208~ (2023)
  34. Xia, J., "Research on collision avoidance algorithm of unmanned surface vehicle based on deep reinforcement learning", IEEE Sensor. J, 23(11), pp.11262~11273 (2022)
  35. Xie, S., "Model predictive ship collision avoidance based on q-learning beetle swarm antenna search and neural networks", Ocean. Eng, 193, pp.106609~ (2019)
  36. Zhai, P., "Intelligent ship collision avoidance algorithm based on ddqn with prioritized experience replay under COLREGs", J. Mar. Sci. Eng, 10(5), pp.585~ (2022)
  37. Zhang, J., "A distributed anticollision decision support formulation in multi-ship encounter situations under COLREGs", Ocean. Eng, 105, pp.336~348 (2015)
  38. Zhao, L., "Simulation method to support autonomous navigation and installation operation of an offshore support vessel", Ph.D, pp.2019
  39. Zhao, L., "COLREGs-compliant multiship collision avoidance based on deep reinforcement learning", Ocean. Eng, 191, pp.106436~ (2019)
  40. Zhao, Y., "A real-time collision avoidance learning system for unmanned surface vessels", Neurocomputing, 182, pp.255~266 (2016)
  41. Zhao, L., "Control method for path following and collision avoidance of autonomous ships based on deep reinforcement learning", J. Mar. Sci. Technol, 27(4), pp.293~310 (2019)
  42. Zheng, R., "Autonomous collision avoidance system in a multiship environment based on proximal policy optimization method", Ocean. Eng, 272, pp.133779~ (2023)
  43. Zhou, J., "Navigation safety domain and collision risk index for decision support of collision avoidance of USVs", Int. J. Nav. Archit. Ocean Eng, 13, pp.340~350 (2021)

관련 논문AI추천

심층 결정론적 정책 경사법을 이용한 선박 충돌 회피 경로 결정김동함, 남종호, 이성욱, 요시타카 후루카와대한조선학회 · 대한조선학회 논문집2019 · 제56권 제1호A Study on Ship Route Generation with Deep Q Network and Route Following Control김민규, 이형탁한국항해항만학회 · 한국항해항만학회지2023 · 제47권 제2호심층 강화학습 기반 자율운항 CTV의 해상풍력발전단지 내 장애물 회피 시스템김진균, 노재규, 전해명대한임베디드공학회 · 대한임베디드공학회논문지2024 · 제19권 제3호Obstacle Zone by Target 기반 선박 충돌회피 알고리즘 개발에 관한 연구이성욱, 이찬욱대한조선학회 · 대한조선학회 논문집2024 · 제61권 제2호Ship route planning for collision avoidance based on the improved isochrone method노명일, 이혜원, 하지상대한조선학회 · International Journal of Naval Architecture and Ocean Engineering2024 · 제16권 Ship collision avoidance route planning using CRI-based A∗ algorithm권철현, 노유정, 박선영, 서찬희, Abebe Misganaw, Kang Young-Jin대한조선학회 · International Journal of Naval Architecture and Ocean Engineering2023 · 제15권

이 간행물의 다른 논문 (58건)

Interpretable, data-driven models for predicting shaft power, fuel consumption, and speed considering the effects of hull fouling and weather conditions노명일, 김현수pp. 1-20Exploring the performance impact of soft constraint integration on reinforcement learning-based autonomous vessel navigation: Experimental insightsXin Jiang, Jiawen Li, Zhenkai Huang, Ji Huang, Ronghui Lipp. 1-17Hull form optimization of fully parameterized small ships using characteristic curves and deep neural networks노명일, 여인창, 김진혁pp. 1-12Multi-vessel target tracking with camera fusion for unmanned surface vehicles이혜원, 노명일, 조영민, Jeong-Ho Park, 하지상, Nam-Sun Sonpp. 1-22Impact load measurement of small multi-bubble explosions near solid wall정노택, Nyo Me Thet Naing, Seung-Ho Hyunpp. 1-11
원문 보기