심층 결정론적 정책 경사법을 이용한 선박 충돌 회피 경로 결정
Determination of Ship Collision Avoidance Path using Deep Deterministic Policy Gradient Algorithm
1
조회
0
다운로드
4
인용
0
참고문헌
키워드
초록
The stability, reliability and efficiency of a smart ship are important issues as the interest in an autonomous ship has recently been high. An automatic collision avoidance system is an essential function of an autonomous ship. This system detects the possibility of collision and automatically takes avoidance actions in consideration of economy and safety. In order to construct an automatic collision avoidance system using reinforcement learning, in this work, the sequential decision problem of ship collision is mathematically formulated through a Markov Decision Process (MDP). A reinforcement learning environment is constructed based on the ship maneuvering equations, and then the three key components (state, action, and reward) of MDP are defined. The state uses parameters of the relationship between own-ship and target-ship, the action is the vertical distance away from the target course, and the reward is defined as a function considering safety and economics. In order to solve the sequential decision problem, the Deep Deterministic Policy Gradient (DDPG) algorithm which can express continuous action space and search an optimal action policy is utilized. The collision avoidance system is then tested assuming the 90°intersection encounter situation and yields a satisfactory result.
Abstract
The stability, reliability and efficiency of a smart ship are important issues as the interest in an autonomous ship has recently been high. An automatic collision avoidance system is an essential function of an autonomous ship. This system detects the possibility of collision and automatically takes avoidance actions in consideration of economy and safety. In order to construct an automatic collision avoidance system using reinforcement learning, in this work, the sequential decision problem of ship collision is mathematically formulated through a Markov Decision Process (MDP). A reinforcement learning environment is constructed based on the ship maneuvering equations, and then the three key components (state, action, and reward) of MDP are defined. The state uses parameters of the relationship between own-ship and target-ship, the action is the vertical distance away from the target course, and the reward is defined as a function considering safety and economics. In order to solve the sequential decision problem, the Deep Deterministic Policy Gradient (DDPG) algorithm which can express continuous action space and search an optimal action policy is utilized. The collision avoidance system is then tested assuming the 90°intersection encounter situation and yields a satisfactory result.
참고문헌 (21)
- Bergstra, J., "Random search for hyper-parameter optimization", Journal of Machine Learning Research, 13, pp.281~305 (2012)
- Furukawa, Y., "Development of automatic course modification system using fuzzy inference", International Federation of Automatic Control Proceedings, 37(10), pp.77~82 (2004)
- Jeong, J.S., "Korean e-Navigation goal and government plan", Telecommunications Technology Association Journal, 159, pp.20~27 (2015)
- Kijima, K., "Development of collision avoidance algorithm using fuzzy inference", Proceedings of ISOPE Pacific, 2002, 123~130
- Kijima, K., "Automatic collision avoidance system using the concept of blocking area", International Federation of Automatic Control Proceedings, 36(21), pp.223~228 (2003)
- Kijima, K., "On the practical prediction method for ship manoeuvring characteristics", Transaction of the West-Japan Society of Naval Architects, 105, pp.21~31 (2003)
- 김동진, "국내 해양선박사고의 인적 오류의 요인 평가", 대한인간공학회지, 30(1), pp.87~98 (2011), 대한인간공학회
- (2017)[URL]
- Kose, K., "A new collision-avoidance-supporting-system and its application to coastal-cargo-ship “SHOYO MARU”", IFAC Proceeding, 31, pp.263~268 (1998)
- Lee, H.J., "Development of collision avoidance system by using expert system and search algorithm", International Shipbuilding Progress, 48, pp.197~212 (2001)
- Lee, W.W., "Reinforcement Learning with Python and Keras" (2017), Wikibook
- Li, Y., "Deep Reinforcement Learning: An Overview", arXiv preprint arXiv:1701.07274, ~ (2017)
- Lillicrap, T.P., "Continuous control with deep reinforcement learning", International Conference on Learning Representations (2016)
- Mnih, V., "Playing Atari with Deep Reinforcement Learning", Neural Information Processing Systems (2013)
- Mori, S., "Note of Ship Form Design(24)", FUNE-NO-KAGAKU, 48, pp.40~49 (1995)
- Ota, D., "Development of automatic collision avoidance system for ships using reinforcement learning", Proceedings of 7th PAAMES and AMEC2016 (2016)
- 심우성, "e-navigation 국제 표준화 현황 및 국내 대응방안 고찰", 한국정보통신학회논문지, 14(5), pp.1057~1063 (2010), 한국정보통신학회
- Silver, D., "Mastering the game of go with deep neural networks and tree search", Nature, 529(7587), pp.484~489 (2016)
- Silver, D., "Deterministic policy gradient algorithms", International Conference on Machine Learning, 32, pp.387~395 (2014)
- 손남선, "가변공간 탐색법을 이용한 다중선박의 충돌회피 알고리즘에 관한 연구", 한국해양환경•에너지학회지, 12(1), pp.15~22 (2009), 한국해양환경·에너지학회
- Planning research for development of core technologies for smart ship, "Van, S.H.", KORDI, ~, pp.Report No. UCPM0147A-42-7 (2007)