다중 에이전트 강화학습을 이용한다중 AGV의 충돌 회피 경로 제어
Collision Avoidance Path Control of Multi-AGV Using Multi-Agent Reinforcement Learning
3
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
산업 응용 분야에서 AGV는 공장이나 창고와 같은 대규모 산업 시설의 무거운 자재를 운송하기 위해 자주 사용된다. 특히, 주문처리 센터에서는자동화가 가능하여 유용성이 극대화된다. 이러한 주문처리 센터와 같은 창고에서 생산성을 높이기 위해서는 AGV들의 정교한 운반 경로 제어가요구된다. 본 논문에서는 대중적인 협력 MARL 알고리즘인 QMIX에 적용될 수 있는 구조를 제안한다. 성능은 두 종류의 주문처리 센터 레이아웃에서세 가지의 메트릭으로 측정하였으며, 결과는 기존 QMIX의 성능과 비교하여 제시된다. 추가적으로, AGV들의 행동 패턴에 대한 가시적인 분석을위해 훈련된 AGV들의 운반 경로를 시각화한 히트맵을 제공한다.
Abstract
AGVs are often used in industrial applications to transport heavy materials around a large industrial building, such as factories orwarehouses. In particular, in fulfillment centers their usefulness is maximized for automation. To increase productivity in warehousessuch as fulfillment centers, sophisticated path planning of AGVs is required. We propose a scheme that can be applied to QMIX, a popularcooperative MARL algorithm. The performance was measured with three metrics in several fulfillment center layouts, and the results arepresented through comparison with the performance of the existing QMIX. Additionally, we visualize the transport paths of trained AGVsfor a visible analysis of the behavior patterns of the AGVs as heat maps.
참고문헌 (22)
- L. Buşoniu, "Innovations in Multi-agent Systems and Applications-1", pp.183~221 (2010)
- J. Cui, "Multi-agent reinforcement learning-based resource allocation for UAV networks", IEEE Transactions on Wireless Communications, 19(2), pp.729~743 (2019)
- X. Li, "A cooperative multi-agent reinforcement learning framework for resource balancing in complex logistics network", Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems (2019)
- X. Li, "A multi-agent reinforcement learning routing protocol for underwater optical sensor networks", Proceedings of IEEE International Conference on Communications (2019)
- F. A. Oliehoek, "Optimal and approximate Q-value functions for decentralized POMDPs", Journal of Artificial Intelligence Research, 32, pp.289~353 (2008)
- F. A. Oliehoek, "A concise introduction to decentralized POMDPs, SpringerBriefs in Intelligent Systems" (2016), Springer
- J. J. Enright, "Optimization and coordinated autonomy in mobile fulfillment systems", Proceedings of the AAAI Workshop on Automated Action Planning for Autonomous Mobile Robots, pp.33~38 (2011)
- 배정연, "A Heuristic for a Heterogeneous Automated Guided Vehicle Routing Problem", International Journal of Precision Engineering and Manufacturing, 18(6), pp.795~801 (2017), 한국정밀공학회
- Z. Han, "Multi-AGV path planning with double-path constraints by using an improved genetic algorithm", PloS one, 12(7) (2017)
- Y. Lian, "Improved A* multi-AGV path planning algorithm based on grid-shaped network", 2019 Chinese Control Conference (2019)
- R. Kamoshida, "Acquisition of automated guided vehicle route planning policy using deep reinforcement learning", IEEE International Conference on Advanced Logistics and Transport (ICALT) (2017)
- Y. Yang, "Multi-robot path planning based on a deep reinforcement learning DQN algorithm", CAAI Transactions on Intelligence Technology, 5(3), pp.177~183 (2020)
- C. J. C. H. Watkins, "Q-learning", Machine Learning, 8, pp.279~292 (1992)
- V. Mnih, "Human-level control through deep reinforcement learning", Nature, 518(7540), pp.529~533 (2015)
- M. Tan, "Multi-agent reinforcement learning: Independent vs. cooperative agents", Proceedings of the Tenth International Conference on Machine Learning, pp.330~337 (1993)
- P. Sunehag, "Value-decomposition networks for cooperative multi-agent learning based on team reward", Proceedings of 17th International Conference on Autonomous Agents and Multiagent Systems (2018)
- T. Rashid, "QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learning", Proceedings of the 35th International Conference on Machine Learning (2018)
- O. Vinyals, "Starcraft II: A new challenge for reinforcement learning", arXiv preprint arXiv:1708.04782, ~ (2017)
- D. Ye, "Mastering complex control in moba games with deep reinforcement learning", Proceedings of the AAAI Conference on Artificial Intelligence, pp.6672~6679 (2020)
- S. Huang, "A closer look at invalid action masking in policy gradient algorithms", arXiv preprint arXiv:2006.14171, ~ (2020)
- J. Chung, "Empirical evaluation of gated recurrent neural networks on sequence modeling", NIPS 2014 Workshop on Deep Learning (2014)
- D. Ha, "Hypernetworks", Proceedings of the International Conference on Learning Representations (ICLR) (2017)