음성 명령 및 객체 인식 기반 로봇 작업 계획
Robot Task Planning Based on Voice Command and Object Detection
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Recently, studies on human-robot interaction (HRI) have been actively conducted in the field of robotics. For effective interaction, a robot should be equipped with an ‘eye’ for environmental perception, localization capabilities, and language skills to communicate with people. In this paper, we integrate vision-based object detection and speech recognition-based command delivery technologies to improve operational efficiency in line with user intent, and explore the implementation of a robot system for recycling. Our system utilizes an OpenMANIPULATOR-X robot arm and uses voice commands recognized by Python’s SpeechRecognition library to issue control commands to the robot, where the speech recognition module classifies tasks by extracting keywords from the text converted from voice commands. A vision system equipped with an Intel D435i camera then identifies the status of transparent cups using YOLO and AlexNet models. Based on this information and the commanded task, the robot executes an appropriate action. Experimental results demonstrate our system’s capability to effectively automate recycling, showing success rates of 91% and 95% for voice command recognition and object detection respectively.
Abstract
Recently, studies on human-robot interaction (HRI) have been actively conducted in the field of robotics. For effective interaction, a robot should be equipped with an ‘eye’ for environmental perception, localization capabilities, and language skills to communicate with people. In this paper, we integrate vision-based object detection and speech recognition-based command delivery technologies to improve operational efficiency in line with user intent, and explore the implementation of a robot system for recycling. Our system utilizes an OpenMANIPULATOR-X robot arm and uses voice commands recognized by Python’s SpeechRecognition library to issue control commands to the robot, where the speech recognition module classifies tasks by extracting keywords from the text converted from voice commands. A vision system equipped with an Intel D435i camera then identifies the status of transparent cups using YOLO and AlexNet models. Based on this information and the commanded task, the robot executes an appropriate action. Experimental results demonstrate our system’s capability to effectively automate recycling, showing success rates of 91% and 95% for voice command recognition and object detection respectively.
참고문헌 (10)
- L. Kunze, "Artificial intelligence for long-term robot autonomy : A survey", IEEE Robotics and Automation Letters, 3(4), pp.4023~4030 (2018)[DOI]
- M. Cho, "An analysis of quality attributes and service satisfaction for artificial intelligence-based guide robot", Journal of Korea Robotics Society, 18(2), pp.216~224 (2023)[DOI]
- B. Fang, "Artificial intelligence for waste management in smart cities : A review", Environmental Chemistry Letters, 21(4), pp.1959~1989 (2023)[DOI]
- A. Bonarini, "Communication in human-robot interaction", Current Robotics Reports, 1(4), pp.279~285 (2020)[DOI]
- H. Jeon, "Deep reinforcement learningbased cooperative robot using facial feedback", Journal of Korea Robotics Society, 17(3), pp.264~272 (2022)[DOI]
- Z. Shi, "Audio-visual sound source localization and tracking based on mobile robot for the cocktail party problem", Applied Sciences, 13(10), pp.1~14 (2023)[DOI]
- P. Haghighatkhah, "Better hit the nail on the head than beat around the bush : Removing protected attributes with a single projection", 2022 Conference on Empirical Methods in Natural Language Processing, pp.8395~8416 (2022)
- A. Martin, "Learning natural language generation with truncated reinforcement learning", 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp.12~37 (2022)
- A. Mukanova, "LLM-powered natural language text processing for ontology enrichment", Applied Sciences, 14(13), pp.1~14 (2024)[DOI]
- H. Z. Ting, "Kinematic analysis for trajectory planning of open-source 4-DoF robot arm", International Journal of Advanced Computer Science and Applications, 12(6), pp.768~775 (2021)[DOI]