MediaPipe Face Mesh를 이용한 얼굴 제스처 기반의 사용자 인터페이스의 성능 개선
Performance Improvement of Facial Gesture-based User Interface Using MediaPipe Face Mesh
0
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
본 논문은 MediaPipe Face Mesh 모델을 이용해 일련의 프레임 시퀀스에서 얼굴 제스처를 인식해 해당 사용자이벤트를 처리하는 얼굴 제스처 기반의 사용자 인터페이스 선행 연구의 성능 개선 방안을 제안함에 그 목적이 있다. 선행 연구는 MediaPipe Face Mesh 모델에서 선택한 7개의 랜드마크의 3차원 좌표들로부터 얼굴 제스처를 인식해해당 사용자 이벤트를 발생시키고 이에 대응하는 명령을 수행하는 것이 특징이다. 제안된 방법은 그 과정에서 커서 위치들에 적응형 이동 평균 처리를 적용해 미세 떨림을 완화함으로써 커서 안정화를 도모하고, 양안 동시 개폐 시에 양안의 일시적 개폐 불일치를 차단해 그 성능을 개선하였다. 제안된 얼굴 제스처 인터페이스의 사용성 평가 결과, 얼굴 제스처의 평균 인식률이 선행 연구에서 95.8%였던 것에 비해 98.7%로 상향되는 것이 확인되었다.
Abstract
The purpose of this paper is to propose a method to improve the performance of the previous research is characterized by recognizing facial gestures from the 3D coordinates of seven landmarks selected from the MediaPipe Face Mesh model, generating corresponding user events, and executing corresponding commands. The proposed method applied adaptive moving average processing to the cursor positions in the process to stabilize the cursor by alleviating microtremor, and improved performance by blocking temporary opening/closing discrepancies between both eyes when opening and closing both eyes simultaneously. As a result of the usability evaluation of the proposed facial gesture interface, it was confirmed that the average recognition rate of facial gestures was increased to 98.7% compared to 95.8% in the previous research.
참고문헌 (28)
- F. Karray, "Human-Computer Interaction : Overview on State of the Art", International Journal on Smart Sensing and Intelligent Systems, 1(1), pp.137~159 (2008)
- T. H. Tsai, "Design of Hand Gesture Recognition System for Human-computer Interaction", Multimedia Tools and Applications, 79(9-10), pp.5989~6007 (2020)
- 김규민, "딥러닝 기반 실시간 손 제스처 인식", 멀티미디어학회논문지, 22(4), pp.424~431 (2019), 한국멀티미디어학회[DOI]
- B. Kumar, "Handbook of Research on AI-Based Technologies and Applications in the Era of the Metaverse", pp.542~358 (2023)
- Q. Gao, "Dynamic Hand Gesture Recognition Based on 3D Hand Pose Estimation for Human-robot Interaction", IEEE Sensors Journal, pp.17421~17430 (2021)
- H. Kaur, "A Review : Study of Various Techniques of Hand Gesture Recognition", Proceedings of 2016 IEEE 1st International Conference on Power Electronics, Intelligent Control and Energy Systems, pp.1~5 (2016)
- Y. Li, "Gesture Interaction in Virtual Reality", Virtual Reality and Intelligent Hardware, pp.84~112 (2019)
- C. A. Cruz, "Sequential Eyelid Gestures for User Interfaces in VR", Proceedings of 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (2023)
- A. Shimada, "Hand Gesture Based TV Control System— Towards Both User-Machine-friendly Gesture Applications", Proceedings of The 19th Korea-Japan Joint Workshop on Frontiers of Computer Vision, pp.121~126 (2013)
- H. Stern, "A Real-Time Hand Gesture Interface for Medical Visualization Applications", Applications of Soft Computing, 36, pp.153~162 (2006), Springer
- G. Pala, "Machine Learning-based Hand Sign Recognition", Proceedings of 2021 International Conference on Artificial Intelligence and Smart Systems, pp.356~363 (2021)
- M. Iskandar, "Artificial Intelligence-based Human Gesture Tracking Control Techniques of Tello EDU Quadrotor Drone", Proceedings of International Conference on Green Energy, Computing and Intelligent Technology (2023)
- "MANO"[URL]
- N. Qian, "HTML : A Parametric Hand Texture Model for 3D Hand Reconstruction and Personalization", Proceedings of the European Conference on Computer Vision, pp.54~71 (2020)
- S. An, "Fast Hand: Fast Monocular Hand Pose Estimation on Embedded Systems", Journal of Systems Architecture, 122 (2022)
- Z. Fan, "Learning to Disambiguate Strongly Interacting Hands via Probabilistic Per-pixel Part Segmentation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.1~10 (2021)
- 장명수, "플로팅 홀로그램 캐릭터 조작을 위한 사용자 제스처 인식 시스템 구현", 한국인터넷방송통신학회 논문지, 19(2), pp.143~149 (2019), 한국인터넷방송통신학회[DOI]
- L. Chen, "MVHM : A Large-Scale multi-View Hand Mesh Benchmark for Accurate 3D Hand Pose Estimation", Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition, pp.836~845 (2021)
- "Google MediaPipe"[URL]
- 허경용, "손 표현 인식을 위한 계층적 손 자세 모델", 한국정보통신학회논문지, 25(10), pp.1323~1329 (2021), 한국정보통신학회[DOI]
- 허경용, "가상 칠판을 위한 손 표현 인식", 한국정보통신학회논문지, 25(12), pp.1770~1776 (2021), 한국정보통신학회[DOI]
- 송복득, "사용자 손 제스처 인식 기반 입체 영상 제어 시스템 설계 및 구현", 한국정보통신학회논문지, 26(3), pp.396~402 (2022), 한국정보통신학회[DOI]
- 송락빈, "MediaPipe Hands 모델 기반의 손 제스처 인식을 이용한 사용자 인터페이스", 멀티미디어학회논문지, 26(2), pp.103~115 (2023), 한국멀티미디어학회
- J. Prameela, "Mouse Handling Using Facial Gesture", International Research Journal of Modernization in Engineering Technology and Science, 04(5), pp.468~475 (2022)
- S. Sreeni, "Mouse Cursor Control Using Facial Movements-An HCI Application", International Journal of Techno-Engineering, 15(2), pp.270~274 (2023)
- Z. Sharifisoraki, "A Novel Face Recognition Using Specific Values from Deep Neural Network-based Landmarks", Proceedings of 2023 IEEE International Conference on Consumer Electronics (2023)
- S. Thino, "Developing a Program to Detect Face Direction and the State of Partially Closed Eyes", Master, pp.2023
- J. Mok, "Facial Gesture-based User Interface Using MediaPipe Face Mesh", Proceedings of 2023 Summer Annual Conference of The Institute of Electronics and Information Engineers, pp.1407~1411 (2023)