See-through형 HMD의 마커리스 얼굴 모션 캡처 기술의 성능 비교에 관한 탐색적 연구

Abstract

Facial motion capture technology is gaining attention as a key technology in the metaverse and augmented reality environments,as it enables real-time facial reproduction and communication in virtual spaces, as well as the expression of emotions. However,when wearing the HMD, one of the main equipments used in the metaverse, there is a possibility that motion capture performancemay be degraded, depending on the see-through type, as it partially obscures the face. Generally, see-through HMDs are classifiedinto two types: optical see-through (OST) and video see-through (VST). The objective of this pilot study is to quantitatively compareand analyze the capture recognition rate, which represents the performance of markerless facial motion capture technologies, underfacial occlusion conditions associated with the two see-through types. (Representative markerless facial motion capture technologiessuch as ARKit and MediaPipe are used in this paper.) To this end, we used OST and VST HMD to measure the estimation accuracyof various blend shape features. The pilot study results showed a tendency toward higher facial motion capture accuracy for theOST type compared to the VST type in both ARKit and MediaPipe, when comparing the see-through types of HMD. Additionally,MediaPipe scored significantly higher accuracy than ARKit, regardless of the HMD type in a comparison of facial motion capturetechnologies.