A Study on Hand Gesture Classification for Mobile Game

Abstract

Before standalone mobile VR (Virtual Reality) were actively developed, there was a movement to make smartphones into HMD (Head-Mounted Display). Google's cardboard and Samsung's GearVR are examples. However, smartphone-based VR has shifted to AR (Augmented Reality) due to the release of standalone VR and the limitations of mobile devices. As examples of using the AR, we can wear accessories even without trying them on, and we can be also guided by arrows on the map when finding a place that we have never visited before. Over the past decade, mobile devices' performance has improved with the development of mobile VR and AR. As a result, the combination of hand gesture classification beyond hand tracking can be utilized for interaction with VR and AR content. In this paper, we describe how to use the hand gesture for the user interaction for the mobile VR and AR. When realizing the gesture classification, we used Manomotion to implement the gesture estimation and 2D/ 3D skeleton was tracked in real time. In this paper, we suggest the methods to classify the hand gesture and verify them through successfully recognizing the gesture and using them at a practical game. We also measure the performance, and the suggested algorithm requires only 0.425 msec to recognize the gesture. This interface can be also used to improve the convenience of users other than games for mobile VR and AR.