Korean Sign Language Recognition Using LSTM and Video Datasets

Abstract

Deaf individuals primarily use sign language, which consists of hand gestures and body movements, as their main means of communication. It is difficult for non-disabled people to understand the visual form of sign language, and sign language recognition technology is required to facilitate communication. However, unlike spoken languages used by the general population, sign languages take a visual form and can be recognized through video or image data before being translated into other languages. In this study, we proposed a Korean sign language recognition and sentence conversion system based on long short-term memory (LSTM) using video datasets. To build a Korean sign language dataset, we automatically collected and preprocessed video data of sign language gestures, which were then used as input for the LSTM model. LSTM has strengths in processing sequential data and can effectively recognize the sequences and patterns of sign language gestures. The experimental results measured the accuracy of the model and analyzed its performance based on sign language gesture recognition and display. This study confirmed the effectiveness of the proposed approach and is expected to contribute to the advancement of Korean sign language recognition technology.