수어 번역을 위한 3차원 컨볼루션 비전 트랜스포머
Three-Dimensional Convolutional Vision Transformer for Sign Language Translation
0
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
한국에서 청각장애인은 지체장애인에 이어 두 번째로 많은 등록 장애인 그룹이다. 하지만 수어 기계 번역은 시장 성장성이 작고, 엄밀하게주석처리가 된 데이터 세트가 부족해 발전 속도가 더디다. 한편, 최근 컴퓨터 비전과 패턴 인식 분야에서 트랜스포머를 사용한 모델이 많이 제안되고있는데, 트랜스포머를 이용한 모델은 동작 인식, 비디오 분류 등의 분야에서 높은 성능을 보여오고 있다. 이에 따라 수어 기계 번역 분야에서도트랜스포머를 도입하여 성능을 개선하려는 시도들이 제안되고 있다. 본 논문에서는 수어 번역을 위한 인식 부분을 트랜스포머와 3D-CNN을 융합한3D-CvT를 제안한다. 또, PHOENIX-Wether-2014T [1]
Abstract
In the Republic of Korea, people with hearing impairments are the second-largest demographic within the registered disabilitycommunity, following those with physical disabilities. Despite this demographic significance, research on sign language translationtechnology is limited due to several reasons including the limited market size and the lack of adequately annotated datasets. Despitethe difficulties, a few researchers continue to improve the performacne of sign language translation technologies by employing the recentadvance of deep learning, for example, the transformer architecture, as the transformer-based models have demonstrated noteworthyperformance in tasks such as action recognition and video classification. This study focuses on enhancing the recognition performanceof sign language translation by combining transformers with 3D-CNN. Through experimental evaluations using the PHOENIX-Wether-2014Tdataset [1], we show that the proposed model exhibits comparable performance to existing models in terms of Floating Point OperationsPer Second (FLOPs).
참고문헌 (21)
- N. C. Camgoz, "Neural sign language translation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)
- N. C. Camgoz, "Sign language transformers : Joint end-to-end sign language recognition and translation", pp.2020, Proceedings of the IEEE
- H. Zhou, "Improving sign language translation with monolingual data by sign back-translation", Proceedings of the IEEE, 2021, ~
- Y. Chen, "A simple multi-modality transfer learning baseline for sign language translation", Proceedings of the IEEE, 2022, ~
- Y. Chen, "Two-stream network for sign language recognition and translation", Advances in Neural Information Processing Systems, 35, pp.17043~17056 (2022)
- H. Wu, "Cvt : Introducing convolutions to vision transformers", Proceedings of the IEEE, 2021, ~
- A. Graves, "Connectionist temporal classification : Labelling unsegmented sequence data with recurrent neural networks", Proceedings of the 23rd International Conference on Machine Learning (2006)
- S. Xie, "Rethinking spatiotemporal feature learning : Speed-accuracy tradeoffs in video classification", Proceedings of the European Conference on Computer Vision(ECCV) (2018)
- D. Li, "Word-level deep sign language recognition from video : A new large-scale dataset and methods comparison", Proceedings of the IEEE, 2020, ~
- Y. Liu, "Multilingual denoising pre-training for neural machine translation", Transactions of the Association for Computational Linguistics, 8, pp.726~742 (2020)
- K. Papineni, "Bleu : a method for automatic evaluation of machine translation", Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (2002)
- A. J. Piergiovanni, "Rethinking video vits : Sparse video tubes for joint image and video learning", Proceedings of the IEEE, 2023, ~
- G. Bertasius, "space-time attention all you need for video understanding?", ICML, 2(3) (2021)
- A. Arnab, "Vivit : A video vision transformer", Proceedings of the IEEE, 2021, ~
- J. Guo, "Cmt : Convolutional neural networks meet vision transformers", Proceedings of the IEEE, 2022, ~
- J. Deng, "Imagenet : A large-scale hierarchical image database", 2009 IEEE Conference on Computer Vision and Pattern Recognition (2009), IEEE