눈동자 추적 기반 입력 및 딥러닝 기반 음성 합성을 적용한 루게릭 환자 의사소통 지원 시스템
Communication Support System for ALS Patient Based on Text Input Interface Using Eye Tracking and Deep Learning Based Sound Synthesis
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Accidents or disease can lead to acquired voice dysphonia. In this case, we propose a new input interface based on eye movements to facilitate communication for patients. Unlike the existing method that presents the English alphabet as it is, we reorganized the layout of the alphabet to support the Korean alphabet and designed it so that patients can enter words by themselves using only eye movements, gaze, and blinking. The proposed interface not only reduces fatigue by minimizing eye movements, but also allows for easy and quick input through an intuitive arrangement. For natural communication, we also implemented a system that allows patients who are unable to speak to communicate with their own voice. The system works by tracking eye movements to record what the patient is trying to say, then using Glow-TTS and Multi-band MelGAN to reconstruct their own voice using the learned voice to output sound.
Abstract
Accidents or disease can lead to acquired voice dysphonia. In this case, we propose a new input interface based on eye movements to facilitate communication for patients. Unlike the existing method that presents the English alphabet as it is, we reorganized the layout of the alphabet to support the Korean alphabet and designed it so that patients can enter words by themselves using only eye movements, gaze, and blinking. The proposed interface not only reduces fatigue by minimizing eye movements, but also allows for easy and quick input through an intuitive arrangement. For natural communication, we also implemented a system that allows patients who are unable to speak to communicate with their own voice. The system works by tracking eye movements to record what the patient is trying to say, then using Glow-TTS and Multi-band MelGAN to reconstruct their own voice using the learned voice to output sound.
참고문헌 (14)
- "What is Lou Gehrig's disease?" (2018)[URL]
- Sung-hyeon Jo, "Introduction to Eye Tracking Technology", The Magazine of the IEEE, 45(8), pp.23~32 (2018)
- Kim-JongHa, "Eye-tracking and Perception", Review of Architecture and Building Science, 58(9), pp.21~26 (2014)
- Byoung-jin Kim, "Reliability Measurement Technique of The Eye Tracking System Using Gaze Point Information", Journal of Digital Contents Society, 17(5), pp.367~373 (2016)[DOI]
- Eunseo Jeon, "Mobile Platform Communication System based on Eye Movement Detection for Amyotrophic Lateral Sclerosis Patients", Journal of the Korean Society of Information Science, pp.1701~1703 (2022)
- Jong-Hyun Kim, "Efficient Mobile Writing System with Korean Input Interface Based on Face Recognition", Journal of The Korea Society of Computer and Information, 25(6), pp.49~56 (2020)[DOI]
- Hyungseop Son, "A Study on Efficient Korean Voice Synthesis Model for The Visually handicapped", Master, pp.2022
- Lee Hee-man, "Voice synthesis engine for TTS application Speech Synthesis Engine for TTS", The Journal of Korean Institute of Communications and Information Sciences, 23(6), pp.1443~1453 (1998)
- "Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to –Speech"[URL]
- "Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search"[URL]
- "MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis"[URL]
- Su-hyeon Oh, "Effective Learning Rate Scheduling for Speaker-Adaptive TTS", Korean Society of Electronics Engineers Conference, pp.1173~1175 (2022)
- T. Okamoto, "Tacotron-Based Acoustic Model Using Phoneme Alignment for Practical Neural Text-to-Speech Systems", IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Singapore, 2019, pp.214~221 (2019)
- Ahn, H, "Convolutional Neural Networks Using Skip Connections with Layer Groups for Super-Resolution Image Reconstruction Based on Deep Learning", Applied Sciences, 10(6), pp.1959~1968 (2020)[DOI]