음질, 운율, 발음 특징을 이용한 마비말장애 중증도 자동 분류
Automatic severity classification of dysarthria using voice quality, prosody, and pronunciation features
1
조회
0
다운로드
5
인용
0
참고문헌
키워드
초록
This study focuses on the issue of automatic severity classification of dysarthric speakers based on speech intelligibility. Speech intelligibility is a complex measure that is affected by the features of multiple speech dimensions. However, most previous studies are restricted to using features from a single speech dimension. To effectively capture the characteristics of the speech disorder, we extracted features of multiple speech dimensions: voice quality, prosody, and pronunciation. Voice quality consists of jitter, shimmer, Harmonic to Noise Ratio (HNR), number of voice breaks, and degree of voice breaks. Prosody includes speech rate (total duration, speech duration, speaking rate, articulation rate), pitch (F0 mean/std/ min/max/med/25quartile/75 quartile), and rhythm (%V, deltas, Varcos, rPVIs, nPVIs). Pronunciation contains Percentage of Correct Phonemes (Percentage of Correct Consonants/Vowels/Total phonemes) and degree of vowel distortion (Vowel Space Area, Formant Centralized Ratio, Vowel Articulatory Index, F2-Ratio). Experiments were conducted using various feature combinations. The experimental results indicate that using features from all three speech dimensions gives the best result, with a 80.15 F1-score, compared to using features from just one or two speech dimensions. The result implies voice quality, prosody, and pronunciation features should all be considered in automatic severity classification of dysarthria.
Abstract
This study focuses on the issue of automatic severity classification of dysarthric speakers based on speech intelligibility. Speech intelligibility is a complex measure that is affected by the features of multiple speech dimensions. However, most previous studies are restricted to using features from a single speech dimension. To effectively capture the characteristics of the speech disorder, we extracted features of multiple speech dimensions: voice quality, prosody, and pronunciation. Voice quality consists of jitter, shimmer, Harmonic to Noise Ratio (HNR), number of voice breaks, and degree of voice breaks. Prosody includes speech rate (total duration, speech duration, speaking rate, articulation rate), pitch (F0 mean/std/ min/max/med/25quartile/75 quartile), and rhythm (%V, deltas, Varcos, rPVIs, nPVIs). Pronunciation contains Percentage of Correct Phonemes (Percentage of Correct Consonants/Vowels/Total phonemes) and degree of vowel distortion (Vowel Space Area, Formant Centralized Ratio, Vowel Articulatory Index, F2-Ratio). Experiments were conducted using various feature combinations. The experimental results indicate that using features from all three speech dimensions gives the best result, with a 80.15 F1-score, compared to using features from just one or two speech dimensions. The result implies voice quality, prosody, and pronunciation features should all be considered in automatic severity classification of dysarthria.
참고문헌 (27)
- Bhat, C., "Automatic assessment of sentence-level dysarthria intelligibility using BLSTM", IEEE Journal of Selected Topics in Signal Processing, 14(2), pp.322~330 (2020)
- Boersma, P., "Praat, a system for doing phonetics by computer", Glot International, 5(9, pp.10) (2001), 341~345
- Choi, D. L., "Dysarthric speech database for development of QoLT software technology", Proceedings of the 8th International Conference on Language Resources and Evaluation, pp.3378~3381 (2012)
- Clarke, W. M., "Predictive measures of speech proficiency in cerebral palsied speakers", Journal of Communi-cation Disorders, 13(5), pp.385~394 (1980)
- Darley, F. L., "Differential diagnostic patterns of dysarthria", Journal of Speech and Hearing Research, 12(2), pp.246~269 (1969)
- Dellwo, V., "Relationships between speech rate and rhythm", Proceedings of the 15th International Congress of the Phonetic Sciences (2003)
- Hernandez, A., "Prosody-based measures for automatic severity assessment of dysarthric speech", Applied Sciences, 10(19), pp.6999~ (2020)
- Hernandez, A., "Dysarthria detection and severity assessment using rhythm-based metrics", Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH 2020), pp.2897~2901 (2020)
- 홍새미, "이완형 마비말장애 화자의 말속도와 쉼 특성", 한국산학기술학회논문지, 15(5), pp.2930~2936 (2014), 한국산학기술학회[DOI]
- 홍새미, "마비말장애 발화의 청지각적 평가방법 비교: 세부평가와 범주평가", Communication Sciences and Disorders, 23(1), pp.242~253 (2018), 한국언어청각임상학회[DOI]
- Jadoul, Y., "Introducing Parselmouth : A Python interface to Praat", Journal of Phonetics, 71, pp.1~15 (2018)
- Janbakhshi, P., "Pathological speech intelligibility assessment based on the short-time objective intelligibility measure", Proceedings of the International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2019)
- Kadi, K. L., "Discriminative prosodic features to assess the dysarthria severity levels", Proceedings of the World Congress on Engi- neering (2013)
- 강영애, "파킨슨병 환자의 음향 모음 공간 파라미터 비교", 말소리와 음성과학, 2(4), pp.185~192 (2010), 한국음성학회
- Kim, M. J., "Combination of multiple speech dimensions for automatic assessment of dysarthric speech intelligibility", Proceedings of the 13th Annual Conference of the International Speech Communication Association (INTERSPEECH) (2012)
- Kim, M. J., "Automatic intelligibility assessment of dysarthric speech using phonologically-structured sparse linear model", IEEE, 2015, pp.23(4), 694~704
- 김성윤, "경직형 마비말장애의 말 명료도와 모음공간 특성", Communication Sciences and Disorders, 19(3), pp.352~360 (2014), 한국언어청각임상학회[DOI]
- Lansford, K. L., "Vowel acoustics in dysarthria: Speech disorder diagnosis and classification", Journal of Speech, Language, and Hearing Research, 57(1), pp.57~67 (2014)
- 이영미, "마비말장애인의 조음오류 유형에 따른 말명료도 분석", Communication Sciences and Disorders, 17(1), pp.130~142 (2012), 한국언어청각임상학회
- 이은경, "마비말장애 화자의 말 속도에 따른말 명료도 및 말 용인도", 언어치료연구, 21(3), pp.127~144 (2012), 한국언어치료학회[DOI]
- Mairano, P., "La dimensione temporale del parlato", pp.79~100 (2010), EDK
- McFee, B., "Librosa: Audio and music signal analysis in Python", Proceedings of the 14th Python in Science Conference, pp.18~25 (2015)
- Narendra, N. P., "Dysarthric Speech Classification Using Glottal Features Computed from Non-words, Words and Sentences", Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH 2020), pp.3403~3407 (2018)
- Narendra, N. P., "Automatic assessment of intelligibility in speakers with dysarthria from coded telephone speech using glottal features", Computer Speech & Language, 65, pp.101117~ (2021)
- 서인효, "연결발화에서 마비말화자의 음질 특성", 말소리와 음성과학, 5(4), pp.33~41 (2013), 한국음성학회[DOI]
- Seo I. H, "Acoustic measures of voice quality and phonation types across speech conditions in dysarthria", Doctor, pp.2014
- Whitehill, T. L., "Speech errors in Cantonese speaking adults with cerebral palsy", Clinical Linguistics &Phonetics, 14(2), pp.111~130 (2000)