Wav2Vec2.0 코드벡터를 활용한 음소 범주 사이 L2 오류 패턴 탐구
Uncovering between-categorical details in L2 pronunciation errors using Wav2Vec2.0 code vectors
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
L2 발음 오류는 모국어 전이 현상에 의해 둘 이상의 표준 음소 범주를 아우르는 특성을 띤다. 이 같은 비범주성을 고려한 평가를 위해서는 음소보다 세분화된 발음 분석 단위가 필요하다. 선행 연구들에서는 그 예시로 음소 사후확률 (phonemic posterior-gram, PPG)을 제안했지만, 사전 정의된 음소 집합과의 연계성으로 소리를 표현하므로 범주적 개여를 온전히 벗어나지 못한다. 이에 본 연구는 자기지도 학습(self-supervised learning, SSL) 모델의 표현 학습이 외부 음소적 규제로부터 자유롭다는 점에 착안해, Wav2Vec2.0 코드벡터를 비범주적 양상 분석의 새로운 단위로 활용한다. 먼저 자질이 L1과 L2 음성을 다르게 인지하는지 확인하기 위해, 내용적으로 통제된 L1 (CMU ARCTIC)과 L2 (L2 ARCTIC) 화자별 데이터에서 사용되는 코드벡터 목록들을 비교했다. 그 후 범주적 정의 내 세부 변이양상들을 조사하고자, 선차적으로 음소 단위 오류탐지를 실시해 분석할 교체오류들을 선별했다. 이때 모델 파인튜닝에 사용될 데이터는 L1 TIMIT이며, 분석 대상은 NIA 037 교육용 한국인의 영어 음성 데이터다. 이후 교체 유형별로 오류 표본들의 음성 프레임에 대응되는 코드벡터열을 추출한 뒤, 우세한 패턴들을 도출했다. 도출된 패턴들은 L1 TIMIT의 코드벡터별 음소 분포 조건부 확률과 현존하는 코드벡터 집합의 군집화 결과를 참조해 최종 해석된다. 분석 결과, 코드벡터는 L2 변별력과 더불어 발음 연속체 속 오류 패턴의 범주 간 위치를 수치화할 능력을 보유하고 있었다. 이른바, 각 오류 패턴을 대표하는 자질들 사이 유클리디안 거리와 패턴 분포는 현존하는 L2 발음 연구에서 기술된 현상들을 반영했다.
Abstract
L2 pronunciation errors, influenced by L1 phonetic transfer, often span the boundaries of two or more canonical phonemes. This phenomenon necessitates moving beyond the categorical judgments by analyzing pronunciation using units more granular than phonemes. While previous studies proposed phonemic posterior-grams (PPG) in this context, these features remain tied to a categorical framework when derived through phonemic association. This study employs Wav2Vec2.0 code vectors as an alternative description unit, granted that the self-supervised learning (SSL) representation is free from external phonemic stipulation. We first evaluate whether code vectors encode L1 and L2 speech differently and use them to identify patterns among categorically identical error samples. To this end, the used inventory in single-speaker data of L1 (CMU ARCTIC) and L2 (L2 ARCTIC) is first compared. Then, substitution errors are selected from our L2 analysis NIA037 data using the model finetuned with TIMIT. For each error type, sub-categorical inspection concerns deriving dominant patterns from corresponding featural sequences and interpreting them using raw code vector clustering results and phoneme-code vector co-occurrence probabilities, both constructed from the L1 reference data TIMIT. The findings revealed that, beyond discerning L2 speech, code vectors can effectively quantify the between-categorical positions of error patterns in the pronunciation spectrum. Specifically, Euclidean distances between pattern vectors reflected phonetically grounded variation phenomena documented in existing L2 literature.
참고문헌 (42)
- "Unsupervised speech intelligibility assessment with utterance level alignment distance between teacher and learner Wav2Vec-2.0 representations" (2023)[URL]
- Bannò, S., "Assessment of L2 oral proficiency using self-supervised speech representation learning", Proceedings of 9th Workshop on Speech and Language Technology in Education (SLaTE), pp.126~130 (2023)
- Boersma, P., "Praat, a system for doing phonetics by computer", Glot International, 5(9), pp.341~345 (2001)
- Chen, S., "WavLM: Large-scale self-supervised pre-training for full stack speech processing", IEEE Journal of Selected Topics in Signal Processing, 16(6), pp.1505~1518 (2022)
- Cho, M., "Perception and production of English vowels by Korean learners: A case study", Studies in Phonetics, Phonology and Mophology, 19(1), pp.155~177 (2013)
- Choi, H., "Asymmetrical production of English diphthongs", eɪ, oʊ, pp.by Korean learners of English, Studies in Linguistics[DOI]
- "Unsupervised cross-lingual representation learning for speech recognition" (2020)[URL]
- (2019)[URL]
- (2024)[URL]
- DARPA TIMIT acoustic-phonetic continous speech corpus CD-ROM, "Garofolo, J. S.", ~, pp.NIST speech disc 1-1.1. NASA STI (1993), Recon technical report n, 93
- Hong, H., "A corpus-based analysis of English segments produced by Korean learners", Journal of Phonetics, 46, pp.52~67 (2014)
- Hsu, W. N., "HuBERT: Self-supervised speech representation learning by masked prediction of hidden units", IEEE, 2021, pp.29, 3451~3460
- "Automatic pronunciation assessment using self-supervised speech representation learning" (2022)[URL]
- Kim, R. E., "A study on English liquids in the rated L2 English speech corpus of Korean learners", Korean Journal of English Language and Linguistics, 19(1), pp.53~75 (2019)
- Kominek, J., "The CMU Arctic speech databases", Proceedings of 5th ISCA Speech Synthesis Workshop, pp.223~224 (2004)
- Koo, H., "A Study of production ability of English liquid", l, r, pp.by Korean learners of English, Korean Education Inquiry[DOI]
- Koo, H. S., "An analysis of English vowels of Korean learners of English and English native speakers", Korean Education Inquiry, 16, pp.1~12 (2001)
- "Learning pronunciation from a foreign language in speech synthesis networks" (2018)[URL]
- Li, X., "Unsupervised discovery of non-native phonetic patterns in L2 English speech for mispronunciation detection and diagnosis", Proceedings of Interspeech 2018, pp.2554~2558 (2018)
- "Deep segmental phonetic posterior-grams based discovery of non-categories in L2 English speech" (2020)[URL]
- Liu, S., "Audio self-supervised learning: A survey", Patterns, 3(12), pp.100616~100616 (2022)
- Mao, S., "Unsupervised discovery of an extended phoneme set in L2 English speech for mispronunciation detection and diagnosis", Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.6244~6248 (2018)
- "Probing self-supervised speech models for phonetic and phonemic information: A case study in aspiration" (2023)[URL]
- McAuliffe, M., "Montreal forced aligner: Trainable text-speech alignment using Kaldi", Proceedings of the 18th Interspeech 2017, pp.498~502 (2017)
- : A comparison between Korean-English bilingual speakers and Korean learners of English, "Linguistic Research", 2013, 323~343
- Park, A. S., "Unsupervised pattern discovery in speech", IEEE Transactions on Audio, Speech, and Language Processing, 16(1), pp.186~197 (2007)
- Park, H., "An experimental study on the lengths of English diphthongs", Speech Science, 8(3), pp.7~14 (2001)
- "What do self-supervised speech models know about words?" (2023)[URL]
- Peng, L., "A study on fine-tuning wav2vec2.0 model for the task of mispronunciation detection and diagnosis", Proceedings of INTERSPEECH 2021, pp.4448~4452 (2021)
- "Self-supervised representation learning for speech using visual grounding and masked language modeling" (2022)[URL]
- Schmidt, A. M., "Cross‐language identification of consonants. Part 1. Korean perception of English", The Journal of the Acoustical Society of America, 99(5), pp.3201~3211 (1996)
- "Wave to syntax: Probing spoken language models for syntax" (2023)[URL]
- Smith, B., "Learner English: A teacher’s guide to interference and other problems" (2001), Cambridge University Press
- Tsukada, K., "A developmental study of English vowel production and perception by native Korean adults and children", Journal of Phonetics, 33(3), pp.263~290 (2005)
- Wang, Y. B., "Toward unsupervised discovery of pronunciation error patterns using universal phoneme posteriorgram for computer-assisted language learning", Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.8232~8236 (2013)
- Wang, Y. B., "Supervised detection and unsupervised discovery of pronunciation error patterns for computer-assisted language learning", IEEE, 2015, pp.23(3), 564~579
- Wu, M., "Transformer based end-to-end mispronunciation detection and diagnosis", Proceedings of INTERSPEECH 2021, pp.3954~3958 (2021)
- Xu, X., "Explore wav2vec 2.0 for mispronunciation detection", Proceedings of INTERSPEECH 2021, pp.4428~4432 (2021)
- Yang, B., "College students’ production and perception of English vowels", English Language Teaching, 22(4), pp.165~184 (2010)
- Yang, B., "A comparative study of relative distances among English front vowels produced by Korean and American speakers", Phonetics and Speech Sciences, 5(4), pp.99~107 (2013)
- Zhao, G., "L2-arctic: A nonnative english speech corpus", Proceedings of IINTERSPEECH 2018, pp.2783~2787 (2018)