A Comparison of L1 and L2 Speech Phonetic Posteriorgrams for Applications in Pronunciation Training
0
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
This study investigated the pronunciation differences in L1 and L2 English speech via the phonetic posteriorgram (PPG) vectors obtained from acoustic model trained for automatic speech recognition (ASR). PPG is the frame-wise phone-classification result in conventional ASR systems, which has been widely applied to Computer-Assisted Pronunciation Training (CAPT). More recently, PPG has been applied to foreign accent conversion, a field of voice conversion research that can be applied to user-adaptable pronunciation training. This study was aimed at investigating the L1-L2 PPG pattern differences between the speech data previously used for these research endeavors. PPG vectors of the 18 native and 24 non-native English speakers were extracted using a deep neural network (DNN) based ASR system. When analyzed in phone-level, results showed that L1 PPGs were more concentrated on correct phone dimension than L2 PPG, which appeared more dispersed across dimensions. The result is in line with the previous PPG-based CAPT research that identified distinct L2 patterns in English speech. The findings are further discussed in the context application to pronunciation training.
Abstract
This study investigated the pronunciation differences in L1 and L2 English speech via the phonetic posteriorgram (PPG) vectors obtained from acoustic model trained for automatic speech recognition (ASR). PPG is the frame-wise phone-classification result in conventional ASR systems, which has been widely applied to Computer-Assisted Pronunciation Training (CAPT). More recently, PPG has been applied to foreign accent conversion, a field of voice conversion research that can be applied to user-adaptable pronunciation training. This study was aimed at investigating the L1-L2 PPG pattern differences between the speech data previously used for these research endeavors. PPG vectors of the 18 native and 24 non-native English speakers were extracted using a deep neural network (DNN) based ASR system. When analyzed in phone-level, results showed that L1 PPGs were more concentrated on correct phone dimension than L2 PPG, which appeared more dispersed across dimensions. The result is in line with the previous PPG-based CAPT research that identified distinct L2 patterns in English speech. The findings are further discussed in the context application to pronunciation training.
참고문헌 (30)
- Aryal, S., "Can voice conversion be used to reduce non-native accents?", International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2014, pp.7879~7883 (2014)
- Berndt, D., "Using dynamic time warping to find patterns in time series", Workshopon Knowledge Knowledge Discovery in Databases, 398, pp.359~ (1994)
- Ding, S., "Golden speaker builder – An interactive tool for pronunciation training", Speech Communication, 115, pp.51~66 (2019)
- Felps, D., "Foreign accent conversion in computer assisted pronunciation training", Speech communication, 51(10), pp.920~932 (2009)
- Gupta, C., "Automatic Pronunciation Evaluation of Singing", Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), 2018, pp.1507~1511 (2018)
- Han, J., "Data mining", pp.39~82 (2012), Elsevier
- 홍승아, "“아는 게 많을수록 외국어 학습에 좋을까?” -모음 음소 목록의 크기와 새로운 모음의 지각을 중심으로-", 외국어교육연구, 33(4), pp.281~299 (2019), 외국어교육연구소[DOI]
- Kominek, J., "The CMU Arctic Databases for Speech Synthesis", Proc. ISCA Workshop on Speech Synthesis, pp.223~224 (2004)
- Lee, A., "Mispronunciation detection via dynamic time warping on deep belief network-based posteriorgrams", International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2013, pp.8227~8231 (2013)
- Li, X., "Unsupervised discovery of non-native phonetic patterns in l2 english speech for mispronunciation detection and diagnosis", Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), 2018, pp.2554~2558 (2018)
- Liu, S., "Jointly Trained Conversion Model and WaveNet Vocoder for Non-Parallel Voice Conversion Using Mel-Spectrograms and Phonetic Posteriorgrams", Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), 2019, pp.714~718 (2019)
- Liu, S., "Voice Conversion Across Arbitrary Speakers Based on a Single Target-Speaker Utterance", Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), 2018, pp.496~500 (2018)
- Ma, C., "A comparative study on system combination schemes for LVCSR", International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2010, pp.4394~4397 (2010)
- Mao, S., "Unsupervised discovery of an extended phoneme set in l2 english speech for mispronunciation detection and diagnosis", International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018, pp.6244~6248 (2018)
- McAuliffe, M., "Montreal Forced Aligner: Trainable Text-Speech Alignment Using Kaldi", Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), 2017, pp.498~502 (2017)
- Panayotov, V., "Librispeech: an asr corpus based on public domain audio books", International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015, pp.5206~5210 (2015)
- Probst, K., "Enhancing foreign language tutors–in search of the golden speaker", Speech Communication, 37(3-4), pp.161~173 (2002)
- 노채환, "몽골인 한국어 학습자들의 한국어 이중모음 오류 분석", 외국어교육연구, 34(1), pp.215~235 (2020), 외국어교육연구소[DOI]
- Sun, L., "Voice conversion using deep bidirectional long short-term memory based recurrent neural networks", International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015, pp.4869~4873 (2015)
- Sun, L., "Phonetic posteriorgrams for many-to-one voice conversion without parallel data training", International Conference on Multimedia and Expo(ICME), 2016, pp.1~6 (2016)
- Sun, L., "Personalized, Cross-Lingual TTS Using Phonetic Posteriorgrams", Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), 2016, pp.322~326 (2016)
- Toda, T., "Voice conversion based on maximum-likelihood estimation of spectral parameter trajectory", IEEE Transactions on Audio, Speech, and Language Processing, 15(8), pp.2222~2235 (2007)
- Wu, Y. C., "Locally Linear Embedding for Exemplar-Based Spectral Conversion", Proce dings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), 2016, pp.1652~1656 (2016)
- Van der Maaten, L., "Visualizing data using t-SNE", Journal of machine learning research, 9(11) (2008)
- Zhang, X., "Improving deep neural network acoustic models using generalized maxout networks", International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2014)
- Zhao, G., "Exemplar selection methods in voice conversion", International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017, pp.5525~5529 (2017)
- Zhao, G., "L2-ARCTIC: A non-native English speech corpus" (2018), Perception Sensing Instrumentation Lab
- Zhao, G., "Using phonetic posteriorgram based frame pairing for segmental accent conversion", IEEE, 2019, pp.27(10), 1649~1660
- Zhao, G., "Foreign Accent Conversion by Synthesizing Speech from Phonetic Posteriorgrams", Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), 2019, pp.2843~2847 (2019)
- Zhou, Y., "Cross-lingual voice conversion with bilingual phonetic posteriorgram and average modeling", International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019, pp.6790~6794 (2019)