한국어 text-to-speech(TTS) 시스템을 위한 엔드투엔드 합성 방식 연구
An end-to-end synthesis method for Korean text-to-speech systems
0
조회
0
다운로드
5
인용
0
참고문헌
키워드
초록
A typical statistical parametric speech synthesis (text-to-speech, TTS) system consists of separate modules, such as a text analysis module, an acoustic modeling module, and a speech synthesis module. This causes two problems: 1) expert knowledge of each module is required, and 2) errors generated in each module accumulate passing through each module. An end-to-end TTS system could avoid such problems by synthesizing voice signals directly from an input string. In this study, we implemented an end-to-end Korean TTS system using Google's Tacotron, which is an end-to-end TTS system based on a sequence-to-sequence model with attention mechanism. We used 4392 utterances spoken by a Korean female speaker, an amount that corresponds to 37% of the dataset Google used for training Tacotron. Our system obtained mean opinion score (MOS) 2.98 and degradation mean opinion score (DMOS) 3.25. We will discuss the factors which affected training of the system. Experiments demonstrate that the post-processing network needs to be designed considering output language and input characters and that according to the amount of training data, the maximum value of n for n-grams modeled by the encoder should be small enough.
Abstract
A typical statistical parametric speech synthesis (text-to-speech, TTS) system consists of separate modules, such as a text analysis module, an acoustic modeling module, and a speech synthesis module. This causes two problems: 1) expert knowledge of each module is required, and 2) errors generated in each module accumulate passing through each module. An end-to-end TTS system could avoid such problems by synthesizing voice signals directly from an input string. In this study, we implemented an end-to-end Korean TTS system using Google's Tacotron, which is an end-to-end TTS system based on a sequence-to-sequence model with attention mechanism. We used 4392 utterances spoken by a Korean female speaker, an amount that corresponds to 37% of the dataset Google used for training Tacotron. Our system obtained mean opinion score (MOS) 2.98 and degradation mean opinion score (DMOS) 3.25. We will discuss the factors which affected training of the system. Experiments demonstrate that the post-processing network needs to be designed considering output language and input characters and that according to the amount of training data, the maximum value of n for n-grams modeled by the encoder should be small enough.
참고문헌 (27)
- Arik, S., "Deep Voice: Real-time neural text-to-speech", Proceedings of the 34th International Conference on Machine Learning, pp.195~204 (2017)
- Arik, S., "Deep Voice 2: Multi-speaker neural text-to-speech", Advances in Neural Information Processing Systems, 30, pp.2966~2974 (2017)
- "Neural machine translation by jointly learning to align and translate" (2014)[URL]
- Bengio, Y., "Curriculum learning", Proceedings of the 26th Annual International Conference on Machine Learning, pp.41~48 (2009)
- "Learning phrase representations using RNN encoder-decoder for statistical machine translation" (2014)[URL]
- "Empirical evaluation of gated recurrent neural networks on sequence modeling" (2014)[URL]
- Collins, J., "Capacity and trainability in recurrent neural networks", Proceedings of the 5th International Conference on Learning Representations (2017)
- Griffin, D., "Signal estimation from modified short-time fourier transform", IEEE Transactions on Acoustics, Speech, and Signal Processing, 32(2), pp.236~243 (1984)
- He, K., "Deep residual learning for image recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.770~778 (2016)
- Hochreiter, S., "Long short-term memory", Neural Computation, 9(8), pp.1735~1780 (1997)
- Hunt, A. J., "Unit selection in a concatenative speech synthesis system using a large speech database", Proceedings of the 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing, pp.373~376 (1996)
- Ioffe, S., "Batch normalization: Accelerating deep network training by reducing internal covariate shift", Proceedings of the 32nd International Conference on Machine Learning, pp.448~456 (2015)
- Kawahara, H., "Speech representation and transformation using adaptive interpolation of weighted spectrum: Vocoder revisited", Proceedings of the 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing, pp.1303~1306 (1997)
- "Fully character-level neural machine translation without explicit segmentation" (2016)[URL]
- Morise, M., "WORLD : A vocoder-based high-quality speech synthesis system for real-time applications", IEICE Transactions on Information and Systems, 99(7), pp.1877~1884 (2016)
- "WaveNet: A generative model for raw audio" (2016)[URL]
- "Deep voice 3: Scaling text-to-speech with convolutional sequence learning" (2017)[URL]
- Rabiner, L., "Theory and applications of digital speech processing" (2011), Pearson
- Raffel, C., "Online and linear-time attention by enforcing monotonic alignments", Proceedings of the 34th International Conference on Machine Learning, pp.2837~2846 (2017)
- "Natural TTS synthesis by conditioning wavenet on mel spectrogram predictions" (2017)[URL]
- Srivastava, N., "Dropout : A simple way to prevent neural networks from overfitting", Journal of Machine Learning Research, 15(1), pp.1929~1958 (2014)
- "Highway networks" (2015)[URL]
- Sutskever, I., "Sequence to sequence learning with neural networks", Advances in Neural Information Processing Systems, 27, pp.3104~3112 (2014)
- Tokuda, K., "Speech synthesis based on hidden markov models", Proceedings of IEEE, 101(5), pp.1234~1252 (2013)
- Vinyals, O., "Grammar as a foreign language", Advances in Neural Information Processing Systems, 28, pp.2773~2781 (2015)
- "Tacotron: Towards end-to-end speech synthesis" (2017)[URL]
- Wu, Z., "Merlin: An open source neural network speech synthesis system", Proceedings of the 9th ISCA Speech Synthesis Workshop, pp.218~223 (2016)