음질 및 속도 향상을 위한 선형 스펙트로그램 활용 Text-to-speech
Text-to-speech with linear spectrogram prediction for quality and speed improvement
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Most neural-network-based speech synthesis models utilize neural vocoders to convert mel-scaled spectrograms into high-quality, human-like voices. However, neural vocoders combined with mel-scaled spectrogram prediction models demand considerable computer memory and time during the training phase and are subject to slow inference speeds in an environment where GPU is not used. This problem does not arise in linear spectrogram prediction models, as they do not use neural vocoders, but these models suffer from low voice quality. As a solution, this paper proposes a Tacotron 2 and Transformer-based linear spectrogram prediction model that produces high-quality speech and does not use neural vocoders. Experiments suggest that this model can serve as the foundation of a high-quality text-to-speech model with fast inference speed.
Abstract
Most neural-network-based speech synthesis models utilize neural vocoders to convert mel-scaled spectrograms into high-quality, human-like voices. However, neural vocoders combined with mel-scaled spectrogram prediction models demand considerable computer memory and time during the training phase and are subject to slow inference speeds in an environment where GPU is not used. This problem does not arise in linear spectrogram prediction models, as they do not use neural vocoders, but these models suffer from low voice quality. As a solution, this paper proposes a Tacotron 2 and Transformer-based linear spectrogram prediction model that produces high-quality speech and does not use neural vocoders. Experiments suggest that this model can serve as the foundation of a high-quality text-to-speech model with fast inference speed.
참고문헌 (17)
- "Wasserstein GAN" (2017)[URL]
- "HiFiSinger:Towards high-fidelity neural singing voice synthesis" (2020)[URL]
- Griffin, D., "Signal estimation from modified short-time Fourier transform", IEEE Transactions on Acoustics, Speech, and Signal Processing, 32(2), pp.236~243 (1984)
- "Towards robust neural vocoding for speech generation: A survey" (2020)[URL]
- "MelGAN:Generative adversarial networks for conditional waveform synthesis" (2019)[URL]
- "Neural speech synthesis with transformer network" (2019)[URL]
- Perraudin, N., "A fast Griffin-Lim algorithm", Proceedings of the 2013 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, pp.1~4 (2013)
- "WaveGlow: A flow-based generative network for speech synthesis" (2018)[URL]
- Ren, Y., "FastSpeech: Fast, robust and controllable text to speech", Proceedings of the 33rd Annual Conference on Neural Information Processing Systems, pp.3156~3164 (2019)
- Sharma, A., "Fast Griffin Lim based waveform generation strategy for text-to-speech synthesis", Multimedia Tools and Applications, 79(41), pp.30205~30233 (2020)
- Shen, J., "Natural TTS synthesis by conditioning WaveNet on Mel spectrogram predictions", Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing, pp.4779~4783 (2018)
- Song, W., "Efficient WaveGlow: An improved WaveGlow vocoder with enhanced speed", Proceedings of the 21st Annual Conference of the International Speech Communication Association, pp.225~229 (2020)
- Tachibana, H., "Efficiently trainable text-to-speech system based on deep convolutional networks with guided attention", Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing, pp.4784~4788 (2018)
- "WaveNet: A generative model for raw audio" (2016)[URL]
- "Attention is all you need" (2017)[URL]
- Wang, Y., "Tacotron: Towards end-to-end speech synthesis", Proceedings of the 18th Annual Conference of the International Speech Communication Association, pp.4006~4010 (2017)
- Zhu, X., "Real-time iterative spectrum inversion with look-ahead", Proceedings of the 2006 IEEE International Conference on Multimedia and Expo, pp.229~232 (2006)