Transformer 네트워크를 이용한 음성신호 변환
Voice-to-voice conversion using transformer network
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Voice conversion can be applied to various voice processing applications. It can also play an important role in data augmentation for speech recognition. The conventional method uses the architecture of voice conversion with speech synthesis, with Mel filter bank as the main parameter. Mel filter bank is well-suited for quick computation of neural networks but cannot be converted into a high-quality waveform without the aid of a vocoder. Further, it is not effective in terms of obtaining data for speech recognition. In this paper, we focus on performing voice-to-voice conversion using only the raw spectrum. We propose a deep learning model based on the transformer network, which quickly learns the voice conversion properties using an attention mechanism between source and target spectral components. The experiments were performed on TIDIGITS data, a series of numbers spoken by an English speaker. The conversion voices were evaluated for naturalness and similarity using mean opinion score (MOS) obtained from 30 participants. Our final results yielded 3.52±0.22 for naturalness and 3.89±0.19 for similarity.
Abstract
Voice conversion can be applied to various voice processing applications. It can also play an important role in data augmentation for speech recognition. The conventional method uses the architecture of voice conversion with speech synthesis, with Mel filter bank as the main parameter. Mel filter bank is well-suited for quick computation of neural networks but cannot be converted into a high-quality waveform without the aid of a vocoder. Further, it is not effective in terms of obtaining data for speech recognition. In this paper, we focus on performing voice-to-voice conversion using only the raw spectrum. We propose a deep learning model based on the transformer network, which quickly learns the voice conversion properties using an attention mechanism between source and target spectral components. The experiments were performed on TIDIGITS data, a series of numbers spoken by an English speaker. The conversion voices were evaluated for naturalness and similarity using mean opinion score (MOS) obtained from 30 participants. Our final results yielded 3.52±0.22 for naturalness and 3.89±0.19 for similarity.
참고문헌 (24)
- "Parrotron: An end-to-end speech-to-speech conversion model and its applications to hearing-impaired speech and speech separation" (2019)[URL]
- "Empirical evaluation of gated recurrent neural networks on sequence modeling" (2014)[URL]
- "BERT:Pre-training of deep bidirectional transformers for language understanding" (2018)[URL]
- Griffin, D., "Signal estimation from modified short-time Fourier transform", IEEE Transactions on Acoustics, Speech, and Signal Processing, 32(2), pp.236~243 (1984)
- Hochreiter, S., "Long short-term memory", Neural Computation, 9(8), pp.1735~1780 (1997)
- "Voice transformer network: Sequence-to-sequence voice conversion using transformer with text-to-speech pretraining" (2019)[URL]
- "Direct speech-to-speech translation with a sequence-to-sequence model" (2019)[URL]
- "Vocoder-free end-to-end voice conversion with transformer Network" (2020)[URL]
- Kim, Y., "Convolutional neural networks for sentence classification", Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.1746~1751 (2014)
- "Adam: A method for stochastic optimization" (2015)[URL]
- Kwon, S., "Preprocessing for elderly speech recognition of smart devices", Computer Speech &Language, 36, pp.110~121 (2016)
- Lee, J., "Fully character-level neural machine translation without explicit segmentation", Transactions of the Association for Computational Linguistics, 5, pp.365~378 (2017)
- Leonard, R. G., "Tidigits speech corpus" (1993), Texas Instruments
- Li, N., "Neural speech synthesis with transformer network", Proceedings of the 33rd AAAI Conference on Artificial Intelligence, 33, pp.6706~6713 (2019)
- Liu, R., "Voice conversion with transformer Network", Proceedings of the ICASSP 2020−2020IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.7759~7759 (2020)
- McAuliffe, M., "Montreal forced aligner: Trainable text-speech alignment using Kaldi", Interspeech, 2017, pp.498~502 (2017)
- Morise, M., "WORLD : A vocoder-based high-quality speech synthesis system for real-time applications", IEICE Transactions on Information and Systems, 99(7), pp.1877~1884 (2016)
- Potamianos, A., "Automatic speech recognition for children", Proceedings of the 5th European Conference on Speech Communication and Technology, pp.2371~2374 (1997)
- "Improving language understanding by generative pre-training" (2018)[URL]
- Schuster, M., "Bidirectional recurrent neural networks", IEEE Transactions on Signal Processing, 45(11), pp.2673~2681 (1997)
- Shen, J., "Natural tts synthesis by conditioning wavenet on MEL spectrogram predictions", Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.4779~4783 (2018)
- Sutskever, I., "Sequence to sequence learning with neural networks", Advances in neural information processing systems 27 (NIPS 2014), pp.3104~3112 (2014)
- Tanaka, K., "AttS2S-VC: Sequence-to-sequence voice conversion with attention and context preservation mechanisms", Proceedings of the ICASSP 2019−2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.6805~6809 (2019)
- Vaswani, A., "Attention is all you need", Advances in neural information processing systems, pp.5998~6008 (2017)