어텐션 기반 엔드투엔드 음성인식 시각화 분석
Visual analysis of attention-based end-to-end speech recognition
0
조회
0
다운로드
3
인용
0
참고문헌
키워드
초록
An end-to-end speech recognition model consisting of a single integrated neural network model was recently proposed. The end-to-end model does not need several training steps, and its structure is easy to understand. However, it is difficult to understand how the model recognizes speech internally. In this paper, we visualized and analyzed the attention-based end-to-end model to elucidate its internal mechanisms. We compared the acoustic model of the BLSTM-HMM hybrid model with the encoder of the end-to-end model, and visualized them using t-SNE to examine the difference between neural network layers. As a result, we were able to delineate the difference between the acoustic model and the end-to-end model encoder. Additionally, we analyzed the decoder of the end-to-end model from a language model perspective. Finally, we found that improving end-to-end model decoder is necessary to yield higher performance.
Abstract
An end-to-end speech recognition model consisting of a single integrated neural network model was recently proposed. The end-to-end model does not need several training steps, and its structure is easy to understand. However, it is difficult to understand how the model recognizes speech internally. In this paper, we visualized and analyzed the attention-based end-to-end model to elucidate its internal mechanisms. We compared the acoustic model of the BLSTM-HMM hybrid model with the encoder of the end-to-end model, and visualized them using t-SNE to examine the difference between neural network layers. As a result, we were able to delineate the difference between the acoustic model and the end-to-end model encoder. Additionally, we analyzed the decoder of the end-to-end model from a language model perspective. Finally, we found that improving end-to-end model decoder is necessary to yield higher performance.
참고문헌 (15)
- Abdel-Hamid, O., "March). Applying convolutional neural networks concepts to hybrid NN-HMM model for speech recognition", Proceedings of the 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.4277~4280 (2012)
- Chan, W., "March). Listen, attend and spell : A neural network for large vocabulary conversational speech recognition", Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.4960~4964 (2016)
- (2014)[URL]
- Dayhoff, J. E., "Artificial neural networks: Opening the black box", Cancer: Interdisciplinary International Journal of the American Cancer Society, 91(S8), pp.1615~1635 (2001)
- Graves, A., "Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks", Proceedings of the 23rd International Conference on Machine Learning, pp.369~376 (2006)
- Graves, A., "Speech recognition with deep recurrent neural networks", Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.6645~6649 (2013)
- Explainable artificial intelligence(XAI), "Gunning, D.", Defense Advanced Research Projects Agency (DARPA), ~ (2017)
- Hinton, G., "Deep neural networks for acoustic modeling in speech recognition", IEEE Signal Processing Magazine, 29(6), pp.82~97 (2012)
- "The MNIST database of handwritten digits" (1998)[URL]
- Miao, Y., "EESEN: End-to-end speech recognition using deep RNN models and WFST-based decoding", Proceedings of the 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), pp.167~174 (2015)
- Mikolov, T., "Recurrent neural network based language model", Proceedings of the 11th Annual Conference of the International Speech Communication Association, pp.1045~1048 (2010)
- Panayotov, V., "Librispeech: An ASR corpus based on public domain audio books", Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.5206~5210 (2015)
- (2017)[URL]
- Sutskever, I., "Sequence to sequence learning with neural networks", Proceedings of the Advances in Neural Information Processing Systems, pp.3104~3112 (2014)
- (2018)[URL]