RoutingConvNet: 양방향 MFCC 기반 경량 음성감정인식 모델
RoutingConvNet: A Light-weight Speech Emotion Recognition Model Based on Bidirectional MFCC
2
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
본 연구에서는 음성감정인식의 적용 가능성과 실용성 향상을 위해 적은 수의 파라미터를 가지는 새로운 경량화 모델 RoutingConvNet(Routing Convolutional Neural Network)을 제안한다. 제안모델은 학습 가능한 매개변수를 줄이기 위해 양방향 MFCC(Mel-Frequency Cepstral Coefficient)를 채널 단위로 연결해 장기간의 감정 의존성을 학습하고 상황 특징을 추출한다. 저수준 특징 추출을 위해 경량 심층 CNN을 구성하고, 음성신호에서의 채널 및 공간 신호에 대한 정보 확보를 위해 셀프어텐션(Self-attention)을 사용한다. 또한, 정확도 향상을 위해 동적 라우팅을 적용해 특징의 변형에 강인한 모델을 구성하였다. 제안모델은 음성감정 데이터셋(EMO-DB, RAVDESS, IEMOCAP)의 전반적인 실험에서 매개변수 감소와 정확도 향상을 보여주며 약 156,000개의 매개변수로 각각 87.86%, 83.44%, 66.06%의 정확도를 달성하였다. 본 연구에서는 경량화 대비 성능 평가를 위한 매개변수의 수, 정확도 간 trade-off를 계산하는 지표를 제안하였다.
Abstract
In this study, we propose a new light-weight model RoutingConvNet with fewer parameters to improve the applicability and practicality of speech emotion recognition. To reduce the number of learnable parameters, the proposed model connects bidirectional MFCCs on a channel-by-channel basis to learn long-term emotion dependence and extract contextual features. A light-weight deep CNN is constructed for low-level feature extraction, and self-attention is used to obtain information about channel and spatial signals in speech signals. In addition, we apply dynamic routing to improve the accuracy and construct a model that is robust to feature variations. The proposed model shows parameter reduction and accuracy improvement in the overall experiments of speech emotion datasets (EMO-DB, RAVDESS, and IEMOCAP), achieving 87.86%, 83.44%, and 66.06% accuracy respectively with about 156,000 parameters. In this study, we proposed a metric to calculate the trade-off between the number of parameters and accuracy for performance evaluation against light-weight.
참고문헌 (29)
- 임명진, "상담 챗봇의 다차원 감정 인식 모델", 스마트미디어저널, 10(4), pp.21~27 (2021), (사)한국스마트미디어학회
- 신주현, "텍스트와 음성의 앙상블을 통한 다중 감정인식 모델", 스마트미디어저널, 11(8), pp.65~72 (2022), (사)한국스마트미디어학회[DOI]
- H.J. Vögel, "Emotion-awareness for intelligent vehicle assistants: A research agenda", Proc. of the 1st International Workshop on Software Engineering for AI in Autonomous Systems, pp.11~15 (2018)
- 임명진, "Word2Vec과 LSTM을 활용한 이별 가사 감정 분류", 스마트미디어저널, 9(3), pp.90~97 (2020), (사)한국스마트미디어학회[DOI]
- J. Parry, "Analysis of Deep Learning Architectures for Cross-Corpus Speech Emotion Recognition", Interspeech, pp.1656~1660 (2019)
- A. Vaswani, "Attention is all you need", Proc, of Conference on Neural Information Processing Systems, pp.5998~6008 (2017)
- Z. Zhao, "Combining a parallel 2D CNN with a self-attention Dilated Residual Network for CTC-based discrete speech emotion recognition,”", Neural Network, 141, pp.52~60 (2021)
- S. Sabour, "Dynamic routing between capsules", Proc, of Conference on Neural Information Processing Systems, pp.3856~3866 (2017)
- F. Liu, "LGCCT : A light gated and crossed complementation transformer for multimodal speech emotion recognition", Entropy, 24(7), pp.1010~1025 (2022)
- C.W. Wu, "ProdSumNet : reducing model parameters in deep neural networks via product-of-sums matrix decompositions" (2018)
- J. Ye, "Temporal Modeling Matters : A Novel Temporal Emotional Modeling Approach for Speech Emotion Recognition" (2022)
- S. Zhang, "Speech Emotion Recognition Using Deep Convolutional Neural Network and Discriminant Temporal Pyramid Matching", IEEE Transactions on Multimedia, 20(6), pp.1576~1590 (2017)
- F. Burkhardt, "A Database of German Emotional Speech", pp.1~4 (2005), Interspeech
- S.R Livingstone, "The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English", Plos, 13(5), pp.e0196391~ (2018)
- C. Busso, "IEMOCAP : Interactive emotional dyadic motion capture database", Language resources and evaluation, 42(4), pp.335~359 (2008)
- P. Nantasri, "A light-weight artificial neural network for speech emotion recognition using average values of MFCCs and their derivatives", 17th International conference on electrical engineering, 2020, 41~44
- K. Atsavasirilert, "A light-weight deep convolutional neural network for speech emotion recognition using mel-spectrograms", 14th International Joint Symposium on Artificial Intelligence and Natural Language Processing (iSAI-NLP), pp.1~4 (2019)
- A. Krizhevsky, "Imagenet classification with deep convolutional neural networks", Communications of the ACM, 60(6), pp.84~90 (2017)
- J.X Ye, "GM-TCNet : Gated Multi-scale Temporal Convolutional Network using Emotion Causality for Speech Emotion Recognition", Speech Communication, 145, pp.21~35 (2022)
- J. L. Bautista, "Speech Emotion Recognition Based on Parallel CNN-Attention Networks with Multi-Fold Data Augmentation", Electronics, 11(23), pp.3935~3949 (2022)
- D. Tang, "End-to-end speech emotion recognition using a novel context-stacking dilated convolution neural network", EURASIP Journal on Audio, Speech, and Music Processing, 2021(1), pp.1~16 (2021)
- S. Loffe, "Batch normalization : Accelerating deep network training by reducing internal covariate shift", Proc 32nd International Conference on International Conference on Machine Learning, pp.448~456 (2015)
- D.A. Clevert, "Fast and accurate deep network learning by exponential linear units (ELUs)" (2015)
- J. Tompson, "Efficient object localization using convolutional networks", Proc. of the IEEE conference on computer vision and pattern recognition, pp. 648-656, Boston, USA, Jun (2015)
- B. McFee, "librosa: Audio and Music Signal Analysis in Python", Proc. of the 14th python in science conference, pp.18~25 (2015)
- D.P. Kingma, "Adam: A Method for Stochastic Optimization" (2014)
- B. Nagarajan, "Deep Learning as Feature Encoding for Emotion Recognition" (2018)
- K. Chauhan, "Speech Emotion Recognition Using Convolution Neural Networks", international conference on artificial intelligence and smart systems(ICAIS), pp.1176~1181 (2021)
- X. Wu, "Neural Architecture Search for Speech Emotion Recognition", 2022 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP)IEEE, pp.6902~6906 (2022), IEEE