Enhancing Speech Emotion Recognition with Hybrid Graph Neural Networks: A GCN-GAT Framework
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
This paper proposes a speech emotion recognition method based on modeling speech signals as circular or linear graphs, enabling the extraction of node characteristics and practical analysis of relationships between nodes. The proposed method combines Graph Convolutional Networks (GCN) and Graph Attention Networks (GAT) layers to leverage the strengths of each in processing graph data. Precisely, GCN captures local relationships between nodes by aggregating information from neighboring nodes. The GAT mechanism better captures complex global relationships between nodes by assigning weights to neighboring nodes. Experiments validate our approach using the IEMOCAP dataset, demonstrating performance comparable to state-of-the-art models in emotion recognition tasks. The results of this study provide new insights and methodologies for further exploration in the field of speech signal processing.
Abstract
This paper proposes a speech emotion recognition method based on modeling speech signals as circular or linear graphs, enabling the extraction of node characteristics and practical analysis of relationships between nodes. The proposed method combines Graph Convolutional Networks (GCN) and Graph Attention Networks (GAT) layers to leverage the strengths of each in processing graph data. Precisely, GCN captures local relationships between nodes by aggregating information from neighboring nodes. The GAT mechanism better captures complex global relationships between nodes by assigning weights to neighboring nodes. Experiments validate our approach using the IEMOCAP dataset, demonstrating performance comparable to state-of-the-art models in emotion recognition tasks. The results of this study provide new insights and methodologies for further exploration in the field of speech signal processing.
참고문헌 (21)
- Mao, Shuiyang, "Deep Learning of Segment-Level Feature Representation with Multiple Instance Learning for Utterance-Level Speech Emotion Recognition", Interspeech, pp.1686~1690 (2019)
- Shirian, Amir, "Compact graph architecture for speech emotion recognition", ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP), pp.6284~6288 (2021), IEEE
- Liu, Jiawang, "Graph based emotion recognition with attention pooling for variable-length utterances", Neurocomputing, 496, pp.46~55 (2022)
- Ye, Jihua, "Attention-based spatio-temporal graph convolutional network considering external factors for multi-step traffic flow prediction", Digital Communications and Networks, 8(3), pp.343~350 (2022)
- Lee, Junhyun, "Self-attention graph poolin", International conference on machine learning (2019), pmlr
- Issa, Dias, "Speech emotion recognition with deep convolutional neural networks", Biomedical Signal Processing and Control, 59, pp.101894~ (2020)
- Busso, Carlos, "IEMOCAP: Interactive emotional dyadic motion capture database", Language resources and evaluation, 42, pp.335~359 (2008)
- Eyben, Florian, "Recent developments in opensmile, the munich open-source multimedia feature extractor", Proceedings of the 21st ACM international conference on Multimedia, pp.835~838 (2013)
- Fiedler, Miroslav, "Algebraic connectivity of graphs", Czechoslovak mathematical journal, 23(2), pp.298~305 (1973)
- Shuman, David I., "The emerging field of signal processing on graphs : Extending high-dimensional data analysis to networks and other irregular domains", IEEE signal processing magazine, 30(3), pp.83~98 (2013)
- Dai, Qiguo, "GraphCDA: a hybrid graph representation learning framework based on GCN and GAT for predicting disease-associated circRNAs", Briefings in Bioinformatics, 23(5), pp.bbac379~ (2022)
- Sokolova, Marina, "A systematic analysis of performance measures for classification tasks", Information processing & management, 45(4), pp.427~437 (2009)