딥러닝 모델의 환경음 분류 성능 향상을 위한 시간-주파수 표현
Time-Frequency Representations for Improving Environmental Sound Classification with Deep Learning Models
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
스펙트로그램은 오디오 신호처리 연구에서 주파수 성분의 강도를 효과적으로 분석하기 위해 널리 활용하지만, 시간에 따라 변화하는 위상 정보를 표현하는데 한계가 있다. 이러한 한계를 극복하기 위해, 본 논문에서는 파워 스펙트로그램(Power Spectrogram)과 순시 주파수(Instantaneous Frequency)를 결합한 시간-주파수표현을 탐구하며, 다양한 딥러닝 아키텍처를 활용한 환경음 분류 작업을 통해 효과성을 검증하였다. ESC-50 데이터세트를 대상으로 한 실험에서, 파워 스펙트로그램과 순시 주파수의 수직적 결합 방식을 적용한ConvNeXt 모델이 87.16%의 분류 정확도를 기록하였으며, 이는 기존 방식 대비 1.7%의 성능향상을 보여준다. 혼동 행렬 분석에서는 물과 관련된 소리와 사이렌 소리가 비슷한 패턴으로 인해 오분류 되었으며, 이를 개선하기 위해 추가 정보가 필요함을 시사한다. 본 연구는 중소규모 데이터세트를 활용하는 오디오 관련 작업에서 딥러닝 모델의 성능을 개선할 가능성을 제시하며, 다양한 소리 관련 응용에 폭넓게 활용될 수 있을 것으로 기대한다.
Abstract
Spectrograms are widely utilized in audio signal processing research to effectively analyze the magnitude of frequency components but they are limited in representing time-varying phase information. To overcome this limitation, this paper explores a time-frequency representation that combines Power Spectrogram (PS) and Instantaneous Frequency (IF) features and validates its effectiveness through environmental sound classification tasks using various deep learning architectures. Experiments on the ESC-50 dataset demonstrate that ConvNeXt model, leveraging the vertical integration of PS and IF, achieves a classification accuracy of 87.16%, reflecting a 1.7% improvement over conventional methods. The confusion matrix analysis reveals that misclassifications often occur for water-related sounds and sirens, as they exhibit highly similar time- frequency patterns, making them challenging to distinguish. This study highlights the potential of the proposed approach to enhance the performance of deep learning models in audio-related tasks, particularly for small- to medium-scale datasets and anticipates broad applicability in sound-related applications.
참고문헌 (22)
- K. Zaman, "A Survey of Audio Classification Using Deep Learning", IEEE Access, 11, pp.106620~106649 (2023)[DOI]
- B. Michael, "At Your Service : Designing Voice Assistant Personalities to Improve Automotive User Interfaces", Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp.1~11 (2019)
- B. J. Zhang, "Nonverbal Sound in Human-Robot Interaction : A Systematic Review", ACM Transactions on Human-Robot Interaction, 12(4), pp.1~46 (2023)[DOI]
- Y. Yuki, "Audio-Processing-Based Human Detection at Disaster Sites with Unmanned Aerial Vehicle", IEEE Access, 8, pp.101398~101405 (2020)[DOI]
- G. Ciaburro, "Improving Smart Cities Safety Using Sound Events Detection Based on Deep Neural Network Algorithms", MDPI Informatics, 7(3), pp.23~ (2020)[DOI]
- A. Mesaros, "DCASE 2017 Challenge setup: Tasks, datasets and baseline system", DCASE 2017-workshop on detection and classification of acoustic scenes and events (2017)
- J. F. Gemmeke, "Audio Set : An ontology and human-labeled dataset for audio events", International Conference on Acoustics, Speech, and Signal Processing (ICASSP) (2017)
- K. J. Piczak, "ESC : Dataset for Environmental Sound Classification", Proceedings of the 23rd ACM international conference on Multimedia, pp.1015~1018 (2015)
- J. Salamon, "A Dataset and Taxonomy for Urban Sound Research", Proceedings of the 22nd ACM international conference on Multimedia, pp.1041~1044 (2014)
- A. Khamparia, "Sound Classification Using Convolutional Neural Network and Tensor Deep Stacking Network", IEEE Access, 7, pp.7717~7727 (2019)[DOI]
- F. Demir, "A new pyramidal concatenated CNN approach for environmental sound classification", Applied Acoustics, 170, pp.107520~ (2020)[DOI]
- W. Mu, "Environmental sound classification using temporal-frequency attention based convolutional neural network", Scientific Reports, 11 (2021)[DOI]
- Y. Zhang, "Spectrogram Transformers for Audio Classification", 2022 IEEE International Conference on Imaging Systems and Techniques (IST) (2022)
- J. Luo, "Vision Transformer based Audio Classification using Patch-level Feature Fusion", 2022 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) (2022)
- K. He, "Deep Residual Learning for Image Recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.770~778 (2016)
- Z. Liu, "A ConvNet for the 2020s", Proceedings of the IEEE, 2022, 11976~11986
- Z. Liu, "Swin Transformer : Hierarchical Vision Transformer using Shifted Windows", 2021 IEEE, 2021, 10012~10022
- J. Deng, "ImageNet : A large-scale hierarchical image database", 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp.248~255 (2009)
- Z. Zhang, "Learning Attentive Representations for Environmental Sound Classification", IEEE Access, 7, pp.130327~130339 (2019)[DOI]