Music Transformer 기반 음악 정보의 가중치 변형을 통한 멜로디 생성 모델 구현
Implementation of Melody Generation Model Through Weight Adaptation of Music Information Based on Music Transformer
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
In this paper, we propose a new model for the conditional generation of music, considering key and rhythm, fundamental elements of music. MIDI sheet music is converted into a WAV format, which is then transformed into a Mel Spectrogram using the Short-Time Fourier Transform (STFT). Using this information, key and rhythm details are classified by passing through two Convolutional Neural Networks (CNNs), and this information is again fed into the Music Transformer. The key and rhythm details are combined by differentially multiplying the weights and the embedding vectors of the MIDI events. Several experiments are conducted, including a process for determining the optimal weights. This research represents a new effort to integrate essential elements into music generation and explains the detailed structure and operating principles of the model, verifying its effects and potentials through experiments. In this study, the accuracy for rhythm classification reached 94.7%, the accuracy for key classification reached 92.1%, and the Negative Likelihood based on the weights of the embedding vector resulted in 3.01.
Abstract
In this paper, we propose a new model for the conditional generation of music, considering key and rhythm, fundamental elements of music. MIDI sheet music is converted into a WAV format, which is then transformed into a Mel Spectrogram using the Short-Time Fourier Transform (STFT). Using this information, key and rhythm details are classified by passing through two Convolutional Neural Networks (CNNs), and this information is again fed into the Music Transformer. The key and rhythm details are combined by differentially multiplying the weights and the embedding vectors of the MIDI events. Several experiments are conducted, including a process for determining the optimal weights. This research represents a new effort to integrate essential elements into music generation and explains the detailed structure and operating principles of the model, verifying its effects and potentials through experiments. In this study, the accuracy for rhythm classification reached 94.7%, the accuracy for key classification reached 92.1%, and the Negative Likelihood based on the weights of the embedding vector resulted in 3.01.
참고문헌 (19)
- R. Kamien, "Music : An Appreciation" (2007), McGraw-Hill
- J. London, "Hearing in Time : Psychological Aspects of Musical Meter" (2004), Oxford University Press
- H. W. Dong, "MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment", Proceedings of Thirty-Second AAAI Conference on Artificial Intelligence (2018)
- S. Hochreiter, "Long Short-term Memory", Neural Computation, 9(8), pp.1735~1780 (1997)
- I. Goodfellow, "Generative Adversarial Nets", Advances in Neural Information Processing Systems, 27, pp.2672~2680 (2014)
- A. Vaswani, "Attention Is All You Need", Advances in Neural Information Processing Systems, 30, pp.5998~6008 (2017)
- A. R. Selfridge-Field, "Concepts of MIDI: Musical Instrument Digital Interface", Computing in Musicology, 8, pp.3~5 (1992)
- L. Cohen, "Time-frequency Distributions-a Review", Proceedings of the IEEE, 77(7), pp.941~981 (1989)
- Y. LeCun, "Deep Learning", Nature, 521(7553), pp.436~444 (2015)
- H. Huang, "Music Transformer: Generating Music with Long-Term Structure", Proceedings of ICLR (2019)
- S. Davis, "Comparison of Parametric Representations for Monosyllabic Word Recognition in Continuously Spoken Sentences", IEEE Transactions on Acoustics, Speech, and Signal Processing, 28(4), pp.357~366 (1980)
- T. Qian, "cMelGAN: An Efficient Conditional Generative Model Based on Mel Spectrograms", ~ (2022)
- D. P. Kingma, "Auto-Encoding Variational Bayes", Proceedings of the 2nd International Conference on Learning Representations (ICLR) (2014)
- "MusicVAE: Creating a Palette for Musical Scores wit h Machine Learning" (2018)[URL]
- N. Jaques, "Tuning Recurrent Neural Networks with Reinforcement Learning", ~ (2017)
- A. Oord, "WaveNet: A Generative Model for Raw Audio" (2016)
- A. Vaswani, "Attention Is All You Need", Advances in Neural Information Processing Systems, 30, pp.5998~6008 (2017)
- H. Huang, "Music Transformer: Generating Music with Long-Term Structure", Proceedings of the 7th International Conference on Learning Representations (ICLR) (2019)