Implementation of a Personalized Music Content Creation Platform Utilizing Emotion Recognition Technology and Generative AI

Abstract

In modern society, music plays an integral role in daily life, making it crucial for individuals to easily find content that aligns with their musical preferences and emotional state. Traditional music production methods, which rely on playlists and basic visuals, are limited in their ability to offer personalized experiences. To address these limitations, a new application utilizing AI technology was developed to generate personalized music and multimedia content. The AI model analyzes the user's emotional state and optimizes musical elements to create custom music. This is achieved through the extraction of features like Mel-frequency cepstral coefficients (MFCC) and tempo from the user's music data, which are then used to generate prompts that reflect their emotional state. These prompts are fed into the MusicGen model to create new, emotion-based music. Additionally, a text-to-image-to-video conversion pipeline using ComfyUI was implemented, allowing the creation of diverse multimedia content based on the user's emotions and preferences. This approach not only addresses the time and cost constraints of traditional music production but also enhances the personalization and quality of the content. The method holds potential for future applications across various industries by providing more refined and tailored user experiences.