A Basic Study on the Conversion of Color Image into Musical Elements based on a Synesthetic Perception
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
The final aim of the present study is to build a system of converting a color image into musical elements based on a synesthetic perception, emulating human synesthetic skills, which make it possible to associate a color image with a specific sound. This can be done on the basis of the similarities between physical frequency information of both light and sound. As a first step, an input true color image is converted into hue, saturation, and intensity domains based on a color model conversion theory. In the next step, musical elements including note, octave, loudness, and duration are extracted from each domain of the HSI color model. A fundamental frequency (F0) is then extracted from both hue and intensity histograms. The loudness and duration are extracted from both intensity and saturation histograms, respectively. In experiments, the proposed system on the conversion of a color image into musical elements was implemented using standard C and Microsoft Visual C++(ver. 6.0). Through the proposed system, the extracted musical elements were synthesized to finally generate a sound source in a WAV file format. The simulation results revealed that musical elements, which were extracted from an input RGB color image,reflected in its output sound signals.
Abstract
The final aim of the present study is to build a system of converting a color image into musical elements based on a synesthetic perception, emulating human synesthetic skills, which make it possible to associate a color image with a specific sound. This can be done on the basis of the similarities between physical frequency information of both light and sound. As a first step, an input true color image is converted into hue, saturation, and intensity domains based on a color model conversion theory. In the next step, musical elements including note, octave, loudness, and duration are extracted from each domain of the HSI color model. A fundamental frequency (F0) is then extracted from both hue and intensity histograms. The loudness and duration are extracted from both intensity and saturation histograms, respectively. In experiments, the proposed system on the conversion of a color image into musical elements was implemented using standard C and Microsoft Visual C++(ver. 6.0). Through the proposed system, the extracted musical elements were synthesized to finally generate a sound source in a WAV file format. The simulation results revealed that musical elements, which were extracted from an input RGB color image,reflected in its output sound signals.
참고문헌 (16)
- Cytowic, R. E., "Synesthesia : A Union of the Senses" (2002), The MIT Press
- Robertson, L. C., "Synesthesia : Perspectives from Cognitive Neuroscience" (2004), Oxford University Press
- Kim, G. H., "Sound Color Harmonism" (2003), Impress
- Osmanovic, N., "A testbed for auralization of graphic art, IEEE Region 5, 2003 Annual Technical Conference", IEEE Region 5, 2003 Annual Technical Conference, (45-49) (2003)
- Matta, S., "Discriminative analysis for image to sound mapping", Intelligent Sensing and Information Processing, pp.119~122 (2004)
- Bologna, G., "On the use of the auditory pathway to represent image scenes in real-time", Neurocomputing, 72(4-6), pp.839~849 (2009)
- Meijer, P. B. L., "An Experimental System for Auditory Image Representations", Transaction on Biomedical Engineering, 39(2), pp.112~121 (1992)
- Ward, J., "Sound-Colour Synaesthesia: to What Extent Does it Use Cross-Modal Mechanisms Common to us All", Cortex;a journal devoted to the study of the nervous system and behavior, 42(2), pp.264~280 (2006)
- Foner, L. N., "Artificial synesthesia via sonification : A wearable augmented sensory system", Mobile networks and applications : MONET, 4(1), pp.75~81 (1999)
- 강건우, "피치 기반 사운드－컬러이미지 변환에 관한 기초연구", 감성과학, 15(2), pp.231~238 (2012), 한국감성과학회
- Reinhard, E., "Color Imaging: Fundamentals and Applications" (2008), A K Peters, Ltd
- Freeman, M., "Mastering Color Digital Photography" (2004), Ilex Press
- Kim, G. H., "Method and apparatus for harmonizing colors by harmonics and converting sound into colors mutually", Korean Intellectual Property, 10-99-34242, ~ (1999)
- Kim, G. H., "Sound Color Harmonism" (2003), Impress
- Loy, G., "Musimathics : The Mathematical Foundations of Music(Volume 1)" (2006), The MIT Press
- Loy, G., "Musimathics : The Mathematical Foundations of Music(Volume 2)" (2007), The MIT Press