A New Vision Of Sound
Synaesthesia is a condition (or ability) whereby two or more senses are experientially intermingled; synaesthetes often perceive letters as having a particular colour, or can experience music not only as sound but also as having form, texture and colour. Although synaesthesia is typically an involuntary congenital condition, a research group are developing a device that may allow blind people to ‘see’ by converting video images to sound…
Introducing Audio Vision
The vOICe learning edition is a system that aims to help blind people to experience a form of vision that can be assimilated aurally. Basically, the user can wear a pair of glasses fitted with a small video camera which feeds into a UMPC that they can carry in a backpack.
The camera then feeds video data into a software program that converts the images into a sound output, which can be played back through headphones. The basic conversion principles are that scanning is effected from left to right, so the L/R position of a sound between the speakers equates to the position in the image. Elevation is communicated through pitch, with higher pitch indicating a higher elevation. Sound volume is used to indicate brightness, with black being signalled with no sound, and white being the loudest sound.
Can We Learn To See With Our Ears?
The idea behind this project is intriguing, and the concept is certainly a good one – after all, navigation through sound is quite common in nature, with the high-frequency sonar system of bats being an obvious example.
However, such a system really depends on the brain being able to interpret the data it is receiving, which is no easy task. The vOICe site has some example mp3 files of encoded images, and correlating the image to the sound is not particularly intuitive.
Time To Paint The Music
While synaesthetes may naturally correlate various inputs across their senses, there doesn’t seem to be any standard at work – for example, there is no consensus that a piano playing C# is always green. Studying larger groups does reveal certain trends, however. Generally speaking, higher pitched sounds are perceived as being of a brighter hue. One of the engineers interviewed in Bobby Owsinski’s ‘Mixing Engineer’s Handbook’ reported that he ‘saw’ high frequencies as being silver or gold in colour – no need for a spectrum analyser then.
Synaesthesia does seem to lend itself to artistic work – this page contains some examples of paintings created by a sound/colour synaesthete that represent how she sees certain songs.