beau's blog

Long week; didn’t get a chance to post anything. As penance, I’m going to write something every day until the sloth is washed away.

Melissa Saenz and Christof Koch at Caltech offer up the first lab-tested reports of visual-audio synesthesia. They’ve found four people who hear sound when they see motion or flashes of light. The sounds they hear are “simple” (I’m not sure what that means) beeps, taps, and whirrs. No sound is perceived due to eye movement, so it really does seem to be triggered by the perception of stuff-tagged-as-motion, and not something lower-level, right off the retina.

To validate the claims of synesthesia, pairs of short rhythms were played to subjects, either as visual flashes or auditory beeps. Subjects reported whether they thought the rhythms in each pair were the same or different. Typically people are quite good at identifying auditory rhythms and bad with visual ones. As expected, everybody did well on the auditory task, but only the synesthetes could accurately compare the visual rhythms, presumably because they could hear them. Interestingly, over the course of the experiment, the synesthetes reported that the synesthetic sounds they heard along with the visuals changed to match the real sounds played during the auditory tests.

A neat direction to go with this would be to play various visual stimuli for the synesthetes and collect phenomenological descriptions of the sounds heard, with the aim of mapping the visual-audio mapping. Are the sounds similar in dynamic profile to the visuals which trigger them? What causes the sounds to change even as the visuals stay the same, as they did during the experiment? What are the intersubjective differences in the synesthetic sounds?

References:

• Saenz, M.; Koch, C. The sound of change: visually-induced auditory synesthesia. Current Biology (2008) vol. 18 (15) pp. R650-R651