Visual-audio synesthesia

August 12th, 2008

Long week; didn’t get a chance to post anything. As penance, I’m going to write something every day until the sloth is washed away.

Melissa Saenz and Christof Koch at Caltech offer up the first lab-tested reports of visual-audio synesthesia. They’ve found four people who hear sound when they see motion or flashes of light. The sounds they hear are “simple” (I’m not sure what that means) beeps, taps, and whirrs. No sound is perceived due to eye movement, so it really does seem to be triggered by the perception of stuff-tagged-as-motion, and not something lower-level, right off the retina.

To validate the claims of synesthesia, pairs of short rhythms were played to subjects, either as visual flashes or auditory beeps. Subjects reported whether they thought the rhythms in each pair were the same or different. Typically people are quite good at identifying auditory rhythms and bad with visual ones. As expected, everybody did well on the auditory task, but only the synesthetes could accurately compare the visual rhythms, presumably because they could hear them. Interestingly, over the course of the experiment, the synesthetes reported that the synesthetic sounds they heard along with the visuals changed to match the real sounds played during the auditory tests.

A neat direction to go with this would be to play various visual stimuli for the synesthetes and collect phenomenological descriptions of the sounds heard, with the aim of mapping the visual-audio mapping. Are the sounds similar in dynamic profile to the visuals which trigger them? What causes the sounds to change even as the visuals stay the same, as they did during the experiment? What are the intersubjective differences in the synesthetic sounds?

References:

Brain wars, part 2: nano-intentionality

July 29th, 2008

In my previous entry I discussed a talk at Neukom ‘08 by Daniel Dennett where he revises his model of consciousness to account for recent thought regarding competition for resources within the brain. He refers us to a paper by Tecumseh Fitch which both provides a detailed explanation of how this competition came about and offers a defense of intrinsic intentionality—the possibility of things being intrinsically directed toward other things.

Fitch begins by providing his interpretation of Dennett’s thought in The Intentional Stance. He sees Dennett’s work as a full-force attack on the possibility of intrinsic intentionality (all citations Fitch 2007):

The thrust of Dennett’s argument is that this “something” (which he terms variously “original” and “intrinsic” intentionality) is an illusion, a deep philosophical mistake that derives from our unwillingness to fully bite the bullet and accept natural selection (and its products, especially ourselves) for the blind and goal-less process that it is. Dennett criticizes intrinsic intentionality with arguments that need to be taken seriously, arguments which seem to force us into a corner where we must choose between the following propositions: either accept that all intentionality, including our own, is derivative (and then admit that a thermostat has a little bit of intentionality, too), or we retreat from this unpalatable option into a mysterian, outmoded belief in “original/intrinsic intentionality”, a concept that under close scrutiny leads to contradiction and paradox at every turn.

At Neukom, Dennett says Fitch misinterpreted his work on intentionality, but doesn’t explain exactly how. Regardless, he thinks Fitch is right, and he’s on board with nano-intentionality and its implications.

From the beginning of his paper, Fitch focuses on a key difference between living things and machines: the ability of living things, especially small ones like cells and neurons, to adapt to local circumstances by changing their shape. Formal changes act as both a kind of memory and a way in which the cell is “about” its environment. This inherent, formal aboutness is nano-intentionality. Fitch argues that the kind of inherent intentionality we attribute to minds is a consequence of the nano-intentionality of cells and neurons. “When combined properly into large interconnected systems, this combined mass-action of cellular nano-intentionality yields intrinsic intentionality in the typical philosopher’s sense, as well as both consciousness and the efficacy of our subjectively sensed self to move the body and perform other acts of will (‘intentionality’ in its general English sense), just as the mass and velocity of gas molecules contained in a volume are constitutive of its temperature, pressure and weight.”

Here is how the argument evolves: Start with an amoeba. We understand much of how amoebas work on a mechanical level, and we know they came to behave this way because of natural selection. This is Fitch’s “prototypical example of [...] nano-intentionality”. The “purpose” of an ameoba is “realized via its physical form: it is a complex arrangement of matter serving to do useful things like find food and avoid toxins”, etc. Critically, an ameoba can deal with novelty using trial and error, “remembering” successful strategies by modifying its form. In contrast, a thermostat can only interact with what it was designed to interact with; it is only “about” what it was designed to be “about”. But an ameoba is about its whole environment, a world of things.

In multi-celluar organisms, groups of specialized nano-intentional cells cooperate, creating intentional subsystems which are “about” different things. Fitch’s example is the sponge. Each subsystem in the sponge is “about” something different, like creating efficient water flows or digesting. Fitch argues that the intentionality of these groups allows the sponge to be intentional itself, independent of an observer taking Dennett’s intentional stance. A sponge is “about” filter-feeding because its constituant parts allow it to filter feed, because the sponge evolved that way. “The aboutness of the sponge is reflected in its structure, but is constituted by its evolutionary history: the story of sponge-kind.” I.e., the way in which individual cells are “about” their environment changes when their environment contains other, collaborating cells: a kind of group intentionality emerges. The same is true on the next level of abstraction: collaborating groups of groups of cells like a sponge.

Next Fitch looks at jellyfish. They have a basic nervous system, a nano-intentional subsystem which is about information processing. In the nervous system, neurons don’t just collaborate with other neurons to perform a simple task like flaggellation, they interact with each other, such that structural changes in one cell are changes in the environment of the others. Indeed, a change in the nervous system of an organism is a change in the environment of many other cells in that organism, neurons or not. Neurons are “‘about’ amplification of information into locally adaptive patterns of action”. Neuronal events, however, are about whatever environmental change triggered the event. The organism as a whole can now react to the world around it. Fitch calls systems with this capacity “proto-mental.”

Fitch goes all the way up. The nervous system evolves from simply reacting to environmental changes, to representing those changes, and eventually to acting as a generative model, a source of ideas about “possible worlds”. Neuronal assemblages which are successful, i.e. whose representations correspond to actions actually taken by the organism in the world, are “tagged” by other neuronal assemblages, and from this emerges the serial experience of consciousness. Here we have intentionality in the philosopher’s sense, and a theory about how it arises. Fitch concludes that any system which has this kind of possible-world-tagging system must have some subjective experience, some awareness:

“To the extent that this argument is correct, awareness is the intrinsic subjective side of an objectively-verifiable capability of some types of nervous system to both entertain multiple hypothesis at a given time, and to later learn from their mistakes and successes. This leads to a rather strong and surprising thesis: that any nervous system objectively capable of considering and choosing among mental options, and of learning from its past decisions, will have at least a little bit of awareness as it does so: a little bit of (serial) consciousness.”

Fitch is, essentially, biting the natural-selection bullet offered by Dennett. But rather than looking for ways in which some internal mind could interface with an external world, he shows us how mind and world are inextricably related, and how directedness emerges from this relation.

The paper also includes an interesting discussion of how this way of thinking will affect artificial intelligence research, among other things.

References:

Dennett, D. How Mindless Algorithms Build Minds. May 9, 2008. Online video clip, accessed on July 21, 2008. http://neukominstitute.com/index.php/site/feature/dennett_talk/symposia/49/symposium08

Dennett, D. (1987) The Intentional Stance. MIT Press.

Fitch, W.T. (2007) Nano-Intentionality: A Defense of Intrinsic Intentionality. Biology & Philosophy, Vol. 23: 157-177.

Brain wars, part 1: the death of intrinsic intentionality

July 22nd, 2008

Daniel Dennett gave a great lecture at the recent Neukom Symposium. I recently had a chance to revisit it and check out some of the fascinating papers he cites, especially Tecumseh Fitch’s recent work on what he calls “nano-intentionality”. Today’s post will summarize some of Dennett’s talk and then introduce his attack on intrinsic intentionality. The follow-up post later this week will dig deeper, exploring Fitch’s nano-intentionality and how it brings intrinsic intentionality back from the dead.

Dennett’s talk takes as its starting point the claim that the brain is a computer, and to support this he introduces a distinction between cooperative and competitive computation—corresponding (funnily enough) to a distinction between brains and personal computers of the sort we use every day. Within a personal computer there are a number of processes which are systematically assigned resources used to accomplish goals. The brain is different: we don’t have the same kind of resource allocation protocol. Inside of us numerous, massively parallelized processes engage in heated, vicious, life-or-death competition for resources with which they may or may not accomplish their goals. According to Dennett the resource allocation paradigm of the personal computer underpins our intuitive, “default model” of computation—but the “serious, deadly” competition of cells, neurons, and higher level processes for resources in the brain is computational as well, and of great importance to the understanding of consciousness.

This represents a significant update to Dennett’s multiple-drafts/fame-in-the-brain model of consciousness. Very, very briefly—I can not do this idea justice in a single sentence—there is no one place in the brain where consciousness arises (e.g. Descartes’s pineal gland), but we become conscious of things when useful information about them is shared between various, independently operating cognitive modules (which are somewhat chaotic themselves, a la Selfridge’s pandemonium). The latest addition to this model is that all of these different units, or their component parts, are competing for resources, essentially working to starve competing processes out of existence. Much of Dennett’s talk focuses on competition between high-level, abstract units, like metaphors, turns of phrase, or memes. According to this theory, ideas compete for resources, and the ideas we express are the ones which “win”, starving their competitors into oblivion. This is where I diverge from his talk, which expands upon the computer metaphor, discussing the mind as software, ideas as virtual machines, and the implications of competition in the brain for normal and abnormal psychology, sociology, etc. I’m going to go in the opposite direction, into the low-level, micro-scale, really dirty stuff.

How might this competition in the brain really play out on lower levels, such as that of the cell, the synapse, or the neuron? Dennett refers us to Fitch’s paper on nano-intentionality. In order to make sense of nano-intentionality, we’ve got to understand normal-sized intentionality first, an area in which Dennett has been a major player.

Intentionality is a tricky subject. It is the “aboutness” of things; their directedness. One of the hot areas of contention in modern philosophy is the means by which things acquire directedness, especially when those things are minds or thoughts. There is some consensus that there are at least two basic types of intentionality: derived and intrinsic. For example, there is clearly a difference between how a word acquires its directedness—by definition and consensus, from the intentions of the word’s designers—and the way a thought seems to have a kind of intrinsic, original intentionality. We don’t need to define the meaning of a thought about something, in fact the very idea sounds ridiculous. The target of a thought seems somehow inextricably wrapped up in the form of the thought itself, long before the definitive power of language can get inside and fiddle with it. Another canonical example is the difference between a thermostat and the human heart. The thermostat is designed; it is “about” the ambient temperature by virtue of the intentions of its designer. What is the human heart about? Pumping blood? How? Where does this aboutness come from? Is it derived, or intrinsic? And, further, if there is intrinsic intentionality, how exactly does it arise? That last one is a killer.

In The Intentional Stance Dennett suggests that the whole discussion of derived/intrinsic intentionality is pretty confused, and what we are actually doing when we assess intentionality is using one of three tools: the physical, design, or intentional stances. When we take the physical stance, we are evaluating how something operates in a reductive, low-level way. When we take the design stance, we determine the possible function of an item and the reason for its design. Finally, when we take the intentional stance we are speculating about the conscious state of an item; its goals, desires, emotions, what it knows or does not know. The design stance roughly corresponds to assessment of derived intentionality. But where does intrinsic intentionality go? We can adopt the intentional stance toward a thermostat just as well as we can adopt it toward a person. It turns out we do this all the time, referring to inanimate objects as if they have belief and desire. The thermostat “knows” the ambient temperature and “wants” to keep it within a certain range.

The apparent intrinsic intentionality of mental phenomena, then, is just a useful folk-psychological metaphor we use to understand each other, and doesn’t correspond to a real, extant thing. In Fitch’s words, Dennett wants us to “bite the bullet and accept natural selection (and its products, especially ourselves) for the blind and goal-less process that it is.” (Fitch 2007) If we have some sort of intentionality, it is derived from the process of natural selection, the same kind of intentionality possessed by a thermostat. This doesn’t sit right with people, it either seems to ascribe too little to the mind or too much to the thermostat.

Fitch tries, I think successfully, to resurrect intrinsic intentionality. Dennett says he’s on board with Fitch’s approach. Details later this week.

References:

Dennett, D. How Mindless Algorithms Build Minds. May 9, 2008. Online video clip, accessed on July 21, 2008. http://neukominstitute.com/index.php/site/feature/dennett_talk/symposia/49/symposium08

Dennett, D. (1987) The Intentional Stance. MIT Press.

Fitch, W.T. (2007) Nano-Intentionality: A Defense of Intrinsic Intentionality. Biology & Philosophy, Vol. 23: 157-177.

Scholarpedia

July 18th, 2008

I remember when Scholarpedia was brand new and completely empty. Today I found Daniel Dennett curates an article there on his Multiple Drafts model of consciousness. Poking around, I found a bunch of other great stuff. Here goes:

The fMRI article makes no mention of recent pattern classification/MVPA techniques… but neither does the fMRI Wikipedia article. Still room to grow!

Bone Alphabet

July 16th, 2008

Recently I’ve gotten pretty interested in some of Brian Ferneyhough’s ideas about music, especially what he has to say about his use of rhythm, duration and time. I have misgivings about a lot of his work, but I’ve become fond of Bone Alphabet, a piece for seven percussion instruments of indeterminate sound where adjacent instruments must not be of the same family. There’s an article by Steven Schick about the process of learning Bone Alphabet, which took him a maybe-not-totally-justifiable nine months. To get it right he cut up the score and pasted it on to graph paper so he could see the rhythms more clearly, did quite a bit of math to understand the polyrhythms, memorized stickings measure by measure, and other such feats of dedication and devotion. The result was, interestingly, a feeling of being deeply situated within his body. He talks quite a bit about the importance of bodily movement to the piece, mentioning at one point that an audio recording alone would not really convey the work. Here’s a good excerpt (Schick 1994):

My method of actually learning the piece involved first cutting out each bar and gluing it on graph paper so that I could better calculate rhythmical relationships. Then I made all decisions regarding sticking and mallet choice and memorized them before going on to the next measure. The advantage to this approach was that, by memorizing as the first and not the last step in the process, I could more quickly embed the material I was learning in the realm of physical gesture. As a result, from the first instant the piece became a theatrical arena where physical gesture was not the simple by-product of performance, but an integral part of a growing interpretive point of view. The instrument became a kind of stage for the enactment of, in Ferneyhough’s words, “a theatre of the body.”

Ross Karre has recorded his interpretation of Bone Alphabet and put it on YouTube. Unfortunately embedding is disabled, so to view it you’ll have to follow these links:

And, finally, a couple of quotes from Ferneyhough to round this all out. From his essay Duration and rhythm as compositional resources:

Whilst the impulse-structure and its audibility are clearly variably perceptible in concrete compositional situations, I maintain that enough of a correspondence is maintained in the middle to long term to enable the flow of space/density ratios demonstrated capable of carrying the main weight of formal organization. According to this principle, degrees of compression, distortion, convergence or mutual interference are calculable in respect of the degree to which the sense of clock time is supported or subverted by the specific tactility of impulse density setting the ‘inner clock’ of a particular metric space.

As always in the discussion of matters artistic, it is not the clear-cut cases which prove most pertinent, but the ill-defined and fluctuating ‘grey zones’ where a given rhythmic phenomenon may be called on to assume multiple functional roles.

References: