beau's blog

I had the pleasure of attending the Neukom Institute’s 2008 symposium at Dartmouth College, “The Human Algorithm.” Videos of the symposium are now online, so I’m going to pull out my notes and do some follow-up on on what I thought were the most interesting presentations. The sleeper hit of the conference was James Haxby’s “The Minority Report: Update 2008″, a summary of recent developments in multi-voxel pattern analysis (MVPA), which is essentially the application of pattern classification algorithms to fMRI data.

The use of MVPA represents a bit of a paradigm shift in neuroimaging research. We can do some pretty serious mind-reading. This gets a little geeky, but bear with me, it’s worth it.

With MVPA, rather than just looking at hot spots in the data, like single activated voxels or contiguous groups of voxels, distributed patterns of activation in a region are considered. Haxby begins the talk by discussing an experiment of his where subjects in the scanner were shown images of faces, houses, shoes, and other objects. The resultant dataset was split in half, and a pattern classification algorithm was run on each subset. Haxby et al. then demonstrated that the patterns of activity for each category in both subsets matched the other, e.g. shoe-related patterns from one half matched shoe-related patterns from the other half better than patterns representing different categories. Given a pattern without a label, they were able to guess its category with 90% accuracy (Haxby et al. 2001).

This is really neat. It’s basically mind-reading. If you’re in an fMRI scanner, and you’re shown a picture of a shoe, and they’ve mapped out your brain a little bit, they can confidently say “You’re looking at a shoe.”

In 2005, Kamitani & Tong pushed it even further. They looked at V1, the primary visual cortex, and showed subjects images of parallel lines with various angles of orientation. Using MVPA, they identified patterns of activation associated with with each given angle (Kamitani et al. 2005). The voxel resolution was 3 cubic millimeters, and multiple orientation-sensitive cells exist within any given voxel. The success of this experiment shows that with MVPA we can cheat and examine patterns of neural activity in finer detail than the fMRI voxel resolution. Researchers at UC Berkeley used a similar model of orientation-sensitive areas in V1, but showed subjects photographs of real-world scenes. They were able to predict what images people were looking at with 80% accuracy (Kay et al. 2008). When these models make mistakes, they make pretty smart mistakes, never choosing, for example, an angle orthagonal to the correct answer.

Further, MVPA allows for hierarchical clustering of similar patterns of activation. Analysis of the data from Haxby’s 2001 study showed that patterns associated with faces, for example, are all more similar to each other than patterns associated with houses. They built a similarity tree, showing human faces were represented similarly to cat faces, inanimate objects were represented dissimilarly from animate objects, and so on (Hanson et al. 2004). This hierarchy of cognitive representational similarity was replicated in a test of 600+ individual neurons of a macaque, finding that the monkey made similar categorical distinctions between animate and inanimate, mammal and reptile, etc (Kiani et al. 2007).

(image from Kiani et al. 2007)

That’s it for now. I’m going to discuss some of the implications MVPA and mind-reading research further in a future post.

References and papers mentioned in Haxby’s talk:

• Haxby, J.V.; Gobbini, M.I.; Furey, M.L.; Ishai, A.; Schouten J.L.; Pietrini, P. (2001). Distributed and Overlapping Representations of Faces and Objects In Ventral Temporal Cortex. Science 293: 2425-2430.
• Kamitani, Y.; Tong, F. (2005). Decoding the visual and subjective contents of the human brain. Nature Neuroscience 8: 679-85.
• Haynes, J.; Rees, G. (2005) Predicting the orientation of invisible stimuli from activity in human primary visual cortex. Nature Neuroscience 8: 686-691.
• G.M. Boynton. (2005) Imaging orientation selectivity: decoding conscious perception in V1. Nature Neuroscience 8: 541-542.
• Kay, K.N.; Naselaris, T; Prenger, R.J.; Gallant, J.L. (2008) Identifying natural images from human brain activity. Nature 452: 352-356.
• Hanson, S.J.; Matsuka, T.; Haxby, J.V. (2004) Combinatorial codes in ventral temporal lobe for object recognition: Haxby (2001) revisited: is there a “face” area? NeuroImage 23: 156-166.
• O’Toole, A.J.; Jiang, F.; Abdi, H.; Haxby, J.V. (2005) Partially Distributed Representations of Objects and Faces in Ventral Temporal Cortex. Journal of Cognitive Neuroscience 17: 580-590
• Kiani, R.; Esteky, H.; Mirpour, K.; Tanaka, K. (2007) Object Category Structure in Response Patterns of Neuronal Population in Monkey Inferior Temporal Cortex. Journal of Neurophysiology 97: 4296-4309.
• Norman, K.A.; Polyn, S.M.; Detre, G.J.; Haxby, J.V. (2006) Beyond mind-reading: multi-voxel pattern analysis of fMRI data. Trends in Cognitive Science Vol. 10, No. 9: 424-430.
This entry was posted on Tuesday, July 1st, 2008 at 7:52 pm and is filed under Cognitive science and philosophy. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.