Arts & Sciences

Humans seem to have a powerful curiosity about how their minds work. We're constantly asking "Why did he do that?" or "Why did she say that?" or even "Why did I do that?" In the cognitive sciences, researchers draw from psychology, computer science, philosophy, linguistics and now most centrally neurobiology in order to procure converging evidence on how the mind works.

By and large, cognitive scientists have found that simply asking a person what they think about something will typically give spurious results. Essentially, such introspective reports are contaminated by the fact that the person is no longer just "thinking about X," but is now "thinking about thinking about X," which changes the process significantly.

Instead, cognitive scientists use more subtle measures that provide a glimpse into what really goes on in someone's mind. One of the most common experimental methods in cognitive psychology consists of recording a person's reaction time. A longer reaction time in responding to a particular stimulus will often indicate that the person has a weak or somehow noisy mental representation of the meaning of that stimulus. Another popular methodology, used by cognitive neuroscientists, termed functional magnetic resonance imaging (fMRI), involves measuring the blood flow to brain regions that subserve various cognitive tasks. These methodologies provide extremely useful information about how the mind/brain works, but they have their limitations. With reaction times, it can sometimes be difficult to know exactly what one can conclude from one stimulus eliciting a response that is a mere 50 milliseconds faster than another stimulus's response. With fMRI, human participants must be almost entirely motionless while their heads are engulfed in the surprisingly loud fMRI machine-not the most naturalistic circumstances in which to study human cognition.

A new methodology for studying cognitive processes that avoids these disadvantages involves recording participantsŐ eye movements with a headband-mounted eyetracker. Humans and other primates have evolved to have high spatial resolution in the center of the retina (the fovea), with poor resolution and little or no color perception in the periphery of their field of view. Therefore, they systematically aim their eyes precisely at regions of the field of view that contain objects most relevant for the potential action that is demanding the most consideration at that point in time. Moreover, the muscles that move the eyes are surprisingly fast, allowing the eyes to move from region to region about every 250 milliseconds. The result is that people tend to look at several different objects in quick succession, and certainly not at random. By studying where people look while they carry out certain cognitive tasks, we can gain insight into what the mind is considering at a sampling rate of about 4 times per second.

With this methodology, human participants wear a lightweight headband on which are attached two cameras: an eye-camera recording the image of the eye as it moves, and a scene-camera recording the participant's field of view. For data analysis, we superimpose crosshairs indicating the participant's direction of gaze onto the scene-camera's view. Participants can move around naturally (even walking short distances), and when they systematically look at Object A before looking at (and then acting on) Object B, one can safely infer that Objects A and B have competing mental representations in the human brain.

Our first findings with this methodology were in the study of language comprehension, published in 1995 in Science with co-authors Michael Tanenhaus, Kathy Eberhard, and Julie Sedivy. In this study, participants were given spoken instructions to move objects around on a table. The patterns with which their eyes examined the various objects consistently demonstrated that the participants used the visual display to constrain their processing of spoken language at the level of word recognition and grammatical processing. For example, when instructed to "pick up the candy" with a display of objects including a bag of candy, a fork, a napkin, and a diskette, participants typically looked at the candy about 100 milliseconds after the end of the word "candy" and then picked it up. In contrast, when the same instruction was given with a display containing a bag of candy, a candle, a napkin, and a diskette, participants took significantly longer to look at the candy, and frequently looked at the candle before finally looking at (and picking up) the candy. The human brain quickly maps the initial sounds onto two possible visual representations (the object that is the candy, and the object that is the candle). An early decision of which object to look at will frequently result in an eye movement to the wrong object. This work showed that, although language areas of the brain and visual areas of the brain are anatomically separate from one another, the two interact rather closely and quickly.

In related work, Viorica Marian and I have demonstrated that, contrary to traditional theory, the two language systems in bilingual brains are not independent of one another. For example, when given spoken instructions to pick up a postage stamp in Russian ("Padnimi marku"), Russian-English bilinguals frequently looked initially at a marker because the mental representation of the English word "marker" (not the Russian name for marker, "flomaster") is phonologically similar to "marku." If the English language system were completely independent of the Russian language system, then mental representations of English words would not be activated during a purely Russian conversation. This research appeared in the May 1999 issue of Psychological Science.

Our most recent work shows that eyetracking can be informative even when there's nothing in the visual display! Recently presented at the annual meeting of the Society for Computers in Psychology, Joy Geng, Daniel Richardson, and I have demonstrated that eye movements are tightly linked with visual imagery and memory. When listening to a story describing a dynamic visual scene with objects moving consistently upward or downward, participants staring at a blank wall made eye movements predominantly in the direction corresponding to the direction of the dynamic imagery. Similarly, when trying to remember a fact recited by a speaker who is no longer present, participants frequently looked at the (empty) location in space where the speaker was originally observed. This work suggests that brain regions that are responsible for visual imagery and memory automatically send signals to brain regions that control eye movement even in the absence of external visual stimuli.

Overall, the findings emerging from this new paradigm of cognitive investigation point to an understanding of how the mind/brain works not in terms of separate and independent processors, but in terms of overlapping and interacting processors (for language, vision, imagery, memory, etc.), with bidirectional synaptic pathways connecting anatomically separate neural systems. The human mind/brain is not a collection of sub-minds. Rather, the human mind/brain is an interactive dynamic system composed of subsystems that are intrinsically enmeshed with one another and indeed with the rest of the organism.

Michael Spivey is assistant professor of psychology. His research is funded by a fellowship in neuroscience from the Sloan Foundation and a grant from the Consciousness Studies Program at the University of Arizona. Among other courses, he teaches Introduction to Cognitive Science, which gives 200+ freshman and sophomores a taste of psychology, neuroscience, computer science, linguistics, and philosophy.