Research

Our laboratory investigates the neural underpinnings of human social behavior. We are pursuing questions such as: How do we recognize emotion from facial expressions? How do we make social judgments about other people? How do we look at people’s faces (how do we move our eyes when looking at them)? How do we make decisions that are influenced by emotion? How do we remember emotional events in our lives? How do we make moral judgments about what is right and wrong?

These and other questions are being addressed by us using a variety of techniques. We study the impairments following specific brain damage in neurological patients; we record electrical activity from the brains of neurosurgical patients; we probe the behavior of people with neuropsychiatric diseases such as autism and Williams syndrome. And we are investigating behavior and brain activations in normal individuals, studies that include an examination of the differences between people in social cognition.

The laboratory currently has two sites: one located at the University of Iowa, and one located at the California Institute of Technology. This is website is for the latter. To go to the University of Iowa lab website, and links to people there, click here

Effective Information in Faces

Project Leaders: Ralph Adolphs and Frederic Gosselin

This project investigates which features in faces convey specific kinds of information. What features in faces do you use to judge the emotion, gender, age, or other socially relevant information of the person? For some of these questions we can predict what the answer would be, for instance, it would come as no surprise that we use information about the mouth (a smile) to judge that someone looks happy. But how do you judge that someone looks afraid? Trustworthy? Dominant? Arrogant? In some of these experiments, we are showing people random pieces of faces to find out which parts of faces convey such information. This technique is called reverse correlation, and has the power to show us patterns that we would not have predicted to start with.
The Figure below shows results from one such study (Adolphs et al., Nature 2005). It shows the effective information people use to discriminate fear in facial expressions. On the left is the information that healthy subjects use. On the right is the information used by a neurological subject with lesions to the amygdala. The neurological subject does not make normal use of the eye region of faces.


Information used by normal subjects	Information used by patient with amygdala lesions

Eye Movements when Viewing Faces

We are also monitoring viewers' eye movements as they look at faces and other visual stimuli ... (More)

Project Leaders: Michael Spezio et al

We are also monitoring viewers' eye movements as they look at faces and other visual stimuli. When healthy people look at faces, they spend a lot of time looking at the eyes and the mouth, as shown in the figure below. People with damage to the amygdala, with agenesis of the corpus callosum, and with autism, all look at faces abnormally. Detailed measurement of how they direct their gaze onto other people's faces is providing insight into what information in the social environment they seek out in the first place.

Eye movements when looking at a face. The face that viewers seen is shown underneath, and superimposed on it are the paths that people's eye gaze makes as they look at this face. Where viewers fixate is a white circle, and the red lines are saccades between fixations.

Face-to-face eyetracking: Michael Spezio, Lisa Lyons, and Sam Huang have developed methods for high-resolution, binocular eyetracking during face-to-face social interactions. In these studies, research participants wear a head-mounted eye tracker (the subject seated on the right in the figure below) that has a small scene camera on the forehead, as well as two small cameras below the eyes to monitor the subject's direction of gaze. When interacting with another person (e.g., the person on the left in the white shirt), we can measure fixations made onto that person's face. This is shown in the figure on the right below: the person in the white shirt is seen in a frame from the scene camera worn by the research subject, and the colors denote the density of the fixations (over a 10-minute interaction) that the subject made. Hotter colors encode a higher density of fixations. In this case, the subject looked most at the other person's eyes.

We are now applying both the eyetracking methods, and also the effective information methods described in the above section, to neurological subjects with focal brain lesions (e.g., focal lesions of the amygdala), as well as to psychiatric populations such as people with a diagnosis of autism or Asperger Syndrome.

Agenesis of the Corpus Callosum

Project Leader: Dr. Lynn Paul

Dr. Lynn Paul in our lab is investigating emotional and social behavior, and brain structure and function, in rare patients who lack the normal connection between the left and right hemispheres of the brain. This connection, the corpus callosum, is normally a massive bundle of fibers comprising some 200 Million axons in the human brain. It can be seen in saggital MR images, as shown on the top left of the figure below, and also using diffusion spectrum imaging, as shown in the bottom left of the figure. The right side of the figure shows the brain of a subject with agenesis of the corpus callosum. This research investigates how communication between the left and right hemispheres of the brain contributes to social behavior.

Agenesis of the corpus callosum. At the top are saggital structural MR images of a normal (left) and acallosal (right) brain. The corpus callosum is the page semicircular structure in the middle of the brain (absent in the image on the right). At the bottom are maps in which the direction of fractional anisotropy of water movement in the brain is encoded by color. Red denotes axons that go from left to right in this top-down view of a brain. The large red structure in the middle of the image on the left is the corpus callosum, absent in the acallosal brain on the right.

More Information: Agenesis of the Corpus Callosum Research Program

Social Cognition in Autism

We are investigating the neural underpinnings of autism ... (More)

Project Leader: Joseph Piven

In a collaboration with Joseph Piven at the University of North Carolina, we are investigating the neural underpinnings of autism. We are studying high-functioning people with autism and with Asperger Syndrome, as well as their first-degree relatives. We are using many of the methods described above: effective information use from faces, eye tracking, and imaging. The studies will help us to better understand the difficulties in social behavior seen in people with autism. For instance, in a recent study we have examined in detail how people with autism make use of information from faces. This study looked at the effective use of information from faces, and found subtle differences in how people with autism process faces compared to matched control subjects.
For more details see Spezio et al., in press

More Information: Autism Research Project

Moral Judgments about Autobiographical Memories

This project investigates how we recollect emotional autobiographical events, and how we make moral judgments about them ... (More)

Project Leaders: Jessica Edwards

This project investigates how we recollect emotional autobiographical events, and how we make moral judgments about them. We are asking people to recollect events from their lives, and to rate them in terms of their moral and emotional significance. Try remembering the worst thing you ever did. How about the best thing you ever did? Do you experience emotions when you remember these things? In retrospect, how blameworthy or praiseworthy do you think those actions of yours were? Were they justified, given the circumstances. We are asking people questions such as these, both about their own memories, and about the (anonymous) transcribed memories of others. Would you judge the same event differently if it was you that did it, as opposed to someone else? Would you judge it differently if it had been performed by a man or by a woman? Do your judgments correlate with your overall attitudes towards ethics? Do such judgments change following specific damage to the brain, such as damage to the frontal lobes?

Connectivity of emotional and visual regions

We are interested how the different cortical regions involved in emotional processing and the visual cortex communicate with each other and how this system evolved ... (More)

Project Leader: Dirk Neumann

Evaluation of probabilistic fiber tracking in the olfactory bulb of a mouse brain [Poster]

Imagine you spot a snake while jogging on one of the stunning Californian mountain trials or you recognize an old friend while strolling through the crowded streets of Old Town Pasadena: your emotional reactions will be immediate and instantaneously interrupt you in your trot and engange your attention. Our brain constantly monitors the emotional meaning of incoming visual information and this processing can be much faster and sometimes independend of our conscious visual experience. We are interested how the different cortical regions involved in emotional processing and the visual cortex communicate with each other and how this system evolved.

To trace the cortical connections between the different brain regions we measure the diffusion direction and speed of water molecules using magnetic resonance imaging. In addition, we are currently developing models to reliably analyze such data by combining methods used in physics to model stationary flow fields and probabilistic sampling techniques. Using Caltech's 3T MRI scanner we can visualize the connectivity of the human brain, smaller animal brains are scanned with high field scanners of up to 12T.