Chapter I: General Introduction
1.7 Thesis overview
My thesis investigates how people pay attention when interacting with other people and the environment. I am particularly interested in how socially relevant cues attract and compete for attention and how these cues stand out from non-social stimuli. It is
important to create a vocabulary of social cues including eye gaze, face, head direction, finger gesture, and body posture, in order to investigate whether there is also a saliency map for these social cues. It is also useful to extend the investigation to lexical cues: what sequence of words captures people’s attention? Again, some examples are not hard to think of: your name, taboo words, exclamation marks all work fairly well. To break down the question, I am interested in quantifying the set of cues, or set of dimensions, that determine social saliency; in inquiring whether and how these are related to and interact with standard (non-social) “bottom-up” saliency; in exploring to what extent there are individual differences (e.g., in people from different cultures, in people with autism, or in males vs. females); and in understanding the neural mechanisms underlying these processes.
To approach these questions, I used four primary experimental techniques. One was high- resolution eye-tracking, which measures where people look, and in turn indicates where the attention goes. Using eye-tracking with high spatial and temporal resolution, we could understand the dynamic deployment of attention. A second was single-neuron recording in neurosurgical patients, which directly probes the neural correlates of perception and judgments. A third was a lesion approach, in particular in amygdala lesion patients, which tests the causal functional role of the amygdala, a key neural structure of the “social brain”. A fourth was testing neuropsychiatric populations such as people with autism, which is able to reveal possible behavioral deficits, especially social deficits, and trace these ultimately to their neural source. Importantly, combining these techniques can answer the same question from different angles and thus have a more holistic view of the question under investigation. For example, testing patients with amygdala lesions and people with autism on the identical task can inform whether amygdala dysfunction will lead to social deficits in autism. Testing a neurological population with concurrent eye- tracking can explore possible deficits of visual attention, while recording single-neurons from neurosurgical patients co-morbid with autism can reveal neuronal mechanisms that lead to behavioral impairment. Therefore, combining different approaches can yield more insights and often leads to more exciting findings. Along with this idea, we have been
conducting single-neuron recordings with concurrent eye-tracking in neurosurgical patients, in order to elucidate the neural mechanisms of one of the most important questions—how do we direct our gaze rapidly to salient objects in our visual environment.
On the other hand, I also employed a variety of experiments to investigate visual attention. In Chapter II, I employed a cued speeded task, in which different locations with respect to the center of optic flow were probed by reaction times in order to study the deployment of spatial attention in optic flow. In Chapter III, I employed a “change detection” protocol, in which subjects were exposed to alternations between two complex scenes that switched back and forth and were entirely identical except for a single change. It is well known that people are remarkably bad at detecting the single item that is changing between the alternating scenes (hence the name, “change blindness”), and this method has been widely used to study which stimuli automatically capture attention and become objects of our conscious awareness. In Chapter IV, we showed degraded emotional faces (a ‘bubbles task’) and asked subjects to judge emotions shown in the faces. With an adaptive learning algorithm implemented to keep a roughly constant performance, we were able to induce enough errors to investigate how neurons responded in the error trials. In Chapter V, I adopted a standard visual search task and directly compared fixations onto social vs. non-social objects in the cluttered search array. Taken together, I combined diverse experimental strategies of cognitive psychology with multiple neuroscience techniques, together with specific neurological and psychiatric populations, to elucidate the neural mechanisms that come into play. The neurobiological approaches provide information on the brain-end of my question: just like I am interested in determining what it is about social stimuli that captures attention, I am interested in whether there are specialized systems in the human brain for processing social stimuli.
My thesis is organized as follows. In Chapter I, I reviewed the social cues and the neural structures, particularly the amygdala, involved in processing saliency. I argued that people with autism may have altered saliency representation of the visual environment
and that amygdala dysfunction may partially account for this deficit. Then I start to investigate saliency from non-social cues. People constantly move in the environment and pay attention to their locomotion. In Chapter II, I studied the spatial cues that attract attention during locomotion using a cued speeded discrimination task and found that motion cues indicating a forward motion are the strongest to attract attention. However, compared to inanimate objects and cues, people preferentially attend to animals and faces, processing in which the amygdala is thought to play an important role. In Chapter III, I employed a change detection protocol and tested four rare patients with bilateral amygdala lesions. All four amygdala patients showed a normal pattern of reliably faster and more accurate detection of animate stimuli. People not only attend to faces, but also pay attention to others’ facial emotions. Humans have a dedicated system to process faces and the amygdala has long been associated with a key role in recognizing facial emotions. In Chapter IV, I studied the neural mechanism of emotion perception and found that single neurons in the human amygdala are selective for subjective judgment of others’ emotions. Lastly, normal people pay more attention to faces and conspecifics than to inanimate objects, but people with autism spectrum disorders (ASD) might not. To further study social attention, in Chapter V, I employed a visual search task and revealed a deficit of social attention in people with ASD. This deficit is independent of the amygdala but dependent on task demands. Overall, through visual psychophysics with concurrent eye-tracking, my thesis found and analyzed socially salient cues and compared social vs. non-social cues and healthy vs. clinical populations. Neural mechanisms underlying social saliency were elucidated through electrophysiology and lesion studies.