robust neural signatures than neutral ones, and can consequently profi t from preferential access to further cognitive processing, behavior control and awareness.
It has been suggested that the prioritization of emotional information is driven by dedi- cated neural circuits (Brosch et al., 2011; Vuilleumier, 2005; Vuilleumier & Brosch, 2009), separate from the fronto-parietal networks that are involved in endogenous and exog- enous attention. In this model, the amygdala, a limbic region critically involved in the monitoring of the environment for emotionally relevant information (Cunningham &
Brosch, 2012; Sander, Grafman, & Zalla, 2003) is thought to play a critical role by modu- lating the processing of incoming sensory stimuli through direct feedback projections to visual cortex (Amaral, Behniea, & Kelly, 2003) and biasing signals to fronto-parietal atten- tion regions (Pourtois, Th ut, Grave de Peralta, Michel, & Vuilleumier, 2005). Consistent with this suggestion, several neuroimaging studies have reported that cortical increases were signifi cantly correlated with amygdala responses; i.e., the more the amygdala was sensitive to a stimulus, the more modulation was observed in sensory areas. Th e boost- ing by the amygdala may not only directly impact on sensory cortices, but it can also bias the fronto-parietal endogenous attention network toward the location of the stimulus, so that subsequent information arising at the same location as emotional cues will benefi t from enhanced processing resources. Functional magnetic resonance imaging record- ings during the emotional dot probe revealed greater activation in the IPS when targets were preceded by a fearful face than a neutral face, consistent with enhanced attentional orienting (Pourtois, Schwartz, Seghier, Lazeyras, & Vuilleumier, 2006). Taken together, neuroimaging work has demonstrated how socio-aff ective stimuli can induce a distinc- tive cascade of neural events which does not only boost the processing of the stimulus itself, but also infl uences mechanisms responsible for orienting and shift ing attention in space, such that subsequent information arising at the same location as a socio-aff ective cue will also benefi t from enhanced processing resources (see also Pourtois, Schettino, &
Vuilleumier, 2013; Vuilleumier & Brosch, 2009).
JOINT ATTENTION: MECHANISMS OF EYE GAZE CUEING 87
primates do not have white sclerae but darker ones, which is much less conducive to a rapid read-out of gaze direction, see Kobayashi & Kohshima, 1997). And indeed, humans are especially accurate at discriminating diff erent directions of eye gaze, using both contrast information and geometrical information for the computation of gaze direction (for review, see Frischen, Bayliss, & Tipper, 2007). From a very early age, humans are able to detect eyes and use the provided information to direct their atten- tion, learn the names of objects, and gain an insight into the mind of their caregivers.
For example, during their fi rst year, infants start following their caregivers’ head turns (Scaife & Bruner, 1975) and turn their gaze into the direction of a pair of eyes pre- sented on a computer screen (Hood, Willen, & Driver, 1998). Th ese very early cases of gaze cueing seem, however, to be based on reactions toward the perception of move- ment in the eyes, not on a real understanding of the communicative intent of others, whereas “real” joint attention develops between 12 and 18 months (Brooks & Meltzoff , 2005). Th e acquisition of this skill may form the basis for a rapid subsequent devel- opment of other socio-cognitive functions. For example, by the age of three, children use gaze information to ascribe internal states such as desires to others (Baron-Cohen, Campbell, Karmiloff -Smith, Grant, & Walker, 1995).
Most empirical investigations of the eff ects of gaze on attention have been conducted using the gaze cueing task (see Fig. 6.2b). In this task, a centrally presented person looks either to the left or to the right, and then a target is presented to the left or to the right of the person. Th e participant has to detect the target. Participants’ responses are faster in trials where the target appears at the location indicated by the gaze compared with tar- gets appearing at the opposite location. Th is eff ect is observed even when the gaze is not predictive, i.e., the eye gaze indicates the correct side in only 50% of the trials (Friesen &
Kingstone, 1998), consistent with an automatic cueing of attention by gaze. Recordings of eye movements have shown that the observation of an averted gaze may trigger eye movements towards the cued location, even before the target appears (Mansfi eld, Farroni, & Johnson, 2003), which indicates that an observed eye gaze may activate similar motor programs in the observer. Interindividual diff erences have been observed in the strength of the attentional shift . For example, participants who report low levels of self-esteem show a larger gaze cueing eff ect than people with high self-esteem, indicating that low self-esteem is related to a higher tendency to use another individual’s gaze as a refer- ence for how their attention should be allocated (Wilkowski, Robinson, & Friesen, 2009).
Social interaction partners may not only use their eye gaze to indicate a common point of interest, but also to mislead the observer by guiding his attention away from a potentially interesting location. In a variant of the gaze cueing paradigm, diff erent identities showing diff erent behaviors were used for the centrally presented face: “trustworthy” faces that always looked toward the location where the target would appear, “untrustworthy” faces that never looked to the target location, and “neutral” faces that looked to the correct location in 50% of the trials (Bayliss & Tipper, 2006). Participants followed the gaze of all faces, no matter how predictive they were for the task, consistent with a strong automatic- ity of gaze cueing. However, aft er the experiment, participants rated the faces that always
looked in the right direction as more likeable and trustworthy than faces that looked in the wrong direction.
Th us, observing someone’s gaze reliably and automatically triggers shift s of attention in the corresponding direction. In addition, eye gaze may interact with emotional expres- sions, which may refl ect additional information about the relevance of the attended object and may thus be used to inform the evaluations and behavior of the observer. For exam- ple, a fearful face looking at a particular location may indicate the presence of a threat at this location. Th is threat may also be dangerous for the observer, thus attentional shift s towards the cued location should have higher priority compared with locations cued by neutral face (see also Bruder, Fischer, & Manstead, Chapter 10, this volume). Even though this interaction eff ect has not been observed in all the studies that set out to fi nd it (see Graham & Labar, 2012, for a review), studies using dynamic stimuli have shown that a fearful face can lead to increased gaze cueing compared with a neutral face (Tipples, 2006; see also Putman, Hermans, & Van Honk, 2006). Information transmitted by facial expressions may also inform the explicit evaluations of the observer. When participants observed a person with a happy expression looking at an object, they gave higher liking ratings to the objects compared to objects looked at with an expression of disgust (Bayliss, Frischen, Fenske, & Tipper, 2007). Similar eff ects have been observed in social evalua- tions: male faces that were looked at by females with smiling faces were rated as more attractive by female participants than males looked at with neutral expressions. Revealing an interesting gender diff erence, in the same experiment male participants preferred the male faces that were being looked at by female faces with neutral expressions (Jones, DeBruine, Little, Burriss, & Feinberg, 2007).
Taken together, these fi ndings demonstrate that other people’s eyes provide impor- tant cues for the attention system. Humans use information from the eye gaze of oth- ers to infer the attentional focus of their interaction partners and automatically shift their own attentional focus to the same location. Furthermore, additional information about the relevance of the attended object, as refl ected in emotional expressions of the interaction partner, may be integrated to inform the observer’s appraisal of the environment.
Compared to the attentional prioritization mechanisms outlined in the fi rst part of this chapter, the joint attention mechanism has a much stronger interactive component.
Using this mechanism, an individual can guide (or misguide) another’s attention and thus another’s perception of the environment, and can furthermore inform and modify the other’s aff ective evaluation of objects or persons that are jointly attended to. Joint atten- tion thus serves as an automatic sharing mechanism for relevant information. Emotions are defi ned as reactions to events that are appraised as relevant to the goals, needs and values of an individual (see, e.g., Brosch et al., 2010). Collective emotions, in turn, can be considered as “responses to events that are appraised as relevant to the goals, needs and values of several individuals simultaneously.” Th us, joint attention may constitute a key mechanism underlying the synchronization of the appraisals of several individuals that is necessary for the elicitation of collective emotions.
JOINT ATTENTION: MECHANISMS OF EYE GAZE CUEING 89
Th e large number of brain areas dedicated to the processing of the human face refl ects the importance of this information for humans. Face perception is mediated by a large distributed system including visual, limbic, and prefrontal regions (Ishai, 2008).
Specialized core brain regions of face processing are located in inferior occipital gyrus and fusiform gyrus. In these regions, incoming visual information is encoded struc- turally and transformed into a perspective-independent model of the face (Rotshtein, Henson, Treves, Driver, & Dolan, 2005). Th e superior temporal sulcus (STS) is involved in the processing of diff erent types of biological motion, and thus plays an important role in the decoding of eye gaze direction (Calder et al., 2007). Studies in humans and monkeys provide converging evidence that the STS contains neurons sensitive to dif- ferent gaze directions, as well as certain static combinations of face positions and gaze (e.g., frontal view of the face with averted eye gaze, see Nummenmaa & Calder, 2009, for a review). Th is enables the observer to infer the direction of someone’s attention under a variety of visual conditions. STS furthermore has many direct connections to other brain regions implicated in aff ective and attentional processes, such as the amygdala and the dorsal fronto-parietal attention system. Th e amygdala is involved in the detection of emotionally and motivationally relevant information, which includes eyes and eye gaze.
Patients with amygdala damage have problems in discriminating gaze directions (Young et al., 1995). Furthermore, they show defi cits in recognizing others’ emotions, largely because they fail to orient their attention towards the eyes (Adolphs et al., 2005). STS fur- thermore has connections with the IPS, a region in the dorsal fronto-parietal attention network (Corbetta & Shulman, 2002), and may input information about gaze direction via these connections into the attention system to bias attention accordingly. Recordings of event-related potentials during the gaze cueing task have revealed the speed of this modulation, showing rapid attentional shift s of eye gaze cues as soon as 300 ms aft er gaze onset (Holmes, Mogg, Garcia, & Bradley, 2010; Schuller & Rossion, 2001). Recent neu- roimaging studies furthermore show an involvement of medial prefrontal regions in the processing of gaze cues (Nummenmaa & Calder, 2009). Th ese regions have been impli- cated in the attribution of mental states to others across a number of diff erent tasks (Van Overwalle, 2009). It has also been suggested that the mirror system plays a role in eye gaze cueing, in that the observation of another’s eye movement may immediately trigger motor responses in FEF that match the observed movement—however, empirical fi nd- ings are contradictory, and the role of mirror neurons in joint attention is still debated in the literature (see Frischen et al., 2007, for a discussion). Altogether, neuroimaging fi ndings suggest that joint attention is implemented by a distributed network centered on the STS, including regions involved in aff ective processing, attentional selection, and higher social cognition. During initial face processing, interactions of STS and amyg- dala may underlie the rapid detection of other people’s eyes and the analysis of their eye movements and gaze direction. Connections between STS and fronto-parietal regions such as the IPS may then be used to input information about the direction of the eye gaze into the attention system to initiate an orienting into the corresponding direction.
Finally, activation in mentalizing regions such as medial prefrontal cortex may refl ect
the involvement of higher-order social inferences, e.g., about the goals, intentions, and preferences of the observed person.
Conclusion
Signals that indicate the aff ective state, interests, and intentions of others are enormously relevant for the “social animal.” It is thus highly adaptive to prioritize socially relevant information out of the incoming information stream. I have reviewed behavioral and neuroimaging research demonstrating rapid attentional prioritization of a large number of socially relevant signals, including emotional expressions, facial confi gurations, and social group identities, which results in more robust neural representation of the pri- oritized stimuli as well as perceptual enhancements such as faster stimulus detection or increased contrast sensitivity. Converging evidence suggests that the underlying mecha- nisms are based on a rapid automatic appraisal of stimulus relevance, are sensitive to mul- tiple classes of potentially relevant socio-aff ective stimuli, and are highly fl exible to recent changes in motivational contingencies. Th is rapid attentional prioritization of socially relevant information establishes sensory contact between individuals, which is necessary for the interpersonal sharing of internal (e.g., via emotional contagion or empathy) and external (via joint attention) information, and thus constitutes an important precursor to collective emotion.
In the second part of the chapter, I have focused on the mechanisms of joint attention, which enable individuals to use information from the eye gaze of others to infer their attentional focus and shift their own attention to the same location. Via this mechanism, individuals can guide another’s perception of the environment and can modify their aff ec- tive evaluation of objects or persons. As joint attention allows the sharing of information about the relevance of an external event, it may constitute a key mechanism for the syn- chronization of the appraisals of several individuals that may underlie the elicitation of collective emotions.
References
Adolphs , R. , Gosselin , F. , Buchanan , T. W. , Tranel , D. , Schyns , P. , & Damasio , A. R. ( 2005 ). A mechanism for impaired fear recognition aft er amygdala damage . Nature , 433 , 68–72 .
Amaral , D. G. , Behniea , H. , & Kelly , J. L. ( 2003 ). Topographic organization of projection-s from the amygdala to the visual cortex in the macaque monkey. Neuroscience , 118 , 1099–1120 .
Baron-Cohen , S. , Campbell , R. , Karmiloff -Smith , R. , Grant , J. , & Walker , J. ( 1995 ). Are children with autism blind to the mentalistic signifi cance of the eyes? British Journal of Developmental Psychology ,
13 , 379–398 .
Bayliss , A. P. , Frischen , A. , Fenske , M. J. , & Tipper , S. P. ( 2007 ). Aff ective evaluations of objects are infl uenced by observed gaze direction and emotional expression. Cognition , 104 , 644–653 .
Bayliss , A. P. , & Tipper , S. P. ( 2006 ). Predictive gaze cues and personality judgments: Should eye trust you? Psychological Science , 17 , 514–520 .
Bertels , J. , Kolinsky , R. , & Morais , J. ( 2010 ). Emotional valence of spoken words infl uences the spatial orienting of attention. Acta Psychologica , 134 , 264–278 .
REFERENCES 91
Brewer , M. B. ( 1988 ). A dual process model of impression formation. In T. Srull & R. Wyer (Eds.),
Advances in Social Cognition (Vol. 1, pp. 1–36 ). Hillsdale, NJ : Erlbaum .
Brooks , R. , & Meltzoff , A. N. ( 2005 ). Th e development of gaze following and its relation to language.
Developmental Science , 8 , 535–543 .
Brosch , T. , Grandjean , D. , Sander , D. , & Scherer , K. R. ( 2009 ). Cross-modal emotional attention: Emotional voices modulate early stages of visual processing. Journal of Cognitive
Neuroscience , 21 , 1670–1679 .
Brosch , T. , Pourtois , G. , & Sander , D. ( 2010 ). Th e perception and categorization of emotional stimuli: A review. Cognition and Emotion , 24 , 377–400 .
Brosch , T. , Pourtois , G. , Sander , D. , & Vuilleumier , P. ( 2011 ). Additive eff ects of emotional, endogenous, and exogenous attention: behavioral and electrophysiological evidence.
Neuropsychologia , 49 , 1779–1787 .
Brosch , T. , Sander , D. , Pourtois , G. , & Scherer , K. R. ( 2008 ). Beyond fear: Rapid spatial orienting towards positive emotional stimuli. Psychological Science , 19 , 362–370 .
Brosch , T. , Sander , D. , & Scherer , K. R. ( 2007 ). Th at baby caught my eye. . . Attention capture by infant faces . Emotion , 7 , 685–689 .
Brosch , T. , & Van Bavel , J. J. ( 2012 ). Th e fl exibility of emotional attention: Accessible social identities guide rapid attentional orienting. Cognition , 125 , 309–316 .
Calder , A. J. , Beaver , J. D. , Winston , J. S. , Dolan , R. J. , Jenkins , R. , Eger , E. , Henson , N. A. ( 2007 ).
Separate coding of diff erent gaze directions in the superior temporal sulcus and inferior parietal lobule. Current Biology , 17 , 20–25 .
Calder , A. J. , Rhodes , G. , Johnson , M. , & Haxby , J. V. ( 2011 ). Oxford Handbook of Face Perception . Oxford : Oxford University Press .
Corbetta , M. , & Shulman , G. L. ( 2002 ). Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews Neuroscience , 3 , 201–215 .
Cunningham , W. A. , & Brosch , T. ( 2012 ). Motivational salience: Amygdala tuning from traits, needs, values, and goals. Current Directions in Psychological Science , 21 , 54–59 .
Desimone , R. , & Duncan , J. ( 1995 ). Neural mechanisms of selective visual attention. Annual Review of Neuroscience , 18 , 193–222 .
Driver , J. ( 2001 ). A selective review of selective attention research from the past century. British Journal of Psychology , 92 , 53–78 .
Egeth , H. E. , & Yantis , S. ( 1997 ). Visual attention: Control, representation, and time course. Annual Review of Psychology , 48 , 269–297 .
Emery , N. J. ( 2000 ). Th e eyes have it: Th e neuroethology, function and evolution of social gaze.
Neuroscience and Biobehavioral Reviews , 24 , 581–604 .
Fazio , R. H. ( 1986 ). How do attitudes guide behavior? In R. M. Sorrentino & E. T. Higgins (Eds.),
Handbook of Motivation and Cognition: Foundations of Social Behavior (pp. 204–243 ). New York, NY : Guilford Press .
Fox , E. , Russo , R. , & Georgiou , G. A. ( 2005 ). Anxiety modulates the degree of attentive resources required to process emotional faces. Cognitive, Aff ective & Behavioral Neuroscience , 5 , 396–404 .
Friesen , C. K. , & Kingstone , A. ( 1998 ). Th e eyes have it! Refl exive orienting is triggered by nonpredictive gaze. Psychonomic Bulletin & Review , 5 , 490–495 .
Frischen , A. , Bayliss , A. P. , & Tipper , S. P. ( 2007 ). Gaze cueing of attention: Visual attention, social cognition, and individual diff erences. Psychological Bulletin , 133 , 694–724 .
Frischen , A. , Eastwood , J. D. , & Smilek , D. ( 2008 ). Visual search for faces with emotional expressions.
Psychological Bulletin , 134 , 662–676 .
Graham , R. , & Labar , K. S. ( 2012 ). Neurocognitive mechanisms of gaze-expression interactions in face processing and social attention. Neuropsychologia , 50 , 553–566 .
Holmes , A. , Mogg , K. , Garcia , L. M. , & Bradley , B. P. ( 2010 ). Neural activity associated with attention orienting triggered by gaze cues: A study of lateralized ERPs. Social Neuroscience , 5 , 285–295 .
Hood , B. M. , Willen , J. D. , & Driver , J. ( 1998 ). Adult’s eyes trigger shift s of visual attention in human infants. Psychological Science , 9 , 131–134 .
Ishai , A. ( 2008 ). Let’s face it: It’s a cortical network. Neuroimage , 40 , 415–419 .
Jones , B. C. , DeBruine , L. M. , Little , A. C. , Burriss , R. P. , & Feinberg , D. R. ( 2007 ). Social transmission of face preferences among humans. Proceedings of the Royal Society B: Biological Sciences , 274 , 899–903 .
Kobayashi , H. , & Kohshima , S. ( 1997 ). Unique morphology of the human eye. Nature , 387 , 767–768 .
MacLeod , C. , Mathews , A. , & Tata , P. ( 1986 ). Attentional bias in emotional disorders. Journal of Abnormal Psychology , 95 , 15–20 .
Mansfi eld , E. M. , Farroni , T. , & Johnson , M. H. ( 2003 ). Does gaze perception facilitate overt orienting?
Visual Cognition , 10 , 7–14 .
Marois , R. , & Ivanoff , J. ( 2005 ). Capacity limits of information processing in the brain. Trends in Cognitive Sciences , 9 , 296–305 .
Mogg , K. , & Bradley , B. P. ( 1999 ). Orienting of attention to threatening facial expressions presented under conditions of restricted awareness. Cognition and Emotion , 13 , 713–740 .
Nummenmaa , L. , & Calder , A. J. ( 2009 ). Neural mechanisms of social attention. Trends in Cognitive Sciences , 13 , 135–143 .
Öhman , A. , & Mineka , S. ( 2001 ). Fears, phobias, and preparedness: Toward an evolved module of fear and fear learning. Psychological Review , 108 , 483–522 .
Olofsson , J. K. , Nordin , S. , Sequeira , H. , & Polich , J. ( 2008 ). Aff ective picture processing: An integrative review of ERP fi ndings. Biological Psychology , 77 , 247–265 .
Posner , M. I. ( 1980 ). Orienting of attention. Quarterly Journal of Experimental Psychology , 32 , 3–25 .
Posner , M. I. , Snyder , C. R. , & Davidson , B. J. ( 1980 ). Attention and the detection of signals. Journal of Experimental Psychology , 109 , 160–174 .
Pourtois , G. , Grandjean , D. , Sander , D. , & Vuilleumier , P. ( 2004 ). Electrophysiological correlates of rapid spatial orienting towards fearful faces. Cerebral Cortex , 14 , 619–633 .
Pourtois , G. , Schettino , A. , & Vuilleumier , P. ( 2013 ). Brain mechanisms for emotional infl uences on perception and attention: What is magic and what is not . Biological Psychology , 92 (3), 492–512.
Pourtois , G. , Schwartz , S. , Seghier , M. L. , Lazeyras , F. , & Vuilleumier , P. ( 2006 ). Neural systems for orienting attention to the location of threat signals: An event-related fMRI study. Neuroimage , 31 , 920–933 .
Pourtois , G. , Th ut , G. , Grave de Peralta , R. , Michel , C. , & Vuilleumier , P. ( 2005 ). Two electrophysiological stages of spatial orienting towards fearful faces: Early temporo-parietal activation preceding gain control in extrastriate visual cortex. Neuroimage , 26 , 149–163 .
Putman , P. , Hermans , E. , & Van Honk , J. ( 2006 ). Anxiety meets fear in perception of dynamic expressive gaze. Emotion , 6 , 94–102 .
Raymond , J. E. , Shapiro , K. L. , & Arnell , K. M. ( 1992 ). Temporary suppression of visual processing in an RSVP task: An attentional blink? Journal of Experimental Psychology: Human Perception and Performance , 18 , 849–860 .
Rotshtein , P. , Henson , R. N. , Treves , A. , Driver , J. , & Dolan , R. J. ( 2005 ). Morphing Marilyn into Maggie dissociates physical and identity face representations in the brain. Nature Neuroscience , 8 , 107–113 .
Sander , D. , Grafman , J. , & Zalla , T. ( 2003 ). Th e human amygdala: An evolved system for relevance detection. Reviews in the Neurosciences , 14 , 303–316 .
Scaife , M. , & Bruner , J. S. ( 1975 ). Th e capacity for joint visual attention in the infant. Nature , 253 , 265–266 .