Whereas these paradigms were initially developed to delineate the operations of dif- ferent attention systems using highly controlled and artifi cial stimuli (such as letters or arrows), they have been quickly adapted to investigate the attentional prioritization of emotionally and socially relevant information. For example, in new versions of the visual search task, participants have been asked to search for faces with diff erent expressions, which has revealed more effi cient search for emotional faces as compared with neutral faces (see Frischen, Eastwood, & Smilek, 2008, for a review). Similarly, the attentional blink defi cit was shown to be greatly attenuated for faces with emotional expressions, which were reported with higher accuracy than neutral expressions when appearing as second target (Fox, Russo, & Georgiou, 2005). Th is indicates that important social
signals are selected preferentially from a perceptual temporal stream and have facilitated access to awareness.
One task that has been especially useful in elucidating the psychological principles and neurocognitive mechanisms underlying the eff ects of socio-aff ective information on attention is the dot probe task (MacLeod, Mathews, & Tata, 1986, see Fig. 6.2a). In this task, participants must detect a target that replaces one of two simultaneously pre- sented cues. One of these cues may, for example, be a face with an emotional expres- sion, the other one a face with a neutral expression. Importantly, the cues are equated on basic physical properties such as brightness, contrast, and color, so that any observed attentional eff ect is not due to exogenous attention capture based on low-level stimulus diff erences, but can be attributed to the aff ectivity of the cues. As in the exogenous cue- ing paradigm, the cue is not predictive of target location; the emotional expression is thus task-irrelevant. Typically, participants show faster responses to targets replacing the emotional rather than the neutral cue, which has been demonstrated both for the visual
Fixation (a)
(b) Fixation
Cue
Cue
Target
Target time
time
Fig. 6.2 Attentional cueing paradigms from social cognitive research showing (a) attention capture by socio-affective signals, and (b) attention shifts based on other people’s eye gaze.
MECHANISMS OF ATTENTION CAPTURE BY SOCIO-AFFECTIVE INFORMATION 83
(Brosch, Sander, & Scherer, 2007) and the auditory modality (Bertels, Kolinsky, & Morais, 2010). Th ese eff ects may even operate across sensory modalities, i.e., for visual targets following auditory cues (Brosch, Grandjean, Sander, & Scherer, 2009), suggesting that the automatic prioritization of important social information is organized supramodally across multiple sensory channels.
Many of the initial studies investigating attention capture by emotional information were conducted in the theoretical framework of the fear module , an evolved module subserving attentional prioritization of evolutionarily prepared potentially threatening (“fear-relevant”) stimuli such as angry faces, snakes, or spiders (Öhman & Mineka, 2001).
For example, some studies using the dot probe task compared the impact of fear-relevant (angry and fearful) and happy facial expressions on attentional prioritization and subse- quent target processing, and reported attention modulation by fear-relevant, but not by happy faces (Mogg & Bradley, 1999; Pourtois, Grandjean, Sander, & Vuilleumier, 2004).
Th ese and other results were used to support the theoretical claim that the rapid modula- tion of attention is restricted to fear-relevant stimuli (Öhman & Mineka, 2001).
We have proposed an alternative mechanism, claiming that the crucial determinant of attentional selection is a rapid appraisal of the emotional-motivational relevance of a stimulus for the needs, goals and values—in short: the well-being of the organism—
with threatening stimuli being merely one of many potentially relevant stimulus classes (Brosch, Sander, Pourtois, & Scherer, 2008). Th e relevance hypothesis of attention has several important theoretical implications: (1) instead of a modular specifi city to evolu- tionarily fear-relevant stimuli, it predicts attentional prioritization of a large number of emotional-motivationally relevant stimuli; and (2) instead of a hard-wired system based on a biological preparedness to fear certain kinds of stimuli, it predicts a high fl exibility of attentional prioritization, since the needs and goals of a perceiver may change frequently to refl ect the changing context and motivations.
In a series of studies we tested and provided empirical evidence for the two major claims of our model (Brosch et al., 2007; Brosch et al., 2008; Brosch & Van Bavel, 2012).
We fi rst set out to test whether not only fear-relevant, but also positive social stimuli are prioritized by attention. As mentioned earlier, most previous studies compared threaten- ing and happy facial expressions. However, it may be argued that happy expressions have a lower level of immediate relevance to the observer than threat-related expressions. In real life, if someone stares at you with an angry face, you need to worry about impend- ing aggression. If someone looks at you with a fearful face, you need to determine the cause of the fear in order to evaluate possible consequences for your own well-being.
In both cases, you need to rapidly allocate attention to prepare an appropriate response.
However, when someone smiles at you, whether from friendliness or personal happiness, no urgent response is required; the smile may be a safety signal, suggesting that you can relax. Th us, the response-demand characteristics of threatening expressions are a priori diff erent from those of expressions of happiness. Proponents of the fear module approach have suggested that threat-related stimuli are prioritized because of their long evolution- ary history and signifi cance (Öhman & Mineka, 2001). Th us, they should be compared
with positive social stimuli that have a similar phylogenetic basis. We thus investigated the attentional eff ect of photos of infant faces. Infants are a prototypical example of a class of social stimuli that are positive and highly biologically relevant. Th ey can be expected to elicit a phylogenetically based readiness for response preparation (as threat does) because appropriate behavior toward newborns, such as providing warmth and nurturance, is relevant for survival of the species. And indeed, congruent with our hypothesis (but incongruent with the theoretical account of the fear module) we observed rapid attention capture both for infant faces and for threatening expressions (Brosch et al., 2007; Brosch et al., 2008) with neuroimaging data indicating identical (or highly similar) underlying selection mechanisms.
In a second series of studies (Brosch & Van Bavel, 2012), we investigated the fl exibility of attentional prioritization mechanisms. Th e relevance hypothesis of attention predicts a high degree of fl exibility, since the relevance of a stimulus may change frequently to refl ect the changing motivational contingencies of the perceiver. To experimentally manipulate the social relevance of our stimuli, we capitalized on the important role of social catego- rization in human cognition (e.g., Brewer, 1988). Assigning people to a social group can lead to changes in evaluation and behavior toward in-group and out-group members.
In other words, targets are rapidly endowed with aff ective meaning when they are cat- egorized as social in-group or out-group members. Th us, we randomly assigned photos of individual faces to an in-group (the participants’ university) or out-group (another university) by presenting them on diff erent background colors during a learning phase, and then used these faces as cues in a dot probe task (see earlier in this section). All faces showed neutral expressions, so socio-aff ective relevance was not derived from emotional expressions, but from recent changes in social group membership. Our results revealed rapid attentional capture by images of out-group members, (Brosch & Van Bavel, 2012, experiment 1). Previous research had shown that white participants rapidly orient towards photos of black males, their racial out-group (Trawalter, Todd, Baird, & Richeson, 2008).
Th is eff ect had been interpreted in terms of the fear module: “the stereotype that young Black men are threatening and dangerous has become so robust and ingrained in the col- lective American unconscious that Black men now capture attention, much like evolved threats such as spiders and snakes” (Trawalter et al., 2008, p. 1322). However, we showed a conceptually similar pattern of results following a very brief 1-minute group-learning task. Th erefore, our fi ndings support the idea that attention mechanisms subserving the selection and prioritization of socio-aff ectively relevant aspects of the environment are not hard-wired to respond to threatening stimuli, but rapidly adapt to recent changes in the immediate motivational context.
Th ese fi ndings led us to suggest a potential underlying mechanism: attentional prior- itization of socio-aff ective information may be driven by the reactivation of aff ectively charged memory representations that were encoded as the result of a previous experience, in this case the association of a stimulus with a salient social group. When the stimulus is encountered again, these representations may allow for a fast evaluation of the stimulus (“appraisal shortcuts,” see Brosch, Pourtois, & Sander, 2010; Fazio, 1986), and subsequently
MECHANISMS OF ATTENTION CAPTURE BY SOCIO-AFFECTIVE INFORMATION 85
infl uence rapid attentional processes. To evaluate this hypothesis, in a second experiment we manipulated the accessibility of the socio-aff ective relevance of our stimuli. During the attention task, we presented photos that participants had previously encoded during the learning task (familiar in-group/out-group), but also photos of unfamiliar individuals displayed on the same background colors (unfamiliar in-group/out-group). We hypoth- esized that rapid attention capture would be observed for familiar, but not for unfamiliar out-group faces, which may require a more time-consuming “on-line” appraisal. Indeed, rapid, refl exive attentional orienting was specifi c to familiar out-group faces (i.e., faces for whom the out-group membership had been encoded in the learning phase). Th is fi nding is consistent with our suggestion that rapid attentional prioritization of socio-aff ective stimuli may be driven by the reactivation of aff ectively charged memory traces—appraisal shortcuts (Brosch & Van Bavel, 2012, experiment 2).
Taken together, attentional prioritization has been shown for a large number of socially relevant signals such as emotional expressions, facial confi gurations, and social group identities. Converging evidence suggests that the underlying mechanisms are based on a rapid automatic appraisal of stimulus relevance, are sensitive to multiple classes of potentially relevant socio-aff ective stimuli, and are highly fl exible to recent changes in motivational contingencies. Th is very rapid mechanism operates at the individual level, i.e., it serves to reorient the attentional focus of one person towards another person.
However, it may be conceptualized as a necessary precursor or even fi rst step of inter- personal communication that may cause collective emotion. It establishes, in an auto- matic fashion, a rudimentary form of contact between individuals, which is required for the interpersonal sharing of information, both about the internal state of the individu- als (e.g., via emotional contagion or empathy mechanisms, see Hess, Houde, & Fischer, Chapter 7, this volume; Lamm & Silani, Chapter 5, this volume; Hatfi eld, Carpenter, &
Rapson, Chapter 8, this volume) and about relevant events in the environment (e.g., via joint attention mechanisms, see “Joint attention: mechanisms of eye gaze cueing”
section).
Consistent with the behavioral fi ndings reviewed up to now, brain imaging studies have consistently revealed increased neural responses to many diff erent socio-aff ective stimuli compared to neutral stimuli, both in early sensory areas like primary visual cortex, and in higher-level regions associated with object and face recognition. Enhanced responses have been observed for emotional faces in the fusiform face area, emotional voices in the temporal voice area, and emotional body movements in the fusiform body area, suggest- ing a selective modulation of brain regions involved in the processing of specifi c stimu- lus categories (Vuilleumier & Brosch, 2009). Research using electroencephalography has yielded similar results, revealing modulatory eff ects of emotional expressions at several stages of cortical processing, including both early, sensory-related processes (as soon as 100 ms aft er stimulus onset) and later processes related to more elaborate evaluations of these stimuli, subsequent autonomic arousal, and/or memory formation (see Olofsson, Nordin, Sequeira, & Polich, 2008, for review). Th us, brain imaging and electrophysiologi- cal data converge to show that socio-aff ectively relevant stimuli are represented by more
robust neural signatures than neutral ones, and can consequently profi t from preferential access to further cognitive processing, behavior control and awareness.
It has been suggested that the prioritization of emotional information is driven by dedi- cated neural circuits (Brosch et al., 2011; Vuilleumier, 2005; Vuilleumier & Brosch, 2009), separate from the fronto-parietal networks that are involved in endogenous and exog- enous attention. In this model, the amygdala, a limbic region critically involved in the monitoring of the environment for emotionally relevant information (Cunningham &
Brosch, 2012; Sander, Grafman, & Zalla, 2003) is thought to play a critical role by modu- lating the processing of incoming sensory stimuli through direct feedback projections to visual cortex (Amaral, Behniea, & Kelly, 2003) and biasing signals to fronto-parietal atten- tion regions (Pourtois, Th ut, Grave de Peralta, Michel, & Vuilleumier, 2005). Consistent with this suggestion, several neuroimaging studies have reported that cortical increases were signifi cantly correlated with amygdala responses; i.e., the more the amygdala was sensitive to a stimulus, the more modulation was observed in sensory areas. Th e boost- ing by the amygdala may not only directly impact on sensory cortices, but it can also bias the fronto-parietal endogenous attention network toward the location of the stimulus, so that subsequent information arising at the same location as emotional cues will benefi t from enhanced processing resources. Functional magnetic resonance imaging record- ings during the emotional dot probe revealed greater activation in the IPS when targets were preceded by a fearful face than a neutral face, consistent with enhanced attentional orienting (Pourtois, Schwartz, Seghier, Lazeyras, & Vuilleumier, 2006). Taken together, neuroimaging work has demonstrated how socio-aff ective stimuli can induce a distinc- tive cascade of neural events which does not only boost the processing of the stimulus itself, but also infl uences mechanisms responsible for orienting and shift ing attention in space, such that subsequent information arising at the same location as a socio-aff ective cue will also benefi t from enhanced processing resources (see also Pourtois, Schettino, &
Vuilleumier, 2013; Vuilleumier & Brosch, 2009).