I returned to Singapore in the summer of 2009 to work with Annett Schirmer and have been working with Annett ever since. The first empirical chapter then begins with the observation that people in the environment are constantly moving.
General Introduction
Overview
Third, I analyzed in detail a particularly salient social stimulus—faces—and its neural representation in the amygdala. Finally, I directly compared social stimuli from people and faces with non-social stimuli of e.g. food, gadgets and electronics in an attention-oriented visual search task.
Socially salient cues
- Eye gaze
- Faces
- Head direction
- Finger gestures
- Postures, actions and biological motion
- Personal distance
- Social touch
- Social rewards
Deficits in eye gaze processing are associated with complex disabilities such as autism spectrum disorder (Pelphrey et al., 2002). In healthy individuals, social and monetary reward learning share overlapping neural substrates ( Lin et al., 2012b ).
Neural representation of saliency
V4 neurons not only enhance responses to a preferred stimulus in their receptive field when the stimulus matches a target feature, but also enhance responses to candidate targets selected for saccades ( Bichot et al., 2005 ). Both in single saccade tasks (Chelazzi et al., 1993, Tolias et al., 2001, Ogawa and Komatsu, 2004) and tasks with natural science free viewing (Sheinberg and Logothetis, 2001, Mazer and Gallant, 2003, Bichot et al., 2005 ), several studies have reported that temporal cortical neurons increase responses to visual stimuli presaccadicly when a stimulus in the receptive field becomes a target, suggesting that task-relevant target salience is encoded by these neurons.
Saliency and the amygdala
In rats, rapid strengthening of thalamo-amygdala synapses mediates cue-reward learning ( Tye et al., 2008 ). Overall, the amygdala may function as a detector of perceptual salience and biological relevance (Sander et al., 2005, Adolphs, 2008).
Saliency and autism
Furthermore, two-year-olds with autism orient to non-social events rather than biological movement (Klin et al., 2009). However, people with autism show a disproportionate impairment in learning to choose social rewards compared to monetary rewards (Lin et al., 2012a).
Amygdala theory of autism
People with autism also have reduced preference and sensitivity for donations to human charities compared to donations to the other charities (Lin et al., 2012c). It is therefore plausible that autism may be partly caused by an amygdala abnormality (Baron-Cohen et al., 2000).
Thesis overview
It is also noteworthy that a bona fide lesion of the amygdala shows no autistic symptoms by clinical examination and autism diagnosis (Paul et al., 2010). In Chapter I, I reviewed the social cues and the neural structures, particularly the amygdala, involved in the processing of salience.
Spatial Attention is Attracted in a Sustained Fashion towards Singular
Overview
It is important to discuss the neural computations underlying this distinct non-social cue—the optic flow, which can be contrasted with the neural computations underlying social salience. We will directly compare social and non-social cues in Chapter III, where we also test the dependence of social salience on the amygdala—a key neural structure of the "social brain."
Summary
Introduction
Motion, the change in object size over time (or temporal size gradient, TSG) and the spatial depth structure implied by object size distribution (or spatial size gradient, SSG) correspond to one of the three axes of the cube. We found strong attentional effects only when the optical flow of the scene expanded (i.e., by zooming toward a single point in the scene), but not when it contracted (i.e., zoomed away from the point).
Methods
- Experiment 1: Speeded discrimination under background dot motion
- Subjects and apparatus
- Task
- Stimuli
- Data analysis
- Experiment 2: Change detection with zooming in and out
- Subjects and apparatus
- Zooming algorithm
- Procedure
- Stimuli
- Data analysis
In the “SSG on” condition, the size of the dots increased gradually in proportion to the distance from the singular dot in the first frame of the stimulus movie. In condition 2, stationary points with size gradient imply a 3D scene structure, which implies the location of the single point.
Results
- Experiment 1: Speeded discrimination under background dots
- Motion is a strong cue while TSG and SSG act as auxiliary cues
- Target eccentricity and SOA on the attentional effects
- Laterality of the attentional effects
- Analysis of error trials
- Experiment 2: Change detection with zooming in and out
Fifth, we compared the magnitude of the attention effects between the expanding and contracting conditions (Figure 2.4B, the red and blue bars are for the expanding and contracting motion, respectively). Dependence of the attention effects on target eccentricity (A,B,E,F) and SOA (C,D,G,H) for all conditions averaged (A–D) and for condition 8 (motion = on, SSG = on, TSG = on) (E–H).
Discussion
- Attention is attracted towards the singular point defined by the expansive, but not
- Sustained attentional effects
- Mechanisms of computation of the FOE
- Advantage of our stimulus design
- Laterality effects of attention
Human psychophysical studies have shown perception of visual expansion without optic flow (Schrater et al., 2001), indicating that judgment of size (or scale) change is independent of local translational motion. Behaviorally, lateralized effects have been reported for the sensory and cognitive processing of language, face, and emotion (MacNeilage et al., 2009). In normal subjects, a strong asymmetry in the attentional resolution between the upper and lower visual fields has been reported (He et al., 1996).
Conclusion
Acknowledgements
Preferential attention to animals and people is independent of the
Overview
Summary
Introduction
One category of salient stimuli that has recently been explored is animated (living) stimuli (New et al., 2007; Mormann et al., 2011). Furthermore, images of animals and humans are preferentially detected during change blindness tasks (New et al., 2007), an approach we took advantage of here. This raises the question of whether the strong neuronal responses tuned to animals in the amygdala (Mormann et al., 2011) have a behavioral consequence, such as increased attention to animals (New et al., 2007).
Methods
- Subjects
- Stimuli and apparatus
- Task
- Eye-tracking
- Data analysis
A subset of the images had been used in previous studies and showed reliably faster detection of animals and humans (New et al., 2007; New et al., 2010). The construction and validity of the stimuli, stimulus properties, and further control experiments using inverted stimuli have been discussed in previous studies ( New et al., 2007 , New et al., 2010 ). Patient SM and two matched controls performed the task with a subset of the stimuli (identical arrangement and stimuli as (New et al., 2010), which did not include the change of head direction category).
Results
- Phenomenological change blindness and conscious detectability
- Rapid detection of animate stimuli by explicit behavioral reports of change
- Implicit measures of change detection from eye-tracking
- Detection advantages to animals were not lateralized
There was no difference in the proportion of any of the above trial types between amygdala patients and matched controls (all t-tests ps > 0.05). No target category showed significant differences between amygdala patients and their matched controls (two-tailed t -tests, all ps > 0.14; bootstrap with 1000 trials, all ps > 0.32). for inanimate; see Table 3.2, Hit Rate Analysis, for statistics) and there was no difference between amygdala patients and control subjects.
Discussion
- Advantages of our change detection task and comparison with other tasks
- Possible caveats
- Lateralized effects of category attention
- Amygdala lesions and plasticity
- The role of the amygdala in attention and saliency
Change detection within the first second probably required the target object to be the first attended element in the scene ( New et al., 2007 ). It has been shown that contextual knowledge can drive change detection performance (e.g. (Rensink et al., 1997)) and, interestingly, as a function of semantic inconsistency (Hollingworth and Henderson, 2000). Indeed, evidence for compensatory functioning (on an unrelated task) has been reported in one of the patients we studied (Becker et al., 2012).
Conclusion
Socially relevant information in faces is largely expressed in the eye region, including gaze directions ( Argyle et al., 1973 , Whalen et al., 2004 ), and viewers fixate primarily on the eyes, a tendency normally associated with amygdala activation ( Gamer) and Büchel, 2009). A range of psychiatric disorders have abnormal fixations on faces, including abnormal fixations on the eye region of faces, and several of these are thought to involve the amygdala (Baron-Cohen et al., 2000, Baron-Cohen, 2004, Dalton et al. ., 2005). Although the amygdala is by no means eliminated as one structure contributing to social dysfunction in these diseases, the data from the present study argue that it may not play a key online role in those components involving orientation and attentional mechanisms.
Acknowledgements
Neurons in the human amygdala selective for perceived emotion
Overview
The role of the amygdala in social attention will be further explored in the next chapter.
Summary
Introduction
Patients with schizophrenia (Sasson et al., 2007), social phobia (Horley et al., 2004), and autism (Pelphrey et al., 2002) also show abnormal face scanning patterns, which have been hypothesized to result from amygdala dysregulation. Baron-Cohen et al., 2000). The functional role of the amygdala is supported by its connection with visual cortices specialized for face processing (Vuilleumier et al., 2004, Moeller et al., 2008, Hadj-Bouziane et al., 2012) as well as reciprocal connections with many visual areas. responsive in the temporal lobes (Desimone and Gross, 1979, Amaral et al., 2003, Freese and Amaral, 2006) and frontal lobes (Ghashghaei and Barbas, 2002). Neurons in the monkey and human amygdala respond to the emotional expression of faces, but it remains unknown whether these responses are driven primarily by the image properties of the stimuli, by the perceiver's perceptual judgments, or by the categorization of behavior in terms of motor output.
Methods
- Subjects
- Task
- Data analysis: spikes
- Data analysis: selection of emotion-selective and interactive units
- Data analysis: response index
- Data analysis: split analysis and permutation test
- Electrode localization from structural MRIs
- Eye tracking
The cumulative distribution function (CDF) (see Figure 4.8D and Figure 4.9A,C) was constructed by calculating for each possible value x of the response index how many examples are less than x. On each run, we randomly selected half of the appropriate trials to identify emotion-selective units and to determine neuron type (as described above). First, we selected trials based on the overlap of the bubbles with the specified eye and.
Results
- Behavioral performance
- Eye tracking
- Emotion-selective neurons
- Interactive neurons encode perceptual judgment of emotions other than ground truth
- Emotion-selective neurons encode perceptual judgment
- A full inventory of neurons in the amygdala that encode perceptual judgment
- Neuronal response characteristics dependent on facial information revealed
- Specificity of the amygdala neurons in coding subjective judgment
- Reaction time (RT) and laterality analysis
Thus, responses during incorrect trials tended to be similar to correct trials of the opposite emotional category. In contrast, the value of the population effect metric was 25.0% for correct trials (Figure 4.10, ed; p.). Importantly, the metric from the incorrect trials was significantly negative and thus at the opposite end of the null distribution compared to the metric from correct trials (Figure 4.10, blue).
We then calculated response indices for the remaining half of correct trials and all incorrect trials. The distribution of incorrect trials was shifted in the opposite direction relative to the distribution of correct trials.
Discussion
- Possible confounds
- Comparison with neuroimaging studies
- Selectivity of amygdala neurons
- Functional role of the amygdala
- The amygdala, consciousness and perception
Our findings are also consistent with previous research showing that presenting the eyes but not the mouth results in a significant BOLD response in the amygdala (Morris et al., 2002; Whalen et al., 2004). We have previously shown that neurons in the human amygdala respond selectively to whole faces compared to face parts, suggesting a predominant role of the amygdala in representing global information about faces (Rutishauser et al., 2011). Electrophysiological data from monkeys show that stimulation of face-selective areas in the temporal cortex (the anterior medial (AM) facial patch) can induce activation in the lateral nucleus of the amygdala ( Moeller et al., 2008 ).
Conclusion
Acknowledgements
Autism spectrum disorder, but not amygdala lesions, impairs social
Overview
Summary
Introduction
Connecting the above-mentioned abnormal eye fixations to faces in ASD, and a correlation with amygdala processing, functional neuroimaging studies have found associations between abnormal fixation behavior and abnormal amygdala activation in people with ASD ( Dalton et al., 2005 ; Kliemann et al., 2012). One recent study even found evidence for abnormal processing of information from the ocular region of faces in single cells recorded from the amygdala in neurosurgical patients with ASD (Rutishauser et al., 2013). Previous findings have shown, in children and adolescents (Sasson et al., 2008), as well as in 2-5 year olds (Sasson et al., 2011), that participants with ASD are less fixated on social images than controls when they freely view the arrays.
Methods
- Subjects
- Stimuli and apparatus
- Task
- Eye tracking
- Data analysis
For each group, we also had 2 catch trials, i.e., the target was not among the objects in the search array (one catch trial with a social target and one with a nonsocial target). The experimental setup of Experiment 2 was identical to Experiment 1, except that the low-level properties of social and nonsocial objects were equated within each search group. Social and nonsocial objects did not differ in standard low-level visibility as.
Results
- Behavioral Performance: Accuracy and Reaction Time
- Eye tracking: general social preference does not differ between subject groups 171
- The attentional deficit in ASD could not be explained by low-level visual properties
- The attentional deficit in ASD is more severe with high task demands
- Missing detection of targets was not prominent in amygdala lesion patients
NUS controls (two-tailed t-tests separately for social vs. non-social targets, all ps > 0.05). When low-level salience was equated between social and non-social objects, people with ASD showed completely normal general social preference compared to ASD controls (one-way ANOVA across three subject groups, p > 0.05 for all fixations and means; two-tailed -t-tests compared to ASD controls, p > 0.05 for all fixations and means; see Table 5.4). All subjects showed rapid and sustained target-relevant effects, for both social targets and non-social targets (Figure 5.7).
Discussion
- Visual search in autism
- Missing detection of targets and task difficulty
- The amygdala and saliency
- Amygdala theory of autism
Conclusion
Acknowledgements
Future Directions
Computational modeling of saliency
Investigating face perception
- Investigating how neurons in the amygdala respond to morphed faces
- Investigating faces along many dimensions
- Eye-tracking
Investigating saliency in visual search
More ecologically valid stimuli
Conclusion
