In most of the research discussed so far, the target was equally likely to appear anywhere within the visual display and so search was essentially random.
This is very different from the real world. Suppose you are outside in the garden looking for your cat. Your visual search would be highly selective – you would ignore the sky and focus mostly on the ground (and perhaps the trees). Thus, your search would involve top-down processes based on your knowledge of where cats are most likely to be found.
Ehinger et al. (2009) studied top-down processes in visual search. They recorded eye fixations of observers searching for a person in numerous real-world outdoor scenes. Observers typically fixated the regions of each scene most likely to be relevant (e.g., pavements or sidewalks) and ignored irrelevant regions (e.g., sky, trees) (see Figure 5.11). Observers also fixated locations that differed considerably from neighbouring regions and areas containing visual features characteristic of a human figure.
Figure 5.11
The first three eye fixations made by observers searching for pedestrians. As can be seen, the great majority of their fixations were on regions in which pedestrians would most likely be found. Observers’
fixations were much more similar in the left-hand photo than in the right-hand one, because there are fewer likely regions in the left-hand one.
From Ehinger et al. (2009). Reprinted with permission from Taylor & Francis.
How can we account for these findings as well as those discussed earlier? Wolfe et al. (2011b) put forward a dual-path model (see Figure 5.12). There is a selective pathway of limited capacity (indicated by the bottleneck) in which objects are individually selected for recognition. This pathway has been the subject of most research until recent times.
There is also a non-selective pathway in which the “gist” or essence of a scene is processed. Such processing can then help to direct or guide processing within the selective pathway (represented by the arrow labelled “Guidance”). This pathway is of far more use in the real world than traditional laboratory research. It allows us to take advantage of our stored knowledge of the environment (e.g., the typical layout of a kitchen).
Findings
Wolfe et al. (2011a) compared visual search for objects presented within a scene setting or at random locations against a white background. As predicted, the rate of search was much faster in the scene setting (10 ms per item vs. 40 ms per item, respectively). They explained this difference in terms of what they called “functional set size” – visual search in scenes is efficient because observers can eliminate most regions because they are very unlikely to contain the object.
Figure 5.12
A two-pathway model of visual search. The selective pathway is capacity limited and can bind stimulus features and recognise objects. The non-selective pathway processes the gist of scenes. Selective and non-selective processing occur in parallel to produce effective visual search.
From Wolfe et al. (2011b). Reprinted with permission from Elsevier.
Strong support for the notion of functional set size was reported by Võ and Wolfe (2012) in a study in which observers detected objects in scenes (e.g., a jam jar in a kitchen). The key finding was that 80% of each scene was rarely fixated by observers. For example, they did not look at the sink or cupboard
doors when searching for a jam jar.
The findings of Võ and Wolfe (2012) show we can use our general knowledge of scenes to facilitate visual search. Hollingworth (2012) wondered whether specific knowledge of scenes would also enhance visual search. Their participants performed a visual search task for objects in scenes. Some had previously viewed the scenes with instructions to decide which object was least likely to appear in a scene of that type. Visual search was significantly faster when the specific scenes used had been seen previously, indicating the usefulness of specific scene knowledge.
More evidence that learning where targets are likely to be found often plays a major role in visual search was reported by Chukoskie et al. (2013). An invisible target was presented at random locations within a relatively small area on a blank screen. Observers were rewarded by hearing a tone when they fixated the target. There was a strong learning effect – fixations were initially distributed across the entire screen but rapidly became increasingly focused on the area within which the target might be present (see Figure 5.13).
Evaluation
Our knowledge of likely and unlikely locations of any given object in a scene is the most important determinant of visual search in the real world. The dual-path model is an advance on most previous theories of visual search in that it fully recognises the importance of scene knowledge. The notion that scene knowledge facilitates visual search by reducing functional set size is an important theoretical contribution that has received much support.
What are the limitations of the dual-path model? First, the processes involved in using gist knowledge of a scene extremely rapidly to reduce the search area remain unclear. Second, there is insufficient focus on the learning processes that often greatly facilitate visual search. For example, experts in several domains detect target information faster and more accurately than non-experts (see Gegenfurtner et al., 2011; see also Chapter 12). The model would need to be developed further to account for such effects.
Figure 5.13
The region of the screen fixated on early trials (blue circles) and on later trials (red circles).
From Chukoskie et al. (2013). © National Academy of Sciences. Reproduced with permission.
CROSS-MODAL EFFECTS
Nearly all the research discussed so far is limited in that the visual (or auditory) modality was studied on its own. This approach has been justified on the grounds that attentional processes in each sensory modality (e.g., vision, hearing) operate independently from those in other modalities. That assumption is incorrect. In the real world, we often coordinate information from two or more sense modalities at the same time (cross-modal attention). An example is lip reading, in which we use visual information about a speaker’s lip movements to facilitate our understanding of what they are saying (see Chapter 9).
KEY TERM
Cross-modal attention
The coordination of attention across two or more modalities (e.g., vision and audition).
Suppose we present participants with two streams of lights (as was done by Eimer and Schröger (1998)), with one stream of lights being presented to the left and the other to the right. At the same time, we also present participants with two streams of sounds (one to each side). In one condition, participants detect deviant visual events (e.g., longer than usual stimuli) presented to one side only. In the other condition, participants detect deviant auditory events in only one stream.
Event-related potentials (ERPs; see Glossary) were recorded to obtain information about the allocation of attention. Unsurprisingly, Eimer and Schröger (1998) found ERPs to deviant stimuli in the relevant modality were greater to stimuli presented on the to-be-attended side than the to-be-ignored side. Thus, participants allocated attention as instructed. Of more interest is what happened to the allocation of attention in the irrelevant modality. Suppose participants detected visual targets on the left side. In that case, ERPs to deviant auditory stimuli were greater on the left side than the right. This is a cross-modal effect in which the voluntary or endogenous allocation of visual attention also affected the allocation of auditory attention. In similar fashion, when participants detected auditory targets on one side, ERPs to deviant visual stimuli on the same side were greater than ERPs to those on the opposite side.
Thus, the allocation of auditory attention influenced the allocation of visual attention as well.