SENSATION AND PERCEPTION
3.2 Audition
3.2.1 Auditory System
The sensory receptors for hearing are sensitive to sound waves, which are moment-to-moment fluctuations in air pressure about the atmospheric level. These fluctuations are produced by mechanical disturbances, such as a stereo speaker moving in response to signals that it is receiving from a music source and amplifier. As the speaker moves forward and then back, the disturbances in the air go through phases of compression, in which the density of molecules—and hence the air pressure—is increased, and rarefaction, in which the
density and air pressure decrease. With a pure tone, such as that made by a tuning fork, these changes follow a sinusoidal pattern. The frequency of the oscillations (i.e., the number of oscillations per second) is the primary determinant of the sound’s pitch, and the amplitude or intensity is the primary determinant of loudness. Intensity is usually specified in decibels (dB), which is 20 log(p/p0), where p is the pressure corresponding to the sound and p0 is the standard value of 20 μPa. When two or more pure tones are combined, the resulting sound wave will be an additive combination of the components. In that case, not only frequency and amplitude become important but also the phase relationships between the components, that is, whether the phases of the cycles for each are matched or mismatched. The wave patterns for most sounds encountered in the world are quite complex, but they can be characterized in terms of component sine waves by means of a Fourier analysis. The auditory system must perform something like a Fourier analysis, since we are capable to a large extent of extracting the component frequencies that make up a complex sound signal, so that the pitches of the component tones are heard.
Ear A sound wave propagates outward from its source at the speed of sound (344 m/s), with the amplitude proportional to 1/(distance)2. It is the cyclical air pressure changes at the ear as the sound wave propagates past the observer that starts the sensory process. The outer ear (see Figure 10), consisting of the pinna and the auditory canal, serves to funnel the sound into the middle ear; the pinna will amplify or attenuate some sounds as a function of the direction from which they come and their frequency, and the auditory canal amplifies sounds in the range of approximately 1–2 kHz. A flexible membrane, called the eardrum ortympanic membrane, separates the outer and middle ears. The pressure in the middle ear is maintained at the atmospheric level by means of the Eustachian tube, which opens into the throat, so any deviations
External auditory canal Pinna
Skin
Bone Malleus
Incus
Semicircular canals
Oval window
Nerve to brain
Sensory cells of the cochlea Eustachian tube
Round window Stapes
Eardrum Path of sound waves
Figure 10 Structure of the human ear. (From Schiffman, 1996.)
SENSATION AND PERCEPTION 77 from this pressure in the outer ear will result in a
pressure differential that causes the eardrum to move.
Consequently, the eardrum vibrates in a manner that mimics the sound wave that is affecting it. However, changes in altitude, such as those occurring during flight, can produce a pressure differential that impairs hearing until that differential is eliminated, which cannot occur readily if the Eustachian tube is blocked by infection or other causes.
Because the inner ear contains fluid, there is an impedance mismatch between it and the air that would greatly reduce the fluid movement if the eardrum acted on it directly. This impedance mismatch is overcome by a lever system of three bones (the ossicles) in the middle ear: themalleus, incus,andstapes. The malleus is attached to the eardrum and is connected to the stapes by the incus. The stapes has a footplate that is attached to a much smaller membrane, the oval window, which is at the boundary of the middle ear and the cochlea, the part of the inner ear that is important for hearing. Thus, when the eardrum moves in response to sound, the ossicles move, and the stapes produces movement of the oval window. Muscles attached to the ossicles tighten when sounds exceed 80 dB, thus protecting the inner ear to some extent from loud sounds by lessening their impact.
However, because this acoustic reflex takes between 10 and 150 ms to occur, depending on the intensity of the sound, it does not provide protection from percussive sounds such as gunshots.
The cochlea is a fluid-filled, spiral structure (see Figure 11). It consists of three chambers, the vestibular and tympanic canals, and the cochlear duct, which separates them except at a small hole at the apex called thehelicotrema. Part of the wall separating the cochlear duct from the tympanic canal is a flexible membrane called thebasilar membrane. This membrane is narrower and stiffer nearer the oval window than it is nearer the helicotrema. The organ of Corti, the receptor organ that transduces the pressure changes to neural impulses, sits on the basilar membrane in the cochlear duct. It contains two groups of hair cells whose cilia project into the fluid in the cochlear duct and either touch
or approach the tectorial membrane, which is inflexible.
When fluid motion occurs in the inner ear, the basilar membrane vibrates, causing the cilia of the hair cells to be bent. It is this bending of the hair cells that initiates a neural signal. One group of hair cells, the inner cells, consists of a single row of approximately 3500 cells; the other group, the outer cells, is composed of approximately 12,000 hair cells arranged in three to five rows. The inner hair cells are mainly responsible for the transmission of sound information from the cochlea to the brain; the outer hair cells act as a cochlear amplifier, increasing the movement of the basilar membrane at frequencies contained in the sounds being received (Hackney, 2010). Permanent hearing loss most often is due to hair cell damage that results from excessive exposure to loud sounds or to certain drugs.
Sound causes a wave to move from the base of the basilar membrane, at the end near the oval window, to its apex. Because the width and thickness of the basilar membrane vary along its length, the magnitude of the displacement produced by this traveling wave at different locations will vary. For low-frequency sounds, the greatest movement is produced near the apex; as the frequency increases, the point of maximal displacement shifts toward the base. Thus, not only does the frequency with which the basilar membrane vibrates vary with the frequency of the auditory stimulus, but so does the location.
Auditory Pathways The auditory pathways after sensory transduction show many of the same properties as the visual pathways. The inner hair cells have synapses with the neurons that make up the auditory nerve. The neurons in the auditory nerve show frequency tuning. Each has a preferred or characteristic frequency that corresponds to the location on the basilar membrane of the hair cell from which it receives input but will fire less strongly to a range of frequencies about the preferred one. Neurons can be found with characteristic frequencies for virtually every frequency in the range of hearing. The contour depicting sensitivity of a neuron to different tone frequencies is called a tuning curve.
Cochlea
Reissner’s membrane
Basilar membrane Vestibular canal
Cochlear duct Tympanic canal
Helicotrema
Figure 11 Schematic of the cochlea uncoiled to show the canals. (From Schiffman, 1996.)
The tuning curves typically are broad, indicating that a neuron is sensitive to a broad range of values, but asymmetric: The sensitivity to frequencies higher than the characteristic frequency is much less than that to frequencies below it. With frequency held constant, there is a dynamic range over which as intensity is increased the neuron’s firing rate will increase. This dynamic range is on the order of 25 dB, which is considerably less than the full range of intensities that we can perceive.
The first synapse for the nerve fibers after the ear is the cochlear nucleus, After that point, two separate pathways emerge that seem to have different roles, as in vision. Fibers from the anterior cochlear nucleus go to the superior olive, half to the contralateral side of the brain and half to the ipsilateral side, and then on to the inferior colliculus. This pathway is presumed to be involved in the analysis of spatial information. Fibers from the posterior cochlear nucleus project directly to the contralateral inferior colliculus.
This pathway analyzes the frequency of the auditory stimulus. From the inferior colliculus, most of the neurons project to the medial geniculate and then to the primary auditory cortex. Frequency tuning is evident for neurons in all of these regions, with some neurons responding to relatively complex features of stimulation. The auditory cortex has a tonotopic organization, in which cells responsive to similar frequencies are located in close proximity, and contains
neurons tuned to extract complex information. As with vision, the signals from the auditory cortex follow two processing streams (Rauschecker, 2010). The posterior- dorsal stream analyzes where a sound is located, whereas the anterior-ventral stream analyzes what the sound represents.
3.2.2 Basic Auditory Perception
Loudness and Detection of Sounds Loudness for audition is the equivalent of brightness for vision.
More intense auditory stimuli produce greater amplitude of movement in the eardrum, which produces higher amplitude movement of the stapes on the oval window, which leads to bigger waves in the fluid of the inner ear and hence higher amplitude movements of the basilar membrane. Thus, loudness is primarily a function of the physical intensity of the stimulus and its effects on the ear, although as with brightness, it is affected by many other factors. The relation between judgments of loudness and intensity follows the power function
L=aI0.6
whereL is loudness,a is a constant, andI is physical intensity.
Just as brightness is affected by the spectral proper- ties of light, loudness is affected by the spectral proper- ties of sound. Figure 12 showsequal-loudness contours
120
Feeling
100
80
60
40
20
0
20 100 500 1000
Frequency (Hz) Sound pressure level (dB from 0.0002 dyn/cm2)
5000 10000 Loudness
level (phons) 120
110 100 90 80 70 60 50 40 30 20 10 0
Figure 12 Equal-loudness contours. Each contour represents the sound pressure level at which a tone of a given frequency sounds as loud as a 1000-Hz tone of a particular intensity. (From Schiffman, 1996.)
SENSATION AND PERCEPTION 79 for which a 1000-Hz tone was set at a particular inten-
sity level and tones of other frequencies were adjusted to match its loudness. The contours illustrate that humans are relatively insensitive to low-frequency tones below approximately 200 Hz and, to a lesser extent, to high- frequency tones exceeding approximately 6000 Hz. The curves tend to flatten at high intensity levels, particularly in the low-frequency end, indicating that the insensi- tivity to low-frequency tones is a factor primarily at low intensity levels. This is why most audio ampli- fiers include a “loudness” switch for enhancing low- frequency sounds artificially when music is played at low intensities. The curves also show the maximal sensi- tivity to be in the range 3000–4000 Hz, which is critical for speech perception. The two most widely cited sets of equal-loudness contours are those of Fletcher and Mun- son (1933), obtained when listening through earphones, and of Robinson and Dadson (1956), obtained for free- field listening.
Temporal summation can occur over a brief period of approximately 200 ms, meaning that loudness is a function of the total energy presented for tones of this duration or less. The bandwidth (i.e., the range of the frequencies in a complex tone) is important for determining its loudness. With the intensity held constant, increases in bandwidth have no effect on loudness until a critical bandwidth is reached. Beyond the critical bandwidth, further increases in bandwidth result in increases in loudness.
Extraneous sounds in the environment can mask targeted sounds. This becomes important for situations such as work environments, in which audibility of specific auditory input must be evaluated with respect to the level of background noise. The degree of masking is dependent on the spectral composition of the target and noise stimuli. Masking occurs only from frequencies within the critical bandwidth. Of concern for human factors is that a masking noise will exert a much greater effect on sounds of higher frequency than on sounds of lower frequency. This asymmetry is presumed to arise primarily from the operation of the basilar membrane.
Pitch Perception Pitch is the qualitative aspect of sound that is a function primarily of the frequency of a periodic auditory stimulus. The higher the frequency, the higher the pitch. The pitch of a note played on a musical instrument is determined by what is called its fundamental frequency, but the note also contains energy at frequencies that are multiples of the fundamental frequency, called harmonics or overtones. Observers can resolve perceptually the lower harmonics of a complex tone but have more difficulty resolving the higher harmonics (Plomp, 1964). This is because the perceptual separation of the successive harmonics is progressively less as their frequency increases.
Pitch is also influenced by several factors in addition to frequency. A phenomenon of particular interest in human factors is that of the missing fundamental effect.
Here, the fundamental frequency can be removed, yet the pitch of a sound remains unaltered. This suggests that pitch is based on the pattern of harmonics and not just the fundamental frequency. This phenomenon
allows a person’s voice to be recognizable over the telephone and music to be played over low-fidelity systems without distorting the melody. The pitch of a tone also varies as a function of its loudness. Equal-pitch contours can be constructed much like equal-loudness contours by holding the stimulus frequency constant and varying its amplitude. Such contours show that as stimulus intensity increases, the pitch of a 3000-Hz tone remains relatively constant. However, tones whose frequencies are lower or higher than 3000 Hz show systematic decreases and increases in pitch, respectively, as intensity increases.
Two different theories were proposed in the nine- teenth century to explain pitch perception. According to Ernest Rutherford’s (1886)frequency theory, the criti- cal factor is that the basilar membrane vibrates at the frequency of an auditory stimulus. This in turn gets transduced into neural signals at the same frequency such that the neurons in the auditory nerve respond at the frequency of the stimulus. Thus, according to this view, it is the frequency of firing that is the neural code for pitch. The primary deficiency of frequency theory is that the maximum firing rate of a neuron is restricted to about 1000 spikes/s. Thus, the firing rate of individ- ual neurons cannot match the frequencies over much of the range of human hearing. Wever and Bray (1937) provided evidence that the range of the auditory spec- trum over which frequency coding could occur can be increased by neurons that phase lock and then fire in volleys. The basic idea is that an individual neuron fires at the same phase in the cycle of the stimulus but not on every cycle. Because many neurons are responsive to the stimulus, some neurons will fire on every cycle.
Thus, across the group of neurons, distinct volleys of firing will be seen that when taken together match the frequency of the stimulus. Phase locking extends the range for which frequency coding can be effective up to 4000–5000 Hz. However, at frequencies beyond this range, phase locking breaks down.
According to Hermann von Helmholtz’s (1877)place theory, different places on the basilar membrane are affected by different frequencies of auditory stimulation.
He based this proposal on his observation that the basilar membrane was tapered from narrow at the base of the cochlea to broad at its apex. This led him to suggest that it was composed of individual fibers, much like piano strings, that would resonate when the frequency of sound to which it was tuned occurred. The neurons that receive their input from a location on the membrane affected by a particular frequency would fire in its presence, whereas the neurons receiving their input from other locations would not. The neural code for frequency thus would correspond to the particular neurons that were being stimulated. However, subsequent physiological evidence showed that the basilar membrane is not composed of individual fibers.
Von B´ek´esy (1960) provided evidence that the basilar membrane operates in a manner consistent with both frequency and place theory. Basically, he demonstrated that waves travel down the basilar membrane from the base to the apex at a frequency corresponding to that of the tone. However, because the width and thickness
of the basilar membrane vary along its length, the magnitude of the traveling wave is not constant over the entire membrane. The waves increase in magnitude up to a peak and then decrease abruptly. Most important, the location of the peak displacement varies as a function of frequency. Low frequencies have their maximal displacement at the apex; as frequency increases, the peak shifts systematically toward the oval window.
Although most frequencies can be differentiated in terms of the place at which the peak of the traveling wave occurs, tones of less than 500–1000 Hz cannot be. Frequencies in this range produce a broad pattern of displacement, with the peak of the wave at the apex. Consequently, location coding does not seem to be possible for low-frequency tones. Because of the evidence that frequency and location coding both operate but over somewhat different regions of the auditory spectrum, it is now widely accepted that frequencies less than 4000 Hz are coded in terms of frequency and those above 500 Hz in terms of place, meaning that at frequencies within this range both mechanisms are involved.
3.3 Vestibular System and Sense