auditory masking auditory masking in audio signal processing, auditory masking occurs when the perception of one sound is affected by the presence of another sound. auditory masking in the frequency domain is known as simultaneous masking, frequency masking or spectral masking. auditory masking in the time domain is known as temporal masking or non-simultaneous masking. masked threshold the unmasked threshold is the quietest level of the signal which can be perceived without a masking signal present. the masked threshold is the quietest level of the signal perceived when combined with a specific masking noise. the amount of masking is the difference between the masked and unmasked thresholds. figure a – adapted from gelfand (2004) gelfand provides a basic example. let us say that for a given individual, the sound of a cat scratching a post in an otherwise quiet environment is first audible at a level of 10 db spl. however, in the presence of a masking noise (for example, a vacuum cleaner that is running simultaneously) that same individual cannot detect the sound of the cat scratching unless the level of the scratching sound is at least 26 db spl. we would say that the unmasked threshold for that individual for the target sound (i.e., the cat scratching) is 10 db spl, while the masked threshold is 26 db spl. the amount of masking is simply the difference between these two thresholds: 16 db. the amount of masking will vary depending on the characteristics of both the target signal and the masker, and will also be specific to an individual listener. while the person in the example above was able to detect the cat scratching at 26 db spl, another person may not be able to hear the cat scratching while the vacuum was on until the sound level of the cat scratching was increased to 30 db spl (thereby making the amount of masking for the second listener 20 db). simultaneous masking simultaneous masking occurs when a sound is made inaudible by a noise or unwanted sound of the same duration as the original sound. for example, a powerful spike at 1 khz will tend to mask out a lower-level tone at 1.1 khz. also, two sine tones at 440 and 450 hz can be perceived clearly when separated. they cannot be perceived clearly when presented simultaneously. critical bandwidth if two sounds of two different frequencies are played at the same time, two separate sounds can often be heard rather than a combination tone. the ability to hear frequencies separately is known as frequency resolution or frequency selectivity. when signals are perceived as a combination tone, they are said to reside in the same critical bandwidth. this effect is thought to occur due to filtering within the cochlea, the hearing organ in the inner ear. a complex sound is split into different frequency components and these components cause a peak in the pattern of vibration at a specific place on the cilia inside the basilar membrane within the cochlea. these components are then coded independently on the auditory nerve which transmits sound information to the brain. this individual coding only occurs if the frequency components are different enough in frequency, otherwise they are in the same critical band and are coded at the same place and are perceived as one sound instead of two. the filters that distinguish one sound from another are called auditory filters, listening channels or critical bandwidths. frequency resolution occurs on the basilar membrane due to the listener choosing a filter which is centered over the frequency they expect to hear, the signal frequency. a sharply tuned filter has good frequency resolution as it allows the center frequencies through but not other frequencies (pickles 1982). damage to the cochlea and the outer hair cells in the cochlea can impair the ability to tell sounds apart (moore 1986). this explains why someone with a hearing loss due to cochlea damage would have more difficulty than a normal hearing person in distinguishing between different consonants in speech. masking illustrates the limits of frequency selectivity. if a signal is masked by a masker with a different frequency to the signal, then the auditory system was unable to distinguish between the two frequencies. by experimenting with conditions where one sound can mask a previously heard signal, the frequency selectivity of the auditory system can be tested. similar frequencies figure b – adapted from ehmer how effective the masker is at raising the threshold of the signal depends on the frequency of the signal and the frequency of the masker. the graphs in figure b are a series of masking patterns, also known as masking audiograms. each graph shows the amount of masking produced at each masker frequency shown at the top corner, 250, 500, 1000 and 2000 hz. for example, in the first graph the masker is presented at a frequency of 250 hz at the same time as the signal. the amount the masker increases the threshold of the signal is plotted and this is repeated for different signal frequencies, shown on the x axis. the frequency of the masker is kept constant. the masking effect is shown in each graph at various masker sound levels. figure c – adapted from gelfand 2004 figure d – adapted from gelfand 2004 figure b shows along the y axis the amount of masking. the greatest masking is when the masker and the signal are the same frequency and this decreases as the signal frequency moves further away from the masker frequency. this phenomenon is called on-frequency masking and occurs because the masker and signal are within the same auditory filter (figure c). this means that the listener cannot distinguish between them and they are perceived as one sound with the quieter sound masked by the louder one (figure d). figure e – adapted from moore 1998 the amount the masker raises the threshold of the signal is much less in off-frequency masking, but it does have some masking effect because some of the masker overlaps into the auditory filter of the signal (figure e) figure f – adapted from moore 1998off-frequency masking requires the level of the masker to be greater in order to have a masking effect; this is shown in figure f. this is because only a certain amount of the masker overlaps into the auditory filter of the signal and more masker is needed to cover the signal. lower frequencies the masking pattern changes depending on the frequency of the masker and the intensity (figure b). for low levels on the 1000 hz graph, such as the 20–40 db range, the curve is relatively parallel. as the masker intensity increases the curves separate, especially for signals at a frequency higher than the masker. this shows that there is a spread of the masking effect upward in frequency as the intensity of the masker is increased. the curve is much shallower in the high frequencies than in the low frequencies. this flattening is called upward spread of masking and is why an interfering sound masks high frequency signals much better than low frequency signals. figure b also shows that as the masker frequency increases, the masking patterns become increasingly compressed. this demonstrates that high frequency maskers are only effective over a narrow range of frequencies, close to the masker frequency. low frequency maskers on the other hand are effective over a wide frequency range. figure g – adapted from a diagram by gelfand harvey fletcher carried out an experiment to discover how much of a band of noise contributes to the masking of a tone. in the experiment, a fixed tone signal had various bandwidths of noise centered on it. the masked threshold was recorded for each bandwidth. his research showed that there is a critical bandwidth of noise which causes the maximum masking effect and energy outside that band does not affect the masking. this can be explained by the auditory system having an auditory filter which is centered over the frequency of the tone. the bandwidth of the masker that is within this auditory filter effectively masks the tone but the masker outside of the filter has no effect (figure g). this is used in mp3 files to reduce the size of audio files. parts of the signals which are outside the critical bandwidth are represented with reduced precision. the parts of the signals which are perceived by the listener are reproduced with higher fidelity. effects of intensity figure h – adapted from moore 1998 varying intensity levels can also have an effect on masking. the lower end of the filter becomes flatter with increasing decibel level, whereas the higher end becomes slightly steeper. changes in slope of the high frequency side of the filter with intensity are less consistent than they are at low frequencies. at the medium frequencies (1–4 khz) the slope increases as intensity increases, but at the low frequencies there is no clear inclination with level and the filters at high center frequencies show a small decrease in slope with increasing level. the sharpness of the filter depends on the input level and not the output level to the filter. the lower side of the auditory filter also broadens with increasing level. these observations are illustrated in figure h. temporal masking temporal masking or non-simultaneous masking occurs when a sudden stimulus sound makes inaudible other sounds which are present immediately preceding or following the stimulus. masking which obscures a sound immediately preceding the masker is called backward masking or pre-masking and masking which obscures a sound immediately following the masker is called forward masking or post-masking. temporal masking's effectiveness attenuates exponentially from the onset and offset of the masker, with the onset attenuation lasting approximately 20 ms and the offset attenuation lasting approximately 100 ms. similar to simultaneous masking, temporal masking reveals the frequency analysis performed by the auditory system; forward masking thresholds for complex harmonic tones (e.g., a sawtooth probe with a fundamental frequency of 500 hz) exhibit threshold peaks (i.e., high masking levels) for frequency bands centered on the first several harmonics. in fact, auditory bandwidths measured from forward masking thresholds are narrower and more accurate than those measured using simultaneous masking. temporal masking should not be confused with the ear's acoustic reflex, an involuntary response in the middle ear that is activated to protect the ear's delicate structures from loud sounds. other masking conditions figure i – ipsilateral simultaneous masking ipsilateral ("same side") masking is not the only condition where masking takes place. another situation where masking occurs is called contralateral ("other side") simultaneous masking. in this case, the instance where the signal might be audible in one ear but is deliberately taken away by applying a masker to the other ear. the last situation where masking occurs is called central masking. this refers to the case where a masker causes a threshold elevation. this can be in the absence of, or in addition to, another effect and is due to interactions within the central nervous system between the separate neural inputs obtained from the masker and the signal. effects of different stimulus types experiments have been carried out to see the different masking effects when using a masker which is either in the form of a narrow band noise or a sinusoidal tone. when a sinusoidal signal and a sinusoidal masker (tone) are presented simultaneously the envelope of the combined stimulus fluctuates in a regular pattern described as beats. the fluctuations occur at a rate defined by the difference between the frequencies of the two sounds. if the frequency difference is small then the sound is perceived as a periodic change in the loudness of a single tone. if the beats are fast then this can be described as a sensation of roughness. when there is a large frequency separation, the two components are heard as separate tones without roughness or beats. beats can be a cue to the presence of a signal even when the signal itself is not audible. the influence of beats can be reduced by using a narrowband noise rather than a sinusoidal tone for either signal or masker. mechanisms of masking there are many different mechanisms of masking, one being suppression. this is when there is a reduction of a