Localization Uncertainty in Time-Amplitude Stereophonic Reproduction
Enzo De Sena Zoran Cvetkovic Huseyin Hacihabiboglu Marc Moonen Toon van Waterschoot
eess.AS






speaker


  PlaneWave/.pic=
  	(-1.5,0) – (1.5,0);
  	(-1.5,0.4) – (1.5,0.4);
  	(-1.5,0.8) – (1.5,0.8);
  	[thick,->] (0, 1.4) – (0,-0.5);
  ,
  Mic/.pic=
    [fill=gray!40,pic actions] 
    	(-0.15,0) rectangle (0.15,-0.8);
    [fill=gray] 
      (0,0) ellipse [x radius=0.25cm, y radius=0.25cm];
  ,
  Speaker/.pic=
    [fill=gray!40,pic actions] 
    (-15pt,0) – 
      coordinate[midway] (-front) 
    (15pt,0) – 
    ++([shift=(-6pt,8pt)]0pt,0pt) coordinate (aux1) – 
    ++(-18pt,0) coordinate (aux2) 
    – cycle 
    (aux1) – ++(0,6pt) – coordinate[midway] (-back) ++(-18pt,0) – (aux2);
  ,
  Human/.pic=
    [fill=skin] 
      (0,0.7cm) ellipse [x radius=7pt,y radius=3pt]
      (0,-0.7cm) ellipse [x radius=7pt,y radius=3pt];    
    [fill=skin] 
      (0.7,4pt) – (0.7,-4pt) – (0.95,0pt) – cycle;    
    [fill=skin] 
      (0,0) ellipse [x radius=0.8cm, y radius=0.7cm];
    [fill=brown!85!black] 
      (0.5,0.55) arc[start angle=50,end angle=310,x radius=0.8cm, y radius=0.7cm]
      to[out=60,in=230] cycle;
  






 




Copyright  2020 IEEE
IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 28, 2020De Sena et al.: Localization Uncertainty in Time-Amplitude Stereophonic Reproduction
Localization Uncertainty in Time-Amplitude Stereophonic Reproduction
    Enzo De Sena, Senior Member, IEEE, 
		Zoran Cvetković, Senior Member, IEEE, 
		Hüseyin Hacıhabiboğlu, Senior Member, IEEE, 
        Marc Moonen, Fellow, IEEE, 
		Toon van Waterschoot, Member, IEEE
Copyright  2020 IEEE. Personal use of this material is permitted. However, permission to use this material for any other purposes must be obtained from the IEEE by sending a request to pubs-permissions@ieee.org. https://ieeexplore.ieee.org/document/9004547

Manuscript received July 18, 2019; revised January 13, 2020; accepted February 12, 2020. Date of publication February 20, 2020.  
The work reported in this paper was partially funded by (i) EPSRC Grant EP/F001142/1, (ii) Turkish Scientific and Technological Research Council (TUBITAK) Research Grant 119U254 “Audio Signal Processing for Six Degrees of Freedom (6DOF) Immersive Media”,  (iii) KU Leuven internal funds C2-16-00449 “Distributed Digital Signal Processing for Ad-hoc Wireless Local Area Audio Networking”, (iv) European Commission under Grant Agreement no. 316969 within the FP7-PEOPLE Marie Curie Initial Training Network “Dereverberation and Reverberation of Audio, Music, and Speech (DREAMS)" (v) European Research Council under the European Union's Horizon 2020 research and innovation program / ERC Consolidator Grant: SONORA (no. 773268). 
This paper reflects only the authors' views, and the European Union is not liable for any use that may be made of the contained information.
Parts of this work were previously presented in <cit.>, <cit.> and <cit.>. 
The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Stefan Bilbao. (Corresponding author: Enzo De Sena.)

Enzo De Sena is with the Institute of Sound Recording at the University of Surrey (UK) (e.desena@surrey.ac.uk). Zoran Cvetković is with the Department of Engineering at King's College London (UK). Hüseyin Hacıhabiboğlu is with the Graduate School of Informatics, Middle East Technical University (METU) (Turkey). Marc Moonen and Toon van Waterschoot are with the Department of Electrical Engineering at KU Leuven (Belgium).

    December 30, 2023
=======================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================







This paper studies the effects of inter-channel time and level differences in stereophonic reproduction on perceived localization uncertainty, which is defined as how difficult it is for a listener to tell where a sound source is located.
Towards this end, a computational model of localization uncertainty is proposed first.
The model calculates inter-aural time and level difference cues, and compares them to those associated to free-field point-like sources. 
The comparison is carried out using a particular distance functional that replicates the increased uncertainty observed experimentally with inconsistent inter-aural time and level difference cues.
The model is validated by formal listening tests, achieving a Pearson correlation of 0.99.
The model is then used to predict localization uncertainty for stereophonic setups and a listener in central and off-central positions.
Results show that amplitude methods achieve a slightly lower localization uncertainty for a listener positioned exactly in the center of the sweet spot. 
As soon as the listener moves away from that position, the situation reverses, with time-amplitude methods achieving a lower localization uncertainty. 



Stereophony, panning, recording and reproduction, localization uncertainty, auditory modelling.





§ INTRODUCTION


Despite significant advancements in the field of multichannel audio <cit.>, the most common reproduction system in use today remains the two-channel stereophonic system.


In typical stereophonic panning, the two loudspeakers are positioned at ± 30^∘ with respect to the listener's look direction and reproduce delayed and attenuated versions of the same signals. 
The differences in time and level are typically frequency-independent, and are referred to as ictd and icld, respectively. 
For ictd smaller than 1 ms, 
the listener does not perceive the two loudspeaker signals as separate,
but rather a single fused auditory event, 
often referred to as “phantom source”.
The perceived location of the phantom source depends on both the ictd and icld.
This psychoacoustic effect is called “summing localization”, and is at the basis of stereophonic panning <cit.>. 

The phantom source can be moved using icld alone, i.e. amplitude panning, ictd alone, i.e. time panning, or both icld and ictd, i.e. time-amplitude panning. 

Panning is inherently linked to recording. 
Consider a plane wave impinging on two microphones, each connected to a loudspeaker without mixing.
The distance between the microphones dictates the ictd, while the ratio between the two directivity patterns, e.g. cardioid or figure-8, dictates the icld.



Recording with coincident microphones is equivalent to amplitude panning, while recording with non-coincident omnidirectional microphones is equivalent to time panning.
Recording with microphones that are neither omnidirectional nor coincident is equivalent to time-amplitude panning. For point-like sources, distance attenuation means that small, distance-dependent level differences will also be observed at non-coincident microphones.


Amplitude panning is widely used in sound mixing, with most mixing desks and digital audio workstation (DAW) software implementing a version of the sine/tangent law <cit.>. 
Amplitude recording methods are used in a wide variety of methods, e.g. the original Blumlein pair <cit.>,  Ambisonics <cit.>, and the spatial decomposition method <cit.>. 
While it is possible to pan a stereophonic image using time panning for certain signals, it provides higher localization uncertainty for sustained higher frequency stimuli <cit.>.



Time-amplitude recording methods <cit.>, on the other hand, are popular within the audio engineering community, 
mainly for their strong sense of spaciousness,
which may be attributed to the higher decorrelation 
between the microphone signals <cit.>.
Widely-spaced microphones with inter-microphone distances in the order of meters, while extensively used, are often criticized for 
their unstable imaging <cit.> and irregular distribution of reproduced auditory events <cit.>. 

Near-coincident stereophonic microphones such as the ORTF (Office de Radiodiffusion Télévision Française), NOS (Nederlandse Omroep Stichting) and DIN (Deutsches Institut für Normung)  pairs have also been used widely in practice <cit.>. 
These arrays are preferred by practitioners for providing a stable and natural stereophonic image. 
ORTF was shown to have a localization curve most similar to a binaural recording <cit.>. 
A 3D extension to ORTF was recently proposed <cit.> and was shown to provide a good overall localization and auditory spaciousness in comparison with a coincident recording setup <cit.>.

More recent work on  time-amplitude recording techniques generally consider the problem from a constrained perspective, by using standard microphone directivities, by evaluating spatial acuity only at the sweet spot, or both. 
For example, a psychoacoustical evaluation of equal segment microphone array (ESMA) <cit.>, revealed that the selection of the recording array dimensions has a distinct effect on the localization accuracy for an ESMA using cardioid microphones <cit.>. 
In another subjective study, first-order Ambisonics with max-rE encoding  provided lower stereophonic image shifts in comparison with ESMA for a central listening position <cit.>.

















While the design of microphone arrays for recording spatial audio has traditionally been an ad hoc process driven by practical evidence that is not necessarily objectively validated, systematic approaches have also been proposed. A recently proposed design tool called microphone array recording and reproduction simulator (MARRS) allows the designer to design a stereophonic microphone pair using standard microphone directivity patterns and to observe the performance of the design both by visualization of the resulting localization curves and auralization of the simulation <cit.>.  


A similar systematic framework for the design 
of amplitude and time-amplitude circular multichannel recording and reproduction systems was proposed in <cit.>, based on earlier work <cit.>.
An objective analysis based on active intensity fields
showed that for stable rendition of plane waves it is beneficial to render each such wave by no more than two loudspeakers, thus re-framing the multichannel problem as a stereophonic one.





Using available psychoacoustic curves, a family of optimal microphone directivity patterns was obtained, parametrized by the array radius. 
The obtained directivity patterns are too spatially selective to be implemented using first-order microphone patterns but can be implemented using higher-order microphones, e.g. differential microphones <cit.>. 
Formal listening experiments were carried out for a microphone array with 15.5 cm radius <cit.>.
Results showed a significantly improved localization accuracy with respect to Johnston's array, and comparable to that of vbap and Ambisonics when the listener is in the center of the loudspeaker array.
The experiments also assessed the localization uncertainty, defined as how difficult it is for the listener to tell where the sound source is located. 
Results showed an improvement in localization uncertainty with respect to vbap and Ambisonics when the listener is in a position 30 cm off-center. 


This paper explains why that is the case and shows that this is a more general characteristic of time-amplitude methods. 




Towards this end, this paper proposes a computational model of localization uncertainty. 
The model also allows prediction of the perceived direction, but this is left for future work.




Formal listening experiments require very careful design, 
and carrying them out is expensive and time-consuming <cit.>.
Computational models provide a fast and repeatable alternative. 




Spatial hearing involves several stages of processing 
of sound waves impinging on the listener's head, which have been the subject of intensive study over the years.
Well-established models now exist for the early stages of the mechanisms of spatial hearing, such as the effects of head diffraction, cochlear filtering and neural transduction.
On the other hand, the higher levels of processing, where the spatial cues are combined, are not well understood yet.
Various models have been proposed in the literature, but none has proved to be capable of predicting all characteristics of human hearing. 

This paper proposes a model that first calculates itd and ild cues that arise from stereophonic reproduction and then compares them to the ones associated to point-like sources. 
The comparison is carried out using a distance function which replicates the auditory event splitting observed with inconsistent itd-ild cues.
It is shown that predictions of localization uncertainty based on this model are highly correlated with subjective scores. 
This model is then applied to generic ictd-icld values in central and off-central positions to assess how time-amplitude stereophony affects localization uncertainty. 
















The paper is organized as follows. 
Section <ref> provides the background on time-amplitude stereophony and on auditory system modelling.
Section <ref> presents the proposed model to predict localization uncertainty. 
Section <ref> discusses how time-amplitude panning affects localization uncertainty in stereophonic reproduction.
Section <ref> narrows the focus on a specific family of time-amplitude panning curves proposed in <cit.>. 
Section <ref> concludes the paper.



















































































































































§ BACKGROUND


















 §.§ Stereophonic Reproduction


Consider a stereophonic reproduction setup as shown in  <ref> with base angle ϕ_0 and loudspeaker distance r_l. 
The two loudspeakers are reproducing delayed and attenuated versions of the same signal. 
The gains applied to the left and right loudspeaker are denoted as g_L and g_R, respectively, while the delays are τ_L and τ_R.
The icld is defined as =20log_10g_L/g_R=G_L-G_R, where G_L=20log_10g_L and G_R=20log_10g_R. 
The ictd, on the other hand, is defined as τ_R-τ_L.
Notice how these definitions are given such that whenever ictd and icld have the same sign, their effect is consistent to one another. 
For instance, when both are positive, the left loudspeaker is louder and its signal also arrives earlier. 

If the ictd is below the echo threshold,
the listener will perceive a single fused auditory event <cit.>.
The echo threshold is strongly stimulus-dependent, 
and varies between 2 ms (for clicks) and 40 ms (for speech) <cit.>.
For ictd between 1 ms and the echo threshold, 
the auditory event is localised at the 
loudspeaker whose signal arrives first <cit.>.
This effect is called “law of the first wavefront" <cit.>.
For ictd smaller than 1 ms, 
a single fused “phantom” sound source is localised in a position that depends on both ictd and icld.
This psychoacoustic effect is called “summing localization" <cit.>. 

In the literature, summing localization and the law of the first wavefront 
are collectively referred to as “precedence effect" <cit.>.



 <ref> shows Williams time-amplitude psychoacoustic curves, 
which represent all the icld-ictd pairs that render the phantom source in the direction of the left and right loudspeaker <cit.>.
Other time-amplitude psychoacoustic curves are also available in the literature <cit.>. 

The psychoacoustic curves show that if one wishes to render a phantom source in the direction of the left (respectively, right) loudspeaker it is possible to use an icld of about 15 dB (respectively, -15 dB) without any ictd (amplitude panning).
It is then possible to continuously pan between the two loudspeakers using icld that vary between these two extremes (notice that the psychoacoustic curves do not give information on how to do this exactly, and only provide information about the extreme directions).


 <ref> also shows that it is possible to render a sound source in the direction of the left loudspeaker by reducing the icld but increasing the ictd (time-amplitude panning). 

If the icld is reduced to zero (time panning), it is still possible to displace the phantom source all the way to the left (respectively, right) loudspeaker with an ictd of about 1 ms (respectively, -1 ms), which corresponds to the onset of the law of the first wavefront. 



Note that most of the available stereophonic panning curves assume the standard stereophonic setup involving a phantom source panned between two loudspeakers positioned symmetrically to the left and right of the listener at a specified distance and can only provide suboptimal performance for panning lateral sources.







 §.§ Stereophonic recording




Consider the reference system in  <ref>. 
Here, the microphones are positioned on a circle with radius r_m facing outwards.
The inter-microphone distance is denoted by  d and is related to the array radius by 

    d=2r_msin(ϕ_m/2) ,

where ϕ_m is the angle separating the two microphones, typically referred to as microphone base-angle. 
The value θ_s denotes the angle of an incoming plane wave.
The arrangement of the two microphones on a circle (as opposed to positioning them on the x-axis) is preferred in this paper because it facilitates the extension to the multichannel case, and aids the comparison with psr, as discussed later. 

The first systematic approach to the problem of two-channel stereophonic recording
is attributed to Blumlein <cit.>. 
The Blumlein pair consists of two figure-8 microphones positioned 
orthogonally to each other (ϕ_m=90^∘) in the same position (d=0 cm).
Here, each microphone is connected to the corresponding loudspeaker without mixing, which means that the inter-microphone time and level differences are identical to ictd and icld, respectively. 
Systems without mixing are also the focus of the rest of this paper.
In the specific case of the Blumlein pair, the ictd are zero, since the microphone pair is coincident.




















A variety of other stereophonic arrangements have been proposed in the past few decades (see e.g. <cit.>).
These include the ORTF, DIN, NOS and cardioid XY pair, the characteristics of which are summarised in  
Table <ref>. 
The operating curve associated to the ORTF is shown in  <ref>.






























psr


 §.§ Perceptual Soundfield Reconstruction

The methods discussed in the previous subsection were mostly designed on a trial-and-error basis. 

A systematic framework for the design of circular microphone arrays was proposed in <cit.>, based on earlier work by Johnston and Lam  <cit.> and termed psr.
This section summarizes the design procedure, but for the specific case of a stereophonic setup.



In psr, each microphone is connected to a corresponding loudspeaker. 
The microphone base-angle, ϕ_m, is identical to the loudspeaker base-angle, ϕ_0, which allows a straightforward extension to full 360^∘ multichannel rendering. 
The inter-microphone delay (and thus ictd) is: 

    (θ_s)=d/csinθ_s=1/c2r_msin(ϕ_0/2)sinθ_s ,
 
where c is the speed of sound (c=343 m/s in dry air at 20^∘). 
Notice that in the psr literature, the  is expressed as a function of the array radius, r_m.
The remainder of this paper, on the other hand, uses the inter-microphone distance, d, which facilitates the comparison with other popular stereophonic microphone arrangements.







Consider the ictd obtained for a plane wave with θ_s=ϕ_0/2, i.e. in the direction of the left microphone:

    (ϕ_0/2) = d/csin(ϕ_0/2) .




Given the value of (ϕ_0/2), psychoacoustic time-amplitude curves can be used to find the minimum icld necessary to render a phantom source in the direction of the left loudspeaker. 
Rather than using Franssen curves as in <cit.>, which are considered to be not sufficiently precise for quantitative design <cit.>, this paper uses Williams curves.






Setting the  to be equal to the Williams value yields the constraint  (ϕ_0/2)=_W,L((ϕ_0/2)), where _W,L () is the Williams curve associated to the left loudspeaker (the top curve in  <ref>).
The value _W,L((ϕ_0/2)) will denoted by _W in the following.
For θ_s=-ϕ_0/2 (i.e. the direction of the right loudspeaker) one has (-ϕ_0/2)=_W,R((-ϕ_0/2)),
which, due to the symmetry of the problem, is equivalent to (-ϕ_0/2)=-_W. 





A third trivial point can be added, i.e. (0)=0 dB. 


It remains to choose how to connect these three points.
Two choices were explored in <cit.>: a simple straight line <cit.>, or a modified version of the tangent panning law  <cit.>.
While both approaches were shown to lead to a good localization accuracy (i.e. the phantom source was shown to be perceived close to the intended direction θ_s), the latter approach allows to link the design with other methods (e.g. vbap <cit.>) and consists of the following parametric function: 

    (θ_s)=20log_10sin(ϕ_0/2+β+θ_s)/sin(ϕ_0/2+β-θ_s) ,

where β is a free parameter that is used to satisfy the constraint (ϕ_0/2)=_W (the other two points are then also satisfied due to symmetry), which results in 

    β = arctan(10^_W/20sin(ϕ_0)/1-10^_W/20cos(ϕ_0)) .


To summarize, in order to obtain the psr panning curve one should (a) set the free parameter d (or, equivalently,  r_m), (b) obtain (ϕ_0/2) from (<ref>), (c) obtain _W from Williams psychoacoustic curve and β from (<ref>) and (d) obtain (θ_s) and (θ_s) from (<ref>) and (<ref>), respectively.









The result of this procedure is a family of panning curves parametrized by the value of the inter-microphone distance d.
In the extreme case d=0, one obtains an amplitude-only method ((θ_s)=0 ∀ θ_s).
As the value of d increases, the ictd increase while the icld dictated by (<ref>) decrease, thus achieving stereophonic rendering with a different time/amplitude trading ratio.
 <ref> shows the panning curve obtained for d=18.7 cm.


The so-obtained (θ_s) and (θ_s) can be viewed either in the context of stereophonic panning, whereby they can be used to directly control appropriate loudspeaker gains and delays, or in the context of stereophonic recording.
In the latter case, the microphone directivity patterns can be designed so as to emulate (θ_s).
Thus, one wishes to set (θ_s)=20log_10Γ_L(θ_s)/Γ_R(θ_s), where Γ_L(θ_s) and Γ_R(θ_s) denote the directivity patterns of the left and right microphones.
In general, first-order microphones (i.e. microphones with Γ(θ_s) of the type Γ(θ_s)=a_0+a_1cos(θ_s)) are not sufficiently directive to achieve the necessary icld.
Second-order microphones are already sufficient for this purpose <cit.>.
The remainder of this paper will refer to PSR panning/recording to emphasize the fact that application is possible in both contexts.

Notice that PSR aims at panoramic reproduction where a listener is allowed not only to move their head (e.g. to reduce localization ambiguities) but also to freely rotate their orientation with respect to an arbitrary, reference reproduction axis. 
In order to provide an orientation-independent homogeneous localization for a symmetric loudspeaker setup, stereophonic pairwise panning laws can be used as a starting point. 
Despite the fact that these laws provide suboptimal localization performance for lateral sources, sources that are positioned within the range of near-peripheral azimuths (i.e. ±ϕ_0/2 with respect to the listener's frontal direction) can be rendered accurately. 
From a purely physical point of view, PSR was also shown, via an energetic analysis of the reproduced sound field, to provide a good reproduction of directional properties of the sound field in a wide listening area <cit.>.



 §.§ Auditory system modelling



The auditory system estimates the directions of sound sources based on a combination 
of monaural and binaural cues <cit.>.
Localization in the horizontal plane is mostly reliant on binaural cues, particularly on differences in the time of arrival and on difference in level of a sound wave at the two ears.
itd are caused by the different time of arrival of sound waves radiated by sources outside the median plane.
At low frequencies the auditory system
analyses the interaural time difference between the signals' fine structure <cit.>.
At higher frequencies this mechanism becomes ambiguous, 
and the time differences between the signals' envelopes are used instead <cit.>.
The maximum naturally occurring itd is approximately 0.65 ms <cit.>.
ild are caused by the acoustical shadowing of the head
and are strongly frequency-dependent.
At low frequencies the head is approximately transparent 
to the sound wave and the level differences are small.
As the wavelength approaches the size of the human head, the level differences become sensible.
The highest natural ild is in the region of 20 dB <cit.>.


A common confusion in this context is to assume that ictd are identical to itd, and icld are identical to ild, which is incorrect. 
The wavefronts of each loudspeaker reach both ears and form interference patterns at the position of the ears,
which, in turn, lead to a complex relationship between ictd-icld and itd-ild.
Also notice that, as opposed to ictd and icld, itd and ild are frequency-dependent. 








































The mechanisms by which the auditory system interprets the itd and ild cues
are complex and not yet fully understood <cit.>.
Experimental evidence
suggests that humans use two main mechanisms for source localization, 
and that these mechanisms are to a certain degree independent from one another <cit.>.
The first interprets the interaural time shifts between the signals' fine structure and uses signal components below 1.6 kHz.
The second interprets the interaural level differences and time shifts of the envelopes jointly.
The latter mechanism seems to be dominant for signals with significant
frequency content above 1.6 kHz <cit.>.


A first, most notable attempt to model binaural processing
was made by Jeffress in 1948 <cit.>, who hypothesised that sound localization is governed by a mechanism of
running cross-correlation between the two channels.
While today this is still considered to be an adequate mean of measuring 
itd, it does not account for the presence of ild <cit.>.
Lindemann <cit.> 
proposed a model that incorporates this information in the cross-correlation 
mechanism by way of inhibitory elements that are physiologically plausible.
Gaik <cit.> extended this model further, 
based on the observation that itd and ild due to point-like sources in free field come in specific pairs.
For instance the itd and ild values for a source in the median plane are both small.
On the other hand, for a source to the right/left, both itd and ild are high.
In fact, in these cases the acoustic wave arriving 
at the far ear is both attenuated (because of head shadowing) 
and delayed (because of propagation time).


Gaik observed that when inconsistent itd-ild pairs (e.g. a left-leading ild and a right-leading itd) are presented over headphones, the auditory event width increases, and sometimes two separate events appear <cit.>.
In other words, inconsistent itd-ild pairs cause increased localization uncertainty.

These unnatural conditions can arise also with multiple sources radiating coherent signals, as in stereophonic reproduction.
Indeed, although each loudspeaker acts as a free-field source, the signals due to the different loudspeakers add up at the ears, creating interference phenomena that may result in inconsistent itd-ild cues.
Quantifying the deviation between the reproduced itd-ild pairs and
the ones associated to natural sources is therefore useful
to study the localization uncertainty due to different multichannel methods.
A study presented by Pulkki and Hirvonen in <cit.> 
goes in this direction.
For a given multichannel method they 
find the angle of the closest free-field source in terms of ild 
and itd, separately.
This model gives useful predictions when the angles corresponding to the ild and to the itd coincide.
However, in most cases the itd and ild cues provide contradicting information, 
and therefore the model output is hard to interpret <cit.>.











§ LOCALIZATION UNCERTAINTY MODEL






The first step of the model is to calculate itd-ild pairs of single point-like free-field sound sources in a number of directions on the horizontal plane. 
The so-obtained pairs are referred to as free-field itd-ild pairs.
Similarly to <cit.> and <cit.>, it is hypothesized here that the auditory system uses the free-field itd-ild pairs as a dictionary to interpret all other acoustical conditions. 
The itd-ild pairs for the acoustical scene to be estimated are calculated and compared to the free-field itd-ild pairs using a given distance functional.
Finally, the information is combined across critical bands to obtain an overall estimate of the localization uncertainty. 




 §.§ Calculation of ild and itd



The itd and ild values are calculated as follows.
Acoustic sources are modelled as point sources in the free field. 

The hrtf are taken as the Kemar mannequin measurements from the CIPIC database (subject 25) <cit.>. 
The sampling frequency is 44.1 kHz.
The response of the cochlea is modelled using a gammatone filter-bank <cit.> with 24 center frequencies equally spaced on the erb scale between 60 kHz and 15 kHz <cit.>.


As a rough model of the neuron firing probability, the bandpass signals are half-wave rectified below 1.5 kHz, while above 1.5 kHz the envelope of each bandpass signal is taken using the discrete Hilbert transform <cit.>. 

The resulting signals are fed to 24 binaural processors that calculate the itd and ild values independently.
The itd is calculated as the location of the maximum of the cross-correlation function evaluated over time lags between [-0.7, 0.7] ms <cit.>.
The ild is calculated as the energy ratio of the left and right channel <cit.>.
Altogether, the model produces a set of 24 itd-ild pairs (48 values in total).







 <ref> show the itd-ild pairs associated to free-field sources in the frontal horizontal plane.
The free-field sources are reproducing 50 ms long white noise sample, multiplied by a Tukey window with taper parameter 5%. 
Each simulation is repeated ten times to average out the effect of different noise realizations. 
Each point on the plot corresponds to a free field source positioned at angles between θ=-90^∘ and θ=90^∘ with an angular resolution of 5^∘.
In the remainder of this paper, these values will be denoted as _i(θ) and _i(θ), respectively, where i is the critical band index and θ is the free-field source angle with respect to the listener's look direction.




It can be observed in  <ref> that the interaural cues are highly correlated, i.e. larger itd values are typically associated to larger ild values, which is due to the concurrent effect of sound propagation and diffraction around the head <cit.>.
Also, the maximum ild values increase with frequency, which is due to the increasing head shadowing  associated to decreasing wave lengths <cit.>.
Some small asymmetries are observed, which are possibly due to noise or asymmetries in the measurement setup of the hrtf dataset. 















 §.§ Distance between itd-ild pairs


Let _i and _i denote the itd and ild values in the i-th critical band as observed by a listener under stereophonic reproduction (or other acoustical conditions). 

In order to combine the information of itd and ild cues across critical bands, it is useful to normalize all quantities to the maximum values of the free-field cues:

    _i    = _i/max_θ|_i(θ)| , 
    _i   = _i/max_θ|_i(θ)| .

The free-field pairs, _i(θ) and _i(θ), 
are normalized in the same way and are
denoted as _i(θ) and _i(θ), respectively.

Notice that while _i(θ)∈[-1,1] and _i(θ)∈[-1,1],
_i and _i can in theory be outside that range.
Indeed, there is no guarantee that the acoustical condition being analyzed has itd-ild values outside the range of values occurring for free-field sources.



















The aim is now to select a distance functional between 
{_i, _i} and {_i(θ), _i(θ)}
according to some meaningful psychoacoustic criterion.




Consider the distance defined by the classical p-norm:

    (|_i-_i(θ)|^p+|_i-_i(θ)|^p)^1/p .


 <ref> shows an example of an observed {_15, _15} point positioned at {0.5,-0.5} in the 15th critical band (the one centered at 6.07 kHz).
The distance between each free-field source {_15(θ), _15(θ)} and the {_15, _15}={0.5,-0.5} point is plotted in  <ref> as a function of the free-field source angle, θ, for different distance functionals.

The experimental evidence shows that subjects presented with contradicting itd-ild pairs are likely to report split auditory events <cit.>.
The Euclidean distance (p=2) does not emulate this behaviour, as it leads to a single minimum in θ=0, as shown in  <ref>. 
The Manhattan distance (p=1) is nearly constant in the [-30^∘,30^∘] angular sector.
The 0.5-distance, on the other hand,
causes two sharp minima, 
one of which is centred in the direction corresponding to the itd cue,
which is compatible with the psychoacoustic evidence <cit.>. 
Other values of p close to 0.5 would also retain this behaviour.
In the next section it is shown that
the model has very good predictive power even without careful tuning of p.

The distance defined in (<ref>) does not satisfy the triangle inequality for p<1.
However, the same distance raised to power p, does satisfy all the properties of a distance <cit.>.
Hence, the p-norm distance used here is:

    ξ_i(θ|_i,_i) = |_i-_i(θ)|^p+|_i-_i(θ)|^p .


The objective is to obtain a function of θ that quantifies the likelihood that a sound is perceived in that direction.
Intuitively, this function should be inversely proportional to ξ_i(θ|_i,_i), such that whenever the distance is small, the likelihood is high (and viceversa).
Let this function be

    f_i(θ|_i,_i) =
    
    Ke^-ξ_i(θ|_i,_i) ,

where K is a positive constant. 

Although other choices are available<cit.>, the advantage of (<ref>) is that it gives the model an explicit statistical interpretation in the ml framework, as will be discussed in the next subsection.



The next step is to integrate the information from the different critical bands.
The mechanisms governing this stage of perception are generally regarded 
as complex and not well understood <cit.>.
Here, the information across critical bands is combined as a loudness-weighted average:

    f(θ|,) = K/N∑_i=1^N w_i e^-ξ_i(θ|_i,_i) ,

where w_i are the loudness weights,  denotes the vector =[_1,...,_N] and =[_1,...,_N] , and N is the number of critical bands (N=24).

The loudness weights are set according to the procedure proposed in <cit.>, which results in critical bands with a higher signal level weighting more than critical bands with little active content <cit.>.
The procedure involves (a) calculating the SPL levels in each critical band, (b) converting them to phon levels through the BS ISO 226:2003 equal-loudness contours, and (c) converting them to w_i weights using a function where a 10 phon reduction leads to a halving of the weight, in line with Stevens's model <cit.>.  































 §.§ Statistical interpretation

Suppose that {_i, _i} are noisy observations of a point-like free-field source at angle θ_0 associated to a set of true {_i(θ_0), _i(θ_0)} values:

    _i   =_i(θ_0)+u_i ,
    _i   =_i(θ_0)+v_i .

where u_i and v_i are the noise components, which may arise for instance as a consequence of room reverberation or reproduction with multiple loudspeakers.
Let the joint distribution of the noise components be the following mixture of zero-mean bivariate theta-generalized normal distributions <cit.>:

    f(u_1,...,u_N;v_1,...,v_N) = 1/N∑_i=1^N Ce^-|u_i|^p+
    |v_i|^p/2σ^2 ,


where σ represents a standard deviation and C is a normalization constant. 

Notice that if 1/2σ^2=1, equation (<ref>) becomes identical to (<ref>).
Suppose now that the auditory system estimates the true value θ_0 using an ml approach. 
Within this context, f(θ|,), seen here as a function of θ,  takes the meaning of a likelihood function. 

Although the objective of the proposed model is not to estimate θ itself (which is left for future work), the shape of the likelihood function gives information about how difficult it is to estimate it, as will be explained in the next subsection.




 §.§ Calculation of the localization uncertainty



The likelihood function quantifies the probability that a subject would perceive a source in a given direction θ. 
From this perspective, a uniform (constant) likelihood function would result in a maximally uncertain (diffuse) event.
At the other extreme, an impulsive likelihood function would result in a minimally uncertain event.
Various measures can be used in this context.
This paper uses a modified version of the circular variance <cit.>.
This measure is drawn from the field of directional statistics <cit.>, and has values between 0, which is associated to an impulsive function, and 1, which is associated to a constant function.
Calculating the modified circular variance involves normalising f(θ|,) so that it sums to one (e.g. by adjusting the K constant), and calculating the (modified) first cosine and sine moments:

    α   =∫_0^2πf(2θ|,)cos(θ)dθ,
    β   =∫_0^2πf(2θ|,)sin(θ)dθ ,

where the modification consists of using f(2θ|,) instead of f(θ|,). 
This modification is motivated by the fact that the likelihood function only takes values in [-π/2,π/2], which would not lead to a circular variance of 1 in case of a constant likelihood function. 
Then, the circular variance is given by <cit.>

    H(,)=1-√(α^2+β^2) .

 
 







The final step is to normalize the value of H.
Indeed, since f(θ|,) is never impulsive (even for free-field sources), the value of H is always larger than zero.
For instance, for p=0.7, the minimum H is 0.49. 
The following normalization yields values close to zero for estimates associated to free-field sources:

    H(,)=H(,)-H_min/1-H_min

where H(,) denotes the final localization uncertainty estimates of the proposed model and H_min=min_θH((θ),(θ)).

It should be noted that, in practice, all calculations above are made using a discretization of the angles θ. 
The simulations in the remainder of this paper use a resolution of 5^∘.












 §.§ Model validation


A formal listening test with 19 subjects was carried out in <cit.> using a modified MUSHRA test <cit.>.
The subjects answered the question 
“How certain are you of the direction of the source?"
by giving a score on a continuous scale from 0 to 100.
















The test was carried out in an audio booth using four synthesized 5-channel surround sound methods: (a) pair-wise tangent panning law <cit.> (equivalent to horizontal vbap); (b) near-field corrected second-order Ambisonics 
with mode-matching decoding at low frequency and
maximum-energy decoding at high frequency
 <cit.>;
(c) second-order Ambisonics with in-phase decoding <cit.>;
and (d) the quasi-coincident microphone array proposed in <cit.>. 
The test was run for three sound source directions 
and two seating positions, and included both a reference (a loudspeaker in the intended direction) and an anchor (an approximately diffuse soundfield). 
The rendered virtual source directions were in front of the listener and ±18^∘ to the right and left of the front direction. 
Three excerpts (female speech, african bongo and cello) were used as representatives of common program material, and the resulting subjective scores were averaged across the three excerpts.
Details of the experiment are available in <cit.>. 














































In order to validate the proposed model, the experiment was replicated here through simulations.
The stimuli used in the simulations were (a) long white noise burst (500 ms), (b) short white noise burst (50 ms), (c) short pink noise burst (50 ms), and (d) impulsive sound. 
All stimuli (except for the impulsive sound) were multiplied by a Tukey window with a 5% taper parameter.

The loudspeaker signals for each of the 5-channel surround sound methods mentioned above were then calculated. 
Notice that the experiment and the simulations employed multichannel sound reproduction where more than two loudspeakers can be substantially active in some of the methods used (i.e. Ambisonics). 
While the model will be used in a stereophonic context in this paper, it is independent from the number of loudspeakers that the input ear signals originate from. 

The acoustic path from each loudspeaker to the head was simulated in free-field conditions using appropriate time delays and distance attenuation (inverse square law) that depend on the position of the head.
The ear pressure signals were obtained using the Kemar mannequin hrtf.
The so-obtained ear pressure signals associated with multiple loudspeakers were added together. 
Finally, the resulting ear pressure signals were fed to the proposed model, and localization uncertainty estimates were obtained using (<ref>).

 <ref> shows the absolute Pearson correlation between the subjective scores and 
the localization uncertainty estimates, as a function of the parameter p for different stimuli. 
All the types of stimuli result in an absolute Pearson correlation stronger than -0.98 for p∈[0.7,0.9], 
confirming that the model predictions are strongly correlated with the experimental data.
The reason why the correlation is negative is that subjects' answers to the question “How certain are you of the direction of the source?" are merely inverted with respect to localization uncertainty.

All types of random noise perform particularly well. 
This indicates that the choice of stimulus does not appear to be critical when aiming to predict the localization uncertainty of an experiment involving a varied programme material like the one in <cit.> (which included speech, bongos and cellos).
Furthermore, since the free-field pairs {_i(θ), _i(θ)} were obtained using short white noise bursts, the model appears to be robust to a mismatch between the stimulus used for the free-field pairs and the one used in the acoustic scene under analysis (in a machine learning context, these would be akin to training and testing, respectively).








Fine tuning of the p-norm distance is also not critical as long as contradicting itd-ild cues lead to a an approximately bimodal likelihood function.
For the short white noise burst, values of p between 0.5 and 1.0 all give correlation coefficients of approximately -0.99. 


The Euclidean distance (p=2), on the other hand, yields a much weaker correlation of -0.84.
At the other extreme, a small p also leads to weak correlations, indicating that norms closer to the l_0-norm do not provide an effective measure in this context.

The simulations in the remainder of this paper use the short white noise burst, and a distance function with p=0.7 (an approximate midpoint of the p∈[0.5,1.0] interval with -0.99 correlation). 
 <ref> shows the scatter plot comparing the model estimates with the subjective scores.

Note that the selection of an earlier set of results from an experiment using a five channel setup for the validation of the proposed model is deliberate. 
While the derivation of the model is based on two-channel stereophony as the simplest possible spatial audio reproduction system, its predictive power extends beyond that as is shown.

Note also that the experiment did not use any head or position tracking and the listeners were instructed to keep as still as possible. 
Such a listening scenario is not entirely ecologically valid. 
However, since the proposed model is neither dynamic nor does it incorporate any mechanism to account for listener motion, the results from the experiment are indicative of the predictive power of the model. 




























§ LOCALIZATION UNCERTAINTY IN STEREOPHONIC REPRODUCTION














































 §.§ Localization uncertainty in the center of the sweet-spot





 <ref> shows the localization uncertainty 
produced by the proposed model 
for a listener in the center of the sweet-spot and for a stereophonic reproduction system with base angle ϕ_0=60^∘ and r_l=2 m (see  <ref>).
Two areas in the second and fourth quadrants have a significant localization uncertainty.
These areas correspond to cases where icld and ictd provide inconsistent information: one 
loudspeaker is leading in terms of icld (i.e. it is louder)
while the other is leading in terms of ictd (i.e. it arrives earlier).
This indicates that inconsistent ictd-icld pairs somehow 
translate to unnatural itd-ild cues,
which is in agreement with the experimental findings of Leakey in <cit.>.
It may also be observed in  <ref> 
that large icld values (outside ≈± 13 dB) result in a
low localization uncertainty for all ictd.
Here, one loudspeaker signal is masking the other.

Finally, it can be  observed that for  ictd around ± 0.3 ms
the localization uncertainty increases even in the first and third quadrants, where  ictd-icld pairs are consistent.
At  ictd of approximately ± 0.3 ms, the two loudspeaker signals arrive at the same time at one of the two ears (more specifically, at the left ear for ≈+0.3 ms and at the right ear for  ≈-0.3 ms). 
Appendix <ref> proves that these ictd can be approximated as

    τ_o ≈±r_h/c[cos(θ_e-ϕ_0/2)+ϕ_0/2+θ_e-π/2] ,

where r_h denotes the head radius and θ_e denotes the angle between the forward-looking direction and the ear.
For r_h=9 cm, θ_e=100^∘ and ϕ_0=60^∘, then τ_o=±0.27 ms.
If signals of both loudspeakers arrive at the same time at one of the ears,  that ear effectively receives  one instance of the rendered acoustic event, whereas the other ear receives two instances. 
A number of psychoacoustic studies investigated effects of presenting three coherent stimuli, two to one ear and the third to the other ear. 
It was found that the perceived event had “complex spatial structure" including cases where, depending on relative delays between the three stimuli, two distinct acoustic sources could be perceived <cit.>. 
The results of the proposed model are in agreement with the findings of these experiments.
In fact, simulations not shown here for space reasons, confirm that as the head radius r_h changes, the areas with higher ictd in the first and third quadrants move in accordance to (<ref>).
If one wishes to avoid these areas, the ictd values should be restricted to the open interval ∈]-τ_o, τ_o[. 

In  <ref>, notice how the psr operating curve associated to d=18.7 cm avoids the areas with higher localization uncertainty around ≈± 0.3 ms. 
This is so by construction, as will be discussed later in Section <ref>.
















































 §.§ Localization uncertainty in off-center positions


















Figure <ref> shows the localization uncertainty 
for a listener in a position 10 cm to the left, and in a position 10 cm and 20 cm to the right of the sweet-spot.
The listener is still looking ahead, in a direction parallel to the y-axis.
It may be observed that these plots are almost identical to the on-center plot of  <ref> but shifted horizontally.
This implies that the dominant effect is the change in the ictd observed by the listener as a consequence of having moved closer to the right loudspeaker.
The change in observed icld, on the other hand, appears to have a minor effect. 
Likewise, the relative change of direction of the loudspeakers also appears to have a minor effect (in the 0.2 m position, the two loudspeakers appear at 25^∘ and -35^∘ with respect to the listener, compared to ± 30^∘ for the central position). 




Notice how amplitude panning methods, which are associated to the line =0 ms (corresponding to the y-axis), lie in an area with increased localization uncertainty for these off-center positions. 
For the rightward positions in  <ref> and <ref>, the localization uncertainty is particularly high for positive icld values, which are meant to render phantom sources between the midline and the left loudspeaker. 
Viceversa, for the leftward position  <ref>, the localization uncertainty is high for negative icld values, 
which are meant to render phantom sources between the midline and the right loudspeaker. 


Notice also how the psr panning curve associated to d=18.7 cm largely avoids areas with higher localization uncertainty in both leftward and rightward positions 10 cm off-center. 










 §.§ Definition of relative inter-channel time and level differences


In order to aid the interpretation of Figures <ref>, it is useful to define two new quantities: the relative ictd and icld, i.e. equivalent inter-channel time and level difference as they are observed at a point away from the center of the loudspeaker array.
These quantities will be referred to as rictd and ricld, respectively.


Let the observation point be positioned at (x,y) (see  <ref>).
Appendix <ref> provides a closed-form approximation for the ricld and rictd under the assumption of small displacement compared to the loudspeaker distance: 

    ≈ - x/r_l20sin(ϕ_0/2)/log_e(10) , 
       ≈-x2/csin(ϕ_0/2) .




 §.§ Considerations on the effect of rictd and ricld in amplitude panning



Consider now again the 𝐱={10,0} cm off-center plot in  <ref>.
The point =0 ms and =5 dB (i.e. the left loudspeaker leads in amplitude) is associated to =-0.29 ms and =4.7829 dB, which have opposite sign and thus provide inconsistent information. 
The ricld is now smaller than =5 dB because the right loudspeaker is closer to the listener.
The change, however, is small (0.22 dB), and the left loudspeaker still leads in amplitude. 
The rictd, on the other hand, has changed significantly with respect to ictd, and the right loudspeaker is now leading in time. 
To summarise, the left loudspeaker is louder and thus leads in terms of amplitude, but the right loudspeaker is now closer and thus leads in terms of time.
Even though the ictd and icld were consistent, the horizontal shift of the listener caused the rictd and ricld to become inconsistent, which, in turn, led to a high localization uncertainty. 

Negative values of icld, on the other hand, are less problematic in terms of localization uncertainty for a listener in this position. 
For instance, =0 ms and =-5 dB is associated to relative values =-0.29 ms and =-5.2171 dB, which have the same sign and provide consistent information. 

Similar arguments can be made for a listener moving to the left (i.e. x<0), as shown in  <ref>. 
In this case, however, the plots shift horizontally to the left and the critical area of the plot becomes the bottom one. 


In summary, amplitude methods lead to higher localization uncertainty whenever the listener moves to one side but aims to render sound sources in directions closer to the opposite side.
Section <ref> will show that time-amplitude methods provide a lower localization uncertainty in positions away from the center of the sweet-spot. 



 §.§ Further considerations on rictd and ricld

Notice how the ricld is a function of x/r_l, while the rictd is independent from r_l. 
This implies that for large loudspeaker arrays, movement of the observation point away from the center of the sweet-spot causes a larger change of rictd compared to ricld.
This holds also even for living-room-sized arrays.
Consider for instance the rictd that will result in full perceived shift of the phantom source to one loudspeaker, i.e.  =1 ms.
Assuming =0 ms, =0 dB and ϕ_0=π/3 (60 degrees), this is already achieved at x=0.001 c/2sin(ϕ_0/2)=0.34 m. 


In that position, the ricld, on the other hand, is only 1.5 dB.
In other words, a phantom source that appears halfway between loudspeakers for a listener in the center of the sweet-spot, collapses onto one of the loudspeakers when the listener moves to a position 34 cm on either side. 
This compares to approximately 3 cm on either side for binaural cross-talk cancellation using two  loudspeakers <cit.>. 
It should be noted that the degradation is gradual in both cases, which makes defining quantitatively the sweet-spot size difficult.




Vertical displacement is less significant than horizontal displacement.
This can be inferred by the fact that the first-order approximation (<ref>) does not  depend on y (it can be shown that y appears starting from the second-order term). 
This is confirmed by  <ref>, which shows the localization uncertainty as a function of listener position for PSR panning/recording associated to d=0 cm. 
The plot shows that the localization uncertainty varies almost exclusively as a function of the lateral displacement, x. 

For this reason, the simulations in the remainder of the paper will focus on listener displacement along the x axis (y=0).







§ TIME-AMPLITUDE PANNING/RECORDING METHODS











This section will show how small ictd allow to render consistent ricld-rictd values (and thus a lower localization uncertainty) in a slightly larger area around the center of the sweet-spot. 
The starting point of the analysis will be PSR panning/recording, which enables to vary the time-amplitude trade-off using the the inter-microphone distance, d, as a free parameter. 
Then, the performance of popular microphone arrangements will be assessed.
The section concludes with some comments on how the results extend to the multichannel case.




 §.§ Effect of time-amplitude trading on localization uncertainty using PSR panning/recording






 <ref> shows the localization uncertainty for PSR panning/recording as a function of inter-microphone distance d and plane wave angle θ_s for three different observation positions. 
A number of observations can be made:

    
  * As the distance from the center of the sweet spot increases, so does the maximum localization uncertainty. 
    
  * In the center position ( <ref>), the localization uncertainty is lowest around d=0 (i.e. amplitude methods). Two “sidebands” appear in the plot beyond d=20 cm. These sidebands correspond to the vertical stripes observed in   <ref> at ±τ_o.
    
  * At x=10 cm ( <ref>), the localization uncertainty is higher for inter-microphone distance d=0 cm, i.e. amplitude methods.
    It remains high in plane wave directions around the midline, while it reduces for other directions, especially around the θ_s∈[10^∘, 20^∘] range.
    This can be observed also in  <ref>, which shows the panning curve associated to d=18.7 cm. 
    Here, the panning curves avoids the areas with higher localization uncertainty coming in from the left.
    
  * At x=20 cm ( <ref>), the localization uncertainty tends to reduce for higher d, where it also has a lower variability as a function of source angles θ_s. 



 <ref> shows the localization uncertainty as a function of inter-microphone distance d. 

Here the results are averaged across angles between the midline and the left loudspeaker, which are more difficult to render when the listener moves to the right of the center.
The curve associated to the center position shows that, in that position, the inter-microphone distance with the lowest uncertainty is close to d=0 cm, as expected.

In typical use cases, one would like to minimize the uncertainty as it is observed in a whole region around the center of the sweet spot, instead of specific positions. 
 <ref> shows the localization uncertainty averaged for listener displacements between x=0 cm and x=5 cm (with a 1 cm resolution). 
Here, the absolute minimum is achieved at d=20 cm. 
 <ref> also shows the localization uncertainty averaged for displacements between x=0 cm and x=15 cm. 
In this case, the average uncertainty increases significantly at small d, and the absolute minimum moved to d≈ 30 cm.



In summary, the results suggest that smaller inter-microphone distances (amplitude methods) are preferable for positions close to the center, while larger inter-microphone distances result in lower localization uncertainty at larger distances. 

There is thus a trade off between lower localization uncertainty in the center of the sweet spot and away from the center.



Notice that if one can assume that only one listener is present, and that some form of user tracking is available, then the results here suggest that it would be beneficial to correct the s such that the =0. 
In other words, if the location of the (single) listener is known, one can correct the time delays such that the listener experiences amplitude panning. 

If, on the other hand, the listener's position is not being tracked, it is typically difficult to predict how far the listener/s will move away from the center of the sweet spot. 
A possible compromise is to use the inter-microphone distance that results in the largest ictd to be no larger than τ_o, and thus avoid the areas with slightly higher localization uncertainty in the first and third quadrant shown in  <ref>.
One can find this inter-microphone distance by equating _max with τ_o, and isolating d:





    d= r_h cos(θ_e-ϕ_0/2)+ϕ_0/2+θ_e-π/2/sin(ϕ_0/2) .

For instance, the inter-microphone distance for θ_e=100^∘ and r_h=9 cm, is d=18.7 cm, which is the one used for the panning curve shown in  <ref> and <ref>.




 §.§ Comparison with popular stereophonic microphone arrangements


This subsection presents the results of a comparison between three versions of PSR for d=0, 18.7, 37.2 cm, and the popular stereophonic microphone arrangements summarized in Table <ref>. 
The comparison involves choosing which source angles θ_s to simulate. 

While it is clear that for PSR the source angles should be within the microphone base-angle (60^∘), the choice for other microphone arrangements is ambiguous. 
Indeed, sound engineers typically make a choice of the relative position of the sources of interest (e.g. an orchestra) often based on artistic considerations. 
In order to make a more unambiguous choice here, the range of angles is chosen according to which source angles results in - pairs within the Williams curves.
The resulting ranges are termed here coverage angles, and are reported in Table <ref> for the popular microphone arrangements.
In the simulations that follow, a set of thirty θ_s are uniformly taken within the coverage angle for each microphone arrangement.

 <ref> shows the localization uncertainty as a function of lateral displacement x. 
 <ref> shows the average across source angles, while  <ref> shows the excursion across source angles, defined as the difference between the largest and the smallest localization uncertainty across source angles.


A number of observations can be made:


  * The amplitude methods, i.e. Blumlein pair, PSR with d=0 cm, and cardioid XY pair (all of which are indicated by a × marker) have very similar performance; the difference between them stems from the slightly different distribution of s across source angles θ_s.

  * In comparison to time-amplitude methods, the average uncertainty of the amplitude methods is lower for |x|<2 cm but higher beyond |x|>10 cm.

  * The uncertainty excursion of the amplitude methods is significantly higher than  time-amplitude methods beyond |x|>10 cm, reaching almost 0.5 at |x|=20 cm.
This is associated to the difficulty in rendering sound sources associated to direction opposite to the lateral movement of the listener, as discussed in Section <ref>, and it means that that sound sources at different angles will be perceived differently.

  * In the center (x=0 cm), the average uncertainty is lowest for the amplitude methods, and then, in order, for PSR (d=18.6 cm), ORTF (d=17 cm), DIN (d=20 cm), PSR (d=37.2 cm) and NOS (d=30 cm). PSR (d=18.6 cm) is lowest among time-amplitude methods because it is the only one that does not cross the ±τ_o threshold (see   <ref>).

  * The PSR with d=18.6 cm has a reasonably low average uncertainty across the lateral displacement, a low excursion within |x|<15 cm, but a higher excursion compared to other time-amplitude methods beyond |x|>15 cm.

  * The NOS and PSR version with d=37.2 cm have a higher average uncertainty in the center, but a smaller variation of the average uncertainty across x, and a low overall excursion. 

  * The PSR (d=18.6 cm), ORTF (d=17 cm), DIN (d=20 cm), and NOS (d=30 cm) all have low excursion, with little discernible difference between them across x. 


Overall, these results further confirm that amplitude methods have a better performance at the center of the sweet-spot, while time-amplitude methods have a lower overall uncertainty and a lower excursion away from the sweet-spot. 
A lower excursion means that whenever multiple sound sources are present at different angles, they are rendered with a similar localization uncertainty, which is an appealing property. 

A further advantage of psr over the other time-amplitude methods, is that it has been shown to have a high localization accuracy, meaning that subjects would perceive sound sources in directions close to the intended θ_s  <cit.>.
This is also related to the fact that in PSR the coverage angle is identical to the loudspeaker base-angle, while the other methods will have (at least) a compression/decompression of angles from the coverage angle to the loudspeaker base-angle. 

An additional advantage of having the coverage angle identical to the loudspeaker base-angle (and microphone base-angle) is that the extension to the multichannel case is straightforward. 








 §.§ Comments on the relation to multichannel PSR

The PSR stereophonic setup considered thus far can be seen as a subset of a multichannel circular array. 
In <cit.>, it was shown that having only the two loudspeakers closest to θ_s as active yields a lower spatial fluctuation of the active intensity vector field and thus a larger sweet spot. 
This turns the overall multichannel design problem into a number of stereophonic problems. 


Consider the case of M microphones distributed uniformly around the circle with ϕ_m=ϕ_0=2π/M.
Each of these microphones is connected to a loudspeaker in the same angular configuration, without mixing.
Johnston and Lam proposed in <cit.> a microphone array with M=5, resulting in a base angle of ϕ_0=2π/5=72^∘, and radius r_m=15.5 cm. 
This array radius is very close to the value needed to avoid ictd larger than τ_o=±0.3 ms. 
Indeed, using (<ref>) and (<ref>) for ϕ_0=2π/5=72^∘ yields r_m=16.2 cm. 

Johnston and Lam stated that r_m=15.5 cm would conserve itd cues that the listener would have experienced in the recording space, but without providing a rigorous explanation (it could indeed be argued, as Bernfeld did in <cit.>, that correct itd cues can also be achieved by using icld alone).
This paper shows that Johnston and Lam's choice of array radius is the one that leads to a good compromise between low localization uncertainty in central and non-central listener positions.
The same choice was made in <cit.>, where the directional accuracy was also improved significantly. 












§ SUMMARY AND CONCLUSIONS




















Two-channel stereophony remains to this day the most commonly used audio reproduction system.



This paper focused on the effect of inter-channel time and level differences on perceived localization uncertainty. 
A computational model was proposed, based on calculating a distance functional between the observed ild-itd values and the ones associated to free-field sound sources.
The distance functional was chosen as the 0.7-norm which is capable of modelling the splitting of auditory events observed experimentally in case of contradicting ild-itd cues.
The model predictions had a high correlation with results of formal listening experiments.
The model can also be used to predict the perceived localization angle, but this is left to future research. 

The model was then used to predict the localization uncertainty under stereophonic reproduction.
It was found that contradicting ictd-icld pairs were associated to a high localization uncertainty. 
Closed form approximations of the ictd and icld relative to the listener (denoted as rictd and ricld) were presented.


It was observed that when a listener moves away from the center of the sweet-spot, the ricld remains largely unchanged, while the rictd changes even for small displacements.
In off-center positions, then, one can obtain contradicting ricld-rictd pairs even if the original icld-ictd pairs were not.
It was then explained that non-zero ictd (i.e. time-amplitude methods) will delay the onset of contradicting ricld and rictd. 
More significant use of ictd results in a higher uncertainty in the center of the sweet-spot, but a relatively lower one in off-center positions. 
A comparison of amplitude methods (Blumlein, XY) and time-amplitude methods (psr, ORTF, DIN and NOS) confirms that time-amplitude methods have a lower localization uncertainty in off-center positions, and that the localization uncertainty has a lower variability as a function of source angle. 

These results reveal that near-coincident microphone pairs, which are often used for their sense of spaciousness deriving from lower inter-channel decorrelation <cit.>, also have an advantage in terms of localization uncertainty.


In this paper it was not possible to analyze widely spaced microphone arrays such as the ones criticized by  Lipshitz <cit.> or time panning as criticized by Lee and Rumsey <cit.>, since they exceed the 1 ms ictd limit beyond which one has to also model the law of the first front. 
This paper is therefore not in contradiction with the findings in <cit.> and <cit.>, as the conclusions made here are limited to coincident and near-coincident microphone arrays. 
The computational model could be modified to account for the law of the first front, e.g. by means of inhibitions mechanisms <cit.>, thus allowing analysis of widely spaced microphone arrays. 


Throughout this paper, it was implicitly assumed that a low localization uncertainty is desirable. 
This is motivated by the fact that an actual plane wave is well localized spatially, and that 
to render it accurately it needs to have a low localization uncertainty. 


This may not be necessarily the case e.g. in an artistic context, where one may prefer to render sound sources as difficult to localize.
The results in this paper are useful in this context too as they can also be read with an implicit preference for high uncertainty. 












§ 


This appendix provides a closed-form approximation of the ictd that results in the two loudspeaker signals to arrive at the same time at the left ear (the corresponding case for the right ear will follow).
Let the head radius be denoted by r_h and the angle between the forward-looking direction and the ear  be denoted by θ_e.
Let θ_0=cos^-1(r_h/r_l) be the angle of tangential incidence on the spherical head. 
Assuming that |r_h/r_l|<<1, i.e. the head radius is much smaller than the loudspeaker distance, then θ_0≈π/2.

The angle between the left loudspeaker and the left ear is θ_LL=θ_e-ϕ_0/2 
and under most conditions, θ_LL<θ_0. 
The distance between the left loudspeaker and the left ear is then:

    d_LL   =√(r_l^2+r_h^2-2rr_hcos(θ_LL)) 
       =r_l√(1+r_h^2/r_l^2-2r_h/r_lcos(θ_e-ϕ_0/2))
       ≈ r_l-r_hcos(θ_e-ϕ_0/2) ,

where in the last step, the quadratic term r^2_h/r^2_l is ignored and the square root is approximated using the first-order Taylor series approximation.

The angle between the right loudspeaker and the left ear is θ_RL=ϕ_0/2+θ_e 
and under most conditions, θ_RL>θ_0. 
The distance between the right loudspeaker and the left ear, then, is given by the summation of the distance between the loudspeaker and the point of tangential incidence, and the remaining angular sector:

    d_RL=√(r_l^2+r_h^2)+r_h(θ_RL-θ_0) .

For small r_h/r_l, the quadratic term (r_h/r_l)^2 can be ignored:

    d_RL≈ r_l+r_h(ϕ_0/2+θ_e-π/2) .


The delay applied to the left loudspeaker, τ_o, that makes the two loudspeaker signals arrive at the same time at the left ear satisfies the equation
d_LL/c + τ_o=d_RL/c .
Replacing equations (<ref>) and (<ref>), and solving for τ_o yields

    τ_o≈r_h/c[cos(θ_e-ϕ_0/2)+ϕ_0/2+θ_e-π/2] .


Due to the symmetry of the problem, the  that results in the loudspeaker signals arriving at the same time at the right ear is -τ_o.





§ 


This appendix provides a closed-form approximation of the rictd and ricld. 
Towards this end, approximations of the relative distance between an observation point (x,y) and the two loudspeakers at a distance r_l from the center of the sweet-spot (0,0) are sought first. 
It will be assumed  that |x/r_l|<<1 and |y/r_l|<<1.
The distance between the observation point (x,y) and the right loudspeaker can be approximated as 

    d_R^2   =[r_lsin(ϕ_0/2)-x]^2+[r_lcos(ϕ_0/2)-y]^2
       ≈ r_l^2[1-2x/r_lsin(ϕ_0/2)-2y/r_lcos(ϕ_0/2)] ,

where in the second step it is assumed that (x/r_l)^2≈ 0, and (y/r_l)^2≈ 0.
It is useful to define a=2x/r_lsin(ϕ_0/2) and b=2y/r_lcos(ϕ_0/2), so that d_R can be rewritten as
d_R≈ r_l√(1-(a+b)).
The first-order Taylor series approximation of the square root yields:

    d_R≈ r_l(1-a+b/2)


Using similar steps, d_L can be approximated by
d_L≈  r_l[1+(a-b)/2] .
The time delay between the two loudspeakers observed in (x,y) can now be approximated as

    d_R-d_L/c≈ -r_l a/c=-x2/csin(ϕ_0/2) .



The relative level observed in (x,y) for the two point-like loudspeakers can now be approximated as 

    20log_10(1/d_L/1/d_R)
        ≈ 20log_101-x/2r_lsin(ϕ_0/2)-y/2r_lcos(ϕ_0/2)/1+x/2r_lsin(ϕ_0/2)-y/2r_lcos(ϕ_0/2) ,

and, using a first-order Taylor series approximation:

    20log_10(1/d_L/1/d_R)≈ -x/r_l20/log_e(10)sin(ϕ_0/2) ,








Using (<ref>) and (<ref>), one obtains ≈ - x/r_l and ≈-x2/csin(ϕ_0/2).












§ ACKNOWLEDGMENTS

The authors would like to thank Niccolò Antonello for pointing out that the proposed computational model has a statistical interpretation. 

IEEEtran


[
    < g r a p h i c s >
]Enzo De Sena
(S'11-M'14-SM'19) is a Lecturer in Audio at the Institute of Sound Recording at the University of Surrey. He received the M.Sc. degree (cum laude) in 2009 from the Università degli Studi di Napoli “Federico II" in Telecommunication Engineering and the Ph.D. degree in Electronic Engineering in 2013 from King’s College London. Between 2012 and 2013 he was a Teaching Fellow at King's College London. Between 2013 and 2016 he was a Postdoctoral Research Fellow at the Department of Electrical Engineering at KU Leuven. He held visiting  positions at Stanford University (2013), Aalborg University (2014-2015), Imperial College London (2016), and King's College London (2018-current). He is a former Marie Curie Fellow. His current research interests include room acoustics modelling, multichannel audio, microphone beam forming, and binaural modelling. For more information see desena.org.

-2plus -1fil
[
    < g r a p h i c s >
]Zoran Cvetković (SM'04) is Professor of Signal Processing at King’s College London. He received his Dipl. Ing. and Mag. degrees from the University of Belgrade, Yugoslavia, the M.Phil. from Columbia University, and the Ph.D. in electrical engineering from the University of California, Berkeley. He held research positions at EPFL, Lausanne, Switzerland (1996), and at Harvard University (2002-04). Between 1997 and 2002, he was a member of the technical staff of AT&T Shannon Laboratory. His research interests are in the broad area of signal processing, ranging from theoretical aspects of signal analysis to applications in audio and speech technology, and neuroscience. He served as an Associate Editor of IEEE Transactions on Signal Processing. 

-2plus -1fil
[
    < g r a p h i c s >
]Hüseyin Hacıhabiboğlu (S'96-M'00-SM'12) is an Associate Professor of Signal Processing at Graduate School of Informatics, Middle East Technical University, Ankara, Turkey. He received the B.Sc. (honors) degree from the Middle East Technical University (METU), Ankara, Turkey, in 2000, the M.Sc. degree from the University of Bristol, Bristol, U.K., in 2001, both in electrical and electronic engineering, and the Ph.D. degree in computer science from Queen's University Belfast, Belfast, U.K., in 2004. He held research positions at University of Surrey, Guildford, U.K. (2004-2008) and King's College London, London, U.K. (2008-2011). His research interests include audio signal processing, room acoustics, multichannel audio systems, psychoacoustics of spatial hearing, microphone arrays, and game audio. Dr. Hacıhabiboğlu is a Senior Member of the IEEE, Member of the IEEE Signal Processing Society, Member of UKRI International Development Peer Review College, Audio Engineering Society (AES), Turkish Acoustics Society (TAD), and the European Acoustics Association (EAA) and an associate editor of IEEE/ACM Transactions on Audio, Speech, and Language Processing. 

-2plus -1fil
[
    < g r a p h i c s >
]Marc Moonen
 (M'94, SM'06, F'07) is a Full Professor at the Electrical Engineering
Department of KU Leuven, where he is heading a research team working in the area of numerical algorithms and
signal processing for digital communications, wireless communications,
DSL and audio signal processing.

He is a Fellow of the IEEE (2007) and a Fellow of EURASIP (2018).
He received the 1994 KU Leuven Research Council Award, the 1997 Alcatel
Bell (Belgium) Award (with Piet Vandaele), the 2004 Alcatel Bell
(Belgium) Award (with Raphael Cendrillon), and was a 1997 Laureate of
the Belgium Royal Academy of Science. He received journal best paper
awards from the IEEE Transactions on Signal Processing (with Geert Leus and with Daniele Giacobello)
and from Elsevier Signal Processing (with Simon Doclo).

He was chairman of the IEEE Benelux Signal Processing Chapter
(1998-2002), a member of the IEEE Signal Processing Society Technical Committee on Signal Processing for Communications, and President of EURASIP (European Association for Signal Processing, 2007-2008 and 2011-2012).
He has served as Editor-in-Chief for the EURASIP Journal on Applied
Signal Processing (2003-2005), Area Editor for Feature Articles in IEEE Signal Processing Magazine (2012-2014), and has been a member of the editorial board of Signal Processing, IEEE Transactions on Circuits and Systems II, IEEE Signal Processing Magazine, Integration-the VLSI Journal, EURASIP Journal on Wireless Communications and Networking and EURASIP Journal on Advances in Signal Processing.

-2plus -1fil
[
    < g r a p h i c s >
]Toon van Waterschoot(S'04, M'12) received MSc (2001) and PhD (2009) degrees in Electrical Engineering, both from KU Leuven, Belgium, where he is currently an Associate Professor and Consolidator Grantee of the European Research Council (ERC). He has previously also held teaching and research positions at Delft University of Technology in The Netherlands and the University of Lugano in Switzerland. His research interests are in signal processing, machine learning, and numerical optimization, applied to acoustic signal enhancement, acoustic modeling, audio analysis, and audio reproduction. 

He has been serving as an Associate Editor for the Journal of the Audio Engineering Society and for the EURASIP Journal on Audio, Music, and Speech Processing, and as a Guest Editor for Elsevier Signal Processing. He is a Director of the European Association for Signal Processing (EURASIP), a Member of the IEEE Audio and Acoustic Signal Processing Technical Committee, a Member of the EURASIP Special Area Team on Acoustic, Speech and Music Signal Processing, and a Founding Member of the EAA Technical Committee in Audio Signal Processing. He was the General Chair of the 60th AES International Conference in Leuven, Belgium (2016), and has been serving on the Organizing Committee of the European Conference on Computational Optimization (EUCCO 2016), the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA 2017), and the 28th European Signal Processing Conference (EUSIPCO 2020). He is a member of EURASIP, IEEE, ASA, and AES.





