Sound event localization and detection (SELD) is an emerging research topic that aims to unify the tasks of sound event detection and direction-of-arrival estimation. As a result, SELD inherits the challenges of both tasks, such as noise, reverberation, interference, polyphony, and non-stationarity of sound sources. Furthermore, SELD often faces an additional challenge of assigning correct correspondences between the detected sound classes and directions of arrival to multiple overlapping sound events. Previous studies have shown that unknown interferences in reverberant environments often cause major degradation in the performance of SELD systems. To further understand the challenges of the SELD task, we performed a detailed error analysis on two of our SELD systems, which both ranked second in the team category of DCASE SELD Challenge, one in 2020 and one in 2021. Experimental results indicate polyphony as the main challenge in SELD, due to the difficulty in detecting all sound events of interest. In addition, the SELD systems tend to make fewer errors for the polyphonic scenario that is dominant in the training set.

DCASE, error analysis, polyphony, sound event localization and detection

§ INTRODUCTION

Sound event localization and detection (SELD) has many applications in urban sound sensing [1], wildlife monitoring [2], surveillance [3], autonomous driving [4], and robotics [5]. SELD is an emerging research field that aims to combine the tasks of sound event detection (SED) and direction-of-arrival estimation (DOAE) by jointly recognizing the sound classes, and estimating the directions of arrival (DOA), the onsets, and offsets of detected sound events [6].

The introduction of the SELD task in the 2019 Challenge on Detection and Classification of Acoustic Scenes and Events (DCASE) has significantly accelerated SELD research. Many significant contributions have been made over the last three years in terms of datasets, evaluation metrics, and algorithms [7]. The TAU Spatial Sound Events dataset [8] used in DCASE 2019 included only stationary sound sources, with 72 room impulse responses (RIRs) from 5 different locations, and only 20 distinct samples for each of the 11 sound classes. The TAU-NIGENS Spatial Sound Events dataset [9] used in DCASE 2020 saw an introduction of moving sound sources, more RIRs from 15 different locations, and 14 sound classes extracted from the NIGENS General Sound Events Database [10], with around 30 to 50 distinct samples per class. The 2021 edition [11] introduced unknown directional interferences, making the sound scenes more realistic, in addition to the increase in the maximum polyphony of target events to three, from two in the 2019 and 2020 runs. The number of sound classes was reduced to 12, as some classes were used as interferences.

All three SELD datasets provide both first-order ambisonic (FOA) and microphone array (MIC) formats.

The SELD evaluation metrics have evolved over the past three years. In DCASE 2019, SED and DOAE performances were evaluated independently. Segment-wise error rate (ER) and F1 score evaluation were used for SED [12], while frame-wise DOA error and frame recall were used for DOAE [13]. Since 2020, SED and DOAE were evaluated jointly with location-dependent ER and F1 score for SED, and class-dependent localization error (LE) and localization recall (LR) for DOAE [14]. The 2021 metrics further take into account overlapping same-class events [11].

On the algorithm aspect, there have been many developments for SELD, inside and outside the DCASE Challenges, in the areas of data augmentation, feature engineering, model architectures, and output formats. In 2015, an early monophonic SELD work by Hirvonen [15] formulated SELD as a classification task, where each output class represents a sound class-location pair. In 2018, Adavannepioneered a seminal polyphonic SELD work that used a single-input multiple-output convolutional recurrent neural network (CRNN) model, SELDnet, to jointly detect sound events and estimate the corresponding DOAs [6]. In 2019, Caoproposed a two-stage strategy by training separate SED and DOA models [16], then using the SED outputs as masks to select the DOA outputs, significantly outperforming the jointly-trained SELDnet. Caolater proposed an end-to-end SELD network [17] that used soft parameter sharing between the SED and DOAE encoder branches and output trackwise predictions. An improved version of this network [18] replaced the bidirectional gated recurrent units (GRU) with multi-head self-attention (MHSA) to decode the SELD outputs [18]. In 2020, Shimadaproposed a new output format for SELD which unified SED and DOAE into one loss function [19]. This was amongst the few works which successfully used the linear-frequency for spectrograms and interchannel phase differences as input features, instead of the mel spectrograms. A new CNN architecture, D3Net [20], was adapted into a CRNN for this work and showed promising results. In another research direction, Nguyenproposed to solve SED and DOAE separately, use a bidirectional GRU to match the SED and DOAE output sequences, then produce event-wise SELD outputs [21, 22]. This was based on the observation that different sound events often have different onsets and offsets, resulting in temporal matching in the SED and DOAE output sequences. In 2021, Nguyenproposed a new input feature, SALSA, which spectrotemporally aligns the spatial cues with the signal power in the linear-frequency scale to improve SELD performance [23].

The top SELD system for DCASE 2019 trained four separate models for sound activity detection, SED, single-source DOAE, and two-source DOAE [24]. The top systems for both DCASE 2020 and 2021 synthesized a larger dataset from the original data, employed many data augmentation techniques, and combined different SELD models into ensembles [25, 26]. Other highly ranked solutions also intensively used data augmentation and ensemble methods.

Since SELD consists of both SED and DOAE tasks, it inherits many challenges from both SED and DOAE, such as noise, reverberation, interference, polyphony, and non-stationarity of sound sources. Furthermore, SELD often faces an additional challenge in correctly associating SED and DOAE outputs of multiple overlapping sound events. In an attempt to dissect the difficulties of the SELD task, Politiscompared the performances of the same SELD system in different acoustic environments [11] with different combinations of noise, reverberation, and unknown interferences. The authors founded that, in absence of unknown interferences, ambiance noise has little negative effects on SELD performance, while reverberation significantly reduces the SELD performance in all noise combinations. Unknown interferences degrade SELD performances by the largest margin compared to noise and reverberation. In addition, using the FOA format generally produces better performance than the MIC format.

To further understand the challenges facing SELD, we performed detailed error analysis on the SELD outputs, with the focus on polyphony, moving source, class-location interdependence, class-wise performance, and DOA errors, using our two SELD systems which both ranked second in the team category for the 2020 and 2021 DCASE Challenges [27, 23]. Experimental results showed that polyphony is the main factor that decreases the SELD performance across all the evaluation metrics, explaining why unknown interferences reduced the SELD performance by the largest extent. Interestingly, we also found that SELD systems do not necessarily favor single-source scenarios, which is easier than polyphonic cases. Instead, SELD systems achieved lower error rates in polyphonic cases which dominate the training dataset. The rest of the paper is organized as follows. <Ref> describes our analysis method. <Ref> presents the experimental results and discussions. Finally, we conclude the paper in <Ref>.

§ ANALYSIS METHOD

In this section, brief descriptions of the SELD datasets and systems are provided. Error analyses were performed on the SELD outputs of the two SELD systems which both ranked second in the team category for the 2020 and 2021 DCASE Challenges [27, 23]. The 2021 version of the evaluation metrics was used in all analyses. For convenience, the TAU-NIGENS Spatial Sound Events 2020 and 2021 datasets used in the DCASE Challenges [9, 11] are referred to here as the SELD 2020 and 2021 datasets, respectively.

§.§ Dataset

Characteristics     2020     2021

Channel format                      FOA       FOA

Moving sources

Ambiance noise

Unknown interferences

Maximum degree of polyphony         2         3

Number of target sound classes      14        12

Evaluation split                    eval      test

Comparison between 2020 and 2021 SELD datasets

Table <ref> summarizes some differences between the two SELD datasets. Since both of the SELD systems require the FOA format, only the FOA subset of the datasets were used in our experiments. Each of the dataset consists of 400, 100, 100, and 200 one-minute audio recordings for the train, validation, test, and evaluation splits respectively. The azimuth and elevation ranges are $[\SI{-180}{\degree}, \SI{180}{\degree})$ and $[\SI{-45}{\degree}, \SI{45}{\degree}]$, respectively.
During the developmental stage, the validation set was used for model selection while the test set was used for evaluation. During the evaluation stage, the train, validation, and test data (collectively known as the development split) were used for training evaluation models. For the 2020 SELD dataset, the results on the evaluation split were used for the error analyses. Since the ground truth for the evaluation split of the 2021 SELD dataset has not been publicly released at the time of writing, the results on the test split of the 2021 SELD dataset were used for error analysis instead.

§.§ Evaluation metrics

To evaluate the SELD performance, we used the official SELD evaluation metrics [7] from the DCASE 2021 Challenge. The metrics not only jointly evaluate SED and DOAE, but also take into account the cases where multiple instances of the same class overlap. The SELD evaluation metrics consist of location-dependent error rate ($\text{ER}_{\le T}$) and F1 score ($\text{F}_{\le T}$) for SED; and class-dependent localization error ($\text{LE}_\text{CD}$), and localization recall ($\text{LR}_\text{CD}$) for DOAE. A sound event is considered a correct detection only if it has a correct class prediction and its estimated DOA is also less than $T$ away from the DOA ground truth, where $T=\SI{20}{\degree}$ for the official challenge. The DOAE metrics are also class-dependent, that is, the detected DOA is only counted if its corresponding detected sound class is correct. A good SELD system should have low $\text{ER}_{\le T}$, high $\text{F}_{\le T}$, low $\text{LE}_\text{CD}$, and high $\text{LR}_\text{CD}$.

§.§ SELD systems

We denote two of our SELD systems that ranked second in the team categories of the 2020 and 2021 DCASE challenges as NTU'20 and NTU'21, respectively. Table <ref> shows the performances of the baselines, the top-ranked solutions, and our second-ranked systems in 2020 and 2021. NTU'20 is an ensemble of sequence matching networks [21, 27] while NTU'21 is an ensemble of different models trained on our new proposed SALSA features for SELD [23]. Both systems use the class-wise output format, which can only detect a maximum of one event of a particular class at a time. Both systems outperformed the respective baselines by a large margin, and only perform slightly worse than the respective top-ranked system. The 2020 results in Table <ref> were computed using the 2020 SELD evaluation metrics. For subsequent sections, the results of the NTU'20 system were recomputed using the 2021 metrics.

Year     System
$\text{ER}_{\le \SI{20}{\degree}}$
$\text{F}_{\le \SI{20}{\degree}}$

Baseline [9]
0.69     0.413     23.1     0.624

#1: USTC'20 [25]
0.20     0.849     6.0     0.885

#2: NTU'20 [27]
0.23     0.820     9.3     0.900

Baseline [11]
0.73     0.307     24.5     0.448

#1: Sony'21 [26]
0.43     0.699     11.1     0.732

#2: NTU'21 [23]
0.37     0.737     11.2     0.741

Performance of selected SELD systems.

§ EXPERIMENTAL RESULTS AND DISCUSSION

In each subsection concerning a factor of variation, we performed an analysis on the data distribution of 2020 and 2021 SELD datasets, followed by an analysis of the SELD results. Overall, the 2021 dataset is much more challenging than the 2020 dataset. For detailed analyses, $\text{ER}_{\le T}$ is further broken down into substitution, deletion, and insertion errors, while $\text{F}_{\le T}$ is further broken down into precision and recall. Since the SELD metrics are segment-based, i.e., outputs are divided into segments of 1 before being evaluated, we used the provided ground truth to group the segments based on polyphony (0, 1, 2, and 3 sources), static and moving sources to compute the metrics for each case.

§.§ Effect of polyphony

(a) Polyphonic distribution

(b) Static vs moving

Segment-wise polyphonic and static distribution per year.

(lr)2-4 (lr)5-8

$\downarrow$ $\text{ER}_{\le \SI{20}{\degree}}$

$\downarrow$ Substitution

$\downarrow$ Deletion

$\downarrow$ Insertion

$\uparrow$ $\text{F}_{\le \SI{20}{\degree}}$

$\uparrow$ Precision

$\uparrow$ Recall

$\downarrow$ $\text{LE}_\text{CD}$

$\uparrow$ $\text{LR}_\text{CD}$

SELD performance w.r.t. degree of polyphony

<Ref>(a) shows the segment-wise polyphonic distribution of 2020 and 2021 datasets, which are dominated by single-source and two-source segments, respectively. On average, there are 1.11 and 1.85 events per segment in the 2020 and 2021 datasets, respectively. <Ref> shows the breakdown of the SELD performance for each polyphonic case. The DOAE metrics clearly show that polyphony is a major cause of performance degradation. For both NTU'20 and NTU'21 systems, as the number of overlapping sources increases, increases and decreases. Interestingly, polyphony does not always degrade SED performance. The peak performances of $\text{ER}_{\le \SI{20}{\degree}}$ and precision were achieved in the degree of polyphony that dominates the respective dataset, which is single-source for the 2020 dataset and two-source for the 2021 dataset. This result suggests that one possible solution to tackle polyphony is to introduce more data samples for difficult cases.

When the number of overlapping sources increases, the SED error compositions also change. The deletion error rate rapidly increases, the insertion error rate sharply decreases, and the substitution error rate increases. In addition, the recall rate decreases significantly. It is clear that the SELD systems struggle to detect all the present events in polyphonic cases.

In the absence of any event of interest, the insertion error rates are 0.030 and 0.122 for NTU'20 and NTU'21 systems, respectively. When comparing the SELD performances between the 2020 and 2021 setups, the single-source results in 2021 are significantly worse than those in 2020 across all metrics. In addition, the substitution errors across all degrees of polyphony are much higher in the 2021 setup, than in 2020. These results show the detrimental effect of unknown interferences that were introduced in the 2021 dataset, consistent with the findings in [11].

§.§ Effect of moving sound sources

(lr)2-4 (lr)5-7

$\downarrow$ $\text{ER}_{\le \SI{20}{\degree}}$
0.214       0.239       0.232
0.379         0.357     0.372

$\uparrow$ $\text{F}_{\le \SI{20}{\degree}}$
0.854       0.841       0.845
0.731         0.745       0.737

$\downarrow$ $\text{LE}_\text{CD}$
8.7        10.0       9.4
10.5       11.7       11.2

$\uparrow$ $\text{LR}_\text{CD}$
0.847       0.846       0.846
0.725       0.751       0.741

$\downarrow$ $\text{ER}_{\le \SI{180}{\degree}}$
0.166       0.168       0.171
0.334       0.298       0.318

$\uparrow$ $\text{F}_{\le \SI{180}{\degree}}$
0.898       0.891       0.892
0.778       0.800       0.789

SELD performance of static and moving sources.

<Ref>(b) shows the segment-wise distribution of static and moving sound sources, not counting empty segments, based on the provided ground truth. A segment is considered a moving one if at least one sound source is moving. Since there are more overlapping sources in the 2021 dataset, the proportion of moving segments is significantly higher than the 2020 dataset. <Ref> presents the SELD performance for both cases. The $\text{LE}_\text{CD}$ of moving-source cases is higher than those of static-source cases, as expected. For the 2020 dataset, the $\text{LR}_\text{CD}$ are similar for both cases, and the performance gap for SED disappears when we compute location-independent SED metrics (by setting the DOA threshold to $T=180\si{\degree}$). These results suggest that moving sources have little effect on SED performance and mainly affect DOAE. For the 2021 dataset, all metrics are better for moving-source cases compared to single-source cases. This contradictory result may be due to the skewed distribution and requires further investigation once the evaluation ground truth is made available.

§.§ Class and location interdependency

SED performance across different DOA thresholds.

(a) Localization error

(b) Localization recall

Localization error and recall by class dependencies.

To understand the dependency of location-dependent SED metrics on the correctness of the detected DOAs, we investigate the effect of the different DOA thresholds $T\si{\degree}$ on $\text{ER}_{\le T\si{\degree}}$ and $\text{F}_{\le T\si{\degree}}$, as shown in <Ref>. The gaps between the SED metrics for $T=\SI{20}{\degree}$ and the location-independent $T=\SI{180}{\degree}$ are not significantly large, suggesting that many estimated DOAs are within the $\SI{20}{\degree}$ threshold. However, the location-dependent SED metrics deteriorate quickly as the DOA threshold reduces to $10\si{\degree}$, suggesting a significant number of the estimated DOAs deviate by more than $10\si{\degree}$ from the ground truth.

To understand the dependency of classification-dependent DOA metrics on the correctness of the predicted classes, we show the classification-dependent and classification-independent LE and LD in <Ref>. When not accounting for the predicted class, the LR significantly increases, leading to some unwanted rise in LE.

§.§ Class-wise performance

(a) Class distribution

(b) Class-wise $\text{F}_{\le \SI{20}{\degree}}$

Segment-wise class distribution of 2021 SELD dataset (test split) and class-wise location-dependent F score of NTU'21 system.

Due to space constraints, we only included the segment-wise class distribution and the class-wise performance of 2021 setup in <Ref>. The segment-wise class distribution in <Ref>(a) is highly skewed, with the footstep class accounting for the highest proportion of 21.2, while the female speech accounting for the lowest at 1.3. However, the class-wise
$\text{F}_{\le \SI{20}{\degree}}$ scores are more even, and the class with the highest segment-wise proportion does not correspond to highest $\text{F}_{\le \SI{20}{\degree}}$ score. One possible reason is that it is difficult to detect all footstep sound due to discontinuities, low bandwidth, and low energy. In addition, class-wise performance is highly dependent on the SELD model and the quality of training samples. Interestingly, the female speech class with the highest $\text{F}_{\le \SI{20}{\degree}}$ score of 94.2 has the lowest segment-wise proportion. Other classes such as knock and male speech also have high $\text{F}_{\le \SI{20}{\degree}}$ scores despite the low segment-wise proportions.

§.§ Azimuth vs elevation error

For the NTU'20 system, the $\text{LE}_\text{CD}$ contributed by azimuth and elevation are 6.3 and 5.3, respectively. For the NTU'21 system, the $\text{LE}_\text{CD}$ contributed by azimuth and elevation are 7.9 and 6.2, respectively. The azimuth and elevation errors are similar although the azimuth range of $[\SI{-180}{\degree}, \SI{180}{\degree})$ is much larger than elevation range of $[\SI{-45}{\degree}, \SI{45}{\degree}]$, suggesting that it is more difficult to estimate elevation angles than azimuth angles.

§ CONCLUSION

In realistic acoustic conditions with noise and reverberation, polyphony and unknown interferences appear to be the biggest challenges for SELD. In the presence of unknown interferences, SELD systems tend to make more substitution errors. When there are several sound events, either due to polyphony or unknown interferences, the SELD systems struggle to detect all events of interests, leading to low recall and high deletion error rate. Interestingly, the overall SED error rate is at the lowest for the polyphonic case that dominates the dataset. Moving sound sources mainly increase the localization errors, leading to small reduction in location-dependent SED metrics. High segment-wise representation of a class also does not necessary translate to high SED performances. Localization error reduction poses significant challenge beyond a threshold, especially as elevation errors are often as high as azimuth errors. The study of same-class polyphonic events is left for future works due to the limitations of the current systems studied.

[1]
J. Salamon and J. P. Bello, “Deep Convolutional Neural Networks and Data
Augmentation for Environmental Sound Classification,”
IEEE Signal Process. Lett., vol. 24, no. 3, pp.
279–283, 2017.

[2]
D. Stowell, M. Wood, Y. Stylianou, and H. Glotin, “Bird detection in audio: A
survey and a challenge,” in IEEE International
Workshop on Machine Learning for Signal Processing, 2016, pp. 1–6.

[3]
P. Foggia, N. Petkov, A. Saggese, N. Strisciuglio, and M. Vento, “Audio
Surveillance of Roads: A System for Detecting Anomalous Sounds,”
IEEE Trans. Intell. Transp. Syst., vol. 17,
no. 1, pp. 279–288, 2016.

[4]
M. K. Nandwana and T. Hasan, “Towards smart-cars that can listen: Abnormal
acoustic event detection on the road,” in
Proceedings of the Annual Conference of the
International Speech Communication Association, 2016, pp. 2968–2971.

[5]
J. M. Valin, F. Michaud, B. Hadjou, and J. Rouat, “Localization of
simultaneous moving sound sources for mobile robot using a frequency-domain
steered beamformer approach,” in Proceedings of
the IEEE International Conference on Robotics and Automation, 2004, pp.

[6]
S. Adavanne, A. Politis, J. Nikunen, and T. Virtanen, “Sound Event
Localization and Detection of Overlapping Sources Using Convolutional
Recurrent Neural Networks,” IEEE J. Sel. Topics
Signal Process., vol. 13, no. 1, pp. 34–48, 2019.

[7]
A. Politis, A. Mesaros, S. Adavanne, T. Heittola, and T. Virtanen, “Overview
and Evaluation of Sound Event Localization and Detection in {DCASE}
2019,” IEEE/ACM Trans. Audio, Speech, Lang.
Process., vol. 29, pp. 684–698, 2020.

[8]
S. Adavanne, A. Politis, and T. Virtanen, “A Multi-room Reverberant Dataset
for Sound Event Localization and Detection,” in
Proceedings of the 4th Workshop on Detection and
Classification of Acoustic Scenes and Events, 2019, pp. 10–14.

[9]
A. Politis, S. Adavanne, and T. Virtanen, “A Dataset of Reverberant Spatial
Sound Scenes with Moving Sources for Sound Event Localization and
Detection,” arXiv, 2020.

[10]
I. Trowitzsch, J. Taghia, Y. Kashef, and K. Obermayer, “The NIGENS General
Sound Events Database,” arXiv, 2019.

[11]
A. Politis, S. Adavanne, D. Krause, A. Deleforge, P. Srivastava, and
T. Virtanen, “A Dataset of Dynamic Reverberant Sound Scenes with
Directional Interferers for Sound Event Localization and Detection,”
arXiv, 2021.

[12]
A. Mesaros, T. Heittola, and T. Virtanen, “Metrics for polyphonic sound event
detection,” Applied Sciences (Switzerland),
vol. 6, no. 6, p. 162, 2016.

[13]
S. Adavanne, A. Politis, and T. Virtanen, “Direction of arrival estimation
for multiple sound sources using convolutional recurrent neural network,”
in Proceedings of the European Signal Processing
Conference, 2018, pp. 1462–1466.

[14]
A. Mesaros, S. Adavanne, A. Politis, T. Heittola, and T. Virtanen, “Joint
measurement of localization and detection of sound events,” in
Proceedings of the IEEE Workshop on Applications
of Signal Processing to Audio and Acoustics, 2019, pp. 333–337.

[15]
T. Hirvonen, “Classification of spatial audio location and content using
Convolutional neural networks,” in Proceedings
of the 138th Audio Engineering Society Convention 2015, 2015, pp. 622–631.

[16]
Y. Cao, Q. Kong, T. Iqbal, F. An, W. Wang, and M. D. Plumbley, “Polyphonic
Sound Event Detection and Localization using a Two-Stage Strategy,” in
Proceedings of the 4th Workshop on Detection and
Classification of Acoustic Scenes and Events, 2019.

[17]
Y. Cao, T. Iqbal, Q. Kong, Y. Zhong, W. Wang, and M. D. Plumbley,
“Event-independent Network for Polyphonic Sound Event Localization and
Detection,” in Proceedings of the 5th Workshop
on Detection and Classification of Acoustic Scenes and Events, 2020.

[18]
Y. Cao, T. Iqbal, Q. Kong, F. An, W. Wang, and M. D. Plumbley, “An Improved
Event-Independent Network for Polyphonic Sound Event Localization and
Detection,” in Proceedings of the IEEE
International Conference on Acoustics, Speech and Signal Processing, 2021,
pp. 885–889.

[19]
K. Shimada, Y. Koyama, N. Takahashi, S. Takahashi, and Y. Mitsufuji, “ACCDOA:
Activity-Coupled Cartesian Direction of Arrival Representation for Sound
Event Localization And Detection,” in
Proceedings of the IEEE International Conference
on Acoustics, Speech and Signal Processing, 2021, pp. 915–919.

[20]
N. Takahashi and Y. Mitsufuji, “Densely connected multidilated convolutional
networks for dense prediction tasks,” in
Proceedings of the IEEE/CVF Conference on
Computer Vision and Pattern Recognition, 2021.

[21]
T. N. T. Nguyen, D. L. Jones, and W. Gan, “A Sequence Matching Network for
Polyphonic Sound Event Localization and Detection,” in
Proceedings of the IEEE International Conference
on Acoustics, Speech and Signal Processing, 2020, pp. 71–75.

[22]
T. N. T. Nguyen, N. K. Nguyen, H. Phan, L. Pham, K. Ooi, D. L. Jones, and W.-S.
Gan, “A General Network Architecture for Sound Event Localization and
Detection Using Transfer Learning and Recurrent Neural Network,” in
Proceedings of the IEEE International Conference
on Acoustics, Speech and Signal Processing, 2021, pp. 935–939.

[23]
T. N. T. Nguyen, K. Watcharasupat, N. K. Nguyen, D. L. Jones, and W. S. Gan,
“DCASE 2021 Task 3: Spectrotemporally-aligned Features for Polyphonic Sound
Event Localization and Detection,” IEEE AASP
Challenge on Detection and Classification of Acoustic Scenes and Events
2021, 2021.

[24]
S. Kapka and M. Lewandowski, “Sound Source Detection, Localization and
Classification using Consecutive Ensemble of CRNN Models,”
IEEE AASP Challenge on Detection and
Classification of Acoustic Scenes and Events 2019, 2019.

[25]
Q. Wang, H. Wu, Z. Jing, F. Ma, Y. Fang, Y. Wang, T. Chen, J. Pan, J. Du, and
C.-H. Lee, “The USTC-iFlytek System for Sound Event Localization and
Detection of DCASE2020 Challenge,” IEEE AASP
Challenge on Detection and Classification of Acoustic Scenes and Events
2020, 2020.

[26]
K. Shimada, N. Takahashi, Y. Koyama, S. Takahashi, E. Tsunoo, M. Takahashi, and
Y. Mitsufuji, “Ensemble of ACCDOA- and EINV2-based Systems with D3Nets and
Impulse Response Simulation for Sound Event Localization and Detection,”
IEEE AASP Challenge on Detection and
Classification of Acoustic Scenes and Events 2021, 2021.

[27]
T. N. T. Nguyen, D. L. Jones, and W. S. Gan, “Ensemble of sequence matching
networks for dynamic sound event localization, detection, and tracking,” in
Proceedings of the 5th Workshop on Detection and
Classification of Acoustic Scenes and Events, 2020, pp. 120–124.