1]Beijing Key Laboratory of Computational Intelligence and Intelligent System
, Faculty of Information Technology, Beijing University of Technology,
Beijing, 100124, China

2]AcousticDSP Consulting LLC, St Paul, MN, United States

# Exploring the Power of Pure Attention Mechanisms in Blind Room Parameter
Estimation

Chunxi Wang<EMAIL_ADDRESS>Maoshen Jia<EMAIL_ADDRESS>Meiran Li<EMAIL_ADDRESS>Changchun Bao<EMAIL_ADDRESS>Wenyu
Jin<EMAIL_ADDRESS>[  [

###### Abstract

Dynamic parameterization of acoustic environments has drawn widespread
attention in the field of audio processing. Precise representation of local
room acoustic characteristics is crucial when designing audio filters for
various audio rendering applications. Key parameters in this context include
reverberation time (RT60) and geometric room volume. In recent years, neural
networks have been extensively applied in the task of blind room parameter
estimation. However, there remains a question of whether pure attention
mechanisms can achieve superior performance in this task. To address this
issue, this study employs blind room parameter estimation based on monaural
noisy speech signals. Various model architectures are investigated, including
a proposed attention-based model. This model is a convolution-free Audio
Spectrogram Transformer, utilizing patch splitting, attention mechanisms, and
cross-modality transfer learning from a pretrained Vision Transformer.
Experimental results suggest that the proposed attention mechanism-based
model, relying purely on attention mechanisms without using convolution,
exhibits significantly improved performance across various room parameter
estimation tasks, especially with the help of dedicated pretraining and data
augmentation schemes. Additionally, the model demonstrates more advantageous
adaptability and robustness when handling variable-length audio inputs
compared to existing methods.

###### keywords:

Acoustic Environments, Blind Room Parameter Estimation, Pure Attention
Mechanisms

##  1 Introduction

In recent years, there has been a growing focus on the dynamic
parameterization of evolving acoustic environments. The parameters that
describe local rooms or other acoustic spaces hold significance as they can be
harnessed in the modelling and design of audio filters for a diverse range of
applications. Understanding the specific acoustic properties of the
surrounding room can be applied to improve speech signals and support
dereverberation algorithms, ultimately improving word error rate for Automatic
Speech Recognition (ASR) and the clarity of voice communication systems [1, 2,
3]. Additionally, spatial sound reproduction systems could leverage this data
to enhance their performance in tasks related to acoustic room equalization
either using predefined filters [4, 5] or in an adaptive manner [6].

Furthermore, the successful realization of audio augmented reality (AAR)
necessitates the seamless integration of virtual acoustic objects into the
physical environment. This integration underscores the importance of achieving
a harmonious alignment between the acoustical properties of virtual elements
and the characteristics of the local space [7]. In pursuit of this goal, a
significant challenge lies in the accurate estimation of related acoustical
parameters of a room to enhance the realism of immersive audio. Notably, Jot
et al. [8] introduced the concept of a “reverberation fingerprint”, comprising
the room’s volume and its frequency-dependent diffuse reverberation decay
time. This innovative concept was proposed to characterize rooms specifically
for the realistic binaural rendering achievable with audio augmented reality
headphones. It’s worth noting that this fingerprint primarily focuses on the
part of reverberation that is independent of the position, treating a room’s
acoustic characteristics in isolation from the orientation and directivity of
sound sources and receivers.

Conventionally, room parameters like reverberation time (RT60) and direct-to-
reverberant ratio (DRR) are typically obtained through a direct analysis of
measured Room Impulse Responses (RIRs). Meanwhile, room volume is closely
linked to the determination of a concept known as the “critical distance.”
This critical distance is defined as the distance at which the direct and
reverberant power components of a sound source become equal, effectively
making the DRR reach 0 dB. In cases where we assume an ideal diffuse
soundfield, the relationship between these parameters is mathematically
described by Sabine’s well-known equation [9]:

$RT_{60}(b)\approx 0.16\frac{V}{\alpha(b)\cdot S},$ (1)

where $S$ denotes the total area of the room’s surfaces and $\alpha(b)$ is the
area-weighted mean absorption coefficient in octave band $b$.

In practical applications, conducting in-situ measurements of RIRs and
determining the volumes of users’ local acoustic spaces can often present
significant challenges. A compelling alternative involves blind estimation of
room acoustic parameters from audio recordings obtained using microphones,
even when the sound sources are unknown and in the presence of background
noise. The 2015 ACE challenge [10] established a benchmark for blind
estimation of RT60 and DRR from noisy speech recordings. The leading systems
in this challenge primarily relied on signal modeling-based approaches [11,
12]. Meanwhile, room volume estimation has long been formulated as a
classification problem [13, 14]. On the other hand, the estimation of room
volume has long been framed as a classification problem. Audio forensics
systems described in [13, 14] make use of Mel-frequency cepstral coefficients
(MFCC)-based features to identify the specific room associated with an
environmental sound or speech recording, typically within a predefined closed
set of possibilities.

Due to the recent advancements in Deep Neural Networks (DNNs), there is a
growing trend to reframe the challenge of blind room acoustic parameter
estimation as a regression problem. This approach leverages Convolutional
Neural Network (CNN) models in combination with time-frequency
representations, offering an increasingly relevant and effective solution.
Gamper et al. [15] introduced a CNN designed to directly estimate RT60 from a
four-second recording of reverberant speech. The experimental results
demonstrate that this CNN outperforms other methods in the ACE challenge,
offering both superior performance and higher computational efficiency. The
same approach was also applied to blind volume estimation in [16] and results
show that it can estimate a broad range of volumes within a factor of 2 on
real-measured data. CNN-based systems with similar methodologies have been put
forward for the blind estimation of room acoustic parameters, utilizing either
single-channel [17, 18, 19] or multi-channel speech signals [20]. These
systems have showcased promising outcomes in terms of both accurate parameter
estimation and resilience to temporal variations in dynamic acoustic
environments. Notably, in contrast to the conventional approach of log-energy
calculations for spectro-temporal features used in prior studies, Ick et al.
[21] introduced a set of phase-related features. Their research demonstrated
clear improvements in the context of estimating reverberation fingerprints for
real-world rooms that had not been previously seen, highlighting the enhanced
efficacy of this method.

CNNs are widely considered in the fore-mentioned approaches due to their
suitability for learning two-dimensional time-frequency signal patterns for
end-to-end modelling. CNNs can be extended by a recurrent layer to form
convolutional recurrent neural networks (CRNN) that exploit sequential
dependencies in the data [22] and improve the capability of processing input
sequences of variable length [23]. To further enhance the capture of long-
range global context, hybrid models combining Convolutional Neural Networks
(CNNs) with self-attention mechanisms have yielded state-of-the-art results in
a range of tasks, including acoustic event classification [24, 25] and various
audio pattern recognition endeavors [26, 27]. Gong et al. [28] pushed the
boundaries even further by introducing purely attention-based models for audio
classification. Their creation, the Audio Spectrogram Transformer (AST), was
evaluated on several audio classification benchmarks, achieving new state-of-
the-art results. This underscores that CNNs may not always be essential in
this particular context.

Building on the inspiration derived from the research presented in [28], our
study introduces a convolution-free, purely attention-based model for the
blind estimation of acoustic room parameters by extending our previous work in
[29]. To the best of our knowledge, this marks the inaugural application of an
attention-based system in the field of blind acoustic room parameter
estimation. The proposed system utilizes Gammatone magnitude spectral
coefficients as well as the low-frequency phase spectrogram as inputs and
captures long-range global context, even in the lower layers of the model.
Furthermore, to enhance system performance, we apply transfer learning through
the use of a pretrained transformer model from ImageNet. For the evaluation of
the proposed method, we curate a RIR corpus that includes publicly available
RIRs, synthesized RIRs, and RIRs obtained through in-house measurements of
real-world rooms. Experimental results clearly demonstrate the superiority of
our proposed model when compared to CNN-based blind acoustic parameter
estimation systems, particularly when dealing with previously unseen real-
world rooms using single-channel recordings of variable length.

The remainder of the article is organized as follows.

Section 2 introduces the construction of RIR datasets, including real-world
and simulated datasets. Section 3 demonstrates the generation of audio data
with reverberation and noise using constructed RIR datasets, followed by data
preprocessing, augmentation, and feature extraction schemes for neural network
training. Section 4 details the model structures of a CNN-based model, a CRNN-
based model, the proposed attention-based systems. Section 5 conducts a
comprehensive evaluation of the proposed system against state-of-the-art
methods in various room parameter estimation tasks and its performance under
variable-length inputs. Section 6 draws the conclusion.

##  2 Data generation pipeline

Applying neural network methods to address blind room parameter estimation is
a challenging task, as it generally requires a substantial amount of data.
Since this task necessitates the need of having audio samples from rooms with
various acoustic characteristics, manually creating a suitably diverse dataset
would incur exorbitant costs and time. In this work, audio samples are created
from public real-world RIR datasets, the BJUT Reverb dataset, and a room-
simulation-based RIR dataset.

###  2.1 Public Real-world RIR Datasets

In this study, six publicly available real-world RIR datasets that include 44
authentic rooms are considered, with the aim of encompassing a wide range of
acoustic room parameters.

The majority of the data targets at geometrically regular rooms, including
spaces like offices, classrooms, and auditoriums/lecture halls. These datasets
include the ACE Challenge dataset [10], the Aachen Impulse Response (AIR)
dataset [30], the Brno University of Technology Reverb Database (BUT ReverbDB)
[31], the C4DM dataset [32], and the dEchorate dataset [33]. Additionally, the
OpenAIR dataset [34] primarily covers larger acoustic spaces, such as
churches, nuclear reactor halls, and other substantial structures. As a
result, a large variety of real-world room configurations with different
volume parameters are incorporated.

Furthermore, RT60 values vary widely, ranging from less than half a second to
over ten seconds, and these values are calculated using the Schroeder method
[35].

In addition to the public datasets described above, RIRs from 11 distinct
rooms at the campus of Beijing University of Technology were measured,
including elevator shafts, classrooms, auditoriums, seminar rooms, and more.
The parameters of these selected rooms were recorded. The aim of this endeavor
is to bridge the natural gap in available real-world acoustic spaces within
the volume range of 12$m^{3}$ to 7000$m^{3}$. Three RIR measurements were
conducted at different positions within the selected rooms. Specifically,
measurements were taken at the geometric center of the room, a location near
the wall, and a position near the corner, to capture the RIR with a sequence
length of 4 seconds. The microphone and loudspeaker positions are illustrated
in Fig. 1.

Figure 1: Room Measurement Layout Diagram

###  2.2 Simulated RIR Dataset

The real-world data is supplemented by introducing 30 simulated RIRs derived
from virtual rooms with various geometries. This aims to enhance the dataset’s
representation of less frequently encountered room volumes, thereby achieving
a normal distribution of total volume.

The specific approach involves simulating a single sound source positioned
near the center of each virtual room and evenly distributing five-point
receivers throughout the volume of each room. To create this synthetic
dataset, the pyroomacoustics [36] software package is deployed, which utilizes
the image-source model to simulate RIRs for rooms with specific volumes.
Although this geometric model does not account for phenomena like diffraction
and scattering, empirical evidence demonstrates that the utilization of
simulated data contributes to enhancing the model’s performance, enabling it
to effectively generalize to real-world data [16].

##  3 Preprocessing

In this section, we provide a detailed explanation of how audio data with
reverberation and noise is generated. We started with convolving acoustic
response with audio signals and adding various types of noises for subsequent
neural network comparisons.

To ensure the quality and consistency of the dataset, we performed a series of
data preprocessing. Firstly, we partitioned the audio signals into training,
validation, and test sets. Only real-world RIRs were selected in the test set
to asses system performance on unseen non-simulated rooms.

Furthermore, we employed a data augmentation technique called SpecAugment that
aims to enhance the neural network’s ability to generalize in unknown rooms
and noisy environments.

Lastly, we discussed the method for audio feature extraction. Gammatone ERB
filterbank was used to generate time-frequency representations. After
processing, these features resulted in a two-dimensional feature block used as
input to the neural network, allowing it to handle various datasets and
provide accurate blind room parameter estimation performance.

###  3.1 Audio Generation

In the acquired RIR dataset, a total of 55 real-world rooms and 30 simulated
rooms are included, comprising a total of 570 RIRs. The volume labels span
from 11.88$m^{3}$ to 21,000$m^{3}$, while the range of RT60 varies from
0.41$s$ to 19.68$s$. Due to the significant differences in volume labels
spanning multiple orders of magnitude, we chose to represent them using a
logarithmic base 10 scale. Additionally, to ensure consistency across all
datasets, all RIRs were downsampled to 16 kHz. The distribution of volume and
RT60 in different datasets is shown in Fig. 2.

From a given RIR dataset with room parameter labels, we generated audio data
with reverberation and noise for the purpose of feeding it into different
neural networks for comparison. To achieve this, we mapped the acoustic
response $r(t)$ of different types of rooms in the RIR dataset onto the audio
signal $y(t)$.

We used source speech signals $x(t)$ recorded in anechoic chambers without
reverberation and convolve them with $r(t)$ in the time domain. The source
speech signals $x(t)$ are obtained from the ACE dataset [10], where samples
are recorded without reverberation, and include both male and female speakers.
In the RIR dataset, some rooms have more RIR measurements than others. To
ensure a uniform representation, each room is equally sampled, so that the
distribution of audio samples in our dataset matches the volume distribution
in our RIR dataset.

Additional noise signals $n(t)$ were added to simulate recordings at four
different signal-to-noise ratio (SNR) levels, including [+30, +20, +10, +0]
decibels. The noise $n(t)$ comprises two types of noise, namely white noise
and babble noise [37].

In summary, the audio signal $y(t)$ is constructed by convolving source speech
signals $x(t)$ with room impulse response $r(t)$ and adding additional noise
$n(t)$, represented as:

$\displaystyle y(t)$ $\displaystyle=x(t)*r(t)+n(t)$ $\displaystyle=s(t)+n(t)$
(2)

Here, $t$ represents the discrete time index. $s(t)$ represents the
reverberation audio signal without noise.

Figure 2: The distribution graphs of volume and RT60 in different datasets.

We spilt 32,000 audio signals $y(t)$ into training, validation, and test sets
in a 6-2-2 ratio. During the training process, we randomly sampled a subset
from both real and simulated rooms for room validation. A subset was also
extracted from real-world unseen rooms for room testing. Rooms with these
specific parameters were not included in the training set. The purpose of this
step is to assess whether the model, when confronted with room parameters not
encountered during the training process, can still demonstrate robust
predictive performance under noisy and reverberant conditions. Overall,
_Dataset I_ is formulated as listed in Table 1.

Table 1: Summary of Data Splits for _Datasets I_ & _II_ Data |  $\\#$ of |  $\\#$ of | Real | Simulated
---|---|---|---|---
Spilt | _Dataset I_ | _Dataset II_ | Rooms | Rooms
Train | 19200 | 24000 | 34 | 18
Validation | 6400 | 6400 | 21 | 12
Test | 6400 | 6400 | 21 | 0

###  3.2 Audio Augmentation

To enhance the generalizability of neural networks in unknown rooms and noisy
environments, we employed the widely-used data augmentation technique known as
SpecAugment [38]. This method enhances the model’s robustness to unknown
conditions through modifications and augmentations to the available training
data. Specifically, we selected reverberation signals without noise $s(t)$ as
described in equation 2. Subsequently, these audio signals were transformed
into log Mel-frequency spectrograms and subjected to time warping, frequency
masking, and time masking, as shown in Fig. 3.

Figure 3: Augmentation schemes applied to reverberation signals without noise.

Time warping was implemented using TensorFlow’s sparse image warping function.
For a log Mel-frequency spectrogram with $\tau$ time steps, we treated it as
an image with the time axis horizontal and the frequency axis vertical. In the
image, we randomly selected points within the interval $[W_{m},\tau-W_{m}]$
located between time steps and applied random warping. The warping distance,
$w$, was chosen uniformly from the range $[0,W_{m}]$, where $W_{m}$ is the
time warp parameter. Six anchor points were fixed at the boundaries of the
image, including the four corners and the midpoints of the vertical edges.

Frequency masking was applied as follows: a continuous set of Mel frequency
channels $[f_{0},f_{0}+f_{1})$ is masked, where $f_{1}$ is initially chosen
from a uniform distribution $[0,F_{m}]$, and $f_{0}$ is chosen from $[0,\nu-
f_{1})$, where $F_{m}$ is the frequency mask parameter, and $\nu$ is the
number of Mel frequency channels.

Time masking was applied in a similar manner: a continuous set of time steps
$[t_{0},t_{0}+t_{1})$ is masked, with $t_{1}$ being initially chosen from a
uniform distribution $[0,T_{m}]$, and $t_{0}$ chosen from $[0,\tau-t_{1})$,
where $T_{m}$ is the time mask parameter, and $\tau$ is the total number of
time steps. The Mel-frequency spectrogram with masking applied is then
converted back to the time-domain signal.

Finally, 4800 speech sequences with these masking effects were added to the
original training dataset for neural network training, and this dataset is
labeled as _Dataset II_ , as shown in Table 1. This comprehensive data
augmentation strategy aims to help the neural network better adapt to various
environments and conditions, ultimately improving its generalization
performance.

###  3.3 Featurization

Audio feature extraction is crucial in convolutional neural networks, as it
directly influences the model’s performance. However, combining multiple
feature extraction methods into one model led to complex models and requires a
substantial amount of data and expensive training costs. Therefore, it is
necessary to balance the addition of feature extraction methods while
retaining key acoustic information to ensure that the model can handle a
variety of datasets and provide general and accurate blind room parameter
estimation performance.

Prior works in [16, 15, 39] emphasize the importance of low-frequency
information for room acoustic parameter estimation. Consequently, feature
representation is restricted to the relatively low-frequency range ($<$2000
kHz). The Gammatone ERB filterbank is used to generate time-frequency
representations, comprising 20 frequency bands covering the frequency range
from 50 Hz to 2000 Hz. The audio is computed using a 64-sample Hann window
with a 32-sample hop size, resulting in a $20\times 1997$ complex Gammatone
spectrogram.

Furthermore, the phase information extracted from the audio is also retained
following the work in [21]. Phase angles are computed for each time-frequency
bin to generate phase features. These features are then truncated to include
only the frequency bands associated with frequencies below 500 Hz (i.e.,
$5\times 1997$) since lower-frequency components generally carry more
information related to room volume. Additionally, the first-order derivatives
of the phase coefficients along the frequency axis are calculated (i.e.,
$5\times 1997$). This feature configuration aligns with the “ +_phase_ ” model
described in [21], which has been proven to outperform methods based solely on
amplitude spectrogram features.

By combining spectral features, phase features, and first-order derivatives of
phase coefficients, a two-dimensional feature block is obtained. The dimension
of the feature block is $30\times 1997$, where 30 represents the feature
dimension ($F$), and 1997 represents the time dimension ($T$).

##  4 Model architecture

In this section, different architectures for audio data processing models are
described for blind room parameter estimation tasks. These models include a
CNN-based model, a CRNN-based model, and proposed attention-based systems.

Firstly, the CNN-based model utilizes multiple convolution and pooling layers
to capture features of the audio data through convolution operations, followed
by reducing the parameter count using pooling operations. Secondly, the CRNN-
based model combines CNN and LSTM networks, designed to handle time series
data better, capturing both time and frequency features of the audio data.
Finally, the proposed model employs a completely different approach, relying
solely on attention mechanisms without using convolution. This model has a
unique structure, breaking inputs into patches, processing data through
embedding layers and positional encoding layers, ultimately extracting
features and producing results using Transformers. Additionally, the study
also employs transfer learning by utilizing pretrained image models to process
audio data, improving performance and efficiency.

###  4.1 Convolutional neural network

In this section, a model based on a CNN following the “+_phase_ ” model in
[21] is introduced, for processing two-dimensional feature blocks extracted
from audio data. The model comprises six convolutional layers with
corresponding average pooling layers, and each convolutional layer is followed
by a Rectified Linear Unit (ReLU) activation function. To prevent overfitting,
dropout layers, which discard 50% of the connections, are introduced within
the network structure. Taking the estimation of room volume parameters as an
example, the final output layer is a fully connected layer, mapping the output
dimension to downstream tasks. In particular, the structure of its last layer
is dynamically adjusted according to the requirements of the blind room
parameter estimation task to meet the performance needs of different tasks.
Its system architecture is illustrated in Fig. 4.

Figure 4: The system architecture of the CNN-based model.

###  4.2 Convolutioanl recurrent neural network

CRNN is designed to capture both temporal and frequency features in audio data
while also having a memory to handle time series data. CRNN efficiently
extracts features from data and models sequences, better accommodating
variable-length inputs, making it highly suitable for practical blind room
parameter estimation problems.

A CRNN-based model is introduced in this section as it combines the
parameteric efficiency of CNNs with the capability of sequential modelling
from gated RNNs. The system architecture of CRNN is illustrated in Fig. 5.

Figure 5: The system architecture of the CRNN-based model.

Overall, the model consists of six convolutional layers with ReLU activation,
followed by an LSTM layer, a max-pooling layer, a dropout layer, and a time-
distributed fully connected layer.

Convolutional blocks gradually reduce the size and dimensions of the feature
maps, allowing more sequences to enter the network. Simultaneously, the max-
pooling layer is applied to extract useful features from the raw audio data.

The LSTM layer, which serves as a key component of the CRNN, is used for
processing time series data, capturing the temporal relationships and sequence
information in the input data.

Subsequently, the model includes a dropout layer before the dense layer. The
data is then flattened and passed to the fully connected layer, whose output
size can be adjusted as needed.

Finally, the model outputs the estimated room parameters. In this example,
blind indoor volume estimation is used as the task, which is a regression task
with an output size of 1. The structure can be adapted to specific application
scenarios and datasets.

###  4.3 In-Depth: Convolution-Free Audio Spectrogram Transformer

####  4.3.1 Audio Spectrogram Transformer

In this section, we introduce a model based purely on attention mechanisms
without convolution for blind room parameter estimation. The design of this
model is inspired by the workings of the Audio Spectrogram Transformer
described in [28], which has shown remarkable performance in end-to-end audio
classification tasks. However, it is noted that this attention-based approach
has not been extensively explored in other domains.

The primary goal of this section is to apply the proposed model that is purly
attention-based to the blind room parameter estimation problem and compare its
performance with traditional CNN and CRNN models.

In this work, two-dimensional feature block with dimensions of $30\times 1997$
as input for the proposed model is used. To better capture local information
in the audio, the two-dimensional feature block is divided into $P$ patches,
each sized $16\times 16$. The goal of patch split is to ensure a better
capture of local features within the audio signal. Additionally, to maintain
consistency in both feature and time dimensions, each patch has a 6-feature
dimension and 6-time dimension overlap with its surrounding blocks. As a
result, the number of patches $P$ is determined to be 398, shown as:

$\displaystyle{P=\bigl{\lceil}\displaystyle\frac{F-16}{10}\bigr{\rceil}\bigl{\lceil}\displaystyle\frac{T-16}{10}\bigr{\rceil}},$
(3)

where $F$ represents the feature dimension, and $T$ represents the time
dimension.

To further process these patches, we introduced a linear projection layer.
This layer’s role is to flatten each $16\times 16$ patch into a one-
dimensional patch embedding with a dimension of $768$, referred to as the
patch embedding layer. This embedding process helps reduce the data’s
dimensionality, making it more suitable for subsequent processing in the
model.

Since these patches are not arranged in chronological order, and traditional
Transformer architectures do not directly handle input sequences, we
introduced trainable positional embeddings of dimension 768 in each patch. By
introducing these trainable positional embeddings, the model is better able to
understand the spatial structure of the audio spectrogram and grasp the
positional information between patches.

Furthermore, the feature sequence is fed into the Transformer. Similar to
[28], each feature sequence begins with a [CLS] token. In this model, the
encoding and feature extraction of the input sequence only utilizes the
encoder part of the original Transformer architecture [40]. The advantage of
using the original Transformer structure is that it is a standard architecture
already available in PyTorch and TensorFlow, making it easy to reproduce.
Secondly, we plan to apply transfer learning to this task, and the standard
architecture facilitates transfer learning. Specifically, the embedding size
of the Transformer encoder we use is 768, with 12 layers and 12 heads, which
are the same as those in [41, 42].

We adjusted the output of the encoder based on the type of room parameters
being estimated. Taking room volume estimation as an example, the input
consists of a sequence formed by a feature block with the dimensions of
$30\times 1997$, and the output is a single label used for volume prediction.
The entire output of the Transformer serves as the feature representation for
the two-dimensional audio feature block, which is subsequently mapped to
labels for volume estimation using a linear layer with a Sigmoid activation
function. The system architecture of the proposed model is depicted in Fig. 6.

Figure 6: The system architecture of the proposed model.

In summary, traditional convolutional neural networks typically have multiple
layers, small kernels, and stride sizes. In contrast to the proposed model,
which includes patch embeddings (viewed as a single convolutional layer with
large kernels and strides) and projection layers within Transformer blocks
(viewed as $1\times 1$ convolution). The proposed model used in this study can
be referred to as a convolution-free model, distinguishing it from CNN and
CRNN [41, 42].

####  4.3.2  ImageNet Pretraining

Many researchers have noted that Transformers lack some of the inductive
biases inherent to CNNs, such as translation equivariance and locality [43].
Consequently, they exhibit poorer generalization capabilities under conditions
of limited training data, compared to simpler models like CNNs and CRNNs [41].

To achieve accurate blind room parameter estimation, a substantial amount of
publicly available data with correctly labeled room parameters is required to
train the network. Therefore, two approaches are adopted:

1) As introduced in Section 2.2, the use of an image-source model to
synthesize RIR datasets.

2) Transfer learning.

Transfer learning has been widely explored in previous research, particularly
in transferring from visual tasks to audio tasks. This transfer often focuses
on using CNN-based models [25, 44, 45, 46], where ImageNet-pretrained CNN
weights are used as initial weights for audio classification tasks. However,
the computational cost of training state-of-the-art visual models could be
relatively high. Fortunately, some common architectures like ResNet [46] and
EfficientNet [47] offer readily available ImageNet-pretrained models for
TensorFlow and PyTorch, making transfer learning more convenient.

For image classification tasks, research indicates that Transformer models
start to outperform traditional models in performance when the dataset size
exceeds 14 million. However, audio datasets for blind room parameter
estimation tasks typically cannot provide such massive amounts of data, posing
a challenge. Therefore, we have decided to explore cross-modality transfer
learning for the task of audio spectrogram processing, leveraging the similar
format between image and audio data.

In this study, we use a pretrained off-the-shelf Vision Transformer (ViT)
model from ImageNet [42] to simplify the transfer learning process. Afterward,
we make appropriate adjustments to adapt it to the blind room parameter
estimation task. Although both ViT and the proposed model employ the standard
Transformer with the same patch and embedding sizes, their architectural
similarities require adjustments to the structure before migration to ensure
compatibility with the blind room parameter estimation task. Three adjustments
are implemented:

1) One challenge is that the proposed model’s input is a single-channel
feature block, whereas ViT’s input is a three-channel image. To overcome this
issue, we adopted an approach to calculate the average weights corresponding
to each of the three input channels of the ViT patch embedding layer. We then
used these averaged weights for the patch embedding layer. This essentially
extends the single-channel spectrogram to a three-channel image with the same
content but higher computational efficiency. This approach helps us better
adapt to the differences in model input, thus improving the efficiency and
performance of the research.

2) Another issue encountered in this study is that the input dimensions of ViT
are fixed, whereas in practical tasks, the model needs to adapt to variable-
length audio inputs. As the audio length changes, the dimensions of the
feature block also change. While the Transformer naturally supports variable
input lengths and can be directly transferred from ViT to the proposed model,
special handling of positional encodings was required. This is because the ViT
model learns to encode spatial information during ImageNet training.
Therefore, we used the ‘Cut and bi-linear interpolate’ method [25] to adjust
the input size and manage positional encodings. This way, even with different
input shapes, we can pass on the two-dimensional spatial knowledge obtained
from the pretrained ViT to the proposed method, allowing the model to adapt to
audio inputs of varying lengths. This method helps us better handle data under
different input conditions.

3) To adapt to different sub-tasks in blind room parameter estimation, we take
the example of blind room volume estimation. We reinitialize the final
classification layer of ViT to output corresponding volume labels in the
proposed model.

These adjustments are crucial to ensuring that the pretrained ViT model can be
effectively used for the specific task of blind room parameter estimation and
achieve improved efficiency and performance.

##  5 Experiment

In this section, an in-depth exploration of blind room parameter estimation
tasks is conducted, utilizing two different datasets (_Dataset I_ and _Dataset
II_) to assess the performance of various models. We employed log-scaled and
normalized data to better handle the magnitude differences between room
parameters and utilized multiple evaluation metrics to comprehensively assess
the accuracy and robustness of the models.

We employed various model architectures (CNN-based model, CRNN-based model,
and the purely attention-based method in Section 4.3) and conducted a detailed
comparison of their performance on different tasks and datasets.

Overall, through these experiments, we examined the performance of different
models in various blind room parameter estimation tasks and assessed their
adaptability in handling variable-length audio inputs.

###  5.1 Datasets

In the task of blind room parameter estimation, _Dataset I_ and _Dataset II_
mentioned in sections 3.1 and 3.2 were utilized. In the preprocessing phase,
room volume labels (in m3 units, in logarithmic scale) were exclusively read,
and four models, CNN-based model, CRNN-based model, the Proposed Method, and
the “proposed method w/ pretrain” model, were individually evaluated for their
performance on _Dataset I_ and _Dataset II_. For the blind room parameter
estimation with variable-length audio input, _Dataset II_ was employed.
Similarly, in the preprocessing phase, only room volume labels (in m3 units,
in log-scaled) were considered. However, a modification was made to the test
set of _Dataset II_. Specifically, samples were extracted from 1 to 4 seconds
with a step size of 0.5 seconds, and zero padding was applied to different
lengths of audio samples to match the original length. This was done to assess
the performance of different models in handling blind room parameter
estimation under audio inputs with different length.

Finally, in the task of joint estimation of room parameters, _Dataset II_ was
used. In the preprocessing phase, the model simultaneously reads room RT60 (in
seconds) labels and room volume labels (in m3 units). In order to overcome the
significant scale differences between these two parameters, we adopted an
approach where we mapped the values of RT60 to volume values and applied a
logarithmic scaling to them. It is worth emphasizing that this data processing
method is reversible, allowing us to revert all parameters to standard units
at any time. The advantage of mapping the parameter relationship rather than
standard normalization is that it eliminates the need for frequent adjustment
of hyperparameters when dealing with different blind room parameter estimation
tasks, as it effectively addresses the differences in units and magnitudes
among the parameters. This is done to evaluate the performance of different
models in joint room parameters estimation.

###  5.2 Evaluation Metrics and Loss Function

As shown in Fig. 2, due to large span of room volume and RT60 ranges, the
estimation error could be related to its order of magnitude. Therefore, a
log-10 estimation is more suitable than a linear estimation. This way, larger
acoustic spaces in training are not disproportionately affected due to the
relatively high contribution of error estimation. Using a logarithmic
estimation better handles estimation errors of different orders of magnitude.

Four evaluation metrics using a base-10 logarithm are considered. They are as
follows: 1) Mean Squared Error (MSE): MSE is the average of the squared
differences between estimated room parameters and ground truth room
parameters. It is used to measure the degree of dispersion between estimated
values and ground truth values. The smaller the average of squared
differences, the closer the estimated values are to the ground truth values.
2) Mean Absolute Error (MAE): MAE represents the average of the absolute
differences between estimated values and ground truth values. It provides the
average deviation between estimated values and ground truth values and is
commonly used to measure the accuracy of estimated values. 3) Pearson
Correlation Coefficient ($\rho$): The Pearson correlation coefficient is used
to measure the strength and direction of the linear relationship between two
variables. It is used to describe the relationship between estimated room
parameters and ground truth room parameters, with values ranging from -1 to 1.
Negative values indicate a negative correlation, positive values indicate a
positive correlation, and 0 indicates no correlation. 4) MeanMult MM: MM is
the mean absolute logarithm of the ratio between the estimated room volume and
the ground truth room volume. This metric provides an overview of the mean
error in the ratio between estimated room parameters and ground truth room
parameters. Taking the logarithm of the ratio helps reduce the impact of data
points with significant differences. For example, for the estimated volume
parameter, given the estimated volume $\hat{V}_{n}$ and the ground truth
volume $V_{n}$:

$\textit{MM}=e^{\frac{1}{N}\sum_{n=1}^{N}|\ln\left(\frac{\hat{V}_{n}}{V_{n}}\right)|},$
(4)

where “$n$” represents the sample index, and “$N$” represents the total number
of samples.

During model training stages, MSE was used as the loss function to minimize
the error between estimated room parameters and ground truth room parameters.
In the “Estimation of room parameter” and “Room parameter estimation under
variable-length audio input” tasks, the loss function $\mathrm{L_{1}}$ formula
was as follows:

$\text{L}_{1}=\frac{1}{B}\sum_{n=1}^{B}(\hat{V}_{n}-V_{n})^{2},$ (5)

where “$n$” represents the sample index, and “$B$” represents the batch size
during training.

In contrast, for the task of “Joint estimation of room parameters,” which
involves the simultaneous estimation of RT60 and volume parameters. To avoid
differences in units and orders of magnitude between different parameters, as
well as the impact of parameter scaling methods, the normalized MSE was used
instead of the MSE in Eq. 5. The loss function $\mathrm{L_{2}}$ was formulated
as follows:

$\text{L}_{2}={\lambda_{1}}*\frac{\sum_{n=1}^{B}(\hat{U}_{n}-U_{n})^{2}}{B\sum_{n=1}^{B}(U_{n})^{2}},+{\lambda_{2}}*\frac{\sum_{n=1}^{B}(\hat{V}_{n}-V_{n})^{2}}{B\sum_{n=1}^{B}(V_{n})^{2}},$
(6)

where $\hat{U}_{n}$ and $U_{n}$ represent the estimated and the ground truth
RT60, respectively. $\lambda_{1}$ and $\lambda_{2}$ are weights used to
control the balance between the RT60 and volume normalized MSE loss functions.
These weights are employed to adjust the relative importance of these two
functions during model training. Based on empirical evidences and experimental
results, $\lambda_{1}$ is set to 1, and $\lambda_{2}$ is set to 2. This weight
configuration can be adjusted according to the specific task and model
performance to better meet the training requirements.

###  5.3 Experiment configurations

Different MSE loss functions were chosen based on the task’s requirements.
Each model utilized the Adam optimizer from PyTorch. During the training
process, L2 regularization was applied to prevent over-fitting. Furthermore,
grid search was employed to tune hyperparameters, including initial learning
rate, batch size, and regularization strength.

For the “Estimation of room parameter” task, to facilitate comparative
testing, we switched between _Dataset I_ and _Dataset II_ as well as
determined whether to use a pretrained model from ImageNet. To ensure
consistency in model configurations, hyperparameters were kept constant. CNN-
based and CRNN-based models were trained for 1000 epochs with an initial
learning rate of 5e-4.

The proposed attention-based method and the “proposed method w/ pretrain”
model were trained for 150 epochs with an initial learning rate of 5e-6. For
the “Joint estimation of room parameters” task, CNN-based and CRNN-based
models were trained for 2000 epochs with an initial learning rate of 2e-4. The
proposed method and the “proposed method w/ pretrain” model were trained for
300 epochs with an initial learning rate of 2e-6.

To ensure fairness, all models were trained on devices equipped with an Intel
Core i9 processor and an NVIDIA GeForce 4090 GPU.

##  6 Results and Discussion

###  6.1 Estimation of room volume parameter

To investigate whether comparable performance similar to that of CNN and CRNN
can be achieved by using a pure attention mechanism, we extracted audio data
from _Dataset I_ for the purpose of estimating room volume parameter. We
transformed audio data into a feature block, as described in Section 3.3.
Subsequently, we separately input these feature blocks into the CNN-based
model, CRNN-based model, and the proposed method (the base version without
ImageNet pretraining) for training. We then compared the predicted volume
labels to the ground truth values. The results of these three models are
presented in Table 2. Note that in this section we mainly focus on estimation
of room volumes as this task has been shown to be more challenging than RT60
estimation in the literature [15, 16, 21].

Table 2: The comparison between the CNN-based model [21], the CRNN-based model, and the base version of the proposed method. Method | MSE | MAE | $\rho$ | MM
---|---|---|---|---
CNN [21] | 0.3863 | 0.4837 | 0.6984 | 3.0532
CRNN | 0.3572 | 0.4265 | 0.7262 | 2.6701
Proposed method | 0.2650 | 0.3432 | 0.8077 | 2.2039

Based on experimental results above, we can see that the proposed method using
the pure attention mechanism significantly outperforms both CNN and CRNN-based
approaches, even with a lower-layer network configuration and relatively fewer
training epochs. This suggests that the proposed method can accurately capture
the acoustic characteristics in the audio data, thereby improving the accuracy
and stability of room volume estimation.

Meanwhile, the four evaluation metrics show that the CRNN-based model performs
better than the CNN-based model. This can be attributed to the advantages of
CRNN, which combines CNN with LSTM. CRNN can better handle the time series
audio data while capturing local features, which is crucial for blind room
parameter estimation tasks.

Table 3:  Performance comparison of different models with and without the application of SpecAugment. Method | _Dataset I_ | _Dataset II_
---|---|---
MSE | MAE | $\rho$ |  MM | MSE | MAE | $\rho$ |  MM
CNN [21] | 0.3863 | 0.4837 | 0.6984 |  3.0532 | 0.3154 | 0.4136 | 0.7678 |  2.5921
CRNN | 0.3572 | 0.4265 | 0.7262 |  2.6701 | 0.2818 | 0.3684 | 0.7898 |  2.3471
Proposed | 0.2650 | 0.3432 | 0.8077 |  2.2039 | 0.1981 | 0.2884 | 0.8580 |  1.9427
method
Proposed | 0.2157 | 0.3111 | 0.8529 |  2.0470 | 0.1541 | 0.2423 | 0.8929 |  1.7470
method
w/pretrain

To further investigate the impact of ImageNet pretraining on the proposed
method’s performance, the “proposed method w/ pretrain” model was trained on
_Dataset I_. Simultaneously, to examine the effect of the SpecAugment data
augmentation method on the performance of existing models, we retrained the
existing four models on _Dataset II_. The results of the above experiments are
shown in Table 3.

Based on the training results of different models on _Dataset I_ , we can
observe a significant improvement in the performance of the proposed method in
the “Estimation of room parameter” task with the use of the ImageNet
pretraining method. Furthermore, when the four models were retrained on
_Dataset II_ , the application of the SpecAugment method elevated the models’
performance to a new level. In particular, this method demonstrates a
significant improvement in the performance of the “proposed method w/
pretrain” model. It confirms the effectiveness of SpecAugment in mitigating
overfitting and enhancing model generalizability.

Meanwhile, in order to provide a more illustrative example, we rescaled the
experimental results to a linear scale. In this experiment, the test set room
volume ranges from 12 to 21,000 $m^{3}$. We compared the performance of the
best-performing models, namely the CNN-based model, CRNN-based model, and the
“proposed method w/ pretrain” model. They were trained on _Dataset II_ , and
their model’s median as well as mean absolute error are shown in Table 4.

Table 4: Comparison of median and mean absolute error for volume parameters among the best-performing models. Method | Median | Mean absolute error
---|---|---
($m^{3}$) |  $(m^{3}$)
CNN [21] | 353 | 1919
CRNN | 257 | 1644
Proposed method | 155 | 1219
w/pretrain

From the table, the “proposed method w/ pretrain” model exhibits the best
performance in terms of both median and mean absolute error, having the lowest
error values.

(a) CNN-based model

(b) CRNN-based model

(c) The “proposed method w/ pretrain” model

Figure 7: Confusion Matrices for the best-performing models trained on
_Dataset II_ in the “Estimation of room volume parameter” task

The Fig. 7 displays the confusion matrices for these three best-performing
models in the “Estimation of room volume parameter” task, with the x-axis and
y-axis representing the log-10 exponent of volume size. From the
visualization, it is evident that the “proposed method w/ pretrain” model
exhibits excellent performance across the entire test range. Its distribution
consistently closely surrounds the ground truth, clearly outperforming the
CNN-based and CRNN-based models.

Results in this section indicate that the proposed purely attention-based
model is capable of capturing relevant features and representations in the
context of room volume regression efficiency. More importantly, it
demonstrates remarkable generalization capabilities, effectively applying the
patterns learned from the training data to real-world rooms, even for rooms
the model has not encountered before, resulting in accurate volume estimates.
This outcome provides a strong theoretical foundation for our approach and
underscores its potential in more blind estimation practical problems, which
will be addressed in the following section.

###  6.2 Room parameter estimation under variable-length audio input

In this section, model performances under variable-length audio inputs are
evaluated for the “Room parameter estimation” task. The selected models were
tested with different lengths of audio inputs, and their performances were
assessed using four objective evaluation metrics as shown in Fig. 8. It is
evident from the figure that the accuracy of the models in predicting room
volume parameter significantly depends on the length of the input audio. As
the input audio length shortens, the estimation performance of all models
inevitably experiences degradation.

By observing the curves of MSE and MAE metrics in Fig. 8, it can be noted that
the CRNN-based model, the proposed model, and the “proposed method w/
pretrain” model exhibit smaller decay slopes. This suggests that, compared to
CNN-based models, they can better handle time sequences of variable length.
The smaller decay slopes of the evaluation metrics can be considered an
indication that the models better maintain performance stability, even when
the input length decreases, maintaining relatively good performance.

Results in Fig. 8 also indicate that the “proposed method w/ pretrain” model
performs the best at the same input length. For the shortest input sample,
i.e., when the input length is 1 second, the MSE for the “proposed method w/
pretrain” model is 0.6458. In comparison, to achieve the same performance
level, the CNN-based model, CRNN-based model, and the proposed model would
require input audio lengths of approximately 2.8 seconds, 2.0 seconds, and 1.2
seconds, respectively. This advantage facilitates the proposed attention-based
models to outperform both CNN and CRNN systems with significantly less
temporal context, which can be a valuable merit when dealing with speech-based
blind estimation problems in practice.

Figure 8: Performance comparison of different models under the “Room parameter
estimation under variable-length audio input” task.

###  6.3 Joint estimation of room parameters

This section aims to address the “Joint estimation of room parameters” task,
which involves training a single model to simultaneously estimate multiple
room parameters. Specifically, due to the shared acoustic characteristics of
room volume and RT60 parameters, it is possible to estimate them concurrently
by extracting reverberation-related information. Considering difficulties in
collecting groundtruth of real data for other room parameters, such as total
surface area and average surface absorption coefficient in real-world room
datasets, this experiment focuses on the joint estimation of room volume and
RT60.

In this task, we selected three models, namely the CNN-based model, CRNN-based
model, as well as the “proposed method w/ pretrain” model, and trained them on
_Dataset II_. Their network architectures were fundamentally similar to those
used for the “Estimation of room volume parameter” task, with minor
modifications. In the “Joint estimation of room parameters” task, the three
models are required to output two parameters, i.e. room volume and RT60,
instead of a single parameter. Consequently, the final output layers of the
models were modified to include two fully connected layers for estimating
different room parameters. During the training process, hyperparameters were
fine-tuned (as described in Section 5.3), and the loss function was adjusted
(as shown in Eq.6).

It is worth noting that, in order to mitigate issues related to different
units and scales among parameters, as well as the impact of parameter scaling
during normalization, we chose to use only the $\rho$ as the evaluation
metric. This helps ensure consistency among the estimated parameters. The
corresponding results are presented in the Table 5.

Table 5: Pearson correlation coefficients of best-performing models in “Joint estimation of room parameters” task. Method | CNN [21] | CRNN |  Proposed method w/ pretrain
---|---|---|---
vol | RT60 | vol | RT60 | vol |  RT60
$\rho$ | 0.6187 | 0.9133 | 0.6584 | 0.9488 | 0.8287 |  0.9681

From these results, it can be clearly seen that the “proposed method w/
pretrain” model outperforms the other models, achieving the highest $\rho$ for
both room volume and RT60, indicating its effectiveness in jointly estimating
these room parameters.

In this experiment, the test set room volume ranges from 12 to 21,000 $m^{3}$
while the RT60 range from 0.41 to 19.68 seconds. We rescaled the experimental
results to a linear scale. The median, as well as mean absolute error for the
three models regarding volume and RT60, are displayed in Table 6.

Furthermore, we conducted a comparative study between the volume estimation in
the joint model and the estimation of volume results for the “Estimation of
room volume parameter” task by comparing results in Table 3 and Table 5, as
well as Table 4 and Table 6. Despite the fact that the joint estimation models
aim to simultaneously handle multiple parameters, it is clear that their
volume estimation results, while experiencing some degree of attenuation, are
overall very similar to the results obtained from estimating only a single
parameter. This suggests that the performance of the joint model is in par
with that of models designed for estimating a single parameter.

Table 6: Comparison of median and mean absolute error for volume as well as RT60 parameters among the best-performing models. Method | Median | Mean absolute error
---|---|---
vol | RT60 | vol | RT60
($m^{3}$) | (seconds) | ($m^{3}$) | (seconds)
CNN [21] | 728 | 0.64 | 2481 | 1.32
CRNN | 329 | 0.39 | 2265 | 0.71
Proposed method | 294 | 0.31 | 2208 | 0.61
w/pretrain

Fig. 9 shows confusion matrices for volume and RT60 parameter estimation in
the “Joint estimation of room parameters” task, highlighting the best-
performing models. The x-axis and y-axis represent the log-10 exponent of room
parameters (volume and RT60). From the visualization results, it can be
observed that estimation performances of the CNN-based model, CRNN-based
model, and the “proposed method w/ pretrain” model gradually improves, and
their fitting capabilities increase progressively.

(a) CNN-based model(volume)

(b) CRNN-based model(volume)

(c) The “proposed method w/ pretrain” model(volume)

(d) CNN-based model(RT60)

(e) CRNN-based mode(RT60)

(f) The “proposed method w/ pretrain” model(RT60)

Figure 9: Confusion Matrices for the best-performing models trained on
_Dataset II_ in the “Joint estimation of room parameters” task

The comprehensive analysis of experimental results in this study demonstrates
the effectiveness of the joint estimation model for the blind room parameter
estimation task. This method involves utilizing a single model to
simultaneously estimate both room volume and RT60 parameters, providing a more
holistic understanding of acoustic environmental characteristics.
Particularly, the “proposed method w/ pretrain” model achieves the highest
$\rho$ for both room volume and RT60 parameters. This highlights the model’s
capability of capturing the intricate characteristics of acoustic environments
through the joint estimation of room parameters.

##  7 Conclusion and future work

In this study, we aim to explore the feasibility of using attention-based
models to address audio processing tasks, specifically including the
“Estimation of room volume parameter,” “Room parameter estimation under
variable-length audio input,” and “Joint estimation of room parameters” tasks.
We employ different training strategies to evaluate performances of a CNN-
based model, a CRNN-based model, the proposed attention-based model, and the
“proposed method w/ pretrain” model.

Experimental results based on unseen real-world rooms and realistic noise
scenario indicate that our proposed method shows significant superiority in
terms of accurately capturing the acoustic characteristics of audio data. This
demonstrates that neural networks based on pure attention mechanisms can
effectively handle regression problems related to audio and exhibit potential
advantages in handling joint estimation tasks and variable-length inputs.

Future research directions will focus on optimizing and enhancing the
performance of attention-based audio processing models in real-world
applications. We plan to further improve the model structure, including
considering more efficient variants, to better capture the complex features of
audio data. Additionally, we will strive to collect more comprehensive and
diverse room data to enhance the model’s generalization capabilities. These
efforts will contribute to advancing the application of attention-based audio
processing models in real-world scenarios.

## References

  *   * Zhang et al. [2018] Zhang, Z., Geiger, J., Pohjalainen, J., Mousa, A.E.-D., Jin, W., Schuller, B.: Deep learning for environmentally robust speech recognition: An overview of recent developments. ACM Trans. Intell. Syst. Technol. 9(5) (2018)
  * Wu et al. [2017] Wu, B., Li, K., Ge, F., Huang, Z., Yang, M., Siniscalchi, S.M., Lee, C.-H.: An end-to-end deep learning approach to simultaneous speech dereverberation and acoustic modeling for robust speech recognition. IEEE Journal of Selected Topics in Signal Processing 11(8), 1289–1300 (2017) https://doi.org/10.1109/JSTSP.2017.2756439
  * Mohammadiha and Doclo [2016] Mohammadiha, N., Doclo, S.: Speech dereverberation using non-negative convolutive transfer function and spectro-temporal modeling. IEEE/ACM Transactions on Audio, Speech, and Language Processing 24(2), 276–289 (2016)
  * Cecchi et al. [2018] Cecchi, S., Carini, A., Spors, S.: Room response equalization – a review. Applied Sciences 8(1) (2018)
  * Jin and Kleijn [2015] Jin, W., Kleijn, W.B.: Theory and design of multizone soundfield reproduction using sparse methods. IEEE/ACM Transactions on Audio, Speech, and Language Processing 23(12), 2343–2355 (2015)
  * Jin [2016] Jin, W.: Adaptive reverberation cancelation for multizone soundfield reproduction using sparse methods. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 509–513 (2016)
  * Neidhardt et al. [2022] Neidhardt, A., Schneiderwind, C., Klein, F.: Perceptual matching of room acoustics for auditory augmented reality in small rooms - literature review and theoretical framework. Trends in Hearing 26, 23312165221092919 (2022)
  * Jot and Lee [2016] Jot, J.-M., Lee, K.S.: Augmented reality headphone environment rendering. In: Audio Engineering Society Conference: 2016 AES International Conference on Audio for Virtual and Augmented Reality (2016). Audio Engineering Society
  * Kuttruff [2016] Kuttruff, H.: Room acoustics. In: CRC Press (2016)
  * Eaton et al. [2016] Eaton, J., Gaubitch, N.D., Moore, A.H., Naylor, P.A.: Estimation of room acoustic parameters: The ACE challenge. IEEE/ACM Transactions on Audio, Speech, and Language Processing 24(10), 1681–1693 (2016) https://doi.org/10.1109/TASLP.2016.2577502
  * de M. Prego et al. [2015] M. Prego, T., Lima, A.A., Zambrano-López, R., Netto, S.L.: Blind estimators for reverberation time and direct-to-reverberant energy ratio using subband speech decomposition. In: 2015 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), pp. 1–5 (2015). https://doi.org/10.1109/WASPAA.2015.7336954
  * Loellmann et al. [2015] Loellmann, H., Brendel, A., Vary, P., Kellermann, W.: Single-channel maximum-likelihood T60 estimation exploiting subband information. arXiv preprint arXiv:1511.04063 (2015)
  * Moore et al. [2014] Moore, A., Brookes, M., Naylor, P.: Room identification using roomprints. Journal of the Audio Engineering Society (2014)
  * Peters et al. [2012] Peters, N., Lei, H., Friedland, G.: Name that room: Room identification using acoustic features in a recording. In: Proceedings of the 20th ACM International Conference on Multimedia, pp. 841–844 (2012)
  * Gamper and Tashev [2018] Gamper, H., Tashev, I.J.: Blind reverberation time estimation using a convolutional neural network. In: 2018 16th International Workshop on Acoustic Signal Enhancement (IWAENC), pp. 136–140 (2018)
  * Genovese et al. [2019] Genovese, A.F., Gamper, H., Pulkki, V., Raghuvanshi, N., Tashev, I.J.: Blind room volume estimation from single-channel noisy speech. In: Proc. ICASSP, pp. 231–235 (2019). IEEE
  * Bryan [2020] Bryan, N.J.: Impulse response data augmentation and deep neural networks for blind room acoustic parameter estimation. In: Proc. ICASSP, pp. 1–5 (2020). IEEE
  * Götz et al. [2022] Götz, P., Tuna, C., Walther, A., Habets, E.A.P.: Blind reverberation time estimation in dynamic acoustic conditions. In: Proc. ICASSP, pp. 581–585 (2022). IEEE
  * Saini and Peissig [2023] Saini, S., Peissig, J.: Blind room acoustic parameters estimation using mobile audio transformer. In: 2023 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), pp. 1–5 (2023)
  * Srivastava et al. [2021] Srivastava, P., Deleforge, A., Vincent, E.: Blind room parameter estimation using multiple multichannel speech recordings. In: 2021 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), pp. 226–230 (2021)
  * Christopher et al. [2023] Christopher, I., Mehrabi, A., Jin, W.: Blind acoustic room parameter estimation using phase features. In: Proc. ICASSP, Rhodes Island, Greece, pp. 1–5 (2023). IEEE
  * Callens and Cernak [2020] Callens, P., Cernak, M.: Joint blind room acoustic characterization from speech and music signals using convolutional recurrent neural networks. arXiv preprint arXiv:2010.11167 (2020)
  * Deng et al. [2020] Deng, S., Mack, W., Habets, E.: Online blind reverberation time estimation using crnns. In: Interspeech (2020). https://api.semanticscholar.org/CorpusID:226202121
  * Kong et al. [2020] Kong, Q., Cao, Y., Iqbal, T., Wang, Y., Wang, W., Plumbley, M.D.: Panns: Large-scale pretrained audio neural networks for audio pattern recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing 28, 2880–2894 (2020)
  * Gong et al. [2021] Gong, Y., Chung, Y.-A., Glass, J.: Psla: Improving audio tagging with pretraining, sampling, labeling, and aggregation. IEEE/ACM Transactions on Audio, Speech, and Language Processing 29, 3292–3306 (2021)
  * Li et al. [2018] Li, P., Song, Y., McLoughlin, I.V., Guo, W., Dai, L.-R.: An attention pooling based representation learning method for speech emotion recognition (2018)
  * Rybakov et al. [2020] Rybakov, O., Kononenko, N., Subrahmanya, N., Visontai, M., Laurenzo, S.: Streaming keyword spotting on mobile devices. arXiv preprint arXiv:2005.06720 (2020)
  * Gong et al. [2021] Gong, Y., Chung, Y.-A., Glass, J.: AST: Audio Spectrogram Transformer. In: Proc. Interspeech 2021, pp. 571–575 (2021)
  * Wang et al. [2023] Wang, C., Jia, M., Li, M., Bao, C., Jin, W.: Attention is all you need for blind room volume estimation. arXiv preprint arXiv:2309.13504 (2023)
  * Jeub et al. [2009] Jeub, M., Schafer, M., Vary, P.: A binauralroom impulse response database for the evaluation of dereverberation algorithms. Digital Signal Processing, 1–5 (2009)
  * Szöke et al. [2019] Szöke, I., Skácel, M., Mošner, L., Paliesek, J., Černocký, J.: A binauralroom impulse response database for the evaluation of dereverberation algorithms. In: IEEE Journal of Selected Topics in Signal Processing, vol. 13, pp. 864–876 (2019)
  * Stewart and Sandler [2010] Stewart, R., Sandler, M.: Database of omnidirectional and b-format room impulse responses. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 13(4), 165–168 (2010)
  * Carlo et al. [2021] Carlo, D.D., Tandeitnik, P., Foy, C., Bertin, N., Deleforge, A., Gannot, S.: dechorate: a calibrated room impulse response dataset for echo-aware signal processing. EURASIP Journal on Audio, Speech, and Music Processing, Springer 2021(1), 1–15 (2021)
  * Murphy and Shelley [2010] Murphy, D., Shelley, S.: Openair: an interactive auralization web resource and database. Journal of the Audio Engineering Society (2010)
  * Schroeder [1965] Schroeder, M.R.: New method of measuring reverberation time. The Journal of the Acoustical Society of America 37(3), 409–412 (1965) https://doi.org/10.1121/1.1909343 https://doi.org/10.1121/1.1909343
  * Scheibler et al. [2017] Scheibler, R., Bezzam, E., Dokmanić, I.: Pyroomacoustics: A Python package for audio room simulations and array processing algorithms. arXiv e-prints, 1710–04196 (2017) https://doi.org/10.48550/arXiv.1710.04196 arXiv:1710.04196 [cs.SD]
  * Krishnamurthy and Hansen [2009] Krishnamurthy, N., Hansen, J.H.L.: Babble noise: Modeling, analysis, and applications. IEEE Transactions on Audio, Speech, and Language Processing 17(7), 1394–1407 (2009) https://doi.org/%****␣main.bbl␣Line␣550␣****10.1109/TASL.2009.2015084
  * Park et al. [2019] Park, D.S., W. Chan, Y.Z., Chiu, C.-C., Zoph, B., Cubuk, E.D., Le, Q.V.: Specaugment: A simple data augmentation method for automatic speech recognition. Interspeech, 2613–2617 (2019)
  * Srivastava et al. [2022] Srivastava, P., Deleforge, A., Vincent, E.: Realistic sources, receivers and walls improve the generalisability of virtually-supervised blind acoustic parameter estimators. arXiv preprint arXiv:2207.09133 (2022)
  * Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., N.Gomez, A., Kaiser, L., Polosukhin, I.: Attention is all you need. In: NIPS (2017)
  * Dosovitskiy et al. [2020] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
  * Touvron et al. [2021] Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., Jégou, H.: Training data-efficient image transformers & distillation through attention. In: International Conference on Machine Learning, pp. 10347–10357 (2021). PMLR
  * Sun et al. [2021] Sun, H., Liu, X., Xu, K., Miao, J., Luo, Q.: Emergency vehicles audio detection and localization in autonomous driving. arXiv preprint arXiv:2109.14797 (2021)
  * Gwardys and Grzywczak [2014] Gwardys, G., Grzywczak, D.: Deep image features in music information retrieval. International Journal of Electronics and Telecommunications 60, 321–326 (2014)
  * Guzhov et al. [2021] Guzhov, A., Raue, F., Hees, J., Dengel, A.: Esrsuppesnet: Environmental sound classification based on visual domain models. In: 2020 25th International Conference on Pattern Recognition (ICPR), pp. 4933–4940 (2021). IEEE
  * He et al. [2019] He, K., Girshick, R., Dollár, P.: Rethinking imagenet pre-training. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4918–4927 (2019)
  * Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR