# Domain Agnostic Few-shot Learning for Speaker Verification

###### Abstract

Deep learning models for verification systems often fail to generalize to new
users and new environments, even though they learn highly discriminative
features. To address this problem, we propose a few-shot domain generalization
framework that learns to tackle distribution shift for new users and new
domains. Our framework consists of domain-specific and domain-aggregation
networks, which are the experts on specific and combined domains,
respectively. By using these networks, we generate episodes that mimic the
presence of both novel users and novel domains in the training phase to
eventually produce better generalization. To save memory, we reduce the number
of domain-specific networks by clustering similar domains together. Upon
extensive evaluation on artificially generated noise domains, we can
explicitly show generalization ability of our framework. In addition, we apply
our proposed methods to the existing competitive architecture on the standard
benchmark, which shows further performance improvements.

Index Terms: Few-shot Learning, Domain Generalization, Pseudo-label
Clustering, Fine-tuning

##  1 Introduction

Deep learning models often fail to generalize to new domains and new classes,
even though they produce highly discriminative representations. Domain
generalization [1, 2, 3, 4] and few-shot learning [5, 6, 7] have produced
attractive solutions to address problems about unseen domains and classes,
respectively. However, only few studies [8, 9] have focused on both problems,
simultaneously. In particular, identification systems always face both
problems in the test phase while most approaches only consider the same set of
categories and domains as they are trained upon. Consequently, these closed-
set models perform poorly on novel categories and novel domains. This paper
addresses this problem for speaker verification, which is the task of
accepting or rejecting the claimed identity of a speaker test utterance based
on few enrolled utterances.

Previous approaches [10, 11, 12, 13, 14, 15, 16, 17, 18] on speaker
verification train an embedding network using classification or metric
learning loss on a dataset consisting of multiple speakers (users) in various
domains. However, these methods do not generalize well to novel users and
novel domains not involved during training. There are several works [19, 20,
21] using few-shot learning approaches that generalize to new users. In
training, they exploit prototypical networks [6], and apply an episodic
learning scheme to mimic the test conditions of new users. For classification,
they use prototypes [19] or a class-centroid-based formulation [21] to compute
the distance functions over all the classes. These meta-learning based methods
consider generalization on novel users but do not consider novel domains. To
achieve both objectives, we propose a domain generalization component that
mimics the presence of both novel users and novel domains in the episodic
training phase.

In our framework, we consider two types of episode generations with domain-
specific and domain-aggregation prototypical networks. Firstly, we generate
the episodes consisting of the support and query set to mimic the cases of
enrolling a novel user with few labeled samples (support set) and correctly
classifying the test samples (query set). Here, the episodes are sampled from
a specific domain and all domains for the domain-specific and domain-
aggregation network, respectively. With the generated episodes and the
prototypical loss function [6, 22], both networks are trained to adapt well to
a new user. Secondly, we generate the episodes to mimic the cases of testing
the network in a novel domain. Here, we consider a domain mismatched way:
given a domain and its domain-specific network, the support and query sets are
sampled from the other domain which is unseen to the specific network. Then,
the support and query sets are fed into the domain-specific and domain-
aggregation networks, respectively. With this episode generation and the
prototypical loss function, we can train the domain-aggregation network with
better domain generalization ability.

For the speaker verification task, the domains can be various environments
such as background noises, recording devices, etc. However, modeling the
domain-specific network for each domain may require much memory and
computations. For the sake of efficiency, we consider a pseudo domain label
approach where similar domains are clustered together using style features
adopted in [23, 24].

We extensively evaluated our framework with two types of domain differences:
(a) artificially generated and (b) those intrinsically present in a dataset.
Firstly, using the VoxCeleb1 dataset [25], we artificially generated several
domains with noise augmentations and split them into training and testing set.
In the experiments, we observed that the proposed framework shows better
generalization ability for both new speakers and new domains in test dataset.
Also, the pseudo domain labels are beneficial for efficient training and
further performance boost. Secondly, our framework when applied on top of
[22], learns representations that can tackle intrinsic domain differences in
VoxCeleb2 [26]. The results also show performance improvements on the standard
evaluation benchmark: VoxCeleb1, SITW [27] and CNCeleb [28].

Figure 1:  Our framework consists of: (a) Domain clustering: we assign pseudo
domain labels to all samples using domain-discriminative features from the
domain-aggregation network $F_{agg}$ pre-trained by $L_{agg}$; (b) Training
domain-specific network: $F_{j}$ is trained on the corresponding source domain
after clustering and re-assignment of domains; (c) Domain-agnostic few-shot
learning: episodes are generated to mimic the few-shot test case for novel
users and domain-mismatched case for novel domains.

##  2 Proposed Framework

###  2.1 Task Description

In this paper, our task is speaker verification, which is a decision process
that accepts or rejects an input utterance $\bf x$ by comparing the utterance
with a trained reference speaker model ${\bf X}_{ref}$. Mathematically, the
decision process is represented as:

$\displaystyle f({\bf X}_{ref},{\bf
x})\underset{reject}{\overset{accept}{\gtrless}}\tau$ (1)

where $f(\cdot,\cdot)$ is a similarity metric score between ${\bf X}_{ref}$
and $\bf x$. If the score is greater than a pre-defined threshold $\tau$, $\bf
x$ is accepted as the utterance of the reference speaker; otherwise, $\bf x$
is presumed to be from an impostor. Our goal is to train a speaker
verification model that generalizes to novel users and domains.

###  2.2 Problem Setting and Notation

We have a source dataset $D=\\{D_{1},D_{2},…,D_{N}\\}$ consisting of multiple
speakers from $N$ domains, where $D_{j}$ indicates the $j^{th}$ domain. Each
domain contains utterances and their corresponding speaker labels such that
$D_{j}=\\{({\bf x}_{j}^{i},y_{j}^{i})\\}_{i=1}^{n_{j}}$, where $n_{j}$ is the
number of samples in the $j^{th}$ domain. We use the source data to train an
embedding network $F_{agg}$ such that it generalizes well to novel target
users in novel testing domains $D_{*}$ with different characteristics from the
source domains. Note that it is different from data augmentation techniques
that use noise and reverberation to enforce test domains to be included in the
training domains. We denote a domain-specific network and a domain-aggregation
network by $F_{j}$ and $F_{agg}$, respectively. $F_{j}$ specialized in
$j^{th}$ domain is learned so that $F_{agg}$ learned with all domains can
extract domain agnostic features even for novel domain data. Fig. 1 shows our
overall framework.

###  2.3 Domain Clustering

Our learning scheme requires $N$ domain-specific networks that consume lots of
memory in training, which we can reduce by modifying the original source
dataset $D$ with pseudo domain labels by clustering. We extract domain-
discriminative features and cluster them to assign the same pseudo domain
labels to the samples of similar domains. For domain-discriminative features,
we exploit the features proposed in style transfer algorithms for different
image domains [23, 29, 24] that are recently shown applicable to audio domains
[30, 31, 32]. In these methods, the mean and the standard deviation that are
calculated across the spatial dimensions represent the input style. We obtain
the domain-discriminative feature of an input ${\bf x}$ by stacking these
feature statistics as follows:

$\\{\mu(\phi_{agg,1}({\bf x})),\sigma(\phi_{agg,1}({\bf
x})),...,\mu(\phi_{agg,l}({\bf x})),\sigma(\phi_{agg,l}({\bf x}))\\}$

where $\phi_{agg,l}$ indicates the output of the $l^{th}$ layer in $F_{agg}$.
Before extracting features, we first train $F_{agg}$ with the source domain
dataset $D$. Then, with the extracted domain-discriminative features, we
perform k-means clustering [33] to obtain $M$ clusters and assign the pseudo
domain label to each sample. This is illustrated in Fig. 1(a). Accordingly, we
obtain a modified source dataset
$D=\\{{\widetilde{D}}_{1},{\widetilde{D}}_{2},...,{\widetilde{D}}_{M}\\}$ with
pseudo domain labels, and $M\leq N$. In contrast to [24], our objective of
assigning pseudo domain labels is to increase memory efficiency by clustering
ambiguous domains and reducing the number of domain-specific networks.

###  2.4 Episodic Training

In this step, we describe our two types of episode generations to mimic the
test cases of the novel user and novel domain. First, we split the dataset $D$
into the support set $D^{s}=\\{({\bf x}^{i},y^{i})\\}^{CK}_{i=1}$ and the
query set $D^{q}$ to mimic the few-shot test case for novel users, which is a
$C$-way $K$-shot problem. Here, way and shot stand for the number of speakers
and samples per speaker in each episode, respectively. We use the prototypical
network [6] to be trained in an episodic manner such that it computes speaker
prototypes with the support set and forces the query set to minimize the
distance from the corresponding prototypes. Note that our algorithm can be
adapted to any prototype-based loss functions, e.g., angular prototypical loss
[22].

As illustrated in Fig. 1(b), the domain-specific network $F_{j}$ is trained
using episodes generated from ${\widetilde{D}}_{j}$, and prototype of speaker
$k$, ${\bf c}_{k}$, is obtained by averaging over embedding vectors in
$j^{th}$ domain support set:

${\bf c}_{k}=\frac{1}{K}\sum_{\\{({\bf
x}_{j}^{i},y_{j}^{i})\\}\in{\widetilde{D}}_{j}^{s}}F_{j}({\bf
x}_{j}^{i})\mathds{1}(y_{j}^{i}=k)$ (2)

where $\mathds{1}(\cdot)$ is the indicator function returning $1$ for true
statements and $0$ otherwise. The episodic training loss $L_{sp}$ for the
domain-specific network is defined as follows:

$\displaystyle L_{sp}=\sum_{\\{({\bf
x}_{j}^{i},y_{j}^{i})\\}\in{\widetilde{D}}_{j}^{q}}-\log(p(y=y_{j}^{i}\mid{\bf
x}_{j}^{i}))$ (3) where $\displaystyle p(y=y_{j}^{i}\mid{\bf
x}_{j}^{i})=\frac{\exp(-d(F_{j}({\bf x}_{j}^{i}),{\bf
c}_{y_{j}^{i}}))}{\sum_{k^{\prime}}\exp(-d(F_{j}({\bf x}_{j}^{i}),{\bf
c}_{k^{\prime}}))},$

and $d(\cdot)$ is the Euclidean distance. Similarly, the domain-aggregation
network $F_{agg}$ is trained by the loss function $L_{agg}$, which is the same
with $L_{sp}$ except that the support sets and the query sets are sampled from
all the source domains $D$.

The second type of episode generation and training is illustrated in Fig.
1(c). Our objective is to train $F_{agg}$ to extract discriminative features
even for the novel domain data. Hence, we construct the support set
${\widetilde{D}}_{u}^{s}$ and query set ${\widetilde{D}}_{u}^{q}$ from
$u^{th}$ domain and mimic the test case of domain generalization exploiting
$F_{j}$ (domain-mismatch). With the domain-mismatch condition, i.e., $j\neq
u$, unseen domain prototypes ${\bf c}_{k}$ are calculated by Eq. 2 using
$F_{j}$ on the novel domain set ${\widetilde{D}}_{u}^{s}$, and embedding
vectors of ${\widetilde{D}}_{u}^{q}$ are extracted from $F_{agg}$. The loss
$L_{dg}$ for few-shot domain generalization episodes is as follows:

$\displaystyle L_{dg}=\sum_{\\{({\bf
x}_{u}^{i},y_{u}^{i})\\}\in{\widetilde{D}}_{u}^{q}}-\log(p(y=y_{u}^{i}\mid{\bf
x}_{u}^{i}))$ (4) where $\displaystyle p(y=y_{u}^{i}\mid{\bf
x}_{u}^{i})=\frac{\exp(-d(F_{agg}({\bf x}_{u}^{i}),{\bf
c}_{y_{u}^{i}}))}{\sum_{k^{\prime}}\exp(-d(F_{agg}({\bf x}_{u}^{i}),{\bf
c}_{k^{\prime}}))}.$

Since the $u^{th}$ domain data is novel to $F_{j}$, $F_{agg}$ learns to map
embedding vectors in a domain-agnostic way. Here, only $F_{agg}$ is updated by
$L_{dg}$ since we need to keep ${\widetilde{D}}_{u}$ novel to $F_{j}$ during
training. Our training procedure is summarized as follows: 1) Train $F_{agg}$
using $L_{agg}$ and extract domain-discriminative features, 2) Perform
clustering on extracted features and assign pseudo domain labels: this leads
to $M$ domains from $N$ input domains, 3) Generate $M$ domain-specific
networks and train them using Eq. 3, 4) Simultaneously, train $F_{agg}$ using
the combined loss $L_{agg}+\lambda_{dg}L_{dg}$ where $\lambda_{dg}$ is a
balancing parameter.

##  3 Experimental Results

###  3.1 Experiment Setup

Dataset. First, to analyze generalization ability of our framework, we
designed the evaluation experiment of simulating domain differences by
artificially producing data augmentations. We selected 1,088 speakers in
VoxCeleb1 [25], consisting of 1,000 speakers for training and 88 for testing.
Each speaker has 45 utterances, and each utterance was randomly cropped into
2-sec-long for the use-cases of short utterance enrollment and testing. To
explicitly generate several domains, we augmented domains by mixing noises.
Then, we split them into train and test sets to simulate the test condition
where data are recorded in novel domains. The source dataset consists of 4
domains: original set (clean) and three noisy domains which were augmented by
synthesizing the clean with babble, car, and music noises (0dB SNR). Each of
the novel speaker’s data in the target domain consists of 5 utterances for
enrollment and 35 utterances for testing. The target domain consists of 7
domains: original set (clean) and 6 noisy domains which were augmented by
synthesizing the clean with babble, car, music, office, ambient room, and
typing noises (0dB SNR). Thus, our model is evaluated on 4 in-domain sets
(i.e. clean, babble, car, music) and 3 out-domain sets (i.e. office, ambient
room, and typing). Additionally, to analyze models in diverse environments, we
augmented VoxCeleb1 with various noise types and SNR values for source and
target domains. Details are described in Section 3.3.

Next, we evaluated our proposed methods on the standard benchmark following
[22], where VoxCeleb2 [26] is used for training and VoxCeleb1 [25], SITW [27],
and CNCeleb [28] are used for testing. Since VoxCeleb2 contains speech from
speakers spanning a wide range of different ethnicities, accents, professions,
and ages recorded in various environments, we assume that there exists domain
difference intrinsically within the dataset. The development set of VoxCeleb2
contains $5,994$ speakers in total and is entirely disjoint from the test
datasets. VoxCeleb1 test set and SITW-Eval.Core are widely-used evaluation
datasets, and it is worth noting that the acoustic condition of them is
similar with that of the training set VoxCeleb2, but there is domain
difference intrinsically present between and within the datasets. CNCeleb is a
large-scale speaker dataset comprised of 11 different genres from 3,000
Chinese celebrities, and we utilize CNCeleb.E for testing. The acoustic
condition of CNCeleb is severely different from that of VoxCeleb.

Model | EER (%) | FRR @ FAR 10%
---|---|---
X-vectors | 5.685 | 3.817
CE loss | 3.879 | 1.716
ProtoNet | 3.496 | 1.257
Ours w/ original domain labels | 3.231 | 1.016
Ours w/ pseudo domain labels | 2.717 | 0.756

Figure 2: ROC curve, EER, and FRR for different methods.

Implementation details. We use the x-vector system [13] and the speaker
embedding network [18] for evaluating on the artificially generated dataset.
The x-vector system is a standard DNN speaker verification system using
temporal statistics. The speaker embedding network (SE network) uses an 1D-CNN
architecture, and [18] shows that SE network can effectively learn with short
utterances. We use both systems for the baseline, and we adapt our episodic
learning algorithm to SE network. We trained the network with $5$-way $5$-shot
training episodes where 5 support and 5 query points per speaker are included.
Given 4 source domains (clean, babble, car, and music), we trained the domain-
specific networks with two different setups: $N=4$ (original) and $M=3$
(clustered). We chose the balancing parameters as $\lambda_{dg}=0.8$ and
$\lambda_{n}=\\{0.1,0.2,0.3\\}$.

For evaluating our framework on the standard benchmark, we exploit Fast
ResNet-34 with angular prototypical loss function [22] as a baseline. No data
augmentation is performed during training, apart from the random sampling. We
trained the model with $200$-way $2$-shot training following [22]. We adapt
our episodic learning algorithm to Fast ResNet-34 with $\lambda_{dg}=0.1$
using the same training strategy with the baseline for a fair comparison. We
adapt our clustering algorithm to assign pseudo domain labels to training
samples and use them for training domain-specific networks and conduct
ablation studies with various number of domains $M=2,3,4$.

Evaluation metrics. We used False Acceptance Rate (FAR) and False Rejection
Rate (FRR). FAR is the percentage of identification instances in which
unauthorised persons (imposters) are incorrectly accepted, while FRR is that
in which the authorised person (target user) is incorrectly denied. The Equal
Error Rate (EER) is measured at which FAR and FRR are the same. The minimum
detection cost of the function (MinDCF) was defined by the NIST SRE evaluation
plans [34]. DCF is a weighted sum of FRR and FAR as
$C_{fr}\cdot\text{FRR}\cdot
P_{target}+C_{fa}\cdot\text{FAR}\cdot(1-P_{target})$. In particular, the
parameters $C_{fr}=1$, $C_{fa}=1$ and $P_{target}=0.05$ are used for the cost
function.

###  3.2 Comparison Studies on Generated Domains

On the artificially generated domains, we compare our framework with the
following methods: the x-vector system (X-vectors) and SE network trained with
cross-entropy loss (CE loss) which do not consider both novel users and
domains, the prototypical network (ProtoNet) which considers test case of
novel users but not novel domains.

Fig. 2 shows the ROC curve and the table of EER and FRR of the 5 different
methods. X-vectors are robust speaker embeddings given long utterances, but
under short-utterance setting, where all utterances are cropped into 2-sec-
long, SE network outperforms x-vectors. For this reason, we applied our
learning scheme to the SE network and showed 4 different results in Fig. 2.
Our method achieved the best performance compared with other learning
frameworks. It shows that our learning scheme can lead the network to extract
discriminative features even for the unseen users and domains. Surprisingly,
our method w/ pseudo domain labels could achieve memory efficiency and produce
less error compared with the original domain labels. For better domain
generalization, the domain-specific network should have distinct domain
characteristics, i.e., trained with well-separated domain features. In our
case, we define noise type as the domain, and some noise types could have
similar characteristics in the feature space. Thus, pseudo domain labels
obtained with clustering performed better than the original domain labels.
Also, this leads to the reduction in memory requirements for domain-specific
networks. At 10% FAR, ours w/ pseudo domain labels shows 55.94%, 39.86%, and
25.59% relative FRR improvements compared to the other three methods using SE
network.

Figure 3: T-SNE plot of the features obtained by ProtoNet (left) and our
framework (right). Color and shape represent speaker and domains,
respectively. O and X indicate the feature of in-domain and out-domain,
respectively. Table 1: Average EER (%) of four methods given in-domain and
out-domain data with different numbers of domains.

Model | In-4 | Out-3 | Out-11 | Out-6
---|---|---|---|---
CE loss | 3.683 | 4.400 | 4.781 | 10.708
ProtoNet | 2.888 | 3.649 | 4.055 | 9.707
Ours w/ original domain labels | 2.895 | 3.417 | 3.873 | 9.265
Ours w/ pseudo domain labels | 2.528 | 2.916 | 3.396 | 7.185

###  3.3 Additional Analyses

Visualization. As shown in the t-SNE plot [35] of embedding vectors in Fig. 3,
our few-shot domain generalization framework extracts well-clustered features
per each user in a domain-agnostic way. Features extracted from ProtoNet are
clustered well per each user but some features of out-domains are far from
those of in-domains and the cluster center. The features could be easily
rejected during verification, and it leads to higher FRR.

Performance on in-domains and out-domains. In Table 1, the first and second
column respectively indicates the EER given the test data from in-domain (4
source domains) and out-domain (3 novel domains) data. CE loss and ProtoNet
use data augmentation to be robust to noise environments. However, they are
vulnerable to novel domains that cannot be covered by noise augmentations. Our
method could adapt well to out-domains by explicitly considering the domain
generalization test case during training.

Test on various target domains. We evaluated our framework using various out-
domain settings: in Table 1, the third and the fourth column shows EER with 11
novel noise types (Out-11) and 3 novel noise types with an additional severe
SNR value (Out-6). Especially, in Out-6 case, we applied 2 different SNR
values (-6, 0dB) to 3 novel noise types. Our framework shows better results
especially for the Out-6 which has large domain gaps due to severe noise. It
demonstrates that the training scheme to deal with test case of unseen domains
can lead the network to extract discriminative features even under the
situation where train and test domains are severely different.

Train with various source domains. To study the effect of pseudo domain
labels, we tested our framework with more source domains by augmenting
original domains with 3 different SNR values (-6, 0, 6dB SNR). Overall, we
have 10 domains: clean and 9 noisy domains. Training 10 domain-specific
networks requires much memory and compute, and we combine the original domains
by clustering. In Table 2, ours w/ pseudo domain labels with $M=4$ that uses 4
domain-specific components achieved comparable performance compared to ours w/
original domain labels. In addition, ours w/ pseudo domain labels with $M=10$
outperformed over ours w/ original domain labels, ensuring the efficacy of
pseudo domain labels.

Table 2: Effect of number of pseudo domains on EER.

| Original domain labels | Pseudo domain labels
---|---|---
|  $N$=10 |  $M$=10 |  $M$=8 |  $M$=6 |  $M$=4
EER (%) | 2.264 | 2.044 | 2.145 | 2.176 | 2.332

###  3.4 Evaluation on the Standard Benchmark

As shown in Table 3, our episodic learning strategy boosts the overall
performance on all test datasets by a clear margin. Our network learns the
generalization ability from environments intrinsically present within
VoxCeleb2. Hence, it works better for novel environments where the domain
difference between training and test sets is not large. i.e., VoxCeleb1 and
SITW. However, it is hard to improve the generalization ability on novel
genres because VoxCeleb2 does not contain multiple genres, which indicates
that our episodic learning setup cannot mimic the test scenario. Thus, the
performance marginally increases on CNCeleb, where the data are recorded for
various genres, e.g., singing, movie, drama, etc. We expect that our episodic
learning strategy on the training set of CNCeleb can improve the performance
on these novel genres, which we leave for future experimental work.

Table 3: EER and MinDCF of the baseline and ours with different number of
pseudo domain labels (* is the number in [22]).

Model | VoxCeleb1 | SITW | CNCeleb.E
---|---|---|---
EER | MinDCF | EER | MinDCF | EER | MinDCF
Fast ResNet-34 [22] | 2.29 (*2.37) | 0.188 | 4.02 | 0.301 | 15.83 | 0.815
Ours w/ M=2 | 2.09 | 0.177 | 3.94 | 0.278 | 15.50 | 0.717
Ours w/ M=3 | 2.12 | 0.174 | 3.91 | 0.285 | 15.44 | 0.735
Ours w/ M=4 | 2.16 | 0.177 | 3.88 | 0.280 | 16.04 | 0.727

##  4 Conclusion

We propose a domain generalization framework by generating two types of
episodes to learn a speaker verification model that generalize to novel users
and domains. We include domain clustering followed by learning domain-specific
and -aggregation networks in source domain. The domain-aggregation network is
learned to be domain-agnostic with episodes mimicking the test case of novel
users and domains. Extensive experiments show that our framework produces
better domain-agnostic features and considerable performance improvements.

## References

  * [1] H. Li, S. J. Pan, S. Wang, and A. C. Kot, “Domain generalization with adversarial feature learning,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2018.
  * [2] D. Li, Y. Yang, Y.-Z. Song, and T. Hospedales, “Learning to generalize: Meta-learning for domain generalization,” in _Proceedings of the Conference on Artificial Intelligence (AAAI)_ , 2018.
  * [3] D. Li, J. Zhang, Y. Yang, C. Liu, Y.-Z. Song, and T. M. Hospedales, “Episodic training for domain generalization,” in _Proceedings of the IEEE International Conference on Computer Vision (ICCV)_ , 2019.
  * [4] J. Kang, R. Liu, L. Li, Y. Cai, D. Wang, and T. F. Zheng, “Domain-invariant speaker vector projection by model-agnostic meta-learning,” in _Proceedings of the INTERSPEECH_ , 2020.
  * [5] C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in _International Conference on Machine Learning (ICML)_ , 2017.
  * [6] J. Snell, K. Swersky, and R. Zemel, “Prototypical networks for few-shot learning,” in _Advances in Neural Information Processing Systems (NeurIPS)_ , 2017.
  * [7] D. Das, S. Yun, and F. Porikli, “ConFeSS: A framework for single source cross-domain few-shot learning,” in _International Conference on Learning Representations (ICLR)_ , 2022.
  * [8] Y. Guo, N. C. Codella, L. Karlinsky, J. V. Codella, J. R. Smith, K. Saenko, T. Rosing, and R. Feris, “A broader study of cross-domain few-shot learning,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2020.
  * [9] H.-Y. Tseng, H.-Y. Lee, J.-B. Huang, and M.-H. Yang, “Cross-domain few-shot classification via learned feature-wise transformation,” in _International Conference on Learning Representations (ICLR)_ , 2020.
  * [10] G. Heigold, I. Moreno, S. Bengio, and N. Shazeer, “End-to-end text-dependent speaker verification,” in _Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2016.
  * [11] D. Snyder, D. Garcia-Romero, D. Povey, and S. Khudanpur, “Deep neural network embeddings for text-independent speaker verification,” in _Proceedings of the INTERSPEECH_ , 2017.
  * [12] L. Wan, Q. Wang, A. Papir, and I. L. Moreno, “Generalized end-to-end loss for speaker verification,” in _Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2018.
  * [13] D. Snyder, D. Garcia-Romero, G. Sell, D. Povey, and S. Khudanpur, “X-vectors: Robust DNN embeddings for speaker recognition,” in _Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2018.
  * [14] Y. Li, F. Gao, Z. Ou, and J. Sun, “Angular softmax loss for end-to-end speaker verification,” in _International Symposium on Chinese Spoken Language Processing (ISCSLP)_ , 2018.
  * [15] H.-S. Heo, J.-w. Jung, I.-H. Yang, S.-H. Yoon, H.-j. Shim, and H.-J. Yu, “End-to-end losses based on speaker basis vectors and all-speaker hard negative mining for speaker verification,” in _Proceedings of the INTERSPEECH_ , 2019.
  * [16] Y. Liu, L. He, and J. Liu, “Large margin softmax loss for speaker verification,” in _Proceedings of the INTERSPEECH_ , 2019.
  * [17] Z. Ren, Z. Chen, and S. Xu, “Triplet based embedding distance and similarity learning for text-independent speaker verification,” in _Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)_ , 2019.
  * [18] S. Yun, J. Cho, J. Eum, W. Chang, and K. Hwang, “An end-to-end text-independent speaker verification framework with a keyword adversarial network,” in _Proceedings of the INTERSPEECH_ , 2019.
  * [19] P. Anand, A. K. Singh, S. Srivastava, and B. Lall, “Few shot speaker recognition using deep neural networks,” _arXiv preprint arXiv:1904.08775_ , 2019.
  * [20] J. S. Chung, J. Huh, S. Mun, M. Lee, H. S. Heo, S. Choe, C. Ham, S. Jung, B.-J. Lee, and I. Han, “In defence of metric learning for speaker recognition,” in _Proceedings of the INTERSPEECH_ , 2020.
  * [21] J. Wang, K.-C. Wang, M. T. Law, F. Rudzicz, and M. Brudno, “Centroid-based deep metric learning for speaker recognition,” in _Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2019.
  * [22] J. S. Chung, J. Huh, S. Mun, M. Lee, H. S. Heo, S. Choe, C. Ham, S. Jung, B.-J. Lee, and I. Han, “In defence of metric learning for speaker recognition,” in _Proceedings of the INTERSPEECH_ , 2020.
  * [23] X. Huang and S. Belongie, “Arbitrary style transfer in real-time with adaptive instance normalization,” in _Proceedings of the IEEE International Conference on Computer Vision (ICCV)_ , 2017.
  * [24] T. Matsuura and T. Harada, “Domain generalization using a mixture of multiple latent domains.” in _Proceedings of the Conference on Artificial Intelligence (AAAI)_ , 2020.
  * [25] A. Nagrani, J. S. Chung, and A. Zisserman, “Voxceleb: a large-scale speaker identification dataset,” in _Proceedings of the INTERSPEECH_ , 2017.
  * [26] J. S. Chung, A. Nagrani, and A. Zisserman, “Voxceleb2: Deep speaker recognition,” in _Proceedings of the INTERSPEECH_ , 2018.
  * [27] M. McLaren, L. Ferrer, D. Castan, and A. Lawson, “The speakers in the wild (sitw) speaker recognition database.” in _Proceedings of the INTERSPEECH_ , 2016, pp. 818–822.
  * [28] L. Li, R. Liu, J. Kang, Y. Fan, H. Cui, Y. Cai, R. Vipperla, T. F. Zheng, and D. Wang, “Cn-celeb: multi-genre speaker recognition,” _Speech Communication_ , 2022.
  * [29] D. Jung, S. Yang, J. Choi, and C. Kim, “Arbitrary style transfer using graph instance normalization,” in _Proceedings of the IEEE international conference on image processing (ICIP)_ , 2020.
  * [30] B. Kim, S. Yang, J. Kim, and S. Chang, “Domain generalization on efficient acoustic scene classification using residual normalization,” _arXiv preprint arXiv:2111.06531_ , 2021.
  * [31] B. Kim, S. Yang, J. Kim, H. Park, J.-T. Lee, and S. Chang, “Towards robust domain generalization in 2d neural audio processing,” 2021.
  * [32] B. Kim, S. Yang, J. Kim, and S. Chang, “Qti submission to dcase 2021: Residual normalization for device imbalanced acoustic scene classification with efficient design,” _DCASE2021 Challenge, Tech. Rep_ , 2021.
  * [33] J. MacQueen _et al._ , “Some methods for classification and analysis of multivariate observations,” in _Proceedings of the fifth Berkeley symposium on mathematical statistics and probability_ , 1967.
  * [34] O. Sadjadi, C. Greenberg, E. Singer, L. Mason, and D. Reynolds, “Nist 2021 speaker recognition evaluation plan,” 2021-07-12 04:07:00 2021.
  * [35] L. v. d. Maaten and G. Hinton, “Visualizing data using t-sne,” _Journal of machine learning research_ , vol. 9, no. Nov, pp. 2579–2605, 2008.


#  Integrated Sensing and Communication in the Finite Blocklength Regime

Homa Nikbakht1, Michèle Wigger2, Shlomo Shamai (Shitz)3, and H. Vincent Poor1
1Princeton University, $\quad{}^{2}$LTCI,
T$\acute{\mbox{e}}$l$\acute{\mbox{e}}$com Paris, IP Paris,
$\quad{}^{3}$Technion
{homa<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

A point-to-point integrated sensing and communication (ISAC) system is
considered where a transmitter conveys a message to a receiver over a discrete
memoryless channel (DMC) and simultaneously estimates the state of the channel
through the backscattered signals of the emitted waveform. We derive
achievability and converse bounds on the rate-distortion-error tradeoff in the
finite blocklength regime, and also characterize the second-order rate-
distortion-error region for the proposed setup. Numerical analysis shows that
our proposed joint ISAC scheme significantly outperforms traditional time-
sharing based schemes where the available resources are split between the
sensing and communication tasks.

##  I Introduction

Integrating sensing capabilities into a communication network is a promising
approach to resolve the challenges of the upcoming sixth generation (6G)
wireless communication system [1, 2, 3, 4, 5]. In fact, network sensing
functionality is a key enabler to allow sensory data collection from the
environment, which is required in applications such as industrial robots and
autonomous vehicles. A recent paradigm, called _integrated sensing and
communication (ISAC)_ , suggests to fully integrate the sensing functionality
into the communication functionality [6, 7, 8]. In other words, ISAC systems
jointly perform both the sensing and communication tasks using common
hardware, antenna(s) and spectrum. The benefits of such a joint approach are
reductions in hardware and signaling costs and improvements in energy
consumption and spectral efficiency [9, 10].

Despite a considerable amount of interesting ISAC research efforts, the
fundamental performance limits, and thus the inherent tradeoffs between
sensing and communication performances of optimal systems, remain unsolved. In
particular, while [12, 13, 14, 15, 16, 17, 18, 19, 20] determined the
information-theoretic fundamental performance limits for the asymptotic
infinite blocklength regime, the focus of this article lies on the
performances of real codes at finite blocklengths.

Specifically, in this work we consider a point-to-point ISAC system in which
the transmitter conveys a message to a receiver over a discrete memoryless
state-dependent channel, and in addition, based on a generalized feedback
signal, it estimates the memoryless state sequence of the channel so as to
minimize a given distortion criterion. We derive achievability and converse
bounds on the optimal tradeoff between the communication rate and decoding
error and the sensing distortion. Our achievability and converse bounds are
close, and coincide up to third-order terms in the asymptotic regimes of
infinite blocklengths. For this asymptotic regime we thus refine the capacity-
distortion result in [14, 16] to the optimal scaling of the rate as a function
of the allowed distortion and decoding error probability. The finite-
blocklength behavior of ISAC has already been studied in [21], however for a
Gaussian channel model where a single state (the channel coefficient) governs
the entire transmission and the receiver wishes to estimate this state with
smallest possible squared-error. In our setup, the state is described by a
memoryless sequence impacting the various channel uses and the goal of the
estimation is to reconstruct this sequence with minimum distortion.

##  II Problem Setup

Consider the point-to-point setup in Figure 1 where a transmitter wishes to
communicate a message $M$, which is uniformly distributed over a set
$\\{1,\ldots,\mathsf{M}\\}$, to a receiver over a state-dependent memoryless
channel and at the same time wishes to estimate the channel state sequence
based on a generalized feedback signal. We consider the discrete memoryless
state-dependent channel with finite input alphabet $\mathcal{X}$, finite
channel state alphabet $\mathcal{S}$, finite feedback alphabet $\mathcal{Z}$,
finite output alphabet $\mathcal{Y}$ and the channel transition law

$P_{Y^{n}Z^{n}|X^{n}S^{n}}(y^{n},z^{n}|x^{n},s^{n})=\prod_{i=1}^{n}W(y_{i},z_{i}|x_{i},s_{i})$
(1)

for a given conditional pmf $W(\cdot,\cdot|\cdot,\cdot)$.

So, if $M=m$, at a given time $i\in\\{1,\ldots,n\\}$ and after observing the
feedback sequence $Z_{i-1}$, the transmitter sends an input symbol

$X_{i}=f_{i}^{(n)}(m,Z^{i-1})$ (2)

where for any $i\in\\{1,\ldots,n\\}$ the encoding function $f_{i}^{(n)}$ is
defined on appropriate domains. The transmitter also estimates the channel
state $S^{n}$ that is i.i.d according to a given distribution $P_{S}$ as

$\hat{S}^{n}=h^{(n)}(Z^{n},X^{n}),$ (3)

based on a block-estimation function $h^{(n)}$, defined on appropriate
domains.

After observing the channel outputs $Y^{n}$, the receiver decodes the message
$M$ as

$\displaystyle\hat{M}$ $\displaystyle=$ $\displaystyle g^{(n)}(Y^{n}),$ (4)

where $g^{(n)}$ is a decoding function on appropriate domains. The quality of
the state estimation at the transmitter is measured by the expected average
per-block distortion

$\displaystyle\Delta^{(n)}:=\mathbb{E}[d(S^{n},\hat{S}^{n})]$ $\displaystyle=$
$\displaystyle\frac{1}{n}\sum_{i=1}^{n}\mathbb{E}[d(S_{i},\hat{S}_{i})]$ (5)

for a given bounded per-symbol distortion function $d(\cdot,\cdot)$.

The decoding error probability is defined as:

$\epsilon^{(n)}:={\mathbb{P}}[\hat{M}\neq M].$ (6)

###### Definition 1

Given a blocklength $n$, the rate-distortion-error triple
$(\mathsf{R},\mathsf{D},\epsilon)$ is said to be achievable, if there exist
encoding, decoding, and estimation functions $\\{f^{(n)},g^{(n)},h^{(n)}\\}$
satisfying

$\displaystyle\frac{1}{n}\log_{2}(\mathsf{M})$ $\displaystyle\geq$
$\displaystyle\mathsf{R},$ (7) $\displaystyle\epsilon^{(n)}$
$\displaystyle\leq$ $\displaystyle\epsilon,$ (8) $\displaystyle\Delta^{(n)}$
$\displaystyle\leq$ $\displaystyle\mathsf{D}.$ (9)

$M$$f_{i}^{(n)}(M,Z^{i-1})$$X_{i}$$W(y_{i},z_{i}|x_{i},s_{i})$$Y_{i}$$g^{(n)}(Y^{n})$$\hat{M}$$S^{n}$$h(X^{n},Z^{n})$$\hat{S}^{n}$$Z_{i-1}$Transmitter
Figure 1: ISAC System model.

##  III Optimal Estimator

For the described memoryless setup, the optimal state estimator is a
symbolwise estimator applied to the transmitter’s observations $X^{n}$ and
$Z^{n}$:

$\displaystyle\hat{S}^{n}=[(\hat{s}^{*}(X_{1},Z_{1}),\hat{s}^{*}(X_{2},Z_{2}),\ldots,\hat{s}^{*}(X_{n},Z_{n})],$
(10)

where

$\displaystyle\hat{s}^{*}(x,z):=\text{arg}\min_{s"\in\hat{\mathcal{S}}}\sum_{s\in\mathcal{S}}P_{S|XZ}(s|x,z)d(s,s^{\prime}),$
(11)

with

$\displaystyle
P_{S|XZ}(s|x,z)=\frac{P_{S}(s)P_{Z|SX}(z|s,x)}{\sum_{\tilde{s}\in\mathcal{S}}P_{S}(\tilde{s})P_{Z|SX}(z|\tilde{s},x)}.$
(12)

The proof of optimality of this symbolwise estimator relies on the Markov
chain relation

$\displaystyle(X^{i-1},X_{i+1}^{n},Z^{i-1},Z_{i+1}^{n})\multimap-(X_{i},Z_{i})\multimap-
S_{i},$ (13)

see [16, Appendix A] for more details.

##  IV Main Results

Given two random variables $X$ and $Y$ having joint probability mass function
(pmf) $P_{XY}(x,y)$, define their information density

$\displaystyle i(X;Y):=\log\frac{P_{Y|X}(y|x)}{P_{Y}(y)},$ (14)

and notice that the expectation of the information density equals the mutual
information $I(X;Y)=\mathbb{E}[i(X,Y)]$. Denote the higher central moments of
the information density as

$\displaystyle\mathsf{V}$ $\displaystyle:=$ $\displaystyle\text{Var}[i(X;Y)]$
(15) $\displaystyle=$
$\displaystyle\sum_{x,y}P_{X}(x)P_{Y|X}(y|x)\log^{2}\frac{P_{Y|X}(y|x)}{P_{Y}(y)}-I(X;Y)^{2},$
$\displaystyle\mathsf{T}$ $\displaystyle:=$
$\displaystyle\mathbb{E}[|i(X;Y)-\mathbb{I}(X;Y)|^{3}]$ (16) $\displaystyle=$
$\displaystyle\sum_{x,y}P_{X}(x)P_{Y|X}(y|x)\left|\log\frac{P_{Y|X}(y|x)}{P_{Y}(y)}-I(X;Y)\right|^{3}.$

Our main results are the following theorems on the rate-distortion-error
tradeoff.

######  Theorem 1 (Achievability Bound)

Given a blocklength $n$, the rate-distortion-error tradeoff
$(\mathsf{R},\mathsf{D},\epsilon)$ is achievable if there exists a $P_{X}$ and
a constant $\mathsf{K}>0$ such that the following two conditions are
satisfied,

$\displaystyle\mathsf{R}$ $\displaystyle\leq$ $\displaystyle
I(X;Y)-\sqrt{\frac{\mathsf{V}}{n}}\mathbb{Q}^{-1}\left(\epsilon-\beta_{u}\right)-\mathsf{K}\frac{\log(n)}{n},$
(17) $\displaystyle\mathsf{D}$ $\displaystyle\geq$
$\displaystyle\sum_{x\in\mathcal{X}}\sum_{s\in\mathcal{S}}\sum_{z\in\mathcal{Z}}d(s,\hat{s}^{*}(x,z))P_{X}(x)P_{S}(s)P_{Z|XS}(z|x,s),$
(18)

with

$\beta_{u}:=\frac{1}{n^{\mathsf{K}}}+\frac{0.7975\mathsf{T}}{\sqrt{n\mathsf{V}^{3}}},$
(19)

and where the mutual information $I(X;Y)$ and the two central moments
$\mathsf{V}$ and $\mathsf{T}$ are defined based on the joint pmf
$P_{XY}(x,y)=P_{X}(x)P_{Y|X}(y|x)$.

###### Proof:

See Section VI. ∎

We also have the following converse bound.

######  Theorem 2 (Converse Bound)

Given the blocklength $n$, a rate-distortion-error triple
$(\mathsf{R},\mathsf{D},\epsilon)$ is not achievable if for all $\delta>0$ and
pmfs $P_{X}$ satisfying (18) the following lower bound holds:

$\displaystyle\mathsf{R}$ $\displaystyle\geq$ $\displaystyle
I(X;Y)-\sqrt{\frac{\mathsf{V}}{n}}\mathbb{Q}^{-1}\left(\epsilon+\beta_{l}\right)+\frac{\log(n)}{2n}-\frac{\log\delta}{n},$
(20)

where

$\beta_{l}:=\frac{0.7975\mathsf{T}}{\sqrt{n\mathsf{V}^{3}}}+\frac{\delta}{\sqrt{n}}.$
(21)

###### Proof:

The proof of the bound in (20) follows similar steps as the proof of [22,
Lemma 58], where one has to integrate the optimal estimator in (10). See
Appendix A for details. ∎

###### Proposition 1

Given $\mathsf{D}$, $\epsilon$ and large blocklengths $n$, the largest rate
$\mathsf{R}$ such that the triple $(\mathsf{R},\mathsf{D},\epsilon)$ is
achievable, is given by

$\mathsf{R}_{\max}(\mathsf{D},\epsilon,n)=\max_{P_{X}}\left[I(X;Y)-\sqrt{\frac{\mathsf{V}}{n}}\mathbb{Q}^{-1}\left(\epsilon\right)+O\left(\frac{\log
n}{n}\right)\right],$ (22)

where the maximum is over all pmfs $P_{X}$ satisfying (18).

###### Proof:

By the differentiability $\mathbb{Q}^{-1}$ and by the forms of $\beta_{u}$ and
$\beta_{l}$ in (19) and (21), we have

$\displaystyle\mathbb{Q}^{-1}(\epsilon-\beta_{u})$ $\displaystyle=$
$\displaystyle\mathbb{Q}^{-1}(\epsilon)+O\left(\frac{1}{\sqrt{n}}\right),$
(23) $\displaystyle\mathbb{Q}^{-1}(\epsilon+\beta_{l})$ $\displaystyle=$
$\displaystyle\mathbb{Q}^{-1}(\epsilon)+O\left(\frac{1}{\sqrt{n}}\right).$
(24)

Substituting (23) into (17), and (24) into (20) proves the proposition. ∎

###### Remark 1

Equality (22) agrees with [22, Theorem 49] which determines the second-order
coding rate of a DMC in the finite blocklength regime.

##  V Comparisons and Examples

In this section, we evaluate Theorems 1 and 2 numerically for a binary example
and compare them also with the performance of two baseline schemes that are
frequently employed in practice.

###  V-A Time-Sharing Schemes

Many practical systems employ a basic resource-sharing approach where a
fraction of the resources (here $(1-\gamma)n$ channel uses) are dedicated only
to the communication task and the remaining resources (here $\gamma n$ channel
uses) to the sensing task, each one completely ignoring the other task. A
slightly improved scheme uses the resources for the communication task also
for some basic sensing, but using the waveform that is best for communication,
and similarly uses the resources for the sensing task also for communication,
but using the best waveform for sensing.

####  V-A1 Basic Resource-Sharing Scheme

Given time-sharing parameter $\gamma\in[0,1]$, the performance of the basic
resource-sharing scheme described above achieves rate

$\mathsf{R}=(1-\gamma)\mathsf{R}_{\max}$ (25)

and distortion

$\mathsf{D}=\gamma\mathsf{D}_{\min}+(1-\gamma)\mathsf{D}_{\textnormal{trivial}},$
(26)

where $\mathsf{R}_{\max}$ is the largest achievable rate:

$\mathsf{R}_{\max}:=\max_{P_{X}}\left[I(X;Y)-\sqrt{\frac{\mathsf{V}}{n}}\mathbb{Q}^{-1}\left(\epsilon-\beta_{u}\right)-\mathsf{K}\frac{\log(n)}{n}\right]$
(27)

and $\mathsf{D}_{\min}$ denotes the best possible distortion while
$\mathsf{D}_{\text{trivial}}$ denotes the distortion achieved by the optimal
trivial estimator that does not exploit the feedback:

$\displaystyle\mathsf{D}_{\min}$ $\displaystyle:=$
$\displaystyle\min_{P_{X}}\sum_{x\in\mathcal{X}}\sum_{s\in\mathcal{S}}\sum_{z\in\mathcal{Z}}P_{X}(x)P_{S}(s)P_{Z|SX}(z|s,x)\hat{s}^{*}(x,z),$
$\displaystyle\mathsf{D}_{\text{trivial}}$ $\displaystyle:=$
$\displaystyle\min_{s^{\prime}\in\mathcal{S}}\sum_{s\in\mathcal{S}}P_{S}(s)d(s,s^{\prime}).$
(29)

####  V-A2 Improved Resource-Sharing Scheme

For a given time-sharing parameter $\gamma\in[0,1]$, the improved resource-
sharing scheme achieves rate

$\mathsf{R}=\gamma\mathsf{R}_{\textnormal{sense}}+(1-\gamma)\mathsf{R}_{\max}$
(30)

and distortion

$\mathsf{D}=\gamma\mathsf{D}_{\min}+(1-\gamma)\mathsf{D}_{\textnormal{comm}},$
(31)

where

$\displaystyle\mathsf{D}_{\textnormal{comm}}:=\sum_{x\in\mathcal{X}}P_{X}^{\star}(x)\sum_{s\in\mathcal{S}}\sum_{z\in\mathcal{Z}}P_{S}(s)P_{Z|SX}(z|s,x)\hat{s}^{*}(x,z)$
(32)

for $P_{X}^{\star}$ the optimizer in (27) and

$\displaystyle\mathsf{R}_{\textnormal{sense}}:=I(X;Y)-\sqrt{\frac{\mathsf{V}}{n}}\mathbb{Q}^{-1}\left(\epsilon-\beta_{u}\right)-\mathsf{K}\frac{\log(n)}{n}$
(33)

evaluated for $P_{XY}=P_{X}^{\prime}P_{Y|X}$ with $P_{X}^{\prime}$ the
optimizer of (V-A1).

###  V-B Binary Channel with Multiplicative Bernoulli State

Consider the channel

$Y=SX,$ (34)

with binary alphabets $\mathcal{X}=\mathcal{S}=\mathcal{Y}\in\\{0,1\\}$ and
where the state is Bernoulli-$q$ with $q\in(0,1)$ and the feedback is perfect,
i.e., $Z=Y$. We consider the Hamming distortion measure
$d(s,\hat{s})=s\oplus\hat{s}$.

To compare the performance specified in Theorems 1 and 2 with each other and
with the performance of the two baseline time-sharing schemes, we parametrize
the binary input distribution $P_{X}$ by $\alpha:={\mathbb{P}}[X=1]$. We also
notice that the channel in (34) is equivalent to a Z-Channel: input $0$ always
leads to the output symbol 0 and input $1$ leads to output 0 with probability
$1-q$ and to output 1 with probability $q$. The mutual information between
input and output of the channel is then obtained as

$\displaystyle I(X;Y)$ $\displaystyle=$ $\displaystyle H_{b}(q\alpha)-\alpha
H_{b}(q),$ (35a) where $H_{b}(x)=-x\log(x)-(1-x)\log(1-x)$ is the binary
entropy function. For the second and third central moments of the information
density we have $\displaystyle\mathsf{V}_{\alpha}$ $\displaystyle=$
$\displaystyle\alpha\left(q\log^{2}\frac{1}{\alpha}+(1-q)\log^{2}\frac{1-q}{1-q\alpha}\right)$
(35b) $\displaystyle+(1-\alpha)\log^{2}\frac{1}{1-q\alpha}-I(X;Y)^{2},$
$\displaystyle\mathsf{T}_{\alpha}$ $\displaystyle=$ $\displaystyle\alpha
q\left|\log\frac{1}{\alpha}-I(X;Y)\right|^{3}$ (35c)
$\displaystyle+(1-q)\left|\log\frac{1-q}{1-q\alpha}-I(X;Y)\right|^{3}$
$\displaystyle+(1-\alpha)\left|\log\frac{1}{1-q\alpha}-I(X;Y)\right|^{3},$

We can then substitute $I(X;Y)$ and $\mathsf{V},\mathsf{T}$ from (35) into
(17) and (20) to obtain the desired bounds on the rate.

To calculate the distortion bound (18), notice that whenever $x=1$, then
$z=y=s$ and thus the distortion is zero. On the other hand, when $x=0$ then
$y=0$ and the transmitter does not receive any information about the state of
the channel. In this case, the optimal estimator is to choose the most likely
state symbol, i.e. $\hat{s}=0$ if $q<1/2$ and $\hat{s}=1$ if $q\geq 1/2$. We
combine these observations to obtain the following bound:

$\displaystyle\mathsf{D}$ $\displaystyle\geq$ $\displaystyle
P_{X}(0)\sum_{s,y}d(s,\hat{s}^{*}(x=0,y))P_{S}(s)P_{Y|XS}(y|x=0,s)$ (36)
$\displaystyle=$ $\displaystyle
P_{X}(0)\sum_{s\in\mathcal{S}}d(s,\hat{s}^{*}(x=0,y=0))P_{S}(s)$ (37)
$\displaystyle=$ $\displaystyle(1-\alpha)\min\\{q,1-q\\}.$ (38)

In other words, a distortion constraint imposes the following bound on
$\alpha$:

$\alpha\geq 1-\frac{\mathsf{D}}{\min\\{q,1-q\\}}.$ (39)

Thus, for this example Theorem 1 states that for any $\mathsf{D}>0$, all
triples $(\mathsf{R},\mathsf{D},\epsilon)$ are achievable if

$\mathsf{R}\leq\max_{\alpha,\mathsf{K}\geq
0}I(X;Y)-\sqrt{\frac{\mathsf{V}}{n}}\mathbb{Q}^{-1}\left(\epsilon-\beta_{u}\right)-\mathsf{K}\frac{\log(n)}{n},$
(40)

where the maximization is over all $\alpha\in[0,1]$ satisfying
$1\geq\alpha\geq 1-\frac{\mathsf{D}}{\min\\{q,1-q\\}}$. Theorem 2 states that
for any $\mathsf{D}>0$ all triples $(\mathsf{R},\mathsf{D},\epsilon)$
satisfying

$\mathsf{R}\geq\max_{\alpha,\delta>0}I(X;Y)-\sqrt{\frac{\mathsf{V}}{n}}\mathbb{Q}^{-1}\left(\epsilon+\beta_{l}\right)+\frac{\log(n)}{2n}-\frac{\log\delta}{n}$
(41)

are not achievable. Here, the maximization is again over values
$\alpha\in\left[\min\left\\{0,1-\frac{\mathsf{D}}{\min\\{q,1-q\\}}\right\\},1\right]$.

Notice that for this channel (which is a Z-channel) the capacity is equal to
[23]

$\displaystyle\mathsf{C}=\log(1+q(1-q)^{\frac{1-q}{q}}),$ (42)

and is achieved for

$P_{X}^{\star}(1)=\alpha^{\star}=\frac{1}{q\left(1+2^{\frac{H_{b}(q)}{q}}\right)}.$
(43)

The distortion achieved with this capacity-achieving $\alpha^{\star}$ is
$\mathsf{D}_{\textnormal{comm}}=(1-\alpha^{\star})\min\\{q,1-q\\}$.

###  V-C Numerical Analysis

Fig. 2 illustrates the achievability and converse bounds on the rate-
distortion-error tradeoff presented in (40) and (41) for $\epsilon=0.05$ and
$q=0.4$. As can be seen from this figure the bounds are tight for large values
of $n$. Notice that for $q=0.4$ the capacity of the channel is
$\mathsf{C}=0.246$ and the achieved distortion is
$\mathsf{D}_{\textnormal{comm}}=0.2432$.

Figure 2: Achievability and converse bounds on the rate-distortion-error
trade-off of Theorems 1 and 2 for $\epsilon=0.05$, $q=0.4$, and different
values of $n$.  Figure 3: Comparison of the rate-distortion-error trade-off in
Theorems 1 and 2 with the basic and improved resource-sharing schemes for
$\epsilon=0.05$, $q=0.4$, and $n=700$.

Fig. 3 compares the rate-distortion-error tradeoff achieved by our scheme with
the tradeoff achieved under the basic and improved resource-sharing schemes.
As can be seen from this figure, our scheme outperforms the other two baseline
schemes.

##  VI Proof of Theorem 1

###  VI-A Codebook Generation

Choose $P_{X}$ satisfying (18). The codebook
$\mathcal{C}=\\{x^{n}(m)\\}_{m=1}^{\mathsf{M}}$ is generated by randomly and
independently choosing each entry according to $P_{X}$.

###  VI-B Encoding

To send a message $m$, the transmitter encodes this message via the codeword
$x^{n}(m)$ and sends it over the channel.

###  VI-C Estimation

After observing the feedback sequence $Z^{n}=z^{n}$, the transmitter estimates
the channel state through (10).

###  VI-D Decoding

Given the channel outputs $Y^{n}=y^{n}$, the receiver estimates the message
$M$ by choosing the index $\hat{m}$ that corresponds to the codeword
$x^{n}(\hat{m})$ that maximizes the information density:

$\hat{m}:=\textnormal{arg}\max_{m}i\left(x^{n}(m);y^{n}\right).$ (44)

The receiver then produces the guess $\hat{M}=\hat{m}$.

###  VI-E Error Analysis

To analyze ${\mathbb{P}}[\hat{M}\neq M]$, we use the threshold-based metric
bound in [22]. For any $\gamma\in\mathbb{R}$, we have

$\displaystyle{\mathbb{P}}[\hat{M}\neq M]$ $\displaystyle\leq$
$\displaystyle{\mathbb{P}}[i(X^{n};Y^{n})\leq\gamma]+\mathsf{M}\cdot{\mathbb{P}}[i(\bar{X}^{n};Y^{n})\geq\gamma],$
(45)

where $\bar{X}^{n}\sim P_{X^{n}}$ and is independent of $X^{n}$ and $Y^{n}$.
We will set

$\displaystyle\gamma$ $\displaystyle:=$
$\displaystyle\log\mathsf{M}+\mathsf{K}\log n,$ (46)

for some $\mathsf{K}>0$, and employ the Berry-Esseen theorem and Bayes’
formula to evaluate the two terms on the right-hand side of (45).

By the strengthening of the Berry-Esseen theorem in [24], and because
$\mathbb{E}[i(X^{n};Y^{n})]=nI(X;Y)$, we have with the definition in (46)

$\displaystyle{\mathbb{P}}\left[i(X^{n};Y^{n})\leq\gamma\right]$
$\displaystyle\leq$
$\displaystyle\mathbb{Q}\left(\frac{-\log\mathsf{M}+nI(X;Y)-\mathsf{K}\log(n)}{\sqrt{n\mathsf{V}}}\right)$
(47) $\displaystyle+\frac{0.7975\mathsf{T}}{\sqrt{n\mathsf{V}^{3}}}.$

To bound ${\mathbb{P}}[i(\bar{X}^{n};Y^{n})\geq\gamma]$, we first use Bayes’
formula to write

$\displaystyle P_{{X}^{n}}(\overline{x}^{n})$ $\displaystyle=$
$\displaystyle\frac{P_{{Y}^{n}}({y^{n}})P_{{X}^{n}|{Y}^{n}}(\overline{x}^{n}|{y}^{n})}{P_{{Y}^{n}|{X}^{n}}({y}^{n}|\overline{x}^{n})}$
(48) $\displaystyle=$ $\displaystyle
P_{\bar{X}^{n}|{Y}^{n}}(\overline{x}^{n}|{y}^{n})2^{-i(\bar{x}^{n};y^{n})}.$
(49)

For any ${y}^{n}\in\mathbb{R}^{n}$, we then have

$\displaystyle\sum_{\bar{x}^{n}\in\mathcal{X}}\mathbbm{1}\left\\{i(\bar{x}^{n};y^{n})>\gamma\right\\}P_{{X}^{n}}(\overline{x}^{n})$
$\displaystyle=\sum_{\bar{x}^{n}\in\mathcal{X}}2^{-i(\bar{x}^{n};y^{n})}\mathbbm{1}\left\\{\frac{P_{Y^{n}|X^{n}}(y^{n}|\overline{x}^{n})}{P_{Y^{n}}(y^{n})}>2^{\gamma}\right\\}$
$\displaystyle\hskip 85.35826pt\cdot
P_{{X}^{n}|{Y}^{n}}(\overline{x}^{n}|{y}^{n})$ (50)
$\displaystyle\leq\sum_{\bar{x}^{n}\in\mathcal{X}}2^{-i(\bar{x}^{n};y^{n})}\frac{P_{Y^{n}|X^{n}}(y^{n}|\overline{x}^{n})}{P_{Y^{n}}(y^{n})}2^{-\gamma}P_{{X^{n}}|{Y^{n}}}(\overline{x}^{n}|{y}^{n})$
$\displaystyle=\sum_{\bar{x}^{n}\in\mathcal{X}}P_{{X}^{n}|{Y}^{n}}(\overline{x}^{n}|{y}^{n})2^{-\gamma}$
$\displaystyle=2^{-\gamma}.$ (51)

As a consequence,

$\displaystyle{\mathbb{P}}[i(\bar{X}^{n};Y^{n})\geq\gamma]\leq 2^{-\gamma}$
(52)

and

$\displaystyle\mathsf{M}{\mathbb{P}}[i(\bar{X}^{n};Y^{n})\geq\gamma]\leq
2^{-\gamma+\log\mathsf{M}}=n^{-\mathsf{K}}.$ (53)

Combining (45), (47), and (53), we obtain

$\displaystyle{\mathbb{P}}[\hat{M}\neq
M]\leq\mathbb{Q}\left(\frac{-\log\mathsf{M}+nI(X;Y)-\mathsf{K}\log(n)}{\sqrt{n\mathsf{V}}}\right)+\beta_{u},$

where $\beta_{u}$ is defined in (19).

Thus, the probability of error stays below $\epsilon$ whenever

$\displaystyle\epsilon-\beta_{u}\geq\mathbb{Q}\left(\frac{-\log\mathsf{M}+nI(X;Y)-\mathsf{K}\log(n)}{\sqrt{n\mathsf{V}}}\right),$
(55)

or equivalently when

$\displaystyle\log\mathsf{M}\leq
nI(X;Y)-\sqrt{n\mathsf{V}}\mathbb{Q}^{-1}(\epsilon-\beta_{u})-\mathsf{K}\log(n),$
(56)

establishing the bound in (17).

###  VI-F Expected Distortion

The expected distortion can be written as

$\displaystyle\Delta^{(n)}$ $\displaystyle=$
$\displaystyle\frac{1}{n}\sum_{i=1}^{n}\mathbb{E}[d(S_{i},\hat{S}_{i})]$
$\displaystyle=$
$\displaystyle\sum_{x\in\mathcal{X}}\sum_{s\in\mathcal{S}}\sum_{z\in\mathcal{Z}}d(s,\hat{s}^{*}(x,z))P_{X}(x)P_{S}(s)P_{Z|XS}(z|x,s).$

By our choice of $P_{X}$, our scheme thus satisfies the requirement on the
distortion.

##  VII Conclusions

We have studied the rate-distortion-error tradeoff of a point-to-point ISAC
system where a transmitter conveys a message to a receiver over a discrete
memoryless state-dependent channel and simultaneously estimates the state of
the channel. We have derived achievability and converse bounds on the rate-
distortion-error tradeoff in the finite blocklength regime. We also have
characterized the second-order rate-distortion-error region of the proposed
setup. Our numerical analysis shows that our joint design scheme significantly
outperforms the resource-sharing baseline schemes where the available
resources are split between the sensing and communication tasks. In our model
the receiver has no state-information. The generality of our model allows
however to obtain results for perfect or partial state-information as special
cases from our Theorems 1 and 2, simply by including the state-information as
part of the receiver’s output. An interesting line of future work is to study
the ISAC problem with general state and channel distribution in the finite
blocklength regime [25].

## Acknowledgment

The work of H. Nikbakht and H. V. Poor has been supported by the U.S National
Science Foundation under Grant CNS-2128448. The work of S. Shamai (Shitz) has
been supported by the US-Israel Binational Science Foundation (BSF) under
grant BSF-2018710 and by the German Research Foundation (DFG) via the German-
Israeli Project Cooperation (DIP), under Project SH 1937/1-1.

##  Appendix A Proof of Theorem 2

The proof of the bound in (20) follows similar steps as the proof of [22,
Lemma 58, Theorem 28]. In the following, we sketch the proof of the converse
bound (20).

Consider a random variable $Y$ on $\mathcal{Y}$ which can take probability
measures $P_{Y|X}$ and $P_{Y}$. Define by $P_{Z|Y}:\mathcal{Y}\to\\{0,1\\}$ a
randomized test between those two distributions where $0$ indicates that the
test chooses $P_{Y}$. Let $\beta_{\alpha}(P_{Y|X},P_{Y})$ be the minimum
probability of error under hypothesis $P_{Y}$ if the probability of error
under hypothesis $P_{Y|X}$ is below $1-\alpha$. I.e.,

$\displaystyle\beta_{\alpha}(P_{Y|X},P_{Y})$ (59) $\displaystyle=$
$\displaystyle\min_{P_{Z|Y}:\sum_{y\in\mathcal{Y}}P_{Y}(y)P_{Z|Y}(1|y)\geq\alpha}\sum_{y\in\mathcal{Y}}P_{Y}(y)P_{Z|Y}(1|y).$

It is known that $\beta_{\alpha}(P_{Y|X},P_{Y})$ is the best performance
achievable among such randomized tests.

It is easy to show that for any $\tilde{\gamma}>0$,

$\displaystyle\alpha\leq{\mathbb{P}}\left[\frac{dP_{Y|X}}{dP_{Y}}\geq\tilde{\gamma}\right]+\tilde{\gamma}\beta_{\alpha}(P_{Y|X},P_{Y}).$
(60)

Equivalently, for any $\gamma_{n}>0$

$\displaystyle\beta_{\alpha}(P_{Y^{n}|X^{n}},P_{Y^{n}})\geq\frac{1}{\gamma_{n}}\left(\alpha-{\mathbb{P}}\left[\log\frac{dP_{Y^{n}|X^{n}}}{dP_{Y^{n}}}\geq\log\gamma_{n}\right]\right).$
(61)

Set

$\displaystyle\log\gamma_{n}=nI(X;Y)+\sqrt{n\mathsf{V}}Q^{-1}(\alpha_{n})$
(62)

with

$\alpha_{n}=\alpha-\frac{0.7975\mathsf{T}}{\sqrt{n\mathsf{V}^{3}}}-\frac{\delta}{\sqrt{n}}$
(63)

for some $\delta>0$. By employing the strengthened version of the Berry-Esseen
theorem [24], we have

$\displaystyle\left|{\mathbb{P}}\left[\log\frac{dP_{Y^{n}|X^{n}}}{dP_{Y^{n}}}\geq\log\gamma_{n}\right]-\alpha_{n}\right|\leq\frac{0.7975\mathsf{T}}{\sqrt{n\mathsf{V}^{3}}}.$
(64)

Consequently

$\displaystyle{\mathbb{P}}\left[\log\frac{dP_{Y^{n}|X^{n}}}{dP_{Y^{n}}}\geq\log\gamma_{n}\right]\leq\alpha-\frac{\delta}{\sqrt{n}}.$
(65)

Substituting (65) into (61), we have

$\displaystyle\beta_{\alpha}(P_{Y^{n}|X^{n}},P_{Y^{n}})\geq\frac{\delta}{\gamma_{n}\sqrt{n}},$
(66)

and by (62)

$\displaystyle\log(\beta_{\alpha}(P_{Y^{n}|X^{n}},P_{Y^{n}}))$ (67)
$\displaystyle\geq$
$\displaystyle\log(\delta)-nI(X:Y)-\sqrt{n\mathsf{V}}\mathbb{Q}^{-1}(\alpha_{n})-\frac{1}{2}\log(n).$

By [22, Theorem 27], every $(\mathsf{M},\epsilon)$-code satisfies

$\displaystyle\log\mathsf{M}\leq-\log(\beta_{1-\epsilon}(P_{Y^{n}|X^{n}},P_{Y^{n}})).$
(68)

By (67) and $\alpha=1-\epsilon$ and the fact that
$\mathbb{Q}^{-1}(1-x)=-\mathbb{Q}^{-1}(x)$,

$\displaystyle\log\mathsf{M}$ $\displaystyle\leq$ $\displaystyle
nI(X:Y)-\sqrt{nV}\mathbb{Q}^{-1}(\epsilon+\frac{0.7975\mathsf{T}}{\sqrt{n\mathsf{V}^{3}}}+\frac{\delta}{\sqrt{n}})$
(69) $\displaystyle+\frac{1}{2}\log(n)-\log(\delta)$

which proves the inequality (20). Combined with the optimal estimator in (18),
this proves the theorem.

## References

  * [1] F. Liu et al., "Integrated sensing and communications: Toward dual-functional wireless networks for 6G and beyond," _IEEE Journal on Selected Areas in Communications_ , vol. 40, no. 6, pp. 1728–1767, June 2022.
  * [2] J. M. Mateos-Ramos et al., “End-to-end learning for integrated sensing and communication," in _Proceedings of the IEEE International Conference on Communications_ , Seoul, Korea, Republic of, pp. 1942–1947, 16-20 May, 2022.
  * [3] H. Zhang et al., “Holographic integrated sensing and communication," _IEEE Journal on Selected Areas in Communications_ , vol. 40, no. 7, pp. 2114–2130, July 2022.
  * [4] A. Liu, M. Li, M. Kobayashi, and G. Caire, “Fundamental limits for ISAC: Information and communication theoretic perspective", in _F. Liu, C. Masouros, Y.C. Eldar, (eds), Integrated Sensing and Communications. Springer_ , Singapore, 2023.
  * [5] A. Liu et al., "A survey on fundamental limits of integrated sensing and communication," _IEEE Communications Surveys & Tutorials_, vol. 24, no. 2, pp. 994–1034, Secondquarter 2022.
  * [6] X. Cheng, D. Duan, S. Gao and L. Yang, “Integrated sensing and communications (ISAC) for vehicular communication networks (VCN)," _IEEE Internet of Things Journal_ , vol. 9, no. 23, pp. 23441–23451, Dec., 2022.
  * [7] H. Li, Z. Han, and H. V. Poor, “Cellular system based integrated sensing and communications for wide-area monitoring," in _Proceedings of the IEEE International Geoscience and Remote Sensing Symposium_ , pp. 592-595, Pasadena, CA, USA, July 16-21, 2023.
  * [8] Z. Wei et al., "Integrated sensing and communication signals toward 5G-A and 6G: A survey," _IEEE Internet of Things Journal_ , vol. 10, no. 13, pp. 11068-11092, 1 July1, 2023.
  * [9] D. K. P. Tan et al., “Integrated sensing and communication in 6G: Motivations, use cases, requirements, challenges and future directions," in _Proceedings of the IEEE International Online Symposium on Joint Communications & Sensing_, Dresden, Germany, pp. 1–6, 2021.
  * [10] C. Chaccour, W. Saad, M. Debbah, and H. V. Poor, “Joint sensing, communication, and AI: A trifecta for resilient THz user experiences,” Online: arXiv:2305.00135, May, 2023.
  * [11] J. Wang, N. Varshney, C. Gentile, S. Blandino, J. Chuang, and N. Golmie, “Integrated sensing and communication: Enabling techniques, applications, tools and data sets, standardization, and future directions," _IEEE Internet of Things Journal_ , vol. 9, no. 23, pp. 23416–23440, Dec. 2022.
  * [12] J. An, H. Li, D. W. K. Ng, and C. Yuen, “Fundamental detection probability vs. achievable rate tradeoff in integrated sensing and communication systems," _IEEE Transactions on Wireless Communications_ , vol. 22, no. 12, pp. 9835–9853, Dec. 2023.
  * [13] H. Joudeh and F. M. J. Willems, “Joint communication and binary state detection,” _IEEE Journal on Selected Areas in Information Theory_ , vol. 3, no. 1, pp. 113–124, 2022.
  * [14] M. Kobayashi, G. Caire, and G. Kramer, “Joint state sensing and communication: Optimal tradeoff for a memoryless case," in _Proceedings of the IEEE International Symposium on Information Theory_ , pp. 111–115, Vail, CO, USA, June 17-22, 2018.
  * [15] M. Kobayashi, H. Hamad, G. Kramer, and G. Caire, “Joint state sensing and communication over memoryless multiple access channels,” in _Proceedings of the IEEE International Symposium on Information Theory_ , pp. 270–274, 2019.
  * [16] M. Ahmadipour, M. Kobayashi, M. Wigger, and G. Caire, “An information-theoretic approach to joint sensing and communication,” _IEEE Transactions on Information Theory_ , vol. 70, pp. 1124 – 1146, 2022.
  * [17] M. Ahmadipour, M. Wigger, and S. Shamai, “Integrated communication and receiver sensing with security constraints on message and state," in _Proceedings of the IEEE International Symposium on Information Theory_ , pp. 2738–2743, Taipei, Taiwan, June 25–30, 2023.
  * [18] M. Ahmadipour and M. Wigger, "An information-theoretic approach to collaborative integrated sensing and communication for two-transmitter systems," _IEEE Journal on Selected Areas in Information Theory_ , vol. 4, pp. 112–127, 2023.
  * [19] H. Hua, T. X. Han, and J. Xu, “MIMO integrated sensing and communication: CRB-rate tradeoff," _IEEE Transactions on Wireless Communications_ , Aug. 2023.
  * [20] J. Yao, L. Mai, and Q. Zhang, “Approximate capacity-distortion region of joint state sensing and communication in MIMO real Gaussian channels," _IEEE Transactions on Communications_ , Dec. 2023.
  * [21] X. Shen, N. Zhao, and Y. Shen, “On the performance tradeoff of an ISAC system with finite blocklength," in _Proceedings of the IEEE International Conference on Communications_ , pp. 4628–4633, Rome, Italy, 28 May-01 June, 2023.
  * [22] Y. Polyanskiy, H. V. Poor and S. Verdú, “Channel coding rate in the finite blocklength regime," _IEEE Transactions on Information Theory_ , vol. 56, no. 5, pp. 2307–2359, May, 2010.
  * [23] L. G. Tallini, S. Al-Bassam and B. Bose, "On the capacity and codes for the Z-channel," in _Proceedings of the IEEE International Symposium on Information Theory_ , pp. 422, Lausanne, Switzerland, 2002.
  * [24] P. Van Beeck, “An application of Fourier methods to the problem of sharpening the Berry-Esseen inequality,” _Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete,_ vol. 23, pp. 187–196, 1972.
  * [25] Y. Chen, T. Oechtering, M. Skoglund, and Y. Luo, “On general capacity-distortion formulas of integrated sensing and communication,” Online: arXiv:2310.11080, Oct. 2023.