ElectroCardioGuard: Preventing Patient Misidentification in
  Electrocardiogram Databases through Neural Networks
Electrocardiograms (ECGs) are commonly used by cardiologists to detect
heart-related pathological conditions. Reliable collections of ECGs are crucial
for precise diagnosis. However, in clinical practice, the assignment of
captured ECG recordings to incorrect patients can occur inadvertently. In
collaboration with a clinical and research facility which recognized this
challenge and reached out to us, we present a study that addresses this issue.
In this work, we propose a small and efficient neural-network based model for
determining whether two ECGs originate from the same patient. Our model
demonstrates great generalization capabilities and achieves state-of-the-art
performance in gallery-probe patient identification on PTB-XL while utilizing
760x fewer parameters. Furthermore, we present a technique leveraging our model
for detection of recording-assignment mistakes, showcasing its applicability in
a realistic scenario. Finally, we evaluate our model on a newly collected ECG
dataset specifically curated for this study, and make it public for the
research community.
Michal SejÃ¡k, Jakub Sido, David Å½ahour
ElectroCardioGuard: Preventing Patient Misidentification in Electrocardiogram Databases through Neural Networks

###### Abstract

Electrocardiograms (ECGs) are commonly used by cardiologists to detect heart-related pathological conditions. Reliable collections of ECGs are crucial for precise diagnosis. However, in clinical practice, the assignment of captured ECG recordings to incorrect patients can occur inadvertently. In collaboration with a clinical and research facility which recognized this challenge and reached out to us, we present a study that addresses this issue. In this work, we propose a small and efficient neural-network based model for determining whether two ECGs originate from the same patient. Our model demonstrates great generalization capabilities and achieves state-of-the-art performance in gallery-probe patient identification on PTB-XL while utilizing 760x fewer parameters. Furthermore, we present a technique leveraging our model for detection of recording-assignment mistakes, showcasing its applicability in a realistic scenario. Finally, we evaluate our model on a newly collected ECG dataset specifically curated for this study, and make it public for the research community.

## 1 Introduction

Accurate interpretation of electrocardiogram (ECG) recordings is crucial for achieving high diagnostic accuracy and minimizing the risk of errors during diagnosis establishment and consecutive treatment. Unfortunately, in administrative practice, instances occur where physicians capture ECG recordings and assign them to incorrect patients by mistake. This issue increases the risk of inaccurate diagnoses and can have serious implications for patients.

In recent years, medicine has witnessed significant advancements through the utilization of advanced technologies and automated systems to enhance diagnostic accuracy and treatment procedures. Artificial neural networks, as a technique of artificial intelligence, have gained increasing prominence in the healthcare sector and hold the potential for revolutionary changes.

It should be emphasized that hospitals often face financial constraints that prevent them from acquiring expensive technologies or hardware not directly used for healthcare. For this reason, it is essential to utilize computationally inexpensive models that are affordable and easy to implement in the hospital environment. These models should be efficient in the analysis of ECG recordings and
could provide an alternative to more expensive technologies without compromising the quality of diagnosis and patient care.

This work is motivated by a hospital seeking a solution to address the issue of errors that occur in their database of ECG recordings, which can have significant implications for patient diagnosis and subsequent treatment.

In this paper, we focus on the development and evaluation of an automated system for detection of misclassification of ECG recordings using a modern convolutional network architecture (CDIL-CNN [7]) suitable for processing sequential data. Our objective is to minimize the number of patient and recording mix-ups prior to ECG evaluations, which can lead to inaccurate diagnoses and compromised patient care. We believe the system could significantly reduce the number of errors in administrative ECG evaluations and contribute to higher accuracy and reliability of diagnoses.

This work makes significant contributions in several key areas: I. The development of an exceptionally efficient system with low computational requirements allows for practical implementation in resource-constrained settings, specifically benefiting hospitals with limited computational resources. This aspect ensures the model's viability and effectiveness in real-world healthcare environments. II. We achieved improved performance by significantly reducing the number of model parameters 760+ times in comparison with concurrent works. This enhancement increases the model's efficiency without compromising its accuracy, providing a more streamlined and effective solution for ECG analysis. III. Additionally, we present an anonymized version of a real-world dataset specifically curated for this study. This contribution not only advances the field of ECG analysis but also provides a valuable resource for future research in this domain. The availability of this dataset fosters collaboration and facilitates further investigations into improving diagnostic accuracy and patient care. IV. The work further simulates the deployment of the developed model under realistic conditions, replicating its application within a clinical setting. By mimicking real-world scenarios, this simulation provides insights into the model's performance and feasibility in practical use, offering valuable perspectives on its potential impact in improving ECG evaluations and supporting healthcare professionals in their decision-making process.

In summary, this work's main contributions include the utilization of an efficient model for resource-constrained settings, achieving improved performance through parameter reduction, the publication of an anonymized real dataset, and simulating real-world deployment. These advancements offer promising avenues for enhanced ECG analysis, opening doors to more accurate diagnoses and improved patient care.

## 2 Related Work

Early approaches to ECG-based patient identification rely on hand-crafted feature extraction methods (FEMs) completely [1, 47]. Some recent works, while utilizing the representational power of neural networks or other function approximators, still utilize preliminary feature extraction methods [26, 11, 42, 30, 3]. These methods include R-peak detection [44, 27, 45, 15, 32] and segmentation [29], measuring onsets and durations of different beat phases [1, 30, 41], Kalman filter transformation [53] or PCA transformation [3].

FEMs are more resistant to noise in the original signal and therefore less prone to over-fitting. However, the choice and tuning of a specific FEM depends on prior expert domain knowledge and is therefore subjected to possible human error and bias. Moreover, transforming the input signal to a different domain using FEMs is likely to - in addition to noise - remove a part of the information relevant to the task we are trying to solve. Suffice to say, designing and optimizing FEMs automatically is difficult.
With the rise of deep-learning methods and availability of better hardware, researchers have applied deep neural networks to the task of of identifying patients using the ECG signal directly [31, 25, 16, 24]. Such models not only require large amounts of data for training and are prone to over-fitting, but are also sensitive to noise, which commonly occurs in the ECG signal; wandering baseline, interference caused by power lines and other electrical devices, muscle contractions, motion artifacts and other high-frequency noise to name a few [39]. To address the presence of noise, researchers pre-process the signal [34, 36, 54, 23, 43] using various high-pass, low-pass or band-stop/pass filters implemented using Fourier transformations, [50] wavelet transformations [36, 5], or other techniques [31].

Regarding neural network architectures, prior art consists mostly of biometric systems based on convolutional neural networks (CNNs) [31, 10, 17, 16, 37] and recurrent neural networks implemented using long-short term memory cells (LSTMs) [29, 25, 24]. LSTM networks represent a natural solution to signal processing as they are designed specifically to handle long, sequential data [20]. However, data cannot be passed through LSTM networks in parallel due to their sequence-processing nature. That motivates the use of CNNs, whose convolutional filters can be parallelized on GPUs very efficiently. In the context of ECG biometrics, CNNs can be either applied to electrocardiogram images using two-dimensional filters [18, 33, 31] or directly to the input signals using one-dimensional filters [17].

A noticeable feature of many previous works is the size of datasets commonly used for training and evaluation of patient identification systems. Such datasets include the MIT-BIH Normal Sinus Rhythm (NSRDB) and MIT-BIH Arrhythmia (MITDB) [38, 13], CYBHi [8], ECG-ID [35] and PTB [4], which contain tens or hundreds of patients at most. Patient ECGs in small datasets are very unlikely to be representative and independent samples of the whole population either due to biases introduced during the selection of patients to monitor or due to specific features of the devices or processes used to collect ECG signals. Evaluation on such data therefore provides limited proof of a system or a model generalizing well beyond the study's scope.

Furthermore, the patient identification task is generally phrased as a direct multi-class classification problem [37, 31, 17, 25, 24, 12], which may be motivated by the presence of small patient databases. In other words, researchers build models that directly map ECG signals to patients using only their trained parameters. Researchers commonly achieve this by designing neural network architectures whose output is a soft-max layer with one "neuron" for each patient in their dataset. The downside of this approach is the fact that such model - in its full form - is incapable of generalization, because the set of patients it works with is already predetermined by its design. The implications are mostly practical: the model cannot be shared for or among different hospital environments, nor can it handle changes in the set of patients; in that case, it has to be fine-tuned again or retrained from scratch.

Works such as [24] amend this by discarding the soft-max layer when attempting to generalize beyond the known set of patients, and instead think of the network's output at the last-but-one layer as an embedding of the ECG signal in some D-dimensional ECG vector space. We call the embedding an ECG vector. To confirm the identity of a patient whose ECG readings are already known, a new ECG is embedded in the vector space (creating an ECG vector \(p_{a}\)) and compared to the average of their other ECG vectors \(\bar{p_{b}}=\frac{1}{k}\cdot(p_{b1}+p_{b2}+...+p_{bk})\) through cosine similarity. The decision whether the new ECG actually originates from the patient whom \(p_{b1}\) to \(p_{bk}\) belong to is obtained by comparing the result of the cosine similarity to some pre-determined threshold value [24]. Although this configuration appears satisfactory, it's uncertain whether cosine similarity can effectively compare ECG vectors of both familiar and unfamiliar patients, given the added complexity of the soft-max layer in transforming an ECG vector into a patient ID. Furthermore, limiting ourselves to the use of cosine similarity can cause our embedding model to "run out" of space on the hyper-sphere of ECG vectors to allocate to new patients. Even if the model was capable of perfectly embedding
previously unseen patient ECGs, then for any threshold we choose, we can find a number of patients large enough for which the model starts failing to discriminate well between individuals.

Oh et al. [40] have designed a model based on convolutional networks and Transformers [55] for representing electrocardiogram recordings as vectors and then fine-tuned it for patient classification using the ArcFace loss [9]. As this model is trained using metric learning [28], it does not require model retraining after the set of patients changes. However, since their model is quite large, it may be difficult to deploy it in a hardware-constrained environment.

In summary, the issues we have discovered in related work are the following: multi-class patient classification, the use of large models, and evaluation on small datasets. Our study addresses these issues by:

* designing a small model which decides whether two ECG recordings originate from the same individual
* more importantly
- evaluating our model on large public datasets (_cca_. 1000x larger than PTB and others)

## 3 Method

Instead of treating the patient identification task as a multi-class classification problem, we approach the task of patient identification indirectly. Instead of having a neural network classify ECG signals into patient classes, we created a model to decide whether two different ECG signals originate from the same patient or not. Such a model does not require us to know the amount of patients beforehand and the set of patients for identification can change over time without the need to retrain the model.

We then train our model in two phases. In the first phase, an embedding model learns to build the ECG vector space through metric learning [28], which is a technique of building vector representations of inputs based on a metric function, such as the Euclidean distance. In the second phase, the embedding model is augmented by a discriminator head and fine-tuned on ECG pairs. The discriminator head's inputs are two ECG vectors created by the embedding model, and its output is a probability estimate of whether those vectors belong to the same individual.

Finally, to verify that a specific patient indeed owns a new, previously unseen ECG signal, we utilize a database of patients, which are clusters of previously classified ECG vectors (embedding model outputs). First, we convert the signal to an ECG vector using the embedding model and then estimate the probability that the vector belongs to the selected cluster using the discriminator head.

The overview of this whole process is captured by Figure 1. We publish our code-base as a GitHub repository at [https://github.com/CaptainTrojan/electrocardioguard](https://github.com/CaptainTrojan/electrocardioguard) for the purpose of reproducibility.

### Datasets

Before delving into the specifics of our training and evaluation procedures, we would like to introduce the datasets used for this purpose.

Our study makes use of four distinct datasets, one of which connects us to the prior art, PTB [4]. This dataset not only contains 12-lead ECG signals, but is also the largest and most relevant among all datasets mentioned in Section 2.

The remaining three datasets are CODE-15% [46], PTB-XL [56], and a private collection of electrocardiogram data provided by the Institute for Clinical and Experimental Medicine (Prague, Czech Republic), which we refer to as "IKEM" in the context of our study. Table 1 shows their
**(a)** The first phase of training, which creates an ECG embedding model based on convolutional operations. The model maps ECG recordings to ECG vectors, thus describing an ECG vector space. In an ideal state, vectors (points) from the same patient are close to one another and far away from other patients - ECG vectors from the same patient have the same color. This model is described in Section 3.2.

**(b)** The second phase of training, which produces a small discriminator head. It compares two vectors from a pair using linear combinations of weighted distances between them and outputs the probability that the recordings, which were converted to these vectors by the embedding model, originate from the same patient. The discriminator learns to output 1 for positive pairs (same patient) and 0 for negative pairs (different patients). This model is described at the end of Section 3.2.

**(c)** Schema of the intended deployment scenario. The hospital staff captures a new ECG recording (blue) and assigns it to a given patient (red). First, our embedding model converts the ECG signal to an ECG vector. Then, all ECG vectors belonging to the selected patient are loaded from a database. Finally, we use the discriminator head to calculate the likelihood that this is indeed the patient who the new ECG recording belongs to. The procedure is described in detail in Section 3.3, paragraph "Overseer simulation".

**Figure 1:** The overview of our ECG-based patient identification method.
respective sizes. Notably, the newly created IKEM dataset is roughly four times larger than PTB-XL and contains more samples per patient on average, which is very useful for tasks involving intra-patient ECG comparison or matching.

Each dataset contains raw electrocardiogram signals paired with an anonymized unique patient ID. We store the signals as 16-bit integers in HDF5 files with granularity 4.88 \(\mu V\). We also remove redundant augmented leads and lead III, which can be calculated from leads I and II. These efforts to save storage space have resulted in size decrease of roughly 60%, saving more than 50 gigabytes of data in total. When we input the signal to our model, we expand the reduced 8 leads back to the original 12, producing 12 voltage values for each time instance.

All leads, regardless of sampling frequency (400-500 Hz) or length (8-10 seconds) were bidirectionally truncated or padded with zeroes to 4096 voltage measurements over time. In the exceptional case of PTB, which contains approx. 30-100 second long signals sampled at 1000 Hz, we down-sampled the input signal to 500 Hz before applying bidirectional truncation. Hence, all our ECG recordings are matrices of size \((4096,12)\).

The train/dev/test split is applied to the sequence of ECG signals, which means that - in small numbers - patients may be shared across splits.

### Architecture

In the following sections, we describe the training and evaluation procedure captured by Figure 1 in greater detail.

Pre-processingBefore an electrocardiogram recording is input to our embedding model, it is pre-processed. As we have stated in Section 2, the raw ECG signal contains various types of noise. We have implemented a baseline wander (Figure 2) and high-frequency noise removal filters (Figure 2) using _ptwt_ (PyTorch Wavelets, [2]). Regarding power-line interference, both power-lines and ECG signals have varying frequencies across countries and data sources, so although the power-line interference noise may be present in our data as well, we have decided to omit this filter. Finally, we normalize the input signal to z-scores (Figure 2) both in order to stabilize the gradient descent procedure and to calibrate the possibly different scales of electrocardiograms across different recording devices. Each of these transformations is applied to all leads separately and is a part of the embedding model, which makes the system resistant to noisy input.

Embedding modelWe have experimented with two different embedding model architectures: a one-dimensional residual convolutional network (1D-RN, Table 2) [19] and a circular dilated convolutional network (CDIL-CNN, Table 2) [7], which is a novel architecture specifically designed for processing long sequences. These models pose an advantage over LSTM networks, Transformers

\begin{table}
\begin{tabular}{l l l l l} \hline \hline Title & \(\mathcal{N}\) ECGs & \(\mathcal{N}\) patients & Size & Train/dev/test split \\ \hline PTB & 549 & 290 & 69MB & 0/0/100 \\ PTB-XL & 21799 & 18869 & 2.1GB & 0/50/50 \\ IKEM & 98130 & 30290 & 6.3 GB & 0/50/50 \\ CODE-15\% & 345106 & 233479 & 22GB & 70/10/20 \\ \hline \hline \end{tabular}
\end{table}
Table 1: The datasets we use in our study. All datasets were separated into three non-overlapping sections for training, validation, and testing of our methods. The exact numbers can be retrieved using the _dataset_stats.py_ script.
[55] and state-space models [51] as embedding models mostly thanks to their small size and short forward-pass duration. This enables hospitals to deploy our models on readily accessible hardware without the need to invest in high-end dedicated graphical processing units.

Metric learningTo initialize the ECG vector space defined by the embedding model in a way that involves good separation of patients, we examine two deep metric learning approaches: _triplet loss_[48, 21] and _circle loss_[52]. Both loss functions accept three ECG vectors: anchor (A), positive (P), and negative sample (N), where the anchor and positive sample come from one patient and the negative from a different patient. Minimizing these losses corresponds to embedding the anchor and positive samples close to one another and far away from the negative sample. In other words, the goal is to maximize inter-patient distances and minimize intra-patient distances with respect to the individual ECG recordings. Figure 3 helps visualize what the end result might look like.

Discriminator headAfter we finish training the embedding model, then, in order to utilize its ECG vector space initialization, we connect it to a small discriminator _head_ (a neural network whose inputs are the outputs of another neural network, see Figure 3(a)). The joint model resembles a Siamese setting (see Figure 3(b)), where the embedding model creates ECG vectors and the discriminator head decides whether they originate from the same patient.

\begin{table}

\end{table}
Table 2: Summaries of evaluated embedding model structures. For further details regarding architecture or implementation of these models, see the original publications for ResNet [19] and CDIL-CNN [7]. The input ECG signal has 12 voltage values for 4096 samples through time. The output ECG vector dimension is 256.
\[l_{1}(p,q) =\sum_{i=1}^{N}|p_{i}-q_{i}| \tag{1}\] \[l_{2}(p,q) =\sum_{i=1}^{N}(p_{i}-q_{i})^{2}\] (2) \[csim(p,q) =\sum_{i=1}^{N}\frac{p_{i}\cdot q_{i}}{\sqrt{\sum\limits_{j=1}^{N }p_{j}^{2}}\cdot\sqrt{\sum\limits_{j=1}^{N}q_{j}^{2}}} \tag{3}\]

More specifically, the discriminator head's prediction is based on a linear combination of the distances between two input ECG vectors (Equations 1, 2 and 3) that the embedding model creates. In order to let the network adapt to the embedding space's shape, we experiment with swapping the sum in the above formulas for a weighted sum instead whilst training those weights by gradient descent. The result is passed through a sigmoid layer in order to truncate the output to the range \((0,1)\), which is interpreted as probability.

The Siamese model is trained on _pairs_ of ECG signals and learns to predict which pairs belong to the same patient and which belong to different patients, which is a binary classification problem. This model can be trained either end-to-end or with a frozen embedding model (its weights are never updated), so we experiment with both variants.

### Evaluation

Finally, we evaluate our model on two tasks relevant to patient identification. The first task called gallery-probe matching is commonly used for this purpose in the context of identification based on similarity of representations [6, 22, 14, 49], including electrocardiogram-based patient identification [40]. Second, in order to evaluate the applicability of our model for ECG misclassification detection, we have designed a task we call the _overseer simulation_. Both of these tasks are described in this section and our GitHub repository contains a stand-alone script pt_evaluate.py that can be used for evaluating future models under the same conditions.

Gallery-probe matchingThe main objective of this task is to analyze the similarity of ECG vectors obtained from the same patient. To achieve this, we select a sample of N patients from a dataset, ensuring that each patient has at least two different ECG recordings, denoted as A and B. We divide these recordings into two distinct sets: the _gallery_ set, which contains all the A recordings, and the _probe_ set, which contains all the B recordings. Each set contains exactly one recording per patient. Our aim is to identify, for each probe element, the most similar gallery element, with the desired outcome of both elements belonging to the same patient. We measure the fraction of correctly matched pairs and report it as _accuracy_.

Here, the embedding model is first used for converting all \(2N\) ECG recordings to ECG vectors, and the discriminator head \(f_{\theta}(u,v)\) is used to calculate the similarity between members of all \(N^{2}\) pairs.

Overseer simulationIn the context of this task, we maintain a database of ECG vectors where each belongs to a specific patient from a dataset. This database is initialized by inserting at least one ECG vector (created by the embedding model) for N random patients from a dataset. Then,
similarly to the previous task, we select another K random ECG recordings in total from some of those N patients, and aim to assign them correctly to their owners. The differences between this setup and the setup in gallery-probe matching are that both the database (gallery) and the probe set can contain multiple ECG recordings from one patient, and that both sets can have different size: database has at least N recordings, whereas the probe set has exactly K recordings.

Since we attempt to approximate the real use-case of our model, we simulate a hospital staff member whose task is to classify the probe ECG vectors instead of using the model directly. And to replicate the errors observed in practice, our simulated hospital staff member makes a _mistake_ (random patient chosen instead of the correct one) with some small probability \(p\). Our model's role in this scenario is that of an _overseer_, whose task is to detect these mistakes. Furthermore, as the elements from the probe set are classified, they are _inserted_ into the database, influencing the overseer's future decisions. If the staff makes a mistake and the overseer does not detect it, the database becomes corrupted.

In summary, at each step of the overseer simulation, we are provided with an ECG vector and a patient selected by the staff, and our goal is to decide whether it is likely that this vector truly originates from the selected patient.

Our notation for the \(j\)-th ECG vector of patient number \(i\) is \(p_{ij}\). One patient \(P_{i}\), then, is a set of vectors \(p_{i1},\ p_{i2},\...,\ p_{in}\), where \(n\) is the number of vectors originating from this patient. The vector being classified is called \(v\), which is the output of an embedding model as seen in Figure 0(c). We aim to calculate the measure of likelihood of \(v\) originating from the patient \(P\) selected by the staff, called \(l(v,P)\), and if this value is smaller than some likelihood _threshold_, we claim that the overseer has detected a mistake.

This decision should be based on the knowledge learned by the discriminator head \(f_{\theta}(u,v)\). Nevertheless, we can only use the discriminator head when comparing \(v\) to a single vector \(u\), as in the gallery-probe matching task, but comparing \(v\) to a _set_ of vectors \(\{p_{ij}\}\) is not trivial. Therefore, we experiment with several different approaches of using the discriminator to calculate the likelihood, which are described in the remained of this section.

VecAvg : Average of vectorsAn initial approach to calculating \(l(v,P_{i})\) is to replicate the approach by Jyotishi et al. [24], which is to take an average of each vector \(p_{ij}\) and use it as a single representative of \(P_{i}\), as shown in Equation 4.

\[l(v,P_{i}) =f_{\theta}(v,\bar{p_{i}}), \tag{4}\] \[\bar{p_{i}} =\frac{1}{n}\sum_{j=1}^{n}p_{ij}\]

Such approach is however vulnerable to outlier vectors \(p_{ij}\). Consider the case where an ECG signal measurement \(p_{ik}\) is projected very far away from the general proximity of \(P_{i}\)'s other vectors: taking an average may now represent a completely different patient.

DiscAvg : Average of discriminator outputsAnother simple approach is to take the average of the individual discriminator outputs between \(v\) and every other \(p_{ij}\), as shown in Equation 5.

\[l(v,P_{i})=\frac{1}{n}\cdot\sum_{j=1}^{n}f_{\theta}(v,p_{ij}) \tag{5}\]

This approach is more resistant to outliers, as each vector is processed individually and contributes to the overall likelihood independently of others. However, it does not eliminate the threat of outlier
noise completely, as distant vectors still have a large impact on the value of \(l\).

WeightedDiscAvg : Average of discriminator outputs with quality weightingTo amend this issue, we can instead estimate the _quality_ of each \(p_{ij}\) as a representative of \(P_{i}\) and weigh the discriminator outputs by this quality value; see Equation 6.

\[\begin{split} l(v,P_{i})&=\frac{1}{\sum\limits_{j=0}^ {n}q_{j}}\cdot\sum\limits_{j=1}^{n}f_{\theta}(v,p_{ij})*q_{j},\\ q_{j}&=q(p_{ij},P_{i})=\sum\limits_{\begin{subarray} {c}k=1\\ k\neq j\end{subarray}}^{n}f_{\theta}(p_{ij},p_{ik})\end{split} \tag{6}\]

This method guarantees that ECG vectors that the discriminator considers as poor representatives of the patient group \(P_{i}\) will have a smaller impact on the general likelihood measure \(l(v,P_{i})\). Notice that \(q_{j}\) does not have to be normalized itself, because \(l(v,P_{i})\) is normalized by the sum of \(q_{j}\) regardless. Note that the difference between the previous and current approach is not recognizable until the cluster size is at least 3.

A possible downside of this approach is perhaps the fact that it fails to utilize the knowledge that some clusters are coherent (according to the discriminator) and some are not. Although the weighting itself allocates more impact to representative vectors \(p_{ij}\), it does not capture the fact that such a weighting was even necessary.

WeightedConsistency : WeightedDiscAvg with cluster consistency weightingThe main idea behind WeightedConsistency is that we calculate an estimate of a consistency measure \(c_{i}\) of \(P_{i}\) and should think of the consistency as an indicator of how much can we rely on the information provided by the intra-cluster and cluster-to-\(v\) discriminator outputs. The exact formulation is captured by Equation 7, where \(l^{\prime}\) is Equation 6.

\[\begin{split} l(v,P_{i})&=c_{i}\cdot l^{\prime}(v,P _{i}),\\ c_{i}&=\frac{1}{n(n-1)}\sum\limits_{\begin{subarray} {c}p,q\in P_{i}\\ p\neq q\end{subarray}}f_{\theta}(p,q)\end{split} \tag{7}\]

Imagine a situation where we are deciding between two clusters, \(P_{1}\) and \(P_{2}\), all elements in both \(P_{1}\) and \(P_{2}\) are very close to \(v\) according to \(f_{\theta}\), and \(P_{2}\) contains an outlier. If we simply weigh down the influence of the outlier, the likelihoods for \(P_{1}\) and \(P_{2}\) would be roughly equivalent, but the mere fact that \(P_{2}\) even contains an outlier means that the patient's data is inconsistent or mislabeled. In any case, we lose confidence that assigning \(v\) to \(P_{2}\) is the right decision. Therefore, in order to decide to add \(v\) to \(P_{2}\), the values of \(f_{\theta}\) between \(v\) and other cluster elements should be large enough to outweigh this.

Note that normalizing \(c_{i}\) itself inside one cluster is redundant, as any positive (not sign-changing) linear transformation of \(l(v,P)\) is monotonic and therefore does not influence the value of its argument maxima. \(c_{i}\) simply has to be normalized by any scalar multiple of \(\frac{1}{n(n-1)}\) so that it is normalized with respect to the size of \(P_{i}\), as different patients can have different amounts of ECG vectors.
Experiments

For training of our model in both phases, we use the largest dataset available to us - the CODE-15%. Given the fact that the PTB and PTB-XL datasets were collected by the same institute, it is reasonable to assume higher dependence between their samples. Hence, we have opted out of including any part of PTB in the validation set. Since we aim to produce a model that generalizes well, we never train it on data outside of CODE-15% and use that data for verification of its ability to generalize. If we had trained the model using a combination of CODE-15% and other large datasets, we might have obtained better results. However, doing so could have undermined our confidence in the validity of the results, as it would have been difficult to ascertain whether the model wasn't simply adapting to the unique features and systematic noise of those specific three datasets.

The amount of both possible A/P/N _triplets_ and _pairs_ is (due to the ratio between patients and signals) approximately quadratic in terms of the number of patients, making its size impractically large for enumeration. Hence, during training and validation, we _sample_ the sets of triplets and pairs instead of enumerating them sequentially. Sampling of positive (same-patient) and negative (different-patients) pairs is strictly balanced. To ensure deterministic validation, we seed the triplet and pair generators with a fixed value and early-stop the training procedure on the validation loss of CODE-15%. The specific part of the dataset that we sample from is defined by the train/dev/test split.

### Optimal configuration

Since our goal is to find out which configuration generalizes the best, we tune various model hyperparameters and look for the combination that maximizes the _minimal_ AUC measure across all three large datasets. The results of this hyper-parameter search are captured by Tables 3, 4 and 4. Due to the vast range of possible combinations, we have first identified an optimal configuration for the embedding model using a full discriminator without a hidden layer, and then optimized the configuration for the discriminator separately.

Regarding the embedding model, we found that the filtering methods we have experimented with have not been beneficial to the model's performance, especially the baseline wander removal filter, which actually hurts the performance (\(-0.03\) AUC). The CDIL-CNN architecture is already resistant to noise, and thus the filters are largely redundant and may even corrupt the original signal in special occasions (for example when iterative filters, such as our wavelet-based BWR, fail to converge).

It is also impossible to say whether freezing the embedding model after the first stage helps or not; it would certainly be beneficial to freeze it if overall training times were a major factor. We have trained our models on various GPU cards in the Metacentrum cloud computing grid service and the overall training time never surpassed 18 hours. The same goes for the model's embedding size, which has turned out to be an insignificant parameter in our setting. We have settled on 256, but reducing the size to 128 should have negligible impact on the system's performance while reducing the required storage space for patient vectors by half.

Note the large value of \(\Delta_{AUC}\) for the discriminator \(l_{1}\) distance parameter. It shows that not letting the model adjust the distance member weights - that is, literally computing the distance between the input ECG vectors - significantly hurts its performance. We can also see that it is likely not optimal to incorporate further distance metrics into the discriminator head, as that causes the model to over-fit slightly and worsen its generalization capabilities. However, this claim is not backed by statistical testing, as we fail to reject the hypothesis that the best setting should exclude the cosine distance completely under confidence level 95%. But even under the assumption that the discriminator indeed should use only a single distance measure, in Table 4, we can see that \(l_{1}\) is not
clearly superior to \(l_{2}\) in this sense.

In an attempt to further regularize the model and thus cause it to generalize better, we have

\begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**DCOS** & **DHS** & **DL1** & **DL2** & **min AUC** & **SEM** \\ \hline merge & 16 & full & merge & 0.958763 & 0.001611 \\ exclude & 16 & exclude & full & 0.959258 & 0.001055 \\ merge & 16 & full & exclude & 0.959435 & 0.001657 \\ full & 16 & full & exclude & 0.959533 & 0.000492 \\ exclude & 16 & full & exclude & 0.961165 & 0.001002 \\ \hline \hline \end{tabular}
\end{table}
Table 4: The best 5 configurations of both models. SEM stands for Standard Error of the Mean = \(\frac{\sigma}{\sqrt{n}}\), which can be used for statistical testing. All discriminator model settings use the best embedder model hyper-parameters. The full table can be found at our GitHub repository.

\begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Hyper-parameter** & **Options** & **Best** & \(\Delta_{AUC}\) & **p-value** \\ \hline Model & 1D-RN, CDIL & CDIL & 0.013 & 1e-8 \\ Embedding loss (EL) & triplet, circle & triplet & 0.016 & 3e-6 \\ Embedding size (ES) & 128, 256, 384, 512 & 256 & 0.002 & 0.151 \\ Embedding fine-tuning (EF) & freeze, end-to-end & end-to-end & 0.004 & 0.037 \\ Normalization (NORM) & apply, exclude & apply & 0.031 & 3e-8 \\ Baseline wander removal (BWR) & apply, exclude & exclude & 0.026 & 4e-10 \\ High-frequency noise removal (HFNR) & apply, exclude & apply & 0.001 & 0.322 \\ \hline Discriminator hidden size (DHS) & 0 (exclude), 16 & 16 & 0.007 & 1e-3 \\ Discriminator \(l_{1}\) distance (DL1) & exclude, merge, full & full & 0.412 & 2e-7 \\ Discriminator \(l_{2}\) distance (DL2) & exclude, merge, full & exclude & 0.002 & 0.044 \\ Discriminator \(cos\) distance (DCOS) & exclude, merge, full & exclude & 0.001 & 0.086 \\ \hline \hline \end{tabular}
\end{table}
Table 3: All examined hyper-parameters and their options. Embedding fine-tuning represents a setting where, in the Siamese model, the embedding model is either frozen or allowed to be trained end-to-end during the second phase of training. The last column, \(\Delta_{AUC}\), is the difference in performance between the best configuration and the best alternate configuration across all other options for each specific hyper-parameter. For example, the best configuration’s performance average is better by 0.031 (AUC) than the same configuration but without normalization. The p-value is that of a one-tailed t test between those corresponding hyper-parameter setting experiments, where \(H_{A}\) is that the best configuration is better than the alternative one. All models were trained using the Adam optimizer with learning rate 0.001, without any LR scheduler or weight regularization.
shortly experimented with shuffling the electrocardiogram leads and adding a small amount of Gaussian noise to them. However, these techniques have led to a significant decrease of performance (-0.02 to -0.06 AUC) and have thus been abandoned.

### Evaluation

Finally, we measure the model's performance on the two evaluation tasks: gallery-probe matching, and overseer simulation. The results are shown in Table 6.

For gallery-probe matching, we have selected a random sample of patients with at least two recordings from the test set of each respective dataset. The exception is PTB-XL, where the gallery and probe sets were built from the whole dataset like in the study by Oh et al. [40] for the purpose of meaningful and fair comparison. It should be noted that although the optimal model configuration was selected based on the minimum dev set performance across all three large datasets, the minimum performance was never that of PTB-XL, which allows us to perform gallery-probe evaluation on the entirety of PTB-XL without compromising the validity of our results. The sample size is included in the aforementioned table.

For overseer simulation, we select 10000 patients to initialize the database with and 1000 further electrocardiograms to classify (probe) under staff mistake rate 2%, meaning that there are 20 mistakes to detect among 980 correct classifications. The mistake rate of 2% is our best estimate of the real mistake rate occurring in practice, as it is the difference in pairwise accuracy of our model on PTB-XL and IKEM. Due to the size requirements, PTB was excluded from the experiments, as it contains only 290 patients, and the number of initial patients for PTB-XL was only \(\approx 9400\).

Since we find out that weighting the discriminator outputs by representational quality (WeightedDiscAvg ) performs the best (Table 5), we consequently report results that were obtained under this approach. We hypothesize that the drop of performance in WeightedConsistency compared to WeightedDiscAvg is caused by this approach being biased towards clusters of size 1 (where consistency is maximized) regardless of whether they are a good fit for the classified ECG vectors or not.

Notably, we can see that the evaluation tasks are much harder than the training objective due to the vast amount of ECG vector pairs the model must correctly recognize as true negatives. During pairwise training and evaluation of the discriminator, positive (same patient) and negative pairs (different patients) are sampled in a balanced manner. However, in the gallery-probe matching task, the discriminator must assign a probability value to a single positive pair that is higher than those of thousands of other negative pairs, which are created between the probe element and the entire gallery. The situation is similar in the overseer simulation task except that there are a few more positive pairs.

Our efforts to minimize the difference in accuracy in the pairwise task across all datasets have resulted in a model with the highest degree of generalization across all three datasets. However, we can notice that even a small decrease in pairwise accuracy has a significant impact on performance in the evaluation tasks, which are much harder. It follows that, for optimal performance in production, institutions should invest in fine-tuning our model on their specific datasets using our published version as a robust starting point instead. In such cases, according to Table 6, if we wanted to detect

\begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Approach** & **VecAvg** & **DiscAvg** & **WeightedDiscAvg** & **WeightedConsistency** \\ \hline P@R95 (median) & 3.66\% & 18.35\% & **22.35\%** & 7.14\% \\ \hline \hline \end{tabular}
\end{table}
Table 5: Comparison of likelihood calculation approaches in the overseer simulation task aggregated across all datasets (CODE-15%, IKEM, PTB-XL) and mistake rates (50%, 5%, 2%, 1%).
19 out of 20 mistakes made in 1000 classifications, our model would cause only 1 false alarm in roughly 23 classifications, which the hospital staff would have to dismiss.

Our model's capacity for generalization is also demonstrated by its performance in gallery-probe matching and overseer simulation (F1) on PTB-XL. Despite never encountering a single example from PTB-XL, our model achieved comparable performance to CODE-15%, which encompasses the entire training dataset. The significant performance drop observed in IKEM could be caused by its different patient-to-recording ratio (roughly twice as much recordings per patient on average, see Table 1) or noisier labels.

Furthermore, our model's performance in the gallery-probe task on PTB-XL matches the state of the art set by Oh et al. [40] (12 leads, best variant), but using a significantly smaller number of parameters. It is difficult to specify precisely how much smaller our model is without knowing the exact number of parameters of the state of the art model, but seeing that it contains BERT-base (110M parameters) without the embedding layer (24M parameters) and includes additional convolutional layers, we can determine that our model is at least 760 times smaller, if not more. Our model even surpasses the state of the art results by 0.6%, but this small improvement could be attributed to the fact that the current version of PTB-XL (1.0.3) contains 16 less patients with at least two recordings than the older version used by Oh et al.

## 5 Discussion

In summary, our work has contributed to the field of automatic electrocardiogram processing and patient identification in multiple ways.

First, we publish a large brand-new electrocardiogram patient identification dataset with many recordings per patient on average, which is suitable for training and evaluation of models based on electrocardiogram representations. The dataset is in the stage of negotiating legal terms for publication. Once the legal arrangements are completed, it will be located in the datasets folder of our public GitHub project.

\begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**Task** & **Metric** & **Dataset** & & & \\  & & CODE-15\% & IKEM & PTB-XL & PTB \\ \hline \hline pairwise (training objective) & AUROC & 0.990 & 0.971 & 0.984 & 0.982 \\  & Accuracy & 95.8\% & 92.1\% & 94.3\% & 93.1\% \\ \hline gallery-probe & Accuracy & 60.3\% & 46.0\% & 58.3\% & 77.0\% \\  & Sample size & 2127 & 2127 & 2111 & 113 \\ \hline overseer simulation & P@R95 & \(0.28\pm 0.09\) & \(0.08\pm 0.02\) & \(0.17\pm 0.08\) & – \\  & F1 & \(0.59\pm 0.04\) & \(0.39\pm 0.06\) & \(0.56\pm 0.06\) & – \\  & CR & \(45\%\pm 6\%\) & \(7\%\pm 3\%\) & \(19\%\pm 5\%\) & – \\ \hline \hline \end{tabular}
\end{table}
Table 6: Overview of the results obtained on test sets across all datasets and evaluation tasks, including the original training objective. In the context of pairwise matching, the decision threshold is optimized for maximizing accuracy on the dev set. For overseer simulation, we report 95% confidence intervals of the mistake detection rates, where P@R95 stands for precision at recall 95, F1 is the F-measure between precision and recall under threshold obtained by achieving recall 95 on the dev set, and CR is the fraction of corrected mistakes (the patient with highest likelihood was the real owner) out of all detected mistakes.
Second, we publish a tiny (700 kB) neural-network model utilizing state-of-the-art techniques for sequence processing capable of deciding whether two ECG recordings originate from the same individual. We further show how can our model be used for misclassification detection by employing it in a streamed clustering process, and evaluate it in a simulation mimicking the real application. Both the simulation and various clustering approach implementations are published as a stand-alone part of our GitHub project. This was the primary goal of our study, which was primarily motivated by the Institute for Clinical and Experimental Medicine's sponsorship, as they are actively preparing to implement our model in their production environment.

Our model demonstrates applicability not only in the intended scenario but also in a variety of other cases. For instance, our model can be employed to address the issue of fixing existing databases, rectifying inconsistencies and improving data quality. Moreover, it provides a solution for situations where a patient's record is missing within a specific time frame, allowing us to identify the inconsistency by cross-referencing the date of addition and determining which 100 patients were present on that day. Another valuable application is the ability to detect potential patient swaps that may occur on a particular day, which is a task similar to gallery-probe matching, but much easier in difficulty. This feature can be especially useful in quickly identifying and resolving any mix-ups, thanks to the capabilities of our system. Lastly, our model can address challenges encountered in remote areas where doctors may not have access to their own electrocardiogram machines. By verifying the integrity of electrocardiogram measurements obtained through external requests, it helps mitigate the risk of local mixing of recordings.

## Acknowledgement

Computational resources were supplied by the project "e-Infrastruktura CZ" (e-INFRA LM2018140) provided within the program Projects of Large Research, Development and Innovations Infrastructures.

This work has been supported by Grant No. SGS-2022-016 Advanced methods of data processing and analysis.

## Conflicts of interest

The authors declare the following potential conflicts of interest:

Michal Sejak and David Zahour received funding from the Institute for Clinical and Experimental Medicine (IKEM) to conduct this research project on detection of patient misclassifications using electrocardiogram recordings. IKEM provided financial support for the research and covered data collection. The funding from IKEM did not involve any restrictions on study design, data analysis, or result interpretation.

It is important to note that despite the funding received from IKEM, the research was conducted independently and the authors maintained full control over the study design, data analysis, and decision to publish. The authors affirm that the study was conducted with scientific rigor, objectivity, and integrity, adhering to established research protocols and methodologies. The evaluation of the model was performed using publicly available datasets, ensuring transparency and minimizing potential biases.

We would like to acknowledge the support provided by IKEM and express our gratitude for their financial assistance in conducting this research. However, the funders had no role in the study design, data analysis or pre-processing, manuscript preparation, or decision to submit for publication.

Please note that all authors have reviewed and approved the contents of this disclosure.