Pulsar Classification: Comparing Quantum Convolutional Neural Networks
  and Quantum Support Vector Machines
Well-known quantum machine learning techniques, namely quantum kernel
assisted support vector machines (QSVMs) and quantum convolutional neural
networks (QCNNs), are applied to the binary classification of pulsars. In this
comparitive study it is illustrated with simulations that both quantum methods
successfully achieve effective classification of the HTRU-2 data set that
connects pulsar class labels to eight separate features. QCNNs outperform the
QSVMs with respect to time taken to train and predict, however, if the current
NISQ era devices are considered and noise included in the comparison, then
QSVMs are preferred. QSVMs also perform better overall compared to QCNNs when
performance metrics are used to evaluate both methods. Classical methods are
also implemented to serve as benchmark for comparison with the quantum
approaches.
Donovan Slabbert, Matt Lourens, Francesco Petruccione
Pulsar Classification: Comparing Quantum Convolutional Neural Networks and Quantum Support Vector Machines

###### Abstract

Well-known quantum machine learning techniques, namely quantum kernel assisted support vector machines (QSVMs) and quantum convolutional neural networks (QCNNs), are applied to the binary classification of pulsars. In this comparitive study it is illustrated with simulations that both quantum methods successfully achieve effective classification of the HTRU-2 data set that connects pulsar class labels to eight separate features. QCNNs outperform the QSVMs with respect to time taken to train and predict, however, if the current NISQ era devices are considered and noise included in the comparison, then QSVMs are preferred. QSVMs also perform better overall compared to QCNNs when performance metrics are used to evaluate both methods. Classical methods are also implemented to serve as benchmark for comparison with the quantum approaches.

## I Introduction

In the field of astronomy, where vast amounts of observational data of various types of astronomical objects are collected annually, cataloging these objects into their respective categories is crucial. This comparative study focuses on pulsars, which are a type of neutron star formed when a massive star reaches the end of its life, collapsing under gravity but not enough to overcome neutron degeneracy pressure [1, 2, 3]. This results in a dense star remnant with a significantly smaller radius and rapid rotation, often on the order of seconds to milliseconds [4]. Pulsars possess strong magnetic fields that emit particle beams from their poles. If Earth happens to intercept these beams we observe periodic pulses of observable signals, typically measured using radio telescopes. These neutron stars exhibiting this periodic signal are known as pulsars.

Identifying pulsars is essential for distinguishing them from typical main sequence stars and other neutron stars, as binary systems consisting of two pulsars can generate gravitational waves [5, 6]. Pulsars' precise rotational periods also make them useful for detecting gravitational waves passing between them and Earth, by observing deviations in their pulses [7, 8, 9]. This capability enhances our ability to study gravitational waves.

With the increasing volume of astronomical data due to improving observational technology, traditional software will eventually struggle to manage and process it effectively. The Square Kilometer Array (SKA), set to launch soon, is expected to produce data on the order of exabytes (\(10^{18}\) bytes) [10]. Real-time data analysis and efficient classification tasks are the ultimate goals in astronomy and data science. While current classical classification methods suffice for now, the problem will become insurmountable as data sizes continue to grow.

It is for this reason that machine learning has been thoroughly explored as a solution to this challenge. Classical machine learning has been applied to pulsar classification [11, 12, 13, 14, 15, 16, 17], however, since classical approaches face limitations with increasing data sizes [18], the problem might have to be approached with a different perspective. Quantum computing (QC), particularly quantum machine learning (QML) that combines the computational power of quantum computers with machine learning, is such a new perspective. We endeavour to explore the applicability of quantum computers to the machine learning task of classifying pulsars. We do this by comparing two quantum approaches. Quantum methods (one of which is also called a QSVM) have been used on the HTRU-2 data set before [19]. Another known quantum approach [20] uses a circuit with a depth that would introduce a lot of noise.

We improve over the accuracy presented in this case [20] with shorter depth circuits. Our study uses QML approaches that have circuits with lower depths to effectively and accurately classify pulsars using the real-world
HTRU-2 data set using noise free and noisy simulated quantum algorithms and in a limited implementation on currently available NISQ devices. The HTRU-2 data has been manually created by experts from a large number of pulsar candidates [21]. This means that the problem falls under the machine learning category of supervised machine learning [22; 23] in that we have a ground truth to use in training, but this ground truth was decided on by human experts. There are eight features present in the data set for each candidate. Each of the eight features are encoded into a quantum state using rotational encoding. The quantum approaches under consideration are a basic 8-qubit implementation of a quantum-assisted support vector machine (QSVM) and an 8-qubit quantum convolutional neural network (QCNN). Both approaches will be compared with each other as well as to their classical counterparts.

The two methods implemented are variants of commonly used QML archetypes, namely quantum kernel methods and variational quantum algorithms. The goal is to investigate which is currently more suited for real-world binary classification and the variants of choice to investigate are the QSVM and QCNN. In [24] the assertion is made that variational quantum algorithms can also be considered to be quantum kernel methods, but the difference in this variational approach is the use of hierarchical ideas inspired by classical convolutional neural networks and the training procedure. The QSVM evaluates the kernel using a quantum device, followed by fitting a classical SVM, whereas the QCNN relies on a classical optimization loop to optimize parameters used in its parameterized circuit. There are theoretical guarantees associated with kernel methods [24] and it is these guarantees that we want to compare the QCNN with using its more manipulable quantum circuit.

We show below how the QCNN, despite its larger circuit, outperforms the QSVM when considering the time taken to train the model and to use it for prediction. Using standard confusion matrices [25] and evaluation metrics [26] the best method is not as apparent. The QSVM generally performs slightly better depending on the metric used, however, of the two methods the QSVM avoids the most false negatives. In this discovery-focused use case, avoiding false negatives is really important. A short noise test illustrates the obvious point that longer quantum circuits undergo more noise. This is because there is an error associated with each quantum logic gate applied [27]. With noise included in the comparison, the QSVM performs better because of its shorter depth.

The rest of the paper is structured as follows: Section 2 is an explanation of binary classification as a supervised machine learning task. It also contains the necessary information regarding the two methods that are implemented. Section 3 is an explanation of how the HTRU-2 data set is used and how data preprocessing was performed. Here the specific methodology used for both approaches are also discussed Section 4 serves as a comprehensive comparison between the two methods. This includes noise free and noisy simulated results as well as limited real quantum device results. Section 5 is a short conclusion discussion.

## II Theory

### Binary Classification

Binary classification is a supervised machine learning problem where samples in a data set are categorized into one of two classes, in this case as being a pulsar or not. The following is the formal problem statement for binary classification [28]:

\[f_{\theta}:\mathbf{X}\rightarrow\mathbf{Y},f_{\theta}(\vec{x})=y,\vec{x}\in\mathbf{X} \:\text{and}\:y\in\mathbf{Y}, \tag{1}\]

where \(\mathbf{X}\) and \(\mathbf{Y}\) are the input and output domain respectively, \(x\) is a \(n\)-dimensional input vector (also called a feature vector) that contains \(n\) entries or features, and \(y\) is a class label from the set \(\{0,1\}\), where a label of \(1\) indicates the positive class (pulsars) and \(0\) the negative class. The function \(f_{\hat{\theta}}\) is the optimal model from a model family, or set of models \(\{f_{\theta}\}\), and the subscript \(\theta\) indicates that this function has a set of parameters that need to be optimised during training.

Training involves the use of a subset of both \(\mathbf{X}\) and \(\mathbf{Y}\), called the training set \(\mathbf{X}_{\text{train}}\) and \(\mathbf{Y}_{\text{train}}\), during an optimisation or fitting process to find the best model from a model family that predicts the testing set \(\mathbf{X}_{\text{test}}\) and \(\mathbf{Y}_{\text{test}}\) labels as accurately as possible. Based on the predictions made, cross-referenced with the original labels, the output can be formulated as a confusion matrix, which is an \(2\times 2\) array containing the prediction outcomes. Confusion matrices are useful for assessing how well a model predicts on new samples by visually representing the distribution of predictions. Once a model has been trained that performs at a level that is required, the model can be used to predict on previously unseen data, where no prior knowledge of the labels are possible.

Various metrics are chosen to quantify how well a model performs at prediction and are calculated using values extracted from the confusion matrices. In our case it is more important to classify possible pulsar candidates instead of finding true non-pulsars. This is to aid the discovery of new pulsars, which implies that true positives, false positives and false negatives are all of interest, since both the false positives and false negatives still have a probability of being a pulsar. It is, however, important to minimize the amount of false negatives, since the cost of misclassifying a pulsar is larger than misclassifying a non-pulsar. This ties to the fact that all positive predictions are double checked after prediction. The true negatives are of the least amount of interest. Eight metrics are included and are defined in Table 1[29; 26].

Accuracy is a metric that measures the proportion of correctly classified samples when compared to the known
labels. The recall, also known as sensitivity, is a measure of the proportion of positive cases being identified as positive. Specificity is equivalent to recall, but for the negative case. Precision measures how many of the positively identified samples were in fact actual positives and the negative prediction value (NPV) does the same for the negative predictions. The higher both the recall and precision are, the better the model is at predicting the positive class. Three more metrics are included that are commonly used: balanced accuracy, geometric mean, and informedness. Balanced accuracy is the average of recall and specificity, supplying a way to move away from a possibly inflated accuracy value if imbalanced testing sets lead to biased prediction. Geometric mean also compensates for imbalanced data, but the focus here is placed on the training set specifically. Our training sets will be balanced, but we include geometric mean for completeness. Finally informedness provides another measure of how well the model predicts both the positive and negative classes. The best model would be the one that maximizes performance targeted at the positive class (recall and precision), while avoiding as many false negatives as possible (NPV).

### Quantum Kernel Assisted Support Vector Machine

Support vector machines are designed as maximal margin classifiers where the data is classified into two classes by a linear decision boundary [30], after calculating a kernel function value, which is simply a chosen distance measure between two feature vectors. Depending on the dimension of the problem, this separating boundary is usually a separating hyperplane and is found by specifying two parameters: the normal vector of the plane and a scalar number, which together determine the offset of the plane. The margin is the distance between the closest two data points from opposing classes and the support vectors are the most important data points used when determining the hyperplane as they are the closest points to the plane and have the highest impact on how the plane should be oriented. This assumes that the data is linearly separable in the \(n\)-dimensional input space. If linear separability is impossible, then a feature map \(\phi(\vec{x})\) can be used to map or embed the feature vectors in input space \(\mathbf{X}\), into a higher-dimensional feature space \(\mathbf{F}\) where it may become linearly separable. In feature space, the kernel function is typically an inner product of two mapped feature vectors and because of the kernel trick [31, 32], only the kernel function needs be evaluated, instead of processing the individual vectors in feature space.

The quantum version of SVMs uses data embedding to encode the data features into quantum mechanical states that are simulable on quantum devices [33, 34, 35]. In this way the feature map maps the data into a feature space that is similar to Hilbert space, where the Hilbert-Schmidt inner product can be used [24]. This implies that the feature map in this case, maps real data to quantum mechanical states described by complex density matrices. The new feature map can be described as [24]:

\[\phi(\vec{x})=\ket{S(\vec{x})}\bra{S(\vec{x})}=\rho(\vec{x}) \tag{2}\]

where the feature map \(\phi(\vec{x})\) is the outer product of a data encoded feature vector \(S(\vec{x})\) with itself. Quantum kernel methods, of which quantum support vector machines are an example, can be split into two steps: data encoding, which can be done in a multitude of ways [28] (amplitude encoding, angle embedding etc.) and measurement, which can be understood as a projective operation using some observable matrix \(\hat{O}\).

Quantum kernel methods use the inner product in feature space, as a distance measure between two feature vectors, to train the model based on these values by finding a linear decision boundary that separates two classes from one another. Stated more explicitly, training here means that the optimal measurement must be found as it determines the decision boundary. The inner product of quantum embedded states is the mutual overlap. This similarity measure is now called the quantum kernel. A matrix of pairwise inner product values can be calculated by calculating the kernel value for each combination of data points. The pairwise inner product matrix calculated for two sets of identical feature vectors results in a matrix called the kernel gram matrix. The kernel matrix in this case is calculated in feature space, meaning that quantum kernels are functions of the form:

\[K(\vec{x},\vec{x}^{\prime})=|\bra{\phi(\vec{x})}\hat{O}\ket{\phi(\vec{x}^{ \prime})}|^{2}=\operatorname{tr}\left[\rho(\vec{x})\hat{O}\rho(\vec{x}^{ \prime})\right], \tag{3}\]

where \(\vec{x}\) and \(\vec{x}^{\prime}\) are two separate \(n\)-dimensional input vectors, each with \(n\) features, \(\phi\) is the feature map determined by the data embedding and \(\hat{O}\) is any observable

\begin{table}
\begin{tabular}{l c} \hline \hline
**Metric** & **Equation** \\ \hline Accuracy & \(\frac{\operatorname{TP+TN}}{\operatorname{TP+TN+FP+FN}}\) \\ Recall & \(\frac{\operatorname{TP}}{\operatorname{TP+FN}}\) \\ Specificity & \(\frac{\operatorname{TN}}{\operatorname{TN+FP}}\) \\ Precision & \(\frac{\operatorname{TP}}{\operatorname{TP+FP}}\) \\ Negative Prediction Value & \(\frac{\operatorname{TN}}{\operatorname{TN+FN}}\) \\ Balanced Accuracy & \(\frac{1}{2}(\operatorname{Recall+Specificity})\) \\ Geometric Mean & \(\sqrt{\operatorname{Recall\cdot Specificity}}\) \\ Informedness & \(\operatorname{Recall+Specificity-1}\) \\ \hline \hline \end{tabular}
\end{table}
Table 1: The expressions for the evaluation metrics used. TP, TN, FP, and FN represent the true positives, true negatives, false positives, and false negatives respectively after using a binary classification model on a testing set. These values form part of confusion matrices.
that can be omitted if the only the overlap is to be calculated. The resultant kernel function is then used for fitting a classical SVM to the kernel gram matrix for training.

The quantum kernel is used in place of a classical kernel function when the classical SVM is fitted to the training data. This implies that only the kernel matrix, or rather its elements, are calculated using the quantum device. The kernel only depends on the data encoding, which implies that the data encoding defines the minima used in training [24]. We will call this method a QSVM from now on.

A major drawback to QSVMs is the fact that in order to calculate the full kernel matrix, a quadratically scalable number of entries must be calculated [24]. One entry for every pair of input feature vectors exists. The hope is that a classically intractable quantum kernel can be found that will improve classification accuracy when compared to classical kernel functions. There is also the hope that an improvement with respect to the runtime can be found, but this might prove difficult with how long quantum circuit evaluations generally take. It is also an added benefit that SVMs generally do not suffer from barren plateaus [24].

### Quantum Convolutional Neural Networks

Convolutional neural networks (CNN) are commonly used architectures for image-related machine learning tasks, such as image recognition, image segmentation, and object detection [36, 37]. Generic CNN architectures consist of interconnected layers, including input, output, and hidden layers, with activation functions determining neuron activation. This is arranged in a graph-like configuration where the edges between nodes indicate connection weights that determine how strongly certain nodes are linked. Training involves passing data through the network, calculating a cost function, and using gradient descent to update weights in back propagation during each epoch or iteration. CNNs have three key components: convolutional layers, pooling layers, and fully connected layers. The fully connected layers operate as a standard neural network. Convolutional layers perform matrix multiplication to learn spatial patterns and pooling layers reduce spatial dimensions to reduce computational costs.

While there are many proposals for the quantum analogue of convolutional neural networks [38, 39, 40], we focus on the framework proposed by Cong et al. [41] and follow the design methodology of Lourens et al. [42]. Here the QCNN implements analogous convolution and pooling operations in a quantum circuit setting. The key components being weight sharing, sequential reduction of system size via pooling, and translational invariance in convolution steps. These operations are applied on a circuit architectural level, where a convolution consists of unitary operations (also called mappings) applied to all available qubits in a given layer. Identical unitaries are utilised, enabling translational invariance and the sharing of weights. The choice of unitary is arbitrary, but should be chosen in such a way to minimize circuit-based noise, while retaining the ability to learn patterns in the data.

Pooling consists of using a portion of the available qubits as control qubits and applying controlled rotations on the targets. This leads to a reduction in system size and allows the number of parameters to scale logarithmically with circuit depth. Pooling introduces non-linearity to the model while also reducing its computational overhead and is analogous to coarse-graining in the classical case. Information is lost, but by connecting two qubits through a separate unitary mapping, in our case a simple CNOT gate, some information is retained for use in classification.

Convolution and pooling operations are repeated until the system size is sufficiently small, in our case when there is one solitary qubit remaining. Circuit architecture may be changed to find possible relationships in the data, such as changing the solitary qubit. We show an example QCNN in Figure 3 along with it's corresponding directed graph representation [42]. In the digraph representation, nodes correspond to qubits and edges to unitaries applied between them. This representation is useful for defining hyperparameters such as strides, steps and offsets that are used to optimise circuit architecture. One advantage of the QCNN is its flexibility in choice of architecture and Lourens et al. [42] showed how changing the layout of unitaries on the circuit can greatly improve model performance. Finally, all variational circuits require a data encoding step, just like for kernel methods. The choice of data encoding is really important for the effectiveness of variational quantum circuits [43].

## III Methodology

### The HTRU-2 data set and Feature Selection

The HTRU-2 data set used for this comparitive study has already been formatted for use in machine learning and consists of 8 features or inputs per candidate sample with a singular classification label of either 0 or 1, where 1 is the label of the positive class (a pulsar). There are eight features that can be grouped into two groups of four, where each of the features are statistical elements of a candidate's integrated field and DM-SNR curves. The four statistical elements of these two properties are the mean, standard deviation, excess kurtosis, and skewness. The data set contains 16259 pulsar candidates, where 1639 have been confirmed to be pulsars manually by experts. To read more about what the integrated field and DM-SNR curves are and why they are important for pulsar classification see [21, 44].

The features were all normalized to be between 0 and \(\pi\) radians. This is done because of the rotational encoding
strategy chosen. The full data set was split in two at a ratio of 70:30. A specified number of samples from 70% of the data set, in this case a balanced number of positive and negative class candidates, were sampled randomly and used as training sets. Another set of samples was also created by randomly sampling from the remaining 30%. This set was used as the testing set. This allows for creating training and testing sets of any size if necessary.

A specific case of batching the training set for the QCNN was done as follows: from the balanced training sets already created through random sampling, more random sampling was used to create balanced batches of 10 candidates each that were used in each epoch. The way in which this was done, allowed for sample overlap, meaning some batches might have duplicate entries.

### QSVM Circuit

We kept the quantum circuit depth as low as possible to circumvent decoherence noise. The quantum circuit used is an extension of the circuit used in [45]. The features were embedded as the phase arguments of Y-gates applied to each of the 8 qubits, one for each feature. This is called angle embedding and is mathematically represented by applying a Y-rotation gate to the initial state \(\ket{0}\), where the parameter specifying by how much the rotation should be applied is a specific feature from a feature vector. The rotation operator can be written as \(R_{Y}(x_{i})=e^{-i\frac{x_{i}}{2}\sigma_{Y}}\), where \(\sigma_{Y}\) is one of the Pauli matrices.

\[R_{Y}(x_{i})\ket{0}, \tag{4}\]

where \(R_{Y}(x_{i})\) is the Y-rotation gate that is applied to one of the 8 qubits. Each qubit was rotated by a Y-rotation of \(x_{i}\), where the argument was the \(i^{th}\) entry to a feature vector. Keeping to the simplistic design a second set of gates, the complex conjugate of the angle embedding this time only with another set of 8 features from a separate feature vector, was applied to the 8 qubits.

\[R_{Y}^{\dagger}(x_{i}^{\prime})R_{Y}(x_{i})\ket{0}=e^{-i(\frac{x_{i}-x_{i}^{ \prime}}{2})\sigma_{Y}}\ket{0}, \tag{5}\]

where we now have a term, \(x_{i}-x_{i}^{\prime}\), referencing the difference between two features from two separate feature vectors. This difference forms part of the distance measure required by SVMs. The combined effect of the two sets of Y-rotation gates can be written as unitary operators \(S(\vec{x})\) and \(S^{\dagger}(\vec{x}^{\prime})\) and their operation on the 8-qubit initial state can be written as:

\[S^{\dagger}(\vec{x}^{\prime})S(\vec{x})\ket{00\dots 0}, \tag{6}\]

where \(\ket{00\dots 0}\) is the 8-qubit initial state vector. Afterwards a projective measurement was taken by projecting the state down to the 8-qubit initial state's density matrix, \(\rho=\ket{00\dots 0}\bra{00\dots 0}\). In other words the density matrix serves as part of a projector matrix before measurements are done in the computational basis. This step forms the final part of calculating the inner product. Without it the measurement would only produce the state after two rotations. This has the effect of calculating the inner product in the computational basis without performing a SWAP test [45] and becomes the kernel function that will be used along with fitting a classical support vector machine to the training data for prediction. This can be seen in the following:

\[\bra{00\dots 0}S(\vec{x}^{\prime})S^{\dagger}(\vec{x})\hat{O}S^{\dagger}( \vec{x}^{\prime})S(\vec{x})\ket{00\dots 0}, \tag{7}\]

where \(\bra{00\dots 0}S(\vec{x}^{\prime})S^{\dagger}(\vec{x})\hat{O}\) is the projector used and a final expression can be found:

\[\ket{\bra{00\dots 0}S^{\dagger}(\vec{x}^{\prime})S(\vec{x})\ket{00\dots 0}}^{2}, \tag{8}\]

that is just the kernel discussed before. The above was done for the entire training set. The consequence of how kernel values are calculated (one value exists for each possible pair of features from two separate feature vectors) the scaling of this method is quadratic. This implies that the kernel will become classical intractable at some point. Therefore, since the kernel function is hard to simulate classically, it is the hope that an actual quantum device would be able to calculate it and reap the benefits of it if there are any. We estimate the training time of the training step of the algorithm on a real device by measuring the time it takes for one circuit execution to run and then multiplying this runtime with the total number of circuit executions. This would extrapolate the training time for more circuit executions. From our perspective, runtime

Figure 1: A correlation matrix showing the correlation coefficients between each possible pair of the features in the HTRU-2 data set.
is the time it takes the model to train on the training set. The predicting time is not irrelevant though, as for QSVMs it scales with the size of the training and testing set. In other words to estimate the training time we use:

\[\Delta t_{\text{Training}}=n_{\text{CE}}\times\Delta t_{\text{CE}}, \tag{9}\]

where \(\Delta t_{\text{Training}}\) is the total training time, \(n_{\text{CE}}\) the number of circuit executions and \(\Delta t_{\text{CE}}\) is the training time of one circuit execution. The total number of circuit executions can be calculated using:

\[n_{\text{CE}}=n_{\text{train}}^{2}, \tag{10}\]

where \(n_{\text{train}}\) is the number of training samples. The prediction time can similarly be calculated, but instead the number of circuit executions is calculated using the number of testing samples:

\[n_{\text{CE}}=n_{\text{train}}\times n_{\text{test}}. \tag{11}\]

No attempt was made to batch the training data and to implement incremental learning, as kernel-assisted support vector machines are one shot solvers of a maximal margin classifier type that fits the decision boundary to training data in one go, meaning no iterations are required. The quantum circuit diagram used as explained, is illustrated in Figure 2. This was all done using the Python libraries Pennylane and scikit-learn.

### QCNN Circuit

Quantum convolutional neural network architecture and consequently the architechture of the quantum circuit representing the QCNN are influenced by the shape of the convolutional and pooling steps. Our circuit makes use of eight qubits, three convolutional steps and three pooling steps, before measuring the last remaining qubit. A stride of five was chosen for the convolutional steps, meaning qubits 1 and 6 will be connected by an edge in the first convolutional step. This will have an effect throughout the circuit. We choose two-qubit unitaries for the convolution steps and attempt to maintain a short circuit depth in our experiments. Convolutional steps have the effect of cross-correlating the two qubits the operator is applied to, which also results in the sharing of weights/parameters in a neural network representation of the circuit. The chosen unitary can be seen in equation 12.

\[U|q_{1}\rangle\otimes|q_{2}\rangle=\text{CNOT}\cdot R_{Y1}(\theta_{1})\cdot R_{ Y2}(\theta_{2})|q_{1}\rangle\otimes|q_{2}\rangle, \tag{12}\]

where the \(R_{Y1}\) and \(R_{Y2}\) are y-rotation gates applied to each qubit and the rotation parameters \(\theta_{1}\) and \(\theta_{2}\) will be optimised every iteration. After the convolutional steps, pooling steps reduce the number of qubits to an eventual solitary qubit that is to be measured for output. In the pooling steps we use CNOT gates to represent the directed edges. Figure 3 shows an example QCNN circuit with the directed graph representation

We considered two cases: 1 - using the entire training set for training every epoch and 2 - batching the training data into smaller batches of 10 samples or candidates per batch for training every epoch. The way the batching was done meant that there was chance of sample overlap between batches, with this overlap being almost guaranteed if the batch size of 10 multiplied by the number of epochs was comparable to the training set size. Ideally it would be best to use large training sets. In both cases the learning rate was set to 0.01 and the iteration limit was set to 150 epochs. The chosen loss function to minimize is the binary cross-entropy loss:

\[\begin{split}\text{BCEL}(y_{\text{pred}},y_{\text{test}})=-\Big{(} y_{\text{test}}\cdot\log(y_{\text{pred}})+\\ (1-y_{\text{test}})\cdot\log(1-y_{\text{pred}})\Big{)}.\end{split} \tag{13}\]

The QCNN quantum circuit used is illustrated in Figure 3. The Python libraries PyTorch and Pennylane were both used in conjunction to iterate over the forward pass, back pass and gradient updating steps.

A consequence of both the circuit architecture and how the method is implemented is a different scaling to the QSVM. We can still approximate the total training time by using equation 9, but now the total circuit executions can be easily controlled and calculated using the total number of epochs and samples used:

Figure 2: The QSVM circuit. We simply apply angle embedding (rotational encoding) using \(Y\)-rotation gates, followed by applying the complex conjugate transpose of this embedding using features from a separate feature vector as parameters. Measurement is done in the computational basis only after a projector matrix was applied to help calculate the inner product without a SWAP test [45].
\[n_{\rm CE}=n_{\rm Epochs}\times n_{\rm Train}, \tag{14}\]

where the number of epochs and training samples can be controlled. Estimating the prediction time is much simpler:

\[n_{\rm CE}=n_{\rm Test}, \tag{15}\]

since only one circuit execution is performed per candidate.

## IV Results

### Simulations

We simulate the above circuits, using Pennylane, and plot the confusion matrices. This gives us an indication of how the methods compare with respect to performance on the chosen data set. The classical device used to simulate all the noise-free and noisy simulations has the following specifications: Lenovo Laptop Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz 2.60 GHz, with 16,0 GB (15,8 GB usable) RAM and a GTX 960M GPU, which was not used to supplement the computational power.

The confusion matrices from Figure 4 (which are plotted for runs at 200 training samples and 400 testing samples) show that both methods perform well at classifying pulsars correctly. Both methods are good at correctly predicting negative cases. False positives are harder to avoid for the QSVM than for the QCNN and the reverse is true for the false negatives. The QSVM predicted a higher proportion of the positive cases correctly, however, the QCNN still managed to correctly predict most positive cases correctly as well. The implication of the confusion matrices is that if the QSVM model predicts a candidate as a non-pulsar, it is almost certainly a non-pulsar, however when the model predicts that a candidate is a pulsar, then that candidate might have to be double checked. The reverse applies to the QCNN. The QCNN performs worse at avoiding false negatives, which is not ideal as we want to minimise the amount of mistakes when predicting the positive case.

It is fortunate that there are more non-pulsar candidates than pulsar candidates that are correctly predicted in both cases. This causes a reduced amount of manual confirmation required, as negative predictions will never be doubled checked. This is because they form the majority of all predictions, which means there are too many to manually confirm. Even though the amount of manual labour is reduced, to be entirely sure of the positive predictions, all positive predictions will have to be manually checked, especially when applying the model to unseen data.

We also simulate the circuits for an increasing number of training samples. The resultant curves of eight performance metrics can be plotted as seen in Figure 5. The expected behaviour would be for the performance metrics to increase as sample size is increased, as more samples would typically lead to a more optimized model. This is marginally true for the lower sample sizes (less than 50 samples) where a slight increase can be observed. It is also observed in the QSVM precision plot and in isolated cases for informedness, geometric mean, balanced accu

Figure 3: Top: The QCNN circuit. Note the stride of 5 in the convolutional steps. The rotational encoding, convolution, and pooling steps are all illustrated as a collection of gates inside coloured blocks. Bottom: The directed graph representation of the entire circuit. Edges in the the convolutional steps are the unitary specified earlier and edges in the pooling steps are CNOT gates. Directed edge arrows indicate which node (qubit) is the target qubit.
racy and recall. The near constant behavior throughout the QCNN case indicates that the saturation point for training is reached early and a possible explanation for this is training over many epochs using the entire training set.

The uncertainty at lower samples is also more prominent, which implies that for a limited number of samples, training is not satisfactory. This is observed in the accuracy, specificity, and precision curves. We observe that as the number of samples are increased, the uncertainty in the accuracy and specificity decreases. This indicates an improved prediction aimed towards the negative case. There is also an increase in the precision curve for the QSVM, implying more samples result in a improved prediction performance towards the positive cases. Recall and precision have a higher uncertainty and lower values than specificity, which again illustrates that it is easier to predict negative cases correctly. All positive predictions have a higher probability to be incorrect and will thus have to be manually checked. This is consistent with the results from the confusion matrices.

Two important points worth noting, are the uncertainty and fluctuating behaviour in the balanced accuracy, geometric mean, and informedness. All three metrics depend on recall as a metric, which indicates the performance on positive cases. Since there are more mistakes for the positive cases, and hence more uncertainty, these values tend to fluctuate more.

Finally, from a general perspective, it is clear that the models perform well when measuring in accuracy, specificity and negative prediction value as metrics and slightly worse when considering precision and recall. It depends on the sample size for the other metrics, but we note that the QSVM outperforms the QCNN on most metrics, which is a strong motivator for its use. If precision is prioritised, then the QCNN can be considered as well, since it has a precision that is high compared with the QSVM.

It is also interesting to note that changing the QCNNs architecture, for example the choice of unitary operators used, may give insight as to how certain data features might be linked. In other words QCNNs might be more interpretable than QSVMs. This is especially true for changing the striding between qubits in the convolutional step and changing the final remaining qubit after all the pooling steps. It might supply a way of gauging which features are more important or more closely related without performing PSA or feature correlation tests. An example of this is a sudden increase in accuracy when using a stride of 5, which can be attributed to an unknown correlation in the features.

### QCNN: Unbatched versus Batched Data

We considered the effect of data batching for the QCNN, which involved randomly batching the training data into balanced groups of 10 each. This means that in each epoch of the training and optimization step, the model is training with a new set of 10 samples. The two cases can be compared in Figure 4.

For the batched case it is clear that the minimization of the loss fluctuated a lot more, since it trained over less samples in each iteration. The overall result in the confusion matrices was nearly identical.

The results from Table 2 shows that the gain from training over all samples in each epoch is not worth the increase in overall training time. It is therefore recom

Figure 4: Confusion matrices for runs at 200 training samples and 400 testing samples using QSVMs (Green), QCNNs (Purple) and batched QCNNs (Blue). The confusion matrices of both QCNN approaches are nearly identical, which is an indication that batching should be strongly considered. The two loss curves are each an example of one of the convergent loss optimization processes. An example of a kernel gram matrix (top right) is provided for 200 training samples. Each entry in the confusion matrices is the average of six distinct runs and the uncertainty is the standard error.
mended that data batching be used for this method as it will drastically decrease the amount of time required to train the model. It is also a good idea to increase the number of epochs to cover as many samples as possible and to limit the amount of sample overlap in each batch. Note that this only applies to the way we batched the training data and can be removed entirely if unique batches are sampled. Since it is possible to use a batched QCNN to benefit from its faster training and prediction times compared to the standard QCNN approach, the batched QCNN can also be implemented on even larger batches or data set sizes. The result should be an increase in performance and a decrease in uncertainty.

### Classical versus Quantum

The runtime plots, that follow later in Figure 7, indicate that there is an intersection in the training time somewhere close to 200 training samples for both quantum models. Assuming that training time would be comparable at this sample size, generic classical versions of a SVM and a CNN were also performed to compare these methods with their classical equivalents. The results are tabulated in Table 2.

Comparing CNNs and SVMs separately shows that classical SVMs perform better compared to quantum SVMs when considering every metric, however, the QSVM does perform similarly when considering speci

Figure 5: Eight performance metrics plotted versus an increasing training sample size. The plots included (alternating from left to right going downwards) are accuracy, balanced accuracy, recall, specificity, precision, negative prediction value, informedness, and geometric mean. Averages are illustrated using x-markers and the uncertainty (standard error) is indicated as a shaded area. Each data point is the average of twelve distinct runs. Green indicates the QSVM and purple the QCNN.
ficty. This means that the QSVM is up to par with the CSVM when predicting negative cases. The QCNN is also outperformed by its classical equivalent, but the difference is that its specificity is actually higher than its classical counterpart. This implies that if you wanted to solve the current problem with a quantum approach, the QCNN would be the better choice of the two, not only because it has the higher specificity, but also its faster training time. CCNNs are typically used for spatial analysis in image data, which can cause shortcomings in their performance when identifying feature vectors. This is clearly observed in the low precision value in the CCNN column. It might be possible to improve the classical CNN and achieve improved results (specifically to increase the specificity and the precision), however, it is promising that if quantum computers become more error robust and have faster operational and readout times that there are some improvements that can be made for quantum approaches. Perhaps when we transcend the NISQ era, meaning when the runtimes are reduced and noise limited enough for these methods to become viable when compared to classical methods, quantum approaches may become mainstream.

### Real Devices and Noisy Simulations

We compare the runtime performance of both models on real quantum devices. This is made possible by Qiskit Runtime [46; 47], which is a cloud-based hybrid quantum-classical computational system to implement quantum calculations and to optimize classically. Qiskit Runtime can be interfaced with Pennylane's qiskit-pennylane plugin [48]. It was observed that with both methods, in order to perform a comparative run at 200 training samples and all other parameters kept exactly as it was set in the simulations, the overall training time would be completely impractical on real devices. The QCNN and QSVM on real devices had an approximate training time of \(142.00\pm 5.10s\) and \(3.33\pm 0.47s\) respectively. Using the scalability statements made earlier, these training times can be used to approximate a total training time value for each method by simply multiplying it with the total number of circuit executions required to finish the entire training and predicting process. The result is that it would take longer than a day for the QCNN and QSVM approaches just to be trained. This becomes worse if we include real device prediction and queue time. These times are calculated for a reduced number of epochs of 10 in the QCNN case. Both numbers are impractical, and even if a data batching workaround was used along with reducing the number of iterations even further, the time of one circuit execution is still too long compared with the near instantaneous runtime of one iteration in the classical counterparts. The QSVM would be the better choice here, since its circuit execution time is faster, however, the scalability of the QSVM has to be factored in.

To compare the impact of noise on both circuits, focus was placed on noisy simulations. We wanted to find the relationship between the depth of the quantum circuits and their intrinsic noise. Apart from how the circuits are changed during the compilation process, where it is translated from the designed architecture to what is has to be on the quantum device due to the device's restricted basis gate set, another source of error is gate-based error. This is where any gate applied in the circuit has a probability of failing due to many factors such as: short coherence times that lead to eventual decoherence of the state, crosstalk between qubits, and imprecise control of the qubits when applying operator gates. The more gates there are in succession, the higher the impact will be. Device specific qubit connectivity can lead to an overall greater depth during compilation which would accentuate these problems. This happens because certain multi-qubit gates require two qubits to be connected in order to be applied. If they are not connected, the compilation process tries to find another gate combination that would

\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Metric** & (1) **QSVM** & (2) **QCNN** & (3) **Batched QCNN** & (4)**CSVM** & (5) **CCNN** \\ \hline Accuracy & \(0.950\pm 0.006\) & \(\mathbf{0.972\pm 0.002}\) & \(0.971\pm 0.003\) & \(0.972\pm 0.004\) & \(0.942\pm 0.009\) \\ Balanced Accuracy & \(0.902\pm 0.009\) & \(0.865\pm 0.011\) & \(0.863\pm 0.014\) & \(\mathbf{0.944\pm 0.007}\) & \(0.935\pm 0.013\) \\ Recall & \(0.842\pm 0.018\) & \(0.734\pm 0.022\) & \(0.730\pm 0.028\) & \(0.910\pm 0.012\) & \(\mathbf{0.928\pm 0.027}\) \\ Specificity & \(0.961\pm 0.006\) & \(\mathbf{0.996\pm 0.001}\) & \(\mathbf{0.996\pm 0.001}\) & \(0.978\pm 0.004\) & \(0.943\pm 0.010\) \\ Precision & \(0.707\pm 0.033\) & \(\mathbf{0.950\pm 0.010}\) & \(0.948\pm 0.018\) & \(0.817\pm 0.030\) & \(0.638\pm 0.036\) \\ Negative Prediction Value & \(0.984\pm 0.002\) & \(0.974\pm 0.002\) & \(0.973\pm 0.003\) & \(0.991\pm 0.001\) & \(\mathbf{0.992\pm 0.003}\) \\ G-Mean & \(0.899\pm 0.009\) & \(0.854\pm 0.012\) & \(0.852\pm 0.017\) & \(\mathbf{0.943\pm 0.007}\) & \(0.935\pm 0.013\) \\ Informedness & \(0.803\pm 0.018\) & \(0.730\pm 0.021\) & \(0.726\pm 0.028\) & \(\mathbf{0.888\pm 0.014}\) & \(0.871\pm 0.026\) \\ Training Time (s) & \(1241.99\pm 29.75\) & \(1262.85\pm 9.08\) & \(61.25\pm 0.63\) & **Negligible** & \(1.77\pm 0.05\) \\ Prediction Time (s) & \(2492.15\pm 60.53\) & \(12.26\pm 0.13\) & \(12.04\pm 0.03\) & **Negligible** & \(0.14\pm 0.01\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: The results of both the QSVM (1) and QCNN (2) approaches, as well as the batched QCNN (3), classical SVM (4), and classical CNN (5) methods. All runs were taken for models trained at 200 training samples and testing was performed on 400 testing samples. Each entry is the average \(\pm\) standard error. The best performing method for each metric is indicated in bold font.
deliver the same result. This increases the depth, adding more gates that have probabilities to fail.

This is the main source of error in larger depth circuits and we model the gate-based error of a quantum computer with simple bit flips, then it can be clearly seen, in Table 3, that the longer depth of the QCNN circuit induces a more error prone method and the eventual measurement results will be more random. This serves as an indication that you would rather choose the QSVM, since it is more resilient to noise during the current NISQ era. We increase the probability of a single gate undergoing a bit flip to see just how random the results can get. This probability is the analogue of the median error rates that can be found on the device page on the IBM Quantum Experience page [49].

As the probability of gate-based bit flips are increased for both the QSVM and QCNN we see a clear downward trend in the balanced accuracy. This is accentuated for the QCNN case where the falloff is more rapid. The effect of noise is more pronounced for the QCNN because of its larger depth. More gates in succession have a higher probability of undergoing a bit flip. These measurements were taken for 50 training samples and 100 testing samples to reduce computational costs when simulating noisy quantum circuits. Finally for a bit flip probability of 0.5, the measurements become truly random, which is exactly what is expected. The reason balanced accuracy was used, and none of the other metrics, was to use a metric that is not inflated when more negative cases than positive cases were predicted correctly. We used a metric not biased towards either the positive or negative cases.

The quantum device used to approximate the total runtime of the two circuits was the IBMQ Guadalupe device, which has 16 qubits of which the ones marked in red in Figure 6 were used for the QCNN. The black numbers indicate which qubit each node represented in the actual implementation. Qubits 0 to 7 were used for the QSVM as no qubit connectivity was explicitly required.

### Runtime

Another figure of merit we considered, as discussed earlier, is runtime. This includes both training and prediction time. The runtime plots can be found in Figure 7.

The training time scaling of both methods can be seen from Figure 7. The QSVM scales quadratically, while the QCNN scales linearly, with the intersection between the two curves indicating that for a low number of samples the QSVM is faster. The QCNN will train faster on a larger sample size. The same intersection for the real devices cannot be observed in the figure, but it is clear the two curves are tending towards a similar intersection.

The prediction time plots show that for QCNNs the prediction time remains constant compared to the linear scaling of the QSVM. This is because QCNN prediction is done for a constant 400 samples. No reference is made to the training set as in the QSVM case. The linear scaling of the prediction time for QSVMs is because it scales linearly with the number of training samples, while the number of prediction samples are held constant. This is a significant aspect to consider when QSVMs are used, since using a QSVM on unseen data will take longer.

## V Conclusion

In this comparative study we implemented an 8-qubit quantum-enhanced support vector machine and an 8-qubit quantum convolutional neural network. The two methods were compared by performing noise free and noisy simulations as well as a limited real device implementation that allowed for approximation of a possible fully implemented run on real devices. The training time and prediction time for both methods were also compared for many sample sizes. Both methods show improvement over the known quantum approach [20] and are also lower depth, which helps with noise robustness.

\begin{table}
\begin{tabular}{l c c} \hline \hline
**Noise Level** & (1) QSVM & (2) QCNN \\ \hline
**p = 0** & \(0.911\pm 0.023\) & \(0.914\pm 0.026\) \\
**p = 0.01** & \(0.879\pm 0.021\) & \(0.852\pm 0.03\) \\
**p = 0.1** & \(0.836\pm 0.031\) & \(0.758\pm 0.055\) \\
**p = 0.5** & \(0.501\pm 0.018\) & \(0.500\pm 0.000\) \\ \hline \hline \end{tabular}
\end{table}
Table 3: The effect of circuit-based noise due to the depth of the quantum circuits on method performance indicated using balanced accuracy. A decreasing trend is observed for both methods. The QSVM case performs better at increasing noise probabilities than the QCNN case because of larger depth in the QCNN circuit. Both methods yield random results at \(p=0.5\). Each table entry is the average of 6 distinct runs and the uncertainties are the standard deviations.

Figure 6: The qubit connectivity diagram for the IBMQ Guadalupe device. The qubits marked with red circles formed part of the QCNN approach and the order in which they were assigned in the circuit is indicated by black numbers. Qubits 0 to 7 (white text) were used for the QSVM.
QSVMs are easy to implement and have a small depth, which is really important for avoiding decoherence and noise in real device runs. QSVMs, however, scale quadratically with sample size, which makes them really unattractive for big-data problems. The kernel function used by the SVM is a black box, meaning that there is no way to tell how the algorithm relates features to prediction. They are, however, really effective at a lower number of training samples and in this regime they can definitely be considered.

QCNNs with a larger depth and a more complex circuit are more prone to noise, but does indeed provide more efficient training and prediction than from the QSVM. A specificity that improved over our classical approaches is also observed. Although the approach is not completely transparent, it also supplies a method to get an intuition as to how the different features are linked to one another. The special case of batching the training data resulted in near identical results (the same high specificity was observed here as well) which makes QCNNs viable for a larger number of samples.

It is true that both methods have their downsides and advantages, but it should be clear that the method of choice should depend on the training set size. Judging by its ease of implementation, shorter depth making it more noise resistant, guarantees of performance compared to classical methods and its shorter runtime for a low sample size when simulated and on real devices, QSVMs should be preferred comparatively over QCNNs. Larger sample sizes after the two training time curves have intersected demands that QCNNs be implemented instead, definitely making use of data batching, however the larger depth will introduce a lot of noise into the system, which means that error mitigation techniques will be required.

It should be clear that quantum machine learning can be used for astronomical object classification and real-world binary classification in general, but in the current NISQ era QML methods applied to classical data falls short when compared to classical methods.

A broad architecture search was already performed by changing some of the hyperparameters like striding, but it may be possible to find a QCNN circuit architecture that performs even better than the one considered here by using different unitaries in the convolutional or pooling steps. It would be good idea to do a more comprehensive circuit architecture search for other 8-qubit QCNN architectures that would provide possible improvements to classification accuracy. This is made easier by using the HierarQcal package, which has been used for a similar application in [42]. Circuit-based noise would be the main limiting factor and is the reason why we kept the QCNN circuit as simple as possible.

There are many ways in which this work can be extended. Multi-dimensional classification, where there are

Figure 7: Training and prediction time in seconds versus an increasing number of samples using a testing set of 400 samples for (Top) simulated runs and (Bottom) approximated real device runs. Both training plots show a linear scaling in the QCNN and a quadratic scaling in the QSVM that has been described before. The prediction time plots indicate that the QSVM takes longer to perform predictions. This is because prediction requires reference to the training set used. Simulated and real device results are taken and approximated for QCNN runs at 150 and 10 epochs, respectively. Note the scale of the y-axis in the real device plots.
more than two classes could be an interesting extension. A typical example of multi-classes here would be standard and millisecond pulsars. These methods can also be applied to other data sets to find out if the same behaviour persists. Once better quantum computers are built, either through fault tolerance, improved qubit connectivity or faster operational and readout times, the real device findings can be revisited to confirm if they still hold true then. Applying the methods on real devices may or may not require improvement using error mitigation techniques. This would, however, be the perfect indicator of the viability of these methods for the application of pulsar classification on real devices. Trying different approaches to the problem of pulsar classification such as quantum anomaly detection may yield interesting results. This would involve training a quantum model on only the non-pulsar data and then testing the model on a testing set consisting of a mix of pulsar and non-pulsar samples to see if the model is able to effectively pick out the anomalous pulsars.

###### Acknowledgements.

 This work was funded by the South African Quantum Technology Initiative (SA QuTi) through the Department of Science and Innovation of South Africa. We also wish to acknowledge Dr. Maria Schuld for clarifying theoretical points regarding support vector machines and some of the initial work done by Shivani Pillay on classification of pulsars using quantum kernel methods.
