LSTM-QGAN: Scalable NISQ Generative Adversarial Network
Cheng Chu Aishwarya Hastak Fan Chen

A Novel Audio-Visual Information Fusion System for Mental Disorders Detection
    Yichun Li, Shuanglin Li, Syed Mohsen Naqvi
Intelligent Sensing and Communications Research Group, Newcastle University, UK

    September 9, 2024
===============================================================================================================================




§ ABSTRACT 
Current quantum generative adversarial networks (QGANs) still struggle with practical-sized data. First, many QGANs use principal component analysis (PCA) for dimension reduction, which, as our studies reveal, can diminish the QGAN's effectiveness. Second, methods that segment inputs into smaller patches processed by multiple generators face scalability issues. 
In this work, we propose , a QGAN architecture that eliminates PCA preprocessing and integrates quantum long short-term memory (QLSTM) to ensure scalable performance. 
Our experiments show that  significantly enhances both performance and scalability over state-of-the-art QGAN models, with visual data improvements, reduced Fréchet Inception Distance scores, and reductions of 5× in qubit counts, 5× in single-qubit gates, and 12× in two-qubit gates.





NISQ, Quantum Generative Adversarial Network, Long Short-Term Memory



























§ INTRODUCTION




Current QGANs.
Recent advancements in Noisy Intermediate-Scale Quantum (NISQ) platforms <cit.> have catalyzed intense research on Quantum Generative Adversarial Networks (QGANs) <cit.>, which are well-suited to the constraints of NISQ systems, such as limited qubit counts and shallow circuit depths<cit.>.
Building on the foundational work <cit.> that established the theoretical superiority of QGANs over classical counterparts, 
early QGAN implementations <cit.> only focused on low-dimensional inputs like single-bit data. 
Subsequent research introduced innovations such as Wasserstein loss <cit.> 
and novel architectures <cit.> to improve training stability.
More recent work <cit.> expanded QGANs to high-dimensional data, like the 28×28 MNIST dataset, by employing dimensionality reduction techniques like Principal Component Analysis (PCA).
The state-of-the-art (SOTA) PatchGAN <cit.> further segments inputs into smaller patches, enabling efficient processing on practical NISQ devices.




Limitations.
Despite recent developments, QGANs continue to face challenges in managing practical-sized data.
First, while pre- and post-processing with PCA and inverse PCA <cit.> enable QGANs to handle large-dimensional data, PCA often dominates the process, diminishing the contributions of the QGANs themselves. 
Second, although PatchGAN <cit.> facilitates the direct processing of practical-sized inputs through multiple small patches, its architectural limitations demand an increasing number of quantum resources as input size grows, leading to serious scalability challenges. 
For instance, generating a single MNIST image requires a prohibitively high 56 sub-quantum generators and 280 qubits.
Third, and more concerning, our preliminary study shows a significant decline in output quality as PatchGAN scales from its original 5-qubit design <cit.> to 8 qubits, severely limiting its effectiveness at larger scales.




Contributions.
We introduce , a novel architecture that eliminates the need for PCA when processing large-dimensional data. 
The design allows for the use of a constant amount of NISQ computing resources as input size increases. 
However, as additional hardware resources become available, the architecture scales efficiently, ensuring consistent and reliable performance.
Our contributions include:



  * Preliminary Analysis.
We conduct experiments on the SOTA QGANs <cit.>, revealing previously undisclosed limitations in PCA pre-processing and model scalability.


  * Scalable Architecture.
We present , a scalable QGAN architecture inspired by recent advances in quantum long-short memory (QLSTM) <cit.>. 
 eliminates the need for PCA, maintains constant NISQ resources as input size grows, and efficiently scales with increasing quantum computing resources.


  * Enhanced Performance.
We conduct evaluations on NISQ computers. Experimental results show that  significantly enhances generative performance and improves scalability compared to SOTA QGANs.

































§ BACKGROUND








QGAN Basics.
Figure <ref>
illustrates a standard QGAN with two parameterized models: the Generator, G(θ_g), which generates synthetic data, and the Discriminator, D(θ_d), which evaluates the generated data against real data. 
G is implemented using a quantum neural network (QNN), typically composed of a data encoder E(·) and repeated layers of a variational quantum circuit (VQC) with one-qubit rotations (i.e., Rot.) and two-qubit entanglement (i.e., Ent.). 
D in SOTA QGANs <cit.> can be implemented with either classical or quantum models.
The objective is to optimize the predefined minmax loss ℒ, as outlined in Equation <ref>, where z represents the latent variable.
The specific loss function can be implemented using various specified functions <cit.>.





The overall goal is to enable G to generate data indistinguishable from real data, while D improves its ability to differentiate between them.



    min_θ_gmax_θ_dℒ{D_θ_d(G_θ_g(z)),  D_θ_d(x) }










SOTA QGANs.
To manage larger-dimensional data  with limited qubits on NISQ computers, SOTA QGANs <cit.>) primarily utilize the following two techniques:



  * Pre- and Post-Processing.
Several recent QGANs <cit.> utilize principal component analysis to reduce input dimensions (e.g., from 784 to 4 in <cit.>) to fit within the limitations of NISQ computers with constrained qubits. 
The key steps in PCA involve: 
(1) standardizing the data to have zero mean and unit variance, and
(2) calculating the covariance matrix 𝐂 and the matrix 𝐕_k, which contains the top k eigenvectors (principal components).
For any data matrix 𝐗 with mean μ, the data can be reduced to the top k principal components by 𝐙=𝐗𝐕_k.
The reduced-dimensional data 𝐙^* can then be reconstructed to approximate the original data through inverse PCA: 𝐗^*=𝐙^*𝐕^⊤_k + μ.


  * Patched Input.
PatchGAN <cit.> segments the input into small regional patches and trains a dedicated sub-generator for each, capable of generating synthesized data that follows the pattern of the corresponding patch. 
This approach makes it a resource-efficient QGAN framework. 
The number of sub-generators scales with the input size; for instance, the 5-qubit design in the original work <cit.> requires 56 sub-generators to process the 784-pixel MNIST dataset, and doubling the input size would proportionally increase the number of sub-generators needed. 




QLSTM.
Long short-term memory <cit.> effectively captures spatiotemporal information, enabling task-specific regulation of data flow. 
Recent work <cit.> has introduced quantum LSTM, extended to various sequential learning tasks <cit.>. 
As shown in Figure <ref>, QLSTM retains the classical LSTM gating mechanism, with the key distinction being the integration of QNNs. 
Due to page limit, we refer readers to <cit.> for detailed insights into QLSTM. 
LSTM has already been applied in classical GANs <cit.>, demonstrating enhanced generative power and reduced computational cost. 
Building on this, we aim to leverage LSTM's ability to selectively retain relevant patterns within a QGAN by training a QLSTM-based generator using different patched inputs, rather than separate sub-generators for different patches as in <cit.>. 








§ PRELIMINARY STUDY AND MOTIVATION






 §.§ Preliminary Study




PCA Overshadows QGANs.
QGANs <cit.> on MNIST utilize PCA and inverse PCA for dimensionality reduction and reconstruction.
To evaluate the impact of PCA, we reduced 28×28 MNIST images to 1×2 vectors using , generating the corresponding 𝐂, 𝐕_2, and μ. 
We then randomly generated 1×2 vectors, applied inverse PCA, and present the reconstructed images in Figure <ref>.
These reconstructions closely resemble the original MNIST data and are comparable to those produced by QGANs <cit.>. This suggests that PCA pre- and post-processing may play a dominant role, potentially overshadowing the effectiveness of QGANs.
These findings raise concerns about the independent validity of QGANs when PCA is involved, underscoring the need to evaluate QGANs using unprocessed data.








Scalability for PatchGAN.
PatchGAN <cit.> claims effectiveness with patch-based processing of high-dimensional inputs, but its original work only reports results using 5 qubits. 
To assess scalability, we increasing the qubit count from 5 to 8. Since PatchGAN employs amplitude encoding and processes one patch at a time, we adjusted the number of sub-generators to cover all 784 pixels in an MNIST image.
As show in Figure <ref>, 
the generated images reveal a rapid deterioration in quality as qubit count increases, with sub-figure titles indicating the number of qubits and required sub-generators (i.e., sub-gens). 
This highlights the poor scalability of the PatchGAN architecture and suggests that even with more qubits, the architecture is unlikely to improve or effectively handle larger-scale inputs.



































 §.§ Motivation



Our preliminary results highlight the critical need for a QGAN model capable of directly processing real-world data without PCA preprocessing, as well as a more scalable architecture to overcome the limitations of existing QGANs. Motivated by these findings, we are exploring the integration of patched inputs inspired by PatchGAN <cit.> to enable direct input processing without PCA. Specifically, we are investigating a scalable QGAN framework that leverages QLSTM as the generator's backbone, utilizing QLSTM's ability to capture spatiotemporal information across patches with a single generator, rather than separate sub-generators as in <cit.>. Additionally, we are reengineering the quantum circuit ansatz within the QLSTM structure to improve hardware efficiency, fully addressing the NISQ constraints overlooked in previous QLSTM studies like <cit.>.




















































































§ 






 §.§ Overall Architecture





As illustrated in Figure <ref>(b),  utilizes QLSTM at the core of the generator to enhance scalability and resource efficiency. 
Like PatchGAN <cit.>, the discriminator in  can be implemented using either a classical or quantum neural network, depending on the available quantum computing resources. 
The following outlines the key components and configurations in .



  * Patch Inputs without PCA.
In line with <cit.>,  processes patched inputs to generate corresponding output patches, which are then recombined into a complete output. Unlike <cit.>,  eliminates the need for PCA and inverse PCA, processing the original data directly. This introduces a trade-off between resources (i.e., qubit number N) and processing latency (i.e., steps T). 
With an N-qubit implementation,  generates 2^N measured probabilities at each step as the output vector for each synthetic patched output. These vectors are then compared to the real patched input data in the discriminator. The total number of steps, T, is determined by D/2^N, where D represents the size of the real data. 


  * Scalable QGAN with LSTM.
The generator in  consists of four QLSTM cells, as shown in Figure <ref>(b). 
The process starts by feeding normally distributed noise z into the generator to produce the initial sub-image, G_θ_g(z). The discriminator then assesses both the synthetic and real input patches, calculating the loss ℒ. 
Unlike PatchGAN <cit.>, which requires a separate generator for each patch—leading to a significant increase in NISQ resource overhead as input size grows— leverages the QLSTM's ability to learn and retain relevant patterns while discarding irrelevant information, regardless of the index of patches. 
To achieve this, gradients from all patches within a single input are averaged and applied to update the model parameters collectively, resulting in an image-adaptive generator that scales with increasing data dimensions while maintaining fixed resource usage.


  * Training Optimization.
Convergence in QGAN training is a critical challenge, significantly influenced by the choice of quantum loss function.
Within the  framework, we evaluated both the conventional binary cross entropy loss <cit.> and the Wasserstein loss <cit.>.
The specific Wasserstein loss used for  is detailed in Equation <ref>,
where 
𝕃_x̂ = x̂∼ P_x̂𝔼 [  ( ∇ _x̂D_θ_d(x̂)  _2 -1   ) ^2],
P_r and P_g represent the real data (i.e., x) and 
generated data (i.e., x̃∈D_θ_d(G_θ_g(z))) distributions, respectively.
The distribution
P_x̂ is uniformly sampled between P_r and P_g, and
λ is a constant.
Experimental results on the impact of QGAN loss functions are discussed in Section  <ref>.




    min_θ_gmax_θ_dx∼ P_r𝔼 [ D_θ_d(x)   ] - x̃∼ P_g𝔼 [ D_θ_d(x̃)   ] - 
    λ𝕃_x̂















 §.§ NISQ Implementation




 offers flexibility in implementing G and D.
For fair comparison, D is implemented as a classical neural network, as in PatchGAN <cit.>.
In the QLSTM cells for G, we employed a hardware-efficient ansatz inspired by recent QNNs <cit.>, instead of the generic circuit from <cit.>.
Figure <ref>(a) shows the QNN circuit, which utilizes seven qubits.
Each VAC block includes , , and  layers, followed by 2-qubit  entanglement layer, with the VQC layers repeated twice.
The measurement layer converts the quantum state into classical vectors. 
Although the gate count matches that in <cit.>, our circuit uses native gates, while the (α, β, γ) gate in <cit.> requires synthesis into multiple native gates.






Design Overhead.
Table <ref> compares the hardware resources required by PatchGAN and  for the MNIST dataset. Due to architectural differences, a QNN in PatchGAN refers to the quantum generator used for each input patch, while in , it refers to the quantum module within the QLSTM. 
The last three rows of Table <ref> highlight that  achieves a significant reduction:
a 5× decrease in qubit counts, a 5× decrease in one-qubit gates (1QG), and a 12× decrease in two-qubit gates (2QG).


























§ EXPERIMENTS AND RESULTS







 §.§ Experimental Setup





Schemes and Benchmarks.
We compare  with PatchGAN <cit.> using the MNIST dataset, which consists of 28×28 grayscale images of handwritten digits 0∼9.
PatchGAN is implemented according to its original design <cit.>, utilizing 5 qubits and 56 sub-generators. Each sub-generator produces a 14-pixel patch, and together, the 56 sub-generators generate the entire 784-pixel MNIST image.
For , we implement the generator with two QLSTM layers, each containing 4 QNNs with 7 qubits. At each time step, the LSTM-QGAN generates a 196-pixel patch, requiring 4 time steps to produce a complete MNIST image.

Simulation. 
All QGANs are implemented with the PennyLane and Torchquantum libraries.
PatchGAN and  are trained using the ADAM optimizer with a 2e-4 learning rate, a 128 batch size, and 1000 epochs. Quantum circuits are run on the NISQ  computer <cit.>.

Evaluation metrics.
We evaluate the generated images using both qualitative (e.g., visual inspection) and quantitative methods. For quantitative assessment, we employ the Fréchet Inception Distance (FID), a widely recognized metric for measuring image similarity in GANs <cit.>. A lower FID score indicates a closer feature distance between real and generated images, signifying higher quality. In our experiments, we randomly select 500 real images and 500 generated images for comparison.















 §.§ Results and Analysis





Comparison of Image Visual Quality.  
Figure <ref>(a)
presents a visual comparison between the images generated by PatchGAN and . 
PatchGAN demonstrates limited generation capabilities, as the outlines of the digits (0∼9) are only vaguely identifiable, with noticeable white noise in the background. 
Additionally, the clarity of more complex digits, such as 4, 5, and 9, is particularly low, further highlighting its deficiencies.
In contrast,  demonstrates superior image generation, producing sharper and more distinct digits with minimal noise, underscoring its enhanced capability in generating high-quality images.



Comparison of Image FID Scores.
Figure <ref>(b)
compares the FID scores of images generated by PatchGAN and across different digit classes. 
The FID scores vary between the two models depending on the complexity and distinctiveness of each digit. 
Overall,  achieves lower FID scores than PatchGAN, indicating higher quality in the generated images. 
Specifically, PatchGAN shows significant variability, with its highest FID score at 445.22 (class 0) and its lowest at 246.56 (class 4). 
In contrast,  consistently outperforms PatchGAN, with its highest FID score at 275.58 (class 7) and its lowest at 134.31 (class 1). 
On average, PatchGAN's FID score is 318.02, while  achieves a significantly lower average FID score of 193.28. 




Impact of Loss Function.
Figure <ref>
illustrates the impact of Wasserstein loss and binary cross-entropy (BCE) loss on the training convergence of , comparing both generator loss (i.e., GL) and discriminator loss (i.e., DL). 
With Wasserstein loss, the DL initially decreases while GL increases, ultimately leading to convergence as training progresses. 
Conversely, with BCE loss, GL rapidly increases after several training cycles and stabilizes around 100, while DL drops sharply—indicating mode collapse, a known issue in GAN training. 
Although  with BCE loss can stabilize, achieving full convergence may require more sophisticated techniques. 
In contrast, Wasserstein loss offers greater training stability, resulting in smoother convergence.






§ CONCLUSION



This work presents , a quantum generative adversarial network (QGAN) architecture that overcomes key limitations in existing models. By eliminating reliance on principal component analysis (PCA) and integrating quantum long short-term memory (QLSTM),  achieves scalable performance with efficient resource use. As the first QGAN to incorporate QLSTM, this approach represents a significant advancement likely to inspire further research.










IEEEbib


