# Signatures of correlated defects in an ultra-clean Wigner crystal in the
extreme quantum limit

P. T. Madathil  Department of Electrical and Computer Engineering, Princeton
University, Princeton, New Jersey 08544, USA  C. Wang  Department of
Electrical and Computer Engineering, Princeton University, Princeton, New
Jersey 08544, USA  S. K. Singh  Department of Electrical and Computer
Engineering, Princeton University, Princeton, New Jersey 08544, USA  A. Gupta
Department of Electrical and Computer Engineering, Princeton University,
Princeton, New Jersey 08544, USA  K. A. Villegas Rosales  Department of
Electrical and Computer Engineering, Princeton University, Princeton, New
Jersey 08544, USA  Y. J. Chung  Department of Electrical and Computer
Engineering, Princeton University, Princeton, New Jersey 08544, USA  K. W.
West  Department of Electrical and Computer Engineering, Princeton University,
Princeton, New Jersey 08544, USA  K. W. Baldwin  Department of Electrical and
Computer Engineering, Princeton University, Princeton, New Jersey 08544, USA
L. N. Pfeiffer  Department of Electrical and Computer Engineering, Princeton
University, Princeton, New Jersey 08544, USA  L. W. Engel  National High
Magnetic Field Laboratory, Florida State University, Tallahassee, Florida
32310, USA  M. Shayegan  Department of Electrical and Computer Engineering,
Princeton University, Princeton, New Jersey 08544, USA

###### Abstract

Low-disorder two-dimensional electron systems in the presence of a strong,
perpendicular magnetic field terminate at very small Landau level filling
factors in a Wigner crystal (WC), where the electrons form an ordered array to
minimize the Coulomb repulsion. The nature of this exotic, many-body, quantum
phase is yet to be fully understood and experimentally revealed. Here we probe
one of WC’s most fundamental parameters, namely the energy gap that determines
its low-temperature conductivity, in record-mobility, ultra-high-purity, two-
dimensional electrons confined to GaAs quantum wells. The WC domains in these
samples contain $\simeq 1000$ electrons. The measured gaps are a factor of
three larger than previously reported for lower quality samples, and agree
remarkably well with values predicted for the lowest-energy, intrinsic, hyper-
corelated bubble defects in a WC made of flux-electron composite fermions,
rather than bare electrons. The agreement is particularly noteworthy, given
that the calculations are done for disorder-free composite fermion WCs, and
there are no adjustable parameters. The results reflect the exceptionally high
quality of the samples, and suggest that composite fermion WCs are indeed more
stable compared to their electron counterparts.

In sufficiently dilute two-dimensional electron systems (2DESs), when the
Coulomb energy is much larger than the kinetic energy, electrons should
arrange themselves into an ordered array called the Wigner crystal (WC) [1].
Quantitatively, a quantum WC is expected when the parameter $r_{s}$, defined
as the ratio of the Coulomb and kinetic energies exceeds $\simeq$35 in a 2DES
[2, 3, 4]. Since the theoretical proposal of Wigner, several experiments in
various platforms have reported signatures of a 2D WC [5, 6, 7, 8, 9, 10, 11,
12]. While reducing the density in an ideal 2DES eventually leads to Wigner
crystallization, the disorder present in real 2DESs makes it extremely
challenging to realize the WC state [10]. Another way to quench the kinetic
energy and induce the WC phase is to subject the 2DES to a large,
perpendicular magnetic field which quantizes the electron energy to discrete,
flat Landau levels. In high quality 2DESs, a magnetic-field-induced WC phase
is generally stabilized in the lowest Landau level at small filling factors
($\nu$) $\lesssim$ 1/5. Several early experiments reported evidence for the WC
phase at high-fields [13, 10]. These included non-linear I-V [14, 15, 16, 17],
noise [18, 19, 20], and microwave resonance [21, 22, 23] measurements. More
recently, some intricate properties of the magnetic-field-induced WC were
probed in NMR [24], bilayer commensurability [25], and tunneling [26]
experiments. However, some of most basic and fundamental characteristics of
this exotic, quantum state of matter are yet to be established.

A fundamental parameter that is intimately linked to the formation and defect
energies of the WC is the activation energy ($E_{A}$) that determines its low-
temperature conduction. Numerical calculations indicate that the formation
energy of the high-field WC is minimized when the crystal makes use of the
composite fermion correlations, as opposed to the traditional, Hartree-Fock
electron crystal [27, 28, 29, 30]. However, the experimental $E_{A}$,
determined from the temperature dependence of the sample resistance at very
low fillings [31, 17, 32], were about 10 times smaller than the theoretical
predictions. Later calculations, that considered the energies associated with
defects in a composite fermion WC, shown in Fig. 1(b) [33], reduced the
discrepancy between theory and experiments [34], but the theoretical defect
energy still remained a factor of $\sim$3 larger than the experimental values.

On the experimental front, the ubiquitous disorder present in 2DESs masks the
intrinsic energy gap and contributes significantly to a reduction of $E_{A}$.
Disorder causes the WC to lose long-range correlations and break into domains
[35, 37, 36, 38]. With less disorder, larger domains can be realized, allowing
for a closer comparison between theory and experiments. A recent breakthrough
[39, 40] in the quality (purity) of dilute 2DESs confined to modulation-doped
GaAs quantum wells has resulted in an improvement of the 2DES mobility by more
than a factor of 10 over previous samples [31, 17, 32], enabling us to probe
the physics of more ideal (less disordered) WCs at very low fillings. We
report here $E_{A}$ measurements made on these ultra-pure 2DESs. The data show
a remarkable quantitative agreement with theoretical calculations, suggesting
the presence of a composite fermion WC with non-classical, exotic defects.

Figure 1:  (a) Longitudinal resistance ($R_{xx}$) vs. magnetic field ($B$) for
a 2DES of density 10$\times 10^{10}$ cm-2, confined to a 50-nm-wide GaAs
quantum well at $T\simeq$ 30 mK in the filling range ($2/3>\nu>1/3)$. (b)
Different defect configurations of a composite fermion WC considered in
calculations by Archer et al. [33]. A vacancy is shown on far left, edge and
center interstitials in the center (top and bottom), and the hyper-correlated
bubble defect on the far right. (c-f) $R_{xx}$ vs. $1/\nu$ for four different
densities. The positions of emerging FQHSs at $\nu=2/11$ and 3/17 are marked.
Note that the scales are the same in all four panels, but the traces were
taken at different temperatures, as indicated. (g) $R_{xx}$ vs. $1/\nu$ for
the four different densities in a semi-log plot at $T\simeq$ 130 mK.

We performed activation energy measurements in four different ultra-pure
samples with densities $n$ = 2.8, 4.4, 7.0 and 10, in units of $10^{10}$ cm-2
which we use throughout the manuscript. The extremely high quality 2DESs are
realized by confining electrons to a GaAs quantum well bounded by AlxGa1-xAs
stepped barriers of varying $x$, and placing Si dopants inside a doping-well
structure on either side of the quantum well [39, 40]. The samples’ well
widths are 89, 70, 58, and 50 nm, respectively, and their mobilities range
from $\simeq 15\times 10^{6}$ for the lowest density, to $\simeq 36\times
10^{6}$ cm2/Vs for the highest density sample. We performed electrical
transport measurements in the van der Pauw geometry, with alloyed In:Sn
serving as contacts at the corners and center edges of 4$\times$4 mm2 samples.
The samples were cooled in a dilution refrigerator and transport experiments
were carried out using low-frequency, lock-in techniques.

Figure 2:  (a) $R_{xx}$ at $\nu=1/6$ vs. 1/$T$ shown in an Arrhenius plot. The
slope of each fits gives the activation energy gap ($E_{A})$. (b) $E_{A}$ vs.
1/$\nu$ for the four different densities in the filling factor range of 6.0
$\leq 1/\nu\leq$ 7.2. (c) Current-voltage ($I-V$) data for the $n=2.8$ sample
at $B$ = 7.4 T (1/$\nu=6.3$) and $T\simeq$ 67 mK.

Figure 1(a) displays a representative trace of the longitudinal resistance
($R_{xx}$) vs. magnetic field ($B$) for a 2DES with $n$ = 10, measured at
$T\simeq 30$ mK. The exceptional quality of the 2DES is evident from the rich
features displayed in the filling range $2/3>\nu>$ 1/3. We see fractional
quantum Hall states (FQHSs) up to $\nu$ = 11/23 and 11/21, corresponding to
the standard Jain sequence for 2-flux CFs on the electron and hole side of
$\nu$ = 1/2, respectively [41]. We also observe emergent unconventional FQHSs
between $\nu$ = 1/3 and 2/5, at filling factors $\nu$ = 3/8, 4/11, 5/13 and
6/17. These very fragile states between the conventional Jain states can be
viewed as FQHSs of interacting CFs [42, 43], and are seen only in the highest
quality samples [39, 46, 44, 45].

Figures 1(c-f) show a part of the high-field traces for the four samples
between the inverse filling factor (1/$\nu$) values of 5.0 and 5.9. Note that
the full $R_{xx}$ scale is 20 M$\Omega$, four orders of magnitude larger than
in the low-field trace shown in Fig. 1(a). The 2DESs in the range $1/\nu>5.2$
exhibit a strongly insulating behavior characteristic of the high-field WC
phases at very small fillings [10, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22,
23]. To measure resistances of the order of 10 M$\Omega$ and larger, we used a
pre-amplifier (SR551) with an input impedance $\gtrsim$ 1 T$\Omega$. The
driving $ac$ current (0.05 nA) and frequency (0.2 Hz) were kept sufficiently
small to avoid non-linear effects, and to maximize the in-phase (resistive)
component of the signal. In Figs. 1(c-f) traces, we see a clear minimum at
$\nu=2/11$ and a minimum or an inflection at $\nu=3/17$, suggesting competing
FQHSs amidst the strong, insulating WC background. Emerging FQHSs deep in the
insulating regime, competing against the insulating, WC solid phase have been
reported for $1/5>\nu>1/6$, and at even a smaller filling of $\nu=1/7$ in very
high quality samples [47, 46, 48]. Note that, in the older samples where
$E_{A}$ was measured [31, 17], none of the high-field FQHSs past $\nu=1/5$
were reported, further attesting to the ultra-high quality of our new 2DESs.

The resistance scale is kept uniform in Figs. 1(c-f). As seen in these
figures, the temperatures at which similar magnitude resistances are attained
for the same $\nu$ are very different for the four samples. Figure 1(g) shows
$R_{xx}$, presented in a log-scale vs. $1/\nu$, for different densities at $T$
$\simeq$ 130 mK. Inflections and local minima, corresponding to the Jain FQHSs
at $\nu=2/11$, 3/17, 2/13, and 1/7 can be seen in some of the traces. Near
$\nu=1/6$, where we have data for all four densities, $R_{xx}$ changes by over
four orders of magnitude from the lowest density to the highest, indicative of
the extremely insulating nature of the samples at these fillings.

Figure 2(a) displays representative Arrhenius activation plots for our
samples. The activation energy, $E_{A}$ is determined from the temperature
dependence of $R_{xx}$, given by the relation $R_{xx}\propto
e^{E_{A}/2k_{B}T}$, where $k_{B}$ is the Boltzmann constant. The data shown in
Fig. 2(a) are obtained at $\nu=1/6$. The data show excellent agreement with
the expected activated behavior over a resistance change of about two decades
for each density. As we move to higher densities, the slope clearly increases,
reflecting a higher activation energy. We repeat this analysis, and obtain
$E_{A}$ for the four different densities in a range of small fillings as shown
in Fig. 2(b).

Figure 3:  Activation energy ($E_{A}$) vs. $\sqrt{B}$ for experimental and
theoretical activation data. The circles are our experimental data points
extracted from measurements on ultra-pure samples. The dotted line is a least-
squares-fit through all the experimental data points. The squares and stars
represent previous experimental data reported in Refs. [31] and [17],
respectively. The solid curves represent theoretical values of the CFWC defect
energy $E_{D}$ reported in Ref. [33]. The yellow band highlights the range of
the calculated $E_{A}$ in the filling range 6.0 $\leq 1/\nu\leq$ 7.2 for each
density, and the orange solid line represents $E_{A}$ calculated for
$1/\nu=6.5$.

Figures 2(b) and 3 contain our main findings. We summarize the extracted
$E_{A}$ for all four samples as a function of 1/$\nu$ (Fig. 2(b)) and
$\sqrt{B}$ (Fig. 3). In both figures, it is clear that the energy gaps are
generally larger at higher densities. The relevant energy scale in the high-
field WC is the Coulomb energy given by the expression,
$E_{C}=e^{2}/4\pi\epsilon\epsilon_{0}l_{B}$, where $l_{B}=\sqrt{\hbar/eB}$. In
an ideal WC, $E_{A}$ (at a fixed filling) would then be expected to have a
$\sqrt{B}$ dependence. Our experimental data overall scale reasonably well
with $\sqrt{B}$, as seen from the dotted line in Fig. 3 which is a least-
squares fit through all the data points. However, there are deviations from
the expected behavior, which we will address later in the manuscript.

Before discussing our measured $E_{A}$ quantitatively, we make a few remarks
regarding the disorder in our samples. The high-field WC phase is generally
considered to be broken into domains because of the ubiquitous disorder
present in the 2DES [35, 36, 37, 38, 49]. The domain size provides a measure
of the correlation length of the WC and its long-range order. Very recent
microwave resonance experiments on samples of quality similar to those we use
in our study report an estimated domain size of $\sim$ 1000 electrons per
domain for $n=4.4$ [50]. This estimate is corroborated by our non-linear
current-voltage I-V measurements [51]. As shown in Fig. 2(c), for our sample
with $n=2.8$, we observe a voltage threshold of $E_{th}\simeq 4$ mV at
$1/\nu\simeq 6.3$, corresponding to an electric field threshold of
$E_{th}\simeq$ 1 V/m. Using the expression $L_{0}^{2}\simeq$
$0.02e/4\pi\epsilon\epsilon_{0}E_{th}$ [52], we find the number of electrons
in a domain, $nL_{0}^{2}$ $\sim$ 600\. Since the number of electrons in a
domain should scale with density, we estimate that the domains contain $\sim$
2000 electrons for our highest-density sample. This is approximately 13 times
larger than in previously reported samples [31, 17, 16, 22, 23]. In Ref. [17],
e.g., the threshold electric field for the onset of non-linear I-V was $\simeq
16$ V/m at $1/\nu\simeq 6.3$ for a sample with $n=11$, implying $\sim$ 150
electrons in a typical WC domain. The much larger domains in our samples
render them more ideal for quantitative comparisons with defect energy models
of the intrinsic WCs that do not contain disorder.

The magnitude of the measured activation energies presented in Fig. 3 are
indeed remarkably consistent with theoretical predictions made for the
intrinsic defect formation energies ($E_{D}$) of an exotic WC. Several
calculations in the clean limit indicate that the WC state at high magnetic
fields is a composite fermion WC (CFWC) rather than a Hartree-Fock electron
crystal [27, 28, 29, 30]. The 2DES takes advantage of the CF correlations to
minimize the energy and form a crystal. Archer et al. [33] also considered the
intrinsic defects in an otherwise pristine CFWC, and calculated their $E_{D}$.
The lowest $E_{D}$ should correspond to the transport gaps measured in
experiments [53]. Archer et al. [33] found that a unique, non-classical,
hyper-correlated bubble defect (see the defect configuration in the far right
of Fig. 1(b)) has the lowest $E_{D}$.

In Fig. 3 we provide a comparison between our measured $E_{A}$ values and
$E_{D}$ calculated in Ref. [33] for the hyper-correlated bubble defect. The
four theoretical curves in Fig. 3 are for the four densities of our samples,
and are based on calculated $E_{D}$ in the filling factor range
$1/6\leq\nu\leq 1/7.2$. . These correspond to 4-flux CFWCs with an additional
vortex attached to the bubble defect. The yellow “band” in Fig. 3 highlights
the range of the calculated $E_{D}$ for $1/6\leq\nu\leq 1/7.2$ [53]. Note
that, at a given $\nu$, the calculated $E_{D}$ follows a $\sqrt{B}$ dependence
and its center (along the y-axis) follows the $\sqrt{B}$ dependence and
intercepts the axes at the origin, as expected for an ideal system with no
disorder (see orange line in Fig. 3). The overall agreement between theory and
experimental gaps is impressive, especially considering the fact that there
are no adjustable parameters in making the comparison. Particularly noteworthy
is the fact that the experimental data closely follow the calculations that
are performed for the clean limit. This reflects the exceptionally high
quality of the new samples, and is consistent with the extremely large WC
domain sizes which are estimated to contain $\sim$ 1000 electrons (or CFs).

It is worth emphasizing that theoretical studies favor the CFWC over the
(Hartree-Fock) electron WC [27, 28, 29, 30]. Moreover, the lowest defect-
formation energy predicted for the Hartree-Fock WC is about a factor of 3
larger than that of the CFWC [33]. Quantitatively, our experimentally measured
$E_{A}$ values are clearly in much better agreement with the calculations for
a CFWC. The observation of developing FQHSs at $\nu=2/11$, 3/17, 2/13, and
1/7, which can be understood as the integer quantum Hall states of 6-flux CFs,
also demonstrates the presence of CFs and supports the formation of a CFWC
near these fillings.

In our data, we do find deviations from theory for the high-density [54] and
low-density samples. In the low-density case, disorder might be a culprit. It
has been theoretically shown that disorder plays a significant role in
lowering $E_{A}$ [37, 38, 55]. In Fig. 3, we also show previous experimental
data reported in Refs. [31] and [17]. The activation energies are lower by a
factor of 2 to 3 compared to our new ultra-pure 2DESs and theoretical
calculations that do not include disorder. This can be attributed to the much
greater disorder and the ensuing smaller WC domains in the old samples,
leading to large deviations in $E_{A}$ from what one would expect in the
disorder-free limit. In Fig. 3, the smaller $E_{A}$ we measure in our low-
density samples, compared to $E_{A}$ predicted for the clean limit, can also
be related to a reduction of the domain size in our most dilute samples.

Archer et al. [33] also report a reduction in $E_{A}$ due to finite (non-zero)
layer thickness of the 2DES. Our lowest-density samples have the largest well
widths (89 and 70 nm), and would therefore be most affected. (To minimize
interface roughness and inter-subband scattering, our samples are fabricated
with as large of a GaAs well width as possible, while ensuring that the upper
electric subband is not occupied [39, 40]). Lastly, the calculations also
ignore the role of Landau level mixing. Since Landau level mixing scales as
1/$\sqrt{B}$, one would again expect the largest reduction in $E_{A}$ for the
most dilute samples. It is possible that more rigorous calculations that
include finite layer thickness and Landau level mixing quantitatively would
lead to an even better agreement with the measured gaps.

In summary, we report energy gap measurements on several ultra-pure, dilute
2DESs, in the very small Landau level filling ($\nu<1/5$) range, where the
2DES is deep in the insulating phase that is the hallmark of the field-induced
WC. Thanks to the world-record-high purity of our samples, the WC domains are
$\sim$ 10 times larger than in previous samples, and contain $\sim$ 1,000
electrons, rendering the new samples much closer to the ideal, zero-disorder
case. Our measured gaps are about a factor of three larger than those
previously reported for lower quality samples. The new gaps agree remarkably
well with calculations for the lowest-energy, intrinsic, correlated defects in
a WC made of flux-electron CFs, rather than defects in a Hartree-Fock electron
crystal. The observation of developing FQHSs in the vicinity of the WC crystal
fillings also demonstrates the formation of CFs, and provides further evidence
that the crystal is composed of CFs as opposed to bare electrons. Finally, we
remark that, while new experimental studies are shedding light on the periodic
structure of the WC, both at zero field [10] and high magnetic fields [56],
the energy gaps reported here provide a very fundamental, macroscopic
parameter which is intimately linked to the microscopic details of a quantum
WC, namely its quasi-particle ingredients and intrinsic defects.

We acknowledge support by the U.S. Department of Energy Basic Energy Office of
Science, Basic Energy Sciences (Grant No. DEFG02-00-ER45841) for measurements.
For sample characterization, we acknowledge support by the National Science
Foundation (NSF) Grant No. DMR 2104771 and, for sample synthesis, NSF Grants
No. ECCS 1906253 and the Gordon and Betty Moore Foundation’s EPiQS Initiative
(Grant No. GBMF9615 to L.N.P.). This research is funded in part by a QuantEmX
travel grant from ICAM and the Gordon and Betty Moore Foundation through Grant
GBMF9616. A portion of this work was performed at the National High Magnetic
Field Laboratory (NHMFL), which is supported by the NSF Cooperative Agreement
No. DMR-1644779 and the state of Florida. We thank A. Bangura, D. Graf, G.
Jones, T. Murphy, and R. Nowell at NHMFL for technical support. We also thank
J. K. Jain for illuminating discussions.

## References

  * [1] E. Wigner, On the Interaction of Electrons in Metals, Phys. Rev. 46, 1002 (1934).
  * [2] B. Tanatar and D. M. Ceperley, Ground state of the two-dimensional electron gas, Phys. Rev. B 39, 5005 (1989).
  * [3] Claudio Attaccalite, Saverio Moroni, Paola Gori-Giorgi, and Giovanni B. Bachelet, Correlation Energy and Spin Polarization in the 2D Electron Gas, Phys. Rev. Lett. 88, 256601 (2002).
  * [4] N. D. Drummond and R. J. Needs, Phase Diagram of the Low-Density Two-Dimensional Homogeneous Electron Gas, Phys. Rev. Lett. 102, 126402 (2009).
  * [5] C. C. Grimes and G. Adams, Evidence for a Liquid-to-Crystal Phase Transition in a Classical, Two-Dimensional Sheet of Electrons, Phys. Rev. Lett. 42, 795 (1979).
  * [6] J. Yoon, C. C. Li, D. Shahar, D. C. Tsui, and M. Shayegan, Wigner Crystallization and Metal-Insulator Transition of Two-Dimensional Holes in GaAs at B = 0, Phys. Rev. Lett. 82, 1744 (1999).
  * [7] M. S. Hossain, M. K. Ma, K. A. Villegas Rosales, Y. J. Chung, L. N. Pfeiffer, K. W. West, K. W. Baldwin, and M. Shayegan, Observation of spontaneous ferromagnetism in a two-dimensional electron system, , Proc. Natl. Acad. Sci. U.S.A. 117, 32244 (2020).
  * [8] You Zhou, Jiho Sung, Elise Brutschea, Ilya Esterlis, Yao Wang, Giovanni Scuri, Ryan J. Gelly, Hoseok Heo, Takashi Taniguchi, Kenji Watanabe, Gergely Zaránd, Mikhail D. Lukin, Philip Kim, Eugene Demler, and Hongkun Park, Bilayer Wigner crystals in a transition metal dichalcogenide heterostructure, Nature 595, 48 (2021).
  * [9] Tomasz Smoleński, Pavel E. Dolgirev, Clemens Kuhlenkamp, Alexander Popert, Yuya Shimazaki, Patrick Back, Xiaobo Lu, Martin Kroner, Kenji Watanabe, Takashi Taniguchi, Ilya Esterlis, Eugene Demler, and Ataç Imamoğlu, Signatures of Wigner crystal of electrons in a monolayer semiconductor, Nature 595, 53 (2021).
  * [10] M. Shayegan, Wigner crystals in flat band 2D electron systems, Nat. Rev. Phys. 4, 212 (2022).
  * [11] J. Falson, I. Sodemann, B. Skinner, D. Tabrea, Y. Kozuka, A. Tsukazaki, M. Kawasaki, K. von Klitzing, and J. H. Smet, Evidence for a Liquid-to-Crystal Phase Transition in a Classical, Competing correlated states around the zero-field Wigner crystallization transition of electrons in two dimensions, Nat. Mater. 21, 311 (2022).
  * [12] Md. S. Hossain, M. K. Ma, K. A. Villegas-Rosales, Y. J. Chung, L. N. Pfeiffer, K. W. West, K. W. Baldwin, and M. Shayegan, Anisotropic Two-Dimensional Disordered Wigner Solid, Phys. Rev. Lett. 129, 036601 (2022).
  * [13] For an early review, see M. Shayegan, in Perspectives in Quantum Hall Effects, edited by S. D. Sarma and A. Pinczuk (Wiley, New York, 1997), pp. 343 - 383.
  * [14] R. L. Willett, H. L. Stormer, D. C. Tsui, L. N. Pfeiffer, K. W. West, M. Shayegan, M. Santos, and T. Sajoto, Current-voltage discontinuities in high-quality two-dimensional electron systems at low Landau-level filling factors, Phys. Rev. B 40, 6432 (1989).
  * [15] V. J. Goldman, M Santos, M Shayegan, and J. E. Cunningham, Evidence for two-dimentional quantum Wigner crystal, Phys. Rev. Lett. 65, 2189 (1990).
  * [16] F. I. B. Williams, P. A. Wright, R. G. Clark, E. Y. Andrei, G. Deville, D. C. Glattli, O. Probst, B. Etienne, C. Dorin, C. T. Foxon, and J. J. Harris, Conduction threshold and pinning frequency of magnetically induced Wigner solid, Phys. Rev. Lett. 66, 3285 (1991).
  * [17] H. W. Jiang, H. L. Stormer, D. C. Tsui, L. N. Pfeiffer, and K. W. West, Magnetotransport studies of the insulating phase around $\nu$= 1/5 Landau-level filling, Phys. Rev. B 44, 8107 (1991).
  * [18] Yuan P. Li, T. Sajoto, L. W. Engel, D. C. Tsui, and M. Shayegan, Low-frequency noise in the reentrant insulating phase around the 1/5 fractional quantum Hall liquid, Phys. Rev. Lett. 67, 1630 (1991).
  * [19] Yuan P. Li, D. C. Tsui, L. W. Engel, M. B. Santos, T. Sajoto, and M. Shayegan, RF resonance in two-dimensional electron transport in the Wigner solid regime, Solid State Commun. 96, 379 (1995).
  * [20] Yuan P. Li, D.C. Tsui, L.W. Engel, M.B. Santos, and M. Shayegan, Inductive anomaly of two-dimensional electrons in the Wigner solid regime, Solid State Commun. 99, 255 (1996).
  * [21] E. Y. Andrei, G. Deville, D. C. Glattli, F. I. B. Williams, E. Paris, and B. Etienne, Observation of a Magnetically Induced Wigner Solid, Phys. Rev. Lett. 60, 2765 (1988).
  * [22] P. D. Ye, L. W. Engel, D. C. Tsui, R. M. Lewis, L. N. Pfeiffer, and K. West, Correlation Lengths of the Wigner-Crystal Order in a Two-Dimensional Electron System at High Magnetic Fields, Phys. Rev. Lett. 89, 176802 (2002).
  * [23] Y. P. Chen, G. Sambandamurthy, Z. H. Wang, R. M. Lewis, L. W. Engel, D. C. Tsui, P. D. Ye, L. N. Pfeiffer, and K. W. West, Melting of a 2D quantum electron solid in high magnetic field, Nat. Phys. 2, 452 (2006).
  * [24] L. Tiemann, T. D. Rhone, N. Shibata, and K. Muraki, NMR profiling of quantum electron solids in high magnetic fields, Nat. Phys. 10, 648 (2014).
  * [25] H. Deng, Y. Liu, I. Jo, L. N. Pfeiffer, K. W. West, K. W. Baldwin, and M. Shayegan, Commensurability Oscillations of Composite Fermions Induced by the Periodic Potential of a Wigner Crystal, Phys. Rev. Lett. 117, 096601 (2016).
  * [26] Joonho Jang, Benjamin M. Hunt, Loren N. Pfeiffer, Kenneth W. West, and Raymond C. Ashoori, Sharp tunnelling resonance from the vibrations of an electronic Wigner crystal, Nat. Phys. 13, 340 (2017).
  * [27] H. Yi and H. A. Fertig, Laughlin-Jastrow-correlated Wigner crystal in a strong magnetic field, Phys. Rev. B 58, 4019 (1998).
  * [28] R. Narevich, G. Murthy, and H. A. Fertig, Hamiltonian theory of the composite-fermion Wigner crystal, Phys. Rev. B 64, 245326 (2001).
  * [29] Chia-Chen Chang, Gun Sang Jeon, and Jainendra K. Jain, Microscopic Verification of Topological Electron-Vortex Binding in the Lowest Landau-Level Crystal State, Phys. Rev. Lett. 94, 016809 (2005).
  * [30] A. C. Archer, K. Park and J. K. Jain, Competing Crystal Phases in the Lowest Landau Level, Phys. Rev. Lett. 111, 146804 (2013).
  * [31] R. L. Willett, H. L. Stormer, D. C. Tsui, L. N. Pfeiffer, K. W. West, and K. W. Baldwin, Termination of the series of fractional quantum hall states at small filling factors, Phys. Rev. B 38, 7881 (1988).
  * [32] M. A. Paalanen, R. L. Willett, R. R. Ruel, P. B. Littlewood, K. W. West, and L. N. Pfeiffer, Electrical conductivity and Wigner crystallization, Phys. Rev. B 45, 13784 (1992).
  * [33] A. C. Archer and J. K. Jain, Quantum bubble defects in the lowest-Landau-level crystal, Phys. Rev. B 90, 201309(R) (2014).
  * [34] Archer et al. [33] find that the defect that has the lowest formation energy, and is therefore relevant to the experimentally measured energy gaps, is a hyper-correlated bubble defect, as schematically shown in Fig. 1(b), far right. For completeness, in Fig. 1(b) we are including other (simpler) defects that were also considered in Ref. [33].
  * [35] D. S. Fisher, B. I. Halperin, and R. Morf, Defects in the two-dimensional electron solid and implications for melting, Phys. Rev. B 20, 4692 (1979).
  * [36] I. M. Ruzin, S. Marianer, and B. I. Shklovskii, Pinning of a two-dimensional Wigner crystal by charged impurities, Phys. Rev. B 46, 3999 (1992).
  * [37] Min-Chul Cha and H. A. Fertig, Orientational Order and Depinning of the Disordered Electron Solid, Phys. Rev. Lett. 73, 870 (1994).
  * [38] Min-Chul Cha and H. A. Fertig, Topological defects, orientational order, and depinning of the electron solid in a random potential, Phys. Rev. B 50, 14368 (1994).
  * [39] Yoon Jang Chung, K. A. Villegas Rosales, K. W. Baldwin, P. T. Madathil, K. W. West, M. Shayegan, L. N. Pfeiffer, Ultra-high-quality two-dimensional electron systems, Nat. Mater. 20, 632 (2021).
  * [40] Yoon Jang Chung, A. Gupta, K. W. Baldwin, K. W. West, M. Shayegan, and L. N. Pfeiffer, Understanding limits to mobility in ultrahigh-mobility GaAs two-dimensional electron systems: 100 million cm2/Vs and beyond, Phys. Rev. B 106, 075134 (2022).
  * [41] J. K. Jain, Composite Fermions (Cambridge University Press, Cambridge, England, 2007).
  * [42] Arkadiusz Wójs, Kyung-Soo Yi, and John J. Quinn, Fractional quantum Hall states of clustered composite fermions, Phys. Rev. B 69, 205322 (2004).
  * [43] Sutirtha Mukherjee, Sudhansu S. Mandal, Ying-Hai Wu, Arkadiusz Wójs, and Jainendra K. Jain, Enigmatic 4/11 State: A Prototype for Unconventional Fractional Quantum Hall Effect, Phys. Rev. Lett. 112, 016801 (2014).
  * [44] W. Pan, K. W. Baldwin, K. W. West, L. N. Pfeiffer, and D. C. Tsui, Fractional quantum Hall effect at Landau level filling $\nu$ = 4/11, Phys. Rev. B 91, 041301(R) (2015).
  * [45] N. Samkharadze, I. Arnold, L. N. Pfeiffer, K. W. West, and G. A. Csáthy, Observation of incompressibility at $\nu$ = 4/11 and $\nu$ = 5/13, Phys. Rev. B 91, 081109(R) (2015).
  * [46] Yoon Jang Chung, D. Graf, L. W. Engel, K. A. Villegas Rosales, P. T. Madathil, K. W. Baldwin, K. W. West, L. N. Pfeiffer, and M. Shayegan, Correlated States of 2D Electrons near the Landau Level Filling $\nu$=1/7, Phys. Rev. Lett. 128, 026802 (2022).
  * [47] W. Pan, H. L. Stormer, D. C. Tsui, L. N. Pfeiffer, K. W. Baldwin, and K. W. West, Transition from an Electron Solid to the Sequence of Fractional Quantum Hall States at Very Low Landau Level Filling Factor, Phys. Rev. Lett. 88, 176802 (2002).
  * [48] V. J. Goldman, M. Shayegan, and D. C. Tsui, Evidence for the Fractional Quantum Hall State at $\nu$ = $\frac{1}{7}$, Phys. Rev. Lett. 61, 881 (1988).
  * [49] R. Chitra, T. Giamarchi, and P. Le Doussal, Dynamical Properties of the Pinned Wigner Crystal, Phys. Rev. Lett. 80, 3827 (1998).
  * [50] Lili Zhao, Wenlu Lin, Yoon Jang Chung, Adbhut Gupta, Kirk W. Baldwin, Loren N. Pfeiffer, and Yang Liu, Dynamic Response of Wigner Crystals, Phys. Rev. Lett. 130, 246401 (2023).
  * [51] P. T. Madathil, K. A. Villegas Rosales, Y. J. Chung, K. W. West, K. W. Baldwin, L. N. Pfeiffer, L. W. Engel and M. Shayegan, Moving Crystal Phases of a Quantum Wigner Solid in an Ultra-High-Quality 2D Electron System, Phys. Rev. Lett. 131, 236501 (2023).
  * [52] H. Fukuyama and P. A. Lee, Dynamics of the charge-density wave. I. Impurity pinning in a single chain, Phys. Rev. B 17, 535 (1978).
  * [53] The relevant theoretical expression in Ref. [33] for activation is $R_{xx}\propto e^{E_{D}/k_{B}T}$. We therefore scale the theoretical values by a factor of two to compare to the experimental results.
  * [54] For the two high-density samples, the measured $E_{A}$ exhibit a faster rise with $B$ compared to the calculations. While we do not fully understand this behavior, we remark that at the highest fields, $R_{xx}$ for these samples reach extremely high values (exceeding $\sim 10^{7}\Omega$). This limits the temperature range over which $R_{xx}$ exhibits an activated behavior, making it difficult to extract accurate values for $E_{A}$.
  * [55] S. T. Chui and K. Esfarjani, Finite-Temperature Transport of a Pinned 2D Electron Lattice, Phys. Rev. Lett. 66, 652 (1991).
  * [56] Yen-Chen Tsui, Minhao He, Yuwen Hu, Ethan Lake, Taige Wang, Kenji Watanabe, Takashi Taniguchi, Michael P. Zaletel and Ali Yazdani, Direct observation of a magnetic field-induced Wigner crystal, arxiv:2312.11632.


# One to Multiple Mapping Dual Learning: Learning Multiple Sources from One
Mixing Signal

Ting Liu, Wenwu Wang, Xiaofei Zhang, Jingwen Yan, and Yina Guo Ting Liu,
Xiaofei Zhang, Jingwen Yan, and Yina Guo are with the Department of
Electronics and Communication Engineering, Taiyuan University of Science and
Technology, Taiyuan, 030024, China. Corresponding author’s e-mail:
<EMAIL_ADDRESS>Wenwu Wang is with the Centre for Vision, Speech and Signal
Processing, University of Surrey, Guildford, Surrey GU2 7XH, U.K. Ting Liu and
Wenwu Wang contributed equally to this work. (e-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>Jingwen Yan is with the Guangdong Provincial Key
Laboratory of Digital Signal and Image Processing Technology.

###### Abstract

Single channel blind source separation (SCBSS) refers to separating multiple
sources from a mixing signal collected by a single sensor. Existing methods
for SCBSS mainly focus on separating two sources and have limited
generalization performance. To address these problems, an algorithm is
proposed in this paper to separate multiple sources from a mixture by
designing a parallel dual generative adversarial Network (PDualGAN) that can
build the relationship between a mixture and the corresponding multiple
sources to achieve one-to-multiple cross-domain mapping. This algorithm can be
applied to any mixing model such as the instantaneous mixing model and
convolutive mixing model. In addition, datasets for one-to-multiple source
separation are created which include the mixtures and corresponding sources
for this study. Experiments were performed on four different datasets
including both one-dimensional and two-dimensional signals. Experimental
results show that the proposed algorithm can achieve high performance in peak
signal-to-noise ratio (PSNR) and correlation, which outperforms state-of-the-
art algorithms.

###### Index Terms:

Single channel blind source separation, parallel dual generative adversarial
network, one-to-multiple mapping, dual learning.

##  I Introduction

Single channel blind source separation (SCBSS) aims to separate multiple
sources from a mixture collected by a sensor which has a variety of
applications in speech [1], image [2] and EEG signal processing [3],
biomedical engineering[4][5][6], multi-source phase retrieval[7][8][9], and
eddy current pulsed thermography [10], such as observe and analyze the
anisotropies in the cosmic microwave background (CMB) radiation and recover
the CMB component as accurately as possible from a noisy mixture. The
challenge of SCBSS is that only a single-channel mixture is available,
besides, there is no prior knowledge, and both the sources and mixing matrixes
are unknown, so this problem is extremely underdetermined which leads to SCBSS
being a very pathological and difficult problem.

Traditional methods like design the optimal filter (such as Wiener
filter)[12][13], empirical mode decomposition and independent component
analysis (EMD-ICA) [14][15], ensemble empirical mode decomposition and
independent component analysis (EEMD-ICA) [16][17], non-negative matrix
factorization (NMF)[18][19] have been proposed to solve the problem of SCBSS.
These methods are better for processing instantaneous mixing signals. However,
filter based methods have high complexity and are not suitable for non-
stationary random signals, the IMFs component obtained by the EMD and EEMD is
prone to modal aliasing, ICA assumes that the signals are independent of each
other, and NMF may ignore the phase. Recently, deep learning networks have
seen increasing popularity as a solution to solving this problem. Auto-
encoders (AEs)[37] have been proposed as an approach to supervised source
separation, but AEs rely on the assumption of an output probability density.
Singing Voice Separation generative adversarial networks (SVSGAN) [38] with a
time-frequency masking function is proposed for separating voice sources.
Wasserstein-GAN [20] and two-stage approach [39] are proposed for speech
separation, which has a high computational cost. Evolving multi-resolution
pooling convolutional neural network (E-MRP-CNN) [40] is proposed for monaural
singing voice separation (MSVS). Synthesis-decomposition (S-D) [21], and
single channel blind deconvolution algorithm based on deep convolution
generating adversarial network (DCSS) [22] are proposed for separating black
and white images such as minist images from a mixture, but they are not
suitable for processing color images. In addition, these methods have limited
performance in separating multiple sources(more than three sources) and
generalization, and they mainly focus on processing single structure signals.

In real life, the signal captured by sensor could be degraded by a number of
conditions. For example, the electroencephalogram (EEG) signals are often
mixed with interference signals of electrocardiogram (ECG), electromyography
(EMG), and eye movement artifacts (EOG). The recorded remote sensing images
are often obscured by clouds and fog. Due to the erosion of time, the loss of
ancient Chinese characters often occurs in unearthed cultural relics and
ancient books. Fingerprint images obtained from criminal investigation scenes
often have multiple fingerprints overlapping each other. In a noisy acoustic
environment, multiple speakers are talking simultaneously, which raises the
challenge of separating multiple sources from a mixing signal. Although many
methods have been proposed, separating multiple sources from a single channel
mixture remains an open challenge.

The aim of this paper is to address the problem of single channel blind source
separation of multiple signals that can be applied to both one-dimensional and
two-dimensional signals based on instantaneous mixing model and convolutive
mixing model, and improve its generalization performance.

Inspired by dual learning and GAN [23] which can be used to build one-to-one
mapping to achieve image-to-image translation, we design a parallel dual
generative adversarial network (PDualGAN) to achieve one-to-multiple mapping,
and formulate SCBSS as a data conversion problem in different domains where
multiple sources are separated from a mixture in terms of the mapping between
a mixture and the corresponding sources. Our novel contributions are as
follows:

  1. 1.

Model. We have formulated a unified model for the instantaneous mixing model
and convolutive mixing model.

  2. 2.

PDualGAN algorithm. A new algorithm is proposed by introducing parallel dual
generative adversarial network (PDualGAN), where the model training is
performed using the mixtures and corresponding multiple sources. The stability
of the network and the quality of the generated data are improved by applying
the Wasserstein GAN gradient penalty (WGAN-GP) loss function, thereby
potentially enhancing source separation.

Our algorithm can be applied to both one-dimensional and two-dimensional
signals. In addition, different mixtures are used as experimental data to test
the effectiveness and generalization performance.

  3. 3.

Datasets. New datasets are created for this study which is composed of two
parts: the mixtures and corresponding original sources. Each mixture is
generated by using randomly generated mixing matrixes and multiple sources
with different weights. The datasets created could be valuable for researchers
working in the area of source separation and signal processing.

This paper is organized as follows: Section II describes the background.
Section III builds a one-to-multiple model for the SCBSS problem. Section IV
presents our PDualGAN algorithm for the problem of SCBSS. Section V shows the
experimental results. Section VI concludes the paper and draws future research
directions.

##  II Background

As we all know, training on datasets of smaller size while maintaining nearly
the same performance would be very beneficial. ‘Less than one’ shot learning
[24] can learn _N_ new classes given only _M_ $\textless$ _N_ examples with
the help of distance-weighted soft-label prototype k-Nearest Neighbors
(SLaPkNN).

Definition 1: A hard label is a vector of length _N_ , indicating that the
membership of a data point is exactly one of the _N_ classes.

Definition 2: A soft label is a vector representation of a data point’s
simultaneous membership to multiple classes.

Definition 3: Soft label prototype (SLaP) is a pair of vectors (_X_ , _Y_),
where _X_ is a feature (or position) vector, and _Y_ is an associated soft
label.

In classical _k_ nearest neighbor (kNN)[25], only hard labels are used. When a
space is partitioned using the kNN classification rule, it is clear that at
least _n_ points are required to separate _n_ classes. SLaPkNN is a
combination of SLaP and kNN, and it generalizes the soft label prototype of
kNN, which can be applied to separate _N_ classes effectively by using only
_M_ soft label prototypes, where _M_ $\textless$ _N_. For example, a SLaPkNN
classifier is fitted on two soft-label prototypes and partitions the space
into three classes, as shown in Fig. 1.

Figure 1: The soft label distribution of each prototype is illustrated by the
pie charts. Figure 2: Working process of DualGAN for image-to-image
translation. Image _u_ $\in$ _U_ is translated to domain _V_ using _G_ A.
Similarly, _v_ $\in$ _V_ is translated to _U_ using _G_ B.

The ‘less than one’ shot learning method requires labels in training, and
therefore, it is not suitable for processing mixing signals. Inspired by this
method, however, we develop an unsupervised mapping method to establish the
relationship between the unlabeled and unpaired data.

A generative adversarial network (GAN) is a deep learning model and one of the
most promising methods for unsupervised learning in recent years. The model
produces good output through adversarial learning of the generator and the
discriminator [26]. However, the original GAN algorithm is not suitable for
one-to-one conversion tasks between paired data.

Inspired by GAN and dual learning [27] from natural language translation, a
dual generative adversarial Network (DualGAN)[23] is developed for one-to-one
unlabeled data from two domains. The original GAN has the limitation that it
can only learn to translate data from domain _U_ to those in domain _V_ , but
the DualGAN can learn to invert the task.

Given two sets of unpaired and unlabeled data selected from domains _U_ and V,
firstly, the task of the DualGAN [23] is to learn a generator GA: _U_
$\rightarrow$ _V_ , a mapping relation is built from _u_ $\in$ _U_ to _v_
$\in$ _V_ , secondly, the dual task is to train an inverse generator _G_ B:
_V_ $\rightarrow$ _U_ , and a mapping relation is built from _v_ $\in$ _V_ to
_u_ $\in$ _U_. This is realized with two GANs which have the same structure.
The original GAN learns the generator _G_ A and discriminator _D_ A that
discriminates between the fake and real data of domain V. Similarly, the dual
GAN learns the generator _G_ B and a discriminator _D_ B. The overall working
process is shown in Fig. 2. 1 The _u_ $\in$ _U_ is translated to domain _V_
with _G_ A. The fitting degree of _G_ A(_u_ , _z_) (_z_ is a random noise) is
evaluated by _D_ A, Then, _G_ A(_u_ , _z_) is translated back to domain _U_
with _G_ B, and output _G_ B(_G_ A(_u_ , _z_), _z_ ′) (_z_ ′ is also random
noise) as a reconstruction of _u_. Similarly, _v_ $\in$ _V_ is translated to
domain _U_ as _G_ B(_v_ , _z_ ′) with _G_ B, and then reconstructed as
GA(GB(_v_ , _z_ ′), _z_) with _G_ A. The discriminator _D_ A is trained with
_v_ as true data and _G_ A(_u_ , _z_) as fake data, however, _D_ B takes _u_
as true and _G_ B(_v_ , _z_ ′) as fake. The generators _G_ A and _G_ B are
trained and optimized to output fake samples to cheat the corresponding
discriminators _D_ A and _D_ B, and to minimize the reconstruction error
$\parallel\emph{v}-$_G_ A(_G_ B(_v_ , _z_ ′),_z_))$\parallel$ and
$\parallel$_u_ \- _G_ B(_G_ A(_u_ , _z_), _z_ ′))$\parallel$.

The cross-entropy loss function of the original GAN [26] is substituted by the
loss function of Wasserstein GAN (WGAN)[28], which performs better in
generator convergence and data quality, and in improving the stability of the
network. The loss function applied in _D_ A and _D_ B can be written as

$l_{A}^{d}\left({u,v}\right)={D_{A}}\left({{G_{A}}\left({u,z}\right)}\right)-{D_{A}}\left(v\right),\
$ (1)
$l_{B}^{d}\left({u,v}\right)={D_{B}}\left({{G_{B}}\left({v,z^{\prime}}\right)}\right)-{D_{B}}\left(u\right),\
$ (2)

where _u_ $\in$ _U_ and _v_ $\in$ _V_.

The same loss function is applied in generators GA and GB as they have the
same objective which adopts _L_ 1 distance to measure the reconstruction
losses, it can be written as

$\displaystyle{l^{g}}\left({u,v}\right)$
$\displaystyle={\lambda_{U}}||u-{G_{B}}\left({{G_{A}}\left({u,z}\right),z^{\prime}}\right)||$
(3)
$\displaystyle+{\lambda_{V}}||v-{G_{A}}\left({{G_{B}}\left({v,z^{\prime}}\right),z}\right)||$
$\displaystyle-
D_{B}\left({{G_{B}}\left({v,z^{\prime}}\right)}\right)-{D_{A}}\left({{G_{A}}\left({u,z}\right)}\right)$

where $\lambda_{U}$ and $\lambda_{V}$ are both constant parameters, which are
typically set to a value within [100, 1000].

Clearly, by training DualGAN, unlabeled and unpaired data can be converted
from _U_ to the corresponding data in _V_ because the data of the two domains
have some similar characteristics. Inspired by DualGAN, single channel blind
source separation can be realized by converting the mixtures to the
corresponding multiple sources which have similar characteristics. In this
paper, we develop a PDualGAN to train multiple DualGAN simultaneously, and
convert the mixtures to corresponding multiple sources using the PDualGAN
algorithm.

##  III Mathematical Model

The instantaneous mixing model and the convolutive mixing model used in single
channel blind source separation are given below.

  1. 1.

Instantaneous mixing model.

For instantaneous mixing model, the observed mixture _x_(_t_) can be defined
as

$x(t)=\sum\limits_{i=1}^{N}{{a_{i}}\left(t\right)}{s_{i}}\left(t\right),\ $
(4)

where _x_(_t_) is a observed mixture, _s_ i(_t_) is the _i_ th source, _a_
i(_t_) is the _i_ th mixing matrix. _i_ =1, 2, …, _N_.

  2. 2.

Convolutive mixing model.

For convolutive mixing model, the observed mixture _x_(_t_) can be defined as

$x(t)=\sum\limits_{i=1}^{N}{{a_{i}}\left(t\right)}*{s_{i}}\left(t\right),\ $
(5)

where _x_(_t_) is a observed mixture, _s_ i(_t_) is the _i_ th source, _a_
i(_t_) is the _i_ th mixing matrix. _i_ =1, 2, …, _N_ , * denotes a
convolution operation which can be defined as

${a_{i}}\left(t\right)*{s_{i}}\left(t\right)=\sum\limits_{i=1}^{N}{{a_{i}}\left({t-v}\right)}{s_{i}}\left(v\right),\
$ (6)

where _t_ is the amount of displacement of a$(-\emph{v})$. So (5) can be
described as

$x\left(t\right)=\sum\limits_{i=1}^{N}{{a_{i}}(t-v)}{s_{i}}(v).\ $ (7)

We can see from (4) and (7) that both the instantaneous mixing model and the
convolutive mixing model can be regarded as a form of matrix multiplication,
and a mixing signal corresponds to multiple sources. The aim of this paper is
to find the one-to-multiple mapping relationship between the mixtures and the
corresponding multiple sources.

##  IV PDualGAN Algorithm

Figure 3: The architecture of PDualGAN algorithm based on the instantaneous
mixing model and convolutive mixing model, including the training of PDualGAN
(a), where _N_ DualGAN are trained simultaneously, which learn the mapping
between the mixture _x_(_t_) $\in$ _U_ and the corresponding sources _s_
1(_t_) $\in$ _V 1_, _s_ 2(_t_) $\in$ _V 2_, …, _s_ N(_t_) $\in$ _V N_
sequentially, each DualGAN learns one of these mappings. The testing of
PDualGAN (b), where mixtures are generated by random mixing matrixes and same
sources of different weights (different from the mixing weights of training).
_x_(_t_) are used as the input for estimating the _N_ sources
${\bar{s}}$1(_t_), ${\bar{s}}$2(_t_), …, ${\bar{s}}_{N}$(_t_) as the output.
Figure 4: Working process of each DualGAN. ${\bar{s}}$i(_t_)$\in$ _V i_ is the
fake data generated by generator _G_ ${}_{A_{i}}$ from the mixture
_x_(_t_)$\in$ _U_ , ${\bar{x}}$(_t_)$\in$ _U_ is reverse generated data by
generator _G_ ${}_{B_{i}}$. Similarly, ${\hat{x}}$(_t_)$\in$ _U_ is the fake
data generated by generator _G_ ${}_{B_{i}}$ from the sources _s i_(_t_)$\in$
_V i_, ${\hat{s}}$i(_t_)$\in$ _V i_ is reverse generated data by generator _G_
${}_{A_{i}}$. _x_(_t_) (i.e. _u_), _s_ i(_t_) (i.e. _v_ i), ${\bar{x}}$(_t_)
(i.e. _G_ ${}_{B_{i}}$(_G_ ${}_{A_{i}}$(_u_ , _z_), _z_ ′)), ${\bar{s}}$i(_t_)
(i.e. _G_ ${}_{A_{i}}$(_u_ , _z_)), ${\hat{x}}$(_t_) (i.e. _G_
${}_{B_{i}}$(_v_ i, _z_ ′)), ${\hat{s}}$i(_t_) (i.e. _G_ ${}_{A_{i}}$(_G_
${}_{B_{i}}$(_v_ ′, _z_ ′), _z_))

In this section, we present a PDualGAN algorithm to address the problem of
separating multiple sources from a mixture based on (4) and (7) which is
applied to one-dimensional and two-dimensional signals. As shown in Fig. 3, we
transform the problem of SCBSS into a data conversion problem in different
domains by using PDualGAN to train _N_ DualGAN simultaneously. The first step
is to train the PDualGAN with the mixtures (instantaneous mixtures or
convolutive mixtures) and corresponding sources. Mixtures are generated by
using randomly generated mixing matrixes and multiple sources with different
weights (i.e. different sources have different proportions), and the mixing
matrixes satisfy the Gaussian or normal distribution. The second step is to
test the effect of the trained model with different mixtures of different
weights of the same sources.

In our algorithm, no specific domain knowledge or pre-trained domain
representation is needed, but the mapping relationship between unpaired
signals is searched. The reconstruction error measures the disparity between
the original signals and the reconstructed signals. PDualGAN contains _N_
DualGANs, which are of the same structure as shown in Fig. 4. The mixing
signals are sampled from _U_ and the corresponding sources are sampled from
_V_ 1, _V_ 2, …, _V_ N respectively, The primary task of our PDualGAN is to
build the one-to-multiple mapping from _x_(_t_)$\in$ _U_ to _s_ 1(_t_), _s_
2(_t_), …, _s_ N(_t_) $\in$ _V 1_, _V 2_,…, _V N_.

Network configuration. Our PDualGAN including _N_ DualGANs of the same
structure, each DualGAN has identical network architecture for _G_
${}_{A_{i}}$ and _G_ ${}_{B_{i}}$(_G_ ${}_{A_{i}}$ and _G_ ${}_{B_{i}}$ are
the generators of the _i_ th DualGAN, _i_ =1, 2, …, _N_). The generator has
the equal number of upsampling and downsampling layers, with skip connections
between them, forming a U-shaped net [29][30], and such a structure enables
low-level information to be shared between the input and output. In addition,
_z_ and _z_ ′ are provided only in the form of dropout and applied to multiple
layers of generators at both training and testing phases, but they are not
explicitly provided. For discriminators, the Markovian PatchGAN architecture
[31] is applied, which is effective in capturing local high-frequency features
(such as texture in an image), and its effectiveness has been verified on
various conversion tasks [27]. Furthermore, it requires fewer parameters, and
therefore runs faster than conventional GAN. This scheme will be used for our
signal separation tasks.

We train the discriminators ncritic steps, then one step on generators. The
number of critic iterations per generator iteration ncritic can be set to 2-4,
$\lambda$U and $\lambda$${}_{V_{i}}$ are all set to 1000, an initial learning
rate is set to 0.00005, and the batch size is assigned as 1. The clipping
parameter is set in [0.01, 0.1], and the epoch is set to 20000.

Training. As the momentum-based methods (such as Adam) would occasionally lead
to instability, we use mini-batch stochastic gradient descent (SGD) and apply
the RMSProp solver which is known to perform well on highly nonstationary
problems [28]. The sigmoid cross-entropy loss of the traditional GAN is
locally saturated and may cause the gradient to disappear. However,
Wasserstein GAN gradient penalty (WGAN-GP)[28] loss is differentiable almost
everywhere, resulting in a better discriminator which can provide more
reliable gradient information.

As shown in Fig. 3 (a) and Fig. 4, for a mixture, a generator _G_
${}_{A_{i}}$: _U_ $\rightarrow$ _V_ i in the _i_ th DualGAN is learned by
mapping the mixture _u_ to a generated corresponding source _G_
${}_{A_{i}}$(_u_ , _z_), while the dual task is to train an inverse generator
_G_ ${}_{B_{i}}$: _V_ i $\rightarrow$ _U_ that maps a generated source _G_
${}_{A_{i}}$(_u_ , _z_) to a generated mixture _G_ ${}_{B_{i}}$(_G_
${}_{A_{i}}$(_u_ , _z_), _z_ ′), where _z_ and _z_ ′ are random noise signals.
_N_ corresponding sources are simultaneously generated by _N_ DualGAN (i.e.
$i=1,2,...,\emph{N}$) from the same mixture with the identical structure.

For an original source, a generator _G_ ${}_{B_{i}}$: _V_ i $\rightarrow$ _U_
in the _i_ th DualGAN is learned by mapping the source _v_ i to a generated
mixture _G_ ${}_{B_{i}}$(_v_ i, _z_ ′), while the dual task is to train an
inverse generator _G_ ${}_{A_{i}}$: _U_ $\rightarrow$ _V_ i that maps a
generated mixture _G_ ${}_{B_{i}}$(_v_ i, _z_ ′) to a generated source _G_
${}_{A_{i}}$(_G_ ${}_{B_{i}}$(_v_ i, _z_ ′), _z_), where _z_ and _z_ ′ are
random noises. _N_ different sources generate the same mixture by _N_ DualGAN
(i.e. $i=1,2,...,\emph{N}$) with the identical structure.

The discriminator _D_ ${}_{A_{i}}$ discriminates the real source _v_ i of
domain _V_ i and the fake outputs _G_ ${}_{A_{i}}$(_u_ , _z_). The
discriminator _D_ ${}_{B_{i}}$ discriminates the real mixture _u_ of domain
_U_ and the fake outputs _G_ ${}_{B_{i}}$(_v_ i, _z_ ′)(_D_ ${}_{A_{i}}$ and
_D_ ${}_{B_{i}}$ are the discriminators of the _i_ th DualGAN, _i_ =1, 2, …,
_N_).

The same loss function is applied in each DualGAN for generators _G_
${}_{A_{i}}$ and _G_ ${}_{B_{i}}$ which is defined as

$\displaystyle{l^{{G_{i}}}}\left({u,{v_{i}}}\right)$
$\displaystyle={\lambda_{U}}||u-{G_{{B_{i}}}}\left({{G_{{A_{i}}}}\left({u,z}\right),z^{\prime}}\right)||$
(8)
$\displaystyle+{\lambda_{V_{i}}}||{v_{i}}-{G_{{A_{i}}}}\left({{G_{{B_{i}}}}\left({{v_{i}},z^{\prime}}\right),z}\right)||$
$\displaystyle-
D_{{B_{i}}}\left({{G_{{B_{i}}}}\left({{v_{i}},z^{\prime}}\right)}\right)-{D_{{A_{i}}}}\left({{G_{{A_{i}}}}\left({u,z}\right)}\right),$

where _u_ $\in$ _U_ , _v_ i $\in$ _V_ i, and $\lambda$U, $\lambda$${}_{V_{i}}$
are two constant parameters which are set to a value within [100.0, 1000.0].

The corresponding loss functions applied in _D_ ${}_{A_{i}}$ and _D_
${}_{B_{i}}$ are defined as:

$l_{{A_{i}}}^{D}\left({u,{v_{i}}}\right)={D_{{A_{i}}}}\left({{G_{{A_{i}}}}\left({u,z}\right)}\right)-{D_{{A_{i}}}}\left({{v_{i}}}\right),\
$ (9)
$l_{{B_{i}}}^{D}\left({u,{v_{i}}}\right)={D_{{B_{i}}}}\left({{G_{{B_{i}}}}\left({{v_{i}},z^{\prime}}\right)}\right)-{D_{{B_{i}}}}\left(u\right),\
$ (10)

where _u_ $\in$ _U_ , and _v_ i $\in$ _V_ i.

The one-to-multiple mapping between a mixture _u_ $\in$ _U_ and _N_ sources
_v_ 1, _v_ 2, …, _v_ N $\in$ _V_ 1, _V_ 2, …, _V_ N, respectively, is built by
training the proposed PDualGAN.

Testing. As shown in Fig. 3 (b), after training the proposed PDualGAN, we save
the trained model and parameters which are then used in the test stage. The
mixtures are passed through the trained model and converted to corresponding
multiple sources to achieve separation of multiple sources from the mixture.

Algorithm 1 PDualGAN Algorithm

0: The mixture _u_ $\in$ _U_ , original sources _v_ i $\in$ _V_ i, _i_ = 1, 2,
…, _N_ , $\lambda$U, $\lambda$${}_{V_{i}}$, initial learning rate, batch size,
clipping parameter.

0: Estimated sources _v_ i $\in$ _V_ i, _i_ = 1, 2, …, _N_.

1: Each mixture _X_(_t_) is mapped to _N_ corresponding sources _s_ 1(_t_),
_s_ 2(_t_), …, _s_ N(_t_) by _N_ generators _G_ ${}_{A_{1}}$, _G_
${}_{A_{2}}$, …, _G_ ${}_{A_{N}}$, and _N_ original sources are mapped to the
same mixture by _N_ generators _G_ ${}_{B_{1}}$, _G_ ${}_{B_{2}}$, …, _G_
${}_{B_{N}}$ simultaneously. The parallel results are saved.

2: Optimize the loss function of generators _G_ ${}_{A_{i}}$ and _G_
${}_{B_{i}}$ in each DualGAN.
$\begin{array}[]{l}{l^{{G_{i}}}}\left({u,{v_{i}}}\right)={\lambda_{U}}||u-{G_{{B_{i}}}}\left({{G_{{A_{i}}}}\left({u,z}\right),z^{\prime}}\right)||\\\
+{\lambda_{{V_{i}}}}||{v_{i}}-{G_{{A_{i}}}}\left({{G_{{B_{i}}}}\left({{v_{i}},z^{\prime}}\right),z}\right)||\\\
-D_{{B_{i}}}\left({{G_{{B_{i}}}}\left({{v_{i}},z^{\prime}}\right)}\right)-{D_{{A_{i}}}}\left({{G_{{A_{i}}}}\left({u,z}\right)}\right)\end{array}$

3: Optimize the loss function of discriminators _D_ ${}_{A_{i}}$ and _D_
${}_{B_{i}}$ in each DualGAN.
$l_{{A_{i}}}^{D}\left({u,{v_{i}}}\right)={D_{{A_{i}}}}\left({{G_{{A_{i}}}}\left({u,z}\right)}\right)-{D_{{A_{i}}}}\left({{v_{i}}}\right)$
$l_{{B_{i}}}^{D}\left({u,{v_{i}}}\right)={D_{{B_{i}}}}\left({{G_{{B_{i}}}}\left({{v_{i}},z^{\prime}}\right)}\right)-{D_{{B_{i}}}}\left(u\right)$

4: end.

##  V Numerical Experiments

In this section, the experiment is conducted to demonstrate the performance of
the proposed PDualGAN algorithm, both one-dimensional and two-dimensional
signals are used in the experiment. Besides, take the separation of four
sources from a mixture as an example, we compare the PDualGAN algorithm with
state-of-the-art baseline algorithms.

Experimental data. We use four kinds of datasets for the experiment: the NWPU-
occlusion image datasets, the ancient Chinese character occlusion (ACC-
occlusion) datasets, the speech datasets, and the EEG datasets. We select a
total of 2000 original sources, each dataset includes 500 sources, mixtures
are generated by using randomly generated mixing matrixes and four sources
with different weights. Each dataset includes two types: (1) instantaneous
mixtures and corresponding sources; (2) convolutive mixtures and corresponding
sources.

Instantaneous mixtures are generated by using randomly generated mixing
matrixes and four sources with different weights. The convolutional mixed
signal is the product of the four sources with different weights and the
randomly generated mixing matrixes after a reversal and shift. The mixing
matrixes satisfy the Gaussian or normal distribution.

[NWPU-occlusion image datasets] A NWPU-occlusion image dataset including the
original images selected from the NWPU-RESISC
dataset111https://hyper.ai/datasets/5449 containing 45 categories and cloud
and fog occlusion images captured by us from Google and Baidu search engines.
Each mixing image is generated by mixing two NWPU-RESISC images and two cloud
and fog occlusion images in different random weights.

[ACC-occlusion datasets] An ACC-occlusion dataset including the ancient
Chinese character images and occlusion images was built. Each mixing image is
obtained by mixing one ancient Chinese character image and three occlusion
images in different random weights.

[Speech datasets] The speech signals are selected from the THCHS-30
dataset222http://www.openslr.org/resources/18/data with each sentence
containing 16384 points randomly clipped and then normalized. Each mixing
signal is obtained by mixing four randomly selected speech signals in
different random weights. The THCHS-30 dataset was recorded by a single carbon
microphone in a quiet office environment, where the signals were sampled at 16
kHz and quantized in 16 bits. Most of the speakers are college students who
can speak fluent Mandarin. In total, 1000 recordings are taken, which have a
total duration of more than 30 hours.

[EEG datasets] The EEG acquisition equipment used in the experiment is the EEG
and evoked potential meter (model NCERP-T-240) of Shanghai Nuocheng Electric
Co., Ltd. We use 24-channel silver-plated electrodes, and the electrode
placement position adopts the international standard 10/20. An attention
device using the TGAM chip was developed to test the attention of the
participants, where a pre-defined threshold _p_ can be used to detect the
attention of the participant. If it is higher than the pre-defined threshold
_p_ (e.g. _p_ $\textgreater$ 60), it indicates that the participant has a good
focus on the auditory stimuli, and at this time, EEG signals can be collected
by the device.

TABLE I: The performence of our algorithm as compared with state-of-the-art algorithms on four different datasets. Algorithm | Datasets
---|---
NWPU-occlusion | ACC-occlusion | Speech | EEG
PSNR | Correlation | PSNR | Correlation | Correlation | Correlation
Instantaneous model based algorithms |  |  |  |  |  |
EMD-ICA | 7.91 | 0.2198 | 7.73 | 0.1583 | 0.1779 | 0.1113
EEMD-PCA-ICA | 8.36 | 0.2730 | 8.84 | 0.2947 | 0.1188 | 0.1332
CEEDMAN-ICA | 7.50 | 0.1666 | 7.61 | 0.2015 | 0.1349 | 0.1564
SSA-ICA | 8.92 | 0.2568 | 8.85 | 0.2768 | 0.1997 | 0.1173
Gaussian WGAN | 7.75 | 0.2961 | 7.92 | 0.2876 | 0.3547 | 0.1451
PDualGAN | 20.15 | 0.7212 | 24.98 | 0.7819 | 0.7413 | 0.6862
Convolutive model based algorithms |  |  |  |  |  |
Conjugate Gradieng | 4.35 | 0.1095 | 5.32 | 0.1152 | 0.1339 | 0.1812
IRLS | 4.04 | 0.1113 | 4.85 | 0.1372 | 0.1538 | 0.1430
S-D | 11.81 | 0.5259 | 13.59 | 0.7049 | 0.4520 | 0.3063
E-MRP-CNN | 9.15 | 0.4313 | 9.52 | 0.4493 | 0.6285 | 0.6031
PDualGAN | 19.38 | 0.7026 | 22.76 | 0.7792 | 0.7422 | 0.6545

A total of 50 participants are included in the EEG signal collection, of which
25 are males and 25 are females, aged between 20-40. All participants are
either teachers or students from Taiyuan University of Science and Technology
who are in good health and meet the requirements for EEG collection
experiments.

We select the EEG signals according to the positions of the EEG electrodes of
the temporal and frontal lobes and pre-processing them to remove artifacts
such as ECG and oculus, then we randomly clipped 16384 points in each channel
and normalize and convert the selected EEG signals into a multi-dimensional
matrix to construct an EEG dataset. Each mixture is obtained by mixing four
original EEG signals from the collected dataset in different random weights.

We conducted experiments on these four datasets according to the two mixing
modes. For one-dimensional and two-dimensional signals, we show the test
results of one of instantaneous mixtures and convolutive mixtures respectively
(i.e. for two-dimensional signals, We show the separation result of the
instantaneous mixtures on the NWPU-occlusion image datasets, and the
separation result of the convolutive mixtures on the ACC-occlusion datasets;
for one-dimensional signals, We show the separation result of the
instantaneous mixtures on the EEG datasets, and the separation result of the
convolutive mixtures on the Speech datasets).

Our PDualGAN algorithm is compared with the state-of-the-art algorithms on
these four kinds of signals. The four kinds of datasets are used to train our
network, all the datasets are pre-processed, each mixture corresponds to four
sources, which as the input data of the PDualGAN network. Finally, different
mixtures of four sources of different weights are selected as test signals to
demonstrate the effectiveness of the proposed algorithm.

Baseline method We compare our PDualGAN algorithm with baseline algorithms for
both instantaneous and convolutive mixing model based algorithms in separating
four sources from a mixture. Instantaneous mixing model based algorithms
including EMD-ICA (empirical mode decomposition and independent component
analysis)[15], EEMD-PCA-ICA (ensemble empirical mode decomposition and
principal component analysis and independent component analysis) [16],
CEEDMAN-ICA (complete ensemble empirical mode decomposition with adaptive
noise and independent component analysis) [32], SSA-
ICA333https://download.csdn.net/download/qq_39065549/12318104(singular
spectrum analysis and independent component analysis) [33] and Gaussian
WGAN444https://github.com/ycemsubakan/sourceseparation_misc [20] algorithms.
Convolutive mixing model based algorithms including Conjugate
Gradient555http://groups.csail.mit.edu/graphics/CodedAperture [36],
IRLS666http://groups.csail.mit.edu/graphics/CodedAperture [36], S-D
(synthesis-decomposition)777https://github.com/qiuqiangkong/gan separation
deconvolution [21], and E-MRP-CNN [40]888https://github.com/tuxzz/emrpcnn_pub
algorithms.

The implement of EMD-ICA, EEMD-PCA-ICA, and CEEDMAN-ICA are downloaded from
https://so.csdn.net/so/search?q=CEEMDAN_V00&t=doc&u=&urw and invoke emd.m,
eemd.m and ceemdan.m respectively, then processed by FastICA to find the
independent components by maximizing the statistical independence of the
dimensionality reduction IMFs, PCA is used to extract the main energy in EEMD-
PCA-ICA algorithm. For CEEDMAN-ICA algorithm, the data needs to be
preprocessed and converted to mat format. The implement of Gaussian WGAN is
achieved by running the main$\\_$timit.py file for single source separation,
and the data needs to be preprocessed and converted to wav format. For SSA-ICA
and S-D algorithms, we adjusted part of the program code so that it can input
and output four sources. The codes of EMD-ICA, EEMD-PCA-ICA, CEEDMAN-ICA, SSA-
ICA, S-D, Gaussian WGAN and E-MRP-CNN algorithms cannot be used directly to
process color images, so we process the R, G, and B channels separately. For
E-MRP-CNN algorithm, we use the mixtures and the corresponding four sources
selected from our datasets as the input data of this algorithm for training
and testing, which are preprocessed and converted to wav format. The E-MRP-CNN
algorithm only estimates two channels, one as the vocal source, and the other
as the interference source composed of the remaining sources, so the average
value is calculated according to the results of these two channels.

For the experiments on the NWPU-occlusion image datasets and ACC-occlusion
datasets, the performance of the proposed PDualGAN algorithm can be evaluated
by the peak signal-to-noise ratio (PSNR) [34] and signal correlation [35]. For
the experiments on the speech datasets and EEG datasets, the performance of
the proposed PDualGAN algorithm can be evaluated by signal correlation [35].

The PSNR can be defined as

${\mathop{\rm PSNR}\nolimits}=10{\log_{10}}\left({\frac{{{{{\mathop{\rm
MAX}\nolimits}}_{I}}^{2}}}{{{\mathop{\rm MSE}\nolimits}}}}\right),$ (11)

where MSE is the mean square error between two images with a size of _w_
$\times$ _v_. MAX represents the maximum value of an image without noise. The
MSE of images _M_ and _H_ can be written as

${\mathop{\rm
MSE}\nolimits}(M,H)=\frac{1}{{wv}}{\sum\limits_{i=0}^{w-1}{\sum\limits_{j=0}^{v-1}{(M(i,j)-H(i,j)}})^{2}}.$
(12)

The correlation coefficient of signals _X_ and _Y_ can be expressed as

$r=\frac{{\sum\limits_{i=1}^{N}{\left({{X_{i}}-\bar{X}}\right)\left({{Y_{i}}-\bar{Y}}\right)}}}{{\sqrt{\sum\limits_{i=1}^{N}{{{\left({{X_{i}}-\bar{X}}\right)}^{2}}}}\sqrt{\sum\limits_{i=1}^{N}{{{\left({{Y_{i}}-\bar{Y}}\right)}^{2}}}}}}.$
(13)

A higher PSNR value and signal correlation means a better effect of separating
multiple sources, and a more closer the estimated source to the real source.

The facilities applied to perform the experiments include Intel I9-10900X 13.7
GHz CPU, 2*NVIDIA RTX 8000 Graphics Card and 6*32 GB memory. Gaussian WGAN,
S-D, E-MRP-CNN and the proposed PDualGAN algorithms are implemented on this
facilities. EMD-ICA[15], EEMD-PCA-ICA [16], CEEDMAN-ICA [32], SSA-ICA [33] and
Conjugate Gradient [36], IRLS [36] algorithms are implemented in MATLAB. TABLE
I shows the results of separating four sources of our algorithm as compared
with the state-of-the-art algorithms in mean values of PSNR and signal
correlation on four different datasets. Clearly, the proposed PDualGAN
algorithm has better performance than the EMD-ICA, EEMD-PCA-ICA, CEEDMAN-ICA,
SSA-ICA, Gaussian WGAN, Conjugate Gradient [36], IRLS [36], S-D [21], and
E-MRP-CNN [40] algorithms.

Figure 5: Average PSNR of the proposed PDualGAN as compared with five state-of-the-art instantaneous mixing model based algorithms. Figure 6: Testing results of PDualGAN approach for instantaneous mixing NWPU-occlusion images. Each group includes three different mixtures ((a), (b) and (c)) of four same images of different random weights (different from the mixtures of training). The $\mathbf{X}$ represents the mixture, $\mathbf{s}$1, $\mathbf{s}$2, $\mathbf{s}$3, and $\mathbf{s}$4 represent the four original ground-truth sources, $\mathbf{\bar{s}}$1, $\mathbf{\bar{s}}$2, $\mathbf{\bar{s}}$3, and $\mathbf{\bar{s}}$4 represent the corresponding estimated sources.  TABLE II: Average correlation of the proposed PDualGAN as compared with five state-of-the-art convolutive mixing algorithms. Algorithm | First group | Second group | Third group
---|---|---|---
(a) | (b) | (c) | (a) | (b) | (c) | (a) | (b) | (c)
EMD-ICA | 0.2077 | 0.2098 | 0.2055 | 0.2132 | 0.2188 | 0.2153 | 0.2754 | 0.2539 | 0.2675
EEMD-PCA-ICA | 0.2769 | 0.2688 | 0.2805 | 0.1432 | 0.1179 | 0.1543 | 0.3134 | 0.3033 | 0.2955
CEEDMAN-ICA | 0.1479 | 0.1598 | 0.1455 | 0.2132 | 0.2136 | 0.2203 | 0.3754 | 0.1839 | 0.2075
SSA-ICA | 0.3112 | 0.2526 | 0.3670 | 0.2122 | 0.3433 | 0.1210 | 0.2714 | 0.2134 | 0.2865
Gaussian WGAN | 0.1689 | 0.3754 | 0.1813 | 0.2019 | 0.1983 | 0.1927 | 0.2154 | 0.2282 | 0.2396
PDualGAN | 0.9907 | 0.9201 | 0.9484 | 0.9948 | 0.8443 | 0.9587 | 0.9784 | 0.9471 | 0.9491

###  V-A PDualGAN for NWPU-occlusion images

In the first set of simulations, we evaluate the separating performance of the
proposed PDualGAN algorithm described in Algorithm 1 for instantaneous
mixtures on the NWPU-occlusion image datasets.

In the testing, we randomly select three groups of the instantaneous mixtures
and corresponding sources from NWPU-occlusion datasets, each group shows the
separating results from three different mixtures ((a), (b), (c) in Fig. 6),
and each mixture is instantaneously mixing generated by randomly selected
mixing matrixes and four same images with different random weights. Fig. 6
shows the test results of separating four images from instantaneous mixtures.
Fig. 5 shows the comparison results of average PSNR values obtained by the
proposed PDualGAN algorithm and EMD-ICA, EEMD-PCA-ICA, CEEDMAN-ICA, SSA-ICA,
Gaussian WGAN instantaneous model based algorithms. TABLE II shows the
comparison results of average correlation obtained by the proposed PDualGAN
algorithm and these state-of-the-art algorithms.

Figure 7: Testing results of PDualGAN approach for convolutive mixing ACC-
occlusion images. Each group includes three different mixtures ((a), (b) and
(c)) of four same images of different random weights. The $\mathbf{X}$
represents the mixture, $\mathbf{s}$1, $\mathbf{s}$2, $\mathbf{s}$3, and
$\mathbf{s}$4 represent the four original ground-truth sources,
$\mathbf{\bar{s}}$1, $\mathbf{\bar{s}}$2, $\mathbf{\bar{s}}$3, and
$\mathbf{\bar{s}}$4 represent the corresponding estimated sources. Figure 8:
Average PSNR of the proposed PDualGAN as compared with four state-of-the-art
convolutive model based algorithms.

As observed from Fig. 5, the average PSNR of the proposed algorithm can reach
34.24 dB, outperforming the baseline algorithms EMD-ICA, EEMD-PCA-ICA,
CEEDMAN-ICA, SSA-ICA and Gaussian WGAN of 8.64 dB, 8.42 dB, 8.22 dB, 9.25 dB,
and 7.86 dB. TABLE II demonstrates the average correlation that the PDualGAN
algorithm can reach 0.9948 which outperforms the baseline algorithms EMD-ICA,
EEMD-PCA-ICA, CEEDMAN-ICA, SSA-ICA and Gaussian WGAN of 0.2754, 0.3134,
0.2203, 0.3670 and 0.3754. Therefore, the PDualGAN algorithm achieves better
results than the baseline instantaneous mixing model based algorithms in terms
of both PSNR and correlation.

###  V-B PDualGAN for ACC-occlusion images.

In the second set of simulations, we evaluate the separating performance of
the proposed PDualGAN algorithm described in Algorithm 1 for convolutive
mixtures on the ACC-occlusion datasets.

The testing is similar to section _A_. we randomly select three groups of the
convolutive mixtures and corresponding sources from the ACC-occlusion image
datasets, each group shows the separating results from three different
mixtures ((a), (b), (c) in Fig. 7), and each mixture is convolutive generated
by random mixing matrixes and four same images with different random weights.
Fig. 7 shows the test results of separating four images from convolutive
mixtures. We compared the PDualGAN algorithm with Conjugate Gradient, IRLS,
S-D and E-MRP-CNN convolutive model based algorithms according to the
different mixtures of Fig. 7. The comparison results of average PSNR values
and average correlation obtained by these state-of-the-art algorithms are
shown in Fig. 8 and TABLE III respectively.

TABLE III: Average correlation of the proposed PDualGAN as compared with four state-of-the-art convolutive mixing algorithms. Algorithm | First group | Second group | Third group
---|---|---|---
(a) | (b) | (c) | (a) | (b) | (c) | (a) | (b) | (c)
Conjugate Gradient | 0.2395 | 0.2481 | 0.2355 | 0.1127 | 0.1628 | 0.1512 | 0.2738 | 0.2522 | 0.2152
IRLS | 0.1513 | 0.1722 | 0.1371 | 0.0921 | 0.1144 | 0.1525 | 0.2314 | 0.2594 | 0.2182
S-D | 0.7422 | 0.6088 | 0.5977 | 0.7004 | 0.7936 | 0.5142 | 0.5754 | 0.7882 | 0.7621
E-MRP-CNN | 0.5188 | 0.2935 | 0.4715 | 0.4132 | 0.4485 | 0.3718 | 0.5711 | 0.3681 | 0.3575
PDualGAN | 0.9998 | 0.9918 | 0.9673 | 0.9806 | 0.9998 | 0.8492 | 0.9998 | 0.9958 | 0.9842

Obviously, the average PSNR of the proposed algorithm can reach 38.98 dB,
outperforming the baseline algorithms Conjugate Gradient, IRLS, S-D, and
E-MRP-CNN of 5.32 dB, 4.85 dB, 13.98 dB, and 10.05 dB. TABLE III demonstrates
the average correlation that the PDualGAN algorithm can reach 0.9998 which
outperforms the baseline algorithms Conjugate Gradient, IRLS, S-D, and E-MRP-
CNN of 0.2738, 0.2594, 0.7621, and 0.5711.

###  V-C PDualGAN for speech signals.

Figure 9: Testing results of PDualGAN approach for convolutive mixing speech signals. Each group including two different mixtures((a), (b)) of four same signals of different random weights. The first column is the mixtures, the second column is the corresponding original sources $\mathbf{s}$1, $\mathbf{s}$2, $\mathbf{s}$3, and $\mathbf{s}$4, and the third column is the corresponding estimated sources $\mathbf{\bar{s}}$1, $\mathbf{\bar{s}}$2, $\mathbf{\bar{s}}$3, and $\mathbf{\bar{s}}$4. TABLE IV: Average correlation of the proposed PDualGAN as compared with four state-of-the-art convolutive mixing algorithms. Algorithm | First group | Second group
---|---|---
(a) $\mathbf{s}$i and $\mathbf{\bar{s}}$i | (b) $\mathbf{s}$i and $\mathbf{\bar{s}}$i | (a) $\mathbf{s}$i and $\mathbf{\bar{s}}$i | (b) $\mathbf{s}$i and $\mathbf{\bar{s}}$i
Conjugate Gradient | 0.3662 | 0.0733 | 0.1987 | 0.1681
IRLS | 0.1588 | 0.2045 | 0.2943 | 0.1479
S-D | 0.3548 | 0.5012 | 0.4988 | 0.4492
E-MRP-CNN | 0.6735 | 0.5466 | 0.6022 | 0.6716
PdualGAN | 0.9432 | 0.9656 | 0.7122 | 0.7041

In the third set of simulations, we evaluate the separating performance of the
proposed PDualGAN algorithm described in Algorithm 1 for convolutive mixtures
on the speech signal datasets.

The testing is similar to section _A_. We randomly select two groups of the
convolutive mixtures and corresponding sources from the Speech datasets, each
group shows the separating results from two different mixtures ((a), (b) in
Fig. 9), and each mixture is convolutive generated by four same signals with
different random weights. Fig. 9 shows the test results for convolutive mixing
signals. Similarly, we compared the PDualGAN algorithm with Conjugate
Gradient, IRLS, S-D and E-MRP-CNN algorithms according to Fig. 9. TABLE IV
demonstrates the comparison results of the correlation obtained by the four
state-of-the-art algorithms.

As shown in TABLE IV, the proposed PDualGAN algorithm can obtain an average
correlation of up to 0.9656, which is better than the baseline algorithms
Conjugate Gradient, IRLS, S-D, and E-MRP-CNN of 0.3662, 0.2943, 0.5012, and
0.6735.

TABLE V: Average correlation of the proposed PDualGAN as compared with five state-of-the-art convolutive mixing algorithms. Algorithm | First group | Second group
---|---|---
(a) $\mathbf{s}$i and $\mathbf{\bar{s}}$i | (b) $\mathbf{s}$i and $\mathbf{\bar{s}}$i | (a) $\mathbf{s}$i and $\mathbf{\bar{s}}$i | (b) $\mathbf{s}$i and $\mathbf{\bar{s}}$i
EMD-ICA | 0.0181 | 0.0124 | 0.0159 | 0.0125
EEMD-PCA-ICA | 0.0259 | 0.0242 | 0.0356 | 0.0301
CEEDMAN-ICA | 0.0948 | 0.0812 | 0.0488 | 0.0392
SSA-ICA | 0.0130 | 0.0154 | 0.0155 | 0.0101
Gaussian WGAN | 0.1780 | 0.1154 | 0.1259 | 0.1982
PdualGAN | 0.7432 | 0.7018 | 0.9743 | 0.9741
Figure 10: Testing results of PDualGAN approach for instantaneous mixing EEG
signals. Each group including two different mixtures((a), (b)) of four same
signals of different random weights. The first column is the mixtures, the
second column is the corresponding original sources $\mathbf{s}$1,
$\mathbf{s}$2, $\mathbf{s}$3, and $\mathbf{s}$4, and the third column is the
corresponding estimated sources $\mathbf{\bar{s}}$1, $\mathbf{\bar{s}}$2,
$\mathbf{\bar{s}}$3, and $\mathbf{\bar{s}}$4.

###  V-D PDualGAN for EEG signals.

In the fourth set of simulations, we evaluate the separating performance of
the proposed PDualGAN algorithm described in Algorithm 1 for instantaneous
mixtures on the EEG signal datasets.

The testing is similar to section _A_. We randomly select two groups of the
instantaneous mixtures and corresponding sources from the EEG datasets, each
group shows the separating results from two different mixtures ((a), (b) in
Fig. 10), and each mixture is instantaneously mixing generated by four same
signals with different random weights. Fig. 10 shows the test results for
instantaneous mixing signals. Similarly, we compared the PDualGAN algorithm
with EMD-ICA, EEMD-PCA-ICA, CEEDMAN-ICA, SSA-ICA and Gaussian WGAN algorithms
according to Fig. 10. TABLE V demonstrates the comparison results of the
correlation obtained by the five state-of-the-art algorithms.

Obviously, the average correlation that the PDualGAN algorithm can reach
0.9743 which outperforms the baseline algorithms EMD-ICA, EEMD-PCA-ICA,
CEEDMAN-ICA, SSA-ICA, and Gaussian WGAN of 0.0181, 0.0356, 0.0948, 0.0155 and
0.1982.

The experimental results show that the proposed PDualGAN algorithm outperforms
state-of-the-art algorithms in both PSNR and correlation on the four different
datasets for the instantaneous mixtures and the convolutive mixtures, which
shows the effectiveness of the proposed algorithm.

##  VI Conclusion

In this paper, a new algorithm for the problem of single channel blind source
separation (SCBSS) has been presented. Our contributions are as follows:

Model. We have formulated a unified model for the instantaneous mixing model
and convolutive mixing model for the research.

Algorithm. Based on the instantaneous mixing model and convolutive mixing
model, we proposed a PDualGAN algorithm. _N_ DualGAN are trained
simultaneously with mixtures and corresponding sources to realize one-to-
multiple mapping, and Wasserstein generative adversarial networks gradient
penalty(WGAN-GP) loss function is applied to improve the stability of the
network.

The proposed algorithm can be used to both one-dimensional and two-dimensional
signals, and different mixtures are applied to test the effectiveness and
generalization performance.

Datasets. We build the one-to-multiple datasets in the experiment which are
composed of two parts: the mixtures and corresponding sources. Each mixture is
generated by using randomly generated mixing matrixes and multiple sources
with different weights. These datasets can be used for related research.

Numerical experiments show that the proposed PDualGAN algorithm performs well
in separating multiple sources from instantaneous mixtures and convolutive
mixtures which outperforms the EMD-ICA, EEMD-PCA-ICA, CEEDMAN-ICA, SSA-ICA,
Conjugate Gradient, IRLS, S-D and DCSS algorithms.

## Acknowledgment

This study was supported by the China Scholarship Council under Grant
[2020]1417, the Key Research and Development Project of Shanxi Province under
Grant 201803D421035, the Natural Science Foundation for Young Scientists of
Shanxi Province under Grant 201901D211313, the Research, Teaching, and
Research Funding Project of Shanxi Province for Returned Overseas Students
under Grant HGKY2019080. The authors would like to thank the anonymous
reviewers for their constructive comments improving this article.

## References

  * [1] P. K. Krishna, K. Ramaswamy. Single channel speech separation based on empirical mode decomposition and hilbert transform. IET Signal Proc., vol. 11, no. 5, pp. 579-586, 2017.
  * [2] F. Fan, K. Yang, M. Xia, et al. Underwater image restoration by means of blind deconvolution approach. Front. Optoelectron. China, vol. 3, no. 2, pp. 169-178, 2010.
  * [3] C. J. James, S. Wang. Blind source separation in single-channel EEG analysis: an application to BCI. Annu Int Conf IEEE Eng Med Biol Proc, pp: 6544-6547, 2006.
  * [4] Y. Zheng, Z. Jiang, H. Zhang, et al. Size-scalable content-based histopathological image retrieval from database that consists of WSIs. IEEE J. Biomedical Health Informat., vol. 22, no. 4, pp. 1278-1287, 2018.
  * [5] W. P. Segars, B. M. W. Tsui, J. Cai, et al. Application of the 4-D XCAT phantoms in biomedical imaging and beyond. IEEE Trans. Med. Imaging., vol. 37, no. 3, pp. 680-692, 2018.
  * [6] D. V. Sorokin, I. Peterlik, M. Tektonidis, et al. Non-rigid contour-based registration of cell nuclei in 2-D live cell microscopy images using a dynamic elasticity model. IEEE Trans. Med. Imaging., vol. 37, no. 1, pp. 173-184, 2018.
  * [7] Y. Guo, A. Wang, W. Wang. Multi-source phase retrieval from multi-channel phaseless STFT measurements. Signal Process, vol. 144, pp. 36-40, 2018.
  * [8] Y. Guo, T. Wang, J. Li, et al. Multiple Input Single Output Phase Retrieval. Circ Syst Signal Process, vol. 38, no. 8, pp. 3818-3840, 2019.
  * [9] Y. Guo , X. Zhao , J. Li, et al. Blind Multiple Input Multiple Output Image Phase Retrieval. IEEE Trans Ind Electron, vol. 67, no. 3, pp. 2220-2230, 2020.
  * [10] B. Gao, L. Bai, W. L. Woo, et al. Automatic Defect Identification of Eddy Current Pulsed Thermography Using Single Channel Blind Source Separation. IEEE Trans. Instrum. Meas., vol. 63, no. 4, pp. 913-922, 2014.
  * [11] C. C. Wang, Y. H. Zeng. Research Status and Prospects of Underdetermined Blind Source Separation Algorithms. J Beijing Univ Posts and Telecommun. vol. 41, no. 6, pp. 103-109, 2018.
  * [12] L. S. Zhan, D. S. Huang, C. H. Zheng, et al. Blind inversion of Wiener system for single source using nonlinear blind source separation, Proc Int Jt Conf Neural Networks, vol. 2, pp. 1235-1238, 2005.
  * [13] L. D. Persia, Diego H. Milone, M. Yanagi. Pseudoanechoic blind source separation with improved Wiener postfilter. 2009.
  * [14] Z. Zhao, X. Yan. Algorithm for EMD and ICA-based Single-channel Voice Blind Source Separation. Electron Sci Technol, 2012.
  * [15] B. Mijovic, M. Vos, I. Gligorijevic, et al. Source Separation From Single-Channel Recordings by Combining Empirical-Mode Decomposition and Independent Component Analysis. IEEE Trans. Biomed. Eng., vol. 57, no. 9, pp. 2188-2196, 2010.
  * [16] Y. Guo, S. Huang, Y. Li. Single-Mixture Source Separation Using Dimensionality Reduction of Ensemble Empirical Mode Decomposition and Independent Component Analysis. Circ Syst Signal Process, vol. 31, no. 6, pp. 2047-2060, 2012.
  * [17] Y. Guo, S. Huang, Y. Li, et al. Edge Effect Elimination in Single-Mixture Blind Source Separation. Circ Syst Signal Process, vol. 32, no. 5, pp. 2317-2334, 2013.
  * [18] D. D. LEE, H. S. SEUNG. Learning the Parts of Objects by Non-Negative Matrix Factorization. Nature, vol. 401, no. 6755, pp. 788-791, 1999.
  * [19] B. Gao, W. L. Woo, S. S. Dlay. Single-Channel Source Separation Using EMD-Subband Variable Regularized Sparse Features. IEEE Trans. Audio Speech Lang. Process., vol. 19, no. 4, pp. 961-976, 2011.
  * [20] Y. C. Subakan, P. Smaragdis. Generative adversarial source separation. ICASSP IEEE Int Conf Acoust Speech Signal Process Proc, pp. 26-30, 2018.
  * [21] Q. Kong, Y. Xu, W. Wang, et al. Single-channel signal separation and deconvolution with generative adversarial networks. IJCAI Int. Joint Conf. Artif. Intell., pp. 2747-2753, 2019.
  * [22] T. Yin, T. Liu, Y. Guo. Research on Single-channel Blind Deconvolution Algorithm for Ancient Chinese Character Repair. J. Chin. Comput. Sys., vol. 42, no. 2, pp. 414-417, 2021.
  * [23] Z. Yi, H. Zhang, P. Tan, et al. DualGAN: Unsupervised Dual Learning for Image-to-Image Translation. In Proc. IEEE Int. Conf. Comput Vision, pp. 2868-2876, 2017.
  * [24] I. Sucholutsky, M. Schonlau. ¡®Less Than One¡¯-Shot Learning: Learning N Classes From M $\textless$ N Samples. arXiv preprint arXiv: 2009.08449, 2020.
  * [25] J. C. Bezdek, L. I Kuncheva,. Nearest prototype classifier designs: An experimental study. International Journal of Intelligent Systems. vol. 16, no. 12, pp. 1445-1473, 2001.
  * [26] I. Goodfellow, J. Pouget-Abadie, M. Mirza, et al. Generative adversarial nets. Adv. neural inf. proces. syst., vol. 2, pp. 2672-2680, 2014.
  * [27] Y. Xia, D. He, T. Qin, et al. Dual learning for machine translation. Adv. neural inf. proces. syst., vol. 29, pp. 820-828, 2016.
  * [28] M. Arjovsky, S. Chintala, and L. Bottou. Wasserstein gan. arXiv preprint arXiv: 1701.07875, 2017.
  * [29] P. Isola, J. Y. Zhu, T. Zhou, et al. Image-to-image translation with conditional adversarial networks. Proc. - IEEE Conf. Comput. Vis. Pattern Recognit, CVPR, pp. 5967-5976, 2017.
  * [30] O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. Lect. Notes Comput. Sci., pp. 234-241, 2015.
  * [31] M. Liu, T. Breuel, and J. Kautz. Unsupervised image-to-image translation networks. Adv. neural inf. proces. syst., pp. 700-708, 2017.
  * [32] Z. Luo, Z. Yan, F. U. Weidong, et al. Electroencephalogram Artifact Filtering Method of Single Channel EEG Based on CEEMDAN-ICA. Chinese J. Sens. Actuators, vol. 31, no. 8, pp. 1211-1216, 2018.
  * [33] S. Jabbari. Source separation from single-channel abdominal phonocardiographic signals based on independent component analysis. Biomed. Eng. Lett., vol. 11, no.1, pp. 55-67, 2021.
  * [34] Q. Huynh-Thu and M. Ghanbari, Scope of validity of psnr in image/video quality assessment. Electron. Lett., vol. 44, no. 13, pp. 800-801, 2008.
  * [35] D. R. Fuhrmann, G. S. Antonio. Transmit beamforming for MIMO radar systems using signal cross-correlation. IEEE Trans.aerosp.electron.syst, vol. 44, no. 1, pp. 1-16, 2008.
  * [36] A. Levin, R. Fergus, F. Durand, et al. Deconvolution using natural image priors. ACM T Graphic, vol. 6, pp. 0-2, 2007.
  * [37] E. M. Grais and M. D. Plumbley. Single channel audio source separation using convolutional denoising autoencoders, IEEE Glob. Conf. Signal Inf. Process., GlobalSIP - Proc., pp. 1265-1269, 2017.
  * [38] Z. Fan, Y. Lai, J.R. Jang. SVSGAN: Singing Voice Separation Via Generative Adversarial Network. ICASSP IEEE Int Conf Acoust Speech Signal Process Proc, 726-730, 2018.
  * [39] Y. Sun, W. Wang, J.A. Chambers, et al. Two-Stage Monaural Source Separation in Reverberant Room Environments Using Deep Neural Networks. IEEE ACM Trans. Audio Speech Lang. Process., 27, 125-139, 2019.
  * [40] W. Yuan, B. Dong, S. Wang, et al. Evolving Multi-Resolution Pooling CNN for Monaural Singing Voice Separation. IEEE ACM Trans. Audio Speech Lang. Process., vol. 29, pp. 807-822, 2021.