# The start of the Abiogenesis: Preservation of homochirality in proteins as a
necessary and sufficient condition for the establishment of the metabolism

Søren Toxvaerd  Department of Science and Environment, Roskilde University,
Postbox 260, DK-4000 Roskilde, Denmark

###### Abstract

Biosystems contain an almost infinite amount of vital important details, which
together ensure their life. There are, however, some common structures and
reactions in the systems: the homochirality of carbohydrates and proteins, the
metabolism and the genetics. The Abiogenesis, or the origin of life, is
probably not a result of a series of single events, but rather the result of a
gradual process with increasing complexity of molecules and chemical
reactions, and the prebiotic synthesis of molecules might not have left a
trace of the establishment of structures and reactions at the beginning of the
evolution. But alternatively, one might be able to determine some order in the
formation of the chemical denominators in the Abiogenesis. Here we review
experimental results and present a model of the start of the Abionenesis,
where the spontaneous formation of homochirality in proteins is the
precondition for the establishment of homochirality of carbohydrates and for
the metabolism at the start of the Abiogenesis.

###### keywords:

Abiogenesis Homochirality Metabolism

††journal: Journal of Theoretical Physics

##  1 The preservation of homochiral structures in millions of years in a
prebiotic aqueous environment.

There have been many proposals to the establishment of homochirality of
carbohydrates and amino acids in a prebiotic environment. It is, however, not
the establishment, but rather the preservation of homochirality, which is the
problem. Both the units of amino acids in the proteins, as well as the central
molecules, Glyceraldehyde and Glyceraldehyde-3-phosphate, in the
polymerization of carbohydrates have active isomerization kinetics [1, 2], and
this will, without other chemical constraints, give racemic compositions in
aqueous solutions [3]. Biosystems are from a physicochemical point of view,
soft condensed matter in aqueous environments, and the Abiogenesis must have
taken millions of years, so the establishment of homochirality should in fact
be readdressed to: the establishment and preservation of homochiral struc-
tures in millions of years in an aqueous environment.

The chiral composition and stability of a system of chiral units is determined
by the ratio between a decrease in entropy, and thereby an increase of Gibbs
free energy by homochiral ordering, ($\approx$ 2 kJ/mol), and the gain in
enthalpy by a chiral discrimination in favour of homochirality [4, 5, 3]. The
chiral discrimination is given by the difference in enthalpy, $\Delta
H_{\textrm{CD}}$, between the enthalpy, $\Delta H_{\textrm{f}}$ of a
homochiral system, $(L)$, and a corresponding racemic mixture, $(DL)$. The
chiral discrimination must be negative and and less than $\div$2 kJ/mol in
order overcome the loss of entropy and to establish homochiral domains. The
formation enthalpy, for for crystals, (c), $\Delta
H_{\textrm{f}}^{\ominus}(c)$, can be determine from combustion energy, and the
chiral discrimination is

$\Delta H_{\textrm{CD}}^{\ominus}(c)=\Delta
H_{\textrm{f}}^{\ominus}(c,L)-\Delta H_{\textrm{f}}^{\ominus}(c,DL).$ (1)

A chiral system in an aqueous environment and with isomerization kinetics and
chiral discrimination of its chiral units tends to a homochiral ordering at
high concentrations of its chiral units and low water activity, but racemizes
at higher water activity. The solution in a cell (cytosol) can be
characterized as an aqueous-like solution with an emulsion of proteins, and in
a recent article [3] it was argued, that the establishment of homochirality
was started and preserved in the hydrophobic core of enzyme-like proteins. The
conclusion was obtained from thermodynamic considerations and from Molecular
Dynamics simulations of a model of peptide chains in aqueous solutions. Here
we shall review and summarize experimental results which support this
hypothesis. The spontaneous formation of homochiral proteins was, however,
also a sufficient, but necessary condition for the establishment of
homochirality of the carbohydrates, and the metabolism at the start of the
Abiogenesis.

##  2 The establishment and preservation of homochiral structures in proteins.

The structures of proteins were predicted by Linus Pauling in a series of
papers in 1951, where he derived the two secondary structures of proteins, the
$\alpha-$helix [6] and the pleated $\beta-$sheet conformation [7]. The
clockwise (positive) $\alpha-$helix structure was derived from spectroscopic
data for bond lengths and angles at the planar peptide units, and with an
$\it{intramolecular}$ stabilizing hydrogen-bond between a hydrogen atom at the
substituted -N-H and an oxygen atom in a -C=O group in the helix. The energy
of the -N-H$\cdot\cdot\cdot$O=C- hydrogen bond ($\approx$ 10 kJ/mol) is only
of the order a half of the energy of a corresponding $\it{intermolecular}$
hydrogen bond [8], and this fact has important consequences. The loss of
intermolecular hydrogen bond energy by the establishment of the peptide bonds
results in an emulsion-like aqueous phase separation with compact globular
proteins, where the weaker intramolecular hydrogen bonds stabilize the
secondary protein structures.

The proposed $\alpha-$helix (Figure 2 in [6] ) is, however, not the clockwise
all L-$\alpha-$helix structure, but its mirror structure, an anticlockwise all
D-helix. [9] . This mix of notation was irrelevant for Linus Pauling and for
the contents of his article, but might be relevant for understanding the
ability to maintain homochirality in the hydrophobic compact part of the
globular protein. The two helices have the same minimum intramolecular energy
(enthalpy), and although one cannot conclude, that helix(+)- structures with a
more racemic composition and with even lower energy does not exist, it is not
likely. The enthalpy of a protein is given by the intramolecular energies,
including the energy from the intramolecular hydrogen bonds and the
”hydrophobic forces” (surface tension) from the loss of intermolecular
hydrogen bonds by the phase separation, which all together ensure the compact
core and a sufficient chiral discrimination for the homochirality. However,
some bioproteins with a few units of D-amino acids exist [10], which indicates
that homochirality in proteins, although it stabilizes the compact helical
state [11], is not a strict necessity for the establishment of the secondary
structures in proteins.

The active isomerization kinetics in proteins affects the stability of the
structure, and the aging of proteins in the cells results in appearance of
units of D-amino acids, and with the loss of higher order structures [12]. The
homochirality in proteins is in [3] predicted to be more stable at a smaller
water activity, e g. at higher ionic concentrations than in the cytosol
solution in biological cells.

TABLE I: Chiral discrimination, $\Delta H_{\textrm{CD}}^{\ominus}(c)$

|  |
---|---|---
Molecule |  $\Delta_{\textrm{CD}}H^{{}^{\ominus}}(c)$ kJ/mol | References
Alanine |  4.3 |  [13, 14, 15]
Valine |  2.4 |  [16]
Leucine |  3.4 |  [17]
Isoleucine |  -5.4 |  [18, 19]
Serine |  7.4 |  [20]
Threonine |  -12.0 |  [21]
Proline |  3.4 |  [22]
Tryptophan |  1.3 | [23, 24]

An indirect support of the hypothesis, that it is the secondary and tertiary
homochiral structures of a protein which ensure the homochirality, can be
obtained from the strength of the chiral discrimination of the individual
amino acids. An updated table with recent data for the chiral discrimination
of crystals of amino acids is given in Table I. The chiral discriminations,
$\Delta H_{\textrm{CD}}^{\ominus}(c)$, are obtained from Eq. 1 and data for
$\Delta H_{\textrm{f}}^{\ominus}(c,L)$ and $\Delta
H_{\textrm{f}}^{\ominus}(c,DL)$. Only data for less than half of the twenty
biological $\alpha$-amino acids exists. But one can, however, see from the
table that most amino acids have a positive chiral discrimination, i.e. in
favor of a racemic crystallization of the amino acids. Only two (Isoleucine
and Threonine) out of the eight amino acids have a sufficient negative chiral
discrimination to favour homochirality in solid state. In an aqueous solution
the chiral discrimination is, however, less (negative) [3].

The homochiral stability of proteins cannot be explained by a direct chiral
discriminations between units of amino acids in aqueous solutions, but must
originate from the $\alpha$-amino acids ability to perform proteins with
higher order compact and stable structures, when they are homochiral. This
property can also explain, why there is no direct relation between the
sequence of the units of L-amino acids in an enzyme, the secondary structure
and its biological function. The sequence of the units in a given enzyme
differs in biosystems, and for homologous enzymes these differences can be
used to obtain the cladistics of the biological evolution [25, 26]. The
sequence in e.g. insulin is on one hand specific for a given biological system
[27], but its tertiary conformation, and thereby the enzymatic function in the
metabolism is on the other hand maintained to a degree, that an insulin
molecule in one biosystem can function in another biosystem.

##  3 The establishment and preservation of homochiral structures in
carbohydrates.

Theformation of a compact hydrophobic core in a protein can explain the
stability of homochiral proteins [3], but one still has to explain the
formation and stability of the D- polysaccharides in a prebiotic aqueous
environment. The central chiral molecule in the formose reaction for
spontaneous synthesis of carbohydrates [28] is Glyceraldehyde, and the
corresponding key molecule in the Glycolysis is Glyceraldehyde-3-phosphate.
Both molecules have an active isomerization kinetics in an aqueous solution,

$\displaystyle\textrm{D-Glyceraldehyde-3-phosphate}\rightleftharpoons\textrm{Dihydroxyacetone
phosphate}$ $\displaystyle\textrm{Dihydroxyacetone
phosphate}\rightleftharpoons\textrm{L-Glyceraldehyde-3-phosphate}$ (2)

and this will, without other chemical constraints, give a racemic composition
at relevant biological concentrations [3]. Furthermore all living systems
contain a very effective enzyme, Triose Phosphate Isomerase, which eliminates
the activation energy in the isomerization kinetics of
Glyceraldehyde-3-phosphate [29]. This simple enzyme is believed to have been
present in LUCA (Last Universal Common Ancestor, also called the Last Common
Unicellular Ancestor) and could be among the very first enzymes in the
evolution [30, 31]. The enzyme will, without further physicochemical
constraints reinforce the formation of a racemic composition of carbohydrates
in biosystems. So the presence of this enzyme indicates, that the
homochirality of the
carbohydrates in biosystems is not obtained by discrimination, but is esta-
blished and preserved by another physicochemical mechanism.

To understand the formation and preservation of homochirality in
polysaccharides and in the metabolism, one must notice, that these chain
reactions are reversible. Take for instant the central step in the Glycolysis,

$\displaystyle\textrm{D-Fructose-1,6-diphosphate}\rightleftharpoons$
Dihydroxyacetone phosphate $\displaystyle\textrm{ +
D-Glyceraldehyde-3-phosphate}.$ (3)

Although $\Delta G^{{}^{\ominus^{\prime}}}$ = 23.97 kJ/mol of this reaction is
strongly positive, it readily proceeds in the forward direction under
intracellular conditions ($\approx 10-100\mu$M), and with a Gibbs free energy
gain, $\Delta_{r}G<0$, by cleaving Fructose. But the reverse reaction is
spontaneous at higher of concentrations of Dihydroxyacetone phosphate and
D-Glyceraldehyde 3-phosphate, and with a gain of free energy by synthesis of
Fructose and a storage of free energy in Frucose-6-phosphate, and in the first
component, Glycose-6-phosphate, in the Glycolysis.

The isomerization kinetics at the chiral carbon atom in Glyceraldehyde is,
however, not active after the condensation of Glyceraldehyde and
Dihydroxyacetone, and the original chirality is conserved by the condensation.
The pentoses and hexoses contain additional chiral carbon atoms, created by
this and other synthesis, and with a keto-enol isomerization between them. But
the chirality at carbon atom No. 5 (No. 4 position for pentoses) is not
affected by these keto-enol isomerizations. The homochiral structures are not
limited to simple D-polysaccharides, also the chains in RNA and DNA are all
D-structures of phosphate esters of Ribose and Deoxyribose, respectively.
D-Ribose-5-phosphate has a metabolism which is linked to the Glycolyse, and
altogether these facts can explain the formation and preservation of a
homochiral carbohydrate world: d.

Figure 1:  Illustration of a chain reaction with a stereospecific enzymatic
(Enzyme E${}_{\textrm{DD}}$) catalyse of one of the steps in the chain
reaction. The figure illustrates polymerization of a homochiral polymer from
chiral monomers with isomerization kinetics and a stereospecific catalyse of
the first step of the polymerization into a dimer DD. In the illustration
there is no enzymes, Enzyme E${}_{\textrm{DL}}$ and Enzyme
E${}_{\textrm{DD}}$, and therefore no polymerization into a non-homochiral
polymer.

If there at the establishment of the consecutive reactions in the metabolism
was an enzyme, which only reduced the activation energy for the reaction with
the D-structure of the carbohydrate, but not for the L-structure, then a
metabolism will drain the aqueous solution with the central units,
Glyceraldehyde-3-phosphate, for the D-conformations. The active isomerization
kinetic will convert L-Glyceraldehyde-3-phosphate to
D-Glyceraldehyde-3-phosphate and ensure a racemic equilibrium of the two
enantiomers in the aqueous solution. But the net result will be a dominance of
D-carbohydrates. Such enzymes do in fact exist. The enzyme Hexokinase
(Glucokinase), which catalyses the first step in the Glycolysis, only
catalyses phosphorylation of D-Glycose, but not L-Glycose. The enzyme
Ribokinase, which phosphorylates D-Ribose does not catalyse the corresponding
phosphorylation of L-Ribose [32]. The first step in the Glycolysis is the
stereospecific phosphorylation of D-Glycose to D-Glycose-6-phosphate. RNA and
DNA, are correspondingly phosphor polyesters, so the phosphorylation is
central for the establishment of the metabolism as well as the genetics. An
explanation of the role of the phosphorylation in the Abiogenesis is given by
Westheimer [33], who argued that the phosphorylation stabilizes the
biomolecules. A phosphorylation must have appeared at the establishment of a
metabolism. If a stereospecific proteins like Hexokinase and Ribokinase by
changes were present at the prebiotic phosphorylation of the carbohydrates, it
is sufficient to ensure the homochirality of carbohydrates in the metabolism.

An illustration of reactions with chiral objects, and with isomerization
kinetics between the chiral units and a stereoselectiv enzymatic catalyse of a
step in the chain of reactions, is shown in Figure 1. The figure illustrates a
polymerization of chiral units. A homochiral protein with a secondary
structure has a stereospecific surface, which can catalyse a chemical
reaction. In the illustration there is only a surface which catalyses the D+D
dimerisation, whereas there is no stereospecific surfaces for D+L or L+L
dimerizations. If a prebiotic environment is enriched with the monomers, in
the case of carbohydrates with Glyceraldehyde by the formose reaction, and
polymerizes, then the isomerization kinetics between the chiral units together
with a D-stereospecific enzyme somewhere in the chain reactions will ensure a
dominating D-world.

A prebiotic existence of stereospecific proteins for a reaction in the
metabolism can explain the formation and stability of the homochiral
carbohydrate world. But it also establish an order in the evolution at the
start of the Abiogenesis. The creation of a prebiotic peptide world with, by
change, a stereospecific enzymatic protein for a step in the metabolism is
sufficient for the establishment of a homochiral carbohydrate world.

Figure 2:  Illustration of an aqueous emulsion of proteins. The four proteins
are from the simulations of peptide chains with isomerization kinetics and in
aqueous solutions [3]. The activity of water in the solution is given by the
blue colour (blue: high activity; white: low activity). The units of amino
acids can have different chirality: red (L) and white (D). The peptide chain
(upper right) consists of 200 units and remained with a racemic composition
during the simulation. The peptides with longer chains, (1000 units), had a
racemic distribution of the chiral units for a high water activity (upper
left), but for lower activity the isomerization kinetics resulted in compact
proteins with homochiral compositions, but with a statistical equal amount of
proteins with either D- (lower left)or L- chirality (lower right).

##  4 The establishment and preservation of the L-protein and D-carbohydrate
world.

Linus Pauling’s anticlockwise all D-peptide helices must be the key-molecules
in a D-protein- and L-carbohydrate world, a world which not exists on planet
Earth. In [4, 5, 3] it was pointed out, that one can obtain a spontaneous
symmetry break in a racemic system and a dominance of one of the chiral units,
when the subdomain of one of the homochiral conformations percolates the
system. But the compact proteins in an aqueous emulsion are autonomous, and
the isomerization kinetics can one hand ensure homochirality in the proteins,
but will on the other hand give an statistical equal amount of homochiral
proteins with either D- or L-units. Enzymes are compact proteins with from 62
to $\approx$ 2500 amino acids. Computer simulations of simple peptide chains
in an aqueous environment and with isomerization kinetics show, that short
chains remained in an open structure with a racemic composition, whereas
longer chains had a compact structure and they were almost homochiral for low
water activity [3]. Figure 2 show four proteins: a relative short peptide
chain with 200 units (upper right) and four proteins with 1000 units. A
prebiotic aqueous emulsion of proteins and without a selective me-
chanism will contain both homochiral D- and L-proteins and not be global
homochiral.

There is, however, a crucial quality of the metabolism and $\it{catabolism}$
of carbohydrates and peptides, which can explain the emergence and
preservation of the L-protein and D-carbohydrate world only. The
L-carbohydrates are not included in the metabolism of carbohydrates, but
simple bacteria [34] and procariotes [35] contain enzymes, which catalyse a
catabolic pathway for L-glycose. The metabolism of amino acids contains
enzymes, which cleave the peptide bonds, and which are not stereosellective,
e.g. Trypsin and Pepsin, and enzymes, which include D-amino acids in the
catabolism, e.g. D-amino acid dehydrogenase [36]. The critical barrier for
growth and stability of one of the chiral worlds must be the establishment of
its metabolism and catabolism. Who came first will eat the other by the
catabolism.

##  5 The Abiogenesis.

The Abiogenesis, or the origin of life, is probably not a result of a series
of single events, but rather the result of a gradual process with increasing
complexity of molecules and chemical reactions. This article argues, however,
that there are some ”milestones” and order in the evolution of the complex
molecular biostructures and networks of chemical reactions.

One of the elements in the Abiogenesis is the establishment by time of an
aqueous emulsion of homochiral compact proteins. The homochirality can be
obtained by the isomerization kinetics and preserved by the compact secondary
and tertiary structures of the proteins in aqueous emulsions with low water
activity (e.g. high ionic concentrations) [3].

Another link in the evolution is the establishment of a metabolism. The first
step in the Glycolysis is the stereospecific phosphorylation of Glucose to
D-Glucose-6-phosphate. RNA and DNA, are correspondingly phosphor polyesters,
so the phosphorylation is central for the establishment of the metabolism as
well as the genetics. The synthesis of peptides and carbohydrates have
appeared simultaneously in the prebiotic aqueous environment. A stereospecific
phosphorylation can be obtained by proteins, which by change acted as enzymes
for phosphorylation of D-glucose and D-Ribose only and thereby established the
D-carbohydrate world. Whereas one today can find a few examples of D-amino
acid units in biosystems, L-carbohydrates does not appear. The explanation for
this lack of L-carbohydrates and dominance of peptides with units of L amino
acids today can very well be the same as the explanation for the formation and
preservation of homochiral carbohydrates given here: the existence of the
stereospecific enzymes like Hexokinase and Ribokinase together with a
stereospecific catabolism of L-carbohydrates is sufficient to establish the
homochirality in the metabolism and to remove the L-carbohydrates and proteins
with D-units.

Can this order in the evolution be verified? Perhaps not, the prebiotic world
has disappeared on planet Earth. But there is a chance that frozen ice in
cavities in the rocks beneath the evaporated oceans on planet Mars contains
prebiotic materials, which can reveal the start of the Abiogenesis.

##  6 Acknowledgment

Jeppe C Dyre and Ulf R Pedersen are gratefully acknowledged. This work was
supported by the VILLUM Foundation’s Matter project, grant No. 16515

## References

  * Bada [1972] Bada J.L., 1972. Kinetics of razimization of amino acids as a function of pH. J. Am. Chem. Soc. 94, 1371-1373.
  * Nagorski and Richard [2001] Nagorski R.W., Richard J.P., 2001. Mechanistic imperatives for aldose-ketose isomerization in water: specific, general base- and metal ion-catalyzed isomerization of glyceraldehyde with proton and hydride transfer. J. Am. Chem. Soc. 123, 794-802.
  * Toxvaerd [2017] Toxvaerd, S., 2017. The role of the peptides at the origin of life. J. Theor. Biol. 429, 164-169.
  * Toxvaerd [2000] Toxvaerd, S., 2000. Molecular dynamics simulations of isomerization kinetics in condensed fluids. Phys. Rev. Lett. 85, 4747-4750 .
  * Toxvaerd [2009] Toxvaerd, S., 2009. Origin of homochirality in biosystems. Int. J. Mol. Sci. 10, 1290-1299 .
  * Pauling et al. [1951] Pauling, L., Corey R.B., Branson, H.R., 1951. The structure of proteins: Two hydrogen-bonded helical configurations of the polypeptide chain. Proc. Natl. Acad. Sci. 37, 205-211.
  * Pauling and Corey [1951] Pauling, L., Corey R.B., 1951. The pleated sheet, a new layer configuration of polypeptide chains. Proc. Natl. Acad. Sci, 37, 251-256.
  * Wendler et al. [2001] Wendler, K., Thar, J., Zahn, S., Kirchner, B., 2010. Estimating the hydrogen bond energy. J. Phys. Chem. A 144, 9529-9536.
  * Dunitz [2001] Dunitz, J.D., 2001. Pauling’s left-handed $\alpha$-helix. Angew. Chem. Int. Ed. 40, 4167-4173.
  * Fujii [2002] Fujii, N., 2002. D-amino acids in living higher organisms. Origins Life Evold. Biosphere 32, 103-127.
  * Nanda et al. [2007] Nanda, V., Andrianarijaona, A., Narayanan, C., 2007. The role of protein homochirality in shaping the energy landscape of folding. Protein Science 16, 1667-1675.
  * Fujii et al. [2010] Fujii, N., Fujii, N., Kida, M., Kinouchi, T., 2010. Influence of L$\beta$-, D$\alpha$\- and D$\beta$-Asp isomers of the Asp-76 residue on the properties of $\alpha$A-crystallin 70-88 peptide. Amino Acids 39, 1393-1399.
  * Contineanu and Marchidan [1984] Contineanu, I.; Marchidan, D.I., 1984. The enthalpies of combustion and formation of D-alanine, L-alanine, DL-alanine, and $\beta$-alanine. Rev. Roum. Chim. 29, 43-48.
  * Kamaguchi et al. [1975] Kamaguchi, A., Sato, T., Sakiyama, M., Seki, S., 1975. Enthalpies of combusion of organic compounds. III.∗ D-, L- and DL-alanines. Bull. Chem. Soc. Jpn. 48, 3749-3750.
  * Silva et al. [2010] Ribeiro da Silva, M.A.V., Ribeiro da Silva, M.D.M.C., Santos, A.F.L.O.M., Roux, M.V., Foces-Foces, C., Notario, R., Guzmán-Mejía, R., Juaristi, E., 2010. Experimental and computational thermodynamical study of $\alpha$-alanine (DL) and $\beta$-alanine. J.Phys. Chem. B 114, 16471-16480.
  * Silva et al. [2000] Ribeiro da Silva, M.A.V., Ribeiro da Silva, M.D.M.C., Santos, L.M.N.B.F., 2000. Standard molar enthalpies of formation of crystalline L-, D- and DL-valine. J. Chem. Therm. 32, 1037-1043.
  * Hufman et al. [1937] )Huffman, H.M., Fox, S.W., Ellis, E.L., 1937. Thermal data. VII. The heats of combustion of seven amino acids. J. Am. Chem. Soc. 59, 2144-2150.
  * Wu et al. [1993] Wu, D., Zhu, Y., Gao, Z., Qu, S., 1993. Determination of combustion heat of some amino acids. Wuhan Daxue Xuebao Ziran Kexueban 78-82.
  * Vasilev et al. [1991] Vasiliév, V.P., Borodin, V.A., Kopnyshev, S.B., 1991. Calculation of the standard enthalpies of combustion and of formation of crystalline organic acids and complexones from the energy contributions of atomic groups. Russ. J. Phys. Chem. (Engl. Transl.) 65, 29-32.
  * Neacşu et al. [2014] Neacşu, A., Gheorghe, D., Contineanu, I., Tǎnǎsescu, S., 2014. A thermochemical study of serine stereoisomers. Therm. Acta 595, 1-5.
  * Contineanu et al. [2013] Contineanu, I., Neacşu, A., Gheorghe, D.,Tǎnǎsescu, S., Perişanu Ş., 2013. The thermochemistry of threonine stereoisomers. Therm. Acta 563, 1-5 (2013).
  * Filipa et al. [2014] Filipa, A., Santos, L.O.M., Notario, R., Ribeiro da Silva, M.A.V., 2014. Thermodynamic and conformational study of proline stereoisomers. J. Phys. Chem. B 118, 10130-10141.
  * Lukyanova et al. [2017] Lukyanova, V.A., Druzhinina, A.I., Pimenova, S.M., Ioutsi, V.A., Buyanovskaya, A.G., Takazova, R.U., Sagadeyev, E.V., Gimadeev, A.A., 2017. Thermodynamic properties of L-tryptophan. J. Chem. Therm. 105, 44-49.
  * Kochergina et al. [2014] Kochergina, L.A., Krutova, O.N., Damrina, K.V., 2014. Standard enthalpies of formation for L-tyrosine, DL-norleucine, DL-tryptophan, DL-$\alpha$-alanyl-DL-norleucine, and products of their dissociation in aqueous solution. Russian J. Phys. Chem. A 89, 755-758.
  * Margoliash [1963] Margoliash, E., 1963. Primary structure and evolution of cytochrome C. Proc. Nath. Acad. Sci. 50, 672-679.
  * van Thoai and Roche [1968] van Thoai, N., Roche, J., 1968. Homologous Enzymes and Biological Evolution. New York:Gordon & Breach.
  * Stretton [2002] Stretton, A.O.W., 2002. The first sequence: Fred Sanger and insulin. Genetics 162, 527-532.
  * Butlerow [1861] Butlerow, A., 1861. Formation synthetique dùne substance sucree. Compt. Rend. Acad. Sci. 53, 145–147.
  * Alberty and Knowles [1976] Alberty, W.J., Knowles, J.R., 1976. Free-energy profile for the reaction catalyzed by triosephosphate isomerase. Biochemistry 15, 5627-5631.
  * Sobolevsky et al. [2013] Sobolevsky, Y., Guimarães, R.C., Trifonov, E.N., 2013. Towards functional repertoire of the earliest proteins. J. Biomol. Struct. Dyn. 31, 1293-1300.
  * de Farias et al. [2016] de Farias, S.T., Rêgo, T.G., Jośe, M.V., 2016. A proposal of the proteome before the last universal common ancestor (LUCA). Int. J. Astrobiol. 15, 27-31.
  * Chuvikovsky et al. [2006] Chuvikovsky, D.V., Esipov, R.S., Skoblov, Y.S., Chupova, L.A., Muravyova, T.I., Miroshnikov, A.I., Lapinjoki, S, Mikhailopulo, I.A., 2006. Ribokinase from $E.$ $coli$: Expression, purification, and substrate specificity. Bioorg. Med. Chem. 14, 6327-6332.
  * Westheimer [1987] Westerner, F.H., 1987. Why nature chose phosphates. Science 235, 1173-1178.
  * Shimizu et al. [2012] Shimizu, T., Takaya, N., Nakamura, A., 2012. An L-glucose catabolic pathway in $Paracoccus$ species 43P. J. Biol. Chem. 287, 40448-40456.
  * Yoshida et al. [2014] Yoshida, H., Yoshihara, A., Teraoka, M., Terami, Y., Takata, G., Izumori, K., Kamitori, S., 2014. X-ray structure os a novel L-ribose isomerase acting on a non-natural sugar L-ribose as its ideal substarate. FEBS J. 281, 3150-3164.
  * Olsiewski et al. [1980] Olsiewski, P.J., Kaczorowski, G.J., Walsh, C., 1980. Purification and properties od D-amino acid dehydrogenase, an inducible membran-bound iron-sulfor flavoenzyme from $Escherichia$ $Coli$ $B$. J. Biol Chem. 255, 4487-4494.


# AdaWCT: Adaptive Whitening and Coloring Style Injection

###### Abstract

Adaptive instance normalization (AdaIN) has become the standard method for
style injection: by re-normalizing features through scale-and-shift
operations, it has found widespread use in style transfer, image generation,
and image-to-image translation. In this work, we present a generalization of
AdaIN which relies on the whitening and coloring transformation (WCT) which we
dub AdaWCT, that we apply for style injection in large GANs. We show, through
experiments on the StarGANv2 architecture, that this generalization, albeit
conceptually simple, results in significant improvements in the quality of the
generated images.

Index Terms—  Image-to-image translation, generative adversarial networks,
whitening and coloring transformation.

##  1 Introduction

Generative Adversarial Networks (GANs) [1] have become what is perhaps the
most important family of methods for image generation and manipulation these
past years. In particular, image-to-image translation (i2i) methods [2] seek
to learn a mapping between two related domains, thereby “translating” and
image from one domain to another while preserving some information from the
original. Here, one key idea is to enforce the model to preserve the “content”
of the image while modifying its “style”. In this context, Adaptive Instance
Normalization (AdaIN) [3] has become the standard method for style injection
in i2i and has been exploited in popular architectures such as StarGAN [4] and
StyleGAN [5].

AdaIN works by modulating the statistics of feature maps inside the network:
it first normalizes them by subtracting their mean and dividing by their
standard deviation; then it injects a (learned) style vector by the reverse of
this operation. Typically, the injected style vector is provided by a mapping
network [5]. Although this method has been used successfully in a variety of
image-to-image translation scenarios, the statistical representation of AdaIN
is limited in that it does not take into account the existing correlations
between the feature maps of the generator. However, in the style transfer
literature, the Whitening & Coloring Transformation (WCT) has become the
preferred approach precisely because of its capacity to take into account
these correlations. Yet, despite its good performance in terms of style
transfer, WCT has so far not been used for style injection.

In this work, we fill this gap by replacing AdaIN with an explicit WCT
operation for style injection in GANs. This change, dubbed “Adaptive WCT”
(AdaWCT), is a generalization of AdaIN that can be used as a drop-in
replacement for the AdaIN blocks (without any additional change) in existing
GAN architectures and we present its impact on image-to-image translation
tasks. Indeed, in conditional image generation tasks where the latent space is
intended to represent the style of the images, we find that AdaWCT helps in
disentangling style from content and significantly improves image
quality—results that we demonstrate through quantitative and qualitative
evaluations on the StarGANV2 [6] architecture.

##  2 Related work

Generative adversarial networks (GANs) [7] have demonstrated promising results
in various applications in computer vision, including image generation [5, 8,
9] and image-to-image translation [10, 11, 12, 13, 14]. Recent work has
striven to improve sample quality and diversity, notably through theoretical
breakthroughs in terms of defining loss functions which provide more stable
training [15, 16, 17] and encourage diversity in the generated images [18].
Architectural innovations also played a crucial role in these advancements [4,
19].

Also dubbed “conditional” [20], reference-guided i2i translation methods seek
to learn a mapping from a source to a target domain while being conditioned on
a specific image instance belonging to the target domain. In this case, some
methods attempt to preserve the “content” of the source image (identity, pose)
and apply the “style” (hair/skin color) of the target. Inspired by the mapping
network of StyleGAN [5], recent methods [4, 6, 10] make use of a style encoder
to extract the style of a target image and feed it to the generator, typically
via adaptive instance normalization (AdaIN) [3].

The whitening and coloring transform used in this work has also been used in
style transfer [21, 22] and self-supervised representation learning [23]. To
our knowledge, this is the first time it is being used for style injection.

##  3 Approach

Style injection aims at modifying the statistics of the activations
$\mathbf{X}\in\mathbb{R}^{C\times HW}$ at a specific layer in the generator,
where $(C,H,W)$ are (number of channels, width, height) respectively. We first
briefly review AdaIN style injection, then present our proposed AdaWCT
generalization.

###  3.1 Adaptive Instance Normalization

Adaptive Instance Normalization [3], or AdaIN, modulates the activations
$\mathbf{X}$ by

$\mathbf{\tilde{X}}_{\mathrm{AdaIN}}=\mathrm{diag}(\boldsymbol{\sigma})\mathrm{diag}(\nicefrac{{1}}{{\hat{\sigma}(\mathbf{X}))}})(\mathbf{X}-\hat{\mu}(\mathbf{X})\mathbf{1}^{\intercal})+\boldsymbol{\mu}\mathbf{1}^{\intercal}\,,$
(1)

where $\hat{\mu}(\cdot)$ and $\hat{\sigma}(\cdot)$ compute the row-wise mean
and standard deviation vectors respectively,
$\mathbf{1}\in\mathbb{R}^{HW\times 1}$ is an all-ones vector, and the
$\mathrm{diag}(\mathbf{v})$ operator creates a diagonal matrix from vector
$\mathbf{v}$. Here, $(\boldsymbol{\mu},\boldsymbol{\sigma})$ are learned
parameters.

###  3.2 Adaptive WCT Normalization

Our AdaWCT approach is a generalization of AdaIN which takes into account the
correlations between features maps and employs a whitening and coloring
transformation (WCT) for style injection. There are two main changes over
AdaIN: 1) instead of normalizing by
$\mathrm{diag}(\nicefrac{{1}}{{\hat{\sigma}(\mathbf{X}))}})$ as in eq. 1, we
compute the whitening matrix $\mathbf{W}$; 2) instead of modulating by
$\mathrm{diag}(\boldsymbol{\sigma})$, we do so with a (learned) coloring
matrix $\boldsymbol{\Gamma}$. Mathematically, eq. 1 becomes

$\mathbf{\tilde{X}}_{\mathrm{AdaWCT}}=\boldsymbol{\Gamma}\mathbf{W}(\mathbf{X}-\hat{\mu}(\mathbf{X})\mathbf{1}^{\intercal})+\boldsymbol{\mu}\mathbf{1}^{\intercal}\,.$
(2)

The following two subsections describe how we obtain the whitening and
coloring matrices $\mathbf{W}$ and $\boldsymbol{\Gamma}$ respectively.

###  3.3 Whitening matrix

The whitening matrix $\mathbf{W}=\mathbf{\Sigma}^{-\nicefrac{{1}}{{2}}}$ can
be computed from the covariance matrix of the activations

$\boldsymbol{\Sigma}=\frac{1}{HW-1}\bar{\mathbf{X}}\bar{\mathbf{X}}^{\intercal}\,,$
(3)

where
$\bar{\mathbf{X}}=\mathbf{X}-\hat{\mu}(\mathbf{X})\mathbf{1}^{\intercal}$.
Since doing so using singular value decomposition (SVD) can be unstable [24],
we instead follow [25] and employ the Newton-Schulz iterative method to
compute the whitening matrix without having to perform an eigendecomposition.
The procedure is summarized in alg. 1.

Input: Covariance matrix $\boldsymbol{\Sigma}$, number of iterations $n$,
$\epsilon>0$

Output: Whitening matrix estimate
$\mathbf{W}\approx\mathbf{\Sigma}^{\nicefrac{{1}}{{2}}}$

$\mathbf{\Sigma}\leftarrow\mathbf{\Sigma}+\epsilon\mathbf{I}$

// pre-conditioning

$\mathbf{Y_{0}}\leftarrow\nicefrac{{\boldsymbol{\mathrm{\Sigma}}}}{{\left\|\boldsymbol{\mathrm{\Sigma}}\right\|_{F}}}$

// pre-compensation

1 $\boldsymbol{\mathrm{Z_{0}}}\leftarrow\boldsymbol{\mathrm{I}}$

2 for  _$i=0:n-1$_ do

3
$\mathbf{Y}_{i+1}\leftarrow\frac{1}{2}\mathbf{Y}_{i}(3\mathbf{I}-\mathbf{Z}_{i}\mathbf{Y}_{i})$
$\mathbf{Z}_{i+1}\leftarrow\frac{1}{2}(3\mathbf{I}-\mathbf{Z}_{i}\mathbf{Y}_{i})\mathbf{Z}_{i}$

4 end for

$\mathbf{W}\leftarrow\nicefrac{{\mathbf{Z}_{n}}}{{\left\|\mathbf{\mathrm{\Sigma}}\right\|_{F}^{\nicefrac{{1}}{{2}}}}}$

// post-compensation

Algorithm 1 Newton-Schulz iterative method.

#### Pre-conditioning

In order to improve the stability of the procedure, a shrinkage operator [26]
which improves the conditioning of $\mathbf{\Sigma}$ is first applied (l. 1 in
alg. 1).

#### Pre-compensation

According to [27], the Newton-Shulz method locally converges only if
$\left\|\mathbf{I}-\mathbf{\Sigma}\right\|_{p}<1$ for $p=1,2$, or $\infty$. In
order to ensure this criterion is respected, the covariance matrix
$\mathbf{\Sigma}$ is first normalized by its Frobenius norm [28] (l. 1 in alg.
1).

#### Post-compensation

While the pre-compensation step ensures convergence, at the final step
$\mathbf{Z}_{n}\approx\left\|\mathbf{\Sigma}\right\|_{F}^{\nicefrac{{1}}{{2}}}\mathbf{\Sigma}^{-\nicefrac{{1}}{{2}}}$
instead of the whitening matrix $\mathbf{\Sigma}^{-\nicefrac{{1}}{{2}}}$. We
therefore post-compensate (l. 1 of alg. 1) by dividing with
$\left\|\mathbf{\Sigma}\right\|_{F}^{\nicefrac{{1}}{{2}}}$.

#### Uniform normalization

We found that the numerical compensation steps introduced by [25] only work in
cases where the matrix norm $\left\|\mathbf{\Sigma}\right\|_{F}$ is small.
This is because these steps amplify any error introduced by the iterative
method. We address this by pre-normalizing $\bar{\mathbf{X}}$, dividing it
with its standard deviation across all channels and pixels, just before
computing the covariance matrix (eq. 3). Since this rescaling is uniform, it
leaves the output of alg. 1 unchanged, while consistently avoiding large
matrix norms.

###  3.4 Coloring matrix

Similar to the way $(\boldsymbol{\mu},\boldsymbol{\sigma})$ are learned
parameters for AdaIN (sec. 3.1), $(\boldsymbol{\mu},\boldsymbol{\Gamma})$ are
learned in our approach by the projection

$\left(\boldsymbol{\mu},\boldsymbol{\Gamma}\right)=\mathbf{\Phi}(\mathbf{w})$
(4)

of a latent vector $\mathbf{w}$ obtained from a standard mapping network [5,
6]. Here, $\boldsymbol{\Phi}$ can be an affine projection layer as in [3].
While a coloring matrix should, strictly speaking, be symmetric positive semi-
definite, we did not find this constraint necessary or helpful in practice.

###  3.5 Group-wise whitening and coloring

Unfortunately, generating a full coloring matrix $\mathbf{\Gamma}$ creates a
parameter explosion in the generator. Take the typical example of a
256-channel feature map and 512-dimensional style vector. The total number of
parameters in the projection subnetwork $\mathbf{\Phi}$ must therefore be at
least $256\times 256\times 512\approx 33\text{M}$. Since several style
injection operations take place within a network, this quickly becomes
intractable.

We therefore employ group-wise whitening and coloring [28, 29] to reduce the
total number of parameters. The idea is to split (along the channel dimension)
the activation maps $\mathbf{X}\in\mathbb{R}^{C\times HW}$ into $n$ subgroups
$\mathbf{G}_{j}\in\mathbb{R}^{G\times HW}$, $j=\\{1,...,n\\}$, that will be
whitened and subsequently colored independently of each other. Here,
$G=\nicefrac{{C}}{{n}}$ is the number of channels in each group.

The process is summarized in alg. 2. Here, we denote
$(\boldsymbol{\mu},\mathbf{\Gamma}_{B})=\mathrm{reshape}(\mathbf{\Phi}(\mathbf{w}))$
as the group-wise projection of style vector $\mathbf{w}$, where
$\mathbf{\Gamma}_{B}$ corresponds to the output of the projection subnetwork
rearranged into a block-diagonal matrix (where each block has dimension
$G\times G$).

Note that this group-wise WCT operation discards all correlations between
activations belonging to different groups. Varying the group size $G\in[1,C]$
affects the _degree_ of WCT applied to the features within the network. If
$G=C$, full whitening and coloring is applied. Conversely, $G=1$ degrades
naturally to AdaIN (sec. 3.1).

Input: Activations $\mathbf{X}$, style vector $\mathbf{w}$, $n=\frac{C}{G}$
groups

Output: Colored activations $\mathbf{\tilde{X}}$

// compute group-wise correlation matrices

$\mathbf{G}_{j\in\\{1,\dotsc,n\\}}\leftarrow\mathbf{X}$

// split into groups

1

2for  _$j=1:n$_ do

$\mathbf{\bar{G}}_{j}\leftarrow\mathbf{G}_{j}-\hat{\mu}(\mathbf{G}_{j})\mathbf{1}^{\intercal}$
$\boldsymbol{\Sigma}_{j}\in\mathbb{R}^{G\times
G}\leftarrow(\nicefrac{{1}}{{HW-1}})\mathbf{\bar{G}}_{j}\mathbf{\bar{G}}_{j}^{\intercal}$

// eq. 3

$\mathbf{W}_{j}\leftarrow\textrm{Newton-Schulz}(\mathbf{\Sigma}_{j})$

// alg. 1

3

4 end for

5$\boldsymbol{W}_{B}=\begin{bmatrix}\boldsymbol{W}_{1}&&\mathbf{0}\\\
&\ddots&\\\ \mathbf{0}&&\boldsymbol{W}_{n}\\\
\end{bmatrix}\in\mathbb{R}^{C\times C}$

$\mathbf{Z}\leftarrow\mathbf{W}_{B}(\boldsymbol{\mathrm{X}}-\hat{\mu}(\mathbf{X})\mathbf{1}^{\intercal})$

// whitening

6

$(\boldsymbol{\mu},\boldsymbol{\Gamma}_{B})\leftarrow\mathrm{reshape}(\mathbf{\Phi}_{B}(\mathbf{w}))$

// projection

$\mathbf{\tilde{X}}\leftarrow\boldsymbol{\mathbf{\Gamma}}_{B}\mathbf{Z}+\boldsymbol{\mu}\mathbf{1}^{\intercal}$

// coloring

7

Algorithm 2 Grouped AdaWCT style injection.

##  4 Experiments

We now present quantitative and qualitative experiments to validate our
method.

| Reference-guided | Latent-guided
---|---|---
| AdaIN | AdaWCT | AdaIN | AdaWCT
FID↓ | 19.78 | 16.20 | 16.18 | 13.07
LPIPS↑ | 0.431 | 0.434 | 0.450 | 0.476
Table 1: Reference- and latent-guided quantitative results. Quantitative FID↓ and LPIPS↑ metrics on the AFHQ dataset comparing our AdaWCT with AdaIN. Numbers correspond to the average obtained on all the domains for each metric. Fig. 1: Reference-guided qualitative results. “Source” refers to the structure (pose, orientation, shape) while “reference” is the style (texture, color and domain) which needs to be preserved in the result. Fig. 2: Latent-guided qualitative results. The same random style vector is used for each source image. First row contains the source images while second and third rows are the images generated using AdaIN and our method respectively. | Reference-guided | Latent-guided
---|---|---
$G$ | FID↓ | LPIPS↑ | FID↓ | LPIPS↑
1 | 24.98 | 0.309 | 18.23 | 0.342
4 | 20.79 | 0.328 | 16.65 | 0.359
8 | 20.48 | 0.338 | 20.48 | 0.368
16 | 18.22 | 0.375 | 14.02 | 0.410
32 | 17.80 | 0.370 | 14.64 | 0.399
64 | 16.63 | 0.382 | 14.24 | 0.408
Table 2: FID↓ (lower is better) and LPIPS↑ (higher is better) metrics as a
function of group size $G$ for both reference- and latent-guided scenarios.
Numbers correspond to the average obtained on all the domains for each metric.
Here, results are computed on images of $128\times 128$ resolution (hence
results are different than in tab. 1).

###  4.1 Network and dataset

We compare our AdaWCT approach with AdaIN within the framework of StarGANv2
[6], which allows for unpaired image-to-image translation across multiple
domains. To adapt StarGANv2 to our AdaWCT, we simply replace all the AdaIN
layers in the generator by our AdaWCT module. The other architectural
components (style encoder, mapping network, discriminator) are unchanged, and
so is the training regimen (loss functions, hyperparameters, etc.).

We present results on the “Animal Faces-HQ” (AFHQ) dataset [6]. This dataset
contains images from three domains: cats, dogs, and wild animals (mix of lion,
fox, tiger, cheetah, wolf, jaguar, and leopard). Each domain is composed of
4,500/500 images for train/test respectively. As in [6] and unless otherwise
noted, all results here are reported on images of $256\times 256$ resolution.

###  4.2 Quantitative and qualitative evaluation

We evaluate the quality and diversity of results obtained with AdaWCT compared
to AdaIN using the FID [30] and LPIPS [31] metrics. Following [6], we show
results in both 1) “reference-guided” (where the target style is obtained from
a reference image); and 2) “latent-guided” (where the target style is sampled
randomly) scenarios. Quantitatively, table 1 presents FID and LPIPS metrics
for the reference- and latent-guided scenarios. The AdaIN results are taken
directly from the official StarGANv2 official pytorch
implementation111https://github.com/clovaai/stargan-v2. AdaWCT (here with
$G=64$) outperforms AdaIN quantitatively all metrics except LPIPS for the
reference-guided scenario. For example, in the reference-based case, AdaWCT
results in an absolute FID decrease of 3.58 (-20%). Similar improvements are
observed in the latent-based case. This impact can be visualized qualitatively
in figs 1–2 for the reference- and latent-guided scenarios respectively. In
fig. 1, note how the AdaWCT results more closely match the appearance of the
reference and the pose of the source, and have fewer artifacts than their
AdaIN counterparts. In fig. 2, observe how the resulting images better match
the pose of the source while showing fewer artifacts.

###  4.3 Sensitivity analysis

We now study the impact of group size $G$ (sec. 3.5) on the quality of the
results. Table 2 shows the results of an experiment on images of $128\times
128$ resolution where $G$ is progressively increased from 1 (equivalent to
AdaIN, c.f. sec. 3.5) to 64. It can be seen that increasing the group from 1
to 4 makes a large difference, but as $G$ grows there are diminishing returns.
The best FIDs are obtained with $G=64$ (above which training is inefficient),
the value used in the experiments in sec. 4.2.

##  5 Discussion

This paper presented a generalization of AdaIN, dubbed AdaWCT, that we use for
style injection in large GANs. To do so, a group-wise whitening and coloring
transformation is presented as a drop-in replacement for AdaIN. Replacing the
style injection mechanism with our method within the established StarGANv2 [6]
architecture results in significantly improved performance for both reference-
and latent-guided image-to-image translation results.

Limitations and future work So far, the AdaWCT approach has been evaluated on
image-to-image applications with the StarGANv2 network. It would be
interesting, as future work, to study its impact in other popular GAN
architectures, for example for unconditional image generation with StyleGAN3
[9]. In addition, we also replaced the affine layers with a small, 3-layer MLP
for projecting the latent style vector to the coloring parameters (sec. 3.4).
Simplifying this to use a single affine projection, as is commonly done [9,
6], would potentially help reducing the number of additional parameters
introduced by our method.

Broader impact Any GAN can be employed for pernicious use. While this work was
conducted with the goal of improving image generation for creative purposes,
we acknowledge this paper enables the creation of more realistic images which
makes them more difficult to be detected and distinguished from real
photographs. We condemn misuse of these technologies, and promote solutions
such as digital authentication.

Acknowledgements This research was funded by NSERC CRDPJ 537961-18 and Gearbox
Studios, with computing resources from Compute Canada and Nvidia GPU
donations.

## References

  * [1] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio,  “Generative adversarial nets,”  in NeurIPS, 2014.
  * [2] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros,  “Image-to-image translation with conditional adversarial networks,”  in CVPR, 2017.
  * [3] X. Huang and S. Belongie,  “Arbitrary style transfer in real-time with adaptive instance normalization,”  in ICCV, 2017.
  * [4] Y. Choi, M. Choi, M. Kim, J.-W. Ha, S. Kim, and J. Choo,  “Stargan: Unified generative adversarial networks for multi-domain image-to-image translation,”  in CVPR, 2018.
  * [5] T. Karras, S. Laine, and T. Aila,  “A style-based generator architecture for generative adversarial networks,”  in CVPR, 2019.
  * [6] Y. Choi, Y. Uh, J. Yoo, and J.-W. Ha,  “Stargan v2: Diverse image synthesis for multiple domains,”  in CVPR, 2020.
  * [7] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio,  “Generative adversarial nets,”  in NeurIPS, 2014.
  * [8] T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila,  “Analyzing and improving the image quality of stylegan,”  in CVPR, 2020.
  * [9] T. Karras, M. Aittala, S. Laine, E. Härkönen, J. Hellsten, J. Lehtinen, and T. Aila,  “Alias-free generative adversarial networks,”  NeurIPS, 2021.
  * [10] X. Huang, M.-Y. Liu, S. Belongie, and J. Kautz,  “Multimodal unsupervised image-to-image translation,”  in ECCV, 2018.
  * [11] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros,  “Image-to-image translation with conditional adversarial networks,”  in CVPR, 2017.
  * [12] H.-Y. Lee, H.-Y. Tseng, J.-B. Huang, M. Singh, and M.-H. Yang,  “Diverse image-to-image translation via disentangled representations,”  in ECCV, 2018.
  * [13] Q. Mao, H.-Y. Lee, H.-Y. Tseng, S. Ma, and M.-H. Yang,  “Mode seeking generative adversarial networks for diverse image synthesis,”  in CVPR, 2019.
  * [14] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros,  “Unpaired image-to-image translation using cycle-consistent adversarial networks,”  in CVPR, 2017.
  * [15] M. Arjovsky, S. Chintala, and L. Bottou,  “Wasserstein generative adversarial networks,”  in ICML, 2017.
  * [16] I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. C. Courville,  “Improved training of wasserstein gans,”  in NeurIPS, 2017.
  * [17] M. Xudong, L. Qing, X. Haoran, L. Raymond Y. K., and W. Zhen,  “Least squares generative adversarial networks,”  in ICCV, 2017.
  * [18] D. Yang, S. Hong, Y. Jang, T. Zhao, and H. Lee,  “Diversity-sensitive conditional generative adversarial networks,”  in ICLR, 2019.
  * [19] T. Karras, T. Aila, S. Laine, and J. Lehtinen,  “Progressive growing of GANs for improved quality, stability, and variation,”  in CVPR, 2018.
  * [20] J. Lin, Y. Xia, T. Qin, Z. Chen, and T.-Y. Liu,  “Conditional image-to-image translation,”  in CVPR, 2018.
  * [21] M. Lu, H. Zhao, A. Yao, Y. Chen, F. Xu, and L. Zhang,  “A closed-form solution to universal style transfer,”  in ICCV, 2019.
  * [22] J. Yoo, Y. Uh, S. Chun, B. Kang, and J.-W. Ha,  “Photorealistic style transfer via wavelet transforms,”  in ICCV, 2019.
  * [23] A. Ermolov, A. Siarohin, E. Sangineto, and N. Sebe,  “Whitening for self-supervised representation learning,”  in ICML, 2021.
  * [24] C. Ionescu, O. Vantzos, and C. Sminchisescu,  “Matrix backpropagation for deep networks with structured layers,”  CoRR, 2015.
  * [25] P. Li, J. Xie, Q. Wang, and Z. Gao,  “Towards faster training of global covariance pooling networks by iterative matrix square root normalization,”  in CVPR, 2018.
  * [26] J. Schäfer and K. Strimmer,  “A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics,”  SAGMB, 2005.
  * [27] N. J. Higham,  Functions of Matrices: Theory and Computation,  Soc. for Ind. and App. Math., 2008.
  * [28] L. Huang, D. Yang, B. Lang, and J. Deng,  “Decorrelated batch normalization,”  in CVPR, 2018.
  * [29] L. Huang, X. Liu, B. Lang, A. W. Yu, and B. Li,  “Orthogonal weight normalization: Solution to optimization over multiple dependent stiefel manifolds in deep neural networks,”  in AAAI, 2018.
  * [30] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, G. Klambauer, and S. Hochreiter,  “Gans trained by a two time-scale update rule converge to a nash equilibrium,”  CoRR, vol. abs/1706.08500, 2017.
  * [31] J. Zhu, R. Zhang, D. Pathak, T. Darrell, A. A. Efros, O. Wang, and E. Shechtman,  “Toward multimodal image-to-image translation,”  CoRR, vol. abs/1711.11586, 2017.