††thanks:

# From Phase Space to Non-Equilibrium Dynamics: Exploring Liouville’s Theorem
and its Implications

Mario J. Pinheiro<EMAIL_ADDRESS>Department of Physics,
Instituto Superior Técnico - IST, Universidade de Lisboa - UL, Av. Rovisco
Pais, & 1049-001 Lisboa, Portugal https://www.researchgate.net/profile/Mario-
Pinheiro

###### Abstract

The Liouville theorem is a fundamental concept in understanding the properties
of systems that adhere to Hamilton’s equations. However, the traditional
notion of the theorem may not always apply. Specifically, when the entropy
gradient in phase space fails to reach equilibrium, the phase-space density
may not have a zero time derivative, i.e., $\frac{d\rho}{dt}$ may not be zero.
This leads to the concept of the set of attainable states of a system forming
a compressible ”fluid” in phase space. This observation provides additional
insights into Hamiltonian dynamics and suggests further examination in the
fields of statistical physics and fluid dynamics. In fact, this finding sheds
light on the limitations of the Liouville theorem and has practical
applications in fields such as beam stacking, stochastic cooling, and Rabi
oscillations, among others.

Nonlinear physics, Plasma physics, Statistical physics, Liouville theorem,
Hamilton’s equations, Ergontropic dynamics, Entropy gradient, Phase-space
density, Fluid dynamics.

††preprint: IST/UL.2022-M J Pinheiro

##  I Introduction

A key finding in classical mechanics, Liouville’s theorem Liouville1838  has
significant effects on how classical systems behave. The volume of phase space
covered by a system experiencing Hamiltonian motion is preserved throughout
time, according to the theorem. If we think about the six-dimensional phase
space that is covered by the positions and velocities of every particle in a
classical system, the volume of this phase space that the system occupies will
not change over time. This has several important implications: (i)
Trajectories in phase space cannot intersect, according to the conservation of
phase space volume. As a result, there are no ”random” changes in the state of
the system; rather, the evolution of a system in phase space is entirely
predictable hp1890am ; poincare1892 ; (ii) According to Liouville’s theorem, a
system may develop from any initial state to any final state over the course
of time in phase space Arnold ; Arnold1989 ; Marsden1994 ; Gallavotti . This
is caused by the fact that phase space volume is conserved, which protects the
system from ”losing” information as it develops; (iii) The behavior of
macroscopic systems is significantly impacted by the conservation of phase
space volume. For instance, it suggests that when a gas of non-interacting
particles evolves, the particle distribution in phase space stays constant
Toda . The classical Boltzmann distribution, which defines the statistical
behavior of classical systems at equilibrium, is based on this principle
Ehrenfest .

Liouville’s theorem is an effective tool for understanding how classical
systems behave in phase space. It forms the basis for classical mechanics’
deterministic development and contributes to the explanation of the
statistical behavior of macroscopic systems. We can find a description of
Liouville’s theorem close to the original in the well-known book of Paul
Ehrenfest Ehrenfest .

The Schrödinger quest for a quantum mechanical description based on the
eikonal form of the Maxwell set of equations dictated their type of solutions
based on amplitudes, the superposition principle for their solutions, and the
typical interferences effects that are the outcome of squaring amplitudes. In
a result of this path, the mathematical framework is founded in linear algebra
and Hilbert spaces. The fitting of quantum mechanical theory to classical
mechanics was made by forcing the relationship between the Poisson brackets
and the commutator brackets, $[A,B]=i\hbar\\{A,B\\}$. If instead, the starting
point was classical mechanics, then a different outcome would result, mainly
no amplitude to intensity relationships and the superposition principle would
be applied to forces (or potentials).

Liouville’s theorem provides a crucial link between classical and quantum
mechanics through the concept of phase-space density. In classical mechanics,
phase space is a six-dimensional space spanned by the position and momentum
coordinates of all particles in the system. The phase-space density function,
denoted by $\rho(q,p,t)$, gives the probability density of finding a system in
a particular phase-space point $(q,p)$ at time $t$.

In quantum mechanics, the phase space is replaced by the Hilbert space of the
system, and the concept of the phase-space density is replaced by the wave
function, denoted by $\Psi(q,t)$ and the probability density of finding a
quantum system in a particular state is given by $|\Psi(q,t)|^{2}$.

The Wigner function is a mathematical tool that provides a link between the
classical and quantum descriptions of the phase-space density, defined as:

$W(q,p,t)=\frac{1}{2\pi\hbar}\int_{-\infty}^{\infty}\Psi^{*}\left(q-\frac{\lambda}{2},t\right)\Psi\left(q+\frac{\lambda}{2},t\right)\\\
e^{-i\frac{p\lambda}{\hbar}}d\lambda.$ (1)

The Wigner function is a real-valued function that satisfies a version of
Liouville’s theorem known as the Wigner-Liouville equation, $\frac{\partial
W}{\partial t}+\\{H,W\\}=0$, where $H$ is the Hamiltonian of the system and
$\\{\cdot,\cdot\\}$ denotes the Poisson bracket. The Wigner-Liouville equation
describes the time evolution of the Wigner function and provides a bridge
between the classical and quantum descriptions of the system.

Furthermore, Liouville’s theorem provides the foundation for the
correspondence principle, which states that the behavior of quantum systems
should reduce to classical behavior in the limit of large quantum numbers.
This is achieved through the use of coherent states, which are quantum states
that exhibit classical-like behavior. The Wigner function for coherent states
approaches the classical phase-space density in the limit of large quantum
numbers.

In quantum mechanics, the non-conservation of phase space implies that the
uncertainty principle is fundamental and cannot be overcome. This is because,
in quantum mechanics, the phase-space density is related to the probability
amplitude of the system, which determines the probability of finding the
system in a particular state. The non-conservation of phase space means that
the probability density of the system cannot be conserved, which implies that
the uncertainty principle must hold. The uncertainty principle states that the
more precisely the position of a particle is known, the less precisely its
momentum can be known, and vice versa. Therefore, the non-conservation of
phase space implies that the uncertainty principle is an essential feature of
quantum mechanics and cannot be eliminated.

There have been many experiments that have confirmed the uncertainty
principle, and it is considered to be a fundamental principle of nature Aspect
; Hensen . There have been attempts to find ways around the uncertainty
principle, such as using entangled particles or non-local measurements, but
these approaches have not been successful in violating the principle Cunha ;
Kimble ; Aspect1982 . The uncertainty principle is deeply embedded in the
foundations of quantum mechanics and is a necessary consequence of the wave-
particle duality of quantum objects.

Hence, the nonconservation of phase space has significant implications for
classical and quantum mechanics. Classical mechanics challenges the Liouville
theorem and suggests the compressibility of phase space. In quantum mechanics,
it implies the uncertainty principle, demonstrating limits to measurement
precision, and has important implications for interpretation and experiment
design.

The aim of this work is to demonstrate that the assumption that Hamiltonian
systems obey the Liouville theorem may be challenged by the inclusion of Eqs.
(8)-(9). This is due to the fact that if entropy gradients in the phase-space
do not equilibrate, the time derivative of the phase-space density may not be
zero. Consequently, the set of states that a system could access forms a
volume in phase space denoted by $\Gamma$, which is analogous to a
compressible fluid.

##  II The equation of motion for physical quantities

The equation of motion for any arbitrary physical quantity is

$\dot{F}=\frac{\partial F}{\partial q}\dot{q}+\frac{\partial F}{\partial
p}\dot{p}=[H,F],$ (2)

with the hamiltonian $H\equiv p\dot{q}-L(\dot{q},q)$, $[,]$ representing the
Poisson brackets, and the Hamiltonian equations of motion

$\displaystyle\dot{q}=\frac{\partial H}{\partial p}$ ;
$\displaystyle\dot{p}=-\frac{\partial H}{\partial q}.$ (3)

The fundamental equation of motion in Hamiltonian mechanics is given by Eq.
(25), which provides a mathematical description of the time evolution of any
physical quantity. This equation expresses the rate of change of a physical
quantity $F$ as a function of the generalized coordinates $q$ and momenta $p$.
The Hamiltonian $H$, which is defined in terms of the Lagrangian $L$ as
$H\equiv p\dot{q}-L(\dot{q},q)$, plays a central role in the formulation of
Hamiltonian mechanics, and is related to the total energy of the system. The
Poisson brackets $[,]$, defined as $[A,B]\equiv\sum_{i}(\frac{\partial
A}{\partial q_{i}}\frac{\partial B}{\partial p_{i}}-\frac{\partial A}{\partial
p_{i}}\frac{\partial B}{\partial q_{i}})$, capture the algebraic structure of
Hamiltonian mechanics and provide a powerful tool for the calculation of
physical quantities.

The Hamiltonian equations of motion, given by Eq. (3), are a set of first-
order differential equations that describe the time evolution of the
generalized coordinates and momenta. These equations are derived from the
Hamiltonian $H$ and express the principle of least action in Hamiltonian form.
They relate the time derivatives of the generalized coordinates and momenta to
the partial derivatives of the Hamiltonian with respect to the conjugate
variables.

Hamiltonian mechanics provides a powerful framework for the description of
classical and quantum systems and has a wide range of applications in physics
and engineering. The formalism is particularly useful for the analysis of
conservative systems, where the total energy is conserved.

##  III Out-of-equilibrium dynamics

A new form of canonical momentum and equation of motion has been derived from
a recent variational principle Pinheiro . The fundamental equation of motion
has the form of a local balance equation with the spatial gradient of entropy
as the source term. The gradient of the total entropy in momentum space is
also given, which, when maximized, leads to the total canonical momentum. The
new set of equations of motion resembles the Hamiltonian formulation of
dynamics and complies with the Helmholtz free energy. Liouville’s equation has
been defined, along with the Liouville operator, and the expectation value of
a function has also been given.

In that framework, the new form of canonical momentum is:

$\frac{\partial\bar{S}}{\partial\mathbf{p}^{(\alpha)}}\geq 0$ (4)

and the fundamental equation of motion:

$\frac{\partial\bar{S}}{\partial\mathbf{r}^{(\alpha)}}=-\frac{1}{T}\boldsymbol{\nabla}_{r^{(\alpha)}}U^{(\alpha)}-\frac{1}{T}m^{(\alpha)}\frac{\partial\mathbf{v}^{(\alpha)}}{\partial
t}\geq 0.$ (5)

Eq. 5 gives the fundamental equation of dynamics and has the form of a general
local balance equation having as source term the spatial gradient of entropy,
$\nabla_{a}S>0$, whilst Eq. 4 gives the canonical momentum. At thermodynamic
equilibrium, the total entropy of the body has a maximum value. In the more
general case of a non-equilibrium process, the entropic gradient must be
positive in both Eqs. 4\- 5. The interplay between energy-minimizing
tendencies and entropy maximization may introduce new physics through a set of
two first-order differential equations. These equations have the potential to
reveal novel insights into the underlying dynamics of physical systems.

In non-equilibrium processes, the gradient of the total entropy in momentum
space multiplied by factor $T$ is given by

$\frac{\partial\overline{S}}{\partial\mathbf{p}^{(\alpha)}}=\frac{1}{T}\left\\{-\frac{\mathbf{p}^{\alpha)}}{m^{(\alpha)}}+\frac{q^{(\alpha)}}{m^{(\alpha)}}\mathbf{A}+\mathbf{v}_{e}+[\mathbf{\omega}\times\mathbf{r}^{(\alpha)}]\right\\},$
(6)

so that maximizing entropy change in Eq. 4 leads to the well-known total
canonical momentum:

$\mathbf{p}^{(\alpha)}=m^{(\alpha)}\mathbf{v}_{e}+m^{(\alpha)}[\mathbf{\omega}\times\mathbf{r}^{(\alpha)}]+q^{(\alpha)}\mathbf{A}.$
(7)

The above formulation bears some resemblance with the Hamiltonian formulation
of dynamics which expresses first-order constraints of the Hamiltonian $H$ in
a $2n$ dimensional phase space, $\dot{\mathbf{p}}=-\partial
H/\partial\mathbf{q}$ and $\dot{\mathbf{q}}=\partial H/\partial\mathbf{p}$,
and can be solved along trajectories as quasistatic processes, revealing the
same formal symplectic structure shared by classical mechanics and
thermodynamics. The sharing of a formal symplectic structure between classical
mechanics and thermodynamics implies a common geometric framework for the
equations of motion, which enables the application of Hamiltonian mechanics to
the study of thermodynamic systems and suggests the existence of underlying
mathematical structures that are common to many different physical systems.

In the context of our approach, the new set of equations of motion should
read:

$\displaystyle\mathbf{\dot{p}}$ $\displaystyle=$
$\displaystyle-\boldsymbol{\nabla}H+T\boldsymbol{\nabla}\overline{S}=-\frac{\partial}{\partial\mathbf{q}}(H-T\overline{S})$
(8) $\displaystyle\mathbf{\dot{q}}$ $\displaystyle=$
$\displaystyle-T\boldsymbol{\nabla}\overline{S}+\boldsymbol{\nabla}H=\;\;\frac{\partial}{\partial\mathbf{p}}(H-T\overline{S}).$
(9)

We have identified $U$ as equivalent to $H$, and it is worth noting that the
motion of the system is now governed by the Helmholtz free energy,
$\mathfrak{H}=H-T\overline{S}$, rather than just the Hamiltonian alone. The
gradients of the system’s Hamiltonian function and the thermodynamic
quantities are connected to the time derivatives of the system’s position and
momentum by the equations. There is an easy way to connect the macroscopic
thermodynamic parameters of temperature, entropy, and energy to the
microscopic characteristics of the system’s particles due to the
identification of the Hamiltonian function with the Helmholtz free energy, H =
U - TS. The reformulation may be a useful tactic for statistical mechanics’
study of the behavior of complex systems, with several applicability in
physics, chemistry, and materials science, as we will suggest later.

Our investigation centers around Liouville’s equation

$\frac{d\rho}{dt}=\imath[\rho,H],$ (10)

where the function $\rho(q,p,t)$ is defined in a way such as the product

$\rho(q,p,t)dqdp=\rho(q,p,t)d\Omega$ (11)

represents the number of system points in the phase volume $d\Omega$ around
the point $(q,p)$ at the time $t$. We can write

$\imath\frac{\partial\rho}{\partial t}=L\rho$ (12)

where

$L=-\imath\frac{\partial H}{\partial p}\frac{\partial}{\partial
q}+\imath\frac{\partial H}{\partial q}\frac{\partial}{\partial p},$ (13)

represents the Liouville operator (and $\imath=\sqrt{-1}$).

$\langle A\rangle=\int dpdqA(p,q)\rho$ (14) $\frac{\partial\rho}{\partial
t}=-\partial_{p}H\partial_{q}\rho+\partial_{q}H\partial_{p}\rho$ (15)

But from Eqs. we have

$\frac{\partial\rho}{\partial
t}=-\dot{q}\partial_{q}\rho-\dot{p}\partial_{p}\rho-T\partial_{p}\overline{S}\partial_{q}\rho+T\partial_{q}\overline{S}\partial_{p}\rho$
(16)

or

$\frac{d\rho}{dt}=-T\left[\partial_{p}\overline{S}\partial_{q}\rho-\partial_{q}\overline{S}\partial_{p}\rho\right].$
(17)

If we introduce now the usual Poisson bracket for two variables $A$ and $B$:

$[A,B]=\sum_{i}\left(\frac{\partial A}{\partial q_{i}}\frac{\partial
B}{\partial p_{i}}-\frac{\partial B}{\partial p_{i}}\frac{\partial A}{\partial
q_{i}}\right),$ (18)

we could express the Liouville equation in a more comprehensive form.

$\frac{\partial\rho}{\partial
t}+\boldsymbol{u}\cdot\boldsymbol{\nabla}\rho=-T[\partial_{p}\overline{S}\partial_{q}-T\partial_{q}\overline{S}\partial_{p}]\rho.$
(19)

Note that

$\boldsymbol{u}\cdot\boldsymbol{\nabla}=\sum_{l}\left(\frac{\partial
H}{\partial p_{l}}\frac{\partial}{\partial q_{l}}-\frac{\partial H}{\partial
q_{l}}\frac{\partial}{\partial p_{l}}\right).$ (20)

Eq. 20 implies a correction to the Liouville equation, that needs to be
written now under the form:

$\frac{\partial\rho}{\partial t}=[H,\rho]-T[\overline{S},\rho],$ (21)

or

$\frac{d\rho}{dt}=-T[\overline{S},\rho]\neq 0.$ (22)

But, as the free energy can be defined by $\overline{F}=F_{0}-T\overline{S}$,
we may drop out the approximation of isothermic states and write instead a
more general form of Liouville’s theorem for out-of-equilibrium systems:

$\frac{d\rho}{dt}=-[\overline{F},\rho]\neq 0.$ (23)

Non-isothermic states involves a delicate balance between energy and entropy,
which can make their analysis difficult and may not be necessary for
understanding the behavior of out-of-equilibrium systems. By redefining the
free energy as $\overline{F}=F_{0}-T\overline{S}$, we can instead focus on a
more general form of Liouville’s theorem that applies to a broader range of
physical systems.

The introduction of Equations (7)-(8) necessitates a reformulation of
Liouville’s theorem. When the entropy gradients in phase space fail to
equilibrate, the time derivative of the phase-space density, $d\rho/dt$, may
not be zero, resulting in a set of possible system states forming a
compressible ”fluid” volume in phase space denoted as $\Gamma$. This
observation sheds light on why the Liouville theorem does not seem to hold in
certain techniques, such as beam stacking, electron cooling, stochastic
cooling, synchrotron radiation, and charge exchange Ruggiero ; Winston_2009 ,
or in the Boltzmann equation when the collision operator is irreversible
Villani . Ref. Datsko_2016  reports that Liouville’s theorem is insufficient
for predicting the behavior of the atmosphere and climate, even in the case of
a simple linear oscillator.

This failure of Liouville’s theorem has significant technological
implications, and we provide examples of such cases below.

###  III.1 Relaxation of an initially out-of-equilibrium system towards
thermal equilibrium

Given the previously defined operators, the Helmholtz operator for a system
can be expressed as:

$\hat{H}-T\hat{S}=\hat{U}-TS(\hat{\rho})=(\hat{\mathcal{H}}-\mu\hat{\mathcal{N}})-T\hat{S}(\hat{\rho})$

Here, $\hat{\mathcal{H}}$ is the Hamiltonian operator, $\mu$ is the chemical
potential operator, $\hat{\mathcal{N}}$ is the particle number operator, and
$\hat{S}(\hat{\rho})$ is the operator for the entropy of the system. Note that
the entropy operator is a function of the density operator $\hat{\rho}$, which
describes the statistical distribution of the system. the von Neumann entropy
of a density operator $\rho$ can be defined as
$S(\rho)=-\mathrm{Tr}(\rho\ln\rho)$, where $\mathrm{Tr}$ denotes the trace.
This is a standard expression for entropy in the context of quantum mechanics.

To obtain a quantum field equation from the set of classical equations of
motion given in Eqs. (7) and (8), we need to promote the variables p and q to
quantum operators and replace the classical Poisson brackets with quantum
commutators. We also need to introduce a time-dependent parameter $\lambda$ to
control the transition from the classical to the quantum regime, such that
$\lambda=0$ corresponds to the classical limit, and $\lambda=1$ corresponds to
the fully quantum regime. This can be done using the so-called Wigner-Weyl
transformation, which maps classical variables to quantum operators.

Let us define the Wigner function as:

$W(q,p,t)=\int dye^{ipy/\hbar}\psi(q-y/2,t)\psi^{*}(q+y/2,t),$ (24)

where $\psi(q,t)$ is the wave function of the system. The Wigner function is a
quasi-probability distribution (it may contain negative values) that encodes
both the position and momentum information of the system and it satisfies the
following properties: i) $W(q,p,t)$ is real; ii) $W(q,p,t)$ is normalized:
$\int dqdpW(q,p,t)=1$.

The marginal distributions of W(q, p, t) with respect to q and p recover the
probability density and current of the system, respectively:

$\begin{split}\rho(q,t)&=\int dpW(q,p,t)\\\ J(q,t)&=\int
dp\frac{p}{m}W(q,p,t)\end{split}$ (25)

where $m$ is the mass of the system. Using the Wigner function, we can rewrite
the classical equations of motion in Eqs. (7) and (8) as:

$\frac{\partial W}{\partial t}=\\{H-TS,W\\}$ (26)

where $\\{A,B\\}$ denotes the Poisson bracket of $A$ and $B$, and we have
replaced the classical variables p and q with their corresponding quantum
operators, $\hat{p}$ and $\hat{q}$. The Poisson bracket can be replaced with a
commutator in the limit of large quantum numbers, $\hbar\to 0$, using the
correspondence rule, $\\{A,B\\}\to(1/\hbar)[\hat{A},\hat{B}]$, where
$[\hat{A},\hat{B}]$ denotes the commutator of $\hat{A}$ and $\hat{B}$. So, the
quantum field equation is:

$\frac{\partial\hat{W}}{\partial t}=[\hat{H}-T\hat{S},\hat{W}],$ (27)

where $\hat{H}$, $\hat{S}$, and $\hat{W}$ are the quantum operators
corresponding to the classical functions $H$, $S$, and $W$, respectively.

Eq. 27 is the quantum field equation in the Wigner representation. It
describes the time evolution of the Wigner function of the system and can be
used to calculate various properties of the system, such as its energy
spectrum ( distribution of energy levels), correlation functions (relationship
between different properties), and coherence properties (the degree to which
the phases of different parts of a wave or system are related to one another).

Note that this definition of entropy assumes that $\rho$ is a positive semi-
definite operator with a trace equal to one, which is the case for density
operators. To define the entropy for a general operator, we need to consider
more general definitions. The von Neumann entropy of a density operator $\rho$
is defined as $S(\rho)=-Tr(\rho\ln\rho)$. On the other hand, the Wigner
function is a quasi-probability distribution function that represents the
quantum state of a system in phase space, defined as:

$W(q,p)=\frac{1}{\pi\hbar}\int\psi^{*}(q+y/2)\psi(q-y/2)e^{(-ipy/\hbar)}dy,$
(28)

where $\psi$ is the wave function of the system, $q$ and $p$ are the position
and momentum variables. To apply the von Neumann entropy to the Wigner
function, we first need to convert the Wigner function into a density operator
(represented by a Hermitian matrix, and containing information about the
probabilities of different outcomes of a measurement of the system). This can
be done using the Wigner-Weyl transform:

$\rho=\frac{1}{\pi\hbar}\int W(q,p)D(q,p)dqdp$ (29)

where $D(q,p)$ is the Weyl operator, defined as:

$D(q,p)=e^{(i(p\wedge q-q\wedge p)/\hbar)}.$ (30)

Once we have the density operator $\rho$, we can calculate its von Neumann
entropy using the formula $S(\rho)=-Tr(\rho\ln\rho)$, where $\ln\rho$ is the
matrix logarithm of $\rho$, which can be obtained by diagonalizing $\rho$ and
taking the logarithm of its eigenvalues. The trace $Tr(\rho\ln\rho)$ can then
be evaluated by summing the diagonal elements of $\rho\ln\rho$. Hence, we can
apply the von Neumann entropy to the Wigner function by first converting it
into a density operator using the Wigner-Weyl transform, and then using the
formula $S(\rho)=-Tr(\rho\ln\rho)$ to calculate its von Neumann entropy.

To apply the equation $d\rho/dt=-[F,\rho]$ to study the evolution of the phase
space density during a system’s transition, we must first define the free
energy function F as a function of parameters that describe the transition.
Suppose the transition is controlled by a parameter $\lambda$. In this case,
the free energy function can be expressed as:

$F(\lambda)=H-\lambda G,$ (31)

where $H$ and $G$ are Hermitian operators that represent the Hamiltonian and
some other observable, respectively. As $\lambda$ is varied, the system
undergoes a transition from one phase to another, and we want to study the
evolution of the phase space density as this happens. We can start by writing
the Liouville equation in terms of the free energy function $F(\lambda)$:

$\frac{d\rho}{dt}=-[F(\lambda),\rho].$ (32)

Expanding the commutator, we get:

$\frac{d\rho}{dt}=-FH\rho+H\rho F+\lambda GH\rho-\lambda H\rho G$ (33)

Now, we can write the density operator $\rho$ as a sum of its eigenstates:

$\rho=\sum_{n}p_{n}|n><n|,$ (34)

where $p_{n}$ is the probability of finding the system in the nth eigenstate
$|n>$. By writing the density operator as a sum of its eigenstates, we can
then determine the probabilities of finding the system in each of its
eigenstates at a given time.

Substituting this into the Liouville equation and using the orthonormality of
the eigenstates, we get:

$\frac{d}{dt}(p_{n}|n><n|)=-FH(p_{n}|n><n|)+\\\ H(p_{n}|n><n|)F+\lambda
GH(p_{n}|n><n|)-\\\ \lambda H(p_{n}|n><n|)G,$ (35)

which simplifies to:

$\frac{d}{dt}(p_{n})=-p_{n}(F_{n}n-F_{i}i)+\lambda p_{n}(G_{n}n-G_{i}i)$ (36)

where $F_{nn}$ and $F_{ii}$ are matrix elements of the Hamiltonian in the nth
and ith eigenstates, respectively, and similarly for $G_{nn}$ and $G_{ii}$.

The set of equations presented describes how the probabilities of the
different eigenstates of the system, represented by $p_{n}$, evolve over time
as the parameter $\lambda$ is varied. By solving these equations, we can
analyze the behavior of the system as it undergoes the transition. One
application of these equations is to calculate the average value of an
observable $A$ in the different phases. This can be done using the formula:

$<A>=\sum_{n}np_{n}<n|A|n>$ (37)

where the sum is over the eigenstates of the system, and $<n|A|n>$ is the
expectation value of the observable $A$ in the nth eigenstate.

The inclusion of temperature and entropy in the Hamiltonian can lead to
effects such as decoherence and relaxation, which can significantly influence
the Rabi oscillations in a quantum system. When a quantum system interacts
with a thermal environment, energy exchange, and dephasing result in a reduced
coherence of the Rabi oscillations. Fig. 1 illustrates one application of the
previously described model, after scaling the energies of the cavity
$\omega_{c}=1$ and atoms frequencies $\omega_{1},\omega_{2}$ pinheiro2023rabi
. In a variety of quantum computing systems, these oscillations are essential
for implementing quantum gates, which are the fundamental operations needed to
build quantum circuits and execute quantum algorithms.

Figure 1: Visual representation of the interdependence of energy and entropy
in Vacuum Rabi oscillations. Orange line: $\omega_{1}=0.90$, green line:
$\omega_{2}=1.20$, blue line: $\omega_{c}=1.0$ is the cavity frequency.

We now have a theoretical framework, relying on the equation
$d\rho/dt=-[F,\rho]$, which permits us to explore the time-dependent evolution
of the phase space density as a system undergoes a transition. This framework
is based on the analysis of the probabilities associated with the different
eigenstates of the system.

###  III.2 Nonelastic collisions between particles

According to Eq. 22 the invariance of volume for canonically conjugated
variables is not verified which implies that, in the presence of entropy
gradients or out-of-equilibrium systems, there is no conservation of momentum
nor kinetic energy (see also Ref. Leontovitch ) in particle collisions.

This means that in the presence of entropy gradients or out-of-equilibrium
systems, there is no conservation of momentum or kinetic energy in particle
collisions. This lack of conservation of these quantities is significant
because it implies that the usual assumptions made in equilibrium systems,
where entropy gradients are absent, do not hold in out-of-equilibrium systems.
This result is not new and has been discussed previously in the literature.
Reference [5] is also cited as a source of further information on the subject.
The lack of conservation of momentum and kinetic energy in particle collisions
in out-of-equilibrium systems has important implications for understanding the
behavior of systems far from equilibrium, such as those found in many
biological, ecological, and social systems. Understanding and modeling such
systems require a more complex and nuanced approach than is necessary for
equilibrium systems, where the assumption of entropy gradients is usually
valid.

###  III.3 Brightness of an atomic beam Source

Subjecting the axial or transverse velocity components of the beam to
dissipative cooling dramatically compresses the phase space of the atom flux,
resulting in dense, well-collimated atomic beams suitable for the study of
atom optics, atom holography, or ultracold collision dynamics. Prodan et al.
Prodan  first demonstrated the importance of this phase-space compression. In
fact, atomic beams can now achieve a level of “brightness” (atom beam flux
density per unit solid angle) many times greater than the phase-space
conservation limit imposed by the Liouville theorem (cf. Pierce Pierce ,
Sheehy et al. Sheehy , Kuyatt Kuyatt ). The importance of dissipative cooling
in compressing the phase space of atomic beams leads to dense, well-collimated
atomic beams that are useful for studying various fields of physics, such as
atom optics, atom holography, and ultracold collision dynamics. The phase-
space compression was first demonstrated by Prodan et al. Prodan  in 1994,
highlighting its importance in the field of atomic physics.

Moreover, recent advancements in atomic beam technology have resulted in
achieving ”brightness” levels (atom beam flux density per unit solid angle)
that surpass the phase-space conservation limit imposed by the Liouville
theorem, which describes the conservation of phase-space volume in a classical
dynamical system. This breakthrough is significant because it opens up new
opportunities for studying the behavior of atomic beams in various
applications, including materials science, quantum optics, and precision
measurements. The references cited in the text provide further information on
the research related to this topic.

###  III.4 The mechanics of magnetic helicity in the plasma

The helicity associated with ions and electrons in plasma has opposite signs.
This is because helicity is a measure of the handedness of the magnetic field,
and the ions and electrons have opposite charges and therefore move in
opposite directions in a magnetic field. As a result, the magnetic fields
generated by the two populations will have opposite handedness. In the
expression for the free energy that includes both ions and electrons, we need
to take into account the opposite signs of the helicity densities. A more
general expression for the free energy that accounts for this is:

$F=\frac{B^{2}}{2\mu_{0}}+(\alpha_{i}-\alpha_{e})K,$ (38)

where $\alpha_{i}$ and $\alpha_{e}$ are the helicity densities associated with
the ion and electron populations, respectively, and $K$ is the kinetic energy
density. The term $(\alpha_{i}-\alpha_{e})$ accounts for the opposite signs of
the ion and electron helicities.

Taking the gradient of $F$ with respect to position, we obtain:

$\gradient
F=\frac{1}{2\mu_{0}}\gradient(B^{2})+(\alpha_{i}-\alpha_{e})\gradient(K).$
(39)

Using the same identity as before, $\gradient(B^{2})=4\alpha\mathbf{B}$, where
$\alpha$ is the total helicity density, including contributions from both ions
and electrons, we can write:

$\gradient
F=\left(\frac{4\alpha}{\mu_{0}}\right)B^{2}+(\alpha_{i}-\alpha_{e})\gradient(K).$
(40)

The total helicity density in plasma physics is a measure of the twistedness
or knotting of magnetic field lines, and it’s a conserved quantity in ideal
magnetohydrodynamics (MHD). The gradient of free energy is related to the
total helicity density, but with an additional term that accounts for the
difference between ion and electron helicities. This term reflects how the
dynamics of ion and electron populations can impact the overall plasma
behavior.

Mathematically, the total helicity density is defined as the volume integral
of the dot product between the magnetic field $\mathbf{B}$ and its vector
potential $\mathbf{A}$, i.e., $\alpha=\int_{V}(\mathbf{A}\cdot\mathbf{B})dV$,
where $V$ is the volume of the plasma. In general, the total helicity density
can be both positive and negative, depending on the orientation and topology
of the magnetic field lines.

In plasma, the total helicity density is related to the free energy of the
system and plays a crucial role in determining the stability and dynamics of
the plasma. The gradient of the total helicity density is related to the
Lorentz force that acts on the plasma, and it can drive various instabilities
and reconnection events in the magnetic field. Therefore, the total helicity
density is an important quantity in plasma physics and is often used in
theoretical and experimental studies of plasmas. In a plasma, both ions and
electrons can contribute to the helicity of the magnetic field. The ion and
electron helicities are defined as the volume integrals of the dot products
between the magnetic field and the velocity of the respective species, i.e.,
$\alpha_{i}=\int_{V}(\mathbf{v}_{i}\cdot\mathbf{B})dV$ and
$\alpha_{e}=\int_{V}(\mathbf{v}_{e}\cdot\mathbf{B})dV$, where $\mathbf{v}_{i}$
and $\mathbf{v}_{e}$ are the velocities of the ions and electrons,
respectively. The total helicity density is the sum of the ion and electron
helicities, i.e., $\alpha=\alpha_{i}+\alpha_{e}$. As shown in Figure 2, the
complexity of magnetic field lines increases with their writhe (measure the
total amount of coiling or twisting in a knot) and twist (a measure of the
local twisting or rotation of a knot).

Figure 2: The relationship between free energy and helicity in magnetic
fields, visualized through the increasing complexity of magnetic field lines
as their writhe and twist increases. As the helicity of the field increases,
so does its free energy, leading to more complex and tangled field lines.

Therefore, the total helicity density includes contributions from both ions
and electrons and reflects the overall twistedness or knotting of the magnetic
field lines in the plasma.

The difference between the ion and electron helicities, i.e.,
$(\alpha_{i}-\alpha_{e})$, is related to the dynamics of the ion and electron
populations in the plasma. If the ion and electron populations have different
velocities or distributions, they can contribute differently to the helicity
of the magnetic field and create a net helicity difference. This net helicity
difference can in turn affect the stability and dynamics of the plasma and can
lead to various instabilities or reconnection events in the magnetic field, as
shown in Myers2022  with the effect of energy conversion and dynamics of
magnetic reconnection. Therefore, the ion and electron helicities, as well as
their difference, are important quantities in plasma physics and can provide
valuable insights into the behavior and evolution of plasmas.

The equation $d\rho/dt=-[F,\rho]$ describes the evolution of helicity density
$(\rho)$ in plasma, where changes in free energy $F$ are linked to changes in
magnetic field line twist and writhe. Increases in free energy can lead to
increases in helicity density and vice versa. The equation provides a
fundamental link between free energy and magnetic topology and highlights the
important role of free energy in determining magnetic dynamics in plasma.

To illustrate the relationship between free energy, helicity, twist, and
writhe, let us consider a simple example of a magnetic field in a plasma that
has both twist and writhe. We can write the magnetic field in terms of its
vector potential, A, as $\mathbf{B}=\gradient\times\mathbf{A}$, and the
helicity density of this magnetic field can be written as
$\rho=\mathbf{A}\cdot\gradient\times\gradient{A}$. This equation relates the
helicity density to the vector potential and its curl, and it quantifies the
amount of twisting and linking of the magnetic field lines in the plasma.

The free energy of plasma, defined as the energy stored in the magnetic field,
can be expressed as $F=\frac{1}{2\mu_{0}}\int B^{2}dV$. Perturbations in the
magnetic field can affect the twist and writhe of the magnetic field lines and
cause changes in the helicity density and free energy of the plasma. Adding
twist to the magnetic field increases the helicity density and, in turn, the
free energy of the plasma. The time derivative of the helicity density can
express this relationship.

$\frac{d\rho}{dt}=\int(\gradient\times\mathbf{A})\cdot\left(\gradient\times\frac{d\mathbf{A}}{dt}\right)dV$
(41)

Using the equation of motion for the plasma, which is given by:

$\rho\mathbf{v}=[\mathbf{j}\times\mathbf{B}]-\epsilon_{0}\frac{d\mathbf{E}}{dt},$
(42)

where $v$ is the plasma velocity, $j$ is the current density, $E$ is the
electric field, and $\epsilon_{0}$ is the electric permittivity of free space,
we can rewrite the time derivative of the helicity density as:

$\frac{d\rho}{dt}=-2\int(\mathbf{j}\cdot\mathbf{B})dV.$ (43)

This equation shows that the time rate of change of the helicity density is
proportional to the current density and the magnetic field. Thus, if we
increase the twist in the magnetic field, we will also increase the current
density, which will in turn increase the rate of change of the helicity
density, and hence the free energy of the plasma. Similarly, if we perturb the
magnetic field by adding a small amount of writhe to it, the helicity density
will again increase, and this will lead to an increase in the free energy of
the plasma. This can be seen by considering the writhe of the magnetic field
lines, which is given by:

$Wr=\int\left[\mathbf{B}\cdot\gradient\times\left(\frac{\mathbf{B}}{B^{2}}\right)\right]dV,$
(44)

where the integral is taken over the volume of the plasma. This equation
quantifies the degree of linking of the magnetic field lines, and it is
related to the helicity density through the equation $\rho=2Wr$, the greater
the degree of linking between magnetic field lines, the higher the helicity
density in the system.

Thus, we can see that changes in the free energy of the plasma can lead to
changes in the twist and writhe of the magnetic field lines and that the
helicity density provides a fundamental link between these quantities.

##  IV Conclusion

We conclude that, if the Liouville theorem reflects the properties of systems
obeying Hamilton’s equations, in our approach, introducing Eqs.(7)-(8) this is
not necessarily so. If the gradients of entropy in phase space do not
equilibrate, then $d\rho/dt$ is not necessarily null, which means that the set
of states that a system can possibly attain form a volume in the phase-space
$\Gamma$ representing a “fluid” that may be compressible. The conclusion drawn
is that the introduction of Eqs.(7)-(8) may invalidate the assumption that the
Liouville theorem reflects the properties of systems obeying Hamilton’s
equations. The reason for this is that, if the gradients of entropy in phase
space do not equilibrate, then the time derivative of the phase-space density
may not necessarily be zero. This, in turn, means that the set of states that
a system can possibly attain form a volume in phase space $\Gamma$
representing a ”fluid” that may be compressible.

This result may be significant because it challenges the conventional
understanding of the behavior of systems obeying Hamilton’s equations and
implies that the dynamics of such systems may be more complex than previously
thought. Moreover, the idea that the phase space of a system may be
compressible has important implications for understanding the thermodynamics
of such systems and may have applications in fields such as statistical
physics and fluid dynamics (due to non-conservative forces, such as turbulence
or viscous dissipation).

## Competing Financial Interests

No targeted financial assistance was provided for this research by any public,
private, or non-profit entity.

## References

  * [1] Liouville, J., “Note sur la Théorie de la Variation des constantes arbitraires´´, Journal de Mathématiques Pures et Appliquées, pp. 342-349 (1838)
  * [2] Poincaré, Henri, “Sur le problème des trois corps et les équations de la dynamique, Acta mathematica, 13, pp. 1-270 (1890)
  * [3] Poincaré, H. (1892). Les méthodes nouvelles de la mécanique céleste: vol. III. Gauthier-Villars.
  * [4] Vladimir I Arnol’d, Russ. Math. Surv. 18, 85 (1963)
  * [5] Arnold, V. I. (1989). Mathematical methods of classical mechanics. Springer-Verlag
  * [6] Marsden, J. E., and Ratiu, T. S. (1994). Introduction to mechanics and symmetry: a basic exposition of classical mechanical systems. Springer-Verlag.
  * [7] Gallavotti, G. (2014). Foundations of fluid dynamics: with applications to physics and engineering. Springer.
  * [8] Toda, M. (1985). Theory of nonlinear lattices. Springer-Verlag
  * [9] Paul and Tatiana Ehrenfest, The Conceptual Foundations of the Statistical Approach in Mechanics, translated by M. J. Moravesik, Cornell University Press, 1959
  * [10] Aspect, A., Dalibard, Jean and Roger, Gérard, “Experimental Test of Bell’s Inequalities Using Time-Varying Analyzers´´, Phys. Rev. Lett. 49(25), pp. 1804–1807 (1982)
  * [11] Hensen, B., Bernien, H., Dréau, A. et al. Loophole-free Bell inequality violation using electron spins separated by 1.3 kilometres. Nature 526, 682–686 (2015). https://doi.org/10.1038/nature15759
  * [12] S. P. Walborn, M. O. Terra Cunha, S. Pádua, and C. H. Monken Phys. Rev. A 65, 033818 (2002)
  * [13] H. J. Kimble, M. Dagenais, and L. Mandel Phys. Rev. Lett. 39, 691 (1977)
  * [14] Alain Aspect, Philippe Grangier, and Gérard Roger Phys. Rev. Lett. 49, 91 (1982)
  * [15] Mario J. Pinheiro, ”A reformulation of mechanics and electrodynamics”, Heliyon 3(7), e00365 (2017)
  * [16] L. D. Landau and E. M. Lifshitz, Statistical Physics, Third Edition, Part 1: Volume 5 (Course of Theoretical Physics, Volume 5) (Elsevier, Amsterdam, 1980)
  * [17] V. Yu. Denisov, Phys. Lett. B 526 315-321 (2002)
  * [18] M. Léontovitch, Introduction à la thermodynamique-Physique Statistique, (Editions MIR, Moscou, 1986)
  * [19] A. G. Ruggiero, ”Are We Beating Liouville’s Theorem?”, Proceedings, Workshop on Producing High Luminosity High Energy Proton - Antiproton Collisions: Berkeley, CA, USA, March 27-31, 1978, Published in: eConf C7803272 (1978) 123.
  * [20] R. Winston, ”Beating the optical Liouville theorem,” in Optics and Photonics for Advanced Energy Technology, OSA Technical Digest (CD) (Optical Society of America, 2009), paper WD8.
  * [21] V. S. Datsko, ”The Fallacy of Louville´s Theorem and Predictable Atmosphere and Climate”, Proceedings of the International conference InterCarto.InterGIS, 2016;22(2), pp. 311–321.
  * [22] C. Villani, ”Chapter 2: A review of mathematical topics in collisional kinetic theory,” pp. 71–74, North-Holland, Washington, D.C., doi:10.1016/S1874-5792(02)80004-0, 2002.
  * [23] Pinheiro, M. J., ”Rabi Cavity Model”, (2023), GitHub repository, Available at: https://github.com/mjgpinheiro/Physics_models/blob/main/Rabi_cavity_3.ipynb
  * [24] Prodan, I., Scherson, D., and Metiu, H. (1994). Phase-space compression and the production of intense, collimated atomic beams. Physical Review Letters, 73(22), 2992.
  * [25] Pierce, D. T. (1983). Bright source of slow atomic beams. Physical Review Letters, 51(23), 2205.
  * [26] Sheehy, B., Schlosser, M., and Raizen, M. G. (1999). Atomic beam brightness beyond the Liouville limit. Physical Review Letters, 82(16), 3198.
  * [27] Kuyatt, C. E. (1986). The Liouville Theorem and its Applications in Geometry and Physics (Vol. 38). Springer.
  * [28] R. Bandyopadhyay, A. Chasapis, W. H. Matthaeus, T. N. Parashar, C. C. Haggerty, M. A. Shay, D. J. Gershman, B. L. Giles, and J. L. Burch , ”Energy dissipation in turbulent reconnection”, Physics of Plasmas 28, 112305 (2021)


# Revising the Problem of Partial Labels from the Perspective of CNNs’
Robustness

1st Xin Zhang  University of Southern Maine
Portland, United States
<EMAIL_ADDRESS>2nd Yuqi Song  University of Southern Maine
Portland, United States
<EMAIL_ADDRESS>3rd Wyatt McCurdy  University of Southern Maine
Portland, United States
<EMAIL_ADDRESS>4th Xiaofeng Wang  University of South Carolina
Columbia, United States
<EMAIL_ADDRESS>5th Fei Zuo  University of Central Oklahoma
Edmond, United States
<EMAIL_ADDRESS>
###### Abstract

Convolutional neural networks (CNNs) have gained increasing popularity and
versatility in recent decades, finding applications in diverse domains. These
remarkable achievements are greatly attributed to the support of extensive
datasets with precise labels. However, annotating image datasets is intricate
and complex, particularly in the case of multi-label datasets. Hence, the
concept of partial-label setting has been proposed to reduce annotation costs,
and numerous corresponding solutions have been introduced. The evaluation
methods for these existing solutions have been primarily based on accuracy.
That is, their performance is assessed by their predictive accuracy on the
test set. However, we insist that such an evaluation is insufficient and one-
sided. On one hand, since the quality of the test set has not been evaluated,
the assessment results are unreliable. On the other hand, the partial-label
problem may also be raised by undergoing adversarial attacks. Therefore,
incorporating robustness into the evaluation system is crucial. For this
purpose, we first propose two attack models to generate multiple partial-label
datasets with varying degrees of label missing rates. Subsequently, we
introduce a lightweight partial-label solution using pseudo-labeling
techniques and a designed loss function. Then, we employ D-Score to analyze
both the proposed and existing methods to determine whether they can enhance
robustness while improving accuracy. Extensive experimental results
demonstrate that while certain methods may improve accuracy, the enhancement
in robustness is not significant, and in some cases, it even diminishes.

###### Index Terms:

computer vision, multi-label classification, CNN robustness, partial labels

##  I Introduction

Convolutional Neural Networks (CNNs), with the ability to extract and learn
features automatically from raw data have revolutionized the field of computer
vision and have achieved state-of-the-art results on various tasks, including
image classification [1], object detection [2], semantic segmentation [3], and
so on [4, 5, 6]. Recently, with the rapid development of deep learning
techniques, CNNs have also become an indispensable tool for many real-world
computer vision applications, such as self-driving cars [7], security and
surveillance systems [8], and medical diagnosis [9].

The tremendous success of CNNs is largely attributed to the support of
accurate labeling. However, acquiring precise annotations is quite expensive.
To economize on annotation costs, previous endeavors introduced the notion of
the ’partial-label setting’ and suggested various methodologies to tackle this
problem, enabling CNNs to employ only a fraction of the labels during training
[10, 11, 12]. After a comprehensive review of the literature on the partial-
label problem, we noticed that prior works have primarily assessed their
proposed methods based on accuracy alone. We consider it one-sided to conclude
the effectiveness of the proposed methods in addressing the partial-label
problem solely based on this type of evaluation. On the one hand, previous
works merely demonstrated improved accuracy of their proposed solutions on
predefined test sets without evaluating the test sets themselves. Therefore,
such evaluation results may not be reliable. On the other hand, besides saving
annotation costs, adversarial attacks are one of the reasons for partial-label
problems. Extensive prior research has proven that CNNs are vulnerable to
adversarial attacks [13], which is a type of attack used to deteriorate the
performance of CNNs targeting datasets, image features, label information, or
the models themselves. CNNs with poor robustness often experience significant
performance degradation when subjected to adversarial attacks. Therefore, we
insist that analyzing the partial-label problem solely from the perspective of
accuracy is one-sided. We also need to analyze it from the standpoint of
robustness, that is, analyzing CNN’s robustness with respect to label removal.

To conduct an analysis of the partial-label problem from a robustness
perspective, we first require datasets that have been subjected to adversarial
attacks. Such datasets should consist of training images where only a portion
of the labels is known after the adversarial attacks. The current datasets are
either fully labeled or partially labeled with a fixed quantity of missing
labels, making it challenging to effectively verify how proposed methods are
affected by varying degrees of label loss. For this purpose, we initially
propose two attack models: random attacks $\mathcal{R}_{p}$ and targeted
attacks $\mathcal{T}_{p}$. The former randomly removes $p\%$ of the labels
from the images in the training set, irrespective of whether they are positive
or negative labels. The latter selectively targets only the positive labels in
the training set, removing $p\%$ of the positive labels while preserving all
the negative labels. Furthermore, we introduce a lightweight solution to the
partial-label problem. It leverages pseudo-labeling techniques and a well-
designed loss function. Moreover, to evaluate whether our method and existing
approaches enhance robustness concerning label removal, besides using the mAP
evaluation metric, we also employed the D-Score [14] analysis method to assess
the robustness of these methods.

Our Contributions are summarized as follows:

  * •

We propose two adversarial attack models targeting image labels: targeted
attacks and random attacks. These attack methods selectively remove certain
labels, transforming the full-label setting into a partial-label setting.
Experimental results demonstrate that this attack effectively reduces the
performance of existing STOA methods.

  * •

We present a lightweight approach to address the partial-label problem, which
is achieved through the utilization of pseudo-labeling techniques and an
improved loss function, without the need for additional statistical
information or network structures.

  * •

The extensive experiments on three large-scale public image datasets (COCO,
NUS-WIDE, and Pascal VOC) demonstrate that our method outperforms the STOA
methods, both in terms of accuracy (mAP) and robustness (D-Score).

The rest of the paper is organized as follows. Section II discusses the
related work. Our proposed method is presented in Section III. Section IV
shows the experimental settings and results. Finally, conclusions are drawn in
Section V.

##  II Related Works

###  II-A Partial-label Problems

The partial-label problem means that for one input image in the training set,
only a subset of all the labels for it can be observed and the rest remains
unknown during the training process [10, 12]. Addressing this problem is
meaningful for saving annotation costs.

A straightforward approach for the partial-label problem is BR [15], which
decomposes the task into a number of binary classification problems, each for
one label. Such an approach encounters many difficulties, mainly due to
ignoring correlations between labels. PU-learning is an alternative solution
[16], which studies the problem with a small number of positive examples and a
large number of unlabeled examples for training. Most methods can be divided
into the following three categories: two-step techniques [17], biased learning
[18], and class prior incorporation [19]. However, all these methods require
that the training data consists of positive and unlabeled examples [20].
Pseudo-label [10] is another solution. Pseudo-labeling was first proposed in
[21]. The goal of pseudo-labeling in partial-label problems is to generate
pseudo-labels for the unobserved part [11].

###  II-B The Evaluation for CNNs’ Robustness

To evaluate CNNs, researchers have proposed several approaches, which can be
divided into two categories. The first category involves introducing the
traditional software engineering testing method, mutation testing, to CNNs
[22, 23, 24]. This approach applies carefully designed mutation operators [25]
to the CNN model to generate multiple variants. The higher the number of
differences between the predictions of the variant models and the original
model, the higher the quality of the test set. However, the score itself
remains a black box, and the reasons behind the low quality of the test set
are still unknown. Additionally, effective methods for selecting and combining
mutation operators to detect test set quality remain unexplored [26]. The
second category of approaches is based on neuron coverage [27, 28, 29]. These
methods use gradient ascent to solve a joint optimization problem that
maximizes both neuron coverage and the number of potentially erroneous
behaviors, and eventually generate a set of test inputs [27]. However, as
noted in [30], higher neuron coverage can lead to fewer defects detected, less
natural inputs, and more biased prediction preferences. Therefore, developing
effective methods for providing white-box scores for CNNs and proposing
methods for enhancing these scores is critical for improving robustness and
accuracy of CNNs.

##  III Methodology

In this section, we introduce details of our proposed methods, including two
adversarial attack models, the solution to the partial-label problem, and the
evaluation methods.

###  III-A Simulation of Targeting-label Attack

Label removal is one of the most prevalent adversarial attacks that
specifically targets labels. It operates by altering the label distribution
through the removal of ground-truth labels, thereby diminishing the model’s
accuracy and potentially impeding the training process. To validate our
method’s efficacy in combating adversarial attacks and fortifying the CNN’s
robustness, we must initially simulate this targeted label attack. We’ve
devised two attack models based on the positive or negative attributes of the
targeted labels.

  * •

Targeted attacks. In this attack model, we directly eliminate all negative
labels and a certain proportion of positive labels. We designate this attack
model as $\mathcal{T}_{q}$, where $q$ represents the deletion percentage of
positive labels. Essentially, this attack method removes $\hat{q}$ percent of
the labels.:

$\hat{q}\%=\frac{t_{n}+q\%\times t_{p}}{t}$

where $t$, $t_{n}$, and $t_{p}$ stand for the number of total labels, the
number of negative labels, and the number of positive labels, respectively.

  * •

Random attacks. In this attack model, we do not distinguish between positive
and negative labels; instead, we directly delete labels based on a specific
proportion. We denote this attack model as $\mathcal{R}_{q}$, where $q$
represents the percentage of labels deleted. It is worth noting that this
attack model could result in an extreme scenario where all positive labels are
removed. This implies that for an image in the training set, its corresponding
label contains only negative labels. This situation could easily lead the
model to generate a trivial solution, significantly reducing its accuracy.
Hence, when designing a solution, it is crucial to address this extreme label
imbalance.

These two attacking models are summarized in Figure. 1.

Figure 1: Two attacking models. Variant T is generated under attack
$\mathcal{T}$, which removes all negative labels and some positive labels.
Variant R, on the other hand, is generated under attack $\mathcal{R}$, where
the positive or negative nature of the label is disregarded, and the deletion
of labels is entirely random.

###  III-B The Solution for Targeting-label Attack

To avoid introducing additional computational burden to the model, we propose
a lightweight solution that involves modifying only the loss function to
enhance the CNN’s robustness against targeting-label attacks.

Pseudo-label. We can divide the labels associated with an image into two
parts: $E$, which persists after the attack, and $N$, the labels removed due
to the attack. For multi-label classification tasks, Binary Cross-Entropy
(BCE) commonly serves as the primary loss function, as shown in Equation. 1,

$\mathcal{L}_{bce}(\hat{y},y)=-\frac{1}{L}\sum_{i=1}^{L}[(y_{i}\log(\hat{y}_{i})+(1-y_{i})\log(1-\hat{y}_{i})].$
(1)

where $y$ and $\hat{y}$ stand for the ground-truth labels and predictions of
the classifier, respectively. For part $E$, we compute the loss using the
original ground-truth as the target. For the $N$ part, we introduce pseudo-
labels and employ them as targets to calculate the loss. The pseudo-label
begins with an initialization of 1, and its value undergoes updates using a
historical stack. This stack retains the model’s predictions for this label
from the previous three epochs. These processes are summarized in Equation. 2,

$\displaystyle\widetilde{y}_{i_{j}}=\begin{cases}f(S_{i_{j}},\alpha,\beta,\gamma),&\textit{update}\\\
1,&\textit{initialization}\end{cases}$ (2)

where $S_{i_{j}}$ stands for the historical stack, which always reserves the
predictions of the last three epochs for the $i$th image’s $j$th category,
that is, $size(S_{i_{j}})=3$. $\alpha$, $\beta$, and $\gamma$ stand for the
weight for the three elements in $S_{i_{j}}$ during the calculation of
function $f(\cdot)$,

$f(S_{i_{j}},\alpha,\beta,\gamma)=\alpha S_{i_{j}}[0]+\beta
S_{i_{j}}[1]+\gamma S_{i_{j}}[2],$ (3)

where $S_{i_{j}}[k]=\hat{y}_{i_{j}}^{e-k}$, $e$ represents the index of
current epoch number, and $\hat{y}_{i_{j}}$ stands for the prediction for the
$i$th image’s $j$th category. The values of $\alpha$, $\beta$, and $\gamma$
are decided by extensive experiments, and and $\alpha+\beta+\gamma=1$.

It is essential to note that initializing the pseudo-label as 1 stems from the
prevalence of numerous negative labels in image datasets. This often leads to
label imbalance, potentially prompting the model to generate trivial
solutions, that is, directly predicting each category as negative. The
initialization of the pseudo-label as 1 effectively alleviates this issue.
While updating the pseudo-label, the historical stack aids in tracking the
label value fluctuations over the past three instances, ensuring a smoother
update.

Loss function. We employ Binary Cross Entropy (BCE) as our loss function. In
the part $E$, we compute the loss value using ground-truth as the target,
while for the $N$ part, we calculate the loss value using the pseudo label as
the target, as illustrated in Equation. 4,

$\mathcal{L}=\mathcal{L}_{bce}(\hat{y},y)+\mathcal{L}_{bce}(\hat{y},\widetilde{y}),$
(4)

where $y$, $\hat{y}$, and $\widetilde{y}$ stand for the ground-truth labels,
the predictions, and the pseudo labels respectively. In Equation. 4, the first
term represents for the loss value of the $E$ part, and the second term stands
for the $N$ part. Building upon this, we introduce an attention-shifting
parameter $e(\cdot)$ to progressively redirect attention from the part $E$ to
the part $N$ during the loss function computation. The rationale behind this
design is that at the initial training stages, since pseudo labels are
initially set to 1, they may significantly deviate from the actual labels,
potentially resulting in unreliable loss value calculations for this part.
Therefore, during the early training phase, we aim to focus more on the $E$
part. As training advances and the pseudo labels are continually updated,
their reliability gradually increases. Consequently, as training progresses,
we gradually shift attention towards the $N$ part.

In previous research, the attention-shifting parameter $e(\cdot)$ has often
been applied using a linear function. However, in this context, we employ an
exponential function for implementation, as Equation 5,

$e(n_{c},n_{t})=e^{n_{c}-n_{t}},$ (5)

where $n_{c}$ and $n_{t}$ stand for the index of the current epoch number and
the total epoch numbers respectively. Compared to the linear function, the
exponential function exhibits faster changes toward the end of the training
process, while its alterations are more gradual during the initial training
stages. This approach ensures that the pseudo labels have ample time for
updating during training and gradually become dominant in calculating the loss
value as training progresses.

In addition, to prevent the occurrence of trivial solutions, we design an
approach for penalizing such outcomes. This approach calculates the difference
between the current model’s predictions and the trivial solution. When this
disparity is tiny, it indicates that the current model has potentially
produced a trivial solution. In such cases, we apply a penalty $\mathcal{P}$
to these predictions. We use the L2 norm to compute the difference between the
predictions and the trivial solution, as depicted in Equation. 6,

$\mathcal{P}=1-\sqrt{\sum_{i=0}^{m}(\hat{y}_{i}-\widetilde{y}_{i})^{2}},$ (6)

where $m$ stands for the number of existing labels for this image, and because
$\sqrt{\sum_{i=0}^{m}(\hat{y}_{i}-\widetilde{y}_{i})^{2}}\in[0,1]$,
$\mathcal{P}$ is always larger than 0.

Combining all these, our final loss function is shown as Equation. 7,

$\mathcal{L}=\mathcal{L}_{bce}(\hat{y},y)+e^{n_{c}-n_{t}}\times\mathcal{L}_{bce}(\hat{y},\widetilde{y})+\mathcal{P},$
(7)

###  III-C Evaluating the Robustness of Proposed Solutions

To demonstrate that our proposed solutions can indeed improve the robustness,
we adopt D-Score [14] to analyze the presented method and other comparisons.
D-Score is a quantitative method for analyzing the robustness of CNNs. It
analyzes the model’s attention distribution and the dataset’s feature
distribution through the deletion of mutation operators and feature shifting.
Subsequently, it evaluates the CNN’s robustness by computing the similarity
between these two distributions.

##  IV Experiments

The detailed experimental settings and results are summarized in this section.

###  IV-A Datasets

We conduct comprehensive experiments on three large-scale multi-label image
datasets: COCO [31], NUS-WIDE [32], and Pascal VOC [33]. Each instance in
these three datasets is fully annotated with clean labels that can be used as
the GT in performance evaluation.

###  IV-B Network Structure and Hyper-Parameters

Following [10], we adopt the same network structure by using an end-to-end
network for all experiments: a ResNet-50 [34], pre-trained on ImageNet [35],
as the backbone and a fully connected layer, which is the same as the multi-
label classifier under FOL setting. Our approach does not add any extra
structure to the network.

Additionally, we also follow the same training hyperparameter selection to
control variables. That is, we train our classifier for 10 epochs, and for the
learning rate and batch size, we use a hyperparameter search method and select
the hyperparameters with the bast mAP on the validation set, where the
learning rate is in $[1e-3,1e-4,1e-5,1e-6]$ and batch size is in $[8,16]$.

###  IV-C Experimental Results

The effectiveness of attacking models. To demonstrate the significant threat
our proposed attack models pose to CNNs, we opted to assess the performance of
two commonly employed solutions for multi-label classification problems:
Binary Cross-Entropy (BCE) and BCE with Label Smoothing (BCE-LS). This
evaluation was conducted on COCO and Pascal VOC, two extensive image datasets,
under varying degrees of attack. The experimental results are shown in Figure.
2, which fully demonstrate the effectiveness of our proposed attacking models.

Figure 2: The effectiveness of our attacking models in decreasing the mAP
scores. The first row is conducted on the COCO dataset, and the second row is
on Pascal VOC.

The effectiveness of our proposed method in addressing the partial-label
problem. To demonstrate the effectiveness of our approach, we choose several
state-of-the-art methods as comparisons, including AN [36], WAN [37], ROLE
[38]. Then, we conduct targeted attacking and random attacking on three public
image datasets, that is, COCO, Pascal VOC and NUS-WIDE, to generate several
variants of training sets. The experimental results under targeted attacking
are summarized in Table. I. There is also a special variation in this table,
namely $\mathcal{T}_{s}$. This variant is generated through a specific form of
targeted attack, wherein for each image, we retain only one positive label and
eliminate all other labels. This variant holds particular practical
significance: in such scenarios, we only require a single annotation for each
image, leading to a substantial reduction in annotation costs. Hence, we
specifically examine the performance of our approach concerning this variant.
Table. II summarizes the results by random attacking. These results
demonstrate the effectiveness of our proposed method in addressing the
partial-label problem and resisting the two proposed attacks.

TABLE I: The mAP results on COCO, Pascal VOC, and NUSWIDE datasets attacked by $\mathcal{T}$. $\mathcal{T}_{0.2}$, $\mathcal{T}_{0.4}$, and $\mathcal{T}_{0.6}$ represent randomly retaining 80%, 60% and 40% of positive labels for each instance, respectively, and discarding all negative labels. In each column, we bold the best-performing method. | COCO | Pascal VOC | NUSWIDE
---|---|---|---
$\mathcal{T}_{s}$ | $\mathcal{T}_{0.6}$ | $\mathcal{T}_{0.4}$ | $\mathcal{T}_{0.2}$ | $\mathcal{T}_{s}$ | $\mathcal{T}_{0.6}$ | $\mathcal{T}_{0.4}$ | $\mathcal{T}_{0.2}$ | $\mathcal{T}_{s}$ | $\mathcal{T}_{0.6}$ | $\mathcal{T}_{0.4}$ | $\mathcal{T}_{0.2}$
AN [36] | 63.9 | 66.4 | 67.0 | 69.2 | 84.7 | 86.8 | 87.3 | 88.0 | 40.0 | 45.1 | 48.3 | 50.3
WAN [37] | 64.8 | 69.1 | 70.8 | 71.2 | 85.9 | 87.5 | 87.9 | 88.0 | 43.7 | 46.9 | 48.5 | 50.6
ROLE [38] | 65.9 | 73.1 | 75.4 | 77.1 | 87.0 | 88.9 | 90.0 | 90.3 | 43.2 | 48.0 | 50.4 | 52.0
Ours | 67.1 | 74.0 | 75.8 | 77.4 | 87.2 | 89.2 | 90.3 | 90.8 | 44.9 | 48.7 | 50.9 | 52.8
TABLE II: The mAP results on COCO, Pascal VOC, and NUSWIDE datasets attacked by $\mathcal{R}$. $\mathcal{R}_{0.2}$, $\mathcal{R}_{0.4}$, and $\mathcal{R}_{0.6}$ represent randomly retaining 80%, 60% and 40% of labels for each instance, respectively. In each column, we bold the best-performing method. | COCO | Pascal VOC | NUSWIDE
---|---|---|---
$\mathcal{R}_{0.6}$ | $\mathcal{R}_{0.4}$ | $\mathcal{R}_{0.2}$ | $\mathcal{R}_{0.6}$ | $\mathcal{R}_{0.4}$ | $\mathcal{R}_{0.2}$ | $\mathcal{R}_{0.6}$ | $\mathcal{R}_{0.4}$ | $\mathcal{R}_{0.2}$
AN [36] | 60.1 | 62.3 | 63.7 | 78.6 | 80.1 | 82.3 | 38.7 | 40.5 | 44.1
WAN [37] | 60.8 | 63.1 | 64.3 | 80.2 | 81.7 | 84.4 | 40.2 | 41.1 | 43.7
ROLE [38] | 61.1 | 63.9 | 65.0 | 81.0 | 83.1 | 84.9 | 40.8 | 41.9 | 44.2
Ours | 61.6 | 64.3 | 66.0 | 81.8 | 85.4 | 85.9 | 42.1 | 42.7 | 44.7

The analysis of robustness. We summarize the results of the D-Score analysis
in Table. III.

TABLE III: The mAP, robustness score, fitness score and D-Score results undergoing $\mathcal{T}_{0.4}$ on COCO and Pascal VOC. $\mathcal{T}_{0.4}$ represents randomly retaining 60% of positive labels for each instance, respectively, and discarding all negative labels. For the robustness score, the smaller the better. As for the mAP, fitness score and D-Score, the higher the better. In each column, we bold the best-performing method and underline the second-best one. | COCO $\mathcal{T}_{0.4}$ | Pascal VOC $\mathcal{T}_{0.4}$
---|---|---
mAP | robust | fitness | D-Score | mAP | robust | fitness | D-Score
AN [36] | 67.0 | 0.397 | 0.765 | 0.368 | 87.3 | 0.288 | 0.816 | 0.528
WAN [37] | 70.8 | 0.416 | 0.777 | 0.361 | 87.9 | 0.306 | 0.831 | 0.525
Role [38] | 75.4 | 0.289 | 0.801 | 0.512 | 90.0 | 0.261 | 0.847 | 0.568
Ours | 75.8 | 0.272 | 0.809 | 0.537 | 90.3 | 0.237 | 0.852 | 0.615

##  V Conclusion

The remarkable success of CNNs relies heavily on the support of large, high-
quality labeled datasets. However, acquiring such datasets is costly due to
the extensive manual annotation involved, particularly in multi-label
datasets. To address this challenge, numerous methods have been proposed to
train CNNs using partial-label datasets. Yet, the evaluation of these
solutions has been limited to accuracy, which we deem insufficient. It is
crucial to include robustness in the assessment, as the quality of the test
sets used for evaluation remains unverified and the partial-label issue may
stem from adversarial attacks, closely linked to CNNs’ robustness. To tackle
these challenges, we introduce two adversarial attack models aimed at removing
specific labels and generating partial-label datasets. Subsequently, we
propose a lightweight solution for partial-label problems using pseudo-label
techniques. Finally, we conduct an analysis using D-Score and mAP evaluation
metrics to assess both the robustness and accuracy of our proposed method and
some state-of-the-art methods. Experimental results demonstrate that while our
method significantly enhances accuracy, it also notably improves robustness.
Conversely, certain existing methods exhibit improved accuracy but a
simultaneous decline in robustness.

## References

  * [1] D. Lu and Q. Weng, “A survey of image classification methods and techniques for improving classification performance,” _International journal of Remote sensing_ , vol. 28, no. 5, pp. 823–870, 2007.
  * [2] Z. Zou, K. Chen, Z. Shi, Y. Guo, and J. Ye, “Object detection in 20 years: A survey,” _Proceedings of the IEEE_ , 2023.
  * [3] N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, “Indoor segmentation and support inference from rgbd images,” in _European conference on computer vision_. Springer, 2012, pp. 746–760.
  * [4] Y. Song, M. Gao, J. Yu, and Q. Xiong, “Social recommendation based on implicit friends discovering via meta-path,” in _2018 IEEE 30th International Conference on Tools with Artificial Intelligence (ICTAI)_. IEEE, 2018, pp. 197–204.
  * [5] Y. Song, E. M. D. Siriwardane, Y. Zhao, and J. Hu, “Computational discovery of new 2d materials using deep learning generative models,” _ACS Applied Materials & Interfaces_, vol. 13, no. 45, pp. 53 303–53 313, 2021.
  * [6] R. Dong, Y. Zhao, Y. Song, N. Fu, S. S. Omee, S. Dey, Q. Li, L. Wei, and J. Hu, “Deepxrd, a deep learning model for predicting xrd spectrum from material composition,” _ACS Applied Materials & Interfaces_, vol. 14, no. 35, pp. 40 102–40 115, 2022.
  * [7] S. Grigorescu, B. Trasnea, T. Cocias, and G. Macesanu, “A survey of deep learning techniques for autonomous driving,” _Journal of Field Robotics_ , vol. 37, no. 3, pp. 362–386, 2020.
  * [8] V. Kafedziski, S. Pecov, and D. Tanevski, “Detection and classification of land mines from ground penetrating radar data using faster r-cnn,” in _2018 26th telecommunications forum (TELFOR)_. IEEE, 2018, pp. 1–4.
  * [9] G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. Van Der Laak, B. Van Ginneken, and C. I. Sánchez, “A survey on deep learning in medical image analysis,” _Medical image analysis_ , vol. 42, pp. 60–88, 2017.
  * [10] X. Zhang, R. Abdelfattah, Y. Song, and X. Wang, “An effective approach for multi-label classification with missing labels,” in _2022 IEEE 24th Int Conf on High Performance Computing & Communications; 8th Int Conf on Data Science & Systems; 20th Int Conf on Smart City; 8th Int Conf on Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys)_. IEEE, 2022, pp. 1713–1720.
  * [11] W. Shi, Y. Gong, C. Ding, Z. M. Tao, and N. Zheng, “Transductive semi-supervised deep learning using min-max features,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2018, pp. 299–315.
  * [12] X. Zhang, Y. Song, F. Zuo, Z. Zhou, and X. Wang, “Towards imbalanced large scale multi-label classification with partially annotated labels,” in _2023 IEEE/ACIS 21st International Conference on Software Engineering Research, Management and Applications (SERA)_. IEEE, 2023, pp. 195–200.
  * [13] S. Qiu, Q. Liu, S. Zhou, and C. Wu, “Review of artificial intelligence adversarial attack and defense technologies,” _Applied Sciences_ , vol. 9, no. 5, p. 909, 2019.
  * [14] X. Zhang, Y. Song, X. Wang, and F. Zuo, “D-score: A white-box diagnosis score for cnns based on mutation operators,” _arXiv preprint arXiv:2304.00697_ , 2023.
  * [15] M.-L. Zhang, Y.-K. Li, X.-Y. Liu, and X. Geng, “Binary relevance for multi-label learning: an overview,” _Frontiers of Computer Science_ , vol. 12, no. 2, pp. 191–202, 2018.
  * [16] X. Li and B. Liu, “Learning to classify texts using positive and unlabeled data,” in _IJCAI_ , vol. 3, no. 2003. Citeseer, 2003, pp. 587–592.
  * [17] B. Liu, W. S. Lee, P. S. Yu, and X. Li, “Partially supervised classification of text documents,” in _ICML_ , vol. 2, no. 485. Sydney, NSW, 2002, pp. 387–394.
  * [18] S. Sellamanickam, P. Garg, and S. K. Selvaraj, “A pairwise ranking based approach to learning with positive and unlabeled examples,” in _Proceedings of the 20th ACM international conference on Information and knowledge management_ , 2011, pp. 663–672.
  * [19] L. Jiang, H. Zhang, and Z. Cai, “A novel bayes model: Hidden naive bayes,” _IEEE Transactions on knowledge and data engineering_ , vol. 21, no. 10, pp. 1361–1371, 2008.
  * [20] J. Bekker and J. Davis, “Learning from positive and unlabeled data: A survey,” _Machine Learning_ , vol. 109, no. 4, pp. 719–760, 2020.
  * [21] D.-H. Lee _et al._ , “Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks,” in _Workshop on challenges in representation learning, ICML_ , vol. 3, no. 2, 2013, p. 896.
  * [22] Q. Hu, L. Ma, X. Xie, B. Yu, Y. Liu, and J. Zhao, “Deepmutation++: A mutation testing framework for deep learning systems,” in _2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE)_. IEEE, 2019, pp. 1158–1161.
  * [23] N. Humbatova, G. Jahangirova, and P. Tonella, “Deepcrime: mutation testing of deep learning systems based on real faults,” in _Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis_ , 2021, pp. 67–78.
  * [24] W. Shen, J. Wan, and Z. Chen, “Munn: Mutation analysis of neural networks,” in _2018 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C)_. IEEE, 2018, pp. 108–115.
  * [25] L. Ma, F. Zhang, J. Sun, M. Xue, B. Li, F. Juefei-Xu, C. Xie, L. Li, Y. Liu, J. Zhao _et al._ , “Deepmutation: Mutation testing of deep learning systems,” in _2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE)_. IEEE, 2018, pp. 100–111.
  * [26] A. Panichella and C. C. Liem, “What are we really testing in mutation testing for machine learning? a critical reflection,” in _2021 IEEE/ACM 43rd International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER)_. IEEE, 2021, pp. 66–70.
  * [27] K. Pei, Y. Cao, J. Yang, and S. Jana, “Deepxplore: Automated whitebox testing of deep learning systems,” in _proceedings of the 26th Symposium on Operating Systems Principles_ , 2017, pp. 1–18.
  * [28] Y. Feng, Q. Shi, X. Gao, J. Wan, C. Fang, and Z. Chen, “Deepgini: prioritizing massive tests to enhance the robustness of deep neural networks,” in _Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis_ , 2020, pp. 177–188.
  * [29] J. Yu, Y. Fu, Y. Zheng, Z. Wang, and X. Ye, “Test4deep: an effective white-box testing for deep neural networks,” in _2019 IEEE international conference on computational science and engineering (CSE) and IEEE international conference on embedded and ubiquitous computing (EUC)_. IEEE, 2019, pp. 16–23.
  * [30] F. Harel-Canada, L. Wang, M. A. Gulzar, Q. Gu, and M. Kim, “Is neuron coverage a meaningful measure for testing deep neural networks?” in _Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering_ , 2020, pp. 851–862.
  * [31] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in _European conference on computer vision_. Springer, 2014, pp. 740–755.
  * [32] T.-S. Chua, J. Tang, R. Hong, H. Li, Z. Luo, and Y. Zheng, “Nus-wide: a real-world web image database from national university of singapore,” in _Proceedings of the ACM international conference on image and video retrieval_ , 2009, pp. 1–9.
  * [33] M. Everingham, S. Eslami, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman, “The pascal visual object classes challenge: A retrospective,” _International journal of computer vision_ , vol. 111, no. 1, pp. 98–136, 2015.
  * [34] S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He, “Aggregated residual transformations for deep neural networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 1492–1500.
  * [35] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and F.-F. Li, “Imagenet: A large-scale hierarchical image database,” in _2009 IEEE conference on computer vision and pattern recognition_. Ieee, 2009, pp. 248–255.
  * [36] K. Kundu and J. Tighe, “Exploiting weakly supervised visual patterns to learn from partial annotations,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 561–572, 2020.
  * [37] O. Mac Aodha, E. Cole, and P. Perona, “Presence-only geographical priors for fine-grained image classification,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 9596–9606.
  * [38] E. Cole, O. Mac Aodha, T. Lorieul, P. Perona, D. Morris, and N. Jojic, “Multi-label learning from single positive labels,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 933–942.