# Soft streaming – flow rectification via elastic boundaries

Yashraj Bhosale  Mechanical Sciences and Engineering, University of Illinois
at Urbana-Champaign, Urbana, IL 61801, USA  Tejaswin Parthasarathy  Mechanical
Sciences and Engineering, University of Illinois at Urbana-Champaign, Urbana,
IL 61801, USA  Mattia Gazzola<EMAIL_ADDRESS>Mechanical Sciences and
Engineering, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA

###### Abstract

Viscous streaming is an efficient mechanism to exploit inertia at the
microscale for flow control. While streaming from rigid features has been
thoroughly investigated, when body compliance is involved, as in biological
settings, little is known. Here, we investigate body elasticity effects on
streaming in the minimal case of an immersed soft cylinder. Our study reveals
an additional streaming process, available even in Stokes flows. Paving the
way for advanced forms of flow manipulation, we illustrate how gained insights
may translate to complex geometries beyond circular cylinders.

††preprint: APS/123-QED

This paper examines the role of body elasticity in two-dimensional viscous
streaming. Viscous streaming [1, 2, 3], an inertial phenomenon, refers to the
time-averaged, rectified steady flows that arise when an immersed body of
length scale $a$ undergoes small-amplitude oscillations in a viscous fluid.
Long understood for rigid bodies of uniform curvature, such as cylinders [1]
or spheres [2], viscous streaming has found application in microfluidics [4,
5, 6, 7, 8, 9, 10, 11, 12, 13], from chemical mixing [14, 4, 5, 15] to vesicle
transport [16, 6], due to its ability to reconfigure flow and particle
trajectories within short length –$\mathcal{O}\left(100\right)\mu$m– and time
–$\mathcal{O}\left(10^{-3}\right)$s– scales [17, 12]. Recent developments have
then furthered opportunities in transport, separation or assembly, through the
use of multi-curvature bodies and associated rich flow topologies [18, 19, 20,
21].

Despite progress, no effort has so far systematically considered the role of
body elasticity in viscous streaming. Yet, modulation by soft interfaces may
be relevant in a multitude of settings, from pulsatile physiological flows
[22, 23, 24] or conformal microfluidics [25, 26, 27] to elastic mini-robots in
fluids [28, 29, 30, 31], with relevance to both medicine and engineering. Soft
biological organisms, such as bacteria [32] or larvae [33], may also take
advantage of streaming for feeding or locomotion. Indeed, a back of the
envelope calculation reveals that a millimeter-size aquatic organism beating
its cilia at $\sim\mathcal{O}\left(10\right)$ Hz would operate at the edge of
viscous streaming viability. Supporting this hypothesis, steady flow patterns
and velocities ($\sim 10^{2}-10^{3}~{}\mu\textrm{m}$/s) consistent with
streaming have been observed in starfish and ribbon-worm larvae [33], although
being ascribed, perhaps inaccurately, to Stokes flow phenomena.

Motivated by these considerations, we dissect the effect of body elasticity on
viscous streaming in the minimal setting of an immersed, oscillating
hyperelastic circular cylinder. The major outcome is that, in these
conditions, the time-averaged streaming flow $\langle\psi_{1}\rangle$ reads

$\langle\psi_{1}\rangle=\sin 2\theta~{}\left[\Theta(r)+\Lambda(r)\right]$ (1)

where $r,\theta$ are cylindrical coordinates, $\Theta(r)$ is the classical
rigid body solution from Holtsmark  _et al._ [1], and $\Lambda(r)$ is a novel,
independent contribution from body elasticity.

Figure 1:  Problem setup. Elastic solid cylinder $\Omega_{e}$ of radius $a$
with a rigid inclusion (pinned zone $\Gamma$ of radius $b$), immersed in the
viscous fluid $\Omega_{f}$. The cylinder is exposed to an oscillatory flow
with far-field velocity $V(t)=\epsilon a\omega\cos(\omega t)$.

The above result is obtained by considering the setup shown in Fig. 1, where a
2D visco-elastic solid cylinder $\Omega_{e}$ with radius $a$ is immersed in a
viscous fluid $\Omega_{f}$. The fluid oscillates with velocity $V(t)=\epsilon
a\omega\cos\omega t$, where $\epsilon$, $\omega$ and $t$ represent the non-
dimensional amplitude, angular frequency and time, respectively. We ‘pin’ the
cylinder’s centre using a rigid inclusion $\Gamma$ of radius $b<a$, to
kinematically enforce zero strain and velocity near the cylinder’s centre. We
denote by $\partial\Omega$ and $\partial\Gamma$ the boundary between the
elastic solid and viscous fluid, and the boundary of the pinned zone,
respectively.

In this setup, we assume fluid and solid to be isotropic, incompressible and
of constant density. Furthermore, we assume the fluid to be Newtonian, with
kinematic viscosity $\nu_{f}$ and density $\rho_{f}$. We assume that the solid
exhibits visco-hyperelastic neo-Hookean behaviour, characteristic of soft
biological materials [34], with shear modulus $G$, kinematic viscosity
$\nu_{e}$ and density $\rho_{e}$.

The dynamics in the elastic and fluid phases, are described by the
incompressible Cauchy momentum equations, non-dimensionalized using the
characteristic scales of velocity $V=\epsilon a\omega$, length $L=a$ and time
$T=1/\omega$

Incomp.
$\displaystyle\begin{cases}\mbox{\boldmath$\nabla$}\cdot\mbox{\boldmath$v$}=0,~{}\mbox{\boldmath$x$}\in\Omega_{f}\cup\Omega_{e}\end{cases}$
(2) Fluid
$\displaystyle\begin{cases}\frac{\partial{\mbox{\boldmath$v$}}}{\partial{t}}+\epsilon({\mbox{\boldmath$v$}}\cdot{\bm{\nabla}}){\mbox{\boldmath$v$}}=\frac{1}{M^{2}}\left(-{\nabla}{p}+{\nabla^{2}}{\mbox{\boldmath$v$}}\right),~{}{\mbox{\boldmath$x$}}\in\Omega_{f}\end{cases}$
Solid $\displaystyle\begin{cases}\alpha
Cau\left(\frac{\partial{\mbox{\boldmath$v$}}}{\partial{t}}+\epsilon({\mbox{\boldmath$v$}}\cdot{\bm{\nabla}}){\mbox{\boldmath$v$}}\right)=\frac{Cau}{M^{2}}\left(-{\nabla}{p}+\beta{\nabla^{2}}{\mbox{\boldmath$v$}}\right)\\\
\hskip
99.58464pt+{\bm{\nabla}}\cdot({\bm{F}}{\bm{F}}^{T})^{\prime},~{}{\mbox{\boldmath$x$}}\in\Omega_{e},\end{cases}$

where $v$ and $p$ are the velocity and pressure fields, and $\bm{F}$ is the
deformation gradient tensor, defined as
$\bm{F}=\bm{I}+\bm{\nabla}\mbox{\boldmath$u$}$, where $\bm{I}$ is the
identity, $\mbox{\boldmath$u$}=\mbox{\boldmath$x$}-\mbox{\boldmath$X$}$ is the
material displacement field, and $x$, $X$ are the position of a material point
after deformation and at rest, respectively. The prime symbol ′ on a tensor
denotes its deviatoric. In addition, the following non-dimensional groups
naturally appear: scaled oscillation amplitude $\epsilon$, Womersley number
$M=a\sqrt{\rho_{f}\omega/\mu_{f}}$, Cauchy number
$Cau=\epsilon\rho_{f}a^{2}\omega^{2}/G$, density ratio
$\alpha=\rho_{e}/\rho_{f}$ and viscosity ratio $\beta=\mu_{e}/\mu_{f}$.
Physically, $M$ represents the ratio of inertial to viscous forces, and $Cau$
represents the ratio of inertial to elastic forces. Thus, increasing $M$
indicates an inertia-dominated environment, and increasing $Cau$ implies a
softer body. The equations are then closed using the boundary conditions

Pinned zone
$\displaystyle\begin{cases}\mbox{\boldmath$u$}=0,\mbox{\boldmath$v$}=0,~{}\mbox{\boldmath$x$}\in\Gamma\end{cases}$
(3.1) Interface velocity
$\displaystyle\begin{cases}\mbox{\boldmath$v$}_{e}=\mbox{\boldmath$v$}_{f},~{}\mbox{\boldmath$x$}\in\partial\Omega\end{cases}$
(3.2) Interface stresses
$\displaystyle\begin{cases}\bm{\sigma}_{f}=-p\bm{I}+(\bm{\nabla}\mbox{\boldmath$v$}+\bm{\nabla}\mbox{\boldmath$v$}^{T}),~{}\mbox{\boldmath$x$}\in\Omega_{f}\\\
\bm{\sigma}_{e}=-p\bm{I}+\beta(\bm{\nabla}\mbox{\boldmath$v$}+\bm{\nabla}\mbox{\boldmath$v$}^{T})\\\
~{}~{}~{}~{}+\frac{M^{2}}{Cau}(\bm{F}\bm{F}^{T})^{\prime},~{}\mbox{\boldmath$x$}\in\Omega_{e},\\\
\mbox{\boldmath$n$}\cdot\bm{\sigma}_{e}\cdot\mbox{\boldmath$n$}=\mbox{\boldmath$n$}\cdot\bm{\sigma}_{f}\cdot\mbox{\boldmath$n$},~{}\mbox{\boldmath$x$}\in\partial\Omega\\\
\mbox{\boldmath$n$}\cdot\bm{\sigma}_{e}\cdot\mbox{\boldmath$t$}=\mbox{\boldmath$n$}\cdot\bm{\sigma}_{f}\cdot\mbox{\boldmath$t$},~{}\mbox{\boldmath$x$}\in\partial\Omega\\\
\end{cases}$ (3.3) Far-field
$\displaystyle\begin{cases}\mbox{\boldmath$v$}(|\mbox{\boldmath$x$}|\to\infty)=\cos\omega
t~{}\hat{i},~{}\mbox{\boldmath$x$}\in\Omega_{f},\end{cases}$ (3.4)

where Eq. 3.1 is the rigid pin constraint, Eq. 3.2 is the no-slip condition,
Eq. 3.3 enforces continuity of stresses, and Eq. 3.4 is the far-field flow.
Next, we identify relevant parameter ranges and solve Eq. 2 accordingly, using
perturbation theory.

Typically, in viscous streaming applications, the scaled oscillation amplitude
is $\epsilon\ll{1}$ [1, 3, 5], and the Womersley number is
$M\geq\mathcal{O}\left(1\right)$ [6, 7]. Additionally, density $\alpha$ and
viscosity $\beta$ ratios are $\sim\mathcal{O}\left(1\right)$. The Cauchy
number $Cau$ requires careful consideration. For a rigid body $Cau=0$, while
for an elastic body $Cau>0$, with $Cau\ll 1$ implying weak elasticity. From a
mathematical perspective, dealing with $Cau\geq\mathcal{O}\left(1\right)$ is
challenging due to the highly non-linear nature of hyperelastic materials.
Here, we assume that the cylinder is only weakly elastic, and in particular
that $Cau=\kappa\epsilon$, where $\kappa=\mathcal{O}\left(1\right)$. This
assumption simplifies the asymptotic treatment, slaving $Cau$ to $\epsilon$ so
that both are equally small and tend to zero at the same rate.

We then look for asymptotic solutions of Eqs.(2) by perturbing all relevant
fields as series of powers of $\epsilon$. We derive the leading order solution
$\mathcal{O}\left(1\right)$, which reduces to a rigid cylinder in a purely
oscillatory flow governed by the unsteady Stokes equation [1]. The next order
solution $\mathcal{O}\left(\epsilon\right)$ is derived in two steps. First, we
obtain the deformation of the elastic solid due to the leading order flow.
Second, we use this deformation to determine the boundary conditions for the
flow at $\mathcal{O}\left(\epsilon\right)$, thus incorporating elasticity
effects into the streaming solution. Steps are mathematically outlined below,
with details in the SI.

We start by perturbing to $\mathcal{O}\left(\epsilon\right)$ all physical
quantities $q$, which include $v$, $u$, $p$, $\Omega$, $n$, $t$, as

$q\sim q_{0}+\epsilon q_{1}+\mathcal{O}\left(\epsilon^{2}\right)$ (3)

and substitute them in Eqs. 2. Subscripts (0, 1, …) indicate the solution
order. Then, we adopt the more convenient cylindrical coordinate system
$(r,\theta)$, with radial coordinate $r$, angular coordinate $\theta$, and
origin at the center of the cylinder. Horizontal axis direction $i$
corresponds to $\theta=0$. At leading order $\mathcal{O}\left(1\right)$, the
governing equations and boundary conditions in the solid reduce to

$\bm{\nabla}\cdot((\bm{I}+\bm{\nabla}\bm{u}_{0})(\bm{I}+\bm{\nabla}\bm{u}_{0})^{T})^{\prime}=0,~{}~{}r\leq
1;~{}~{}~{}\mbox{\boldmath$u$}_{0}|_{r=\zeta}=0$ (4)

where $\zeta=b/a$ is the non-dimensional radius of the pinned zone. Since at
this order $Cau=\kappa\epsilon=0$, the solution of Eqs. 4 is the fixed, rigid
body cylinder

$\partial\Omega_{0}=r=1;~{}~{}\mbox{\boldmath$u$}_{0}=0,~{}~{}\mbox{\boldmath$v$}_{0}=\frac{\partial\mbox{\boldmath$u$}_{0}}{dt}=0,~{}~{}r\leq
1.$ (5)

With these leading order boundary conditions the governing equations in the
fluid reduce to

$\displaystyle M^{2}\frac{\partial\nabla^{2}\psi_{0}}{\partial t}$
$\displaystyle=\nabla^{4}\psi_{0}~{}~{}~{}~{}r\geq 1$ (6) $\displaystyle
v_{0,r}|_{r=1}=\frac{1}{r}\frac{\partial\psi_{0}}{\partial\theta}\biggr{|}_{r=1}=0$
$\displaystyle;~{}v_{0,\theta}|_{r=1}=-\frac{\partial\psi_{0}}{\partial
r}\biggr{|}_{r=1}=0$ $\displaystyle v_{0,r}|_{r\to\infty}=\cos\theta~{}\cos t$
$\displaystyle;~{}v_{0,\theta}|_{r\to\infty}=-\sin\theta~{}\cos t,$

where $\psi$ is the streamfunction defined as
$\mbox{\boldmath$v$}=\bm{\nabla}\times\psi$. This system (Eq. 5, 6) is a rigid
cylinder immersed in an oscillating unsteady Stokes flow, which has the exact
analytical solution [1]

$\psi_{0}=\frac{\sin\theta}{2}\left(r+\frac{H_{2}(m)}{rH_{0}(m)}-\frac{2H_{1}(mr)}{mH_{0}(m)}\right)e^{-it}+c.c.,~{}r\geq
1$ (7)

where $i=\sqrt{-1}$, and $m=\sqrt{i}M$. Here, $H_{i}$ and $c.c.$ refer to the
$i^{\textrm{th}}$ order Hankel function of first kind and complex conjugate.
The leading order field $\psi_{0}$ in the fluid is purely oscillatory, thus no
steady streaming is observed at $\mathcal{O}\left(1\right)$, as expected [1,
3]. Additionally, no effects of elasticity manifest on the flow at this order.

We then proceed to the next order of approximation
$\mathcal{O}\left(\epsilon\right)$, where we instead do expect steady
streaming to emerge and elasticity to play a role. At
$\mathcal{O}\left(\epsilon\right)$, the solid governing equations reduce (SI,
Eqs. 42, 49) to

$\nabla^{4}\psi_{e,1}=0,~{}~{}~{}~{}\mbox{\boldmath$x$}\in\Omega_{e}$ (8)

where we have defined the strain function $\psi_{e}$, so that
$\mbox{\boldmath$u$}=\nabla\times\psi_{e}$ (similar to the streamfunction
$\psi$). Equation 8 shows how the specific choice of solid elasticity model is
irrelevant at $\mathcal{O}\left(\epsilon\right)$, since all non-linear stress-
strain responses drop out due to linearization. Equation 8 is further
complemented by the boundary conditions at the pinned zone interface

$u_{1,r}=\left.\frac{1}{r}\frac{\partial\psi_{e,1}}{\partial\theta}\right|_{r=\zeta}=0;~{}~{}~{}~{}u_{1,\theta}=\left.-\frac{\partial\psi_{e,1}}{\partial
r}\right|_{r=\zeta}=0.$ (9)

Now, the flow solution at $\mathcal{O}\left(1\right)$ exerts interfacial
stresses on the solid, which at $\mathcal{O}\left(\epsilon\right)$, is no
longer rigid but instead deforms. This process is driven by Eq. 3.3, which
yields the following radial and tangential stress conditions

$\displaystyle\frac{M^{2}}{\kappa}\left.\frac{\partial}{\partial
r}\left(\frac{1}{r}\frac{\partial\psi_{e,1}}{\partial\theta}\right)\right|_{r=1}=\left.\frac{\partial
v_{0,r}}{\partial r}\right|_{r=1}$ (10)
$\displaystyle\frac{M^{2}}{\kappa}\left.\left(\frac{1}{r^{2}}\frac{\partial^{2}\psi_{e,1}}{\partial\theta^{2}}-r\frac{\partial}{\partial
r}\left(\frac{1}{r}\frac{\partial\psi_{e,1}}{\partial
r}\right)\right)\right|_{r=1}=$
$\displaystyle\left.\left(\frac{1}{r}\frac{\partial
v_{0,r}}{\partial\theta}+\frac{\partial v_{0,\theta}}{\partial
r}-\frac{v_{0,\theta}}{r}\right)\right|_{r=1},$

where the LHS corresponds to the solid phase (Eq. 8) and the RHS to the fluid
phase (Eq. 7), both evaluated at the leading order interface $r=1$. We note
that although the solid interface does deform, the use of $r=1$ is not
inconsistent. Indeed, as shown in the SI (Eq. 44–46), this approximation
induces higher order $\mathcal{O}\left(\epsilon^{2}\right)$ errors in the
boundary stresses evaluation. The flow quantities on the RHS can be then
directly evaluated

$\displaystyle\left.\frac{\partial v_{0,r}}{\partial r}\right|_{r=1}$
$\displaystyle=0$ (11) $\displaystyle\left.\left(\frac{1}{r}\frac{\partial
v_{0,r}}{\partial\theta}+\frac{\partial v_{0,\theta}}{\partial
r}-\frac{v_{0,\theta}}{r}\right)\right|_{r=1}$
$\displaystyle=\sin{\theta}~{}F(m)~{}e^{-it}+c.c.$

with

$F(m)={-mH_{1}(m)}/{H_{0}(m)}.$ (12)

Once Eqs. 11 and 12 are substituted back in the boundary conditions of Eq. 10,
the biharmonic Eq. 8 can be solved to obtain the
$\mathcal{O}\left(\epsilon\right)$ solid displacement field

$\displaystyle\psi_{e,1}$
$\displaystyle=\frac{\kappa}{M^{2}}\sin{\theta}(r)\left(c_{1}+\frac{c_{2}}{r^{2}}+c_{3}r^{2}+c_{4}\ln(r)\right)F(m)~{}e^{-it}$
(13) $\displaystyle+c.c.,$

where the expressions for $c_{1},c_{2},c_{3},c_{4}$ are reported in the SI.
Equation 13 represents the $\mathcal{O}\left(\epsilon\right)$ solid
displacement field both in the bulk $\Omega_{e}$ and at the boundary
$\partial\Omega$, which directly affects the
$\mathcal{O}\left(\epsilon\right)$ flow. At
$\mathcal{O}\left(\epsilon\right)$, the flow governing equation, in
streamfunction form, reads [1]

$M^{2}\frac{\partial\nabla^{2}\psi_{1}}{\partial
t}+M^{2}\left(\left(\mbox{\boldmath$v$}_{0}\cdot\bm{\nabla}\right)\nabla^{2}\psi_{0}\right)=\nabla^{4}\psi_{1},~{}~{}~{}~{}r\geq
1.$ (14)

Since we are interested in steady streaming, we consider the time average

$\displaystyle\nabla^{4}\langle\psi_{1}\rangle$
$\displaystyle=M^{2}\underbrace{\langle\left(\mbox{\boldmath$v$}_{0}\cdot\bm{\nabla}\right)\nabla^{2}\langle\psi_{0}\rangle\rangle}_{\textrm{RHS}},~{}~{}~{}~{}r\geq
1,$ (15)

where the RHS can be rewritten using Eq. 7 to yield

$\displaystyle\nabla^{4}\langle\psi_{1}\rangle=\sin
2\theta~{}\rho(r),~{}~{}~{}~{}r\geq 1,$ (16)
$\displaystyle\rho(r)=-\frac{M^{4}}{2}~{}\textrm{Im}\left[\frac{H_{2}(mr)}{H_{0}(m)}+\frac{H_{2}(m)H_{0}^{*}(mr)}{H_{0}^{2}(m)r^{2}}\right.$
$\displaystyle\left.+2\frac{H_{0}(mr)H_{2}^{*}(mr)}{H_{0}^{2}(m)}\right],$

with $\textrm{Im}[\cdot]$ representing the imaginary part. To solve this
equation, we first recall the far-field boundary conditions

$\displaystyle\left.\frac{1}{r}\frac{\partial\langle\psi_{1}\rangle}{\partial\theta}\right|_{r\to\infty}=\left.\frac{\partial\langle\psi_{1}\rangle}{\partial
r}\right|_{r\to\infty}=0.$ (17)

Next, we recall the no-slip boundary condition of Eq. 3.2 that needs to be
enforced at the $\mathcal{O}\left(\epsilon\right)$ accurate solid–fluid
interface (SI, Eq. 62)

$\displaystyle\left.\mbox{\boldmath$v$}_{e}\right|_{\partial\Omega}$
$\displaystyle=\left.\mbox{\boldmath$v$}_{e}\right|_{r=1+\epsilon
u_{1,r}}+\mathcal{O}\left(\epsilon^{2}\right)$ (18)
$\displaystyle=\left.\mbox{\boldmath$v$}_{f}\right|_{\partial\Omega}=\left.\mbox{\boldmath$v$}_{f}\right|_{r=1+\epsilon
u_{1,r}}+\mathcal{O}\left(\epsilon^{2}\right),$

Figure 2:  Effect of elasticity on streaming flow. Time averaged streamline
patterns (blue/orange represent clockwise/anti-clockwise rotating regions)
depicting streaming response at $M\approx 8$ with increasing softness $Cau$:
(a) rigid limit ($Cau=0$), (b) $Cau=0.025$ and (c) $Cau=0.05$. Non-dimensional
radius of the pinned zone (green cylinder) is set at $\zeta=0.2$ throughout
the study. For effects of $\zeta$ variation on streaming topology, refer to
SI. (d) Normalized DC layer thickness $\delta_{DC}/a$ vs. inverse of Womersley
number ($1/M$) from theory and simulations, for varying body elasticity $Cau$.
Radial decay of velocity magnitude along $\theta=0^{\circ}$ from theory and
simulations, with increasing softness $Cau$: (e) rigid limit ($Cau=0$), (f)
$Cau=0.025$ and (g) $Cau=0.05$. For simulation details, refer to SI.

where we highlight how, at $\mathcal{O}\left(\epsilon\right)$, the cylinder
interface is no longer fixed at $r=1$, but deforms as $r^{\prime}=1+\epsilon
u_{1,r}$. Here, $u_{1,r}$ is the $\mathcal{O}\left(\epsilon\right)$ accurate
deformation field computed by injecting Eq. 13 into
$\mbox{\boldmath$u$}_{1}=\nabla\times\psi_{e,1}$. To enforce Eq. 18, while
maintaining an analytically tractable formulation, we replace the boundary
flow velocity $\left.\mbox{\boldmath$v$}_{f}\right|_{r=r^{\prime}}$ on the
temporally moving interface $r^{\prime}$ with the velocity that the flow would
need to see on the fixed interface $r=1$ to respond equivalently. We achieve
this by Taylor expanding $\left.\mbox{\boldmath$v$}_{f}\right|_{r=r^{\prime}}$
about $r=1$ (SI, Eq. 64–66)

$\displaystyle\left.\mbox{\boldmath$v$}_{f}\right|_{r=1+\epsilon u_{1,r}}$
$\displaystyle=\left.\left(\epsilon\mbox{\boldmath$v$}_{f,1}+\epsilon\frac{\partial\mbox{\boldmath$v$}_{f,0}}{\partial
r}u_{1,r}\right)\right|_{r=1}+\mathcal{O}\left(\epsilon^{2}\right).$ (19)

Similarly, $\left.\mbox{\boldmath$v$}_{e}\right|_{r=1+\epsilon u_{1,r}}$ (LHS
of Eq. 18) can be computed to $\mathcal{O}\left(\epsilon\right)$ accuracy as
$\left.\partial\mbox{\boldmath$u$}_{1,r}/\partial t\right|_{r=1}$ (SI, Eq.
63). Given that $u_{1,r}$ (Eq. 13) and $\partial v_{f,0}/\partial r$ (Eq. 7)
are known, we can plug Eq. 19 into Eq. 18 to obtain $v_{f,1}$ (SI, Eq. 62–67).
Time averaging yields

$\displaystyle\left.\langle
v_{1,r}\rangle\right|_{r=1}=\left.\frac{1}{r}\frac{\partial\langle\psi_{1}\rangle}{\partial\theta}\right|_{r=1}$
$\displaystyle=0~{}$ (20) $\displaystyle\left.-\langle
v_{1,\theta}\rangle\right|_{r=1}=\left.\frac{\partial\langle\psi_{1}\rangle}{\partial
r}\right|_{r=1}$ $\displaystyle=\frac{\kappa}{M^{2}}\sin
2\theta~{}G_{1}(\zeta)F(m)F^{*}(m)$

with

$G_{1}(\zeta)=0.5\left(\frac{(\zeta^{2}+1)ln(\zeta)}{\zeta^{2}-1}-1\right).$
(21)

Equation 20 tells us that, from the fluid perspective, the no-slip condition
on the moving interface $r^{\prime}$ can be equivalently seen as a rectified
tangential slip velocity ($\langle v_{1,\theta}\rangle|_{r=1}\neq 0$) on the
leading order, fixed interface $r=1$. In our case, this slip velocity stems
from solid elasticity and modifies the Reynolds stresses—$\sin
2\theta~{}\rho(r)$— associated with the rigid body (Eq. 16), thus altering the
overall streaming flow response. We remark that this slip is independent of
the Navier–Stokes non-linear inertial advection. Hence, streaming can be
generated even in the Stokes limit, unlike for rigid bodies.

Given the steady flow of Eq. (16) and boundary conditions of Eqs. 17 and 20,
the streaming solution can finally be written as

$\langle\psi_{1}\rangle=\sin 2\theta~{}\left[\Theta(r)+\Lambda(r)\right].$
(22)

Here, $\Theta(r)$ is the classical rigid body contribution from Holtsmark  _et
al._ [1]

$\displaystyle\Theta(r)=-\frac{r^{4}}{48}\int_{r}^{\infty}\frac{\rho(\tau)}{\tau}~{}\mathrm{d}\tau+\frac{r^{2}}{16}\int_{r}^{\infty}\tau\rho(\tau)~{}\mathrm{d}\tau$
(23)
$\displaystyle+\frac{1}{16}\left(\int_{1}^{r}\tau^{3}\rho(\tau)~{}\mathrm{d}\tau+\int_{1}^{\infty}\frac{\rho(\tau)}{\tau}~{}\mathrm{d}\tau-2\int_{1}^{\infty}\tau\rho(\tau)~{}\mathrm{d}\tau\right)$
$\displaystyle+\frac{1}{r^{2}}\left(-\frac{1}{48}\int_{1}^{r}\tau^{5}\rho(\tau)~{}\mathrm{d}\tau-\frac{1}{24}\int_{1}^{\infty}\frac{\rho(\tau)}{\tau}~{}\mathrm{d}\tau\right.$
$\displaystyle\left.+\frac{1}{16}\int_{1}^{\infty}\tau\rho(\tau)~{}\mathrm{d}\tau\right)$

and $\Lambda(r)$ is the new elastic modification

$\Lambda(r)=0.5\frac{\kappa}{M^{2}}~{}G_{1}(\zeta)F(m)F^{*}(m)\left(1-\frac{1}{r^{2}}\right)$
(24)

with $G_{1}(\zeta)$ and $F(m)$ given in Eq. 21 and Eq. 12. This concludes our
theoretical analysis.

Figure 3:  Extension of compliance-induced streaming to generalised, multiple
curvature bodies. Time averaged simulation flow topologies for a bullet—formed
by hybridizing a circle of radius $a$ with a square of side length $2a$—at
$M\approx 8$, with varying body elasticity $Cau$: (a) rigid limit $Cau=0$, (b)
$Cau=0.05$ and (c) $Cau=0.1$. Increasing body softness results in contraction
and strengthening of the rear DC vortex on the square side, consistent with
our theoretical insights.

Next, we compare our theory against direct numerical simulations [35] and
known analytical results in the rigidity limit [1, 3]. For a rigid cylinder
($Cau=0$) oscillating at $M\approx 8$, numerical time-averaged streamlines are
shown in Fig. 2(a). We highlight the four-fold symmetry and the presence of a
well-defined direct circulation (DC) layer of thickness $\delta_{DC}$.
Holtsmark  _et al._ [1] predicts this flow topology, as well as an increase of
$\delta_{DC}$ with $1/M$ until divergence, at which point the DC layer extends
to infinity. This behaviour is recovered by our theory when $Cau=0$ (i.e.
$\Lambda=0$), and by simulations (black line/dots in Fig. 2d). As the cylinder
becomes soft ($Cau>0$), four-fold symmetry is preserved ($\sin 2\theta$ in Eq.
22), but $\delta_{DC}$ contracts on account of the elastic term $\Lambda\neq
0$. This is confirmed by simulations across a range of $Cau$, as seen in Fig.
2(b-d). Thus, an elastic body can access streaming flow configurations
equivalent to rigid objects exposed to significantly higher oscillation
frequencies (e.g., $\sim 2\times$ at $Cau=0.05$, Fig. 2d). We conclude our
validation by reporting in Fig. 2(e–g) theoretical and simulated radially-
varying, time-averaged velocities $|\langle v\rangle|$ at $\theta=0^{\circ}$,
noting close agreement.

Finally, we demonstrate how gained theoretical intuition extends to geometries
of multiple curvatures. We consider the shape of Fig. 3, previously designed
[19] to attain streaming flows favorable to particle transport [18] and
separation [21]. Both applications rely on the presence of flanking and rear
vortices, and performance is improved by strengthening the vortices via
increasing oscillation frequencies [19]. Figure 3 shows how the same process
can alternatively be achieved by increasing softness only. As a result, the
same flow topologies of [19] are obtained in Fig. 3 for frequencies $\sim
4\times$ lower.

In summary, we derived a viscous streaming theory for the case of an elastic
cylinder, and validated it computationally. Our study reveals an additional,
tunable mode of streaming, accessible through material compliance and
available even in Stokes flow. We demonstrate its use for flow control in the
case of a previously designed streaming body of multiple curvatures, to
illustrate application potential in microfluidics or microrobotics, in
conjunction with the use of elastomeric or biological materials. Further, the
fact that compliance enables streaming effects at frequencies significantly
lower than rigid bodies, supports the hypothesis that biological creatures,
speculated to operate at the edge of viscous streaming viability, may instead
take full advantage of it thanks to their softness.

## References

  * Holtsmark  _et al._ [1954] J. Holtsmark, I. Johnsen, T. Sikkeland, and S. Skavlem, Boundary layer flow near a cylindrical obstacle in an oscillating, incompressible fluid, The journal of the acoustical society of America 26, 26 (1954).
  * Lane [1955] C. Lane, Acoustical streaming in the vicinity of a sphere, The Journal of the Acoustical Society of America 27, 1082 (1955).
  * Bertelsen  _et al._ [1973] A. Bertelsen, A. Svardal, and S. Tjøtta, Nonlinear streaming effects associated with oscillating cylinders, Journal of Fluid Mechanics 59, 493 (1973).
  * Lutz  _et al._ [2003] B. R. Lutz, J. Chen, and D. T. Schwartz, Microfluidics without microfabrication, Proceedings of the National Academy of Sciences 100, 4395 (2003).
  * Lutz  _et al._ [2005] B. R. Lutz, J. Chen, and D. T. Schwartz, Microscopic steady streaming eddies created around short cylinders in a channel: Flow visualization and stokes layer scaling, Physics of Fluids 17, 023601 (2005).
  * Marmottant and Hilgenfeldt [2004] P. Marmottant and S. Hilgenfeldt, A bubble-driven microfluidic transport element for bioengineering, Proceedings of the National Academy of Sciences 101, 9523 (2004).
  * Lutz  _et al._ [2006] B. R. Lutz, J. Chen, and D. T. Schwartz, Hydrodynamic tweezers: 1. noncontact trapping of single cells using steady streaming microeddies, Analytical chemistry 78, 5429 (2006).
  * Wang  _et al._ [2011] C. Wang, S. V. Jalikop, and S. Hilgenfeldt, Size-sensitive sorting of microparticles through control of flow geometry, Applied Physics Letters 99, 034101 (2011).
  * Chong  _et al._ [2013] K. Chong, S. D. Kelly, S. Smith, and J. D. Eldredge, Inertial particle trapping in viscous streaming, Physics of Fluids 25, 033602 (2013).
  * Chen and Lee [2014] Y. Chen and S. Lee, Manipulation of biological objects using acoustic bubbles: a review, Integrative and comparative biology 54, 959 (2014).
  * Klotsa  _et al._ [2015] D. Klotsa, K. A. Baldwin, R. J. Hill, R. M. Bowley, and M. R. Swift, Propulsion of a two-sphere swimmer, Physical review letters 115, 248102 (2015).
  * Thameem  _et al._ [2017] R. Thameem, B. Rallabandi, and S. Hilgenfeldt, Fast inertial particle manipulation in oscillating flows, Physical Review Fluids 2, 052001 (2017).
  * Pommella  _et al._ [2021] A. Pommella, I. Harun, K. Hellgardt, and V. Garbin, Enhancing microalgal cell wall permeability by microbubble streaming flow, arXiv preprint arXiv:2112.08519 (2021).
  * Liu  _et al._ [2002] R. H. Liu, J. Yang, M. Z. Pindera, M. Athavale, and P. Grodzinski, Bubble-induced acoustic micromixing, Lab on a Chip 2, 151 (2002).
  * Ahmed  _et al._ [2009] D. Ahmed, X. Mao, B. K. Juluri, and T. J. Huang, A fast microfluidic mixer based on acoustically driven sidewall-trapped microbubbles, Microfluidics and nanofluidics 7, 727 (2009).
  * Marmottant and Hilgenfeldt [2003] P. Marmottant and S. Hilgenfeldt, Controlled vesicle deformation and lysis by single oscillating bubbles, Nature 423, 153 (2003).
  * Thameem  _et al._ [2016] R. Thameem, B. Rallabandi, and S. Hilgenfeldt, Particle migration and sorting in microbubble streaming flows, Biomicrofluidics 10, 014124 (2016).
  * Parthasarathy  _et al._ [2019] T. Parthasarathy, F. K. Chan, and M. Gazzola, Streaming-enhanced flow-mediated transport, Journal of Fluid Mechanics 878, 647 (2019).
  * Bhosale  _et al._ [2020] Y. Bhosale, T. Parthasarathy, and M. Gazzola, Shape curvature effects in viscous streaming, Journal of Fluid Mechanics 898, A13 (2020).
  * Chan  _et al._ [2022] F. K. Chan, Y. Bhosale, T. Parthasarathy, and M. Gazzola, Three-dimensional geometry and topology effects in viscous streaming, Journal of Fluid Mechanics 933, A53 (2022).
  * Bhosale  _et al._ [2021a] Y. Bhosale, G. Vishwanathan, T. Parthasarathy, G. Juarez, and M. Gazzola, Multi-curvature viscous streaming: flow topology and particle manipulation, arXiv preprint arXiv:2111.07184 (2021a).
  * Jalal  _et al._ [2018] S. Jalal, T. Van de Moortele, A. Nemes, O. Amili, and F. Coletti, Three-dimensional steady and oscillatory flow in a double bifurcation airway model, Physical Review Fluids 3, 103101 (2018).
  * Jacob  _et al._ [2021] C. Jacob, D. G. Tingay, and J. S. Leontini, The impact of steady streaming and conditional turbulence on gas transport during high-frequency ventilation, Theoretical and Computational Fluid Dynamics 35, 265 (2021).
  * Parthasarathy  _et al._ [2020] T. Parthasarathy, Y. Bhosale, and M. Gazzola, A simple, rigorous benchmark for fully coupled flow–structure interaction algorithms, arXiv preprint arXiv:2011.09453 (2020).
  * Someya  _et al._ [2016] T. Someya, Z. Bao, and G. G. Malliaras, The rise of plastic bioelectronics, Nature 540, 379 (2016).
  * Heikenfeld  _et al._ [2018] J. Heikenfeld, A. Jajack, J. Rogers, P. Gutruf, L. Tian, T. Pan, R. Li, M. Khine, J. Kim, and J. Wang, Wearable sensors: modalities, challenges, and prospects, Lab on a Chip 18, 217 (2018).
  * Bandodkar  _et al._ [2019] A. J. Bandodkar, P. Gutruf, J. Choi, K. Lee, Y. Sekine, J. T. Reeder, W. J. Jeang, A. J. Aranyosi, S. P. Lee, J. B. Model, _et al._ , Battery-free, skin-interfaced microfluidic/electronic systems for simultaneous electrochemical, colorimetric, and volumetric analysis of sweat, Science advances 5, eaav3294 (2019).
  * Park  _et al._ [2016] S.-J. Park, M. Gazzola, K. S. Park, S. Park, V. Di Santo, E. L. Blevins, J. U. Lind, P. H. Campbell, S. Dauth, A. K. Capulli, _et al._ , Phototactic guidance of a tissue-engineered soft-robotic ray, Science 353, 158 (2016).
  * Ceylan  _et al._ [2017] H. Ceylan, J. Giltinan, K. Kozielski, and M. Sitti, Mobile microrobots for bioengineering applications, Lab on a Chip 17, 1705 (2017).
  * Huang  _et al._ [2019] H.-W. Huang, F. E. Uslu, P. Katsamba, E. Lauga, M. S. Sakar, and B. J. Nelson, Adaptive locomotion of artificial microswimmers, Science advances 5, eaau1532 (2019).
  * Aydin  _et al._ [2019] O. Aydin, X. Zhang, S. Nuethong, G. J. Pagan-Diaz, R. Bashir, M. Gazzola, and M. T. A. Saif, Neuromuscular actuation of biohybrid motile bots, Proceedings of the National Academy of Sciences 116, 19841 (2019).
  * Spelman and Lauga [2017] T. A. Spelman and E. Lauga, Arbitrary axisymmetric steady streaming: Flow, force and propulsion, Journal of Engineering Mathematics 105, 31 (2017).
  * Gilpin  _et al._ [2020] W. Gilpin, M. S. Bull, and M. Prakash, The multiscale physics of cilia and flagella, Nature Reviews Physics 2, 74 (2020).
  * Bower [2009] A. F. Bower,  _Applied mechanics of solids_ (CRC press, 2009).
  * Bhosale  _et al._ [2021b] Y. Bhosale, T. Parthasarathy, and M. Gazzola, A remeshed vortex method for mixed rigid/soft body fluid–structure interaction, Journal of Computational Physics 444, 110577 (2021b).


# Unveiling Camouflage: A Learnable Fourier-based Augmentation for
Camouflaged Object Detection and Instance Segmentation

Minh-Quan Le1, 2, 3, Minh-Triet Tran1, 2, 4, Trung-Nghia Le1, 2, Tam V.
Nguyen5, Thanh-Toan Do6

###### Abstract

Camouflaged object detection (COD) and camouflaged instance segmentation (CIS)
aim to recognize and segment objects that are blended into their surroundings,
respectively. While several deep neural network models have been proposed to
tackle those tasks, augmentation methods for COD and CIS have not been
thoroughly explored. Augmentation strategies can help improve the performance
of models by increasing the size and diversity of the training data and
exposing the model to a wider range of variations in the data. Besides, we aim
to automatically learn transformations that help to reveal the underlying
structure of camouflaged objects and allow the model to learn to better
identify and segment camouflaged objects. To achieve this, we propose a
learnable augmentation method in the frequency domain for COD and CIS via
Fourier transform approach, dubbed CamoFourier. Our method leverages a
conditional generative adversarial network and cross-attention mechanism to
generate a reference image and an adaptive hybrid swapping with parameters to
mix the low-frequency component of the reference image and the high-frequency
component of the input image. This approach aims to make camouflaged objects
more visible for detection and segmentation models. Without bells and
whistles, our proposed augmentation method boosts the performance of
camouflaged object detectors and camouflaged instance segmenters by large
margins.

## Introduction

Camouflage refers to the use of any combination of materials, coloration, or
illumination for concealment, either by making animals or objects hard to see
or by disguising them as something else. In the context of computer vision,
camouflaged object detection (COD) and camouflaged instance segmentation (CIS)
are tasks that aim to recognize and segment objects that are integrated into
their surroundings. COD is the task of identifying objects that are
“seamlessly” embedded in their surroundings (Le et al. 2019). The high
intrinsic similarities between the target object and the background make COD
far more challenging than traditional object detection tasks(Fan et al.
2020a). CIS is a related task that involves not only detecting camouflaged
objects but also segmenting them at the pixel level (Le et al. 2022). Both COD
and CIS require sophisticated computer vision algorithms that can distinguish
between the target object and its background despite their high visual
similarity. These tasks have many potential applications, including wildlife
monitoring, search and rescue operations (Le et al. 2019), medical diagnosis
(polyp segmentation (Fan et al. 2020b); COVID-19 infection identification from
lung x-rays (Das et al. 2022)).

Figure 1:  Our CamoFourier not only preserves the spatial structure and
resolution of an image but also highlights the underlying structure of
camouflaged objects for better identification and segmentation.

The performance of COD has recently been elevated by convolutional neural
networks (CNNs)-based approaches. Current cutting-edge methods concentrate on
designing detector architectures based on attention (Pang et al. 2022; Fan et
al. 2020a; Jia et al. 2022), and specific features of camouflaged objects (Fan
et al. 2022; Zhai et al. 2021). Moreover, OSFormer (Pei et al. 2022) leverages
ViT (Dosovitskiy et al. 2021) to propose the first architecture for the CIS
task. While several architectures have been designed to address COD and CIS
tasks, augmentation strategies have not been completely investigated even
though they can help improve the model’s ability to generalize to new data and
increase its robustness to variations in the input. Therefore, this work
explores the data-centric approach to COD and CIS problems by proposing a
learnable augmentation method in the frequency domain, named CamoFourier,
which allows deep learning models to learn how to augment their own training
data. In this way, our learnable augmentation aims to reveal the underlying
structure of camouflaged objects and allow the model to learn to better
identify and segment camouflaged objects.

Although several augmentation methods have been proposed for generic object
detection (Zhang et al. 2018; Yun et al. 2019; Zoph et al. 2020) and instance
segmentation (Ghiasi et al. 2021)) tasks, these works could not be directly
applied to COD and CIS tasks as these methods introduce occlusions and
deformations to input images, which may make camouflaged objects even harder
to detect and segment (Zhang et al. 2018). Moreover, these methods do not
consider the intrinsic similarities between the target object and the
background (Ghiasi et al. 2021), the main challenges for COD and CIS.
Therefore, we introduce a learnable augmentation strategy in the frequency
domain via Fourier transform for COD and CIS. The Fourier transform of an
image can encode semantic information and appearance information of the image
in its phase and amplitude components, respectively. Changing the amplitude
information while keeping the phase information of the Fourier transform of an
image can change the appearance of the image, but it is still recognizable.
Additionally, Fourier transform can also preserve the spatial structure and
resolution of an image, see Fig. 1.

Motivated by the above properties of Fourier transform, we propose a learnable
augmentation approach that aims to manipulate the amplitude information of the
Fourier transform of an input image to enhance the visibility of camouflaged
objects in the image. In particular, our method leverages a conditional
generative adversarial network to synthesize a generated image
$\mathcal{I}_{G}$ and cross-attention mechanism to learn the spatial
correspondence and alignment between the input image $\mathcal{I}$ and the
generated one $\mathcal{I}_{G}$, which outputs a reference image
$\mathcal{I}_{R}$. Furthermore, we propose a hybrid swapping with an adaptive
parameter to mix the low-frequency component of the reference image and the
high-frequency component of the input image to control the amount of texture
and color information that is transferred from the reference image
$\mathcal{I}_{R}$ to the input image $\mathcal{I}$. To the best of our
knowledge, our work is the first one that designs a trainable Fourier-based
augmentation specifically for both COD and CIS tasks. In summary, our main
contributions are three-fold.

  * •

We propose a novel learnable Fourier-based augmentation for camouflaged object
detection and instance segmentation, highlighting the camouflaged object of
interest from the background and making it more visible for deep models. The
augmentation framework is flexible and can be adapted to different
segmentation or detection algorithms.

  * •

We introduce an adaptive hybrid swapping approach that can control the amount
of texture and color information that is transferred from the reference image
to the input image. We leverage a simple yet effective cross-attention
mechanism between the input and generated image so that the reference image is
able to transfer its texture and color information to the input image in a
more attentive manner.

  * •

Extensive experimental results on a variety of camouflaged object detection
and instance segmentation methods on different datasets show that the proposed
method significantly improves the performance of existing models.

## Related Work

Figure 2: Overview of the proposed CamoFourier. Our method leverages a
conditional generative adversarial network and cross-attention mechanism to
generate a reference image and an adaptive hybrid swapping with parameters to
mix the low-frequency component of the reference image and the high-frequency
component of the input image.

#### Camouflaged object detection.

Fan et al. (2020a) present Search and Identification Net (SINet), a strong
baseline for the COD task. There is also an enhanced version of SINet called
SINetV2 (Fan et al. 2022) with two well-elaborated sub-modules: neighbor
connection decoder (NCD) and group-reversal attention (GRA). To well involve
the frequency clues into the CNN models, Zhong et al. (2022) introduce the
frequency domain as an additional clue to better detect camouflaged objects
from backgrounds. Several sophisticated architectures have been proposed for
COD, our CamoFourier is a learnable augmentation strategy for COD that can be
plugged into these existing works and improve their performance in an
efficient and easy implementation.

#### Camouflaged instance segmentation.

Le et al. (2022) investigate the interesting yet challenging problem of
camouflaged instance segmentation. To promote the new task of camouflaged
instance segmentation of in-the-wild images, Le et al. (2022) introduce a
dataset, dubbed CAMO++, that extends the preliminary CAMO dataset (camouflaged
object segmentation) in terms of quantity and diversity. To detect and segment
the whole scope of a camouflaged object, camouflaged object detection is
introduced as a binary segmentation task, with the binary ground truth
camouflage map indicating the exact regions of the camouflaged objects. Pei et
al. present OSFormer (Pei et al. 2022), the first one-stage transformer
framework for camouflaged instance segmentation. As the CIS task has not been
deeply explored, our CamoFourier is an efficient plug-and-play tool, which
helps improve the performance of CIS model and is valuable for researchers
working on CIS.

#### Data augmentation.

Data augmentation is a critical component of training deep learning models for
object detection and segmentation tasks (Zhang et al. 2018; Yun et al. 2019;
Zoph et al. 2020; Ghiasi et al. 2021; Luo et al. 2023). Data augmentation has
been shown to significantly improve generic object detection and instance
segmentation, but its potential has not been thoroughly investigated for
camouflaged object detection and camouflaged instance segmentation. The
existing works are augmentation methods in the spatial domain. When images are
augmented in the spatial domain, it is possible to introduce occlusions,
deformations, or noise to the input images, which may make the camouflaged
objects even harder to detect and segment. Moreover, these methods do not
consider the intrinsic similarities between the target object and the
background, which are the main challenges for camouflaged object detection and
camouflaged instance segmentation. Our CamoFourier is an augmentation method
in the frequency domain that is specifically designed for COD and CIS and can
help to avoid these artifacts by preserving the spatial information in the
image.

It is worth noting that Fourier transform has been leveraged for domain
generalization (Xu et al. 2021), and domain adaptation (Yang and Soatto 2020).
Different from those works, in this paper, we explore Fourier transform to
propose a learnable augmentation method for COD and CIS tasks.

## Proposed Method

### Overview Framework

We propose a general framework (Fig. 2) that can be used to provide an
effective and highly accurate solution for any segmentation or detection
algorithm for camouflage objects. Specifically, we propose a learnable data
augmentation module that can be integrated to be end-to-end training with any
existing object detection and segmentation methods. In this framework, the
original input image $\mathcal{I}$ is transformed to synthesize a new input
image $\mathcal{I*}$ that can highlight the camouflaged object of interest
from the background and is more suitable for the main processing algorithm
$\mathcal{F}$, including segmentation and detection. It is worth noting that
the proposed module is trained together with the main algorithm $\mathcal{F}$
in an end-to-end fashion.

As shown in Fig. 2, we aim to synthesize the transformed image $\mathcal{I}*$
from an original input image $\mathcal{I}$ by replacing the amplitude in the
FFT of $\mathcal{I}$ with the amplitude of the FFT of a context-aware
reference image $\mathcal{I}_{R}$. The input image $\mathcal{I}$ is used as
the conditional information for a generative model to generate an RGB image
$\mathcal{I}_{G}$. Then, both the generated image $\mathcal{I}_{G}$ and
original input image $\mathcal{I}$ are fed into a Cross Attention module to
produce a context-aware reference image $\mathcal{I}_{R}$ which captures
information of both $\mathcal{I}$ and $\mathcal{I}_{G}$.

Our intention is to synthesize $\mathcal{I}_{R}$ so that its amplitude
component is more appropriate than that of the original image $\mathcal{I}$ to
differentiate the camouflaged object and its surrounding area. Then, we blend
the original image $\mathcal{I}$ with the context-aware reference image
$\mathcal{I}_{R}$ by swapping the amplitude component from the FFT of
$\mathcal{I}_{R}$ with the one of the original image $\mathcal{I}$. We keep
the phase component from the FFT of $I$ because we want to preserve the
spatial structure and the semantic information of the original input image. We
are inspired by the idea of swapping the amplitude component for style
transfer (Yang and Soatto 2020) and we adopt this idea into our proposed
framework. However, in our solution, instead of using a target style for
reference as in (Yang and Soatto 2020), we train our framework with data to
generate a reference image $\mathcal{I}_{R}$ that provides a better FFT
amplitude component for camouflaged object detection or segmentation.

We do not simply swap entirely the amplitude from a reference image
$\mathcal{I}_{R}$ to the original image $\mathcal{I}$ but we propose an
adaptive hybrid swapping approach. We reuse the amplitude at high frequencies
in the FFT of the original image $\mathcal{I}$ because the high-frequency
components are responsible for the texture and detail of an image and adopt
the amplitude at low frequencies in the FFT of the reference image
$\mathcal{I}_{R}$ for transferring the texture and color information from the
reference image.

Furthermore, we do not set a fixed threshold frequency for the hybrid swapping
step but we decide to train an adaptive threshold for hybrid swapping. In this
way, the amplitude swapping step can exploit adaptively to the original input
image $\mathcal{I}$.

### Learnable Fourier-based Augmentation

#### GAN module.

Let $G$ be the generator network and $D$ be the discriminator network. The
generator $G$ takes an input image $\mathcal{I}$ and a random noise vector
$\mathbf{z}$ as inputs and outputs
$\hat{\mathcal{I}_{G}}=G(\mathcal{I},\mathbf{z})$, which is an image that is
supposed to look realistic given $\mathcal{I}$. The discriminator $D$ takes
the concatenation of $(\mathcal{I},\mathcal{I}_{G})$ as input and outputs
$D(\mathcal{I},\mathcal{I}_{G})$, which is a scalar value indicating how
likely $\mathcal{I}_{G}$ is to be a realistic image given $\mathcal{I}$. The
objective function of the conditional generative adversarial network (cGAN) is
given by:

$\begin{split}&\mathcal{L}_{cGAN}(G,D)=\mathbb{E}_{\mathcal{I}\sim
p_{\text{data }}(\mathcal{I})}[\log D(\mathcal{I},\mathcal{I})]\\\
&+\mathbb{E}_{\mathcal{I}\sim p_{\text{data }}(\mathcal{I}),\mathbf{z}\sim
p_{\mathbf{z}}(\mathbf{z})}\left[\log(1-D(\mathcal{I},G(\mathcal{I},\mathbf{z})))\right].\end{split}$
(1)

Similar to the cycle-consistency loss used in some image-to-image translation
models (Isola et al. 2017), we use a conditional generative adversarial
network with an additional $L1$ loss term to measure the similarity between
the generated image $\mathcal{I}_{G}$ and the input image $\mathcal{I}$. $L1$
loss can be written as:

$\mathcal{L}_{L1}(G)=\mathbb{E}_{\mathcal{I}\sim p_{\text{data
}}(\mathcal{I}),\mathbf{z}\sim
p_{\mathbf{z}}(\mathbf{z})}\left[\|\mathcal{I}-G(\mathcal{I},\mathbf{z})\|_{1}\right],$
(2)

where $\|\cdot\|_{1}$ denotes the $L1$ norm. The total objective function of
your model can be written as:

$\min_{G}\max_{D}\mathcal{L}_{cGAN}(G,D)+\lambda\mathcal{L}_{L1}(G),$ (3)

where $\lambda$ is a hyperparameter that controls the weight of the $L1$ loss.
By adding this term, we encourage the generator to produce images that are not
only realistic but also close to the input in the spatial domain (RGB domain).

#### Basic swapping of amplitude.

After transforming both input image $\mathcal{I}$ and reference image
$\mathcal{I}_{R}$ (produced the Cross Attention module) to the frequency
domain via FFT, we obtain the phase and amplitude of each image. Then we swap
the amplitude of both images. The phase of the input image together with the
amplitude of the reference image is transformed back to the spatial domain via
inverse FFT (iFFT) to get $\mathcal{I}*$. By swapping the amplitude of the
input and reference images, we transfer the texture and color information from
the reference image to the input image, while preserving the shape and
structure information from the input image.

Let $\mathcal{F}(\mathcal{I})$ and $\mathcal{F}(\mathcal{I}_{R})$ be the
Fourier transforms of $\mathcal{I}$ and $\mathcal{I}_{R}$, respectively. We
define $\mathcal{A}(\mathcal{F}(\mathcal{I}))$ and
$\phi(\mathcal{F}(\mathcal{I}))$ to be the amplitude and phase of
$\mathcal{F}(\mathcal{I})$. Similarly, the amplitude and phase of
$\mathcal{F}(\mathcal{I}_{R})$ are denoted by
$\mathcal{A}(\mathcal{F}(\mathcal{I}_{R}))$ and
$\phi(\mathcal{F}(\mathcal{I}_{R}))$. The inverse Fourier transform of the
amplitude-swapped image is given by:

$\mathcal{I}*=\mathcal{F}^{-1}(\mathcal{A}(\mathcal{F}(\mathcal{I}_{R})),\phi(\mathcal{F}(\mathcal{I}))).$
(4)

The image $\mathcal{I}*$ is the output of the amplitude swapping operation. It
is the input image $\mathcal{I}$ with the texture and color information from
the reference image $\mathcal{I}_{R}$.

### Cross-Attention Module

We hypothesize that the reference image $\mathcal{I}_{R}$ must capture
information from the input image $\mathcal{I}$ so that the amplitude of the
reference image can pay attention to specific regions in the input data.
Cross-attention (Chen, Hsieh, and Liu 2021; Rombach et al. 2022) between an
input image and a generated image can help to model the spatial correspondence
of different source images. This can help to extract appropriate features and
achieve adaptive and balanced fusion. It can dynamically learn the spatial
correspondence to derive better alignment of essential details from the two
images $\mathcal{I}$ and $\mathcal{I}_{G}$.

The input to the Cross Attention module is an input image $\mathcal{I}$ and a
generated image $\mathcal{I}_{G}$, both of size $H\times W\times C$, where
$H\times W$ is the spatial dimension and $C$ is the number of channels (e.g.,
3 for RGB images). Inspired by Vision Transformer (Dosovitskiy et al. 2021),
first, we split both images into patches of size $P\times P\times C$,
resulting in $(H/P)\times(W/P)$ patches for each image. Each patch is then
flattened into a vector of size $P^{2}C$ and linearly projected into a token
of size $D$ using a learnable projection matrix $W_{e}$ of size $P^{2}C\times
D$. This can be expressed mathematically as:

$\mathcal{I}^{t}=I^{p}*W_{e}\textrm{ and
}\mathcal{I}_{G}^{t}=\mathcal{I}_{G}^{p}*W_{e},$ (5)

where $\mathcal{I}^{t}$ and $\mathcal{I}_{G}^{t}$ are the token
representations of the input and generated images; $I^{p}$ and
$\mathcal{I}_{G}^{p}$ are the patch representations of the input and generated
images, respectively.

Next, we apply cross-attention between the two sets of tokens to allow the
reference image to pay attention to specific regions of the input image. This
can be expressed mathematically using the scaled dot-product attention
mechanism:

$\begin{split}A&=\mathrm{softmax}((\mathcal{I}^{t}*W_{q})*(\mathcal{I}_{G}^{t}*W_{k})^{T}/\sqrt{D})\\\
O&=A*(\mathcal{I}_{G}^{t}*W_{v}),\end{split}$ (6)

where $W_{q}$, $W_{k}$, and $W_{v}$ are learnable projection matrices of size
$D\times D$ for the query, key, and value representations, respectively, and
$A$ is the attention matrix that represents the attention paid by the
generated image to each patch of the input image. The output $O$ is then
linearly projected via a learnable projection matrix $W_{d}$ of size $D\times
P^{2}C$ and reshaped into an image of size $H\times W\times C$ in the RGB
domain. To ensure that the output $O$ remains in the RGB domain with pixel
values in the range $[0,255]$, we apply a scaling and clipping operation to
the output before reshaping it into an image. This can be done by first
normalizing the output to have zero mean and unit variance, then scaling it by
a factor of $255$, and finally clipping the values to the range $[0,255]$. The
final result of the cross-attention module is a reference image
$\mathcal{I}_{R}$.

Table 1: The effectiveness of our CamoFourier in improving camouflaged object
detection models on three benchmarks: CAMO (Le et al. 2019), COD10K (Fan et
al. 2020a), and NC4K (Lv et al. 2021). Our CamoFourier drastically increases
the performance of cutting-edge methods of COD, improved results are
highlighted in blue color.

Method | COD10K-Test | NC4K-Test | CAMO-Test
---|---|---|---
S${}_{m}\uparrow$ |  $\alpha$E $\uparrow$ |  $w$F $\uparrow$ | M $\downarrow$ | S${}_{m}\uparrow$ |  $\alpha$E $\uparrow$ |  $w$F $\uparrow$ | M $\downarrow$ | S${}_{m}\uparrow$ |  $\alpha$E $\uparrow$ |  $w$F $\uparrow$ | M $\downarrow$
SINet (Fan et al. 2020a) | 0.808 | 0.883 | 0.723 | 0.058 | 0.776 | 0.867 | 0.631 | 0.043 | 0.745 | 0.825 | 0.644 | 0.092
SINet (Fan et al. 2020a) \+ CamoFourier | 0.820 | 0.894 | 0.746 | 0.055 | 0.782 | 0.881 | 0.652 | 0.040 | 0.773 | 0.861 | 0.665 | 0.087
SINetV2 (Fan et al. 2022) | 0.847 | 0.898 | 0.770 | 0.048 | 0.815 | 0.863 | 0.680 | 0.037 | 0.820 | 0.875 | 0.743 | 0.070
SINetV2 (Fan et al. 2022) \+ CamoFourier | 0.859 | 0.914 | 0.778 | 0.043 | 0.826 | 0.890 | 0.704 | 0.032 | 0.858 | 0.893 | 0.752 | 0.066
SegMaR (Jia et al. 2022) | 0.841 | 0.905 | 0.781 | 0.046 | 0.833 | 0.895 | 0.724 | 0.033 | 0.815 | 0.872 | 0.742 | 0.071
SegMaR (Jia et al. 2022) \+ CamoFourier | 0.859 | 0.927 | 0.803 | 0.045 | 0.862 | 0.916 | 0.744 | 0.029 | 0.828 | 0.894 | 0.771 | 0.068
ZoomNet (Pang et al. 2022) | 0.853 | 0.907 | 0.784 | 0.043 | 0.838 | 0.893 | 0.729 | 0.029 | 0.820 | 0.883 | 0.752 | 0.066
ZoomNet (Pang et al. 2022) \+ CamoFourier | 0.872 | 0.923 | 0.801 | 0.037 | 0.864 | 0.911 | 0.740 | 0.025 | 0.852 | 0.906 | 0.774 | 0.060
FDNet (Zhong et al. 2022) | 0.834 | 0.895 | 0.750 | 0.052 | 0.837 | 0.897 | 0.731 | 0.030 | 0.844 | 0.903 | 0.778 | 0.062
FDNet (Zhong et al. 2022) \+ CamoFourier | 0.857 | 0.926 | 0.794 | 0.048 | 0.840 | 0.918 | 0.752 | 0.029 | 0.863 | 0.927 | 0.790 | 0.055

Table 2: Effectiveness of our CamoFourier in improving CIS model OSFormer (Pei
et al. 2022) and generic instance segmenters on two benchmarks: COD10K (Fan et
al. 2020a) and NC4K (Lv et al. 2021). Our CamoFourier significantly boosts the
performance of state-of-the-art methods and improved results are highlighted
in blue color.

Method | COD10K-Test | NC4K-Test
---|---|---
AP | AP50 | AP75 | AP | AP50 | AP75
Mask R-CNN (He et al. 2017) | 25.00 | 55.50 | 20.40 | 27.70 | 58.60 | 22.70
Mask R-CNN (He et al. 2017) \+ CamoFourier | 28.74 | 59.13 | 22.86 | 30.46 | 62.72 | 25.09
Cascade R-CNN (Cai and Vasconcelos 2018) | 25.30 | 56.10 | 21.30 | 29.50 | 60.80 | 24.80
Cascade R-CNN (Cai and Vasconcelos 2018) \+ CamoFourier | 28.65 | 58.90 | 22.93 | 31.25 | 63.14 | 26.38
YOLACT (Bolya et al. 2019) | 24.30 | 53.30 | 19.70 | 32.10 | 65.30 | 27.90
YOLACT (Bolya et al. 2019) \+ CamoFourier | 27.65 | 57.40 | 22.17 | 34.97 | 69.22 | 30.84
SOTR (Guo et al. 2021) | 27.90 | 58.70 | 24.10 | 29.30 | 61.00 | 25.60
SOTR (Guo et al. 2021) \+ CamoFourier | 30.23 | 61.57 | 25.93 | 32.64 | 65.11 | 28.87
OSFormer (Pei et al. 2022) | 41.00 | 71.10 | 40.80 | 42.50 | 72.50 | 42.30
OSFormer (Pei et al. 2022) \+ CamoFourier | 43.52 | 74.84 | 42.65 | 44.95 | 75.67 | 44.28

Table 3: Comparisons between our CamoFourier and existing generic augmentation
methods in camouflaged object detection task. Cutting-edge augmentations show
ineffectiveness when applying to COD while our CamoFourier boosts the
performance of SINetV2 (Fan et al. 2022) significantly.

Method | Augmentation | COD10K-Test | NC4K-Test | CAMO-Test
---|---|---|---|---
S${}_{m}\uparrow$ |  $\alpha$E $\uparrow$ |  $w$F $\uparrow$ | M $\downarrow$ | S${}_{m}\uparrow$ |  $\alpha$E $\uparrow$ |  $w$F $\uparrow$ | M $\downarrow$ | S${}_{m}\uparrow$ |  $\alpha$E $\uparrow$ |  $w$F $\uparrow$ | M $\downarrow$
SINetV2 (Fan et al. 2022) | - | 0.847 | 0.898 | 0.770 | 0.048 | 0.815 | 0.863 | 0.680 | 0.037 | 0.820 | 0.875 | 0.743 | 0.070
Mixup (Zhang et al. 2018) | 0.813 | 0.856 | 0.745 | 0.067 | 0.764 | 0.829 | 0.636 | 0.042 | 0.784 | 0.817 | 0.694 | 0.088
Cutmix (Yun et al. 2019) | 0.806 | 0.838 | 0.723 | 0.071 | 0.752 | 0.816 | 0.630 | 0.056 | 0.771 | 0.805 | 0.683 | 0.079
AutoAugment (Zoph et al. 2020) | 0.839 | 0.903 | 0.766 | 0.051 | 0.812 | 0.858 | 0.675 | 0.040 | 0.823 | 0.872 | 0.746 | 0.074
Copy-Paste (Ghiasi et al. 2021) | 0.820 | 0.861 | 0.754 | 0.055 | 0.806 | 0.849 | 0.673 | 0.042 | 0.811 | 0.864 | 0.727 | 0.080
CamDiff (Luo et al. 2023) | 0.851 | 0.895 | 0.772 | 0.047 | 0.819 | 0.866 | 0.678 | 0.037 | 0.831 | 0.882 | 0.745 | 0.073
| CamoFourier | 0.859 | 0.914 | 0.778 | 0.043 | 0.826 | 0.890 | 0.704 | 0.032 | 0.858 | 0.893 | 0.752 | 0.066

### Adaptive Hybrid Swapping of Amplitude

Inspired by the hybrid image (Oliva, Torralba, and Schyns 2006) which mixes
the low-spatial frequencies of one image with the high spatial frequencies of
another image, we propose an adaptive hybrid swapping the amplitude of the
input image $\mathcal{I}$ and the reference image $\mathcal{I}_{R}$. We
transform the input image $\mathcal{I}$ and the reference image
$\mathcal{I}_{R}$ via FFT. The low-frequency part of the amplitude of the
input image is adaptively replaced by that of the reference image. We denote
$\mathbb{M}_{\beta}$ as a binary mask matrix, whose value is one in the center
region and zero in the rest where $\beta\in(0,1)$:

$\mathbb{M}_{\beta}(h,w)=\mathds{1}_{(h,w)\in[-\beta H:\beta H,-\beta W:\beta
W]}.$ (7)

The inverse Fourier transform of the adaptive hybrid swapping of amplitude is
given by:

$\mathcal{I}*=\mathcal{F}^{-1}\Big{(}\Big{[}\mathbb{M}_{\beta}\odot\mathcal{A}(\mathcal{F}(\mathcal{I}_{R}))+(1-\mathbb{M}_{\beta})\odot\mathcal{A}(\mathcal{F}(\mathcal{I}))\Big{]},\phi(\mathcal{F}(\mathcal{I}))\Big{)}.$
(8)

The parameter $\beta$ is optimized via Bayesian optimization. As a
consequence, the augmented image $\mathcal{I}*$ is the output of the adaptive
hybrid swapping operation. The benefit of adaptive hybrid swapping is that it
is able to control the amount of texture and color information that is
transferred from the reference image $\mathcal{I}_{R}$ to the input image
$\mathcal{I}$. The augmented image $\mathcal{I}*$ is following fed as input to
camouflaged object detection and instance segmentation models for training
with specific objective functions of COD and CIS tasks.

## Experiments

### Experimental Settings

#### Datasets.

For COD, we use 3 popular benchmarks: CAMO (Le et al. 2019), COD10K (Fan et
al. 2020a), and NC4K (Lv et al. 2021). Among them, CAMO consists of 2,500
images, half with camouflaged objects and half without. COD10K (Fan et al.
2020a) contains 5,066 camouflaged, 3,000 background, and 1,934 non-camouflaged
images. NC4K is a large-scale COD dataset with 4,121 images. We follow prior
studies (Fan et al. 2020a; Lv et al. 2021) and use 1,000 images from CAMO (Le
et al. 2019), 3,040 images from COD10K(Fan et al. 2020a), and all images from
NC4K (Lv et al. 2021) for testing.

For CIS, there are few task-specific datasets since this is a new and
challenging task. Fan et al. (2020a) provided a COD dataset called COD10K that
also has instance-level annotations for CIS models. COD10K has 2,026 images
for testing and 3,040 camouflaged images with instance-level labels for
training. Recently, Le et al. (2022) released the CAMO++ dataset, a larger CIS
dataset with 5,500 samples with hierarchical pixel-wise annotation. Lyu et al.
(2021) introduced the NC4K test set for CIS, which has 4,121 images.

Table 4: Comparisons between our CamoFourier and existing generic augmentation
methods in CIS task. State-of-the-art generic augmentations downgrade the
effectiveness of OSFormer (Pei et al. 2022) while our CamoFourier improves the
performance of CIS model drastically.

Method | Augmentation | COD10K-Test | NC4K-Test
---|---|---|---
AP | AP50 | AP75 | AP | AP50 | AP75
OSFormer (Pei et al. 2022) | - | 41.00 | 71.10 | 40.80 | 42.50 | 72.50 | 42.30
Mixup (Zhang et al. 2018) | 39.65 | 68.47 | 38.59 | 39.21 | 68.10 | 38.72
Cutmix (Yun et al. 2019) | 37.85 | 66.32 | 37.14 | 38.96 | 67.18 | 38.25
AutoAugment (Zoph et al. 2020) | 41.30 | 71.48 | 39.26 | 42.74 | 72.31 | 42.55
Copy-Paste (Ghiasi et al. 2021) | 39.82 | 71.25 | 38.63 | 41.90 | 71.58 | 41.43
CamDiff (Luo et al. 2023) | 42.24 | 72.57 | 41.39 | 43.25 | 72.86 | 42.18
| CamoFourier | 43.52 | 74.84 | 42.65 | 44.95 | 75.67 | 44.28

Table 5: Our ablation study in the task of camouflaged object detection.
CamoFourier with a cross-attention mechanism and adaptive hybrid swapping
achieves the best performance when plugged into SINetV2 (Fan et al. 2022).

Method | CamoFourier | COD10K-Test | NC4K-Test
---|---|---|---
Cross-attention | Hybrid swapping | S${}_{m}\uparrow$ |  $\alpha$E $\uparrow$ |  $w$F $\uparrow$ | M $\downarrow$ | S${}_{m}\uparrow$ |  $\alpha$E $\uparrow$ |  $w$F $\uparrow$ | M $\downarrow$
SINetV2 (Fan et al. 2022) | ✗ | ✗ | 0.852 | 0.908 | 0.773 | 0.045 | 0.821 | 0.882 | 0.695 | 0.034
✓ | ✗ | 0.857 | 0.911 | 0.776 | 0.044 | 0.824 | 0.888 | 0.697 | 0.033
✗ | ✓ | 0.853 | 0.910 | 0.774 | 0.045 | 0.825 | 0.884 | 0.699 | 0.033
✓ | ✓ | 0.859 | 0.914 | 0.778 | 0.043 | 0.826 | 0.890 | 0.704 | 0.032

Table 6: Our ablation study in the task of camouflaged instance segmentation.
CamoFourier with cross-attention mechanism and adaptive hybrid swapping
achieves the best performance when plugged into OSFormer (Pei et al. 2022).

Method | CamoFourier | COD10K-Test | NC4K-Test
---|---|---|---
Cross-attention | Hybrid swapping | AP | AP50 | AP75 | AP | AP50 | AP75
OSFormer (Pei et al. 2022) | ✗ | ✗ | 42.41 | 72.86 | 42.04 | 43.87 | 73.66 | 42.95
✓ | ✗ | 42.96 | 74.15 | 42.39 | 44.58 | 75.21 | 44.01
✗ | ✓ | 42.75 | 73.84 | 42.26 | 44.13 | 74.28 | 43.72
✓ | ✓ | 43.52 | 74.84 | 42.65 | 44.95 | 75.67 | 44.28

#### Evaluation metrics.

For COD, we use four common metrics: Structure-measure (Sm) (Fan et al. 2017),
mean absolute error (M) (Perazzi et al. 2012), weighted F-measure ($w$F)
(Margolin, Zelnik-Manor, and Tal 2014), and adaptive E-measure ($\alpha$E)
(Fan et al. 2018). Sm measures the structural similarity between predictions
and ground truth, taking into account both region- and object-awareness. M
measures the pixel-level accuracy between a predicted map and ground truth and
is widely used in salient object detection (SOD) tasks. $w$F measures the
balance between recall and precision. $\alpha$E measures the human visual
perception-based alignment between predictions and ground truth at both pixel
and image levels and is suitable for evaluating the global and local accuracy
of camouflaged object detection results. Regarding CIS, we adopt COCO-style
(Lin et al. 2014) evaluation metrics such as AP50, AP75, and AP. Unlike the
mAP metric used in instance segmentation, we do not consider the class labels
of camouflaged instances, since they are class-agnostic. We only need to
consider the existence of camouflaged instances and ignore the class mean
value.

### Implementation Details

In our CamoFourier, the GAN architecture consists of generator and
discriminator components. We adopt a simple UNet encoder-decoder architecture
for the generator. However, we remove all skip connections between the encoder
and the decoder. Regarding the discriminator, we leverage $70\times 70$
PatchGAN (Isola et al. 2017) to classify if each $N\times N$ patch in an image
is real or fake. The Fast Fourier Transform (FFT) and iFFF are implemented
using the PyTorch FFT package. Our CamoFourier is plugged into several
camouflaged object detectors and instance segmenters for end-to-end training.
The parameter $\beta$ in adaptive hybrid swapping is optimized by Bayesian
optimization in range $(0,1)$ with step size $0.01$. Original images are
resized to $512\times 512$ and transformed using CamoFourier then fed into
specific architectures of COD and CIS. We adopt the Adam optimizer with a
learning rate of $10^{-4}$, $\beta_{1}=0.5$, and $\beta_{2}=0.999$. All
variants of CamoFourier are trained with a batch size of $8$ on a single
NVIDIA RTX 3090 GPU.

### Comparisons with State-of-the-art Methods

We validate the effectiveness of our learnable augmentation in the frequency
domain by integrating our augmentation strategy into several camouflaged
object detection and camouflaged instance segmentation models.

#### Camouflaged object detection.

We integrate the proposed CamoFourier into $5$ different state-of-the-art
camouflaged object detection models including: SINet (Fan et al. 2020a),
SINetV2 (Fan et al. 2022), SegMaR (Jia et al. 2022), ZoomNet (Pang et al.
2022), and FDNet (Zhong et al. 2022). The experimental results presented in
Table 1 show that our CamoFourier improves the performance of the 5 methods on
all considered metrics.

#### Camouflaged instance segmentation.

The CIS task has not been well-studied as only one CIS model (OSFormer (Pei et
al. 2022)) has been proposed to date. Therefore, we also conduct an experiment
on several popular generic instance segmentation models for more comprehensive
evaluation. Following the experiments of OSFormer (Pei et al. 2022), we train
those instance segmentation models on the instance-level COD10K training set
and test them on the COD10K (Fan et al. 2020a) and NC4K (Lv et al. 2021) test
sets. Table 2 shows that our learnable augmentation improves the performance
of the instance segmentation models in the CIS task on all AP metrics.
Specifically, our method increases the AP score of OSFormer (Pei et al. 2022)
by a significant margin of $2.52$. For the generic instance segmentation model
Mask R-CNN (He et al. 2017), our augmentation method boosts the AP score by an
even higher margin of $3.74$.

#### Comparisons with state-of-the-art augmentation methods.

We compare our CamoFourier with CamDiff (Luo et al. 2023) and cutting-edge
augmentation methods for generic object detection and instance segmentation
including Mixup (Zhang et al. 2018), CutMix (Yun et al. 2019), AutoAugment
(Zoph et al. 2020), and Copy-Paste (Ghiasi et al. 2021) on COD and CIS tasks.
In particular, we plug the mentioned augmentation methods into SINetV2 (Fan et
al. 2022) and OSFormer (Pei et al. 2022) for COD and CIS tasks, respectively.
Table 3 and Table 4 indicate that generic augmentation methods are obviously
not suitable for camouflaged object detection and instance segmentation; they
even make the performance of SINetV2 (Fan et al. 2022) and OSFormer (Pei et
al. 2022) deteriorate compared to the ones without augmentation. On the
contrary, our proposed CamoFourier helps improve the effectiveness of SINetV2
(Fan et al. 2022) and OSFormer (Pei et al. 2022) by large margins. Our
CamoFourier also outperforms CamDiff (Luo et al. 2023), the current state-of-
the-art method for camouflaged objects on both COD and CIS tasks.

### Qualitative Results

#### Qualitative evaluation.

Figures 3 and 4 visualize the segmentation results of SINetV2 (Fan et al.
2022) and OSFormer (Pei et al. 2022) with and without our CamoFourier,
respectively. The results demonstrate that SINetV2 and OSFormer equipped with
CamoFourier are able to delineate the camouflaged object in a more accurate
way. More qualitative results for COD and CIS tasks are presented in the
supplementary materials.

#### Visualization of augmented images.

In Fig. 5, we illustrate the outputs of our CamoFourier, which are augmented
images $\mathcal{I}*$. In the CamoFourier framework, the conditional GAN
module and a cross-attention mechanism synthesize a reference image
$\mathcal{I}_{R}$. After that, we transform an input image and a reference
image into the frequency domain and perform basic swapping or adaptive hybrid
swapping of these two amplitudes. We also compare the qualitative results of
our CamoFourier with and without adaptive hybrid swapping. Figure 5 shows that
our adaptive hybrid swapping is able to control the amount of texture and
color information that is transferred from the reference image
$\mathcal{I}_{R}$ to the input image $\mathcal{I}$ thanks to the parameter
$\beta$.

Figure 3: Qualitative comparison of SINetV2 with and without our proposed
CamoFourier in COD task. Figure 4: Qualitative comparison of OSFormer with and
without our proposed CamoFourier in CIS task. Figure 5: Visualization of
augmented images by our CamoFourier. Our proposed augmentation highlights the
underlying structure of camouflaged objects for better identification and
segmentation. We also compare the transformed results of our method with and
without adaptive hybrid swapping. Our adaptive hybrid swapping is able to
control the amount of texture and color information that is transferred from
the reference image to the input image.

### Ablation Study

#### Effectiveness of cross-attention module.

We examine the effectiveness of our cross-attention module in the CamoFourier
framework by comparing the performance of COD (SINetV2 (Fan et al. 2022)) and
CIS (OSFormer (Pei et al. 2022)) models with and without the cross-attention
mechanism. Without the cross-attention mechanism, we directly transform the
input image $\mathcal{I}$ and the generated image $\mathcal{I}_{G}$ from the
GAN module into the frequency domain and perform amplitude swapping. Table 5
shows that the cross-attention module in CamoFourier improves the performance
of COD on the COD10K (Fan et al. 2020a) and NC4K (Lv et al. 2021) test sets on
four metrics. Table 6 shows that our cross-attention module in CamoFourier
also enhances the performance of CIS on the COD10K and NC4K test sets.

#### Significance of adaptive hybrid swapping.

We evaluate the significance of the proposed adaptive hybrid swapping in the
CamoFourier augmentation. We also perform experiments on COD and CIS tasks
with 2 models: SINetV2 (Fan et al. 2022) for COD, OSFormer (Pei et al. 2022)
for CIS. We apply our CamoFourier to these architectures with and without
adaptive hybrid swapping. Without using the proposed hybrid swapping, we use
the basic swapping of amplitude as described in Section Basic swapping of
amplitude.. Tables 5 and 6 show that our adaptive hybrid swapping helps the
CamoFourier augmentation to boost the performance of COD and CIS models. More
ablation studies on the effectiveness of the cross-attention mechanism and
adaptive hybrid swapping on other COD and CIS models are presented in the
supplementary material.

## Conclusion

In this paper, we propose CamoFourier, a novel learnable Fourier-based
augmentation method for camouflaged object detection and instance
segmentation. Our method incorporates a generative model and cross-attention
mechanism to create a reference image and a Fourier transform to mix the low
and high-frequency components of the input and reference images. This makes
the camouflaged objects more visible and easier to be detected and segmented.
We conduct extensive experiments on various camouflaged object detection and
instance segmentation models on different datasets. The experimental results
show that our method significantly improves the existing state-of-the-art COD
and CIS methods by a large margin.

## References

  * Bolya et al. (2019) Bolya, D.; Zhou, C.; Xiao, F.; and Lee, Y. J. 2019.  YOLACT: Real-Time Instance Segmentation.  In _ICCV_.
  * Cai and Vasconcelos (2018) Cai, Z.; and Vasconcelos, N. 2018.  Cascade R-CNN: Delving Into High Quality Object Detection.  In _CVPR_.
  * Chen, Hsieh, and Liu (2021) Chen, D.-J.; Hsieh, H.-Y.; and Liu, T.-L. 2021.  Adaptive Image Transformer for One-Shot Object Detection.  In _CVPR_ , 12247–12256.
  * Das et al. (2022) Das, N. N.; Kumar, N.; Kaur, M.; Kumar, V.; and Singh, D. 2022.  Automated deep transfer learning-based approach for detection of COVID-19 infection in chest X-rays.  _Irbm_ , 43(2): 114–119.
  * Dosovitskiy et al. (2021) Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; Uszkoreit, J.; and Houlsby, N. 2021.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.  In _ICLR_.
  * Fan et al. (2017) Fan, D.-P.; Cheng, M.-M.; Liu, Y.; Li, T.; and Borji, A. 2017.  Structure-Measure: A New Way to Evaluate Foreground Maps.  In _ICCV_.
  * Fan et al. (2018) Fan, D.-P.; Gong, C.; Cao, Y.; Ren, B.; Cheng, M.-M.; and Borji, A. 2018.  Enhanced-alignment Measure for Binary Foreground Map Evaluation.  In _IJCAI_ , 698–704.
  * Fan et al. (2022) Fan, D.-P.; Ji, G.-P.; Cheng, M.-M.; and Shao, L. 2022.  Concealed Object Detection.  _IEEE T-PAMI_ , 44(10): 6024–6042.
  * Fan et al. (2020a) Fan, D.-P.; Ji, G.-P.; Sun, G.; Cheng, M.-M.; Shen, J.; and Shao, L. 2020a.  Camouflaged Object Detection.  In _CVPR_.
  * Fan et al. (2020b) Fan, D.-P.; Ji, G.-P.; Zhou, T.; Chen, G.; Fu, H.; Shen, J.; and Shao, L. 2020b.  PraNet: Parallel Reverse Attention Network for Polyp Segmentation.  In Martel, A. L.; Abolmaesumi, P.; Stoyanov, D.; Mateus, D.; Zuluaga, M. A.; Zhou, S. K.; Racoceanu, D.; and Joskowicz, L., eds., _MICCAI_ , 263–273. Cham.
  * Ghiasi et al. (2021) Ghiasi, G.; Cui, Y.; Srinivas, A.; Qian, R.; Lin, T.-Y.; Cubuk, E. D.; Le, Q. V.; and Zoph, B. 2021.  Simple Copy-Paste Is a Strong Data Augmentation Method for Instance Segmentation.  In _CVPR_ , 2918–2928.
  * Guo et al. (2021) Guo, R.; Niu, D.; Qu, L.; and Li, Z. 2021.  SOTR: Segmenting Objects With Transformers.  In _ICCV_ , 7157–7166.
  * He et al. (2017) He, K.; Gkioxari, G.; Dollar, P.; and Girshick, R. 2017.  Mask R-CNN.  In _ICCV_.
  * Isola et al. (2017) Isola, P.; Zhu, J.-Y.; Zhou, T.; and Efros, A. A. 2017.  Image-To-Image Translation With Conditional Adversarial Networks.  In _CVPR_.
  * Jia et al. (2022) Jia, Q.; Yao, S.; Liu, Y.; Fan, X.; Liu, R.; and Luo, Z. 2022.  Segment, Magnify and Reiterate: Detecting Camouflaged Objects the Hard Way.  In _CVPR_ , 4713–4722.
  * Le et al. (2022) Le, T.-N.; Cao, Y.; Nguyen, T.-C.; Le, M.-Q.; Nguyen, K.-D.; Do, T.-T.; Tran, M.-T.; and Nguyen, T. V. 2022.  Camouflaged Instance Segmentation In-The-Wild: Dataset, Method, and Benchmark Suite.  _IEEE T-IP_ , 31: 287–300.
  * Le et al. (2019) Le, T.-N.; Nguyen, T. V.; Nie, Z.; Tran, M.-T.; and Sugimoto, A. 2019.  Anabranch network for camouflaged object segmentation.  _CVIU_ , 184: 45–56.
  * Lin et al. (2014) Lin, T.-Y.; Maire, M.; Belongie, S. J.; Hays, J.; Perona, P.; Ramanan, D.; Dollár, P.; and Zitnick, C. L. 2014.  Microsoft COCO: Common Objects in Context.  In _ECCV_.
  * Luo et al. (2023) Luo, X.-J.; Wang, S.; Wu, Z.; Sakaridis, C.; Cheng, Y.; Fan, D.-P.; and Gool, L. V. 2023.  CamDiff: Camouflage Image Augmentation via Diffusion Model.  arXiv:2304.05469.
  * Lv et al. (2021) Lv, Y.; Zhang, J.; Dai, Y.; Li, A.; Liu, B.; Barnes, N.; and Fan, D.-P. 2021.  Simultaneously Localize, Segment and Rank the Camouflaged Objects.  In _CVPR_ , 11591–11601.
  * Margolin, Zelnik-Manor, and Tal (2014) Margolin, R.; Zelnik-Manor, L.; and Tal, A. 2014.  How to Evaluate Foreground Maps.  In _CVPR_ , 248–255.
  * Oliva, Torralba, and Schyns (2006) Oliva, A.; Torralba, A.; and Schyns, P. G. 2006.  Hybrid Images.  In _ACM SIGGRAPH_ , 527–532.  ISBN 1595933646.
  * Pang et al. (2022) Pang, Y.; Zhao, X.; Xiang, T.-Z.; Zhang, L.; and Lu, H. 2022.  Zoom in and Out: A Mixed-Scale Triplet Network for Camouflaged Object Detection.  In _CVPR_ , 2160–2170.
  * Pei et al. (2022) Pei, J.; Cheng, T.; Fan, D.-P.; Tang, H.; Chen, C.; and Van Gool, L. 2022.  OSFormer: One-Stage Camouflaged Instance Segmentation with Transformers.  In _ECCV_.
  * Perazzi et al. (2012) Perazzi, F.; Krähenbühl, P.; Pritch, Y.; and Hornung, A. 2012.  Saliency filters: Contrast based filtering for salient region detection.  In _CVPR_ , 733–740.
  * Rombach et al. (2022) Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P.; and Ommer, B. 2022.  High-Resolution Image Synthesis With Latent Diffusion Models.  In _CVPR_.
  * Xu et al. (2021) Xu, Q.; Zhang, R.; Zhang, Y.; Wang, Y.; and Tian, Q. 2021.  A Fourier-Based Framework for Domain Generalization.  In _CVPR_ , 14383–14392.
  * Yang and Soatto (2020) Yang, Y.; and Soatto, S. 2020.  FDA: Fourier Domain Adaptation for Semantic Segmentation.  In _CVPR_.
  * Yun et al. (2019) Yun, S.; Han, D.; Oh, S. J.; Chun, S.; Choe, J.; and Yoo, Y. 2019.  Cutmix: Regularization strategy to train strong classifiers with localizable features.  In _ICCV_ , 6023–6032.
  * Zhai et al. (2021) Zhai, Q.; Li, X.; Yang, F.; Chen, C.; Cheng, H.; and Fan, D.-P. 2021.  Mutual Graph Learning for Camouflaged Object Detection.  In _CVPR_ , 12997–13007.
  * Zhang et al. (2018) Zhang, H.; Cisse, M.; Dauphin, Y. N.; and Lopez-Paz, D. 2018.  mixup: Beyond Empirical Risk Minimization.  In _ICLR_.
  * Zhong et al. (2022) Zhong, Y.; Li, B.; Tang, L.; Kuang, S.; Wu, S.; and Ding, S. 2022.  Detecting Camouflaged Object in Frequency Domain.  In _CVPR_ , 4504–4513.
  * Zoph et al. (2020) Zoph, B.; Cubuk, E. D.; Ghiasi, G.; Lin, T.-Y.; Shlens, J.; and Le, Q. V. 2020.  Learning Data Augmentation Strategies for Object Detection.  In Vedaldi, A.; Bischof, H.; Brox, T.; and Frahm, J.-M., eds., _ECCV_ , 566–583. Cham.