# A robust method for quantification of
surface elasticity in soft solids

Stefanie Heyden<EMAIL_ADDRESS>Department of Materials, ETH
Zürich, 8093 Zürich, Switzerland.  Petia M. Vlahovska
<EMAIL_ADDRESS>Engineering Sciences and Applied Mathematics,
Northwestern University, Evanston, Illinois 60208, USA.  Eric R. Dufresne
<EMAIL_ADDRESS>Department of Materials, ETH Zürich, 8093 Zürich,
Switzerland.

###### Abstract

We propose an approach to measure surface elastic constants of soft solids.
Generally, this requires one to probe interfacial mechanics at around the
elastocapillary length scale, which is typically microscopic. Deformations of
microscopic droplets embedded in soft solids are particularly attractive,
because they avoid intrinsic nonlinearities associated with previous
experiments such as the equilibrium of contact lines and the relaxation of
patterned surfaces. We derive analytical solutions for the shape of droplets
under uniaxial deformation and for the radius of droplets upon hydrostatic
inflation. We couple mechanical deformations to the dissolution of droplets to
assess experimental sensitivities. Combined with experimental data from both
modes of deformation, one should be able to reliably extract the complete set
of isotropic surface material parameters following a specific minimization
procedure.

##  I Introduction

¿epif’an-eia, [¡h]: visible surface of a body. In ancient greek, the
linguistic distinction between an object’s boundary and interior dates back to
the centuries _a.Chr.n._ [18]. It took another two millenia for the
corresponding concepts to be laid out in fluid- and solid mechanics [25, 26,
27, 31, 30, 9, 10]. At the tip of this timeline, the debate on surface
mechanics of soft solids is ongoing [7, 14, 41, 35, 42, 36, 20, 11]. The
controversy around surface elastic effects in soft solids is rooted in their
hybrid nature, incorporating features of both fluids and solids. While model
systems such as polymer networks exhibit fluid-like behavior at scales below
the polymer mesh size, the presence of crosslinks gives rise to a finite shear
modulus at the macroscale as in elastic solids [23].

Experimental evidence for surface elasticity of gels has emerged in a variety
of settings. The coupling of elasticity to capillarity was first examined in
[21]. Further studies include static wetting on strained substrates [41, 42],
soft solid contacts [14], and the strain-dependent topography of soft solids
[1]. In all settings, deformations are induced at the scale of the
elastocapillary length, which is on the order of micrometers in gels (E
$\sim\mathcal{O}$(kPa)). At this scale, surface effects start to dominate over
the bulk behavior.

Previous theoretical studies on surface mechanical properties in solids are
based on the pioneering work of Gurtin and Murdoch [9, 10]. Investigations
have focused on point- or line-forces acting upon an elastic substrate as
induced by static wetting (see, _e.g._ , [15, 34, 19, 5, 29, 7, 23, 11]).
Additional studies have examined surface-tension-induced flattening of nearly
plane elastic solids [13], the strain-dependent topography of soft solids [1],
as well as the effects of capillarity in inhomogeneous soft materials on Euler
buckling [37]. Extensions to the canonical problem of an inclusion embedded in
an elastic matrix (known as _Eshelby’s inclusion problem_) have furthermore
accounted for surface elastic effects at the interface (cf., _e.g._ , [8, 33,
6, 22]). A recent work [43] extends theoretical investigations to the
calibration of surface hyperelastic models in soft solids through a
combination of tensile and torsion tests on slender objects.

However, several drawbacks can impede a clean interpretation of experimental
data: Attendant large deformations (_i.e._ via applied pre-stretches [41, 42]
or initial relaxations of patterned surfaces [1]) can induce material- and
geometric nonlinearities. In addition, geometric singularities such as the tip
of a wetting ridge cause stress divergences within the bulk material [15, 20,
23].

Here, we propose a series of mechanical tests to reliably quantify surface
elasticity in soft solids. For isotropic, linear-elastic surfaces, three
surface material parameters need to be determined: the residual surface stress
$\Upsilon^{0}$, as well as surface shear- and bulk moduli $\mu^{s}$ and
$\kappa^{s}$, respectively. We consider two distinct deformation modes: 1) The
shape of droplets embedded in an elastic matrix under uniaxial strain, and 2)
The dilation of condensing droplets under hydrostatic pressure within an
elastic matrix. Both modes of deformation presented here feature geometrically
smooth surfaces, hence circumventing geometric singularities. In addition, the
proposed deformation modes are accessible in the small deformation regime. The
reference configuration is that of a smooth undisturbed surface, hence
possible nonlinearities are eliminated. Inspired by the dual nature of soft
solids, we utilize methods from both fluid- and solid mechanics to solve for
interface deformation upon loading. The proposed data analysis allows for a
reliable recovery of the complete set of surface parameters
$(\Upsilon^{0},\mu^{s},\kappa^{s})$. In addition, fitting sensitivities are
estimated for all physically admissible classes of parameter ranges. We find
that a robust extraction of relevant quantities is achieved for good
experimental signal to noise ratios.

##  II Gedankenexperimente

Consider a dispersion of microscopic liquid droplets in a soft solid (see
Figure 1). This can be fabricated in two steps [33]. First, the liquid is
dispersed in a polymeric precursor at a temperature, $T^{mix}$. The molecules
making up the droplet phase partially dissolve to reach an equilibrium
concentration in the solid phase, $n_{sat}(T^{mix})$. Then, the sample is
crosslinked, and the droplets are surrounded by a stress-free solid network.
At sufficient dilution, each droplet will respond independently to imposed
stresses.

We propose two complementary mechanical probes of these samples: macroscopic
uniaxial deformation and microscopic inflation. As we will see, these two
tests are sensitive to different surface mechanical parameters. The smooth
surfaces of droplets avoids nonlinear stress concentrations, and the tests can
be conducted in the small deformation limit.

For the first test, the temperature is fixed and the sample is stretched [33],
as shown on the left of Figure 1. Initial droplet radii are recorded and the
final cross-sectional shape of droplets are measured at a fixed uniaxial far-
field strain. Droplets and matrix are assumed to be incompressible. The
equilibrium shapes of the droplets are determined by a balance of elastic and
interfacial stresses with the hydrostatic pressure, $p$, within the droplets.
The hydrostatic pressure $p$ may be decomposed as $p=p^{0}+p^{in}$, where
$p^{0}$ is the Laplace pressure in the undeformed configuration and $p^{in}$
is an incremental pressure increase, induced by deformation.

For the second test, we simply shift the temperature. This changes the
saturation concentration in the solid phase, which in turn induces a driving
pressure $p^{in}$. We record the subsequent change in droplet radii, as shown
in the right of Figure 1. When the temperature dependence of the saturation
concentration is known, the driving pressure can be determined
thermodynamically, [36, 28],

$p^{in}=n_{L}k_{B}T\,\text{ln}\left(\frac{n^{sat}(T,p^{in})}{n^{sat}(T,0)}\right).$
(1)

Here, $n_{L}$ is the number density of molecules in the condensed phase, and
$k_{B}$ denotes Boltzmann’s constant. $n^{sat}(T,p^{in})$ denotes the
equilibrium number density of molecules in the dilute phase, whereas
$n^{sat}(T,0)$ denotes its equilibrium value in the absence of a mechanical
load ($p^{in}=0$).

$T=T^{mix}$$T>T^{mix}$$T<T^{mix}$$T=T^{mix}$$\epsilon_{xx}^{\infty}$$\epsilon_{xx}^{\infty}$
Figure 1: Sketch of the experimental set-up. Both deformation modes of
uniaxial strain (left) and applied pressure (right) follow from the same
starting structure.

##  III Theoretical Background

We seek to derive the deformation of an initially spherical closed elastic
interface embedded in an elastic matrix under two different types of applied
loading. The matrix is assumed to be incompressible. Matrix elasticity is
modeled as a linear elastic isotropic solid exhibiting surface elasticity at
the droplet/matrix interface. The outer matrix boundary is assumed to be far
from the droplet/matrix interface, such that finite size effects of the
elastic matrix may be neglected as well as the matrix deformation sufficiently
far away from the droplet. Droplet concentrations are furthermore taken to be
in the dilute limit, such that interactions between droplets are negligible.

The equilibrium equations of a solid taking into account both bulk- as well as
surface energies follows from the energy functional

$\mathcal{F}[\boldsymbol{\varphi}]=\int_{\Omega}W(\boldsymbol{\nabla}\boldsymbol{\varphi})\,dV+\int_{\partial\Omega^{\gamma}}\gamma(\boldsymbol{\nabla}\boldsymbol{\varphi})\,dS-\int_{\partial\Omega^{t}}\mathbf{t}_{0}\cdot\boldsymbol{\varphi}\,dS.$
(2)

Here, $\boldsymbol{\varphi}(\mathbf{X})$ is the deformation mapping with
$\boldsymbol{\varphi}(\mathbf{X})\in\mathcal{U}=\\{\boldsymbol{u}\in
H^{1}(\Omega):\boldsymbol{\varphi}=\boldsymbol{\varphi}_{0}\,\text{on}\,\partial\Omega^{u}\\}$,
where $\partial\Omega^{u}$ is the boundary over which displacements are
imposed. $W(\nabla\boldsymbol{\varphi})$ is the strain energy density of the
bulk material, $\gamma(\nabla\boldsymbol{\varphi})$ is the strain energy
density associated with the surface $\partial\Omega^{\gamma}$, and
$\mathbf{t}_{0}$ is the applied traction along the traction boundary
$\partial\Omega^{t}$. For a system to be in equilibrium, the principal of
minimum potential energy requires a vanishing first variation of the energy
functional $\delta\mathcal{F}[\boldsymbol{\varphi}]=0$ (for a detailed review
on the derivation of governing equations within the framework of continuum
mechanics, we refer to [12, 22] and references therein). This results in the
set of governing equations

$\displaystyle\text{div}(\boldsymbol{\sigma})=0\quad$
$\displaystyle\text{in}\,\Omega,$ (3a)
$\displaystyle\boldsymbol{\sigma}\mathbf{n}=\mathbf{t}_{0}\quad$
$\displaystyle\text{on}\,\partial\Omega^{t},$ (3b)
$\displaystyle\boldsymbol{\sigma}\mathbf{n}-\text{div${}_{s}$}(\boldsymbol{\sigma^{s}})=\mathbf{t}_{0}\quad$
$\displaystyle\text{on}\,\partial\Omega^{\gamma},\quad\text{and}$ (3c)
$\displaystyle\boldsymbol{u}=\boldsymbol{u}_{0}\quad$
$\displaystyle\text{on}\,\partial\Omega^{u}.$ (3d)

Here, $\boldsymbol{\sigma}=\frac{\partial
W}{\partial(\nabla\boldsymbol{\varphi})}$ and
$\boldsymbol{\sigma^{s}}=\frac{\partial\gamma}{\partial(\nabla\boldsymbol{\varphi})}$
are the first Piola-Kirchhoff stress measures within the bulk- and surface
material, respectively. $\mathbf{n}$ is the outward unit normal to the
surface, which is a function of surface displacements [40]. As usual,
displacements are defined as
$\mathbf{u}=\boldsymbol{\varphi}(\mathbf{X})-\mathbf{X}$. A salient feature of
the proposed experiments is their applicability in the small deformation
regime $|\boldsymbol{\nabla}_{s}\mathbf{u}|\sim\epsilon\ll 1$. The matrix
material is cured around pre-existing incompressible droplets. Therefore, the
reference configuration is close to a critical stress-free configuration. For
an isotropic material, the linearized first Piola-Kirchhoff surface stress
follows as [17]

$\boldsymbol{\sigma}^{s}=\Upsilon^{0}\mathbf{I}^{s}+2\mu^{s}\mathbf{E}^{s}+(\kappa^{s}-\mu^{s})\text{tr}(\mathbf{E}^{s})\mathbf{I}^{s},$
(4)

where it is important to note that the residual surface stress $\Upsilon^{0}$
is of order $\mathcal{O}(\epsilon)$ [22]. Here, $\mathcal{O}(\epsilon^{n})$
implies the asymptotic behavior
$\mathcal{O}(\epsilon^{n})/\epsilon^{n}\rightarrow C\neq 0$ as
$\epsilon\rightarrow 0$, with $n\in\mathbb{R}$.

Figure 2: Left: Elastic interface subjected to a displacement field
$\mathbf{u}^{\infty}$ at infinity. Right: Infinite elastic matrix with an
embedded cavity subjected to an inflation pressure $p$ at $r=R_{0}$. Both
interfaces (gray) exhibit exhibit surface elasticity as highlighted in the
inset.

##  IV Shape of Droplets under Applied Stretch

We consider a static uniaxial strain applied at the far-field boundary of an
elastic matrix with an embedded droplet exhibiting a hydrostatic state of
stress. The droplet is composed of an incompressible liquid, hence the
droplet’s volume is preserved under the applied deformation. The
droplet/matrix interface is modeled as an initially spherical closed elastic
interface. The set of governing equations (3) reduces to

$\displaystyle\text{div}(\boldsymbol{\sigma})=0\quad$
$\displaystyle\text{in}\,\Omega,$ (5a)
$\displaystyle\boldsymbol{\sigma}\mathbf{n}=\mathbf{0}\quad$
$\displaystyle\text{on}\,\partial\Omega^{t},$ (5b)
$\displaystyle\boldsymbol{\sigma}\mathbf{n}-\text{div${}_{s}$}(\boldsymbol{\sigma}^{s})=\mathbf{0}\quad$
$\displaystyle\text{on}\,\partial\Omega^{\gamma},\quad\text{and}$ (5c)
$\displaystyle\mathbf{u}=\mathbf{u}^{\infty}\quad$
$\displaystyle\text{on}\,\partial\Omega^{u}.$ (5d)

Inspired by the dual nature of soft solids, we exploit the equivalence between
linear elastic statics and Stokes flow of Newtonian fluids (see Appendix B for
details). As illustrated in Figure 2, we may thus adopt solutions for an
elastic interface in a straining flow field [39, 3, 2, 4] for the solution of
the elastic problem.

###  IV.1 Interfacial shape equations

The displacement field $\mathbf{u}^{\infty}$ at infinity induces disturbed
displacement fields in the interface vicinity, which are denoted as
$\mathbf{u}^{-}$ for the surrounding- and $\mathbf{u}^{+}$ for the
encapsulated solid. In addition, $\mathbf{u^{s}}$ is the induced deformation
of the elastic interface. A triaxial affine strain follows from a displacement
field $\mathbf{u}^{\infty}=(c_{x}x,c_{y}y,c_{z}z)$, which in spherical
coordinates reads

$\displaystyle\mathbf{u}^{\infty}$
$\displaystyle=r\left(c_{z}\cos(\theta)^{2}+c_{x}\cos(\varphi)^{2}\sin(\theta)^{2}+c_{y}\sin(\theta)^{2}\sin(\varphi)^{2}\right)\mathbf{g}_{r}$
$\displaystyle+r\left(-c_{z}\cos(\theta)\sin(\theta)+c_{x}\cos(\theta)\cos(\varphi)^{2}\sin(\theta)+c_{y}\cos(\theta)\sin(\theta)\sin(\varphi)^{2}\right)\mathbf{g}_{\theta}$
$\displaystyle+r\left(-c_{x}\cos(\varphi)\sin(\theta)\sin(\varphi)+c_{y}\cos(\varphi)\sin(\theta)\sin(\varphi)\right)\mathbf{g}_{\varphi}.$
(6)

Here, ($\mathbf{g}_{r}$, $\mathbf{g}_{\theta}$, $\mathbf{g}_{\varphi}$) denote
the radial-, polar- and azimuthal base vectors, respectively.

In the following, we exploit the spherical geometry of the problem and expand
all variables in terms of spherical harmonics basis functions (see Appendix D
for details). The displacement field of the interface is written as

$\displaystyle\mathbf{u}^{s}$
$\displaystyle=f(\theta,\varphi)\mathbf{g}_{r}+\mathbf{u}^{s}_{||},$
$\displaystyle f(\theta,\varphi)$ $\displaystyle=\sum_{j=1,j\neq
1}^{\infty}\sum_{m=-j}^{j}f_{jm}Y_{jm}(\theta,\varphi)$
$\displaystyle\mathbf{u}^{s}_{||}(\theta,\varphi)$
$\displaystyle=\sum_{j=1}^{\infty}\sum_{m=-j}^{j}(z_{jm}\mathbf{y}_{jm1}(\theta,\varphi)+u_{jm}\mathbf{y}_{jm0}(\theta,\varphi)),$
(7)

where $\mathbf{u}^{s}_{||}$ is the displacement component tangential to the
interface, $Y_{jm}$ are scalar spherical harmonics and $\mathbf{y}_{jmq}$ are
vector spherical harmonics. For the displacement field satisfying Equation
(42) within the spherical harmonics basis, we adopt the fundamental solutions
of the Stokes equations in a spherical geometry. We hence have

$\displaystyle\mathbf{u}^{-}$
$\displaystyle=\sum_{jmq}c_{jmq}^{-}\hat{\mathbf{u}}_{jmq}^{-}(\mathbf{r}),$
$\displaystyle\mathbf{u}^{+}$
$\displaystyle=\sum_{jmq}c_{jmq}^{+}\hat{\mathbf{u}}_{jmq}^{+}(\mathbf{r})\quad\text{and}$
$\displaystyle\mathbf{u}^{\infty}$
$\displaystyle=\sum_{jmq}c_{jmq}^{\infty}\hat{\mathbf{u}}_{jmq}^{+}(\mathbf{r}).$
(8)

Here, $\hat{\mathbf{u}}^{\pm}_{jmq}$ are vector solid spherical harmonics
corresponding to the harmonics in the Lamb solution. Suffix notation with
summation over repeated indices is implied. $q$ takes values $(0,1,2)$.
$\hat{\mathbf{u}}^{\pm}_{jm2}$ is the radial component of the velocity field,
while $\hat{\mathbf{u}}^{\pm}_{jm0}$ and $\hat{\mathbf{u}}^{\pm}_{jm1}$ are
the tangential- and surface-solenoidal components, respectively. Detailed
expressions for flow fields $\hat{\mathbf{u}}^{\pm}_{jmq}$ and resultant
tractions in terms of spherical harmonics are listed in Appendix C. The
coefficients $c^{\infty}_{jmq}$ for a generic straining flow as given in
Equation (6) are

$\displaystyle c^{\infty}_{200}$
$\displaystyle=\sqrt{\frac{6\pi}{5}}c_{z},\,c^{\infty}_{220}=\sqrt{\frac{\pi}{5}}(c_{x}-c_{y})$
$\displaystyle c^{\infty}_{202}$
$\displaystyle=2\sqrt{\frac{\pi}{5}}c_{z},\,c^{\infty}_{222}=\sqrt{\frac{2\pi}{15}}(c_{x}-c_{y}).$
(9)

At linear perturbation order [39], interfacial elastic stresses following from
Equation (4) may be written as

$\displaystyle\tau^{s}_{jm0}$
$\displaystyle=-2C_{\kappa^{s}}^{-1}\sqrt{j(j+1)}f_{jm}+\left((C_{\mu^{s}}^{-1}+C_{\Upsilon^{0}}^{-1})(j-1)(j+2)+C_{\kappa^{s}}^{-1}j(j+1)\right)u_{jm},$
$\displaystyle\tau^{s}_{jm1}$
$\displaystyle=-(C_{\mu^{s}}^{-1}+C_{\Upsilon^{0}}^{-1})(j-1)(j+2)z_{jm},\quad\text{and}$
$\displaystyle\tau^{s}_{jm2}$
$\displaystyle=C_{\Upsilon^{0}}^{-1}(j(j+1)-2)f_{jm}+2C_{\kappa^{s}}^{-1}\left(\sqrt{j(j+1)}u_{jm}-2f_{jm}\right),$
(10)

with non-dimensionalized material constants

$C_{\kappa^{s}}=\frac{\mu R_{0}}{\kappa^{s}},\,C_{\mu^{s}}=\frac{\mu
R_{0}}{\mu^{s}},\,C_{\Upsilon^{0}}=\frac{\mu R_{0}}{\Upsilon^{0}}.$ (11)

Here, $R_{0}$ is the initial radius of the elastic interface. Finally, shape
parameters $(u_{jm},z_{jm},f_{jm})$ are determined from the continuity of
normal tractions at the interface as specified in Equation (5c). For
comparison with experiment, normalized displacements parallel and
perpendicular to the applied strain are needed. Normalized displacements
follow as

$\displaystyle\frac{\mathbf{u}^{s}_{(\theta=0,\varphi=0)}}{R_{0}}$
$\displaystyle=\frac{5ER_{0}(2ER_{0}+9\kappa^{s}+3\mu^{s})c_{z}}{3\left(2E^{2}R_{0}^{2}+12\mu^{s}\kappa^{s}+6\Upsilon^{0}(3\kappa^{s}+2\mu^{s})+ER_{0}(5\Upsilon^{0}+8\kappa^{s}+6\mu^{s})\right)}\mathbf{g}_{r},$
$\displaystyle\frac{\mathbf{u}^{s}_{(\theta=\pi/2,\varphi=0)}}{R_{0}}$
$\displaystyle=\frac{5ER_{0}(2ER_{0}+9\kappa^{s}+3\mu^{s})(c_{x}-c_{y}-c_{z})}{6\left(2E^{2}R_{0}^{2}+12\mu^{s}\kappa^{s}+6\Upsilon^{0}(3\kappa^{s}+2\mu^{s})+ER_{0}(5\Upsilon^{0}+8\kappa^{s}+6\mu^{s})\right)}\mathbf{g}_{r}.$
(12)

Appendix E gives the full form of the interfacial displacement field. To
linear order, displacements
($\mathbf{u}^{s}_{(\theta=0,\varphi=0)}$,$\mathbf{u}^{s}_{(\theta=\pi/2,\varphi=0)}$)
suffice to determine the resultant ellipsoidal interface shape, where
$(l=2(R_{0}+\mathbf{u}^{s}_{(\theta=0,\varphi=0)}),w=2(R_{0}+\mathbf{u}^{s}_{(\theta=\pi/2,\varphi=0)}))$
are the length- and width of the droplet in the $(x,z)$-plane, respectively.
We can furthermore define the ratio of microscopic to macroscopic strains,
which is invariant under arbitrary biaxial loading conditions, as

$\frac{l-w}{R_{0}(c_{z}-c_{x})}=\frac{10ER_{0}(2ER_{0}+9\kappa^{s}+3\mu^{s})}{3\left(2E^{2}R_{0}^{2}+12\mu^{s}\kappa^{s}+6\Upsilon^{0}(3\kappa^{s}+2\mu^{s})+ER_{0}(5\Upsilon^{0}+8\kappa^{s}+6\mu^{s})\right)}.$
(13)

The above measures only depend on the radial component of the interfacial
displacement field. To add more information to the subsequent fitting, it is
beneficial to also consider a measure that depends on interfacial displacement
within the hoop direction. In experiments, this usually requires to track
points at the droplet surface. We circumvent this by noting that there is a
distinct angle at which radial displacements vanish. Here, the resultant
deformation can be purely attributed to hoop displacements (shown as green
points in Figure 3). Using axisymmetric loading conditions $c_{x}=c_{y}$,
Equation (55) gives

$\theta_{0}\coloneqq\theta|_{u^{s}_{r}=0}=\frac{\text{arccos}(-1/3)}{2}.$ (14)

Points at the initial droplet surface at an angle $\theta_{0}$ solely deform
within the hoop direction, thus providing a unique mapping to deformed points
at the ellipsoidal surface. The hoop displacement, $u^{s}_{\theta}$ at
$\theta_{0}$ can be determined trigonometrically from the values of
$(R_{0},l,w)$, as shown in Figure 3.

This is related to the material properties through Equation (55),

$\frac{1}{R_{0}c_{z}}u^{s}_{\theta}|_{\theta=\theta_{0}}=-\frac{5ER_{0}(2ER_{0}+3\Upsilon^{0}+6\kappa^{s})\cos{\theta_{0}}\sin{\theta_{0}}}{2\left(2E^{2}R_{0}^{2}+12\mu^{s}\kappa^{s}+6\Upsilon^{0}(3\kappa^{s}+2\mu^{s})+ER_{0}(5\Upsilon^{0}+8\kappa^{s}+6\mu^{s})\right)}.$
(15)

Quantities given in Equations (13) and (15) are used in the fitting procedure
outlined in Sections IV.2 and VII.

Figure 3: Geometrical construction to determine hoop displacements at distinct
surface points located at $\theta=\theta_{0}$ under axisymmetric loading
$c_{x}=c_{y}$.

Figure 4 illustrates resultant strain profiles at the interface. To feature a
fully generic deformation for which all components of surface strain vary, we
choose an applied displacement of the form
$\mathbf{u}^{\infty}=(-\frac{3}{4}x,-\frac{1}{4}y,z)$. In addition to strains
$\epsilon_{\theta\theta}$ and $\epsilon_{\varphi\varphi}$ within the hoop
directions, the compressive asymmetry in $\mathbf{u}^{\infty}$ also induces
shear strains $\epsilon_{\theta\varphi}$. In the plane perpendicular to the
applied loading, highest tensile strains are attained within the polar hoop
direction $\epsilon_{\theta\theta}$, whereas the azimuthal hoop direction
$\epsilon_{\varphi\varphi}$ exhibits highest compressive strains. Induced
shear strains are of comparably low magnitude.

$\epsilon_{\theta\theta}$$\epsilon_{\varphi\varphi}$$\epsilon_{\theta\varphi}$
Figure 4: Strain profiles at the interface of an incompressible droplet for an
applied displacement field
$\mathbf{u}^{\infty}=(-\frac{3}{4}x,-\frac{1}{4}y,z)$ at the far-field
boundary of the elastic matrix. Material parameters are $E=1000$ kPa,
$\Upsilon^{0}=0.01$ N/m, $\mu^{s}=0.01$ N/m, $\kappa^{s}=0.02$ N/m.

###  IV.2 Fitting droplet shapes under applied stretch

The effect of surface elasticity on the resulting deformation field may be
highlighted in comparison to available experimental data [33]. Under the
assumption of constant surface energy, a best-fit value of residual surface
stress $\Upsilon^{0}=3.6$ mN/m is obtained (see dashed lines in Figure 5). In
contrast, taking the complete set of surface material parameters into account,
best-fit values are ($\Upsilon^{0}=3.9\pm 1.7$ mN/m, $\mu^{s}=2.0\pm 2.6$
mN/m, $\kappa^{s}=510\pm 280$ mN/m). In both cases, a value of
$E=1.7\,\mathrm{kPa}$ is used for the material’s bulk Young’s modulus. While
introducing surface elastic effects does a better job fitting the data (see
solid lines in Figure 5), the uncertainty in these parameters is too large to
be useful.

Figure 5: Sample fitting of experimental data given in [33] to Equation (12).
Dashed lines illustrate predictions for vanishing surface elasticity
$(\Upsilon^{0},0,0)$, whereas solid lines depict the behavior incorporating
surface elastic effects $(\Upsilon^{0},\mu^{s},\kappa^{s})$.

To quantify the reliability of fitting parameters, we proceed with the
generation of artificial data sets. We insert test parameters into the
analytical solution given in Equations 13 and 15. The addition of gaussian
noise results in artificial data sets $D^{strain}_{1}$ and $D^{strain}_{2}$,
respectively. Here, our assumption for the experimental error is based on
available experimental data illustrated in Figure 5, with relative standard
deviation $\sim 12\%$. Based on the calculated standard deviation, a pool of
data sets $D^{strain}_{1}\cup D^{strain}_{2}$ and corresponding solutions to
the nonlinear optimization problem is generated. As highlighted in Figure 6,
the probability distributions of resultant best-fit surface parameters
$(\Upsilon^{0},\mu^{s},\kappa^{s})$ show broad spectra and do not recover test
parameters ($\Upsilon^{0}=0.01\,\text{N/m}$, $\mu^{s}=0.01\,\text{N/m}$,
$\kappa^{s}=0.02\,\text{N/m}$). A reliable estimation of the complete set of
surface material parameters hence calls for an additional mode of deformation
to be taken into account. Ideally, the additional deformation mode only
depends on a subset of surface parameters $(\Upsilon^{0},\mu^{s},\kappa^{s})$,
which is achieved by considering a hydrostatic inflation as outlined in the
following Section.

$P(\Upsilon^{0},\mu^{s},\kappa^{s}|D^{strain}_{1}\cup D^{strain}_{2})$ Figure
6: Joint minimization of the volume preserving data set $D^{strain}_{1}\cup
D^{strain}_{2}$ to Equations (13) and (15) and resultant probability
distribution $P(\Upsilon^{0},\mu^{s},\kappa^{s}|D^{strain}_{u}\cup
D^{strain}_{2})$ for test values $\Upsilon^{0}=0.01\,\text{N/m}$,
$\mu^{s}=0.01\,\text{N/m}$, $\kappa^{s}=0.02\,\text{N/m}$.

##  V Growth and shrinkage in an elastic matrix

The growth of a condensing droplet under an applied pressure is modeled as a
hydrostatic inflation of a spherical closed elastic interface embedded in an
elastic matrix (see sketch in Figure 2). In contrast to the uniaxial
deformation, this hence constitutes a volume distorting deformation mode. The
set of governing equations (3) specifies to

$\displaystyle\text{div}(\boldsymbol{\sigma})=0\quad$
$\displaystyle\text{in}\,\Omega,$ (16a)
$\displaystyle\boldsymbol{\sigma}\mathbf{n}=\mathbf{0}\quad$
$\displaystyle\text{on}\,\partial\Omega^{t}\setminus\partial\Omega^{\gamma},$
(16b)
$\displaystyle\boldsymbol{\sigma}\mathbf{n}-\text{div${}_{s}$}(\boldsymbol{\sigma^{s}})=p\mathbf{n}\quad$
$\displaystyle\text{on}\,\partial\Omega^{\gamma},\quad\text{and}$ (16c)
$\displaystyle\mathbf{u}=\mathbf{0}\quad$
$\displaystyle\text{on}\,\partial\Omega^{u}.$ (16d)

Based on the spherical symmetry of the deformation, the unknown displacement
field simplifies to $\mathbf{u}=u_{r}(r)\,\mathbf{g}_{r}$, where
$\mathbf{g}_{r}$ is the radial base vector in spherical coordinates. Governing
equation (16a) in combination with the constitutive choice of linear elastic
and isotropic material behavior hence gives

$r^{2}\frac{\partial^{2}u_{r}}{\partial r^{2}}+2r\frac{\partial
u_{r}}{\partial r}-2u_{r}=0\quad\text{in}\,\Omega,$ (17)

which is similar to the Rayleigh-Plesset equation modeling pressure pulses
within bubbles [32, 24]. The general solution to Equation (17) in terms of
unknown constants $(C_{1},C_{2})$ is

$u_{r}(r)=C_{1}r+C_{2}r^{-2},$ (18)

where we require $C_{1}=0$ for a vanishing displacement field at infinity. It
remains to determine $C_{2}$ using Equation (16c). With

$\displaystyle\mathbf{g}_{r}\cdot\text{div}_{s}(\boldsymbol{\sigma}^{s})$
$\displaystyle=-\boldsymbol{\sigma}^{s}\cdot\boldsymbol{\nabla}_{s}\,\mathbf{g}_{r}$
$\displaystyle=-\boldsymbol{\sigma}^{s}\cdot\left(\frac{1}{r}\left(\mathbf{g}_{\theta}\otimes\mathbf{g}_{\theta}+\mathbf{g}_{\varphi}\otimes\mathbf{g}_{\varphi}\right)\right),$
(19)

where $\otimes$ denotes an outer product, Equation (16c) reduces to

$\sigma_{rr}=\frac{2\sigma^{s}_{\theta\theta}}{R_{0}}-p.$ (20)

We thus find

$C_{2}=\frac{3R_{0}^{3}(pR_{0}-2\Upsilon^{0})}{4(ER_{0}+3\kappa^{s})},$ (21)

from which

$u_{r}=R-R_{0}=\frac{3R_{0}(pR_{0}-2\Upsilon^{0})}{4(ER_{0}+3\kappa^{s})}$
(22)

determines the radial displacement at the inflation boundary $r=R_{0}$. Since
the Laplace pressure has to be recovered in the limit of vanishing driving
pressure $p^{in}$, we substitute $p=p^{in}+2\Upsilon^{0}/R_{0}$ and define the
normalized radial displacement as

$\hat{u}_{r}=\frac{u_{r}}{R_{0}}\frac{E}{p^{in}}=\frac{3ER_{0}}{4(ER_{0}+3\kappa^{s})}.$
(23)

##  VI Coupling mechanics to solubility

In this Section, we connect the mechanics of deformations derived in Sections
IV and V to the dissolution of droplets governed by Equation 1. This lays the
groundwork for assessing the experimental viability of the tests for surface
elasticity proposed in Section II. In particular, we focus on stretch-induced
dissolution during uniaxial stretch (Section IV), and the sensitivity of
droplet radius to changes of temperature (Section V).

In both cases, the average number density of molecules,

$\displaystyle n^{av}$ $\displaystyle=\phi n_{L}+(1-\phi)n^{sat}(T,p^{in}),$
(24)

is constant throughout deformation/ temperature shifts. We thus have

$\displaystyle\frac{\partial n^{av}}{\partial n^{sat}(T,p^{in})}\frac{\partial
n^{sat}(T,p^{in})}{\partial T}dT+\frac{\partial n^{av}}{\partial
n^{sat}(T,p^{in})}\frac{\partial n^{sat}(T,p^{in})}{\partial
p^{in}}dp^{in}+\frac{\partial
n^{av}}{\partial\phi}\frac{\partial\phi}{\partial R}dR=0,$ (25)

where $\phi=\mathcal{N}\cdot\frac{4}{3}\pi R^{3}$ is the volume fraction of
the condensed phase and $\mathcal{N}$ the number density of droplets.

###  VI.1 Sensitivity of temperature

In Section II, we proposed to drive droplet growth through a temperature-
induced shift in solubility. The viability of this approach depends on a
careful choice of droplet composition. An appropriate system will have a
temperature dependence of solubility that enables ready access to small but
resolvable deformations. Equation (22) provides an expression for the
mechanical pressure inducing droplet growth. In the initial configuration, we
have $p^{in}=0$, $R=R_{0}$, and $n^{sat}(T,p^{in})=n^{sat}(T,0)$, such that

$dp^{in}=\frac{4(ER+3\kappa^{s})}{3R^{2}}dR.$ (26)

Evaluating the individual terms in Equation (25) therefore gives

$\frac{1}{R}\frac{dR}{dT}=-\leavevmode\nobreak\
\frac{1-\phi}{3\phi(n_{L}-n^{sat}(T,0))+\frac{4(1-\phi)n^{sat}(T,0)(ER+3\kappa^{s})}{3\leavevmode\nobreak\
n_{L}k_{B}T\leavevmode\nobreak\ R}}\leavevmode\nobreak\ \frac{\partial
n^{sat}(T,0)}{\partial T}.$ (27)

As we can see from this equation, changes in droplet size are more sensitive
to temperature shifts for small volume fractions of the condensed phase and
large differential solubilities. In addition, the right hand side of Equation
(27) is independent of droplet size for vanishing surface elasticity
$\kappa^{s}=0$.

###  VI.2 Stretch-induced dissolution

Section IV considers a droplet in an elastic matrix under an applied far-field
stretch. This deformation increases the pressure within the droplets, leading
to a driving pressure for dissolution, $p^{in}\approx E\epsilon/3$. Inserting
this pressure change into Equation (25), we find

$\frac{1}{R}\frac{dR}{d\epsilon}\approx-\leavevmode\nobreak\
\frac{1}{9}\leavevmode\nobreak\ \frac{1-\phi}{\phi}\leavevmode\nobreak\
\frac{n^{sat}(T,0)}{n_{L}-n^{sat}(T,0)}\leavevmode\nobreak\
\frac{E}{n_{L}k_{B}T}.$ (28)

As shown by this equation, dissolution/condensation of droplets is more
pronounced for stiff materials, large stretch levels, and small volume
fractions of the condensed phase.

###  VI.3 Assessment for a specific system

For the fluorinated-oil and silicone system explored in [36, 28],
$n_{L}k_{B}T\approx 10^{6}\,\text{Pa}$, $\frac{n^{sat}(T,0)}{n_{L}}\approx
3\cdot 10^{-2}$, $\frac{1}{n_{L}}\frac{dn^{sat}(T,0)}{dT}\approx
10^{-3}\,1/(^{\circ}C)$, and $\phi_{0}\approx 0.04$. Further, we assume that
$\kappa^{s}\approx\,0.06\,\mathrm{N/m}$, consistent with Reference [42].
Taking $E\approx 10^{3}\,\mathrm{Pa}$,

$\frac{1}{R}\left|\frac{dR}{dT}\right|\approx 10^{-2}\,\frac{1}{{}^{\circ}C}.$
(29)

Thus, the droplets should change their radius by about $10\%$ for a
temperature shift of $10^{\circ}$C. This should be easy to resolve, and well
within the range of linear response. For dissolution of droplets under applied
stretch, Equation (28) gives

$\frac{1}{R}\left|\frac{dR}{d\epsilon}\right|\approx 10^{-4}.$ (30)

Thus, uniaxial stretches with small strains will not lead to significant
dissolution.

##  VII Robustness of Fitting Parameters

As a final check on the viability of our approach, we investigate the ability
to extract the desired material parameters from the experimental results. We
start by classifying physically admissible parameter ranges. Surface bulk- and
shear moduli are related via

$\kappa^{s}=\lambda^{s}+\mu^{s},$ (31)

where $\lambda^{s}$ is the first surface Lamé parameter. In two dimensions,
sufficient conditions of stability for a homogenous and isotropic material
require

$\displaystyle\lambda^{s}+\mu^{s}>0,\quad\mu^{s}>0,$ $\displaystyle
E_{2D}>0,\quad-1<\nu_{2D}<1,$ (32)

with $\nu_{2D}=\lambda^{s}/(\lambda^{s}+2\mu^{s})$ [16]. Non-auxetic surfaces
hence satisfy $\lambda^{s}\geq 0$. Under these conditions, five classes of
parameter ranges are identified:

$\displaystyle\Upsilon^{0}$ $\displaystyle\approx(\mu^{s},\kappa^{s}),$
$\displaystyle\Upsilon^{0}\ll(\mu^{s},\kappa^{s})$
$\displaystyle,\quad(\mu^{s},\kappa^{s})\ll\Upsilon^{0},$
$\displaystyle\mu^{s}\ll(\Upsilon^{0},\kappa^{s})$
$\displaystyle,\quad(\Upsilon^{0},\mu^{s})\ll\kappa^{s}.$ (33)

We are now in place to extend the multi-modal nonlinear optimization problem
first introduced in Section IV.2. With the added shape of droplets under
applied hydrostatic pressure, we propose a specific minimization procedure
outlined in Figure 7.

$P(\kappa^{s}|D^{pressure})$$P(\Upsilon^{0},\mu^{s}|D^{strain}_{1}\cup
D^{strain}_{2},\kappa^{s})$$\Upsilon^{0}\sim(\mu^{s},\kappa^{s})$$\Upsilon^{0}=10\,\text{mN/m}$$\mu^{s}=10\,\text{mN/m}$$\kappa^{s}=20\,\text{mN/m}$$\hat{\sigma}\sim
10\%$$\hat{\sigma}\sim 1\%$ Figure 7: Sequential steps of the fitting
procedure. An individual fit of data set $D^{pressure}$ predicts $\kappa^{s}$.
A subsequent joint minimization over $D^{strain}_{1}\cup D^{strain}_{1}$
determines $\Upsilon^{0}$ and $\mu^{s}$. Depicted histograms are based on
surface parameters on the same order of magnitude
$\Upsilon^{0}\sim(\mu^{s},\kappa^{s})$. Results for high(reduced) levels of
experimental error $\hat{\sigma}$ are shown in yellow(green).

Our assumption for the experimental error in this analysis is based on
available experimental data [33], which gives a relative standard deviation
$\sim 12\%$. With further optimization, the experimental error could likely be
reduced to $\sim 1\%$. Here, we examine both levels of experimental noise and
the resulting optimization sensitivities.

The proposed procedure requires an initial minimization over the individual
distribution $P(\kappa^{s}|D^{pressure})$, where we exploit that the shape of
droplets under hydrostatic pressure solely depends on the surface bulk modulus
$\kappa^{s}$. Initial droplet radii are recorded and final droplet radii after
growth/shrinkage as well as the temperature difference $\Delta T$ are
measured. Experimental data is fitted using Equation (23), with inflation
pressure calculated from (1). Figure 7 shows that $\kappa^{s}$ can be
recovered for both levels of experimental noise.

We then turn to the joint probability distribution of $(\Upsilon^{0},\mu^{s})$
conditional on $\kappa^{s}$,
$P(\Upsilon^{0},\mu^{s}|D^{strain}_{1}\cup D^{strain}_{2},\kappa^{s})$.
Initial droplet radii are recorded and the attained final length- and width of
droplets at fixed uniaxial far-field strain is measured. Using axisymmetric
loading conditions, hoop displacements are determined at distinct points
initially located at $\theta=\theta_{0}$ (with $\theta_{0}$ given in Equation
(14)) . As sketched in Figure 3, the distance within the hoop direction
$\mathbf{g}_{\theta}$ to deformed points at the ellipsoidal surface is
measured. Experimental data is fitted using Equations (13) and (15).

As illustrated in Figure 7, a spectrum of narrow width can be recovered in
both $\Upsilon^{0}$ and $\mu^{s}$ even at high levels of experimental noise,
whereby no systematic shifts are encountered. Test parameters at reduced
experimental error $\hat{\sigma}\sim 1\%$ are recovered at high precision.

In the examined test case, all surface parameters are on the same order of
magnitude, $\Upsilon^{0}\sim(\mu^{s},\kappa^{s})$. This case serves as the
worst-case scenario on the resultant fitting accuracy. For all remaining
physically admissible parameter ranges, fitting accuracies are enhanced, as
shown in Section F. For the different parameter ranges illustrated in Figures
8 and 9, we focus on the estimate of ($\Upsilon^{0},\mu^{s}$), which
constitutes the only multi-modal and hence most sensitive minimization
problem. Finally, it bears mentioning that a reduced experimental error does
not qualitatively change the necessity for the outlined minimization
procedure.

##  VIII Conclusions

The quantification of surface elasticity in soft solids calls for a careful
distinction between surface- and bulk effects. Here, we derive analytical
solutions for two modes of deformation which are readily accessible in a
laboratory setting and allow for two key advantages: Both deformations are
amenable to be employed in the small deformation regime, hence circumventing
possible material nonlinearities. Droplets furthermore feature geometrically
smooth surfaces to avoid stress divergences within the bulk material. In
comparison to recent work on the calibration of surface hyperelastic
constitutive models _via_ mechanical tests on a soft cylindrical specimen
[43], the proposed experiments allow for an easy fabrication as well as stable
geometries. In addition, experiments systematically yield a population of
droplets, such that a distribution of droplet sizes is obtained. The derived
modes of deformation are coupled to the dissolution of droplets. It is shown
that droplet dissolution/ condensation is more pronounced for stiff materials,
large stretch levels, and small volume fractions of the condensed phase. We
find that a robust quantification of the complete set of isotropic surface
material parameters $(\Upsilon^{0},\mu^{s},\kappa^{s})$ requires to take into
account both modes of deformation. By following the outlined minimization
procedure, surface elastic constants can be reliably recovered at realistic
levels of experimental noise, even for a worst-case scenario in which all
surface parameters are on the same order of magnitude,
$\Upsilon^{0}\sim(\mu^{s},\kappa^{s})$.

We hope that this new approach to measuring surface elastic constants of soft
solids will resolve the controversy surrounding this topic, and enable
systematic studies to understand the structure-property relationships that
underly surface elasticity.

## Acknowledgements

This work was funded via the _SNF Ambizione_ grant PZ00P$2186041$. Great
thanks goes to Robert Style (ETH Zurich) and Nicolas Bain (ETH Zurich) for
many helpful discussions. We thank Siddhant Kumar (TU Delft) for his advice on
stochastics of multivariate optimizations problems, as well as Pradeep Sharma
(University of Houston) for sharing his expertise in the realm of surface
elasticity. On ‘surface’ matters in a more abstract sense, we enjoyed
discussions on greek philology with Nicola Schmid (University of Zurich).

##  Appendix A Differential surface Operators

The surface projection tensor utilized to project second order tensors onto
curved surfaces is defined as

$\mathbb{P}(\mathbf{X})=\mathbf{I}-\mathbf{N}(\mathbf{X})\otimes\mathbf{N}(\mathbf{X}),$
(34)

where $\mathbf{I}$ is the identity tensor and $\mathbf{N}(\mathbf{X})$ is the
outward unit normal to the tangent surface at $\mathbf{X}$. Let
$\mathbf{T}(\mathbf{X})$ be a smooth second order tensor field and
$\mathbf{t}(\mathbf{X})$ a smooth vector field. On a smooth surface with
ourward unit normal $\mathbf{N}(\mathbf{X})$, their projections are given as

$\mathbf{T}^{s}=\mathbb{P}\mathbf{T}\mathbb{P},\quad\text{and}\quad\mathbf{t}^{s}=\mathbb{P}\mathbf{t}.$
(35)

The surface identity tensor is hence obtained as

$\mathbf{I}^{s}=\mathbb{P}\mathbf{I}\mathbb{P}.$ (36)

Surface gradients of a smooth scalar field
$t(\mathbf{X}):\Omega_{0}\rightarrow\mathbb{R}$ and a smooth vector field
$\mathbf{t}(\mathbf{X}):\Omega_{0}\rightarrow\mathbb{R}^{3}$ follow from

$\boldsymbol{\nabla}_{s}t=\mathbb{P}\boldsymbol{\nabla}t,\quad\text{and}\quad\boldsymbol{\nabla}_{s}\mathbf{t}=(\boldsymbol{\nabla}\mathbf{t})\mathbb{P}.$
(37)

In addition, the surface divergence of a smooth vector field
$\mathbf{t}(\mathbf{X})$ and a smooth second order tensor field
$\mathbf{T}(\mathbf{X})$ are

$\text{div}_{s}(\mathbf{t})=\text{tr}(\boldsymbol{\nabla}_{s}\mathbf{t})\quad\text{and}\quad\mathbf{v}\cdot\text{div}_{s}(\mathbf{T})=\text{div}_{s}(\mathbf{T}^{T}\mathbf{v}),$
(38)

with $\mathbf{v}\in\mathbb{R}^{3}$ being an arbitrary constant vector. In a
more general case, we have

$\mathbf{N}\cdot\text{div}_{s}(\mathbf{T})=\text{div}_{s}(\mathbf{T}^{T}\mathbf{N})-\mathbf{T}\cdot\boldsymbol{\nabla}_{s}\mathbf{N}.$
(39)

##  Appendix B Equivalence between linear elasticity and Stokes flow of
Newtonian fluids

The static equilibrium of an elastic body and steady Stokes flows are
described by equivalent governing equations. Assuming linearized kinematics,
strain tensor $\boldsymbol{\epsilon}$ and displacements $\mathbf{u}$ are
related as

$\boldsymbol{\epsilon}=\frac{1}{2}\left(\boldsymbol{\nabla}\mathbf{u}+\boldsymbol{\nabla}\mathbf{u}^{T}\right).$
(40)

For linear elastic material behavior, stresses follow as

$\boldsymbol{\sigma}=2\mu\boldsymbol{\epsilon}+\left(\kappa-\frac{2}{3}\mu\right)\text{tr}(\boldsymbol{\epsilon})\mathbf{I},$
(41)

where $\mu$ is the shear modulus and $\kappa$ is the bulk modulus. Insertion
into the momentum balance in combination with the condition of
incompressibility $\text{tr}(\boldsymbol{\epsilon})=0$ gives

$-\boldsymbol{\nabla}p+\mu\boldsymbol{\nabla}^{2}\mathbf{u}=\mathbf{0},$ (42)

where $p$ is the hydrostatic pressure term
$p=-\text{tr}(\boldsymbol{\sigma})$.

For Newtonian fluids, similar to linear elasticity, viscous stresses arising
from the flow field are linearly related to the local rate-of-strain-tensor

$\mathbf{D}=\frac{1}{2}\left(\boldsymbol{\nabla}\mathbf{v}+\boldsymbol{\nabla}\mathbf{v}^{T}\right),$
(43)

such that

$\boldsymbol{\sigma}=2\eta\mathbf{D}+\left(\zeta-\frac{2}{3}\eta\right)\text{tr}(\mathbf{D})\mathbf{I},$
(44)

where $\zeta$ is the bulk viscosity and $\eta$ is the shear viscosity.
Insertion into the momentum balance in combination with an incompressible,
divergence-free flow field $\text{div}(\mathbf{v})=0$ again gives Equation
(42), using the substitutions $\eta=\mu$ and $\mathbf{v}=\mathbf{u}$. This
illustrates the equivalence of linear elastic statics and steady Stokes flow
of Newtonian fluids. Solutions to Stoke’s flow in spherical geometries
incorporating elastic interfaces are well developed [3, 2]. Here, we adopt
them for the solution of the volume preserving elastic problem as outlined
below.

##  Appendix C Fundamental set of solutions of the Stokes equations

The velocity basis functions are

$\displaystyle{\bf v}^{-}_{jm0}$
$\displaystyle={\textstyle\frac{1}{2}}r^{-j}\left(2-j+jr^{-2}\right){\bf
y}_{jm0}+{\textstyle\frac{1}{2}}r^{-j}\left[j\left(j+1\right)\right]^{\frac{1}{2}}\left(1-r^{-2}\right){\bf
y}_{jm2}\,,$ (45a) $\displaystyle{\bf v}^{-}_{jm1}$ $\displaystyle=\textstyle
r^{(-j-1)}{\bf y}_{jm1}\,,$ (45b) $\displaystyle{\bf v}^{-}_{jm2}$
$\displaystyle={\textstyle\frac{1}{2}}r^{-j}\left(2-j\right)(\textstyle\frac{j}{j+1})^{\frac{1}{2}}\left(1-r^{-2}\right){\bf
y}_{jm0}+{\textstyle\frac{1}{2}}r^{-j}\left(j+(2-j)r^{-2}\right){\bf
y}_{jm2}\,,$ (45c) $\displaystyle{\bf v}^{+}_{jm0}$
$\displaystyle={\textstyle\frac{1}{2}}r^{j-1}\left(-(j+1)+(j+3)r^{2}\right){\bf
y}_{jm0}-{\textstyle\frac{1}{2}}r^{j-1}\left[j\left(j+1\right)\right]^{\frac{1}{2}}\left(1-r^{2}\right){\bf
y}_{jm2}\,,$ (46a) $\displaystyle{\bf v}^{+}_{jm1}$ $\displaystyle=\textstyle
r^{j}{\bf y}_{jm1}\,,$ (46b) $\displaystyle{\bf v}^{+}_{jm2}$
$\displaystyle={\textstyle\frac{1}{2}}r^{j-1}\left(3+j\right)(\textstyle\frac{j+1}{j})^{\frac{1}{2}}\left(1-r^{2}\right){\bf
y}_{jm0}+{\textstyle\frac{1}{2}}r^{j-1}\left(j+3-(j+1)r^{2}\right){\bf
y}_{jm2}\,.$ (46c)

On a sphere $r=1$ these velocity fields reduce to the vector spherical
harmonics defined by 52

${\bf v}^{\pm}_{jmq}={\bf{y}}_{jmq}\,.$ (47)

The tractions on a sphere due to the displacement fields 45 and 46 are

$\tau^{+}_{jm0}=(2j+1)c^{+}_{jm0}-3\left(\frac{j+1}{j}\right)^{\frac{1}{2}}c^{+}_{jm2}\,,$
(48a)
$\tau^{-}_{jm0}=-(2j+1)c^{-}_{jm0}+3\left(\frac{j}{j+1}\right)^{\frac{1}{2}}c^{-}_{jm2}\,$
(48b)

$\tau^{-}_{jm2}=3\left(\frac{j}{j+1}\right)^{\frac{1}{2}}c^{-}_{jm0}-\frac{4+3j+2j^{2}}{j+1}c^{-}_{jm2}\,,$
(49a)
$\tau^{+}_{jm2}=-3\left(\frac{j+1}{j}\right)^{\frac{1}{2}}c^{+}_{jm0}+\frac{3+j+2j^{2}}{j}c^{+}_{jm2}\,$
(49b)

$\begin{split}\tau^{-}_{jm1}=-(j+2)c^{-}_{jm1}\,,\quad\tau^{+}_{jm1}=(j-1)c^{+}_{jm1}\,,\end{split}$
(50)

##  Appendix D Spherical harmonics

An extensive reference on spherical harmonics is a handbook by Varshalovich et
al. [38]. Their properties in relation to the problems of particle
microhydrodynamics are summarized in Blawzdziewicz et al. [4].

The normalized spherical scalar harmonics are defined as

$Y_{jm}\left(\theta,\varphi\right)=\textstyle\left[\frac{2j+1}{4\pi}\frac{(j-m)!}{(j+m)!}\right]^{\frac{1}{2}}(-1)^{m}P_{j}^{m}(\cos\theta)e^{{\rm
i}m\varphi},$ (51)

where ${\bf\hat{r}}={\bf r}/r$, $(r,\theta,\varphi)$ are the spherical
coordinates, and $P_{j}^{m}(\cos\theta)$ are the Legendre polynomials. The
vector spherical harmonics are defined as

$\begin{split}{\bf{y}}_{jm0}&=\left[j\left(j+1\right)\right]^{-{\frac{1}{2}}}r\nabla_{\Omega}Y_{jm}\,,\quad{\bf{y}}_{jm2}={\bf\hat{r}}Y_{jm}\,,\quad{\bf{y}}_{jm1}=-{\mathrm{i}}{\bf\hat{r}}\times{\bf{y}}_{jm0}\end{split}$
(52)

where $\nabla_{\Omega}$ denotes the angular part of the gradient operator. In
spherical coordinates, the vector spherical harmonics that are tangential to a
sphere are

$\begin{split}{\bf{y}}_{jm0}&=\frac{1}{\sqrt{j(j+1)}}\frac{\partial
Y_{jm}}{\partial\theta}{\bm{\hat{\theta}}}+\frac{{\mathrm{i}}m}{\sqrt{j(j+1)}}\frac{Y_{jm}}{\sin\theta}{\bm{\hat{\varphi}}}\\\
{\bf{y}}_{jm1}&=-\frac{m}{\sqrt{j(j+1)}}\frac{Y_{jm}}{\sin\theta}{\bm{\hat{\theta}}}-\frac{{\mathrm{i}}}{\sqrt{j(j+1)}}\frac{\partial
Y_{jm}}{\partial\theta}{\bm{\hat{\varphi}}}\\\ \end{split}$ (53)

For example

${\bf{y}}_{200}=-\textstyle\sqrt{\frac{15}{32\pi}}\sin(2\theta){\bf{\hat{e}}}_{\theta}\,,\quad{\bf{y}}_{101}=\textstyle-{\mathrm{i}}\sqrt{\frac{3}{8\pi}}\sin\theta{\bf{\hat{e}}}_{\varphi}\,,\quad{\bf{y}}_{202}=\textstyle\frac{1}{8}\sqrt{\frac{5}{\pi}}[1+3\cos(2\theta)]{\bf\hat{r}}\,.$
(54)

##  Appendix E Interfacial displacement field

The interfacial displacement field is given as

$\displaystyle\mathbf{u}$
$\displaystyle=\frac{5e^{-2i\varphi}ER_{0}^{2}(2ER_{0}+9\kappa^{s}+3\mu^{s})(e^{2i\varphi}c_{z}+3e^{2i\varphi}c_{z}\cos{2\theta}+(1+e^{4i\varphi})(c_{x}-c_{y})\sin{\theta}^{2})}{12\left(2E^{2}R_{0}^{2}+12\mu^{s}\kappa^{s}+6\Upsilon^{0}(3\kappa^{s}+2\mu^{s})+ER_{0}(5\Upsilon^{0}+8\kappa^{s}+6\mu^{s})\right)}\mathbf{g}_{r}$
$\displaystyle-\frac{5ER_{0}^{2}(2ER_{0}+3\Upsilon^{0}+6\kappa^{s})(3c_{z}+(c_{y}-c_{x})\cos{2\varphi})\sin{2\theta}}{12\left(2E^{2}R_{0}^{2}+12\mu^{s}\kappa^{s}+6\Upsilon^{0}(3\kappa^{s}+2\mu^{s})+ER_{0}(5\Upsilon^{0}+8\kappa^{s}+6\mu^{s})\right)}\mathbf{g}_{\theta}$
$\displaystyle-\frac{5ER_{0}^{2}(c_{x}-c_{y})(2ER_{0}+3\Upsilon^{0}+6\kappa^{s})\sin{\theta}\sin{2\varphi}}{6\left(2E^{2}R_{0}^{2}+12\mu^{s}\kappa^{s}+6\Upsilon^{0}(3\kappa^{s}+2\mu^{s})+ER_{0}(5\Upsilon^{0}+8\kappa^{s}+6\mu^{s})\right)}\mathbf{g}_{\varphi}.$
(55)

##  Appendix F Fitting sensitivities

Figures 8 and 9 show obtained spectra for different cases of parameter ranges
(blue) in comparison to the spectrum obtained from surface parameters on the
same order of magnitude (yellow). As shown, both $\Upsilon^{0}$ and $\mu^{s}$
may be reliably recovered, whereby we find a decrease in the width of spectra
in all cases.

$\Upsilon^{0}\ll(\mu^{0},\kappa^{0})$$\Upsilon^{0}=1\,\text{mN/m}$$\mu^{0}=10\,\text{mN/m}$$\kappa^{0}=20\,\text{mN/m}$

$(\mu^{0},\kappa^{0})\ll\Upsilon^{0}$$\Upsilon^{0}=10\,\text{mN/m}$$\mu^{0}=1\,\text{mN/m}$$\kappa^{0}=2\,\text{mN/m}$

$\mu^{0}\ll(\Upsilon^{0},\kappa^{0})$$\Upsilon^{0}=10\,\text{mN/m}$$\mu^{0}=1\,\text{mN/m}$$\kappa^{0}=20\,\text{mN/m}$

$(\Upsilon^{0},\mu^{0})\ll\kappa^{0}$$\Upsilon^{0}=1\,\text{mN/m}$$\mu^{0}=1\,\text{mN/m}$$\kappa^{0}=20\,\text{mN/m}$

Figure 8: Fitting sensitivities of $\Upsilon^{0}$ for varying parameter ranges
(blue) in comparison to results for $\Upsilon^{0}\sim(\mu^{s},\kappa^{s})$
(yellow), both at high levels of experimental error $\hat{\sigma}\sim 10\%$.

$\Upsilon^{0}\ll(\mu^{0},\kappa^{0})$$\Upsilon^{0}=1\,\text{mN/m}$$\mu^{0}=10\,\text{mN/m}$$\kappa^{0}=20\,\text{mN/m}$

$(\mu^{0},\kappa^{0})\ll\Upsilon^{0}$$\Upsilon^{0}=10\,\text{mN/m}$$\mu^{0}=1\,\text{mN/m}$$\kappa^{0}=2\,\text{mN/m}$

$\mu^{0}\ll(\Upsilon^{0},\kappa^{0})$$\Upsilon^{0}=10\,\text{mN/m}$$\mu^{0}=1\,\text{mN/m}$$\kappa^{0}=20\,\text{mN/m}$

$(\Upsilon^{0},\mu^{0})\ll\kappa^{0}$$\Upsilon^{0}=1\,\text{mN/m}$$\mu^{0}=1\,\text{mN/m}$$\kappa^{0}=20\,\text{mN/m}$

Figure 9: Fitting sensitivities of $\mu^{s}$ for varying parameter ranges
(blue) in comparison to results for $\Upsilon^{0}\sim(\mu^{s},\kappa^{s})$
(yellow), both at high levels of experimental error $\hat{\sigma}\sim 10\%$.

## References

  * Bain et al. [2021] Bain, N., Smith-Mannschott, K., Heyden, S., Style, R., Dufresne, E., 2021.  Surface tension and the strain-dependent topography of soft solids.  PRL , in press.
  * Barthès and Sgaier [1985] Barthès, D., Sgaier, H., 1985\.  Role of membrane viscosity in the orientation and deformation of a spherical capsule suspended in shear flow.  Journal of Fluid Mechanics 160, 119 – 135.
  * Barthès-Biesel and Rallison [1981] Barthès-Biesel, D., Rallison, J., 1981\.  The time-dependent deformation of a capsule freely suspended in a linear shear flow.  Journal of Fluid Mechanics 113, 251 – 267.
  * Blawzdziewicz et al. [2000] Blawzdziewicz, J., Vlahovska, P., Loewenberg, M., 2000.  Rheology of a dilute emulsion of surfactant-covered spherical drops.  Physica A: Statistical Mechanics and its Applications 85, 121010–1.
  * Bostwick et al. [2014] Bostwick, J., Shearer, M., Daniels, K., 2014.  Elastocapillary deformations on partially-wetting substrates: rival contact-line models.  Soft Matter 37, 7361–7369.
  * Dai et al. [2018] Dai, M., Li, M., Shiavone, P., 2018.  Plane deformations of an inhomogeneity–matrix system incorporating a compressible liquid inhomogeneity and complete gurtin–murdoch interface model.  Journal of applied mechanics 85, 121010–1.
  * Dervaux and Limat [2015] Dervaux, J., Limat, L., 2015\.  Contact lines on soft solids with uniform surface tension: analytical solutions and double transition for increasing deformability.  Proc. R. Soc. Ai 471(2176), 20140813\.
  * Duan et al. [2005] Duan, H., Wang, J., Huang, Z., Karihaloo, B., 2005.  Eshelby formalism for nano-inhomogeneities.  Proc R Soc A 461, 3335–3353.
  * Gurtin and Murdoch [1975] Gurtin, M., Murdoch, A., 1975\.  A continuum theory of elastic material surfaces.  Archive for Rational Mechanics and Analysis 57, 291–323.
  * Gurtin et al. [1998] Gurtin, M., Weissmueller, J., Larché, F., 1998.  A general theory of curved deformable interfaces in solids at equilibrium.  Philosophical Magazine A 78, 1093–1109.
  * Heyden et al. [2021] Heyden, S., Bain, N., Xu, Q., Style, R., Dufresne, E.R., 2021\.  Contact lines on stretched soft solids: modelling anisotropic surface stresses.  Proceedings of the Royal Society A 477, 20200673.
  * Huang and Wang [2013] Huang, Z., Wang, J., 2013.  Micromechanics of Nanocomposites with Interface Energy Effect. Springer.  pp. 51–59.
  * Jagota et al. [2012] Jagota, A., Paretkar, D., Ghatak, A., 2012.  Surface-tension-induced flattening of a nearly plane elastic solid.  Phys Rev E 85, 051602\.
  * Jensen et al. [2017] Jensen, K.E., Style, R.W., Xu, Q., Dufresne, E.R., 2017\.  Strain-dependent solid surface stress and the stiffness of soft contacts.  Phys. Rev. X 7, 041031\.
  * Jerison et al. [2011] Jerison, E., Xu, Y., Wilen, L., Dufresne, E., 2011\.  Deformation of an elastic substrate by a three-phase contact line.  PRL 106, 186103\.
  * Kochmann [2012] Kochmann, D., 2012.  Stability criteria for continuous and discrete elastic composites and the influence of geometry on the stability of a negative-stiffness phase.  Physica Status Solidi B , 1–13.
  * Krichen et al. [2019] Krichen, S., Liu, L., Sharma, P., 2019.  Liquid inclusions in soft materials: Capillary effect, mechanical stiffening and enhanced electromechanical response.  JMPS 127, 332–357.
  * Liddell et al. [1843] Liddell, H., Scott, R., Jones, H., 1843.  Greek-English Lexicon.  Oxford University Press, United Kingdom.
  * Lubarda [2013] Lubarda, V., 2013.  Circular loads on the surface of a half-space: Displacement and stress discontinuities under the load.  International Journal of Solids and Structures 50, 1–14.
  * Masurel et al. [2019] Masurel, R., Roché, M., Limat, L., Ionescu, I., Dervaux, J., 2019.  Elastocapillary ridge as a noninteger disclination.  Phys. Rev. Lett. 122, 248004\.
  * Mora et al. [2010] Mora, S., Phou, T., Fromental, J., Pismen, L., Pomeau, Y., 2010.  Capillarity driven instability of a soft solid.  PRL 105, 214301\.
  * Mozaffari et al. [2020] Mozaffari, K., Yang, S., Sharma, P., 2020.  Surface Energy and Nanoscale Mechanics. Springer International Publishing. chapter 0.  pp. 1949–1974.
  * Pandey et al. [2020] Pandey, A., Andreotti, B., Karpischka, S., van Zwieten, G., van Brummelen, E., Snoeijer, J., 2020\.  Singular nature of the elastocapillary ridge.  Phys Rev X 10, 031067\.
  * Plesset [1949] Plesset, M., 1949.  The dynamics of cavitation bubbles.  J. Appl. Mech. 16, 228–231.
  * Pockels [1891] Pockels, A., 1891.  Surface tension.  Nature 43, 437–439.
  * Pockels [1898] Pockels, A., 1898.  Beobachtungen über die adhäsion verschiedener flüssigkeiten an glas.  Naturwissenschaftliche Rundschau 14\.
  * Pockels [1899] Pockels, A., 1899.  Untersuchungen von grenzflächenspannungen mit der cohäsionswaage.  Annalen der Physik 67, 437–439.
  * Rosowski et al. [2020] Rosowski, K., Sai, T., Vidal-Henriquez, E., Zwicker, D., Style, R., Dufresne, E., 2020\.  Elastic ripening and inhibition of liquid–liquid phase separation.  Nature Physics 16, 422–425.
  * Sauer [2014] Sauer, R., 2014.  A contact theory for surface tension driven systems.  Mathematics and Mechanics of Solids 21, 305–325.
  * Shuttleworth [1950] Shuttleworth, R., 1950.  The surface tension of solids.  Proc Royal Soc A 63.
  * Strutt [1916] Strutt, J., 1916.  On the theory of the capillary tube.  Proc Royal Soc A 92.
  * Strutt [1917] Strutt, J., 1917.  On the pressure developed in a liquid during the collapse of a spherical cavity.  Phil. Mag. 34.
  * Style et al. [2015] Style, R., Boltyanskiy, R., Allen, B., Jensen, K., Foote, H., Wettlaufer, J., Dufresne, E., 2015.  Stiffening solids with liquid inclusions.  Nature Physics 11, 82–87.
  * Style and Dufresne [2012] Style, R., Dufresne, E., 2012\.  Static wetting on deformable substrates, from liquids to soft solids.  Soft Matter 27, 7177–7184.
  * Style et al. [2017] Style, R., Jagota, A., Hui, C.Y., Dufresne, E., 2017\.  Elastocapillarity: Surface tension and the mechanics of soft solids.  Annual Review of Condensed Matter Physics 8, 99–118.
  * Style and Xu [2018] Style, R., Xu, Q., 2018.  The mechanical equilibrium of soft solids with surface elasticity.  Soft Matter 14, 4569–4576.
  * Suñé and Wettlaufer [2021] Suñé, M., Wettlaufer, J.S., 2021\.  Capillary control of collapse in soft composite columns.  Phys. Rev. Materials 5, 055603\.
  * Varshalovich et al. [1988] Varshalovich, D., Moskalev, A., Khersonskii, V., 1988.  Quantum Theory of Angular Momentum.  World Scientific Pub, Singapur.
  * Vlahovska [2016] Vlahovska, P., 2016.  Dynamics of membrane-bound particles: Capsules and vesicles. 4 ed.. Royal Society of Chemistry. chapter 9.  Number 4 in RSC Soft Matter, pp. 313–346.
  * Vlahovska et al. [2005] Vlahovska, P., Loewenberg, M., Blawzdziewicz, J., 2005.  Deformation of a surfactant-covered drop in a linear flow.  Physics of Fluids 17, 103103\.
  * Xu et al. [2017] Xu, Q., Jensen, K., Boltyanskiy, R., Safarti, R., Style, R., Dufresne, E., 2017\.  Direct measurement of strain-dependent solid surface stress.  Nature Communications 555(8), 2041–1723.
  * Xu et al. [2018] Xu, Q., Style, R., Dufresne, E., 2018.  Surface elastic constants of a soft solid.  Soft Matter 14(916), 916–920.
  * Zafar and Basu [2021] Zafar, M., Basu, S., 2021.  Calibrating surface hyperelastic constitutive models in soft solids.  Physical Review E 103, 063003\.


# Label-Aware Distribution Calibration for Long-tailed Classification

Chaozheng Wang1,Shuzheng Gao1,Pengyun Wang2, Cuiyun Gao1,
Wenjie Pei1,Lujia Pan2,Zenglin Xu1 Corresponding author

###### Abstract

> Real-world data usually present long-tailed distributions. Training on
> imbalanced data tends to render neural networks perform well on head classes
> while much worse on tail classes. The severe sparseness of training
> instances for the tail classes is the main challenge, which results in
> biased distribution estimation during training. Plenty of efforts have been
> devoted to ameliorating the challenge, including data re-sampling and
> synthesizing new training instances for tail classes. However, no prior
> research has exploited the transferable knowledge from head classes to tail
> classes for calibrating the distribution of tail classes. In this paper, we
> suppose that tail classes can be enriched by similar head classes and
> propose a novel distribution calibration approach named as Label-Aware
> Distribution Calibration (LADC). LADC transfers the statistics from relevant
> head classes to infer the distribution of tail classes. Sampling from
> calibrated distribution further facilitates re-balancing the classifier.
> Experiments on both image and text long-tailed datasets demonstrate that
> LADC significantly outperforms existing methods. The visualization also
> shows that LADC provides a more accurate distribution estimation.

## Introduction

Figure 1: Visualization for the cosine similarities between the means (a) or
variances (b) of classes in the feature space of CIFAR10-LT. The feature
vectors are computed by the ResNet-32 backbone. The first four classes
including $airplane$, $automobile$, $bird$, and $cat$ are head classes, while
the others are tail classes. The blue rectangles highlight the high
similarities between some head classes and tail classes, such as $cat$ and
$dog$, and $automobile$ and $truck$. Figure 2: Overview of our proposed LADC
method. In Stage 1, we train the backbone with Representation Enhancement. In
Stage 2, we freeze the backbone and modify the classifier by re-sampling with
distribution calibration and LWS-Plus training.

Classification tasks such as image classification and natural language
classification are fundamental and essential for evaluating the learning
ability of neural networks. Generally, the neural networks are trained and
evaluated on balanced datasets including CIFAR and ImageNet (Deng et al.
2009). However, real-word datasets are imbalanced and usually present long-
tailed distributions, that is, head classes contain more training instances
while tail classes are with significantly fewer instances. Models trained on
long-tailed data will result in obvious performance degradation, e.g., much
worse performance for the tail classes compared with the performance for the
head classes. The main challenge lies in the sparsity of tail classes, leading
to estimation of the decision boundaries severely biased to head classes.

Re-weighting methods (Cui et al. 2019; Jamal et al. 2020; Cao et al. 2019) are
one popular solution to tackle the challenge. For example, the work (Cui et
al. 2019) re-design training loss functions to highlight the contribution of
tail classes. Recently, Cao et al. (2019) find that traditional end-to-end re-
weighting methods present non-promising results. So they propose to train the
model in a normal way in the first stage and tune the model with deferred re-
sampling (DRS) and deferred re-weighting (DRW) after annealing the learning
rate. Kang et al. (2019) follow the two-stage training process, and decouple
the learning of backbone and classifier to mitigate the impact of imbalanced
data on biasing the classifier. Specifically, for re-balancing the decision
boundary, they re-train the classifier (cRT) and learn scaling weights for
classifier (LWS) via class-balanced sampling in the second training stage.
Zhong et al. (2021) later propose label-aware smoothing to regularize the
classifier. Although the prior studies make some achievements, they have not
exploited the inter-class similarity, especially the transferable knowledge
from head classes to tail classes.

There are also some studies (Chawla et al. 2002; Li et al. 2021; Kim, Jeong,
and Shin 2020) focusing on synthesizing new training instances for tail
classes to solve the data sparsity issue. For example, M2m (Kim, Jeong, and
Shin 2020) generates minority samples by translating majority samples.
MetaSAug (Li et al. 2021) produces new training instances via implicit
semantic data augmentation (Wang et al. 2019). However, these studies pay more
attention to the surface features, without explicitly exploring the in-depth
feature space.

Inspired by the distribution calibration method (DC) in the few-shot learning
(Yang, Liu, and Xu 2021), which estimates the in-depth feature distribution of
support set by utilizing the knowledge from base set, we propose a novel
label-aware distribution calibration method (LADC) for our long-tailed
scenarios. LADC assumes that the distribution of tail classes can be enriched
by similar head classes in the feature space. Following the DC method, LADC
also assumes that every dimension in the feature space follows a Gaussian
distribution, and similar classes have similar means and variances of the
feature representations. The proposal of LADC is based on the observation that
there exists some similarity between head classes and tail classes. As shown
in Fig. 1, we can observe that the statistics of the head class $automobile$
are highly relevant to those of the tail class $truck$. So the feature
distribution of the $truck$ class can be better calibrated by virtue of the
$automobile$ class which contains sufficient instances for training.

Specifically, LADC decouples the training process into two stages. In the
first training stage, the backbone is trained with data augmentation
techniques including Re-balanced Mixup (Chou et al. 2020) and RandAugment
(Cubuk et al. 2020) to enhance the representation learning. Based on the
trained backbone, LADC computes and records the mean feature and co-variance
matrix of each head class. In the second training stage, LADC selects the most
similar $m$ head classes for computing new mean and co-variance of calibrated
distribution for every instance in tail classes. Finally, new training
instances sampled from the calibrated distributions are employed to rectify
the classifier in the first stage. In addition, we propose LWS-Plus to further
improve the classifier rectification ability. In summary, our contributions
are as follow:

  1. 1.

We are the first to exploit the transferable knowledge from head classes to
tail classes for estimating the feature distributions of tail classes.

  2. 2.

We propose a novel method LADC for calibrating the distribution of tail
classes by borrowing knowledge from head classes. A new sampling strategy is
also proposed based on the calibrated distribution to better balance the
classifier.

  3. 3.

Extensive experiments demonstrate that LADC can significantly outperform
state-of-the-art approaches on long-tailed image datasets including
CIFAR10-LT, CIFAR100-LT, ImageNet-LT and iNaturalist. Experimentation on long-
tailed text classification task indicates that LADC is generalizable to other
long-tailed modals.

## Methodology

In this section, we elaborate on the proposed method LADC. LADC follows a two-
stage training framework, in which a backbone is first trained on the original
long-tailed dataset and fixed on the second stage. Fig. 2 depicts the overview
of the proposed LADC. Equipped with the backbone and the associated
representation space, we illustrate how LADC estimates the distribution of
tail classes by utilizing the statistics of relevant head classes. Then we
introduce the sampling strategy based on the calibrated distribution. We
finally describe how LADC enhances the training of backbone and classifiers.

### Problem Definition

We follow a typical long-tailed classification setting. Given an imbalanced
dataset $D=\\{\bm{x}_{i},y_{i}\\}^{N}_{i=1}$ with $N$ training instances,
where $\bm{x}_{i}$ denotes a instance and $y_{i}$ denotes its label. Assume
that the classes are ordered by cardinality, i.e., when class index $p<q$,
then $n_{p}\geq n_{q}$, where $n_{p}$ is the number of training instances of
the class $p$. Let $\mathcal{F}$ be the backbone and $\Phi$ be the classifier.
Let $\bm{z}^{c}$ represent the class embedding of class $c$ in the feature
space defined by $\mathcal{F}$. Furthermore, we split classes into two groups
with head classes denoted as $C_{h}$ and tail classes as $C_{t}$. Note that we
simplify the notations by dropping class index wherever possible.

### Label-Aware Distribution Calibration

The DC method (Yang, Liu, and Xu 2021) for few shot learning focuses on
$N$-way $K$-shot task where $N$ is fixed. Since long-tailed scenarios require
models to perform well on both head and tail classes, directly adopting DC is
non-optimal. To improve the prediction accuracy of tail classes and preserve
the performance on head classes, we propose a novel label-aware distribution
calibration approach.

#### Calibrating distribution of tail classes

Given the trained backbone, LADC calibrates the distributions of tail classes
with the help of relevant head classes. We assume that each class is Gaussian
distributed and head classes are well represented with sufficient data for
training. So the distribution of each head class can be approximated by
calculating the mean and co-variance of the associated training data:

$p(\bm{z})=N(\bm{z}|\bm{\mu},\bm{\Sigma})$ (1)
$\bm{\mu}=\frac{\sum_{i=1}^{n}\mathcal{F}(\bm{x}_{i})}{n}$ (2)
$\bm{\Sigma}=\frac{1}{n-1}\sum_{i=1}^{n}(\mathcal{F}(\bm{x}_{i})-\bm{\mu})(\mathcal{F}(\bm{x}_{i})-\bm{\mu})^{T}$
(3)

where $\bm{\mu}$ and $\bm{\Sigma}$ denote the mean and variance of the class,
and $n$ denotes the number of training instances in the head class.

To more accurately estimate the distribution of each tail class, we employ the
statistics of a set of similar head classes $S$ as the prior and compute the
posterior by using the feature $\bm{\hat{\mu}}$ of the tail class. The similar
head classes $S$ is measured by the Euclidean distance in the feature space:

$Set^{d}=\\{-||\bm{\mu}^{i}-\hat{\bm{\mu}}||^{2}|i\in C_{h}\\}\\\ $ (4)
$S=\\{i|-||\bm{\mu}^{i}-\hat{\bm{\mu}}||^{2}\in topm(Set^{d})\\}$ (5)

Equipped with $S$, We can compute the prior distribution of $\bm{\hat{\mu}}$:

$\bm{w}_{i}=\frac{n_{i}||\bm{\mu}^{i}-\hat{\bm{\mu}}||^{2}}{\sum_{j\in\mathbb{S}}n_{j}||\bm{\mu}^{j}-\hat{\bm{\mu}}||^{2}}$
(6)

$\bm{\mu}_{0}=\sum_{i\in
S}\bm{w}_{i}\bm{\mu}^{i}\quad\bm{\Sigma}_{0}=\sum_{i\in
S}(\bm{w}_{i})^{2}\bm{\Sigma}^{i}+\alpha\bm{I}$ (7)

$p(\bm{z})=N(\bm{z}|\bm{\mu}_{0},\bm{\Sigma}_{0})$ (8)

where $\alpha$ is a hyper-parameter to control the degree of dispersion and
$\bm{w}$ is the weight vector which is designed to impose tail classes to
learn more from head classes that are more abundant and more similar.

For each tail class, $\mathcal{F}(\bm{x})$ conditioned on $\bm{z}$ is assumed
to follow a Gaussian distribution

$p(\mathcal{F}(\bm{x})|\bm{z})=N(\mathcal{F}(\bm{x})|\bm{z},\bm{L})$ (9)

where $\bm{L}$ is the co-variance matrix and it is related to the uncertainty
of the trained backbone.

Based on the prior distribution, the posterior (calibrated) distribution of
each instance in the tail class can be given by

$p(\bm{z}|\\{\mathcal{F}(\bm{x_{i}})\\}_{i})=N(\bm{z}|\bm{\mu}^{\prime},\bm{\Sigma}^{\prime})$
(10)

By defining $\bm{L}=\beta\bm{\Sigma}_{0}$, we compute $\bm{\mu}^{\prime}$ and
$\bm{\Sigma}^{\prime}$ as follows:

$\bm{\mu}^{\prime}=\frac{\beta}{n_{s}+\beta}\bm{\mu}_{0}+\frac{n_{s}}{n_{s}+\beta}\hat{\bm{\mu}}$
(11)

$\bm{\Sigma}^{\prime}=\frac{\beta}{n_{s}+\beta}\bm{\Sigma}_{0}$ (12)

where $n_{s}$ means the sample size for computing $\hat{\bm{\mu}}$ and is set
as 1. $\beta$ is a hyper-parameter which can be interpreted as the relative
uncertainty of the sample to the prior $\bm{\mu}_{0}$. Higher $\beta$
indicates a higher confidence of the prior.

#### Sampling strategy

In the second training stage, to balance the long-tailed distribution, we
define the sampling probability of each class to be:

$P_{i}=\frac{n_{1}/n_{i}^{\tau}}{\sum_{j}n_{1}/n_{j}^{\tau}}$ (13)

where $n_{1}$ refers to the number of training instances of the most
represented class. $\tau$ is a temperature parameter and it controls the
balanced degree of the sampled distribution. Note that we re-sampling head
classes from the original distributions; while for the tail classes, we draw
samples from the calibrated distributions.

Table 1: Overview of datasets used in our experiments. The “IF” denotes
imbalance factor.

Datasets | #Class | IF | #Train Set | Min. Class Size | Max. Class Size | #Test Set
---|---|---|---|---|---|---
CIFAR10-LT | 10 | 10$\sim$200 | 50,000$\sim$11,203 | 500$\sim$25 | 5,000 | 10,000
CIFAR100-LT | 100 | 10$\sim$200 | 50,000$\sim$9,502 | 500$\sim$2 | 5,00 | 10,000
ImageNet-LT | 1000 | 256 | 115,846 | 5 | 1280 | 50,000
iNaturalist2018 | 8142 | 500 | 437,513 | 2 | 1,000 | 24,426
THUNEWS-LT | 10 | 100 | 44,723 | 180 | 18,000 | 10,000

#### Re-balancing classifier

In Stage 2, Kang et al. (2019) introduce two methods for classifier
adjustment: cRT and LWS. cRT re-trains the classifiers completely, while LWS
retains the direction of the classifiers and only adjusts the scales. In the
work, we choose LWS due to its superior performance in our experiments. On the
basis of LWS and inspired by logits adjustment work (Menon et al. 2020; Tang,
Huang, and Zhang 2020), we add a learnable bias term to the logits, named LWS-
Plus, for adjusting the classification boundaries more flexibly:

$\hat{\Phi}_{i}(\bm{x})=\bm{f}_{i}\cdot\Phi_{i}(\bm{x})+\bm{g}_{i}$ (14)

where $\Phi_{i}(\bm{x})$ refers to the logit score of the class $i$,
$\bm{f}_{i}$ denotes the scaling factor of the classifier magnitude, and
$\bm{g}_{i}$ indicates the added bias.

Table 2: Top-1 accuracy (%) on CIFAR10-LT and CIFAR100-LT. Best results are marked in bold. Approach | CIFAR10-LT | CIFAR100-LT
---|---|---
IF | 200 | 100 | 50 | 10 | 200 | 100 | 50 | 10
Cross-entropy training | 65.87 | 70.14 | 74.94 | 86.18 | 34.70 | 37.92 | 44.02 | 55.73
Class Balance Loss | 68.77 | 72.68 | 78.13 | 86.90 | 35.56 | 38.77 | 44.79 | 57.57
Focal Loss | 65.29 | 70.38 | 76.71 | 86.68 | 35.62 | 38.41 | 44.32 | 55.78
CB Focal Loss | 68.15 | 74.57 | 79.22 | 87.48 | 36.23 | 39.60 | 45.21 | 57.99
LDAM Loss | 66.75 | 73.55 | 78.83 | 87.32 | 36.53 | 40.60 | 46.16 | 57.29
LDAM-DRW | 74.74 | 77.03 | 81.03 | 88.16 | 38.45 | 42.04 | 46.62 | 58.71
MCW with LDAM loss | 77.23 | 80.00 | 82.23 | 87.40 | 39.53 | 44.08 | 49.16 | 58.00
Decoufound-TDE | - | 80.60 | 83.60 | 88.50 | - | 44.10 | 50.30 | 59.60
cRT | 69.48 | 73.02 | 79.56 | 87.90 | 38.16 | 43.30 | 47.37 | 57.86
LWS | 67.90 | 72.44 | 77.77 | 87.41 | 37.52 | 42.97 | 47.40 | 58.08
MiSLAS | 76.73 | 82.10 | 85.70 | 90.00 | 43.53 | 47.00 | 52.30 | 63.20
BBN | - | 79.82 | 81.18 | 88.32 | - | 42.56 | 47.02 | 59.12
LDAM-DRW + SSP | - | 77.83 | 82.13 | 88.53 | - | 43.43 | 47.11 | 58.91
Bag of tricks | - | 80.03 | 83.59 | - | - | 47.83 | 51.69 | -
SMOTE | - | 71.50 | - | 85.70 | - | 34.00 | - | 53.80
Major-to-Minor | - | 79.10 | - | 87.50 | - | 43.50 | - | 57.60
MetaSAug-LDAM | 77.35 | 80.66 | 84.34 | 89.68 | 43.09 | 48.08 | 52.27 | 61.28
LADC | 81.56 | 84.65 | 87.09 | 90.81 | 46.62 | 50.77 | 54.94 | 64.66

#### Representation Enhancement

In Stage 1, Mixup has proven to be an effective regularization strategy for
training deep neural networks (Zhong et al. 2021; Zhang et al. 2021). More
recently, Remix has been proposed for imbalanced data (Chou et al. 2020). In
the work, we adopt Remix in the first training stage due to its better
performance in the experiments. In addition, we combine RandAugment (Cubuk et
al. 2020), a two-stage augmentation policy that uses random parameters in
place of parameters tuned by AutoAugment.

## Experiments

### Experimental Setup

#### Datasets

We evaluate LADC on several popular long-tailed classification tasks,
including both image and text data. Image datasets include CIFAR-10-LT,
CIFAR-100-LT (Cui et al. 2019), ImageNet-LT (Cao et al. 2019) and iNaturalist
(Van Horn et al. 2018). The text dataset includes THUNEWS-LT (Sun et al.
2016). Table 1 shows the detailed statistics of the datasets, where the “IF”
indicates the imbalance factor, i.e., the ratio of the number of the most-
represented class to the number of the least-represented class.

#### Baselines

We choose several types of comparison methods: (1) one-stage training methods.
They include Class-Balanced loss (Cui et al. 2019), Focal loss (Lin et al.
2017), CB-Focal loss (Cui et al. 2019), LDAM loss (Cao et al. 2019), Balanced
Softmax (Ren et al. 2020), Meta-class-weight LDAM (Jamal et al. 2020) and a
causal model Decoufound-TDE (Tang, Huang, and Zhang 2020). For the LDAM loss,
we also consider the results with DRW strategy, i.e., LDAM-DRW. (2) two-stage
training methods. The cRT, LWS (Kang et al. 2019) and MiSLAS (Zhong et al.
2021) reuse the original training data. (3) generative approaches. Feature
space augmentation (FSA) (Chu et al. 2020), Major-to-Minor (Kim, Jeong, and
Shin 2020) and MetaSAug-LDAM (Li et al. 2021) synthesize new training data. We
also involve SMOTE (Chawla et al. 2002), a widely-used generative method for
mitigating oversampling issue. (4) other baselines, including OLTR(Liu et al.
2019), BBN (Zhou et al. 2020), SSP (Yang and Xu 2020), Bag of tricks (Zhang et
al. 2021) and Hybrid-PSC (Wang et al. 2021).

#### Implementation Details.

In the experiments, we regard the most frequent classes occupying at least 60%
of the total training instances as head classes, and the remaining classes as
tail classes. The number of head classes $m$ selected for computing $S$ is
chosen from $\\{2,3\\}$. $\alpha$, $\beta$ and $\tau$ are selected from
$0.1\sim 0.2$, $0.4\sim 1.3$ and $1.2\sim 1.3$, respectively. Detailed
parameter analysis can be referred to the Appendix.

For CIFAR10-LT and CIFAR100-LT, we use ResNet-32 (He et al. 2016) as backbone
following the work (Cao et al. 2019). We first train the backbone with
representation enhancement for 400 epochs with five warm up steps. The base
learning rate is 0.1 and we conduct a multi-step learning rate schedule which
decreases learning rate by 0.01 at the $320^{th}$ and $360^{th}$ epochs. In
the second training stage, we freeze the backbone and train classifier with
LADC for 30 epochs during which the learning rate drops by 0.1 at the
$10^{th}$ and $20^{th}$ epochs.

For ImageNet-LT dataset, we choose ResNet-10 and ResNet-50 as the backbone and
train for 200 epochs in the first stage. The learning rate is initialized as
0.2 and decreases by 0.1 at the $120^{th}$ and $160^{th}$ epochs. For the
iNaturalist dataset, we train our approach for 100 epochs employing the cosine
learning rate schedule (Loshchilov and Hutter 2016) for 100 epochs with
ResNet-50 in the first stage. The training strategy in the second stage is
similar as that in training the CIFAR datasets.

We use SGD optimizer with the momentum at 0.9 and weight decay at $5\cdot
10^{-4}$ for all the experiments. The experimentation is run on a single
NVIDIA Tesla V100 with 32GB graphic memory.

### Experiment Results

#### Main results on CIFAR

Table 2 shows the results on CIFAR10-LT and CIFAR100-LT. As can be seen, the
proposed LADC consistently outperforms all the baselines. For the re-weighting
approaches (i.e., the top 7 baselines), LADC increases the performance by at
least 4.31% and 6.54% on average for CIFAR10-LT and CIFAR100-LT, respectively.
Comparing with the best two-stage approach MiSLAS, LADC shows an improvement
by 2.72% on CIFAR100-LT. LADC also performs better than the state-of-the-art
generative approach MetaSAug, presenting an increase by 3.02% and 3.05% on
average for the two datasets, respectively. The results indicate the
effectiveness of the proposed approach in the long-tailed image
classification.

#### Main results on ImageNet-LT

We conduct experiments on two backbones, i.e., ResNet-10 and ResNet-50. As
shown in Table 3, LADC achieves the best performance among all the approaches
for different backbones. Specifically, LADC outperforms the baselines by at
least 0.47% and 1.13% for the two backbones, respectively. The results imply
that representations learned by a larger model could better benefit LADC.

Table 3: Top-1 accuracy (%) on ImageNet-LT. Best results are marked in bold. BALAMs denotes Balanced Softmax loss with meta sampler. Approach | ResNet-10 | ResNet-50
---|---|---
Focal Loss | 30.50 | 43.70
LDAM-DRW | 40.73 | 48.80
BALAMs | 41.80 | 50.04
OLTR | 35.60 | 41.90
Remix | 37.58 | 46.19
cRT | 41.80 | 47.30
LWS | 41.40 | 47.70
FSA | 35.20 | -
cRT+SSP | 43.20 | 51.30
Bag of Tricks | 43.13 | -
MiSLAS | - | 51.47
MetaSAug | - | 50.52
LADC | 43.67 | 52.60

#### Main Results on iNaturalist

Table 4 presents the results on large real-world dataset iNaturalist with
ResNet-50. We can observe that LADC increases the accuracy of baseline
approaches by 0.58%$\sim$8.21%, which shows the capability of LADC in handling
large long-tailed datasets.

#### Main results on the text dataset THUNEWS-LT

We choose a commonly-used subset of Chinese news dataset THUCNews (Sun et al.
2016) which contains ten classes111https://github.com/649453932/Chinese-Text-
Classification-Pytorch. We build its long-tailed version with the imbalance
factor at 100 and use the pre-trained embedding model provided by Li et al.
(2018). We choose three models including TextCNN (Kalchbrenner, Grefenstette,
and Blunsom 2014), TextRNN (Liu, Qiu, and Huang 2016) and Transformer (Vaswani
et al. 2017) as the backbone. We remove the representation enhancement
component due to the inapplicability of the data augmentation techniques for
the text data. The results illustrated in Table 5 show that LADC achieves the
highest accuracy, increasing the best baselines by 0.5%, 1.49% and 0.99% on
three models, respectively. The results demonstrate the generalizability and
effectiveness of LADC in other long-tailed modals.

Table 4: Top-1 accuracy (%) on iNaturalist2018. All the models are trained up to 100 epochs. Approach | ResNet-50
---|---
CE | 57.16
CB-Focal | 61.12
LDAM-DRW | 68.00
BBN | 66.29
Decoufound-TDE | 65.20
cRT+SSP | 68.10
Hybrid-PSC | 68.10
MetaSAug | 68.75
LADC | 69.33
Table 5: Top-1 accuracy (%) on Chinese text classification. CB loss and BSCE denote Class Balanced loss and Balanced Softmax, respectively. Approach | TextCNN | TextRNN | Transformer
---|---|---|---
CE | 78.65 | 78.79 | 79.12
CB Loss | 80.44 | 80.83 | 80.43
LWS | 83.97 | 81.45 | 82.88
BSCE | 84.28 | 81.06 | 83.02
LADC | 84.78 | 82.94 | 84.01
Table 6: Ablation study on CIFAR-100-LT. RE: the representation enhancement
module in Stage 1. Sampling: sampling from the distributions calibrated by
LADC in Stage 2.

Module | CIFAR100-LT
---|---
RE | Sampling | LWS | LWS-Plus | 200 | 100 | 50
✗ | ✗ | ✗ | ✗ | 34.70 | 37.92 | 44.02
✓ | ✗ | ✗ | ✗ | 38.12 | 42.30 | 46.72
✓ | ✓ | ✓ | ✗ | 45.29 | 49.22 | 54.00
✓ | ✓ | ✗ | ✓ | 46.62 | 50.77 | 54.94

### Discussion

#### Ablation Study

We perform an ablation study to analyze the impact of each module in LADC. We
follow the same two-stage training implementation throughout this study, and
the results are shown in Table 6. Comparing the first two rows of the table,
we can see that RE benefits the classification performance, as expected. In
addition, we can observe that sampling from the calibrated distribution brings
further improvement, as shown by the third row. Comparing the last two rows,
we can achieve that the proposed LWS-Plus better rectifies the decision
boundary than LWS.

#### Results Analysis

Following Wang et al. (2020), we divide the classes into three groups
according to the number of training instances associated with each class,
i.e., Many-shot ($>$100 instances), Medium-shot ($100\sim 20$ instances) and
Few-shot ($<$20 instances). The results of different groups are shown in Table
7. Comparing with the model trained by Cross-Entropy (CE), the baselines
mainly focus on boosting the performance of the few-shot group with slight
improvement on the medium-shot group. Besides, all the baselines expect for
Remix show a performance degradation on the many-shot group. In contrast, the
proposed LADC significantly enhances the accuracy on the few-shot and medium-
shot groups, while achieving almost similar performance on the many-shot group
as CE. The results indicate the effectiveness of LADC on both head and tail
classes.

Table 7: Top-1 accuracy (%) of different groups on CIFAR100-100. Approach | All | Many | Medium | Few
---|---|---|---|---
CE | 37.9 | 65.2 | 36.3 | 8.0
Remix | 39.7 | 69.4 | 37.9 | 7.2
OLTR | 41.2 | 61.8 | 41.4 | 17.6
LDAM-DRW | 42.0 | 61.5 | 41.7 | 20.2
$\tau$-norm | 41.4 | 58.8 | 38.1 | 24.7
cRT | 43.3 | 64.0 | 44.8 | 18.1
LADC | 50.8 | 64.5 | 52.6 | 32.5

#### Calibration strategy

To analyze the benefit of our distribution calibration strategy, we compare
with some alternative variants of our method. One variant is LADCAverage,
indicating calibrating distribution for the whole tail class, i.e.,
$\bm{\hat{\mu}}$ in Equ. (11) is the mean feature of the tail class. In
addition, we compare with the DC method in the few-shot learning field (Yang,
Liu, and Xu 2021). Note that the sampling strategy is the same for all the
comparisons. The results are listed in Table 8. As can be seen, our proposed
LADC significantly outperforms the DC method, which demonstrates that LADC is
more effective than DC in the long-tailed scenario. In addition, comparing
LADC with LADCAverage, we can achieve that instance-based distribution
calibration can better estimate the distributions of classes than class-based
calibration.

#### Comparison with different classifier re-balancing approaches in Stage 2.

Table 9 shows the results of different classifier re-balancing approaches in
the second training stage. The experiments are also conducted on CIFAR100-LT.
We choose class-balanced sampling (CBS), the state-of-the-art regularization
method - label-aware smoothing (LAS) and our proposed label-aware distribution
calibration combined with cRT and LWS, respectively, to re-balance the
classifier in Stage 2. We can observe that LADC significantly improves the
performance of class-balanced sampling method (CBS). Besides, LADC outperforms
LAS under different training approaches in Stage 2. The results show the
effectiveness of the proposed distribution calibration method.

#### Visualization of Generated Samples

Figure 3: Visualization of distribution estimation on CIFAR10-200 with the
ResNet-32 backbone in the feature space.

Fig. 3 presents the visualization of feature distribution of training set and
test set of CIFAR10-200. Visualization is conducted by adding a fully-
connected layer ($D*2$ where $D$ is the dimension of features) which projects
feature into 2-dimension space following Liu et al. (2016) Fig. 3 (a) shows
the original training set, where we can see the tail classes (e.g., label
7,8,9) have few training instances and their distributions are significantly
different from those in the test set, as illustrated in Fig. 3 (c).

After sampling via LADC, we can observe in Fig. 3 (b) that the distributions
of tail classes are obviously extended and more closer to the test set. This
indicates that the difference between the distributions of the original
training set and test set is relieved by sampled instances. Meanwhile, LADC
can well preserve the distributions of head classes.

Table 8: Study on different distribution calibration strategies. Approach | CIFAR100-LT
---|---
200 | 100 | 50 | 10
DC | 44.10 | 48.35 | 51.69 | 62.59
LADCAverage | 41.74 | 48.01 | 47.25 | 62.34
LADC (ours) | 46.62 | 50.77 | 54.94 | 64.66
Table 9: Comparison on different classifier modification approach in Stage 2
given a backbone trained with representation enhancement module. CBS and LAS
denote class-balanced sampling and label-aware smoothing, respectively.

Approach | CIFAR100-LT
---|---
Imbalance Factor | 200 | 100 | 50
cRT | CBS | 45.20 | 50.14 | 54.18
LAS | 45.87 | 50.02 | 53.10
LADC | 46.92 | 50.13 | 55.24
LWS | CBS | 43.77 | 48.23 | 53.04
LAS | 44.02 | 48.21 | 52.59
LADC | 45.29 | 49.22 | 54.00

## Related work

Re-sampling and Re-weighting. Re-sampling incorporates two approach types,
i.e., over sampling tail classes (Shen, Lin, and Huang 2016; Buda, Maki, and
Mazurowski 2018; Byrd and Lipton 2019) and under sampling head classes (Buda,
Maki, and Mazurowski 2018; Japkowicz and Stephen 2002). Over sampling tail
classes can augment training data of tail classes, which may cause severe over
fitting problems on tail classes. Under sampling head classes is feasible to
prevent head classes from dominating training process, however, it inevitably
degrades the generalization ability of models for the reduction of head data.

Broadly speaking, a group of works aim to mitigate the issue of long-tail
distribution by re-weighting training instances in the objective function (Cui
et al. 2019; Tan et al. 2020; Lin et al. 2017; Ren et al. 2020; Cao et al.
2019; Tan et al. 2021). For example, Focal loss (Lin et al. 2017) determines
the weights according to the model’s confidence of the training instances.
Several other works determine the weights in inverse proportion to the class
frequency. Cui et al. (2019) calculate the weights according to the effective
number of each classes. Inspired by the generalization ability of adding
margins, Cao et al. (2019) assign a label-aware margin to each class. While
the re-weighting methods in Cui et al. (2019) and Cao et al. (2019) determine
class weights in a pre-defined manner, meta-class-weight (Jamal et al. 2020)
adopts a learnable class weights via meta-learning. Although these re-
weighting methods enable the training of neural networks in a end-to-end
fashion, they result in sub-optimal performance in comparison with two-stage
training methods, potentially due to the distortion in representation learning
caused by early re-weighting.

Two-stage training. Cao et al. (2019) propose a two-stage training methods, in
which re-weighting or re-sampling is only introduced in the second stage.
Decoupled learning is another form of two-stage training that is introduced by
Kang et al. (2019) who claim that training on imbalance dataset biases the
classifier instead of representation learning. In the first stage, Kang et al.
(2019) train a backbone using a traditional cross-entropy loss. Then the
backbone is fixed and the classifier is re-balanced by one of the following
methods: classifier re-training (cRT), classifier normalization ($\tau$ norm)
and learnable classifier weight scaling (LWS). MiSLAS by Zhong et al. (2021)
is also a decoupled learning approach, which utilizes label-aware smoothing to
regularize the classifier. Although simple and effective, there are some
issues not properly addressed by these two-stage methods. For example, class-
balance sampling can still cause over-fitting to tail classes during the
second training stage. Another issue is the distribution mismatch of tail
classes. Our method aims to address these issues via label-aware distribution
calibration and generative oversampling.

Data synthesis approach. Synthesizing new instances has been used widely to
construct a balanced dataset. SMOTE proposed by Chawla et al. (2002) generates
new data through convex combination of a data point and its neighbors. Based
on SMOTE, several variants have been introduced (Han, Wang, and Mao 2005;
Mullick, Datta, and Das 2019). Although they can potentially alleviate the
issue of over-fitting, these methods cannot address the distribution mismatch
issue of tail classes. More recently, there have been works proposed to
generate data for tail classes relying on head classes. For example, M2m (Kim,
Jeong, and Shin 2020) synthesizes data for tail classes via translating
samples from head classes. And MetaSAug (Li et al. 2021) combines implicit
semantic data augmentation and meta-learning to generate new training
instances. These generative methods are computationally complicated and still
need to combine re-weighting methods.

## Conclusion

In this paper, we propose a novel label-aware distribution calibration (LADC)
method for long-tailed classification. LADC can calibrate the distribution of
tail classes by borrowing knowledge from well-represented head classes. Based
on the decouple learning framework, we also explore representation enhancement
and classifier re-balancing technique. Extensive experiments on several long-
tailed image and text classification datasets demonstrate the effectiveness of
LADC.

## References

  * Buda, Maki, and Mazurowski (2018) Buda, M.; Maki, A.; and Mazurowski, M. A. 2018.  A systematic study of the class imbalance problem in convolutional neural networks.  _Neural Networks_ , 106: 249–259.
  * Byrd and Lipton (2019) Byrd, J.; and Lipton, Z. 2019.  What is the effect of importance weighting in deep learning?  In _International Conference on Machine Learning_ , 872–881. PMLR.
  * Cao et al. (2019) Cao, K.; Wei, C.; Gaidon, A.; Arechiga, N.; and Ma, T. 2019.  Learning imbalanced datasets with label-distribution-aware margin loss.  In _Proceedings of the 33rd International Conference on Neural Information Processing Systems_ , 1567–1578.
  * Chawla et al. (2002) Chawla, N. V.; Bowyer, K. W.; Hall, L. O.; and Kegelmeyer, W. P. 2002.  SMOTE: synthetic minority over-sampling technique.  _Journal of artificial intelligence research_ , 16: 321–357.
  * Chou et al. (2020) Chou, H.-P.; Chang, S.-C.; Pan, J.-Y.; Wei, W.; and Juan, D.-C. 2020.  Remix: Rebalanced Mixup.  In _European Conference on Computer Vision_ , 95–110. Springer.
  * Chu et al. (2020) Chu, P.; Bian, X.; Liu, S.; and Ling, H. 2020.  Feature space augmentation for long-tailed data.  In _Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIX 16_ , 694–710. Springer.
  * Cubuk et al. (2020) Cubuk, E. D.; Zoph, B.; Shlens, J.; and Le, Q. V. 2020.  Randaugment: Practical automated data augmentation with a reduced search space.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , 702–703.
  * Cui et al. (2019) Cui, Y.; Jia, M.; Lin, T.-Y.; Song, Y.; and Belongie, S. 2019.  Class-balanced loss based on effective number of samples.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 9268–9277.
  * Deng et al. (2009) Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei-Fei, L. 2009.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , 248–255. Ieee.
  * Han, Wang, and Mao (2005) Han, H.; Wang, W.-Y.; and Mao, B.-H. 2005.  Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning.  In _International conference on intelligent computing_ , 878–887. Springer.
  * He et al. (2016) He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 770–778.
  * Jamal et al. (2020) Jamal, M. A.; Brown, M.; Yang, M.-H.; Wang, L.; and Gong, B. 2020.  Rethinking class-balanced methods for long-tailed visual recognition from a domain adaptation perspective.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 7610–7619.
  * Japkowicz and Stephen (2002) Japkowicz, N.; and Stephen, S. 2002.  The class imbalance problem: A systematic study.  _Intelligent data analysis_ , 6(5): 429–449.
  * Kalchbrenner, Grefenstette, and Blunsom (2014) Kalchbrenner, N.; Grefenstette, E.; and Blunsom, P. 2014.  A convolutional neural network for modelling sentences.  _arXiv preprint arXiv:1404.2188_.
  * Kang et al. (2019) Kang, B.; Xie, S.; Rohrbach, M.; Yan, Z.; Gordo, A.; Feng, J.; and Kalantidis, Y. 2019.  Decoupling representation and classifier for long-tailed recognition.  _arXiv preprint arXiv:1910.09217_.
  * Kim, Jeong, and Shin (2020) Kim, J.; Jeong, J.; and Shin, J. 2020.  M2m: Imbalanced classification via major-to-minor translation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 13896–13905.
  * Li et al. (2021) Li, S.; Gong, K.; Liu, C. H.; Wang, Y.; Qiao, F.; and Cheng, X. 2021.  MetaSAug: Meta Semantic Augmentation for Long-Tailed Visual Recognition.  _arXiv preprint arXiv:2103.12579_.
  * Li et al. (2018) Li, S.; Zhao, Z.; Hu, R.; Li, W.; Liu, T.; and Du, X. 2018.  Analogical reasoning on chinese morphological and semantic relations.  _arXiv preprint arXiv:1805.06504_.
  * Lin et al. (2017) Lin, T.-Y.; Goyal, P.; Girshick, R.; He, K.; and Dollár, P. 2017.  Focal loss for dense object detection.  In _Proceedings of the IEEE international conference on computer vision_ , 2980–2988.
  * Liu, Qiu, and Huang (2016) Liu, P.; Qiu, X.; and Huang, X. 2016.  Recurrent neural network for text classification with multi-task learning.  _arXiv preprint arXiv:1605.05101_.
  * Liu et al. (2016) Liu, W.; Wen, Y.; Yu, Z.; and Yang, M. 2016.  Large-Margin Softmax Loss for Convolutional Neural Networks.  In _ICML_ , volume 48 of _JMLR Workshop and Conference Proceedings_ , 507–516. JMLR.org.
  * Liu et al. (2019) Liu, Z.; Miao, Z.; Zhan, X.; Wang, J.; Gong, B.; and Yu, S. X. 2019.  Large-scale long-tailed recognition in an open world.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2537–2546.
  * Loshchilov and Hutter (2016) Loshchilov, I.; and Hutter, F. 2016.  Sgdr: Stochastic gradient descent with warm restarts.  _arXiv preprint arXiv:1608.03983_.
  * Menon et al. (2020) Menon, A. K.; Jayasumana, S.; Rawat, A. S.; Jain, H.; Veit, A.; and Kumar, S. 2020\.  Long-tail learning via logit adjustment.  _arXiv preprint arXiv:2007.07314_.
  * Mullick, Datta, and Das (2019) Mullick, S. S.; Datta, S.; and Das, S. 2019.  Generative adversarial minority oversampling.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 1695–1704.
  * Ren et al. (2020) Ren, J.; Yu, C.; Sheng, S.; Ma, X.; Zhao, H.; Yi, S.; and Li, H. 2020.  Balanced Meta-Softmax for Long-Tailed Visual Recognition.  _arXiv preprint arXiv:2007.10740_.
  * Shen, Lin, and Huang (2016) Shen, L.; Lin, Z.; and Huang, Q. 2016.  Relay backpropagation for effective learning of deep convolutional neural networks.  In _European conference on computer vision_ , 467–482. Springer.
  * Sun et al. (2016) Sun, M.; Li, J.; Guo, Z.; Yu, Z.; Zheng, Y.; Si, X.; and Liu, Z. 2016.  Thuctc: an efficient chinese text classifier.  _GitHub Repository_.
  * Tan et al. (2021) Tan, J.; Lu, X.; Zhang, G.; Yin, C.; and Li, Q. 2021.  Equalization Loss v2: A New Gradient Balance Approach for Long-tailed Object Detection.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 1685–1694.
  * Tan et al. (2020) Tan, J.; Wang, C.; Li, B.; Li, Q.; Ouyang, W.; Yin, C.; and Yan, J. 2020.  Equalization loss for long-tailed object recognition.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 11662–11671.
  * Tang, Huang, and Zhang (2020) Tang, K.; Huang, J.; and Zhang, H. 2020.  Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect.  _Advances in Neural Information Processing Systems_ , 33.
  * Van Horn et al. (2018) Van Horn, G.; Mac Aodha, O.; Song, Y.; Cui, Y.; Sun, C.; Shepard, A.; Adam, H.; Perona, P.; and Belongie, S. 2018.  The inaturalist species classification and detection dataset.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 8769–8778.
  * Vaswani et al. (2017) Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, Ł.; and Polosukhin, I. 2017.  Attention is all you need.  In _Advances in neural information processing systems_ , 5998–6008.
  * Wang et al. (2021) Wang, P.; Han, K.; Wei, X.-S.; Zhang, L.; and Wang, L. 2021.  Contrastive Learning based Hybrid Networks for Long-Tailed Image Classification.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 943–952.
  * Wang et al. (2020) Wang, X.; Lian, L.; Miao, Z.; Liu, Z.; and Yu, S. 2020.  Long-tailed Recognition by Routing Diverse Distribution-Aware Experts.  In _International Conference on Learning Representations_.
  * Wang et al. (2019) Wang, Y.; Pan, X.; Song, S.; Zhang, H.; Huang, G.; and Wu, C. 2019.  Implicit semantic data augmentation for deep networks.  _Advances in Neural Information Processing Systems_ , 32: 12635–12644.
  * Yang, Liu, and Xu (2021) Yang, S.; Liu, L.; and Xu, M. 2021.  Free lunch for few-shot learning: Distribution calibration.  _arXiv preprint arXiv:2101.06395_.
  * Yang and Xu (2020) Yang, Y.; and Xu, Z. 2020.  Rethinking the value of labels for improving class-imbalanced learning.  _arXiv preprint arXiv:2006.07529_.
  * Zhang et al. (2021) Zhang, Y.; Wei, X.-S.; Zhou, B.; and Wu, J. 2021.  Bag of Tricks for Long-Tailed Visual Recognition with Deep Convolutional Neural Networks.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 35, 3447–3455.
  * Zhong et al. (2021) Zhong, Z.; Cui, J.; Liu, S.; and Jia, J. 2021.  Improving Calibration for Long-Tailed Recognition.  _arXiv preprint arXiv:2104.00466_.
  * Zhou et al. (2020) Zhou, B.; Cui, Q.; Wei, X.-S.; and Chen, Z.-M. 2020.  Bbn: Bilateral-branch network with cumulative learning for long-tailed visual recognition.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 9719–9728.