pnasresearcharticle Sanchez, Dammann Capillarity-driven flows in nanometer-
sized pores play a dominant role in many natural and technological processes,
ranging from water transport and transpiration in trees, swelling of clay, and
catalysis to transport through microfluidic structures and the fabrication of
battery materials. Here we show by a combination of experiments and computer
simulations on water imbibition in nanopores that the competition between
expansive, surface stress release at pore walls and negative, contractile
Laplace pressures of nanoscale menisci leads to an unusual macroscopic
behavior of the porous medium, which is generic for any liquid/nanoporous
solid combination. The results allow one to quantify surface and Laplace
stresses and to monitor nanoscale flow and infiltration states by relatively
simple length measurements of the porous medium. J.S., Z. L. and P.H.
conceived the experiments. J.S. and Z.L. performed the gravimetric, optical
and dilatometry experiments. J.S. and L.G performed the later experiments
including evaporation effects. L.D. performed the Molecular Dynamics
simulations. J.S., L.D., M.F., R.M., H.A.S., and P.H. interpreted the
experimental and computer simulation data. J.S., L.D., H.A.S., P.H. wrote the
manuscript. All authors proofread the manuscript. The authors declare no
competing interests. 1J.S.(Author One) and L.D. (Author Two) contributed
equally to this work. 2To whom correspondence should be addressed. E-mail:
<EMAIL_ADDRESS>
# Deformation Dynamics of Nanopores
upon Water Imbibition

Juan Sanchez  Institute for Materials and X-Ray Physics, Hamburg University of
Technology, Germany  Lars Dammann  Institute for Materials and X-Ray Physics,
Hamburg University of Technology, Germany  Center for X-ray and Nano Science
CXNS, Deutsches Elektronen-Synchrotron DESY, Germany  Institute of Polymers
and Composites, Hamburg University of Technology, Germany  Laura Gallardo
Institute for Materials and X-Ray Physics, Hamburg University of Technology,
Germany  Centre for the Study of Manuscript Cultures (CSMC), Hamburg
University, Hamburg, Germany  Zhuoqing Li  Institute for Materials and X-Ray
Physics, Hamburg University of Technology, Germany  Center for X-ray and Nano
Science CXNS, Deutsches Elektronen-Synchrotron DESY, Germany  Michael Fröba
Institute of Inorganic and Applied Chemistry, University of Hamburg, 20146
Hamburg, Germany  Robert Meißner  Institute of Polymers and Composites,
Hamburg University of Technology, Germany  Institute of Surface Science,
Helmholtz-Zentrum Hereon, Germany  Howard A. Stone  Department of Mechanical
and Aerospace Engineering, Princeton University, Princeton, New Jersey, USA
Patrick Huber  Institute for Materials and X-Ray Physics, Hamburg University
of Technology, Germany  Center for X-ray and Nano Science CXNS, Deutsches
Elektronen-Synchrotron DESY, Germany  Institute of Surface Science, Helmholtz-
Zentrum Hereon, Germany

###### Abstract

Capillarity-driven transport in nanoporous solids is ubiquitous in nature and
is of increasing importance for the functionality of modern liquid-infused
engineering materials. During imbibition, highly curved menisci are driven by
negative Laplace pressures of several hundred atmospheres, exerting an
enormous contractile load on an increasing portion of the porous matrix. Due
to the challenge of simultaneously monitoring imbibition and deformation with
high spatial resolution, the resulting coupling of solid elasticity to liquid
capillarity has remained largely unexplored. Here, we study water imbibition
in mesoporous silica using optical imaging, gravimetry, and high-resolution
dilatometry. In contrast to an expected Laplace pressure-induced contraction,
we find a square-root-of-time expansion and an additional abrupt length
increase when the menisci reach the top surface. The final expansion is absent
when we stop the imbibition front inside the porous medium in a dynamic
imbibition-evaporation equilibrium, as is typical for water transport and
transpiration in plants. These peculiar deformation behaviors are validated by
single-nanopore molecular dynamics simulations and described by a continuum
model that highlights the importance of expansive surface stresses at the pore
walls (Bangham effect) and the buildup or release of contractile Laplace
pressures as nanoscale menisci collectively advance, arrest, or disappear. Our
model predicts that these observations are valid not only for water imbibition
in silica, but for any imbibition process in nanopores, regardless of the
liquid/solid combination. This also suggests that simple deformation
measurements can be used to quantify surface stresses and Laplace pressures or
transport in a wide variety of natural and artificial porous media.

###### keywords:

nanoporous materials $|$ water confinement $|$ Laplace pressure $|$ surface
stress

This manuscript was compiled on

Liquid-infused nanoporous solids play an increasingly important role as
functional materials. One can find them as soft-hard hybrids with superior
properties in terms of adaptive wettability (1), mechanical actuation (2, 3),
adjustable photonics (4, 5), in nanofluidics (6), and in energy storage and
energy conversion (7) applications. Also, the development of structural
materials that are capable of adapting to changing environmental conditions
can be achieved by combining soft, dynamic liquid phases with static, solid
phases that act as mechanically robust scaffold structures. The resulting
hybrid materials have demonstrated unprecedented properties of stability,
adaptability, and stimulus-responsiveness, as desired in many applications
such as 3D printing, soft robotics, omniphobic surfaces, microfluidics, and
multiphase separation (8, 9, 1).

It has also long been known, starting with the pioneering work of Meehan and
Hardy (10) and Bangham and Fakhoury (11), that stresses and strains are
generated in porous solids during the adsorption and desorption of gases and
liquids (12, 13, 14, 15, 16, 17, 18, 19, 20, 21). Two effects play a major
role in the resulting sorption-induced deformation (22, 18). First, the
adsorption of matter onto the solid pore wall surface typically leads to a
reduction in surface stress. The reduced stress results in an expansion of the
porous solid (Bangham effect) (23). Second, the formation of menisci in the
pore space during capillary condensation can create a negative pressure in the
liquid, the magnitude of which is inversely proportional to the pore size. The
effect of the negative pressure on the solid surface causes the solid to
contract (Laplace effect) (24).

Similar to fluid adsorption processes, deformation of porous solids is also
expected to occur in capillarity-driven spontaneous imbibition experiments
with wetting liquids (25, 12, 26) as well as in forced intrusion of non-
wetting liquids (27, 28). In fact, studies highlight the importance of Laplace
pressure-induced pore space deformations and their impact on imbibition
dynamics for soft (macro)porous media such as cellulose sponges (29, 30, 31).
However, the influence of surface stress on the deformation behavior, which is
particularly important for nanoporous media with their large internal surface-
to-volume ratio and which is predicted to lead to pore space expansion, has
been little studied. Thus, in contrast to gas adsorption-induced deformations,
a comprehensive understanding of imbibition-induced deformations as a result
of the interplay of Bangham and Laplace effects is still lacking for the
important class of liquid-infused nanoporous materials.

Here, we aim to achieve such a mechanistic understanding of imbibition-induced
deformation in nanoporous solids by performing combined experiments on water
imbibition and the corresponding deformation dynamics of a model nanoporous
solid (monolithic Vycor glass). Furthermore, by comparison with atomistic
simulations of water imbibition on a single silica nanopore, we trace the
distinct deformation regimes observed at the macroscopic scale to the single-
pore behavior.

## Results and Discussion

Figure 1: Elastocapillarity dynamics upon water imbibition in nanoporous Vycor
glass on the porous-medium scale. (a) Optical snapshots of water imbibition in
a rod-like Vycor monolith. The time between snapshots is 15 minutes. (b) Mean
front position, (c) mass of water uptake, and (d) sample strain measured as a
function of time by optical imaging, gravimetry, and dilatometry,
respectively. (e) Schematic illustration of the macroscopic deformation of the
glass monolith as a function of sample height (left), caused by the interplay
of sample contraction, negative hydrostatic pressure (purple) and sample
expansion due to surface stress release at the water-silica wall interface per
unit sample length (red) as a function of sample height for one selected time
$t$ during imbibition. The strain sensor or pushrod is represented as a
cylindrical glass piece in contact with the top face of the sample, while the
bottom face is in contact with a liquid reservoir. (f) Schematic deformation
states for selected times during imbibition (regime I) and for complete
filling (regime III); also indicated are the resulting changes in the sample
length (green line) as well as the imbibition front position (blue line).

### Imbibition Experiments with Water: The Interplay of Front Movement, Front
Broadening, and Deformation Dynamics

We study the deformation of nanoporous Vycor glass upon infiltration of liquid
water. Vycor is an almost pure fused silica glass permeated by a 3D network of
interconnected pores (32, 33). It is formed by a leaching process after
spinodal decomposition of a borosilicate glass. Therefore, its geometric
structure can be well represented by clipped Gaussian random fields (34, 35).

To monitor the capillarity-driven spontaneous water imbibition, we perform
optical imaging experiments. They allow us to determine the mean front
position and its broadening, respectively, due to a transparency change
induced by the capillary filling of the pore space and light scattering
typical of the inhomogeneously filled pores near the imbibition front, see
movie S1(36, 37, 38, 39). A set of selected snapshots is shown in Fig. 1a
aligned with the optically determined mean imbibition front position (Fig.
1b), mass uptake of the porous monolith (Fig. 1c), and the measured sample
strain as a function of time (Fig. 1d).

Since the relative humidity ($RH$) and thus potential adsorption of water via
the vapor phase before and during the infiltration experiments is expected to
affect both the deformation and imbibition dynamics (40, 36), we perform the
experiments in a closed cell with controlled humidity. In the following we
first discuss the experiment with a relative humidity ($RH$) of 50%. According
to a water vapor sorption isotherm this corresponds to approximately two pre-
adsorbed monolayers of water molecules at the silica pore walls (36).

Evaporation from the porous block was prevented by shielding the side surface
of the Vycor monolith, leaving the top and bottom surfaces unshielded to allow
unaltered contact with the dilatometer pushrod and liquid water inlet,
respectively. After initiating capillary rise in the porous glass by raising
the level of the bulk liquid reservoir until it reaches the bottom of the
glass cylinder, we observe deformation kinetics that include three distinct
regimes. Regime I lasts from the moment of contact with the bulk liquid
reservoir until the imbibition front reaches the top of the matrix, see Fig.
1d. Here the sample expands in accordance with the mean front motion according
to a square-root-of-time dynamics. This regime is terminated by a relatively
abrupt, additional expansion of the matrix that occurs when the advancing part
of the white imbibition front (see Fig. 1a) reaches the top of the matrix.
This step-like deformation continues until the receding part of the white
front reaches the top surface, and thus until complete filling of the pore
space is achieved. This can be seen in both the optical imaging (see Fig. 1b)
and mass uptake (see Fig. 1c) experiments. In agreement with a constant final
mass uptake and the complete disappearance of the white front, the subsequent
regime III after this “strain jump” indicates the complete filling of the pore
space and is characterized by a deformation plateau. The total strain measured
after complete filling of the matrix is $0.09\text{\,}\mathrm{\char
37\relax}$, which corresponds to a length increase of
$10.79\text{\,}\mathrm{\SIUnitSymbolMicro m}$ of our sample (original length
$11.99\text{\,}\mathrm{mm}$).

Next, we propose a model for the time-dependent strain evolution. The model
requires a macroscopic description of the imbibition dynamics, as well as a
characterization of the strain-inducing elastocapillary phenomena occurring at
the single-nanopore scale and how they translate to the macroscopic porous-
medium scale.

### Elastocapillarity Dynamics: From the Single-Nanopore to the Porous-Medium
Scale

The imbibition dynamics of the Lucas-Washburn front motion can be derived from
a Darcy description of the spontaneous imbibition process (36, 41). Since at
these scales inertial and gravitational effects are negligible compared to
capillarity, the imbibition kinetics are governed by a dynamic equilibrium
between the approximately constant Laplace pressure at the highly curved
menisci of the imbibition front, which drives the flow, and the increasing
viscous drag in the liquid-saturated part of the sample, which opposes the
flow. As outlined in Ref. (36), for pore-space geometries such as those
present in Vycor glass, the following equation is obtained for the imbibition
height $h(t)$ as a function of time $t$:

$h(t)=\sqrt{\frac{\gamma}{2\eta\tau\xi}\frac{(r_{p}-d)^{3}}{r_{p}^{2}}}\sqrt{t},$
(1)

where $r_{p}$ is the pore radius, $\gamma$ is the water-vapor surface tension,
$\eta$ is the viscosity of water, and $\tau$ is the hydraulic tortuosity. The
pre-adsorbed water film on the pore walls has a thickness $d$. This pore wall
coverage results in a reduced, initial porosity $\Phi_{\rm i}$ accessible to
the imbibing water, compared to the original host material porosity
$\Phi_{0}$, measured in vacuum. Therefore, a scaling factor
$\xi=\frac{\Phi_{\rm i}}{\Phi_{\rm 0}}$ must be included in Eq. 1.We will
refer to this equation as the modified LW equation (MLW).

Eq. 1 can be extended to describe the mass uptake of water by multiplying the
host material area $A$, the reduced porosity $\Phi_{\rm i}$, and the water
density $\rho$, giving $\Delta m(t)=\rho\Phi_{\rm i}Ah(t)$. Assuming two
immobile water layers ($d=$ $0.5\text{\,}\mathrm{nm}$), one obtains an
excellent fit to the observed mean imbibition position and mass uptake, see
Fig. 1b,c. This reduction in the hydraulic radius can be attributed to the
strongly bound, highly viscous interfacial water layers at the silica
surfaces, in agreement with previous experiments and computer simulations (42,
36). Note, that the pre-adsorbed water layers lead to an initial Bangham
effect, i.e., adsorption-induced expansion of the Vycor glass relative to the
completely dry sample, even before the imbibition experiment is started.

Upon contact with the bulk reservoir, two processes occur in parallel. In the
part of the pore space filled with liquid, i.e., the part of the sample up to
the capillary rise height $h(t)$, the Bangham expansion takes place due to the
corresponding release of surface stress. In addition, the pressure in the
liquid increases linearly from the tensile Laplace pressure of the menisci in
the advancing imbibition front to the atmospheric pressure at the bulk water
reservoir. This results in a contractile stress contribution that decreases
linearly from the imbibition front to the bulk reservoir. Thus, the total
deformation of the matrix must be considered as a superposition of two
opposing deformation effects, the expansive surface stress release (Bangham
effect) and the contractile Laplace pressure effect, see Fig. 1e.

Since the induced strain and thus pore size changes are small, on the order of
fractions of a percent, the resulting time-dependent changes in the hydraulic
permeabilities and thus capillarity-driven flow dynamics due to the changes in
the pore-sizes are negligible. Therefore, both deformation effects scale, to
good approximation, linearly with the capillary-filling dynamics. More
precisely, the strain dynamics follow the classical square-root-of-time LW
dynamics, as proven by our experimental strain observation in regime I, see
Fig. 1d.

Since we observe the matrix to be expanding, the expanding Bangham
contribution obviously dominates the Laplace pressure effect. The abrupt
expansion observed in regime II is explained by the sudden vanishing of the
contractile Laplace pressure contribution. Upon reaching the top surface of
the matrix the highly curved menisci and the corresponding Laplace pressure
contribution vanish. The disappearance of the contractile contribution leads
to an additional sample expansion, which we refer to as the “Laplace expansion
jump.” The duration of the observed expansion is dictated by the broadening of
the imbibition front upon complete filling of the sample and thus by the time
characteristic of its vanishing, when the front reaches the top of the sample.
This can be seen by a comparison of the vanishing of the white scattering
front in the optical images with the times characteristic of the beginning and
end of deformation regime II, respectively (Fig. 1).

Figure 2: Impact of evaporation on imbibition-induced deformation. Imbibition-
induced deformation of a nanoporous Vycor glass monolith ($r_{p}=$
$3.4\text{\,}\mathrm{nm}$ and $RH\approx$ $5\text{\,}\mathrm{\char 37\relax}$)
measured by in situ dilatometry in two different scenarios: the first without
evaporation (NEv) in (a-b) and the second with evaporation (Ev) in (c-d). In
both cases we present our dilatometry data (NEv: green circles; EV: red
circles), combined with optical imaging. We select a vertical pixel line
centered in the sample and integrate over time. The visible white region
reveals the broadening of the imbibition front separating the imbibed and dry
portions of the porous host. The dashed lines in (a) and (c) correspond to
fitting curves based on a $\sqrt{t}$ scaling. The fitting curves in (c) do not
start at $t=0$ because they do not account for the early times when
unavoidable lateral imbibition resulted from the insertion of the bottom of
the unsealed into the liquid reservoir. The deformation dynamics are
compatible with the expected exponential relaxation of the imbibition process,
supported by the blue solid fitting line (see Supporting Information for
derivation). In the non-evaporation scenario, such a condition is achieved by
sealing the monolith lateral surface as shown in the panel (b). Here we show a
representative 2-D slice of the completely filled porous matrix in contact
with a liquid reservoir. The top of the sample is effectively sealed by the
dilatometer pushrod. The orange arrows represent the water flow from the
reservoir to the pore space (imbibition). Zooming into the upper region, we
see the absence of curved menisci since there is no evaporating outflow to
counterbalance the capillarity-driven imbibition. This contrasts with the
presence of menisci at the saturated-unsaturated interface, as shown in panel
(d). Here we illustrate the dynamic equilibrium reached for an unsealed sample
when both the evaporation and imbibition flow rates are equal. The orange
arrows at the curved water-gas interface symbolize the evaporation of water
molecules.

Water vapor transport through the pore network is very slow compared to
capillarity-driven liquid flow, so the effects of evaporation through the top
surface during imbibition are negligible. Additionally, the upper dilatometer
glass pushrod effectively sealed the upper sample surface. In the absence of a
mechanism to counteract water uptake, the liquid will continue to rise
according to MLW dynamics until the pore space is completely filled (36, 43).
In Fig. 2 (top) we show such dynamics and qualitatively illustrate the
flattening of the menisci once the liquid reaches the upper porous block
surface and thus the upper monolithic glass pushrod.

### Impact of Evaporation on Elastocapillarity Dynamics: The Extreme Case of
Reaching an Evaporation-Imbibition Equilibrium

We performed a second imbibition experiment in which, unlike the first, the
side facets are not sealed against evaporation. Then, the evaporation fluxes
increase in proportion to the wetted, vapor-phase exposed part of the side
facets and thus in proportion to the capillary rise height. The imbibition
front then stops at a height well below the upper monolith surface as the
capillary rise reaches dynamic equilibrium, see movie S2 in the supporting
material. The inflow from the bulk liquid reservoir equals the evaporative
outflow to the bulk vapor phase, so that an evaporation-driven water transport
reminiscent of an (artificial) tree is achieved (44, 45, 46), see Fig. 2a.

In this case, the dynamics of capillary rise deviates from the classical
$\sqrt{t}$ scaling. As shown in Fig. 2 (bottom, left), the water reaches a
final capillary rise height, where the arrested imbibition front consists of
highly curved menisci. From the imbibition front to the water reservoir there
exists a linear pressure gradient that continues to exert a tensile
contraction on the wetted porous material. Therefore, no “Laplace expansion
jump” can be observed in the sample even though the imbibition front stopped
rising.

As outlined in the Supplementary Information the capillary-rise dynamics for
this scenario can be described by

$h(t)=h_{\rm eq}\cdot\sqrt{1-e^{-\frac{4q}{R\Phi_{\rm i}}t}},$ (2)

where $q$ is the evaporation rate of water per surface unit and $h_{\rm
eq}=\sqrt{\frac{K\Delta pR}{2q\mu}}$ represents the equilibrium capillary rise
or saturation height, reached at sufficiently long times. The hydraulic
permeability $K$ and every variable involved in $h_{\rm eq}$ except for $q$
can be obtained for the case of water imbibition in Vycor glass from Refs.
(36, 43).

Since the deformation dynamics evolve proportionally to the capillary rise,
the inclusion of a proportionality factor $C$ allows the use of Eq. 2 as a
fitting function, $\epsilon(t)=C\cdot\sqrt{1-e^{-\beta t}}$, for the
dilatometry data. Our experiment leads to $\beta=$
$2.1\text{\times}{10}^{-4}\text{\,}{\mathrm{s}}^{-1}$, which translates to an
evaporation rate $q=$ $0.05\text{\,}\mathrm{\SIUnitSymbolMicro
m}\text{\,}{\mathrm{s}}^{-1}$. This value is in reasonable agreement with
values reported in literature $q\approx$
$0.045\text{\,}\mathrm{\SIUnitSymbolMicro m}\text{\,}{\mathrm{s}}^{-1}$ for
bulk deionized water at similar conditions of humidity and temperature (47).
The determined evaporation rate $q$ leads to an equilibrium capillary rise of
$h_{\rm eq}=$ $14\text{\,}\mathrm{mm}$ or a filling height of
$74\text{\,}\mathrm{\char 37\relax}$ of the total length
($19\text{\,}\mathrm{mm}$), which is in reasonable agreement with the observed
averaged position of the arrested front visible in the snapshots in Fig. 2c.

Note that a first-order Taylor series expansion of Eq. 2 around $t=0$ leads
back to our MLW law or Eq. 1. This means that for early times ($t\ll t_{\rm
E}=\frac{R\varphi_{\rm i}}{4q}$), where $t_{\rm
E}=$4700\text{\,}\mathrm{s}$\pm{$600\text{\,}\mathrm{s}$}$ corresponds to the
time when the evaporative flow rate has reached 1-1/e=63% of the imbibition
flow rate, imbibition dominates over evaporation and the rising dynamics can
be described exclusively in terms of the classical LW square-root-of-time
dependent law (see Supplementary Information). This is evidenced by the
overlapping of the fitting curves in Fig. 2c, for $t<t_{\rm E}$, where the
continuous line indicates the fitting model based on Eq. 2 and the dashed line
denotes the $\sqrt{t}$-fitting based on Eq. 1. The latter is in reasonable
agreement with the deformation dynamics shown in Fig. 2 a, corresponding to a
non-evaporation scenario (sealed sample), and is very well described by the
MLW model. The fitting coefficients are $C_{\rm unsealed}=$$(8.53\pm
0.50)\text{\times}{10}^{-4}\text{\,}{\mathrm{s}}^{-1/2}$ and $C_{\rm sealed}=$
$(10.4\pm 0.2)\text{\times}{10}^{-4}\text{\,}{\mathrm{s}}^{-1/2}$.

In the derivation of Eq. 2 we consider only the liquid flow in the vertical
imbibition direction, so that a one-dimensional (1-D) problem results for
$h=h(t)$ and no transversal imbibition front height variation is considered.
However, a closer look at the shape of the imbibition front reveals that it
becomes increasingly curved with increasing imbibition time, with $h$ being
maximal in the center of the sample and decreasing symmetrically in the radial
direction towards the sample sides, as evident in the snapshot sequence in
Fig. S1 of the Supplementary. This suggests an increasing importance of the
transversal flow components in the imbibition phenomenology with increasing
imbibition time. In fact, the slight but systematic overestimation of the
strain by our simple 1-D model compared to the measured strain in the
intermediate time range from about $5000\text{\,}\mathrm{s}$ to the onset of
saturation at about $15\,000\text{\,}\mathrm{s}$ remains unexplained. Perhaps
it results from neglecting the curvature of the imbibition front and thus the
lateral flow components, since the average height of the curved front and thus
the averaged Bangham effect is smaller than expected in the simplified model
with a flat imbibition front.

### Quantitative Modeling of the Deformation Dynamics on the Porous-Medium
Scale

To achieve a quantitative description of the deformation kinetics we consider
the monolithic porous glass as a 3-D network of interconnected cylindrical
channels with an isotropic orientation distribution. We neglect the broadening
of the imbibition front and therefore assume that the imbibed part of the
network is completely filled with liquid up to the height $h(t)$.

As outlined in detail in Ref. (23), given the narrow pore-size distribution of
Vycor glass and the disordered channel-like structure of the pores, its
deformation can be considered uniform, isotropic, and a single pore-size
approximation can be used. The deformations are determined by a surface stress
contribution and by the hydrostatic fluid pressure in the pore $p_{\rm\rm F}$,
both acting as an effective loading pressure on the solid and normal to the
pore walls. These contributions result in the equations:

$\begin{split}\epsilon_{\mathrm{Bangham}}=-\frac{1}{M_{\mathrm{PL}}}\frac{\Delta
f}{r}\quad\hbox{and}\quad\epsilon_{\mathrm{Laplace}}=\frac{1}{M_{\mathrm{PL}}}p_{\rm
F},\end{split}$ (3)

where $M_{\mathrm{PL}}$ is the hydrostatic pressure-associated pore-load
modulus (13), $\Delta f$ is the released surface stress, i.e., the difference
in surface stress before and after imbibition, and $r=r_{p}-d$ is the
corrected pore radius, which appears in Eq. 1.

For isotropically distributed cylindrical pores, the total strain of a liquid-
filled representative volume $\epsilon^{*}$ is calculated then as a
combination of the two strains:

$\begin{split}\epsilon^{*}&=\frac{1}{M_{\mathrm{PL}}}\left(-\frac{\Delta
f}{r}+p_{\rm F}\right).\end{split}$ (4)

On the porous-medium scale, the fluid pressure increases linearly from the
negative, contractile Laplace pressure $p_{\rm L}$ at the position of the
advancing imbibition front $h(t)$ up to the bulk reservoir pressure $p_{\rm
0}$, while the change in surface stress per unit length for the imbibed part
of the pore network is constant along the sample height at any imbibition time
$t$. Thus the Laplace pressure-induced deformation is maximal at the
imbibition front and vanishes at the bulk reservoir. By contrast, the Bangham
effect is constant along the imbibed part of the matrix, so that the water-
filled part of the originally dry, cylindrical monolith deforms to a frustum
(see Fig. 1e for the corresponding trapezoidal deformation in side view). The
resulting overall deformation and thus relative sample length change $\Delta
L/L_{\rm 0}$ in the imbibition direction is thus proportional to the fraction
of imbibed sample ($h(t)/h_{\rm final}$). And because of the linearly
increasing magnitude of the fluid pressure from the bulk reservoir to the
imbibition front, it is only half as large as it would be for a porous solid
homogeneously experiencing the tensile pressure $p_{\rm L}$. Additionally
considering that for nanoporous materials the atmospheric pressure is much
smaller than the Laplace pressure, $p_{\rm 0}\ll p_{\rm L}$, we arrive at:

$\begin{split}\epsilon(t)&=-\frac{1}{M_{\mathrm{PL}}}\left(\frac{\gamma+\Delta
f}{r}\right)\frac{h(t)}{h_{\rm final}}.\\\ \end{split}$ (5)

At imbibition time $t_{c}$ the average position of the imbibition front is
equal to the height of the sample $L$ or, in other words, using Eq. 1,
$h(t_{c})=L$. For a single nanocapillary, the tensile Laplace pressure
vanishes suddenly at $t_{c}$, but for our porous matrix such a vanishing is
not instantaneous but gradual, since the vanishing dynamics is governed by the
broadened imbibition front. The Laplace pressure contribution starts to
decrease when the upper limit of the broadening front reaches $h(t_{1})=L$ and
continues to do so until its disappearance when the whole matrix is filled at
$t_{2}$. This time difference $t_{2}-t_{1}$ over which the Laplace pressure
vanishes is indicated in all of our figures as regime II (light blue
background). The extent of the Laplace jump is:

$\begin{split}\Delta\epsilon=\epsilon(t_{2})-\epsilon(t_{1})=\frac{p_{F}}{M_{\mathrm{P_{L}}}}=\\\
=-\frac{p_{L}}{2\cdot M_{\mathrm{PL}}}=-\frac{\gamma}{(r_{p}-d)\cdot
M_{\mathrm{PL}}}\end{split}$ (6)

Figure 3: Contributions to imbibition-induced deformation and influence of
pre-imbibition humidity. (a) Contributions to deformation of the porous-medium
along the imbibition direction that originate from contractile Laplace
pressure (purple) and expansive surface stress contributions (red) along with
the resulting overall deformation (green line), as calculated by a linear
superposition according to our elastocapillary model. The maximum values of
both contributions are represented by the dotted lines. (b) Strain isotherm
measurement for a Vycor glass rod $r_{p}=$ $3.4\text{\,}\mathrm{nm}$, $T=$
$25.8\text{\,}\mathrm{\SIUnitSymbolCelsius}$ and $p_{0}=$
$3.3\text{\,}\mathrm{kPa}$, with the logarithmic vanishing of the Laplace
pressure highlighted with a black line (adapted from Ref. (48)). Overlapping
with such data set we include our own experimental data (green points)
obtained by subtracting the maximum strain (Bangham contribution) in our
imbibition experiments to the maximum strain value in the isotherm
($0.2\text{\,}\mathrm{\char 37\relax}$) represented by the green dashed line.
The red dashed lines represent the measured strain in our dilatometry
experiment and the position along the Y-axis of the influence of the
adsorption-induced strain due to the humidity. (c) Water imbibition-induced
deformation in a nanoporous Vycor sample $r_{p}=$ $3.4\text{\,}\mathrm{nm}$
with pre-imbibition humidity $RH_{2}\approx$ $50\text{\,}\mathrm{\char
37\relax}$) (dark green points) compared to one (light green dots) performed
with lower pre-imbibition humidity ($RH_{1}\approx$ $5\text{\,}\mathrm{\char
37\relax}$). The strain jump and maximum strain are given by the red dashed
and dotted lines for $RH_{1}$ and $RH_{2}$, respectively. d) Schematic
illustrations of the interplay of negative hydrostatic pressure (purple) and a
sample-expanding surface stress release at the water-silica wall interface per
unit of sample length (red) for specific rise heights ($h$): (h1), (h2), (h3)
and (h4). Analogously, for a selected time $t_{s}$, in (e) this the interplay
is shown at the single-pore scale for two distinct pre-imbibition humidity
values. The white arrows represent the parabolic velocity profile for
pressure-driven flow in a cylindrical channel.

The individual contributions to the strain and consequently the total strain
are time dependent and evolve proportional to the MLW dynamics of Eq. 1. Fig.
3a shows a model of both strain contributions and, how the combination of both
allows reconstructing the observed deformation dynamics represented by the
green curve. This figure panel illustrates the competitive deformation process
and does not represent an actual experiment. In this way we understand the
imbibition-induced strain curve of water in nanoporous Vycor glass and why
upon filling, we observe a dynamic expansion proportional to $\sqrt{t}$; the
green curve acceptably describes the first stage of the process, prior to a
strain jump that lasts until time $t_{\rm c}$ of complete filling.

The strain jump in our dilatometry measurement can be quantified according to
Eq. 6 provided the pore-load modulus $M_{\mathrm{PL}}$ of the material is
known. $M_{\mathrm{PL}}$ is extractable through a sorption strain isotherm
measurement (23). The water adsorption-desorption-induced strain on nanoporous
Vycor rods was experimentally studied in the 1950s by McIntosh and Amberg
(48). We work with nanoporous Vycor hosts with comparable pore size and
porosity, and at identical temperature. The pore-load modulus of Vycor is
accessible via the Kelvin-Laplace equation upon reaching bulk liquid-vapor
coexistence. The McIntosh and Amberg data shows the logarithmic vanishing of
the Laplace pressure upon approaching bulk liquid coexistence and thus
vanishing menisci curvature (see Fig. 3b). It results in $M_{\mathrm{PL}}=$
$39.45\text{\,}\mathrm{GPa}$. Introducing this value to Eq. 6 while assuming
the bulk surface tension of water
($72\text{\,}\mathrm{mN}\text{\,}{\mathrm{m}}^{-1}$) we get a value for the
maximum Laplace pressure-induced strain of $0.036\text{\,}\mathrm{\char
37\relax}$. This is in reasonable agreement with the experimental value of
$0.032\text{\,}\mathrm{\char 37\relax}$ (see Fig. 3c). Here the final strain
value after the jump (dotted red line) corresponds to the Bangham effect
contribution to the strain.

An identical experiment is performed for a relative humidity of $\sim 5$% and
also shown in Fig. 3c. Again, the three different deformation regimes are
observed. Note, however, that the total relative expansion after complete
filling is $0.16\text{\,}\mathrm{\char 37\relax}$, almost twice as large as in
the experiment at $RH=$ $50\text{\,}\mathrm{\char 37\relax}$, while the strain
jump at complete filling is the same in both experiments.

Whereas the effect of the Laplace pressure on the strain upon imbibition is
predictable, the surface stress contribution is highly dependent on the pre-
imbibition atmosphere. Higher humidity values in our experimental environment,
prior to the start of the imbibition experiment, will induce larger pre-
imbibition adsorption of water molecules on the pore walls, leading to
expansion of the porous host (as illustrated in Fig. 3d). Such dependency is
evidenced by comparing the dilatometry experiments at different pre-imbibition
humidities. Again in Fig. 3b we show how for a certain relative humidity pre-
condition $RH_{1}$, the measurable imbibition-induced deformation (c) is
comparable to the strain difference between equilibrium states at $RH_{1}$ and
$RH_{f}=p/p_{0}=1$ (fully filled sample).

From the final strain values it is possible to calculate $\Delta f$ using Eq.
4. We find $\Delta f(RH=50\%)=$
$-116\text{\,}\mathrm{mJ}\text{\,}{\mathrm{m}}^{-2}$ and $\Delta f(RH=5\%)=$
$-206\text{\,}\mathrm{mJ}\text{\,}{\mathrm{m}}^{-2}$. Our simple semi-
empirical approach to calculate $\Delta f$ from $\epsilon$ is in reasonable
agreement with an effective medium-based description of surface stress-induced
mechanical deformation as proposed by Weissmüller et al. (49). For a network
of interconnected elongated fibers with circular cross-section they propose

$\epsilon=-\frac{\alpha\Delta f}{3K}\frac{1-\nu}{1-2\nu},$ (7)

with $\alpha$ being the volume-specific surface area and $K$ being the bulk
modulus. The $\Delta f$ values obtained in this way are
$-116\text{\,}\mathrm{mJ}\text{\,}{\mathrm{m}}^{-2}$ and
$-190\text{\,}\mathrm{mJ}\text{\,}{\mathrm{m}}^{-2}$ for $RH=50\%$ and
$RH=5\%$, respectively.

Considering the common case where the surface stress release equals the
difference between the surface energy of the dry solid and the wetted solid,
$\Delta f=\gamma_{\rm sl}-\gamma_{\rm sv}$ (sl: solid-liquid, sv: solid-vapor)
(50), Eq. 4 can be written like

$\epsilon^{*}=-\frac{1}{M_{\rm PL}}\frac{\gamma+\gamma_{\rm sl}-\gamma_{\rm
sv}}{r}=\frac{1}{M_{\rm PL}}\cdot\frac{S}{r}.$ (8)

where the spreading coefficient $S=\gamma-(\gamma_{\rm sl}+\gamma_{\rm s})$ at
the liquid meniscus contact line of the wetting theory (51); $S$ has to be
positive for imbibition to occur. Hence, Eq. 8 predicts always a positive
strain and we arrive at the important conclusion that for any capillarity-
driven, spontaneous imbibition process and for any liquid/porous solid
combination the Bangham effect prevails over the Laplace effect, i.e., in
regime I, an expansive behavior will always be observed. Note that our
calculation assumes a zero liquid/solid contact angle $\theta$, a
consideration that does not affect our reasoning. The inclusion of a finite
contact angle ($\theta>0$) implies the replacement of $\gamma$ by
$\gamma\cos{\theta}$ both for the calculation of the Laplace pressure and the
spreading coefficient. This only reduces the magnitude of the Laplace
expansion jump, but it will not result in a sign change of $\epsilon^{*}$,
since $S$ has always to be positive for spontaneous imbibition to occur.

Unfortunately, the surface energy values for dry amorphous silica and hydrous
amorphous silica interfaces reported in the literature are quite inconsistent,
presumably because they depend sensitively on any contamination, the thickness
of the adsorbed water layers, and the surface chemistry (silanol density and
hydroxilatation of the surface) (52, 53). However, for the surface energies
($\gamma$) reported in Ref. (52) ($\gamma_{\rm sl}=$
$260\text{\,}\mathrm{mJ}\text{\,}{\mathrm{m}}^{-2}$ and $\gamma_{\rm sv}=$
$130\text{\,}\mathrm{mJ}\text{\,}{\mathrm{m}}^{-2}$, respectively) we arrive
at a predicted surface stress release of $\Delta f=$
$-130\text{\,}\mathrm{mJ}\text{\,}{\mathrm{m}}^{-2}$ for our $RH=$
$50\text{\,}\mathrm{\char 37\relax}$ measurements, in reasonably good
agreement with our finding of
$-116\text{\,}\mathrm{mJ}\text{\,}{\mathrm{m}}^{-2}$. This also corroborates
the conclusion in Ref. (23) that for water-silica interfaces the surface
stress corresponds to the surface energy changes.

### Molecular Dynamics Simulations of Water Imbibition-Induced Deformation on
the Single-Nanopore Scale

Figure 4: Elastocapillary dynamics upon water imbibition on the single-
nanopore scale as inferred from molecular dynamics computer simulations. (a)
Snapshots from an exemplary simulation at selected imbibition times as
indicated in the figure. Silicon atoms are depicted in yellow, oxygen in red,
and hydrogen in white. The imbibing water is represented through the blue
colored surface. (b) Imbibition length $h$ of the water column in the silica
pores from one pore side in comparison with a $\sqrt{t}$ fit (black). (c) and
(d) Observed lateral $\epsilon_{\perp}$ and longitudinal
$\epsilon_{\parallel}$ strain in the pore matrix.

As demonstrated above, dilatometry experiments allow the study of the
macroscopic effects of imbibition-induced strain on porous host materials and
a quantification of the total strain. However, the individual strain
contributions originating from the superposition of the Bangham and Laplace
pressure effects on the single-pore scale can not be investigated with this
technique.

To better understand how these microscopic strain contributions accumulate to
yield the total macroscopically measured deformation and how they are related
to individual menisci movements in the pore space, Molecular Dynamics (MD)
computer simulations of the imbibition of water into silica nanopores on the
single-pore scale are conducted. Since in MD simulations atom-atom
interactions for each individual atom in the system are modeled they are well
suited to investigate the microscopic imbibition-induced strain contributions
on a fundamental level. The porous silica system used is based on the original
work of Refs. (54, 55) and has a diameter of around $3\text{\,}\mathrm{nm}$
and a wall thickness of around $1\text{\,}\mathrm{nm}$. The total length the
water can imbibe is about $12\text{\,}\mathrm{nm}$.

Fig. 4 shows the results of an ensemble of 20 MD simulations with independent
starting conditions; see movie S3 of one of the simulations in Supporting
Information. Fig. 4a displays snapshots of slices through a nanopore during
the imbibition process. The snapshots are arranged from left to right in
chronological order at times $0\text{\,}\mathrm{ps}$,
$183\text{\,}\mathrm{ps}$, $585\text{\,}\mathrm{ps}$ and
$780\text{\,}\mathrm{ps}$ as indicated by the dotted vertical lines in figures
4b, 4c, and 4d. In the third snapshot a zoom-in is provided that focuses on
the water meniscus formed at the imbibing water column. Note that with a
$3\text{\,}\mathrm{nm}$ pore diameter the silica pore size is only around one
order of magnitude larger than the size of a water molecule as visible from
the inset.

The imbibition length $h$ of the water column into the silica pore from one
pore side is shown as a function of imbibition time $t$ in Fig. 4b. The error
bars are omitted since the plot markers are larger than the standard deviation
of the simulation ensemble. In the simulations the imbibition length is
measured indirectly through the extent of the simulation box in the
longitudinal pore direction. The initial imbibition dynamics of around
$3\text{\,}\mathrm{ps}$ follows a linear trend, which is consistent with an
initial water inertia-governed dynamics during which the influence of the
water viscosity can be neglected (see Supporting Information for further
information). After the initial $3\text{\,}\mathrm{ps}$ the imbibition
dynamics exhibits a $\sqrt{t}$ dependence as shown by the black fit, which
confirms the validity of the LW-model even for pore sizes that only fit a few
adjacent water molecule layers. These results are in agreement with previous
MD simulations of nanopore imbibition for simple Lennard-Jones fluids and
polymer melts (56). The pore is completely filled with water after
$780\text{\,}\mathrm{ps}$. The measurement of the imbibition length through
the extent of the simulation box in the longitudinal pore direction assumes a
constant pore space volume. For the process of imbibition this is a good
approximation since the pore space volume only changes gradually. However, at
the end of the imbibition process a relatively strong lateral pore expansion
occurs (see Fig. 4c). This lateral expansion leads to a significantly
increased pore space volume that triggers an additional water inflow from the
water reservoir into the pore space, even though the pore is already entirely
filled by the imbibing water columns. The inflow of the water in turn reduces
the reservoir volume and leads to a shrinkage of the simulation box in the
longitudinal pore direction. This effect explains the additional increase of
measured imbibition length even after the water columns fill the entire pore
space.

The green markers in Fig. 4c and 4d display the dynamics of the lateral
$\epsilon_{\perp}(t)$ (perpendicular to the long pore axis) and longitudinal
$\epsilon_{\parallel}(t)$ (parallel to the long pore axis) strains,
respectively. Error bars indicate the standard deviations over the averaged 20
individual simulations. The use of different analysis methods to determine the
lateral and the longitudinal strains explains the different magnitudes of
standard deviation in the measured strains (see Materials and Methods). The
strong changes in strain during the first time steps of the simulation are in
response to the initial inertia-dominated imbibition and initial system
equilibration dynamics.

The temporal strain evolution, laterally $\epsilon_{\perp}(t)$ and
longitudinally $\epsilon_{\parallel}(t)$, can be divided in three regimes.
Regime I from $0\text{\,}\mathrm{ps}$ to about $780\text{\,}\mathrm{ps}$
displays an expansion of the host material with $\sqrt{t}$ dynamics as
indicated by the black curve fit. In agreement with our macroscopic
experiments on the porous-medium scale, the expansive Bangham effect dominates
in comparison to the contractile Laplace pressure effect on the single-pore
scale. However, the influence of the Laplace pressure effect is different for
the two strain directions. In the lateral direction the negative Laplace
pressure leads to a pronounced contractile effect. In contrast, the
longitudinal strain is only influenced by the Laplace pressure through the
Poisson effect. More precisely, the negative Laplace pressure leads to an
expansion in pore wall thickness, which, in turn through the Poisson effect,
leads to a contractile effect of small magnitude in the longitudinal pore
direction. At the end of regime I the menisci at the water columns vanished
upon complete filling of the pore space, which leads to a disappearance of the
contractile Laplace pressure and the resulting strain effects in the lateral
and longitudinal directions. Consequently the strain increases during the
relaxation of the porous material from $780\text{\,}\mathrm{ps}$ to
$1200\text{\,}\mathrm{ps}$ in regime II. Regime III occurs after the expansion
around $1200\text{\,}\mathrm{ps}$ when the porous material exhibits the final
observed strain created by the Bangham effect acting on the entire inner
surface of the nanopore.

Thus, overall the strain evolution at the single-pore scale is analogous to
that observed at the macroscopic porous-medium scale. In particular, by
comparison with the single-pore simulation, the Laplace pressure strain jump
observed when the imbibition front reaches the top of the Vycor matrix can be
attributed to a collective disappearance of menisci in Vycor.

Analogous to the macroscopic case, for the simulated material
$M_{\mathrm{PL}}$ can be calculated from Eq. 6 with the Laplace jump
$\Delta\epsilon=$ $1\text{\,}\mathrm{\char 37\relax}$. This results in a pore-
load modulus value of $M_{\mathrm{PL}}=$ $4.8\text{\,}\mathrm{GPa}$, an order
of magnitude smaller than the $39.45\text{\,}\mathrm{GPa}$ calculated for
Vycor glass. This is due to the significantly smaller silica pore wall
thicknesses in the simulation compared to the experiment, and also explains
why the magnitude of the imbibition-induced strains is an order of magnitude
larger than the experimental one. From the total strain we calculated $\Delta
f_{\rm simulation}=\Delta\gamma_{\rm simulation}=$
$-177\text{\,}\mathrm{mJ}\text{\,}{\mathrm{m}}^{-2}$ provided that
$p_{\mathrm{L}}=(\gamma\cos{\theta})/r_{p}$ and the contact angle is about
$\theta=$ $45\text{\,}\mathrm{\SIUnitSymbolDegree}$, as can be inferred from
the meniscus snapshots in Fig 4. We find that this value (starting from
vacuum) is the same order of magnitude as the calculated values of
$\Delta\gamma$ at low humidity in the macroscopic case (between
$-190\text{\,}\mathrm{mJ}\text{\,}{\mathrm{m}}^{-2}$ and
$-206\text{\,}\mathrm{mJ}\text{\,}{\mathrm{m}}^{-2}$). Thus the single-pore
simulations corroborate not only qualitatively, but also (semi-)quantitatively
the experimental findings on the porous-medium scale.

## Summary

We have studied experimentally and by computer simulation the deformation
behavior of a mesoporous silica monolith (Vycor) and a single cylindrical
nanopore, respectively, as a function of water imbibition. In contrast to the
expected Laplace pressure-induced contraction, we observed two distinct
imbibition-induced expansion kinetics, which can be directly attributed to the
advance, arrest, and disappearance of the imbibition front or single nanoscale
menisci, respectively. In particular, the surface stress release-induced
expansion known as the Bangham effect was found to dominate the Laplace
pressure-induced contraction in both cases. As we derive from general wetting
theory, this is the generic case for any spontaneous imbibition process and
any liquid/solid combination, provided that the surface stress release is
equal to the surface energy difference between the wetted and dry solid.

The observed competition between surface stress and Laplace pressure should be
relevant for any imbibition phenomenon in nanoporous media. For significantly
stiff systems, where deformation leads to negligible changes in pore size and
thus hydraulic permeability, simple LW deformation kinetics are expected. For
soft solids, on the other hand, the deformation could lead to linear hydraulic
resistance gradients and thus to deviations of both the liquid advancement
kinetics and the deformation dynamics from the classical LW behavior.

Our results show that relative length changes can be used to study the liquid
filling kinetics and thus the fabrication of the emerging class of liquid-
infused functional materials in detail, but also under in operando conditions
a study of the deformation behavior allows for monitoring the appropriate
filling state of a liquid-infused materials system.

## Materials and Methods

###  0.1 Humidity-Dependent Dilatometry Experiments

As porous host we work with cylindrical monoliths of porous Vycor glass
(Corning Glass, 7930). All of the samples are identical, only differing in
length, and were previously used by Simon Grüner. An extensive description of
such samples can be found in Ref. (36). Important properties such as the mean
pore radius $r_{p}=$ $4.9\text{\,}\mathrm{nm}$ and porosity $\Phi_{0}=0.3$ are
obtained from a water sorption isotherm measurement. Ref. (36) assumed that
there are two pre-adsorbed monolayers of water molecules of thickness $d=$
$0.5\text{\,}\mathrm{nm}$ attached to the pore walls prior to the sorption
isotherm measurements. Since we measure in comparable atmospheric conditions
we will assume the same value for our measurements. This magnitude of water
layer thickness is consistent with the reduced porosity $\Phi_{0}=0.27$ that
we estimate by measuring the sample mass difference between the beginning and
end of the experiment. The tortuosity value of $\tau=3.6$ is also taken from
the above mentioned study.

The samples are pre-cleaned by immersing them in a mixture of concentrated
sulfuric acid (H2SO4) and a $30\text{\,}\mathrm{\char 37\relax}$ hydrogen
peroxide solution (H2O2) with a 3:1 ratio (Piranha etch solution). The
strongly oxidizing strength of this solution removes most organic matter from
the pore walls while hydroxylating most surfaces (i.e., adding OH groups) in
the process. The porous Vycor monoliths are immersed in the solution for 3
weeks and subsequently for 3 days in ultra pure water. After drying the
process is repeated one more time. Before the experiment the samples are dried
by introducing them in a vacuum chamber for 48 hours. Using pure water for
every experiment avoids the necessity to repeat the whole cleaning process
between different measurements, hence in this case only the drying is to be
performed multiple times.

For the measurements we use a vertically oriented dilatometer (Linseis L75
VS500) with a precision up to $0.1\text{\,}\mathrm{\SIUnitSymbolMicro m}$ and
a temporal resolution of $1\text{\,}\mathrm{s}$.

Humidity control during our experiments is achieved by using a custom-built
humidity chamber that can be integrated into the dilatometry setup. The
desired relative humidity can be maintained by controlling the incoming air
flow, which is a mixture of pure dry nitrogen and water vapor. The samples are
exposed to the desired pre-imbibition humidity, where they remain until
equilibrium is reached (monitored by dilatometry). After equilibration, the
initially empty liquid reservoir is filled by injecting liquid through an
external syringe until the water level reaches the sample, initiating the
capillary filling process.

###  0.2 Molecular Dynamics Simulations

Average quantities from the imbibition process are obtained from 20 individual
Molecular Dynamics simulations with alternating starting velocities of the
water molecules conducted with the Large-scale Atomic/Molecular Massively
Parallel Simulator (LAMMPS) (57). As host material, a periodically repeated
amorphous silica pore initially introduced in Ref. (54, 55) is used. The
silica-silica interactions are modeled by a modified Demiralp potential (58),
water-silica interactions are modelled as described in Ref. (59, 60), and
water-water interactions are modelled with the TIP4P/2005 water model (61).
The simulated pore structure consists of amorphous silica $\mathrm{SiO_{2}}$.
The surface of the porous structure is saturated with hydroxyl groups of
$7.23\text{\,}{\mathrm{nm}}^{-2}$ density. The total longitudinal pore length
is around $13\text{\,}\mathrm{nm}$ along the $z$ axis of the simulation box.
However, in the simulation the water column starts imbibing from just inside
the pore leaving around $12\text{\,}\mathrm{nm}$ of possible imbibition
length. In the lateral direction, that is the $x$ and $y$ direction of the
simulation box, the pores are hexagonally shaped and have a diameter of around
$3\text{\,}\mathrm{nm}$ while the thickness of the pore walls is around
$1\text{\,}\mathrm{nm}$. The imbibition process is simulated with periodic
boundary conditions in all directions in an isothermal-isobaric ensemble (NpT)
at $1\text{\,}\mathrm{bar}$ and $300\text{\,}\mathrm{K}$ employing a
Nosé–Hover barostat (62, 63, 64, 65). To trigger the imbibition process a
water reservoir outside of the nanopores at the longitudinal pore ends is
created. Due to the periodic boundary conditions the water can enter the pore
from both sides. The $x$ and $y$ axes are coupled such that the rate of
expansion or contraction of the simulation box is equal for both axes. The
total simulated time is $2000\text{\,}\mathrm{ps}$ employing a
$1\text{\,}\mathrm{fs}$ timestep (see Supporting information for further
information).

The measured strains are defined as Cauchy strains
$\epsilon(t)=[L(t)-L(t=t_{0})]/L(t=t_{0})$ where $L(t=t_{0})$ constitutes the
pore extensions at the start of the simulation. The radial strain
$\epsilon_{\perp}(t)$ is measured indirectly through the change of the
simulation box extension $L_{\mathrm{box}}$ in the lateral direction. In
contrast the longitudinal strain $\epsilon_{\parallel}(t)$ is measured by the
distances between the groups of atoms forming the last
$0.5\text{\,}\mathrm{nm}$ of bulk material of the pores on each end (see
Supporting information for further information). The thermal motions of the
atoms lead to strongly varying distances during the simulation. These
fluctuations account for the high standard deviation in the measured
longitudinal strain represented in the error bars in Fig. 4.

Patrick Huber dedicates this work to Klaus Knorr, Mainz, Germany, on the
occasion of his 80th birthday, with immense gratitude for his mentorship in
physics and for introducing him to the highly fascinating field of nanoporous
media. We are also indebted to Klaus Knorr and Jörg Weißmüller (Hamburg) for a
critical reading of the manuscript and for their helpful comments. J.S., P.H.
and R.M. acknowledge funding by the Deutsche Forschungsmeinschaft (DFG, German
Research Foundation) within the Research Training Group TRG 2462 ‘Processes in
natural and technological Particle-Fluid-Systems (PintPFS)’ (Project No.
390794421) and within the Collaborative Research Centre CRC 986 ’Tailor-Made
Multi-Scale Materials Systems’ (Project No.192346071). L.D. was supported by
the Data Science in Hamburg HELMHOLTZ Graduate School DASHH, Helmholtz
Association Grant-No. HIDSS-0002. L.G. profited from the DFG research grant
”Dynamic Electrowetting at Nanoporous Surfaces: Switchable Spreading,
Imbibition, and Elastocapillarity”, Project number 422879465 (SPP 2171), and
from the cluster of excellence ”Understanding written artifacts” (EXC 2176).
This research was supported in part through the Maxwell computational
resources operated at Deutsches Elektronen-Synchrotron DESY, Hamburg, Germany.
M.F and P.H. thank the Free and Hanseatic City of Hamburg for funding within
the research group ´Control of the special properties of water in nanopores‘.
We also acknowledge the scientific exchange and support of the Centre for
Molecular Water Science CMWS, Hamburg (Germany).

## References

  * (1) Yao X, et al. (2013) Adaptive fluid-infused porous films with tunable transparency and wettability.  Nature Materials 12(6):529–534.
  * (2) Wang W, et al. (2018) Multifunctional ferrofluid-infused surfaces with reconfigurable multiscale topography.  Nature 559(7712):77–82.
  * (3) Brinker M, Huber P (2022) Wafer-Scale Electroactive Nanoporous Silicon: Large and Fully Reversible Electrochemo-Mechanical Actuation in Aqueous Electrolytes.  Advanced Materials 34:2105923.
  * (4) Sentker K, et al. (2019) Self-assembly of liquid crystals in nanoporous solids for adaptive photonic metamaterials.  Nanoscale 11(48):23304–23317.
  * (5) Cencha LG, Dittrich G, Huber P, Berli CLA, Urteaga R (2020) Precursor Film Spreading during Liquid Imbibition in Nanoporous Photonic Crystals.  Physical Review Letters 125(23):234502.
  * (6) Emmerich T, et al. (2022) Enhanced nanofluidic transport in activated carbon nanoconduits.  Nature Materials pp. 1–8.
  * (7) Simon P, Gogotsi Y (2008) Materials for electrochemical capacitors.  Nature Materials 7:845–854.
  * (8) Zhang J, Chen B, Chen X, Hou X (2021) Liquid-Based Adaptive Structural Materials.  Advanced Materials 2005664\.
  * (9) Wong TS, et al. (2011) Bioinspired self-repairing slippery surfaces with pressure-stable omniphobicity.  Nature 477(7365):443–447.
  * (10) Meehan FT, Hardy WB (1927) The expansion of charcoal on sorption of carbon dioxide.  Proceedings of the Royal Society of London. Series A, Containing Papers of a Mathematical and Physical Character 115(770):199–207.
  * (11) Bangham DH, Fakhoury N (1928) The expansion of charcoal acompanying sorption of gases and vapours.  Nature 122:681–682.
  * (12) Scherer GW (1986) Dilatation of Porous Glass.  Journal of the American Ceramic Society 69(6):473–480.
  * (13) Prass J, Mueter D, Fratzl P, Paris O (2009) Capillarity-driven deformation of ordered nanoporous silica.  Applied Physics Letters 95(8):83121.
  * (14) Gor GY, Neimark AV (2010) Adsorption-Induced Deformation of Mesoporous Solids.  Langmuir 26(16):13021–13027.
  * (15) Schappert K, Pelster R (2014) Unexpected Sorption-Induced Deformation of Nanoporous Glass: Evidence for Spatial Rearrangement of Adsorbed Argon.  Langmuir 30(46):14004–14013.
  * (16) Balzer C, et al. (2014) Relationship between pore structure and sorption-induced deformation in hierarchical silica-based monoliths.  Zeitschrift fuer Physikalische Chemie 229:1189–1209.
  * (17) Grosman A, Puibasset J, Rolley E (2015) Adsorption-induced strain of a nanoscale silicon honeycomb.  EPL (Europhysics Letters) 109(5):56002.
  * (18) Gor GY, Huber P, Bernstein N (2017) Adsorption-induced deformation of nanoporous materials—A review.  Applied Physics Reviews 4(1):011303.
  * (19) Chen M, Coasne B, Guyer R, Derome D, Carmeliet J (2019) Molecular Simulation of Sorption-Induced Deformation in Atomistic Nanoporous Materials.  Langmuir 35(24):7751–7758.
  * (20) Yang Q, et al. (2020) Capillary condensation under atomic-scale confinement.  Nature 588(7837):250–253.
  * (21) Harrellson SG, et al. (2023) Hydration solids.  Nature 619(7970):500–505.
  * (22) Gor GY, et al. (2015) Elastic response of mesoporous silicon to capillary pressures in the pores.  Applied Physics Letters 106:1–13.
  * (23) Gor GY, Bernstein N (2016) Revisiting Bangham’s law of adsorption-induced deformation: changes of surface energy and surface stress.  Physical Chemistry Chemical Physics 18(14):9788–9798.
  * (24) Gor GY, et al. (2015) Elastic response of mesoporous silicon to capillary pressures in the pores.  Applied Physics Letters 106(26):261901.
  * (25) Duprat C, Aristoff J, Stone HA (2011) Dynamics of elastocapillary rise.  Journal of Fluid Mechanics 679:641–654.
  * (26) Weijs JH, Andreotti B, Snoeijer JH (2013) Elasto-capillarity at the nanoscale: On the coupling between elasticity and surface energy in soft solids.  Soft Matter 9(35):8494–8503.
  * (27) Tortora M, et al. (2021) Giant Negative Compressibility by Liquid Intrusion into Superhydrophobic Flexible Nanoporous Frameworks.  Nano Letters 21(7):2848–2853.
  * (28) Michel L, et al. (2022) Bowtie-Shaped Deformation Isotherm of Superhydrophobic Cylindrical Mesopores.  Langmuir 38(1):211–220.
  * (29) Hoberg TB, Verneuil E, Hosoi AE (2014) Elastocapillary flows in flexible tubes.  Physics of Fluids 26(12):122103.
  * (30) Ha J, et al. (2018) Poro-elasto-capillary wicking of cellulose sponges.  Science Advances 4(3):1–7.
  * (31) Siddique JI, Anderson DM, Bondarev A (2009) Capillary rise of a liquid into a deformable porous material.  Physics of Fluids 21(1):013106.
  * (32) Levitz P, Ehret G, Sinha SK, Drake JM (1991) Porous Vycor glass: The microstructure as probed by electron microscopy, direct energy transfer, small-angle scattering, and molecular adsorption.  J. Chem. Phys. 95:6151.
  * (33) Huber P, Knorr K (1999) Adsorption-desorption isotherms and x-ray diffraction of Ar condensed into a porous glass matrix.  Phys. Rev. B 60(18):12657.
  * (34) Levitz P (1998) Off-lattice reconstruction of porous media: Critical evaluation, geometrical confinement and molecular transport.  Advances in Colloid and Interface Science.
  * (35) Gommes CJ (2018) Stochastic models of disordered mesoporous materials for small-angle scattering analysis and more.  Microporous and Mesoporous Materials 257:62–78.
  * (36) Gruener S, Hofmann T, Wallacher D, Kityk AV, Huber P (2009) Capillary rise of water in hydrophilic nanopores.  Physical Review E - Statistical, Nonlinear, and Soft Matter Physics 79(6):067301.
  * (37) Gruener S, et al. (2012) Anomalous front broadening during spontaneous imbibition in a matrix with elongated pores.  Proceedings of the National Academy of Sciences of the United States of America 109(26):10245–50.
  * (38) Gruener S, Hermes HE, Schillinger B, Egelhaaf SU, Huber P (2015) Capillary rise dynamics of liquid hydrocarbons in mesoporous silica as explored by gravimetry, optical and neutron imaging: Nano-rheology and determination of pore size distributions from the shape of imbibition fronts.  Colloids and Surfaces A: Physicochemical and Engineering Aspects 496:13–27.
  * (39) Page JH, Liu J, Abeles B, Deckman HW, Weitz DA (1993) Pore-Space Correlations in Capillary Condensation in Vycor.  Phys. Rev. Lett. 71:1216–1219.
  * (40) Amberg CH, McIntosh R (1952) a Study of Adsorption Hysteresis By Means of Length Changes of a Rod of Porous Glass.  Canadian Journal of Chemistry 30(12):1012–1032.
  * (41) Gruener S, Huber P (2011) Imbibition in mesoporous silica: rheological concepts and experiments on water and a liquid crystal.  J. Phys.: Condens. Matter 23(18):184109.
  * (42) Schlaich A, Kappler J, Netz RR (2017) Hydration Friction in Nanoconfinement: From Bulk via Interfacial to Dry Friction.  Nano Letters 17(10):5969–5975.
  * (43) Maillet B, Dittrich G, Huber P, Coussot P (2022) Diffusionlike Drying of a Nanoporous Solid as Revealed by Magnetic Resonance Imaging.  Physical Review Applied 10(1):1.
  * (44) Seker E, Begley MR, Reed ML, Utz M (2008) Kinetics of capillary wetting in nanoporous films in the presence of surface evaporation.  Applied Physics Letters 92(1):013128.
  * (45) Wheeler TD, Stroock AD (2008) The transpiration of water at negative pressures in a synthetic tree.  Nature 455:208.
  * (46) Camplisson CK, Schilling KM, Pedrotti WL, Stone HA, Martinez AW (2015) Two-ply channels for faster wicking in paper-based microfluidic devices.  Lab on a Chip 15(23):4461–4466.
  * (47) Huang Z, et al. (2021) Fast Water Evaporation from Nanopores.  Advanced Materials Interfaces 8(14).
  * (48) Amberg CH, Mcinthosh R (1952) A Study of Adsorption Hysteresis By Means of Length Changes of A Rod of Porous Glass.  Canadian Journal of Chemistry-revue Canadienne De Chimie 30(12):1012–1032.
  * (49) Weissmüller J, Duan HL, Farkas D (2010) Deformation of solids with nanoscale pores by the action of capillary forces.  Acta Materialia 58(1):1–13.
  * (50) Gor GY, Neimark AV (2010) Adsorption-induced deformation of mesoporous solids.  Langmuir : the ACS journal of surfaces and colloids 26(16):13021–13027.
  * (51) de Gennes PG, Brochard-Wyart F, Quere D (2004) Capillarity and Wetting Phenomena: Drops, Bubbles, Pearls, Waves.  (Springer, New York).
  * (52) Brunauer S, Kantro DL, Weise CH (1956) The surface energies of amorphous silica and hydrous amorphous silica.  Canadian Journal of Chemistry 34(10):1483–1496.
  * (53) Díaz D, et al. (2022) How water wets and self-hydrophilizes nanopatterns of physisorbed hydrocarbons.  Journal of Colloid and Interface Science 606:57–66.
  * (54) Ugliengo P, et al. (2008) Realistic Models of Hydroxylated Amorphous Silica Surfaces and MCM-41 Mesoporous Material Simulated by Large-scale Periodic B3LYP Calculations.  Advanced Materials 20(23):4579–4583.
  * (55) Delle Piane M, Corno M, Pedone A, Dovesi R, Ugliengo P (2014) Large-scale B3LYP simulations of ibuprofen adsorbed in MCM-41 mesoporous silica as drug delivery system.  Journal of Physical Chemistry C 118(46):26737–26749.
  * (56) Dimitrov DI, Milchev A, Binder K (2007) Capillary rise in nanopores: Molecular dynamics evidence for the Lucas-Washburn equation.  Phys. Rev. Lett. 99:54501.
  * (57) Thompson AP, et al. (2022) LAMMPS - a flexible simulation tool for particle-based materials modeling at the atomic, meso, and continuum scales.  Computer Physics Communications 271:108171.
  * (58) Meißner RH, Schneider J, Schiffels P, Colombi Ciacchi L (2014) Computational prediction of circular dichroism spectra and quantification of helicity loss upon peptide adsorption on silica.  Langmuir 30(12):3487–3494.
  * (59) Cole DJ, Payne MC, Csányi G, Spearing SM, Ciacchi LC (2007) Development of a classical force field for the oxidized Si surface: Application to hydrophilic wafer bonding.  Journal of Chemical Physics 127(20):204704.
  * (60) Butenuth A, et al. (2012) Ab initio derived force-field parameters for molecular dynamics simulations of deprotonated amorphous-SiO2/water interfaces.  physica status solidi (b) 249(2):292–305.
  * (61) Abascal JL, Vega C (2005) A general purpose model for the condensed phases of water: TIP4P/2005.  The Journal of Chemical Physics 123(23):234505.
  * (62) Martyna GJ, Tobias DJ, Klein ML (1994) Constant pressure molecular dynamics algorithms.  The Journal of Chemical Physics 101(5):4177–4189.
  * (63) Parrinello M, Rahman A (1981) Polymorphic transitions in single crystals: A new molecular dynamics method.  Journal of Applied Physics 52(12):7182–7190.
  * (64) Coregliano LN, et al. (2006) A Liouville-operator derived measure-preserving integrator for molecular dynamics simulations in the isothermal–isobaric ensemble.  Journal of Physics A: Mathematical and General 39(19):5629.
  * (65) Shinoda W, Shiga M, Mikami M (2004) Rapid estimation of elastic constants by molecular dynamics simulation under constant stress.  Physical Review B 69(13):134103.


# Efficient Facial Landmark Detection for Embedded Systems

###### Abstract

This paper introduces the Efficient Facial Landmark Detection (EFLD) model,
specifically designed for edge devices confronted with the challenges related
to power consumption and time latency. EFLD features a lightweight backbone
and a flexible detection head, each significantly enhancing operational
efficiency on resource-constrained devices. To improve the model’s robustness,
we propose a cross-format training strategy. This strategy leverages a wide
variety of publicly accessible datasets to enhance the model’s
generalizability and robustness, without increasing inference costs. Our
ablation study highlights the significant impact of each component on reducing
computational demands, model size, and improving accuracy. EFLD demonstrates
superior performance compared to competitors in the IEEE ICME 2024 Grand
Challenges PAIR Competition, a contest focused on low-power, efficient, and
accurate facial-landmark detection for embedded systems, showcasing its
effectiveness in real-world facial landmark detection tasks. The code is
available at https://github.com/072jiajia/efficient-fld.

##  1 Introduction

Facial landmark detection plays a crucial role in the face processing
pipeline, serving key applications such as virtual face reenactment, emotion
recognition, and driver status monitoring. Despite the advancements and
successes of deep learning models in addressing these challenges, deploying
them on edge devices, especially smartphones, presents significant issues.
These challenges, including power consumption and time latency, can severely
degrade the user experience.

Existing works [1, 7, 12, 16] propose methods that first predict a heatmap for
each facial landmark, followed by post-processing each heatmap to determine
the corresponding facial landmark position. These methods encounter long
inference times and high computational costs due to their dense prediction,
especially for models using an hourglass structure [11]. To reduce the model’s
resource requirements, the Practical Facial Landmark Detector (PFLD) [3] was
introduced. It directly predicts the position of each facial landmark, thereby
reducing the computational cost of dense predictions and eliminating the need
for post-processing. While compromising on prediction precision, PFLD achieves
a promising balance between prediction accuracy and computational cost, making
it suitable for edge devices. However, further reduction in resource
requirements remains a significant challenge.

In addition to computational demands on edge devices, facial landmark
detection faces the fundamental challenge of model robustness. A
straightforward approach to enhance robustness is to augment the training data
with additional publicly available datasets, allowing the model to learn from
a more diverse range of data. Nevertheless, this approach is not directly
applicable in this task due to variations in landmark formats across different
datasets. For instance, the WFLW dataset [13] employs a 98-point format, Menpo
[15] uses a 68-point format, and the IVSLab Facial-Landmark Detection Dataset
(IVSFD), provided by the competition host, adopts a 51-point format. While
these datasets offer valuable training data, the discrepancies in landmark
formats present a challenge in leveraging data from multiple sources.

To address these challenges, we propose the Efficient Facial Landmark
Detection (EFLD) model, which consists of three key components: the efficient
backbone network, the flexible facial landmark detection head network, and the
cross-format training strategy. The efficient backbone and the flexible facial
landmark detection head are designed to minimize time latency, power
consumption, and memory usage on edge devices. The cross-format training
strategy, on the other hand, is proposed to efficiently utilize existing
publicly available datasets, thereby enhancing the generalizability of our
model without increasing the inference cost.

The main contributions of this work are as follows:

  * •

We introduce an efficient backbone network structure and a facial landmark
detection head network structure that reduce time latency, power consumption,
and memory usage on edge devices.

  * •

We propose a novel cross-format training strategy that utilizes existing
publicly available datasets, even with different facial landmark formats, to
enhance model performance and generalizability without increasing the
inference cost.

  * •

EFLD demonstrates superior performance in the IEEE ICME 2024 Grand Challenges
PAIR Competition, a contest focuses on Low-power, Efficient, and Accurate
Facial-Landmark Detection for Embedded Systems, showcasing its effectiveness
in real-world facial landmark detection tasks.

Fig. 1: Model overview of our proposed method. Our method consists of three
major components: (a) an efficient backbone network that transforms each input
image into a feature vector, (b) multiple facial landmark detection head
networks predicting facial landmarks in various formats, and (c) a cross-
format training strategy that supports training across different facial
landmark formats. When exporting our model for application, we only include
the 51-point component in the exported model.

##  2 Related Works

###  2.1 Facial Landmark Detection

Facial landmark detection has significantly evolved [6] over the years, giving
rise to the development of two primary methodological approaches: direct
landmark regression and heatmap-based methods. Direct landmark regression
methods [3] predict the positions of facial landmarks directly from images,
providing faster inference times and suiting them for edge devices.
Conversely, heatmap-based methods [1, 7, 12, 16] produce 2D heatmaps for each
landmark, subsequently post-processed to determine the corresponding facial
landmark position. Although this approach can result in more accurate
predictions, it demands more computational resources due to its dense
predictions and the additional processing steps involved.

For our proposed method, we adopt direct landmark regression, favored for its
rapid inference time. This choice aligns with our objective to enhance
operational efficiency while maintaining high accuracy in landmark detection.

###  2.2 Lightweight Model Architecture

Deep learning models have demonstrated remarkable capabilities across a
diverse range of tasks. However, deploying these models on edge devices
presents significant challenges, particularly in terms of power consumption
and time latency, which are critical constraints for devices like smartphones.

To mitigate these issues, researchers have developed several innovative
strategies in recent years, including model quantization, efficient
operations, network pruning, knowledge distillation, and neural architecture
search, etc. In our approach, we utilize the following two methods:
Model Quantization: This technique involves reducing the numerical precision
of the model’s parameters from floating-point to lower-bit representations,
such as 16-bit half-precision or 8-bit integers. Despite a potential drop in
performance due to the reduced precision, this modification significantly
lowers the computational demands and memory usage of the model.
Efficient Operations: We reduce the overall computational complexity of our
models by employing techniques such as depthwise separable convolutions [2]
and feature concatenation [5, 8]. Depthwise separable convolutions reduce
computational cost by dividing a standard convolutional layer into a
combination of a 1-by-1 convolution and a depthwise convolution. For feature
concatenation, we specifically employ the One-Shot Aggregation (OSA) Module
[8], which concatenates each layer’s output to reduce computational cost.
These methods significantly enhance the efficiency of our model.

Fig. 2: Detailed architecture of each components The detailed architecture of
the proposed Efficient-OSA (EOSA) module, decoder, and the facial landmark
detection head network. Each module is lightweight due to the use of efficient
operations such as feature concatenation layers and depthwise convolutional
layers.

##  3 Methodology

In this section, we first explain the comprehensive methodology underpinning
our proposed model in section 3.1. Then, we specify the three major components
of our method, including 1) the efficient backbone network structure in
section 3.2, 2) the flexible facial landmark detection head network in section
3.3, and 3) the cross-format training strategy in section 3.4. These
components work harmoniously to ensure the model’s performance is both
lightweight for edge devices and robust for the performance.

###  3.1 Method Overview

As illustrated in figure 1, our proposed method employs the direct landmark
regression approach. Initially, the input image is processed through the
backbone model, which consists of four Efficient-OSA (EOSA) modules, a
modified-version of OSA module. The resultant feature map from the backbone is
then decoded into a feature vector that represents the facial features. This
feature vector is subsequently input into various facial landmark detection
head networks, each tailored to generate predictions for different landmark
formats. We apply the cross-format training strategy, optimizing our model on
multiple publicly available datasets with diverse landmark formats
simultaneously. When deploying our model, we only export the modules necessary
for the application and remove unrelated modules, thus reducing the required
resources.

###  3.2 The Lightweight Backbone Network

Our backbone network consists of four EOSA modules followed by a decoder. As
illustrated in figure 2, each EOSA module inputs a feature map and outputs a
downscaled feature map with stride of 2. This structure incorporates elements
of OSA module [8] and adds an extra convolutional layer. The design aims to
harness OSA module’s lightweight and efficient computation capabilities to
extract more complex features, while the added convolutional layer focuses on
capturing low-level features. To obtain the downscaled feature map, the first
convolutional layer of the OSA module has a stride of 2, followed by
$N_{blocks}-1$ convolutional layers with a stride of 1. Each layer outputs a
downscaled feature map with a channel number of $F_{OSA}$. Furthermore, for
the added convolutional layer, we employ depthwise separable convolutional
layers to further decrease the computational cost of OSA module. This
integration effectively reduces the model size with only a minimal impact on
performance.

The decoder within the backbone is designed straightforwardly. It comprises a
$1\times 1$ convolutional layer with an output channel number of
$F^{decoder}_{out}$, followed by a depthwise convolution layer with a filter
size of $H\times W$, where $H$ and $W$ represent the height and width of the
feature map, respectively. Thus, the depthwise convolution layer outputs a
feature vector with dimension $F^{decoder}_{out}$, representing the features
of the face. This setup decodes the feature map into a feature vector through
the combination of $1\times 1$ convolution and depthwise convolution, thereby
reducing computational resources.

###  3.3 The Facial Landmark Detection Head

As depicted in figure 2, each detection head takes a feature vector
representing the face and outputs the positions of facial landmarks. Inspired
by [5], the first part of the architecture consists of $N_{head}$ linear
layers followed by concatenation steps. Each linear layer outputs a feature
vector with a dimension of $F_{head}$, which is then concatenated with the
input feature vector. The second part of the architecture is a linear layer
responsible for predicting the positions of all landmarks based on the
concatenated feature vector. Notably, an EFLD model may include multiple
detection head networks, each tailored for predicting a specific type of
facial landmark. This design enables the model to predict multiple types of
facial landmarks from a single input image, significantly enhancing the
versatility and utility of the EFLD model across various applications.
Additionally, when exporting our model to applications, since each detection
head is responsible for a single type of facial landmark format, we may only
export the necessary heads to our applications, thereby eliminating
unnecessary computations.

###  3.4 The Cross-Format Training Strategy

The cross-format training strategy is designed to improve the model’s
generalizability and robustness by joint-training on multiple datasets.
Specifically, for training on multiple datasets simultaneously, we create a
detection head for each type of facial landmark format present in the training
data. These datasets are then shuffled, and batch training is applied to this
training pipeline. This approach addresses the diversity in facial landmark
formats, such as the 98-point format used by the WFLW dataset [13], the
68-point format of Menpo [15], and the 51-point format adopted by the IVSLab
Facial-Landmark Detection dataset (IVSFD), which is provided by the
competition host. Our model undergoes simultaneous training on these datasets.
With multiple detection heads tailored for specific landmark formats,
including 51-point, 68-point, and 98-point, the model calculates the loss
function for annotations when available and assigns a zero loss for missing
annotations.

This strategy enables efficient and effective learning from a wide range of
data sources, significantly enhancing the model’s ability to perform
consistently across different landmark configurations. Additionally, this
strategy is highly flexible, as the additional training resources can be
extended without limit. It allows the model to adapt and accurately predict
facial landmarks in a wide array of facial expressions, environments, and
landmark formats, making it highly effective in diverse real-world
applications.

Team | Accuracy | G FLOPs | Size | Time | Power | INT8 | Score
---|---|---|---|---|---|---|---
jijiawu (Ours) | 18.47 | 0.02 | 0.17 | 1.08 | 1.93 | ✓ | 2741.92
yuhs1 | 19.30 | 0.08 | 2.25 | 8.12 | 1.64 |  | 7.78
JordanHsieh | 19.82 | 0.15 | 1.58 | 6.98 | 1.79 |  | 6.86
hiwu.dery | 23.10 | 0.15 | 1.46 | 42.97 | 2.25 |  | 1.10
O_S_O | 25.50 | 0.38 | 2.48 | 15.85 | 2.07 |  | 0.83
ppp458732 [4] | 22.30 | 1.16 | 2.64 | 67.35 | 2.27 |  | 0.048
Weiiiii_0123 | 28.06 | 11.15 | 4.12 | 17.55 | 2.04 |  | 0.017
dar814 | 4.09 | 0.15 | 10.00 | 26.10 | 9.34 |  | 0.011
YUJIE47 | 16.13 | 2.61 | 4.55 | 151.55 | 9.84 |  | 0.0008
asdggg | 19.95 | 4.34 | 28.46 | 110.03 | 2.70 |  | 0.0005
ed1122 | 16.96 | 33.29 | 54.15 | 87.77 | 2.87 |  | 0.00004
Table 1: The proposed method is compared with ten competitors’ methods in the
final round of the contest across various metrics: accuracy, model complexity
(G FLOPs), model size (Size), inference time (Time), power consumption
(Power), the application of int8-quantization (INT8), and final score (Score).
The best method for each metric is indicated in bold.

##  4 Experiments

###  4.1 Implementation Details

Our training pipeline is developed upon TF-Models [14].
Model Hyperparameters. The configuration for the four EOSA modules in the
backbone network, $(F_{OSA},N_{OSA})$, is set as $(4,2)$, $(8,3)$, $(16,3)$,
$(16,3)$, with only the first EOSA module utilizing conventional convolution
with $F_{conv}=8$. The remaining modules employ depthwise separable
convolutional layers with $F_{conv}=8$, $F_{conv}=16$, and $F_{conv}=32$,
respectively. The output feature dimension of the decoder is set to $256$. The
detection head is composed of three linear blocks, each with $32$ units,
followed by concatenation.
Training datasets. Our model is trained across multiple datasets: the IVSFD
dataset, which includes 4437 images; the training set of Menpo [15],
comprising 6655 images; and the WFLW dataset [13], totaling 10000 images.
Training Hyperparameters. We apply the Normalized Mean Error (NME), normalized
by the inter-ocular distance, as our training loss. The input images are
resized to 128, and trained with batch size of $512$ on one NVIDIA $2080$Ti
GPU, with learning rate at $0.001$ using the AdamW [9] optimizer. The training
spans $1500$ epochs, utilizing a cosine schedule for the learning rate
adjustment.
Export TFLite Model. For the final version of our model, we export only the
backbone model along with the 51-point detection head. To optimize our model’s
performance, we implement int8 post-training quantization, conducting
calibration on all images from the IVSFD dataset.
Inference. For each input data, we first utilize MediaPipe [10], which
provides a cross-platform face detection function to identify faces in the
images. Then, our TFLite model is used to predict the facial landmarks for
each detected face.

###  4.2 Performance of Final Submission

Table 1. presents the results of our proposed method alongside the top ten
competitors in the final round of the IEEE ICME 2024 Grand Challenges PAIR
Competition. This table details each method’s performance on the private
testing set provided by the competition host, evaluated on the MediaTek
Dimensity 9300 Series Platform. The reported metrics include accuracy, model
complexity, size, inference time, power consumption, and the application of
int8-quantization. The accuracy metric, designed by the competition host, is
measured using a per-point evaluation criterion: a prediction is determined to
be correct if the predicted facial landmark falls within 5 pixels of the
ground truth; otherwise, it is determined to be incorrect. Additional metrics
include model complexity, measured in GFLOPs; model size, reported in
megabytes; inference time per image, in milliseconds; and power consumption
per image, in watts. The final score for each method is calculated based on
the formula provided by the host:

$Score=\frac{Accuracy}{Time\cdot Complexity\cdot Power\cdot Size}$ (1)

Our method achieved the highest final score, significantly outperforming the
runner-up. We also recorded the best performance in terms of model complexity,
size, inference time, and power consumption. Notably, our team was the only
one to implement int8-quantization, which effectively reduced costs while
still maintaining acceptable performance of accuracy. These results
demonstrate the effectiveness of our Efficient Facial Landmark Detection
(EFLD) method in providing a lightweight and efficient solution for facial
landmark detection on edge devices.

L. Backbone | L. Head | Cross-Format | M FLOPs | Size | NME | FR@10% | CED-AUC@10%
---|---|---|---|---|---|---|---
✓ | ✓ | ✓ | 19.1 | 0.170 | 3.72 | 1.13 | 65.3
| ✓ | ✓ | 25.2 | 0.180 | 3.68 | 1.26 | 65.9
✓ |  | ✓ | 18.8 | 0.229 |  3.62 | 1.08 | 66.0
✓ | ✓ |  | 19.1 | 0.170 | 5.10 | 4.42 | 54.2
Table 2: Ablation study. Performance of different combinations of proposed
components, including the lightweight backbone (L. Backbone), lightweight
facial landmark detection head (L. Head), and the application of the cross-
format training strategy (Cross-Format). We report results across various
metrics: model complexity (M FLOPs), model size (Size), normalized mean error
(NME), failure rate at a threshold of 10% (FR@10%), and cumulative error
distribution - area under curve at a threshold of 10% (CED-AUC@10%). The
method with the largest performance drop for each metric is indicated in red.

###  4.3 Ablation Study

Table 2. showcases the impact of different components on the model’s
performance by comparing Normalized Mean Error (NME), Failure Rate at 10%
(FR@10%), and Cumulative Error Distribution - Area Under Curve at 10% (CED-
AUC@10%), with NME normalized by the inter-ocular distance. In these
experiments, we split the IVSFD dataset provided by the competition host into
a training set with 2218 samples and an evaluation set with 2219 samples.
Additionally, when employing the cross-training strategy, we incorporate
samples from the Menpo [15] and WFLW [13] datasets. In the table, “L.
Backbone” signifies the adoption of depthwise separable convolution within the
EOSA module of the backbone network, in contrast to using a standard
convolutional layer. “L. Head” denotes the use of our proposed detection head
as opposed to PFLD’s detection head [3]. “Cross-Format” refers to the
utilization of additional publicly available datasets for training our model.

The results underscore the significant efficiency of the lightweight backbone
through reduced Million FLOPs, indicating its crucial role in decreasing
computational demands. The lightweight head not only minimizes model size but
also sustains competitive performance, underscoring its effectiveness.
Moreover, the cross-format training strategy significantly enhances model
accuracy and performance, highlighting its essential contribution to improving
model efficacy.

In essence, our ablation study validates the vital significance of each
component in enhancing the model’s performance, affirming their collective
influence on computational efficiency, size reduction, and accuracy
improvement.

##  5 Conclusions

Our work advances the field of facial landmark detection by introducing the
Efficient Facial Landmark Detection (EFLD) model, tailored for optimal
performance on edge devices. By innovating an efficient backbone and a
flexible detection head, we address the critical issues of high power
consumption and latency that hinder deployment on such devices. Moreover, our
novel cross-format training strategy successfully tackles the challenge of
varying landmark formats across datasets, enhancing our model’s
generalizability without increasing the inference cost. Demonstrating superior
performance against competitors, EFLD represents a significant step forward,
promising improved user experience in real-world applications.

## References

  * [1] Bulat, A., Tzimiropoulos, G.: How far are we from solving the 2d & 3d face alignment problem?(and a dataset of 230,000 3d facial landmarks). In: ICCV (2017)
  * [2] Chollet, F.: Xception: Deep learning with depthwise separable convolutions. In: CVPR (2017)
  * [3] Guo, X., Li, S., Yu, J., Zhang, J., Ma, J., Ma, L., Liu, W., Ling, H.: Pfld: A practical facial landmark detector. arXiv preprint arXiv:1902.10859 (2019)
  * [4] Hong, Z.W., Lin, Y.C.: Improving facial landmark detection accuracy and efficiency with knowledge distillation. arXiv preprint arXiv:2404.06029 (2024)
  * [5] Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: CVPR (2017)
  * [6] Khabarlak, K., Koriashkina, L.: Fast facial landmark detection and applications: A survey. arXiv preprint arXiv:2101.10808 (2021)
  * [7] Kumar, A., Marks, T.K., Mou, W., Wang, Y., Jones, M., Cherian, A., Koike-Akino, T., Liu, X., Feng, C.: Luvli face alignment: Estimating landmarks’ location, uncertainty, and visibility likelihood. In: CVPR (2020)
  * [8] Lee, Y., Hwang, J.w., Lee, S., Bae, Y., Park, J.: An energy and gpu-computation efficient backbone network for real-time object detection. In: CVPR workshops (2019)
  * [9] Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: ICLR (2019)
  * [10] Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., Zhang, F., Chang, C.L., Yong, M.G., Lee, J., et al.: Mediapipe: A framework for building perception pipelines. arXiv preprint arXiv:1906.08172 (2019)
  * [11] Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: ECCV (2016)
  * [12] Wang, J., Sun, K., Cheng, T., Jiang, B., Deng, C., Zhao, Y., Liu, D., Mu, Y., Tan, M., Wang, X., et al.: Deep high-resolution representation learning for visual recognition. TPAMI (2020)
  * [13] Wu, W., Qian, C., Yang, S., Wang, Q., Cai, Y., Zhou, Q.: Look at boundary: A boundary-aware face alignment algorithm. In: CVPR (2018)
  * [14] Yu, H., Chen, C., Du, X., Li, Y., Rashwan, A., Hou, L., Jin, P., Yang, F., Liu, F., Kim, J., Li, J.: TensorFlow Model Garden. https://github.com/tensorflow/models (2020)
  * [15] Zafeiriou, S., Trigeorgis, G., Chrysos, G., Deng, J., Shen, J.: The menpo facial landmark localisation challenge: A step towards the solution. In: CVPR (2017)
  * [16] Zhu, C., Li, X., Li, J., Dai, S.: Improving robustness of facial landmark detection by defending against adversarial attacks. In: ICCV (2021)