Optimal decision rules for marked point process models

M.N.M. van Lieshout
CWI, P.O. Box 94079, NL-1090 GB Amsterdam, The Netherlands
Department of Applied Mathematics, University of Twente, P.O. Box 217
NL-7500 AE Enschede, The Netherlands

###### Abstract

We study a Markov decision problem in which the state space is the set of
finite marked point configurations in the plane, the actions represent
thinnings, the reward is proportional to the mark sum which is discounted over
time, and the transitions are governed by a birth-death-growth process. We
show that thinning points with large marks is optimal when births follow a
Poisson process and marks grow logistically. Explicit values for the thinning
threshold and the discounted total expected reward over finite and infinite
horizons are also provided. When the points are required to respect a hard
core distance, upper and lower bounds on the discounted total expected reward
are derived.

2020 Mathematics Subject Classification: 60G55, 90C40.

##  1 Introduction

The classic Markov decision process [3, 8, 22] on a finite state space
${\cal{X}}$ and action set $A$ is defined as follows. Write $A(x)$ for the
subset of $A$ which contains all actions that may be taken in state
$x\in{\cal{X}}$. Then, a policy $\phi$ is a procedure for the selection of an
action at each decision epoch $i\in{\mathbb{N}}_{0}$. Such a policy could be
random or deterministic, and in principle take into account the entire history
of the process. Often though, one may restrict attention to the class of
deterministic Markov policies. Such a policy $\phi=(\phi_{i})_{i=0}^{\infty}$
is a sequence of mappings $\phi_{i}:{\cal{X}}\to A$ that, at time $i$, assign
an action $a=\phi_{i}(x)\in A(x)$ to the current state $x$. In doing so, a
direct reward $r(x,a)$ is earned and a probability mass function
$p(\cdot|x,a)$ on ${\cal{X}}$ governs the next state of the process. Being
Markovian, only the current state and action are important; the past history
is irrelevant. A policy $(\phi_{i})_{i}$ is said to be stationary if its
members $\phi_{i}$ do not depend on the time $i$.

Let $(X_{i},Y_{i})$ denote the stochastic process of states $X_{i}$ and
actions $Y_{i}$. Write ${\mathbb{E}}_{x}^{\phi}$ for its expectation when the
initial state $X_{0}=x$ and the transitions are driven by policy $\phi$. Then
an optimal policy maximises the discounted total expected reward

(1)
$v_{\alpha}^{\phi}(x)={\mathbb{E}}_{x}^{\phi}\left[\sum_{i=0}^{\infty}\alpha^{i}r(X_{i},Y_{i})\right],$

$0\leq\alpha<1$. The reward function is usually assumed to be bounded, in
which case (1) is well-defined. When the state and action spaces are both
finite, it is well known [22, Theorem 5.5.3b] that it suffices to consider
only Markov policies and, by [22, Theorem 6.2.10], one may restrict oneself
even further to the class of Markov policies that are deterministic and
stationary. The maximal discounted total expected reward can be found by
policy iteration [22, Theorem 6.4.2] or value iteration, also known as
successive approximation or dynamic programming.

When the cardinality of the state or action space is infinite, policy
iteration is not guaranteed to converge in a finite number of steps. The
dynamic programming approach on the other hand is amenable to generalisation
to more general state and action spaces. Results in this direction include [4,
7, 10, 27]. The tutorial by Feinberg [6] provides an exhaustive overview with
particular emphasis on inventory control problems.

In this paper, we concentrate on the case where the state space ${\cal{X}}$
consists of finite simple marked point patterns in two-dimensional Euclidean
space. Markov decision theory using spatial point process models has found
many applications in mobile network optimisation. However, the role of the
point process is auxiliary in that it is used to model the spatial
distribution of users, base stations and so on, from which coverage
probabilities and other performance characteristics of the network can be
calculated [1, 13, 16, 11]. Spatial point process models are also convenient
in multi-target tracking [14] and their void probabilities or divergence
measures can form the basis for observer trajectory optimisation [2].

Our focus of interest here is to assume that the actions operate directly on
the point process. More precisely, we assume that, at decision epoch $i$, an
action $\phi_{i}(\mathbf{x})$ maps $\mathbf{x}$ into a subset of $\mathbf{x}$.
In other words, the action set $A(\mathbf{x})$ is the finite power set of
$\mathbf{x}$. When the decision to retain a point or not is based on the mark
or the inter-point distances, it can be interpreted as a (mark-)dependent
thinning [17, 18]. The set-up described above is appropriate for harvesting
problems in forestry [19]. Here, the classical strategy is to use discretised
stand based growth tables and dynamic programming [26]. Point pattern based
policies have been rarer due to ‘a lack of models and to difficulties in
selecting trees to be removed’ [20] and tend to be simulation based [9, 21,
23, 24]. One example is German thinning, which enhances natural selection by
picking trees whose diameter at breast height is at most $d$ and fells a
fraction of them. More formally, if each $X_{i}$ consists of tree locations
marked by diameter at breast height, German thinning fells a fraction of the
set $\\{(x,m)\in X_{i}:m\leq d\\}$ and the Markov transition kernel governs
the growth of the remaining trees (for example using the logistic growth curve
or extensions such as the Richards curve [25]) as well as natural births and
deaths (e.g. a hardcore model [12], the asymmetric soft core models of [15] or
the dynamic models of [23]). French thinning is similar, except that a
fraction of trees with large rather than small sizes is removed to stimulate
forest rejuvenation. In either case, picking a policy amounts to choosing the
level $d$. Simulations suggest that French thinning might be the better
strategy [9].

The paper’s plan is as follows. In Section 2, we study a decision process in
which the actions consist of deleting a subset of the current points and the
reward is proportional to the marks. The stochastic process that governs the
dynamics is a birth-and-death process with independent deaths and a Poisson
process of births; the marks grow logistically. We calculate the discounted
total expected reward function over finite and infinite horizons and show that
French thinning is an optimal policy. An explicit expression for the mark
threshold $d$ is derived too. In Section 3, we move on to allow interaction
between the points and replace the Poisson birth process by one in which no
point is allowed to come too close to another point. In this setting, we
provide upper and lower bounds on the discounted total expected reward
function over finite and infinite horizons. The tightness of the bounds is
investigated by means of some simulated examples. We conclude by mentioning
some topics for further research.

##  2 Marked Poisson process model with logistic growth

###  2.1 Definition of the model

To define a Markov decision process [22, Section 2.3.2], let the state space
${\cal{X}}$ consists of finite simple marked point patterns on a compact set
$W\subset{\mathbb{R}}^{2}$ with marks in $L=[0,K]$ for some $K>0$. When
${\cal{X}}$ is equipped with the Borel $\sigma$-algebra of the weak topology,
by the discussion below [5, Prop 9.1.IV], ${\cal{X}}$ is Polish. When at time
$i\in{\mathbb{N}}_{0}$ the process is in state $\mathbf{x}$, a thinning action
is carried out, resulting in a new state $\mathbf{a}$ that consists of all
retained points $\mathbf{a}\subset\mathbf{x}$. Thus, the action space
$A(\mathbf{x})$ is finite and contains all subsets of $\mathbf{x}$. Define a
stationary reward function
$r_{i}(\mathbf{x},\mathbf{a})=r(\mathbf{x},\mathbf{a})$ by

(2)
$r(\mathbf{x},\mathbf{a})=R\sum_{(x,m)\in\mathbf{x}\setminus\mathbf{a}}m,\quad\mathbf{x}\in{\cal{X}},\mathbf{a}\subset\mathbf{x}.$

Thus, the reward is proportional to the sum of the marks of all removed
points. When $R>0$, $r(\cdot,\cdot)\geq 0$. Since the mark content in an
${\mathbb{R}}^{+}$-marked point process is a random variable [5, Proposition
6.4.V], $r$ is well-defined.

Upon taking action $\mathbf{a}$ in state $\mathbf{x}$, the dynamics that lead
to the next state are modelled as a birth-death-growth process. Specifically,
the marks of the retained points $(x,m)\in\mathbf{a}$ grow according to the
well-known logistic model that was proposed around 1840 by Verhulst and
Quetelet [25]. In this model, when the mark at time $0$ is $m_{0}>0$, the mark
at time $n\in{\mathbb{N}}\cup\\{0\\}$ is

(3) $g^{(n)}(m_{0})=\frac{K}{1+\left(\frac{K}{m_{0}}-1\right)e^{-\lambda n}}.$

By convention, $g^{(n)}(0)=0$. The parameter $\lambda>0$ governs the rate of
growth and $K\geq m_{0}\geq 0$ is an upper bound on the size. In combination
with independent births and deaths, the next state is defined by the following
dynamics:

  * •

delete $\mathbf{x}\setminus\mathbf{a}$;

  * •

independently of other points, let each $(x_{i},m_{i})\in\mathbf{a}$ die with
probability $p_{d}\in(0,1)$ (natural deaths) and otherwise grow to
$(x_{i},g^{(1)}(m_{i}))$ as in (3);

  * •

add a Poisson process on $W$ with intensity $\beta>0$ and mark its points
independently according to a probability measure $\nu$ on $[0,K]$.

Write $(X_{i},Y_{i})_{i=0}^{\infty}$ for the sequence of successive states
$X_{i}$ and actions $Y_{i}$. A randomised policy
$\phi=(\phi_{i})_{i=0}^{\infty}$ is a sequence of conditional probability
kernels $\phi_{i}(\cdot|X_{0}$, $Y_{0},\dots$, $X_{i-1}$, $Y_{i-1}$, $X_{i})$
on $A$ to generate $Y_{i}$ based on the history of the process such that
$\phi_{i}(A(\mathbf{x}_{i})|\mathbf{x}_{0},\mathbf{a}_{0},\dots,\mathbf{x}_{i})=1$.
If the policy is Markov and deterministic, $Y_{i}$ is simply a function of
$X_{i}$, and one may write $Y_{i}=\phi_{i}(X_{i})$. Then, for $0\leq\alpha<1$,
the infinite horizon $\alpha$-discounted total expected reward function (1)
under policy $\phi=(\phi_{i})_{i=0}^{\infty}$ with initial state
$X_{0}=\mathbf{x}$ is

(4)
$v_{\alpha}^{\phi}(\mathbf{x})={\mathbb{E}}^{\phi}\left[\sum_{i=0}^{\infty}\alpha^{i}\left(R\sum_{(x,m)\in
X_{i}\setminus Y_{i}}m\right)\mid X_{0}=\mathbf{x}\right].$

The following Lemma shows that the model is well-defined for the birth-death-
grow dynamics defined above.

###### Lemma 1

The infinite horizon $\alpha$-discounted total expected reward function
$v_{\alpha}^{\phi}(\mathbf{x})$, $\mathbf{x}\in{\cal{X}}$, defined in (4) is
finite for all $0\leq\alpha<1$, all $R>0$ and all policies $\phi$.

Proof: Pick $\mathbf{x}\in{\cal{X}}$ and write $n(\mathbf{x})<\infty$ for its
cardinality. Since the growth function (3) is bounded by $K$,

${\mathbb{E}}\left[\sum_{(x,m)\in X_{0}\setminus Y_{0}}m\mid
X_{0}=\mathbf{x}\right]\leq Kn(\mathbf{x}).$

For $i>0$, $X_{i}$ is the union of survivors from $\mathbf{x}$, from
subsequent generations starting with $X_{1}\setminus X_{0}$ up to
$X_{i-1}\setminus X_{i-2}$ and points born in the last decision epoch.
Therefore, recalling the birth and death dynamics,

${\mathbb{E}}\left[\sum_{(x,m)\in X_{i}\setminus
Y_{i}}m|X_{0}=\mathbf{x}\right]\leq Kn(\mathbf{x})(1-p_{d})^{i}\\\
+K\beta|W|\sum_{k=0}^{i-1}(1-p_{d})^{k}$

where $|W|$ denotes the area of $W$. Hence

$v_{\alpha}^{\phi}(\mathbf{x})\leq
RKn(\mathbf{x})\sum_{i=0}^{\infty}\alpha^{i}(1-p_{d})^{i}+RK\beta|W|\sum_{i=1}^{\infty}\alpha^{i}\sum_{k=0}^{i-1}(1-p_{d})^{k}.$

For all $p_{d}\in(0,1)$, the first series in the right hand side converges to
$1/(1-\alpha(1-p_{d}))$. Since

$\sum_{i=1}^{\infty}\alpha^{i}\sum_{k=0}^{i-1}(1-p_{d})^{k}=\sum_{i=1}^{\infty}\alpha^{i}\frac{1-(1-p_{d})^{i}}{p_{d}}\leq\frac{1}{p_{d}}\sum_{i=1}^{\infty}\alpha^{i}<\infty$

for all $p_{d}\in(0,1)$, $v_{\alpha}^{\phi}(\mathbf{x})$ is finite. $\square$

The reward function $r$ itself is not bounded, so the (N) regime of [4,
Chapter 9] applies.

###  2.2 Optimal policy and reward

The optimal $\alpha$-discounted total expected reward
$v^{*}_{\alpha}(\mathbf{x})$ is defined as the supremum of the
$v_{\alpha}^{\phi}(\mathbf{x})$ over all policies, including randomised ones.
In this section, we will show that French thinning is optimal and give an
explicit expression for the corresponding reward.

By [4, Proposition 9.1], the supremum in the definition of
$v^{*}_{\alpha}(\mathbf{x})$ may be taken over the class of Markov policies,
and, by [4, Proposition 9.8], satisfies the equation

(5)
$v^{*}_{\alpha}(\mathbf{x})=\max_{\mathbf{a}\subset\mathbf{x}}\left\\{R\sum_{(x,m)\in\mathbf{x}\setminus\mathbf{a}}+\alpha{\mathbb{E}}\left[v^{*}_{\alpha}(X)\mid\mathbf{x},\mathbf{a}\right]\right\\}$

where $X$ is distributed according to the one step birth-death-growth dynamics
from state $\mathbf{x}$ under action $\mathbf{a}$. Observe that the optimality
equations (5), $\mathbf{x}\in{\cal{X}}$, are not sufficient conditions for
$v^{*}_{\alpha}$. Nevertheless, $v^{*}_{\alpha}(\mathbf{x})$ can be calculated
as the limit of an iterative procedure [4, Proposition 9.14] known as the
dynamic programming algorithm. Set $v_{0}(\mathbf{x})=0$ for all
$\mathbf{x}\in{\cal{X}}$ and set $n=1$. Define, for every
$\mathbf{x}\in{\cal{X}}$,

$v_{n}(\mathbf{x})=\max_{\mathbf{a}\subset\mathbf{x}}\left\\{R\sum_{(x,m)\in\mathbf{x}\setminus\mathbf{a}}m+\alpha{\mathbb{E}}\left[v_{n-1}(X)\mid\mathbf{x},\mathbf{a}\right]\right\\}.$

Then set $n=n+1$ and repeat. This algorithm converges to
$v^{*}_{\alpha}(\mathbf{x})$ as $n\to\infty$ by [4, Proposition 9.14] but – in
general – is of little help in constructing an optimal policy, let alone a
stationary one. Given a stationary policy $\phi$, a necessary and sufficient
condition for it to be optimal is [4, Prop. 9.13]

(6)
$v_{\alpha}^{\phi}(\mathbf{x})=\max_{\mathbf{a}\subset\mathbf{x}}\left\\{R\sum_{(x,m)\in\mathbf{x}\setminus\mathbf{a}}+\alpha{\mathbb{E}}\left[v_{\alpha}^{\phi}(X)|\mathbf{x},\mathbf{a}\right]\right\\}.$

For our model, the dynamic programming algorithm does suggest an optimal
deterministic and stationary Markov policy.

###### Theorem 1

Consider the Markov decision process with state space ${\cal{X}}$, action
spaces
$A(\mathbf{x})=\\{\mathbf{y}\in{\cal{X}}:\mathbf{y}\subset\mathbf{x}\\}$,
$\mathbf{x}\in{\cal{X}}$, reward function (2) with $R>0$, and birth-death-
growth dynamics based on independent deaths with probability $p_{d}\in(0,1)$,
a Poisson birth process with intensity $\beta>0$ marked independently
according to probability measure $\nu$ on $[0,K]$ for $K>0$ and logistic
growth function (3). Then, for $0\leq\alpha<1$,

$v^{*}_{\alpha}(\mathbf{x})=R\beta|W|\sum_{k=1}^{\infty}\alpha^{k}\int_{0}^{K}s(m)\,d\nu(m)+R\sum_{(x,m)\in\mathbf{x}}s(m),$

where $|W|$ is the area of $W$ and

$s(m)=\sup_{n\in{\mathbb{N}}_{0}}\left\\{\frac{K\alpha^{n}(1-p_{d})^{n}}{1+\left(\frac{K}{m}-1\right)e^{-\lambda
n}}\right\\},\quad m\in[0,K].$

Furthermore, the optimal $\alpha$-discounted total expected reward corresponds
to a French thinning that removes all points with a mark that is at least

$d^{*}_{\alpha}=\sup_{n\in{\mathbb{N}}_{0}}\left\\{\frac{K}{1-e^{-n\lambda}}\left(\alpha^{n}(1-p_{d})^{n}-e^{-n\lambda}\right)\right\\}.$

For $\alpha=1$, the total expected reward $v^{*}_{1}(\mathbf{x})$ is infinite.

Proof: After initialising $v_{0}(\mathbf{x})=0$ for all
$\mathbf{x}\in{\cal{X}}$, clearly the optimal expected reward at time $0$ is
$v_{1}(\mathbf{x})=R\sum_{(x,m)\in\mathbf{x}}m,$ which is attained for action
$\mathbf{a}=\emptyset$, or, in other words, by removing all points with mark
greater than or equal to $d_{1}=0$. The proof proceeds by induction. Set, for
$n\in{\mathbb{N}}$,

(7)
$d_{n}=\max\left\\{0,K\frac{\alpha(1-p_{d})-e^{-\lambda}}{1-e^{-\lambda}},\,\dots,\,K\frac{\alpha^{n-1}(1-p_{d})^{n-1}-e^{-(n-1)\lambda}}{1-e^{-(n-1)\lambda}}\right\\}$

and suppose that the optimal $\alpha$-discounted expected reward over $n$
steps is attained by French thinning at level $d_{n}$ and given by

(8)
$v_{n}(\mathbf{x})=R\beta|W|\sum_{k=1}^{n-1}\alpha^{k}\int_{0}^{K}s_{n-k}(m)\,d\nu(m)+R\sum_{(x,m)\in\mathbf{x}}s_{n}(m)$

where, for $1\leq k\leq n$,

$s_{k}(m)=\max\left\\{m,\alpha\left(1-p_{d}\right)g^{(1)}(m),\,\dots,\,\alpha^{k-1}\left(1-p_{d}\right)^{k-1}g^{(k-1)}(m)\right\\}.$

Now, for $n+1$, the optimal finite horizon $\alpha$-discounted expected reward
is

$v_{n+1}(\mathbf{x})=\max_{\mathbf{a}\subset\mathbf{x}}\left\\{R\sum_{(x,m)\in\mathbf{x}\setminus\mathbf{a}}m+\alpha{\mathbb{E}}\left[v_{n}(X)\mid\mathbf{x},\mathbf{a}\right]\right\\}.$

By the induction assumption, the discounted expectation
$\alpha{\mathbb{E}}\left[v_{n}(X)\mid\mathbf{x},\mathbf{a}\right]$ is the sum
of

$\alpha
R\beta|W|\sum_{k=1}^{n-1}\alpha^{k}\int_{0}^{K}s_{n-k}(m)\,d\nu(m)=R\beta|W|\sum_{k=2}^{n}\alpha^{k}\int_{0}^{K}s_{n+1-k}(m)\,d\nu(m)$

and contributions from the points in $\mathbf{a}$ that survive a decision
epoch as well as from points born in the interval between time $n$ and $n+1$.
These contributions are, respectively,

$\alpha R\sum_{(x,m)\in\mathbf{a}}\left(1-p_{d}\right)s_{n}(g(m))$

and, using the Campbell–Mecke formula [5, Section 6.1],

$\alpha R\beta|W|\int_{0}^{K}s_{n}(m)\,d\nu(m).$

The optimal policy assigns a point $(x,m)\in\mathbf{x}$ to
$\mathbf{x}\setminus\mathbf{a}$ if and only if
$m\geq\alpha(1-p_{d})s_{n}(g^{(1)}(m)).$ By the induction assumption and (3),
this is the case if and only if

(9) $m\geq\alpha^{k}(1-p_{d})^{k}g^{(k)}(m)\Leftrightarrow m\geq
K\frac{\alpha^{k}(1-p_{d})^{k}-e^{-k\lambda}}{1-e^{-k\lambda}}$

for all integers $1\leq k\leq n$. Consequently, $d_{n+1}$ has the required
form. For this allocation rule, the reward is
$\max\left\\{m,\alpha\left(1-p_{d}\right)s_{n}(g^{(1)}(m))\right\\}=s_{n+1}(m)$
and the induction step is complete.

Next, let $n$ go to infinity and fix $m\in[0,K]$. Then $s(m)$ is finite for
all $p_{d}\in(0,1)$ and $0\leq\alpha<1$. Additionally,
$\lim_{n\to\infty}s_{n}(m)=s(m)$. Thus, for any $\mathbf{x}\in{\cal{X}}$,

$R\sum_{(x,m)\in\mathbf{x}}s_{n}(m)\to R\sum_{(x,m)\in\mathbf{x}}s(m)$

as $n\to\infty$. Furthermore,

$\sum_{k=1}^{n-1}\alpha^{k}\int_{0}^{K}s_{n-k}(m)\,d\nu(m)\to\sum_{k=1}^{\infty}\alpha^{k}\int_{0}^{K}s(m)\,d\nu(m),\quad
n\to\infty,$

because of dominated convergence applied to the doubly indexed sequence
$a_{k,n}$ defined by ${\mathbf{1}}\left\\{k\leq n-1\right\\}\alpha^{k}\int
s_{n-k}\,d\nu.$ In conclusion, for each $\mathbf{x}\in{\cal{X}}$,
$\lim_{n\to\infty}v_{n}(\mathbf{x})=v^{*}_{\alpha}(\mathbf{x})$, the optimal
$\alpha$-discounted total expected reward [4, Proposition 9.14], and
$v^{*}_{\alpha}(\mathbf{x})$ has the claimed form.

To complete the proof, we need to show that $v^{*}(\mathbf{x})$ is attained by
the stationary deterministic policy that retains all points with mark smaller
than $d^{*}_{\alpha}$. Denote its infinite horizon $\alpha$-discounted total
expected reward by

$v_{\alpha}^{d^{*}}(\mathbf{x})={\mathbb{E}}\left[R\sum_{i=0}^{\infty}\alpha^{i}\sum_{(x,m)\in
X_{i}}m\,{\mathbf{1}}\\{m\geq d^{*}_{\alpha}\\}\mid X_{0}=\mathbf{x}\right]$

and focus on the contributions of each generation of points. A point
$(x,m)\in\mathbf{x}$, the initial generation, yields a reward
$R\,\alpha^{n}(1-p_{d})^{n}g^{(n)}(m)$ precisely when $g^{(n-1)}(m)$ is less
than $d^{*}_{\alpha}$ but $g^{(n)}(m)\geq d^{*}_{\alpha}$. Since, as in (9),
$g^{(n)}(m)\geq d^{*}_{\alpha}$ if and only if

$g^{(n)}(m)\geq\alpha^{k}(1-p_{d})^{k}g^{(n+k)}(m)$

for all $k\in{\mathbb{N}}_{0}$, we conclude that every point of $\mathbf{x}$
contributes $R\,s(m)$. The points that are born in the first decision epoch
(generation $1$) yield the same total expected reward, but this is discounted
by $\alpha$ due to the later birth date. Similarly, the total expected reward
of points belonging to the second generation is discounted by $\alpha^{2}$,
and so on. Tallying up, the $\alpha$-discounted total expected reward of
generations $k=1,2,\dots$ is

$R\beta|W|\sum_{k=1}^{\infty}\alpha^{k}\int_{0}^{K}s(m)\,d\nu(m)$

on application of the Campbell–Mecke formula. Finally add the contribution
from the initial generation to conclude that the threshold $d^{*}_{\alpha}$
defines an optimal policy. Condition (6) is readily verified. $\square$

As a by-product, the proof of Theorem 1 derives the optimal
$\alpha$-discounted total expected reward (8) for finite time horizons too,
and French thinning with threshold (7) is an optimal policy. The suprema in
$s(m)$ and $d^{*}_{\alpha}$ are attained, which can be seen by considering the
limit for $n\to\infty$.

##  3 Hard core models with logistic growth

###  3.1 Bounds for the optimal discounted total expected reward

In this section, we refine the Poisson model of the previous section to the
case where births are governed by a hard core process. Thus, the state space
${\cal{X}}_{K}$ consists of all finite simple marked point patterns on a
compact set $W$ in the plane that contain no pair $\\{x_{1},x_{2}\\}$ such
that $||x_{1}-x_{2}||\leq K$ with marks in $L=[0,K]$. For the motivating
example from forestry in which the marks correspond to the diameter at breast
height, the condition ensures that all trees can grow to their maximal size.

As in Section 2.1, when at time $i\in{\mathbb{N}}_{0}$ the process is in state
$\mathbf{x}$, a thinning action is carried out, resulting in a new state
$\mathbf{a}$ that consists of all retained points. The reward is defined in
(2).

The dynamics are modified in such a way that the hard core is respected.
Specifically, suppose that action $\mathbf{a}$ is taken in state
$\mathbf{x}\in{\cal{X}}_{K}$. The next state is then governed by the following
birth-death-growth process:

  * •

delete $\mathbf{x}\setminus\mathbf{a}$;

  * •

independently of other points, let each $(x_{i},m_{i})\in\mathbf{a}$ die with
probability $p_{d}\in(0,1)$ and otherwise grow to $(x_{i},g(m_{i}))$ for some
bounded, continuous function $g:[0,K]\to[0,K]$ satisfying $m\leq g(m)$ for
$m\in[0,K]$;

  * •

add a hard core process on $W$ with hard core distance $K$ and intensity
$\beta>0$; mark its points independently according to a probability measure
$\nu$ on $[0,K]$ and remove all points that fall within distance $K$ to a
point in $\mathbf{a}$.

In this framework, the reward function is bounded since the hard core
condition implies an upper bound on the number of points that can be alive at
any time. We are therefore in the (D) regime of [4, Chapter 9].

For $\mathbf{x}\in{\cal{X}}_{K}$, define $v^{*}_{\alpha}(\mathbf{x})$ as the
supremum of (4) over all policies $\phi$. By [4, Proposition 9.1] it suffices
to consider Markov policies only, and $v^{*}_{\alpha}(\mathbf{x})$ is the
limit of the dynamic programming algorithm [4, Proposition 9.14]. The
optimality condition (6) applies. Moreover, since the action sets are finite,
Corollary 9.17.1 in [4] guarantees the existence of an optimal deterministic
stationary policy. An explicit expression seems hard to obtain. However, the
following bounds are available.

###### Theorem 2

Consider the Markov decision process with state space ${\cal{X}}_{K}$, action
spaces
$A(\mathbf{x})=\\{\mathbf{y}\in{\cal{X}}_{K}:\mathbf{y}\subset\mathbf{x}\\}$,
$\mathbf{x}\in{\cal{X}}_{K}$, reward function (2) with $R>0$, and birth-death-
growth dynamics based on independent deaths with probability $p_{d}\in(0,1)$,
a hard core birth process with intensity $\beta>0$ marked independently
according to probability measure $\nu$ on $[0,K]$ for $K>0$ and growth
function $g$. Write $g^{(n)}(m)$ for the $n$-fold composition of $g$.

For $\alpha\in[0,1)$, initialise $v_{0}(\mathbf{x})=0$ for all
$\mathbf{x}\in{\cal{X}}_{K}$. Define, for $n\in{\mathbb{N}}$ and
$\mathbf{x}\in{\cal{X}}_{K}$,

$v_{n}(\mathbf{x})=\max_{\mathbf{a}\subset\mathbf{x}}\left\\{R\sum_{(x,m)\in\mathbf{x}\setminus\mathbf{a}}m+\alpha{\mathbb{E}}\left[v_{n-1}(X)\mid\mathbf{x},\mathbf{a}\right]\right\\}.$

Then $\tilde{v}_{n}(\mathbf{x})\leq
v_{n}(\mathbf{x})\leq\hat{v}_{n}(\mathbf{x})$ where

$\displaystyle\tilde{v}_{n}(\mathbf{x})$ $\displaystyle=$ $\displaystyle
R\sum_{(x,m)\in\mathbf{x}}\tilde{s}_{n}(x,m)+R\beta\sum_{k=1}^{n-1}\alpha^{k}\int_{W}\int_{0}^{K}\tilde{s}_{n-k}(w,l)\,d\nu(l)dw$
$\displaystyle\hat{v}_{n}(\mathbf{x})$ $\displaystyle=$ $\displaystyle
R\sum_{(x,m)\in\mathbf{x}}\hat{s}_{n}(m)+R\beta|W|\sum_{k=1}^{n-1}\alpha^{k}\int_{0}^{K}\hat{s}_{n-k}(l)\,d\nu(l)$

with $\tilde{s}_{0}=\hat{s}_{0}=0$ and, for $n\in{\mathbb{N}}$,

$\hat{s}_{n}(m)=\max\left\\{m,\alpha(1-p_{d})g^{(1)}(m),\,\dots,\,\alpha^{n-1}(1-p_{d})^{n-1}g^{(n-1)}(m)\right\\}$

and, writing $b(x,K)$ for the closed ball centred at $x$ with radius $K$,

$\tilde{s}_{n}(x,m)=\max\\{m,\alpha(1-p_{d})g^{(1)}(m)-\alpha
K\beta|b(x,K)\cap W|,\,\dots,$ $\alpha^{n-1}(1-p_{d})^{n-1}g^{(n-1)}(m)-\alpha
K\beta|b(x,K)\cap W|\sum_{i=0}^{n-2}\alpha^{i}(1-p_{d})^{i}\\}.$

When the growth function is logistic,

$\displaystyle\tilde{s}_{n}(x,m)$ $\displaystyle=$
$\displaystyle\max_{i=0,\dots,n-1}\left\\{\frac{K\alpha^{i}(1-p_{d})^{i}}{1+\left(\frac{K}{m}-1\right)e^{-\lambda
i}}-\alpha K\beta|W\cap
b(x,K)|\frac{1-\alpha^{i}(1-p_{d})^{i}}{1-\alpha(1-p_{d})}\right\\};$
$\displaystyle\hat{s}_{n}(m)$ $\displaystyle=$
$\displaystyle\max_{i=0,\dots,n-1}\left\\{\frac{K\alpha^{i}(1-p_{d})^{i}}{1+\left(\frac{K}{m}-1\right)e^{-\lambda
i}}\right\\}.$

Proof: The proof proceeds by induction. For $n=0$, evidently
$v_{0}\leq\tilde{v}_{0}$. Assume that $\tilde{v}_{k}(\mathbf{x})\leq
v_{k}(\mathbf{x})\leq\hat{v}_{k}(\mathbf{x})$ for all $k\leq n$ and all
$\mathbf{x}\in{\cal{X}}_{K}$ and that $\tilde{v}_{k}$, $\hat{v}_{k}$ have the
required form. Since

(10)
$v_{n+1}(\mathbf{x})=\max_{\mathbf{a}\subset\mathbf{x}}\left\\{R\sum_{(x,m)\in\mathbf{x}\setminus\mathbf{a}}m+\alpha{\mathbb{E}}\left[v_{n}(X)\mid\mathbf{x},\mathbf{a}\right]\right\\}$

and $v_{n}(X)\geq\tilde{v}_{n}(X)$, let us consider the expectation of
$\tilde{v}_{n}(X)$ under the hard core birth-death-growth dynamics when action
$\mathbf{a}$ is taken in state $\mathbf{x}$. By the definition of
$\tilde{v}_{n}$ and distinguishing between surviving and new-born points,

$\displaystyle{\mathbb{E}}\left[\tilde{v}_{n}(X)\mid\mathbf{x},\mathbf{a}\right]$
$\displaystyle=$ $\displaystyle R\,{\mathbb{E}}\left[\sum_{(x,m)\in
X}\tilde{s}_{n}(x,m)\mid\mathbf{x},\mathbf{a}\right]+R\beta\sum_{k=1}^{n-1}\alpha^{k}\int_{W}\int_{0}^{K}\tilde{s}_{n-k}(w,l)\,d\nu(l)dw$
$\displaystyle=$ $\displaystyle
R\sum_{(x,m)\in\mathbf{a}}\left(1-p_{d}\right)\tilde{s}_{n}(x,g^{(1)}(m))+R\beta\sum_{k=1}^{n-1}\alpha^{k}\int_{W}\int_{0}^{K}\tilde{s}_{n-k}(w,l)\,d\nu(l)dw$
$\displaystyle+R\beta\int_{W}\int_{0}^{K}\tilde{s}_{n}(w,l)\,{\mathbf{1}}\\{w\not\in
U_{K}(\mathbf{a})\\}\,d\nu(l)dw$

where the symbol $U_{K}(\mathbf{a})$ signifies the union of closed balls with
radius $K$ around the points in $\mathbf{a}$. The calculation of the last term
above relies on the Campbell–Mecke formula [5, Section 6.1]. Now, the integral
in the last line above can be written as

$R\beta\int_{W}\int_{0}^{K}\tilde{s}_{n}(w,l)\,d\nu(l)dw-R\beta\int_{W}\int_{0}^{K}\tilde{s}_{n}(w,l)\,{\mathbf{1}}\\{w\in
U_{K}(\mathbf{a}\\}\,d\nu(l)dw$

and is bounded from below by

(11) $R\beta\int_{W}\int_{0}^{K}\tilde{s}_{n}(w,l)\,d\nu(l)dw-
RK\beta\sum_{(x,m)\in\mathbf{a}}\int_{W}\int_{0}^{K}{\mathbf{1}}\\{w\in
b(x,K)\\}\,d\nu(l)$

where the induction assumption is invoked for the inequality
$\tilde{s}_{n}\leq K$. Next, return to (10). The bound on
${\mathbb{E}}\left[\tilde{v}_{n}(X)\mid\mathbf{x},\mathbf{a}\right]$ implies

$v_{n+1}(\mathbf{x})\geq\max_{\mathbf{a}\subset\mathbf{x}}\left\\{R\sum_{(x,m)\in\mathbf{x}\setminus\mathbf{a}}m+\alpha{\mathbb{E}}\left[\tilde{v}_{n}(X)\mid\mathbf{x},\mathbf{a}\right]\right\\}\\\
\geq\max_{\mathbf{a}\subseteq\mathbf{x}}\\{R\sum_{(x,m)\in\mathbf{x}\setminus\mathbf{a}}m+$
$\alpha
R\sum_{(x,m)\in\mathbf{a}}\left[(1-p_{d})\tilde{s}_{n}(x,g^{(1)}(m))-K\beta|b(x,K)\cap
W|\right]+R\beta\sum_{k=1}^{n}\alpha^{k}\int_{W}\int_{0}^{K}\tilde{s}_{n+1-k}(w,l)d\nu(l)dw.$

The policy that assigns $(x,m)$ to $\mathbf{x}\setminus\mathbf{a}$ if and only
if

$m\geq\alpha\left[\left(1-p_{d}\right)\tilde{s}_{n}(x,g^{(1)}(m))-K\beta|b(x,K)\cap
W|\right]$

optimises the right hand side and, with

$\tilde{s}_{n+1}(x,m)=\max\left\\{m,\alpha\left(1-p_{d}\right)\tilde{s}_{n}(x,g^{(1)}(m))-\alpha
K\beta|b(x,K)\cap W|\right\\},$

one sees that

$v_{n+1}(\mathbf{x})\geq\tilde{v}_{n+1}(\mathbf{x})=R\sum_{(x,m)\in\mathbf{x}}\tilde{s}_{n+1}(x,m)+R\beta\sum_{k=1}^{n}\alpha^{k}\int_{W}\int_{0}^{K}\tilde{s}_{n+1-k}(w,l)\,d\nu(l)dw,$

an observation that completes the induction argument and therefore the proof
of the lower bound.

For the upper bound $v_{n}\leq\hat{v}_{n}$, as in the proof of Theorem 1, an
induction proof applies based on $\hat{s}_{n}$ but with (11) replaced by the
upper bound

$R\beta\int_{W}\int_{0}^{K}\hat{s}_{n}(w,l)\,d\nu(l)dw.$

$\square$

Over an infinite time horizon, the optimal $\alpha$-discounted total expected
reward is bounded by the same functional forms, which coincide if $\alpha=0$.

###### Corollary 1

The functions $\hat{s}_{n}$ and $\tilde{s}_{n}$ defined in Theorem 2 take
values in $[0,K]$ and increase monotonically to

$\hat{s}(m)=\sup_{n\in{\mathbb{N}}_{0}}\left\\{\alpha^{n}(1-p_{d})^{n}g^{(n)}(m)\right\\},\quad
m\in[0,K],$

and, for $x\in W$ and $m\in[0,K]$,

$\tilde{s}(x,m)=\sup_{n\in{\mathbb{N}}_{0}}\left\\{\alpha^{n}(1-p_{d})^{n}g^{(n)}(m)-\alpha
K|b(x,K)\cap W|\sum_{i=0}^{n-1}\alpha^{i}(1-p_{d})^{i}\right\\}.$

###  3.2 Simulation study

To assess the tightness of the bounds in Theorem 2, we calculated
$\hat{v}_{n}(\mathbf{x})$ and $\tilde{v}_{n}(\mathbf{x})$ in two regimes, a
dense one and a sparse one. For the inital pattern $\mathbf{x}$, a sample from
a Strauss process [12] on $W=[0,5]^{2}$ with interaction parameter set to zero
was chosen. The activity parameter was set to give the required intensity:
$\beta=1.0$ in the sparse regime and $\beta=4.3$ in the dense regime. For the
mark dynamics, we used a logistic growth function with $\lambda=2$ and maximal
size $K=0.1$; the initial marks were sampled from a Beta distribution with
shape parameters $\lambda_{1}=2$ and $\lambda_{2}=20$. The death rate was set
to $p_{d}=0.05$. Finally, we used discount factor $\alpha=0.9$ and reward
parameter $R=1$.

The results are plotted in Figure 1. The left panels show the pattern
$\mathbf{x}$. In the right panels, the solid lines are the graphs of
$\hat{v}_{n}(\mathbf{x})$ as a function of $n$, the dotted lines show
$\tilde{v}_{n}(\mathbf{x})$ plotted against $n$. Integrals were estimated by
the Monte Carlo method with $1,000$ samples. In the sparse regime, the
approximation is quite good, for the denser regime, the gap between the two
graphs is quite wide except for very small $n$. In both cases, the dynamic
programming algorithm converges rapidly.

Figure 1: Left panels: samples $\mathbf{x}$ from a Strauss hard core process
with intensity $\beta=1.0$ (top) and $\beta=4.3$ (bottom) on $[0,5]^{2}$.
Right panels: graphs of $\hat{v}_{n}(\mathbf{x})$ (solid lines) and
$\tilde{v}_{n}(\mathbf{x})$ (dotted lines) against $n$ for the birth-death-
growth dynamics of Section 3.2.

##  4 Conclusion

In this paper we considered optimal policies for Markov decision problems
inspired by forest harvesting. We proved that French thinning is optimal when
births follow a Poisson process and marks grow logistically. When the points
are required to respect a hard core distance, we derived upper and lower
bounds on the discounted total expected reward for general birth-death-growth
dynamics. Although we focused on a homogeneous birth process, the results
carry over to the case where the birth process is governed by some spatially
varying intensity function.

In future it would be of interest to study configuration dependent asymmetric
birth and growth models [14, 15, 24]. Indeed, in a forestry setting, the
growth of well-established, large trees may hardly be hampered by the
emergence of saplings close by, while it would be harder for young and small
trees to flourish near large ones. Moreover, the natural environment, such as
the availability of nutrients, might play a role. Finally, refinements of the
action space that allow for different thresholds in different mark strata
could be investigated.

## References

  * [1] Baccelli, F. and Blaszczyszyn, B. (2009). Stochastic geometry and wireless networks, in two volumes. NOW.
  * [2] Beard, M., Vo, B.T., Vo, B.N. and Arulampalam, S. (2017). Void probabilities and Cauchy–Schwarz divergence for generalized labeled multi-Bernoulli models. IEEE Trans. Signal Process., 65, 5047–5061.
  * [3] Bertsekas, D.P. (1995). Dynamic programming and optimal control. Prentice and Hall.
  * [4] Bertsekas, D.P. and Shreve, S.E. (1978). Stochastic optimal control: The discrete time case. Academic Press.
  * [5] Daley, D.J. and Vere–Jones, D. (2003, 2008). An introduction to the theory of point processes, second edition in two volumes. Springer.
  * [6] Feinberg, E.A. (2016). Optimality conditions for inventory control. Tutorials in Operations Research, INFORMS 2016, pp. 14–44.
  * [7] Feinberg, E.A. and Lewis, M.E. (2007). Optimality inequalities for average cost Markov decision processes and the stochastic cash balance problem. Math. Oper. Res., 32, 769–783.
  * [8] Feinberg, E.A. and Schwartz, A. (2002). Handbook of Markov decision processes. Springer.
  * [9] Fransson, P., Franklin, O., Lindroos, O., Nilsson, U. and Brännström, Å. (2020). A simulation-based approach to a near optimal thinning strategy: Allowing for individual harvesting times for individual trees. Can. J. For. Res., 50, 320–331.
  * [10] Hernández–Lerma, O. and Lasserre, B.J. (1996). Discrete-time Markov control processes: Basic optimality criteria. Springer.
  * [11] Khloussy, E., Gelabert, X. and Jiang, Y. (2015). Investigation on MDP-based radio access technology selection in heterogeneous wireless networks. Comput. Netw., 91, 57–67.
  * [12] Kelly, F.P. and Ripley, B.D. (1976). On Strauss’s model for clustering. Biometrika, 63, 357–360.
  * [13] Lee, W., Jung, B.C. and Lee, H. (2020). DeCoNet: Density clustering-based base station control for energy-efficient cellular IoT networks. IEEE Access, 8, 120881.
  * [14] Lieshout, M.N.M. van (2008). Depth map calculation for a variable number of moving objects using Markov sequential object processes. IEEE Trans. Pattern Anal. Mach. Intell., 30, 1308–1312.
  * [15] Lieshout, M.N.M. van (2009). Sequential spatial processes for image analysis. In Stereology and Image Analysis. ECS10–Proceedings of the 10th European Congress of ISS, V. Capasso et al. (Eds.), 6 pages. Bologna.
  * [16] Lu, X., Salehi, M., Haenggi, M., and Hossain, E. (2021). Stochastic geometry analysis of spatial-temporal performance in wireless networks: A tutorial. IEEE Commun. Surveys & Tutorials, 23, 2753–2801.
  * [17] Matérn, B. (1986). Spatial variation. Springer.
  * [18] Myllimäki, M. (2009). Statistical models and inference for spatial point patterns with intensity-dependent marks. PhD thesis, University of Jyväskylä.
  * [19] Pretzch, H. (2009). Forest dynamics, growth and yield. Springer.
  * [20] Pukkala, T. and Miina, J. (1998). Tree-selection algorithms for optimizing thinning using a distance-dependent growth model. Can. J. For. Res., 28, 693–702.
  * [21] Pukkala, T., Lähde, E. and Laiho, O. (2015). Which trees should be removed in thinning treatments? For. Ecosyst., 2, 1–12.
  * [22] Puterman, M.L. (1994). Markov decision processes. Wiley.
  * [23] Renshaw, E. and Särkkä, A. (2001). Gibbs point processes for studying the development of spatial-temporal stochastic processes. Comput. Stat. Data Anal., 36, 85–105.
  * [24] Renshaw, E., Comas, C. and Mateu, J. (2009). Analysis of forest thinning strategies through the development of space-time growth-interaction simulation models. Stoch. Environ. Res. Risk Assess., 23, 275–288.
  * [25] Richards, F.J. (1959). A flexible growth function for empirical use. J. Exp. Bot., 10, 290–300.
  * [26] Rönnqvist, M. (2003). Optimization in forestry. Math. Program. Ser. B, 97, 267–284.
  * [27] Scha̋l, M. (1993). Average optimality in dynamic programming with general state space. Math. Oper. Res., 18, 163–172.


# Sound-Print: Generalised Face Presentation Attack Detection using Deep
Representation of Sound Echoes

Raghavendra Ramachandra1 Jag Mohan Singh 1 Sushma Venkatesh2
1Norwegian University of Science and Technology (NTNU), Norway. 2AiBA AS,
Norway.
email<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Facial biometrics are widely deployed in smartphone-based applications because
of their usability and increased verification accuracy in unconstrained
scenarios. The evolving applications of smartphone-based facial recognition
have also increased Presentation Attacks (PAs), where an attacker can present
a Presentation Attack Instrument (PAI) to maliciously gain access to the
application. Because the materials used to generate PAI are not deterministic,
the detection of unknown presentation attacks is challenging. In this paper,
we present an acoustic echo-based face Presentation Attack Detection (PAD) on
a smartphone in which the PAs are detected based on the reflection profiles of
the transmitted signal. We propose a novel transmission signal based on the
wide pulse that allows us to model the background noise before transmitting
the signal and increase the Signal-to-Noise Ratio (SNR). The received signal
reflections were processed to remove background noise and accurately represent
reflection characteristics. The reflection profiles of the bona fide and PAs
are different owing to the different reflection characteristics of the human
skin and artefact materials. Extensive experiments are presented using the
newly collected Acoustic Sound Echo Dataset (ASED) with 4807 samples captured
from bona fide and four different types of PAIs, including print (two types),
display, and silicone face-mask attacks. The obtained results indicate the
robustness of the proposed method for detecting unknown face presentation
attacks.

##  1 Introduction

Face Recognition Systems (FRS) are vulnerable to presentation attacks that are
carried out by presenting facial artefacts to the face biometric capture
system. The easy availability of the target face image that can be acquired by
non-intrusive capture or through social networks makes these attacks common to
operational FRS. Furthermore, Presentation Attacks (PAs) can easily be
performed without any knowledge of the underlying operation of the biometric
system. The wider deployment of the FRS in various applications, especially
banking, has further elevated the PA on the FRS.

Figure 1: Illustration of Acoustic reflections for face presentation attack
detection Table 1: Acoustics Face PAD State-Of-The-Art (SOTA) Techniques

Authors | Transmission Signal | Types of PAIs | Database size | Multimodality | SmartPhone
---|---|---|---|---|---
Zhou et al. [20, 21] | FWCM |  | Print Attack
---
Display Attack
45 Subjects |  | RGB face image
---
Acoustic reflections
Samsung S7, S8 and Huawei P9
Chen et al. [1] | FWCM | Print Attack | 6 Subjects | Acoustic reflections | Samsung Galaxy C7
Kong et al. [7] | FWCM |  | Print Attack
---
Display Attack
30 Subjects |  | RGB face image
---
Acoustic reflections
Samsung S9, S21, edge note and Xiaomi Redmi7
Our work | Wide Pulse |  | Print Attack
---
Silicone Mask
Display Attack
35 Subjects | Acoustic reflections | Samsung S10

The vulnerability of FRS to PAs has increased the interest of both academic
and industrial researchers in developing Presentation Attack Detection (PAD)
algorithms. Several high-end smartphones (e.g., Apple iPhone, Samsung) offer
PAD by integrating multiple sensors (multispectral and 3D) and multibiometric
systems. From an academic perspective, PAD algorithms have been extensively
studied and broadly classified into hardware- and software-based methods [19,
13]. Hardware-based approaches employ additional hardware (such as
multispectral cameras and liveness measuring devices) to reliably detect PAs
at high cost and with limited scalability. Software-based approaches analyze
captured face biometrics using either handcrafted features (using textures,
gradients, and other image-based features) [13, 5] or deep learning features
[19, 3, 9, 18, 12, 17, 16, 10, 8]. End-to-end deep learning techniques have
been extensively studied in the literature on face PAD to achieve a reliable
detection accuracy. However, generalizability across different types of PAIs
for face PAD remains challenging.

In contrast to the existing techniques based on visual data (RGB images),
acoustic echoes are used for face verification and PAs [20]. Acoustic echo
processing includes the sound signal transmitted using a smartphone speaker
and recording sound reflections using microphones (see Figure 1). The first
work using acoustic echoes for both face verification and PAD was presented by
Zhou et al. [20, 21]. A multimodal approach using acoustic and visual data was
presented for verification and PAD. The underlying principle of using acoustic
echoes for face verification assumes that the different 3D facial structure
can reflect the sound signal with uneven attenuation at different frequencies
that can be used for verification. For the PAD, the reflected signal has
different reflection characteristics depending on the type of material used to
generate the artefact. Therefore, the reflection profile was further processed
to reliably detect the facial PAD.

In [21] Frequency-Modulated Continuous Wave (FMCW) signals with different
frequencies were used as the transmitted signal, and the received signal was
processed to recover echoes that were buried in the main lobes. Finally, a
shallow serial Convolutional Neural Network (CNN) is proposed independently on
acoustic echoes and facial images to perform verification and PAD. Chen et al.
[1] proposed the acoustic echo-based PAD in which the FMCW signal is used as
the transmitted signal. The received signal was collected using two
microphones located on the smartphones. In total, 16 chirp signals with
different frequency ranges were used for transmission. Recently, Kong et al.
[7] presented a face PAD using acoustics and a face (or RGB) image-based
multimodal system. The transmitted signals were generated using the FMCW
technique, with nine chirp signals at different frequencies. Detection was
performed using a two-branch attention model trained separately on acoustic
and RGB images. Based on the above discussion on SOTA methods, the FMCW signal
is widely used to perform face verification and PAD. However, post-processing
of the received echoes is challenging because they are buried in the strong
main-lobe reflection and background noise. Furthermore, in previous
approaches, experiments have been performed using only print and display
attacks.

In this work, we present a novel method for face PAD based on acoustic signals
on a smartphone. The main objective of the proposed method is to use only
acoustic signals for the genralizable PAD. The proposed method analyzes
reflection echo characteristics to detect bona fides and PAs. The scattering
property of the transmitted signal exhibits different characteristics owing to
the change in the medium/material properties between the different types of
PAIs and Bona fide. Therefore, the proposed method introduces a single wide
pulse as the transmission signal to achieve a high Signal-to-Noise Ratio
(SNR). Furthermore, the signal design also includes the silence period before
transmission of the signal that will allow recording of the background noise,
which is later subtracted from the received signal to reduce the background
noise. After post-processing, the received signal is further represented by
time-frequency components computed using Continuous Wavelet Transform (CWT)
filter banks. The CWT representation was further processed through the pre-
trained deep convolutional neural network (EfficientNet [15]) to obtain deep
features. Finally, PAD was performed using a linear SVM to effectively detect
the PAs. Table 1 summarizes the features of the proposed method and the
existing SOTA method. Thus, the main contributions of this work are as
follows:

  * •

We present a novel PAD based on acoustic signal processing for a face
biometric system. We presented a wide pulse as the transmitted signal to
achieve a high SNR. Furthermore, the proposed signal design incorporates a
silent period to estimate the background noise.

  * •

A new acoustic signal feature extraction method using CWT filter-banks and
deep features are introduced to reliably detect the PAs.

  * •

A new Acoustic Sound Echo Dataset (ASED) is collected with two different types
of print PAIs, display attack PAI and silicone face mask PAI, together with
the bona fide samples from 35 unique data subjects resulting in 4807 acoustic
samples.

  * •

Extensive experiments are performed to benchmark the generalizability of the
proposed method across four different PAIs.

Figure 2: Block diagram of the proposed method

The remainder of the paper is organized as follows: Section 2 discusses the
proposed method, Section 3 discusses the data collection protocols and
dataset, Section 4 presents the experimental results and discussion, and
Section 5 concludes the paper.

##  2 Proposed Method

Figure 2 shows a block diagram of the proposed method that can be structured
into six functional blocks: (1) transmitted signal design, (2) received signal
processing, (3) filter bank using CWT, (4) deep features, (5) detector, and
(6) fusion and final decisions. Each of these functional units of the proposed
method is discussed as follows:

###  2.1 Transmitted sound signal design

The primary goal of the proposed approach is to detect PAs based on the
reflection characteristics recorded by the smartphone microphone. Therefore,
designing a transmission signal is crucial for reliably characterizing
reflections to detect PAs. In this work, we present a new signal considering
two main fundamental characteristics: (a) selection of frequency and duration
of the signal must enable reliable Signal-to-Noise Ratio (SNR) and (b)
robustness to background noise.

The problem of detecting PA using sound signals is based on analyzing the
reflection characteristics of PAI materials. Therefore, the signal design must
facilitate capturing the reflection from bona fide or PA artefacts and
analyzing the reflection characteristics to detect PAs. Therefore, we are not
interested in measuring the range (distance) and/or resolution (distinguishing
between multiple objects) of the bona fide or the PA artefact. With this
motivation, we introduce a wide rectangular pulse to achieve a sound beam (the
straight line that this pulse can travel in space) that can sufficiently
impact the bona fide face or PA artefact so that reflection with sufficient
energy can be recorded to detect the PAs. Furthermore, the background noise
must be effectively mitigated by sensing the environment to recover high-
quality reflection. Therefore, the proposed signal design introduces a silent
period before transmitting the pulse signal, during which the background noise
is recorded. The recorded background signal can model the background while
processing the received echoes to detect the PAs.

Figure 3: Proposed transmitted signal

Figure 3 illustrates the shape of the transmitted sound signal introduced in
this work. The process of sound signal transmission starts by setting the
microphone ON and the speaker OFF. The microphone in the smartphone acts as
the receiver, and the speaker is the transmitter that transmits the sound
waves as shown in Figure 3. The first part of the signal lasted for 1.5
seconds, during which the microphone was set ON to record the background
noise. The rectangular pulse is then transmitted through the speaker. The
majority of existing smartphones support a 44.1 KHz sampling rate for
microphones; therefore, the highest frequency sensed is 22kHz [20]. Thus, the
proposed signal adopts a rectangular pulse of 21KHz to sense the bona
fide/PAs. The pulse duration lasted 2 seconds because the wider bandwidth
permitted more energy to be recorded on the reflections received using the
microphone. Furthermore, the smartphone is held at a distance of 30 to 45 cm
from the face during verification without obstacles, enabling richer and
discriminant information from the reflected signal to detect the PAs reliably.
After the sound signal is transmitted, both speaker and microphone are set to
OFF for 0.5 seconds to avoid the chances of recording the transmitted signal.
This step is essential to avoid direct interference, which can hide the
reflection signals corresponding to the bona fide and/or PAs. In the last part
of the signal, the microphone was set to ON, so that the reflections were
recorded for 1.5 seconds. Thus, the total length of the signal is 1.5 (to
record background) + 2 (transmitted signal) + 0.5 (idle time) + 1.5 (recording
reflections) = 5.5 seconds.

Figure 4: Qualitative results of CWT-FB for bona fide and PAs

###  2.2 Received signal processing

The received signal $R$ is first subtracted from the background signal $B$ to
obtain clean signal $C_{s}$. In the next step, the clean signal $C_{s}$ is
passed through a matched filter to maximize the signal-to-noise ratio, which
is also known as pulse compression [2]. The match-filtering operation is
performed by correlating the transmitted pulse with the received signal. The
output of the matched filter $M_{c}$ is then used to detect the bona fide /
PAs.

###  2.3 Continuous Wavelet Transform Filter Bank (CWT-FB)

In this work, we used the CWT-FB to extract the time-frequency information to
capture the reflection characteristics of the bona fide and attack samples.
Filter banks are designed for the length of the received signals and use a
Morse Wavelet [11]. In this work, we choose a gamma value equal to 3 and a
time-bandwidth product of 60 because, with gamma = 3, the Morse wavelet is
perfectly symmetric in the frequency domain, allowing for a better capture of
the time-frequency information. The designed CWT-FB comprises ten wavelet
bandpass filters; therefore, the highest-frequency passband is set to 20Khz.
Figure 4 illustrates the output of the CWT filter bank for both the bona fides
and PAs employed in this study. These qualitative results indicate different
time-frequency responses for the bona fide and PAs.

###  2.4 Deep features

In the next step, we extracted the features from the CWT-FB using off-the-
shelf pre-trained Convolutional Neural Networks (CNN). In this work, we
employed EfficientNet b0 [15] by considering the robustness and accuracy in
several applications. Given the CWT-FB image, the deep features were extracted
from the last Batch Normalization layer, resulting in $7$ $\times$ $7$
$\times$ $1280$ features. We chose the last BN layer based on an empirical
analysis that indicated the best detection performance compared to the other
layers in EfficientNet b0 [15].

###  2.5 Detection Module

The deep features are then used to train the classifiers to obtain the
comparison scores. In this work, we employed a linear Support Vector Machines
(SVM) classifier to obtain the detection score. Since the deep features is of
the dimension in $7$ $\times$ $7$ $\times$ $1280$, we employ $7$ $\times$ $7$
= $49$ SVM classifiers that are trained independently on the $49$ different
features of dimension $1$ $\times$ $1280$.

###  2.6 Fusion and final decision

Given the test vector of deep features of dimensions $7$ $\times$ $7$ $\times$
$1280$, we obtained 49 independent detection scores. Finally, the detection
scores were fused using the sum rule to obtain the final detection score as
follows: $D_{s}=\sum_{i=1}^{49}S_{i}$, where ${S_{i}}$ indicates the
individual scores obtained using the SVM and $D_{s}$ indicates the final fused
score. The final score $D_{s}$ is compared against the preset threshold to
classify the received signal as bona fide / PA.

##  3 Acoustic Sound Echo Dataset (ASED)

In this work, we introduce a newly collected Acoustic Sound Echo Dataset
(ASED) comprising 35 data subjects and four different PAIs, including two
types of print attacks, display attacks, and silicone face masks. The proposed
acoustic signaling (transmission and reflection) system was implemented as an
Android application and was installed on a Samsung Galaxy S10. The data were
collected in a laboratory setting, particularly in an indoor scenario
reflecting the office environment. The user holds the phone so that the
frontal camera can show the frontal face of the user. The angle of holding the
phone is between 40-60 degrees such that the user can see the face image on
the smartphone. Normally, the smartphone-to-face distance is between 20-40 cm.
Bona fide data collection was conducted for 20 days in multiple sessions
varying from 2 to 10 days, resulting in 35 to 40 samples for each data
subject. We employed facial images from the data subjects to generate PAs
using different types of artefacts.

To capture the display attack, we employed iPad Pro 12.9, in which the face
image was displayed on a smartphone to collect the data. To collect the print
data, we used two different types of printers. Print-I: The data subject’s
face images were printed using a LaserJet printer with normal print paper.
Print-II: The data subject’s face images are printed using the Dye Sublimation
printer with a glossy paper. The use of two different types of printers allows
for the analysis of the reflection characteristics of the proposed method for
detecting PAs. The silicone face mask dataset was collected by wearing the
silicone mask of the subject. Owing to the high cost of silicone masks, we
used only four silicone face masks to collect the dataset. Thus, the ASED
dataset comprised 1433 bona fides, 1234 display attacks, 500 print-I, 500
print-II, and 1140 silicone samples resulting in a total of . This resulted in
4807 samples, including bona fides and PAs.

Table 2: Statistics of Acoustic Sound Echo Dataset (ASED)

Data Type | Train Set | Test Set
---|---|---
Bona fide | 1003 | 430
Display Attack | 899 | 385
Print-I Attack | 350 | 150
Print-II Attack | 350 | 150
Silicone Attack | 798 | 342

###  3.1 Performance evaluation protocol:

We propose a protocol to evaluate the attack detection performance by dividing
the entire dataset into two independent sets. The training set consisted of
samples collected from 25 subjects and the testing set consisted of samples
collected from 10 data subjects. Table 2 lists the statistics of the training
and testing sample distributions used to evaluate PAD algorithms. However, for
the silicone mask data, we have used two Silcone masks corresponding to unique
identities for training and remaining two for testing.

##  4 Experiments and Results

In this section, we present the quantitative performance of the proposed
acoustic-based facial PAD technique. The performance of the face PAD was
benchmarked using ISO/IEC 30107- 3 [6] metrics such as Attack Presentation
Classification Error Rate (APCER) and bona fide Presentation Classification
Error Rate (BPCER). APCER is defined as the proportion of attack presentations
incorrectly classified as bona fide, whereas BPCER is defined as the portion
of the bona fide incorrectly classified as attack presentation. The Detection-
Equal Error Rate (D-EER) indicates the value that the proportion of APCER is
equal to the portion of BPCER. Extensive experiments were performed to
benchmark the performance of the proposed method, highlighting the role of the
background noise subtraction employed in the proposed transmission signal
design. Furthermore, a comparison with the proposed feature extraction method
using EfficientNet was benchmarked with other off-the-shelf pre-trained CNNs,
such as DenseNet [22], ResNet50 [4] and MobileNetV2 [14].

To effectively analyze the performance of the proposed method for
generalizable PAD, we present quantitative results using two different
protocols: inter and intra experiment. Inter experiment protocol: In this
protocol, the PAD systems were trained and tested with different types of PAI.
Hence, this protocol allowed us to analyze the generalizability of the
proposed method to unknown PAI. Intra experimental protocol: In this protocol,
the PAD system is trained and tested with the same type of PAI. Hence, this
protocol allows the analysis of the robustness of the proposed method to known
PAI.

Table 3: Quantitative performance of the propose method with existing pre-
trained without back ground subtraction

Algorithms | Train Data | Test Data | D-EER | BPCER @ APCER =
---|---|---|---|---
5% | 10%
DenseNet | Attack 1 | Attack 1 | 12.38 | 30.75 | 14.31
Attack 2 | 12.16 | 22.77 | 15.49
Attack 3 | 19.59 | 55.16 | 27.94
Attack 4 | 12.15 | 31.45 | 15.49
Attack 2 | Attack 1 | 11.27 | 25.35 | 12.2
Attack 2 | 8.13 | 11.13 | 7.98
Attack 3 | 10.18 | 10.79 | 14.18
Attack 4 | 10.32 | 16.43 | 10.79
Attack 3 | Attack 1 | 10.28 | 24.41 | 10.32
Attack 2 | 14.65 | 42.72 | 22.16
Attack 3 | 7.32 | 12.22 | 5.86
Attack 4 | 9.79 | 18.17 | 9.85
Attack 4 | Attack 1 | 6.81 | 10.19 | 4.46
Attack 2 | 11.45 | 22.77 | 14.78
Attack 3 | 8.71 | 13.61 | 7.74
Attack 4 | 4.17 | 3.99 | 2.18

Algorithms | Train Data | Test Data | D-EER | BPCER @ APCER =
---|---|---|---|---
5% | 10%
ResNet50 | Attack 1 | Attack 1 | 6.2 | 8.45 | 4.69
Attack 2 | 4.69 | 4.69 | 3.28
Attack 3 | 12.14 | 25 | 15.49
Attack 4 | 4.97 | 5.39 | 1.46
Attack 2 | Attack 1 | 10.53 | 19.15 | 11.5
Attack 2 | 7.44 | 16.66 | 6.33
Attack 3 | 12.12 | 27.23 | 14.55
Attack 4 | 9.14 | 14.78 | 8.45
Attack 3 | Attack 1 | 14.12 | 44.13 | 24.41
Attack 2 | 16.72 | 51.64 | 32.62
Attack 3 | 8.14 | 18.32 | 6.14
Attack 4 | 10.16 | 26.76 | 10.32
Attack 4 | Attack 1 | 5.2 | 7.51 | 2.81
Attack 2 | 5.38 | 8.45 | 3.15
Attack 3 | 6.75 | 7.74 | 3.28
Attack 4 | 2.61 | 1.87 | 0.98

Algorithms | Train Data | Test Data | D-EER | BPCER @ APCER =
---|---|---|---|---
5% | 10%
MobileNetV2 | Attack 1 | Attack 1 | 10.77 | 19.48 | 11.73
Attack 2 | 12.83 | 31.22 | 17.61
Attack 3 | 13.51 | 34.97 | 20.42
Attack 4 | 9.79 | 15.72 | 9.35
Attack 2 | Attack 1 | 8.42 | 11.97 | 8.21
Attack 2 | 8.47 | 8.45 | 7.51
Attack 3 | 8.7 | 12.21 | 8.45
Attack 4 | 8.23 | 8.45 | 7.51
Attack 3 | Attack 1 | 10.77 | 10.17 | 11.51
Attack 2 | 12.41 | 23.23 | 13.84
Attack 3 | 9.27 | 15.72 | 8.45
Attack 4 | 10.85 | 14.78 | 11.26
Attack 4 | Attack 1 | 10.28 | 15.72 | 10.32
Attack 2 | 13.97 | 24.41 | 19.95
Attack 3 | 10.88 | 35.44 | 11.73
Attack 4 | 7.32 | 10.56 | 5.39

|  |  |  | BPCER @ APCER =
---|---|---|---|---
Algorithms | Train Data | Test Data | D-EER | 5% | 10%
|  | Attack 1 | 3.46 | 2.58 | 1.17
|  | Attack 2 |  4.29 | 3.99 | 1.64
|  | Attack 3 | 6.75 | 7.51 | 3.99
| Attack 1 | Attack 4 | 2.61 | 2.11 | 0.7
|  | Attack 1 | 9.41 | 9.62 | 9.15
|  | Attack 2 | 8.58 |  8.68 | 8.45
|  | Attack 3 | 9.51 | 9.62 | 9.62
| Attack 2 | Attack 4 | 8.88 | 9.15 | 8.58
|  | Attack 1 | 3.46 | 2.81 | 1.42
|  | Attack 2 | 1.37 | 0.23 |  0.23
|  | Attack 3 | 1.49 | 0.71 | 0.46
| Attack 3 | Attack 4 | 1.82 | 1.4 | 0.46
|  | Attack 1 | 9.41 | 9.38 | 9.38
|  | Attack 2 | 9.39 | 9.38 | 9.38
|  | Attack 3 | 9.39 | 9.38 | 9.38
Proposed Method | Attack 4 | Attack 4 | 6.91 | 7.74 | 6.33

(a) [D-EER(%): Intra evaluation protocol

(b) [D-EER(%): Inter evaluation protocol

Figure 5: D-EER(%) of the proposed and existing methods on inter and intra
experiments without background subtraction

###  4.1 Results and Discussion: Without Background Subtraction

This section discusses the quantitative performances of the proposed and
existing PAD methods without background subtraction. Thus, the features were
computed directly on the received signal, and experiments were performed using
inter and intra evaluation protocols. Table 3 shows the quantitative
performances of the proposed and existing PAD methods. Attack 1, indicated in
Table 3 corresponds to a display attack, Attack 2 corresponds to a print-I
attack, Attack 3 corresponds to a print-II attack, and Attack 4 corresponds to
a silicone face mask attack. Table 3 presents the quantitative results of both
the intra and inter evaluation protocol and Figure 5(a) and 5(b) shows the bar
chart with D-EER(%) values for both intra and inter evaluation protocol. The
bar chart in Figure 5(b) indicates the inter evaluation protocol in which the
D-EER (%) is plotted by taking the mean of D-EER computed on test attacks.
Based on the obtained results, the following important observations were made.

  * •

In general, the intra experiments indicate better results than the inter
experiments on all four different PAIs. However, it is interesting to note
that the difference in performance between intra and inter experiments on PAIs
is not much different, indicating the generalizability of the proposed
acoustic signal analysis.

  * •

Among the four PAIs employed in this work, the attack potential of these PAIs
depends on the type of feature extraction. For example, Attack 1 indicates the
highest D-EER (%) with DenseNet features and Attack 3 indicates the highest
D-EER (%) with MobileNet and ResNet50. Attack 2 indicated the highest D-EER of
the proposed method (%).

  * •

The proposed feature extraction using Efficientnet has indicated the best
performance on Attacks 1 and 3 in inter and intra-experiments compared to the
three different pre-trained networks employed in this work. However, the
proposed method indicated less performance variation between intra and inter
evaluation protocols.

Table 4: Quantitative performance of the propose method with existing pre-
trained with back ground subtraction

Algorithms | Train Data | Test Data | D-EER | BPCER @ APCER =
---|---|---|---|---
5% | 10%
DenseNet | Attack 1 | Attack 1 | 9.41 | 15.96 | 7.74
Attack 2 | 15.4 | 21.59 | 14.31
Attack 3 | 13.51 | 31.22 | 12
Attack 4 | 8.19 | 11.73 | 5.86
Attack 2 | Attack 1 | 11.77 | 31.22 | 15.13
Attack 2 | 6.07 | 11.13 | 2.34
Attack 3 | 20.27 | 58.45 | 37.79
Attack 4 | 10.59 | 23.47 | 11.26
Attack 3 | Attack 1 | 6.81 | 7.98 | 4.69
Attack 2 | 4.12 | 4.22 | 2.81
Attack 3 | 5.38 | 6.8 | 3.23
Attack 4 | 4.17 | 3.52 | 2.34
Attack 4 | Attack 1 | 7.06 | 11.73 | 4.69
Attack 2 | 7.44 | 10.56 | 4.69
Attack 3 | 14.65 | 44.36 | 26.29
Attack 4 | 4.17 | 3.28 | 2.11

Algorithms | Train Data | Test Data | D-EER | BPCER @ APCER =
---|---|---|---|---
5% | 10%
ResNet50 | Attack 1 | Attack 1 | 3.46 | 3.05 | 2.11
Attack 2 | 6.64 | 7.74 | 5.39
Attack 3 | 6.75 | 7.98 | 4.46
Attack 4 | 3 | 2.58 | 2.11
Attack 2 | Attack 1 | 7.56 | 14.78 | 5.88
Attack 2 | 3.32 | 2.34 | 2.11
Attack 3 | 20.16 | 54.22 | 40.61
Attack 4 | 7.16 | 10.19 | 3.99
Attack 3 | Attack 1 | 9.41 | 13.38 | 9.38
Attack 2 | 9.39 | 13.61 | 8.68
Attack 3 | 8.74 | 13.84 | 7.98
Attack 4 | 5.88 | 6.14 | 3.52
Attack 4 | Attack 1 | 9.41 | 15.49 | 8.65
Attack 2 | 12.13 | 22.55 | 12.91
Attack 3 | 14.77 | 37.79 | 20.65
Attack 4 | 3.26 | 1.87 | 1.17

Algorithms | Train Data | Test Data | D-EER | BPCER @ APCER =
---|---|---|---|---
5% | 10%
MobileNetV2 | Attack 1 | Attack 1 | 9.66 | 15.25 | 9.62
Attack 2 | 12.13 | 25.58 | 14.78
Attack 3 | 10.76 | 19.24 | 11.26
Attack 4 | 9.14 | 11.97 | 6.57
Attack 2 | Attack 1 | 12.13 | 25.82 | 13.84
Attack 2 | 8.13 | 13.84 | 4.92
Attack 3 | 20.16 | 50 | 37.79
Attack 4 | 12.14 | 19.71 | 13.34
Attack 3 | Attack 1 | 11.27 | 23.23 | 12.67
Attack 2 | 8.7 | 16.9 | 6.87
Attack 3 | 11.33 | 34.97 | 12.91
Attack 4 | 6.72 | 7.74 | 5.39
Attack 4 | Attack 1 | 8.42 | 12.91 | 7.51
Attack 2 | 7.32 | 10.32 | 5.39
Attack 3 | 9.39 | 18.17 | 7.74
Attack 4 | 4.16 | 3.51 | 2.11

|  |  |  | BPCER @ APCER =
---|---|---|---|---
Algorithms | Train Data | Test Data | D-EER | 5% | 10%
|  | Attack 1 | 1.85 | 1.17 | 0.99
|  | Attack 2 | 2.06 | 1.64 | 1.17
|  | Attack 3 | 1.94 | 1.64 | 1.17
| Attack 1 | Attack 4 |  1.17 | 0.93 | 0.93
|  | Attack 1 | 7.92 | 8.92 | 6.8
|  | Attack 2 | 5.38 | 7.98 | 3.75
|  | Attack 3 | 9.39 | 9.85 | 9.38
| Attack 2 | Attack 4 | 5.61 |  5.86 | 3.75
|  | Attack 1 | 2.85 | 2.34 | 1.64
|  | Attack 2 | 1.49 | 1.48 | 0.94
|  | Attack 3 | 1.48 | 1.4 | 0.93
| Attack 3 | Attack 4 | 1.44 | 0.93 |  0.49
|  | Attack 1 | 2.14 | 1.46 | 0.72
|  | Attack 2 | 2.18 | 2.11 | 1.41
|  | Attack 3 | 2.16 | 2.11 | 0.99
Proposed Method | Attack 4 | Attack 4 | 0.64 | 0.23 | 0.23

(a) D-EER(%): Intra evaluation protocol

(b) D-EER(%): Inter evaluation protocol

Figure 6: D-EER(%) of the proposed and existing methods on inter and intra
experiments with background subtraction

(a) D-EER(%) performance without background Subtraction

(b) D-EER(%) performance with background Subtraction

Figure 7: Average D-EER(%) of inter and intra experiments with and without
background subtraction

###  4.2 Results and Discussion: With Background Subtraction

This section discusses the quantitative results of the proposed and existing
PAD methods when background subtraction was performed. The uniqueness of the
proposed transmission signal lies in its ability to record the background
before the signal is transmitted and received. Therefore, we can subtract the
background signal from the received signal to improve the SNR and contribute
to reliable detection of PAI. Table 4 presents the quantitative performances
of the proposed and existing PAD techniques with inter and intra evaluation
protocols. Figure 6(a) and 6(b) show bar charts with the D-EER(%) for both
intra and inter experiments. Based on the results obtained, the following are
important observations:

  * •

The detection error is less for the intra experiments compared to the inter
experimental protocol with both proposed and existing feature extraction
techniques. However, the average difference in performance between the intra
and inter protocols was minimal. Therefore, the use of acoustic signals can
result in a generalizable PAD.

  * •

Among the four PAIs employed in this work, the attack potential of these PAIs
depends on the type of feature extraction. For example, Attack 1 indicates the
highest D-EER (%) with DenseNet features, and Attack 3 indicates the highest
D-EER (%) with MobileNet and ResNet50. Attack 2 indicates the proposed
method’s highest D-EER (%).

  * •

The proposed feature extraction using Efficientnet has indicated the best
performance on inter and intra experiments compared to the three different
pre-trained networks employed in this work. The results indicated the
robustness of the proposed method to background noise, as background noise
subtraction was performed in these experiments.

Figure 7(a) and 7(b) show the average performance of the proposed and existing
feature extraction methods in the inter and intra experiments with and without
the background subtraction method. The following are the important
observations:

  * •

The detection performance of the PAD algorithms indicates improved performance
when background subtraction is performed. This demonstrated the superiority of
the proposed transmission signal.

  * •

The proposed feature extraction based on the Efficientnet indicates the best
performance on with and without background subtraction compared to other
feature extraction techniques.

  * •

The proposed method indicates the little difference between intra and inter-
performance variation on both with and without background subtraction. The
proposed method indicates an average D-EER (%) of 6.35(%) and 5.11(%) on inter
and intra experiments without using background subtraction. With background
subtraction, the proposed method indicated an average D-EER (%) of 3.36(%) and
2.33(%) in inter and intra experiments respectively. The low difference in the
performance of the proposed method with inter and intra variations indicates
the generalizability of the proposed method.

##  5 Conclusion and Future Work

Reliable detection of unknown PA is essential for enabling trustworthy face
recognition applications on smartphones. In this study, we presented a novel
method for a generalizable face PAD on smartphones using acoustic sound
echoes. A novel signal based on the wide pulse is proposed to effectively
model the background noise and increase the signal-to-noise ratio. The
reflected signals were processed to remove background noise and obtain the
time-frequency representation. We then computed the deep features using pre-
trained EfficientNet by extracting the features from the BatchNorm layer. The
BatchNorm layer provides 49 different embeddings used to train 49 independent
linear SVMs whose decisions are fused to make the final decision. Extensive
experiments are presented to benchmark the performance of the proposed method
using intra and inter evaluation protocols. Additional experiments are
presented to highlight the importance of background subtraction in improving
the robustness and accuracy of the face PAD. The obtained results demonstrated
the generalizability of the proposed method across unknown PAIs. Future work
will extend the proposed method to different types of smartphones. Further
extensive data collection is planned in the noisy conditions to benchmark the
PAD algorithms based on acoustic reflections.

## References

  * [1] H. Chen, W. Wang, J. Zhang, and Q. Zhang.  Echoface: Acoustic sensor-based media attack detection for face authentication.  IEEE Internet of Things Journal, 7(3):2152–2159, 2019.
  * [2] E. C. Farnett, G. H. Stevens, and M. Skolnik.  Pulse compression radar.  Radar handbook, 2:10–11, 1990.
  * [3] A. George and S. Marcel.  Deep pixel-wise binary supervision for face presentation attack detection.  In ICB, pages 1–8. IEEE, 2019.
  * [4] K. He, X. Zhang, S. Ren, and J. Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [5] G. Heusch, A. George, D. Geissbühler, Z. Mostaani, and S. Marcel.  Deep models and shortwave infrared information to detect face presentation attacks.  IEEE Transactions on Biometrics, Behavior, and Identity Science, 2(4):399–409, 2020.
  * [6] ISO/IEC JTC1 SC37 Biometrics.  ISO/IEC 30107-3. Information Technology - Biometric presentation attack detection - Part 3: Testing and Reporting.  International Organization for Standardization, 2017.
  * [7] C. Kong, K. Zheng, S. Wang, A. Rocha, and H. Li.  Beyond the pixel world: A novel acoustic-based face anti-spoofing system for smartphones.  IEEE Transactions on Information Forensics and Security, 17:3238–3253, 2022.
  * [8] H. Li, W. Li, H. Cao, S. Wang, F. Huang, and A. C. Kot.  Unsupervised domain adaptation for face anti-spoofing.  IEEE Transactions on Information Forensics and Security, 13(7):1794–1809, 2018.
  * [9] Y. Liu, Z. Li, S. Pan, C. Gong, C. Zhou, and G. Karypis.  Anomaly detection on attributed networks via contrastive self-supervised learning.  IEEE transactions on neural networks and learning systems, 2021.
  * [10] U. Muhammad, Z. Yu, and J. Komulainen.  Self-supervised 2d face presentation attack detection via temporal sequence sampling.  Pattern Recognition Letters, 156:15–22, 2022.
  * [11] S. C. Olhede and A. T. Walden.  Generalized morse wavelets.  IEEE Transactions on Signal Processing, 50(11):2661–2670, 2002.
  * [12] Y. Qin, Z. Yu, L. Yan, Z. Wang, C. Zhao, and Z. Lei.  Meta-teacher for face anti-spoofing.  IEEE transactions on pattern analysis and machine intelligence, 2021.
  * [13] R. Ramachandra and C. Busch.  Presentation attack detection methods for face recognition systems: A comprehensive survey.  CSUR, 50(1):1–37, 2017.
  * [14] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen.  Mobilenetv2: Inverted residuals and linear bottlenecks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4510–4520, 2018.
  * [15] M. Tan and Q. Le.  Efficientnet: Rethinking model scaling for convolutional neural networks.  In International conference on machine learning, pages 6105–6114. PMLR, 2019.
  * [16] C.-Y. Wang, Y.-D. Lu, S.-T. Yang, and S.-H. Lai.  Patchnet: A simple face anti-spoofing framework via fine-grained patch recognition.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20281–20290, 2022.
  * [17] Z. Yu, R. Cai, Z. Li, W. Yang, J. Shi, and A. C. Kot.  Benchmarking joint face spoofing and forgery detection with visual and physiological cues.  arXiv preprint arXiv:2208.05401, 2022.
  * [18] Z. Yu, X. Li, J. Shi, Z. Xia, and G. Zhao.  Revisiting pixel-wise supervision for face anti-spoofing.  IEEE Transactions on Biometrics, Behavior, and Identity Science, 3(3):285–295, 2021.
  * [19] Z. Yu, Y. Qin, X. Li, C. Zhao, Z. Lei, and G. Zhao.  Deep learning for face anti-spoofing: A survey.  IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022.
  * [20] B. Zhou, J. Lohokare, R. Gao, and F. Ye.  Echoprint: Two-factor authentication using acoustics and vision on smartphones.  In Proceedings of the 24th Annual International Conference on Mobile Computing and Networking, pages 321–336, 2018.
  * [21] B. Zhou, Z. Xie, Y. Zhang, J. Lohokare, R. Gao, and F. Ye.  Robust human face authentication leveraging acoustic sensing on smartphones.  IEEE Transactions on Mobile Computing, 21(8):3009–3023, 2021.
  * [22] Y. Zhu and S. Newsam.  Densenet for dense flow.  In 2017 IEEE international conference on image processing (ICIP), pages 790–794. IEEE, 2017.