#  Wavelet characterizations of the Sobolev wavefront set: bandlimited
wavelets and compactly supported wavelets

Hartmut Führ, Mahya Ghandehari  H. Führ
Lehrstuhl für Geometrie und Analysis
RWTH Aachen University
D-52056 Aachen
Germany  M. Ghandehari
Department of Mathematical Sciences
University of Delaware
Newark, DE 19716
USA

###### Abstract.

We consider the problem of characterizing the Sobolev wavefront set of a
tempered distribution $u\in\mathcal{S}^{\prime}(\mathbb{R}^{d})$ in terms of
its continuous wavelet transform, with the latter being defined with respect
to a suitably chosen dilation group $H\subset{\rm GL}(\mathbb{R}^{d})$. We
derive necessary and sufficient criteria for elements of the Sobolev wavefront
set, formulated in terms of the decay behaviour of a given generalized
continuous wavelet transform. It turns out that the characterization of
directed smoothness of finite order can be performed in the two important
cases: (1) bandlimited wavelets, and (2) wavelets with finitely many vanishing
moments (e.g. compactly supported wavelets).

The main results of this paper are based on a number of fairly technical
conditions on the dilation group. In order to demonstrate their wide
applicability, we exhibit a large class of generalized shearlet groups in
arbitrary dimensions fulfilling all required conditions, and give estimates of
the involved constants.

Keywords: wavefront set; square-integrable group representation; continuous
wavelet transform; anisotropic wavelet systems; shearlets

AMS Subject Classification: 42C15; 42C40; 46F12

##  1\. Introduction

A recurring theme in wavelet analysis and its various generalizations is the
relationship between smoothness of the analyzed function on the one hand, and
the decay of its wavelet coefficients on the other. Global smoothness, as
quantified by norms of smoothness spaces such as Besov spaces, is related to
global decay properties, whereas local smoothness is related to a more
localized decay behaviour of the coefficients. This paper is concerned with
the later class of questions, for a large class of generalized continuous
wavelet transforms.

Our aim is to give criteria for $N$-regular directed points of a tempered
distribution $u\in{\mathcal{S}}^{\prime}({\mathbb{R}}^{d})$ in terms of
continuous wavelet transform decay. A wavelet transform is defined with
respect to a suitably chosen dilation group $H\subset{\rm GL}(\mathbb{R}^{d})$
and a wavelet vector $\phi\in{\mathcal{S}}({\mathbb{R}}^{d})$. We obtain such
criteria for two important cases: (1) when the wavelet vector is compactly
supported in frequency, and (2) when the wavelet vector has a certain minimal
number of vanishing moments; this case allows to use wavelet vectors that are
compactly supported.

The $N$-regular directed points, formally defined in equation (4) below,
describe the oriented local regularity of a tempered distribution $u$: if
$(x,\xi)$ is an $N$-regular directed point of $u$, then $u$ can be considered
smooth of order $N$ at $x$, when viewed in direction $\xi$. The $N$-wavefront
set $WF^{N}(u)$, a.k.a. the Sobolev wavefront set of $u$ of order $N$,
consists of all directed points $(x,\xi)$ that are not $N$-regular directed
points of $u$. We typically describe the wavefront set $WF^{N}(u)$ by giving
criteria for elements in its complement.

Ideally, we would like to achieve results of the following form:

$\displaystyle\quad(x,\xi)\mbox{ is an $N$-regular directed point of $u$}$
$\displaystyle\Leftrightarrow$ $\displaystyle\exists\mbox{ neighborhood
}U\mbox{ of }x\ \ \forall\,y\in U$ (1) $\displaystyle\forall\,h\in
K~{}:|W_{\psi}u(y,h)|\leq C\|h\|^{N}~{},$

where $K\subset H$ is a suitable subset of dilations which explicitly depends
on $\psi$ and a certain (sufficiently small) cone containing directions near
$\xi$. Intuitively, $K$ contains those $h\in H$ such that $\pi(0,h)\psi$ is a
small-scale wavelet oscillating in direction $\xi$. A characterization for
wavefront sets of infinite order - similar to the one in (1) - was given in
[5]; here the decay of a fixed order $N$ on the right-hand side is replaced by
decay of arbitrary degree. For $N$-regular directed points, we will settle for
more modest statements, of the following types:

$\displaystyle(x,\xi)\mbox{ is an $N$-regular directed point of $u$}$
$\displaystyle\Rightarrow$ $\displaystyle\exists\mbox{ neighborhood }U\mbox{
of }x\ \forall\,y\in U\ \forall\,h\in K_{1}~{}:|W_{\psi}u(y,h)|\leq
C\|h\|^{N-\delta_{1}}~{},$

typically called a “direct” theorem, as well as an “inverse” theorem (both in
the parlance of [11]), namely

$\displaystyle\exists\mbox{ neighborhood }U\mbox{ of }x\ \forall\ y\in U\
\forall\,h\in K_{2}:|W_{\psi}u(y,h)|\leq C\|h\|^{\alpha N+\delta_{2}}$
$\displaystyle\Rightarrow$ $\displaystyle(x,\xi)\mbox{ is an $N$-regular
directed point of $u$}$

with suitable constants $\alpha\geq 1,\delta_{1},\delta_{2}\geq 0$ and
suitably defined subsets $K_{1},K_{2}\subset H$ of dilations. For the case of
the shearlet transform in dimension two, a similar characterization had been
obtained by Grohs [11]. As related work, discussing this question with varying
degrees of generality, we mention [13, 16, 12]. We point to the gap between
the exponents of the direct and the inverse statements in (1) and (1). To our
knowledge, this gap cannot be closed; see [11] for results of similar type on
shearlets. Consequently, the results of our paper provide a near
characterization of $N$-regular directed points. In order to extend such
characterizations to a more general setup, in particular for expansion to a
wide variety of generalized wavelets, we adopt the point of view from [5],
with the aim of developing a comprehensive and unified approach. Just as in
the mentioned paper, a recurrent theme in our work is the need to understand
the influence of structural properties of the underlying group on the problem
at hand.

This article contains several substantial generalizations and adaptations of
the setup from [5]. Most importantly, the investigation of finite degrees of
smoothness makes it possible to include larger classes of analyzing wavelets.
The results of [5] relied on using bandlimited wavelets, i.e. Schwartz
functions whose Fourier support is compactly supported, well away from the
zero frequency. This leads to wavelet coefficient decay of arbitrary orders,
provided the analyzed signal is sufficiently smooth. Restricting the
investigation of smoothness and decay behaviour to finite orders allows to
replace the bandlimited analyzing wavelets by a more general class of
wavelets, and thus to employ wavelets that are compactly supported in space
rather than in frequency. The decisive feature of the wavelet which enables
this replacement is the number of vanishing moments (see Definition 4.5 for a
precise formulation of vanishing moments).

Using the vanishing moment criteria results in significant consequences in our
theory. Firstly, the formulation of direct and inverse theorems for the (near)
characterization of $N$-regular directed points in this setting requires lower
bounds for the number of vanishing moments on the analyzing wavelet. The
recurrent theme of the dependence of the developed criteria on the underlying
group becomes visible here, as well. Secondly, wavelets with finitely many
vanishing moments can be real-valued. The symmetry properties of these
wavelets in the Fourier domain make them essentially incapable of
distinguishing the frequency direction $\xi$ from its opposite $-\xi$. In
order to address these issues, we replace the wavefront set by the so-called
unsigned wavefront set, and reformulate the direct and inverse theorems
accordingly. We emphasize that these issues, in particular the inability of
the wavelet transform criteria to distinguish opposite directions of the
wavefront set, are also present in the precursor papers [15, 11] (albeit not
addressed explicitly), but not in [5]. We point to subsection 6.1 for an
extended comparison of our results to the predecessor papers [15, 11].

###  1.1. Overview of the paper and main results

The chief purpose of this paper is to adapt and extend the previously
developed techniques for wavelet characterization of singularities, in order
to allow the treatment of finite degrees of smoothness, using compactly
supported wavelets. The progress of this endeavour can be traced through the
main results of our paper, which are the following:

  * •

Theorem 3.4, which establishes criteria for points in the Sobolev wavefront
set, using wavelet coefficient decay with respect to bandlimited wavelets;

  * •

Theorem 5.4, showing that these criteria can be transferred between different
wavelets, as soon as the cross-kernel associated to the wavelets decays
sufficiently quickly;

  * •

Theorem 5.8 and Corollary 5.9, which establish Sobolev wavefront set criteria
using compactly supported wavelets;

  * •

Theorem 6.12, which guarantees the applicability of the previous results for a
large class of shearlet dilation groups.

The remainder of the paper is divided into 5 sections. Section 2 reviews the
basic definitions regarding the Fourier and wavelet transforms, $(N-)$regular
directed points and similar matters. Subsection 2.3 recalls the conditions on
the dilation group needed to characterize elements of the wavefront set in
terms of wavelet coefficient decay, and recalls related results from [5]. This
section also contains the definition of the cone-affiliated subsets
$K_{o/i}(W,V,R)\subset H$ that are instrumental in the precise formulation of
the local wavelet coefficient decay conditions on which the wavelet based
criteria for regular directed points are based. This section prepares the
necessary background for both the formulation and the proof of Theorem 3.4,
presented in the subsequent section. This result formulates necessary and
sufficient criteria for $N$-regular directed points formulated in terms of
wavelet coefficient decay. Even though its proof is largely an adaptation of
[5, Theorem 3.5], we include a sketch of the proof of Theorem 3.4, as it will
be used in the main result of the subsequent section.

Theorem 3.4 only allows to use analyzing wavelets that are compactly supported
in frequency, which raises the question whether a similar characterization for
other types of wavelets is possible as well. The extension to more general
wavelets requires the introduction of various auxiliary notions, definitions
and results. Introducing these is the main purpose of Section 4. To begin
with, since vanishing moment criteria open the door to the use of real-valued
wavelets, we introduce the unsigned wavefront set and exhibit its basic
properties in Subsection 4.1. We then review the important definitions and
results in connection with vanishing moments, which were introduced in [7, 8,
10], in Subsection 4.2. The chief purpose of these results is to guarantee a
certain decay of wavelet coefficients, once both the wavelet and the analyzed
function are sufficiently smooth, with sufficiently many vanishing moments. We
then set out to write an explicit cross-kernel convolution formula
(Proposition 4.10) which will be used later on.

In Section 5, we use the theory developed in Sections 3 and 4 to extend our
scheme to compactly supported wavelets. Theorem 5.4 is the central tool for
the transfer of wavelet coefficient decay statements between pairs of wavelets
with sufficiently many vanishing moments. As such it is of considerable
independent interest, since it establishes that the local wavelet coefficient
decay of a given signal genuinely reflects properties of the signal, once the
analyzing wavelet has a certain oscillatory behaviour. As a surprising
byproduct of Theorem 5.4, we are able to close a gap in Theorem 3.4; this
result is stated in Corollary 5.6.

We subsequently use Theorem 5.4 to prove Theorem 5.8, showing that wavelets
that are compactly supported in space (rather than in frequency) can also be
used for the near characterization of unsigned regular directed points of
finite order. As a further application we note in Corollary 5.9 that real-
valued wavelets with vanishing moments of arbitrary order can be used to
characterize unsigned regular directed points of infinite order.

The closing section 6 verifies the extensive list of conditions imposed on the
dilation group throughout sections 3 – 5 for the class of shearlet dilation
groups. We thereby exhibit a large class of dilation groups in arbitrary
dimensions to which the main results of this paper are applicable, with
concrete estimates for all constants involved in the criteria. These
observations are summarized in Theorem 6.12. This result also fosters an
understanding of the influence of various features of the underlying group on
properties of the wavelet transform. We close by comparing our results for
generalized shearlet groups to the main results of [15, 11] in Subsection 6.1.

##  2\. Basic definitions regarding regular directed points and wavelet
transforms

###  2.1. $N$-Regular directed points and the wavefront set

First let us introduce some notation. Given $R>0$ and $x\in\mathbb{R}^{d}$, we
let $B_{R}(x)$ and $\overline{B_{R}}\left(x\right)$ denote the open/closed
ball with radius $R$ and center $x$, respectively. We let
$S^{d-1}\subset\mathbb{R}^{d}$ denote the unit sphere. By a neighborhood of
$\xi\in S^{d-1}$, we will always mean a _relatively open_ set $W\subset
S^{d-1}$ with $\xi\in W$. Given $R>0$ and an open set $W\subset S^{d-1}$, we
define the (truncated) cone generated by $W$ as

$C(W):=\left\\{r\xi^{\prime}\,\middle|\,\xi^{\prime}\in
W,\,r>0\right\\}=\left\\{\xi\in\mathbb{R}^{d}\setminus\left\\{0\right\\}\,\middle|\,\frac{\xi}{\left|\xi\right|}\in
W\right\\}\,\,\,\text{ and }\,\,\,C(W,R):=C(W)\setminus\overline{B_{R}}(0).$

Both sets are clearly open subsets of
$\mathbb{R}^{d}\setminus\left\\{0\right\\}$ and thus of $\mathbb{R}^{d}$.

Given a tempered distribution $u$ and an integer $N\in\mathbb{N}$, we call
$(x,\xi)\in\mathbb{R}^{d}\times S^{d-1}$ an $N$-regular directed point of $u$
if there exists $\varphi\in C_{c}^{\infty}(\mathbb{R}^{d})$, identically one
in a neighborhood of $x$, as well as a $\xi$-neighborhood $W\subset S^{d-1}$
and a constant $C_{N}>0$ with

$\forall\,\xi^{\prime}\in C\left(W\right)~{}:~{}\left|\widehat{\varphi
u}(\xi^{\prime})\right|\leq C_{N}(1+|\xi^{\prime}|)^{-N}.$ (4)

Note that this condition effectively only concerns the behaviour at large
frequencies: we may replace $C\left(W\right)$ in inequality (4) by $C(W,R)$
for any $R>0$, without changing the definition of an $N$-regular point. In
practice, the suitable $R$ will be chosen depending on $\xi$. The
$N$-wavefront set $WF^{N}(u)$ consists of all directed points $(x,\xi)$ that
are not $N$-regular directed points of $u$.

###  2.2. Continuous wavelet transforms in higher dimensions

We now introduce the necessary notions pertaining to continuous wavelet
transforms in some detail. We fix a closed matrix group $H<{\rm
GL}(d,\mathbb{R})$, the so-called dilation group, and let
$G=\mathbb{R}^{d}\rtimes H$. This is the group of affine mappings generated by
$H$ and all translations. Elements of $G$ are denoted by pairs
$(x,h)\in\mathbb{R}^{d}\times H$, and the product of two group elements is
given by $(x,h)(y,g)=(x+hy,hg)$. The left Haar measure of $G$ is given by
${\rm d}(x,h)=|\det(h)|^{-1}{\rm d}x\,{\rm d}h$, where ${\rm d}x$ and ${\rm
d}h$ are the left Haar measures of $\mathbb{R}^{d}$ and $H$ respectively. The
group $H$ acts on $\mathbb{R}^{d}$ by matrix multiplication. The dual action
is just the (right) action $\mathbb{R}^{d}\times H\ni(\xi,h)\mapsto
h^{T}\xi\in\mathbb{R}^{d}$.

The group $G$ acts unitarily on ${L}^{2}(\mathbb{R}^{d})$ by the quasi-regular
representation defined by

$[\pi(x,h)f](y)=|\det(h)|^{-1/2}\cdot f\left(h^{-1}(y-x)\right)~{}.$ (5)

The dilation group $H$ is called irreducibly admissible, if the associated
quasi-regular representation $\pi$ is irreducible and square-integrable. By
[6], this property has a characterization in terms of the dual action: $H$ is
irreducibly admissible if and only if there is a unique conull open orbit
$\mathcal{O}=H^{T}\xi_{0}$ of the dual action, with the additional property
that the stabilizer groups associated to $\mathcal{O}$ are compact.

Given functions $\psi,f\in{L}^{2}(\mathbb{R}^{d})$, the continuous wavelet
transform of $f$ with respect to $\psi$ is the function
$W_{\psi}f:G\to\mathbb{C}$, $W_{\psi}f(x,h)=\langle f,\pi(x,h)\psi\rangle$. In
this context, $\psi$ is called the analyzing wavelet. Moreover, $\pi$ induces
an action of $G$ on $\mathcal{S}(\mathbb{R}^{d})$, the space of Schwartz
functions. We write
$\left\langle\cdot\mid\cdot\right\rangle:\mathcal{S}^{\prime}(\mathbb{R}^{d})\times\mathcal{S}\left(\mathbb{R}^{d}\right)\to\mathbb{C}$
for the natural extension of the ${L}^{2}$-scalar product
$\langle\cdot,\cdot\rangle$, which means $\langle
u\mid\psi\rangle:=u\left(\overline{\psi}\right)$. For future reference, let us
observe that a straightforward calculation yields

$\left[\mathcal{F}\left(\pi\left(x,h\right)f\right)\right]\left(\xi\right)=\left|\det\left(h\right)\right|^{1/2}\cdot
e^{-2\pi i\left\langle
x,\xi\right\rangle}\cdot\widehat{f}\left(h^{T}\xi\right)$ (6)

for
$f\in{L}^{1}\left(\mathbb{R}^{d}\right)+{L}^{2}\left(\mathbb{R}^{d}\right)$.
Here, as in the remainder of the paper, we use the convention

$\mathcal{F}f\left(\xi\right)=\widehat{f}\left(\xi\right)=\int_{\mathbb{R}^{d}}f\left(x\right)\cdot
e^{-2\pi i\left\langle x,\xi\right\rangle}\,{\rm d}x$

for the Fourier transform of $f\in L^{1}\left(\mathbb{R}^{d}\right)$.

Using a Schwartz function $\psi$ as analyzing wavelet allows to extend the
continuous wavelet transform to the space of tempered distributions in a
straightforward manner. Namely, given a tempered distribution
$u\in\mathcal{S}^{\prime}\left(\mathbb{R}^{d}\right)$ and some
$\psi\in\mathcal{S}(\mathbb{R}^{d})$, we define the wavelet transform of $u$
with respect to $\psi$ by

$W_{\psi}u:G\to\mathbb{C},~{}(x,h)\mapsto\langle u\mid\pi(x,h)\psi\rangle.$

Throughout this paper, wavelets $\psi$ will always come from the Schwartz
class, and accordingly the wavelet transform is defined on
$\mathcal{S}^{\prime}(\mathbb{R}^{d})$.

A wavelet $\psi\in\mathcal{S}(\mathbb{R}^{d})$ is called admissible if it
fulfills the admissibility condition

$C_{\psi}=\int_{H}|\widehat{\psi}(h^{T}\xi)|^{2}\text{\,d}h<\infty\mbox{ for
all }\xi\in{\mathcal{O}}~{}.$

The most important consequence of the admissibility condition is the inversion
formula

$f=\frac{1}{C_{\psi}}\int_{G}W_{\psi}f(x,h)\,\pi(x,h)\psi\,d(x,h)$

valid (in the weak operator sense) for all $f\in{L}^{2}(\mathbb{R}^{d})$. Note
that this inversion formula does not necessarily apply to tempered
distributions.

###  2.3. Technical conditions on the dual action

We will write $V\Subset\mathcal{O}$ to indicate that
$V,\mathcal{O}\subset\mathbb{R}^{d}$ are open sets and that the closure
$\overline{V}\subset\mathcal{O}$ is a compact subset of $\mathcal{O}$. We
assume that the dilation group is irreducibly admissible, and so the
associated quasi-regular representation is irreducible and square-integrable.
As a consequence of this assumption, all the properties required in [5,
Assumption 2.1] are guaranteed; we spell this out in the following lemma. We
refer to [5, Lemma 2.2] and the text leading up to it for a proof. Observe
that part (b) of the following lemma is slightly stronger than the
corresponding part (b) from [5, Assumption 2.1], but the reasoning from [5] in
fact shows the stronger statement. In particular, observe that
$\xi\in\mathcal{O}$ entails $-\xi\in\mathcal{O}$.

######  Lemma 2.1.

Suppose the dilation group is irreducibly admissible. Then there exists an
open, conull, $H^{T}$-invariant subset $\mathcal{O}\subset\mathbb{R}^{d}$ with
the following properties:

  1. (a)

For each $\xi\in\mathcal{O}$, we have
$\mathbb{R}^{\ast}\xi\subset\mathcal{O}$, where
$\mathbb{R}^{\ast}:=\mathbb{R}\setminus\\{0\\}$.

  2. (b)

Any Schwartz function $\psi$ such that $\widehat{\psi}$ is compactly supported
inside $\mathcal{O}$ is admissible, i.e., satisfies

$\forall\,\xi\in\mathcal{O}~{}:~{}\int_{H}|\widehat{\psi}(h^{T}\xi)|^{2}\,{\rm
d}h<\infty.$

We next formally define the sets $K_{i}$ and $K_{o}$ which associate dilations
to directions.

######  Definition 2.2.

Let $\varnothing\neq W\subset S^{d-1}$ be open with $W\subset\mathcal{O}$
(which implies $C\left(W\right)\subset\mathcal{O}$). Furthermore, let
$\varnothing\neq V\Subset\mathcal{O}$ and $R>0$. We define

$K_{i}(W,V,R):=\left\\{h\in H\,\middle|\,h^{-T}V\subset C(W,R)\right\\}$

as well as

$K_{o}(W,V,R):=\left\\{h\in H\,\middle|\,h^{-T}V\cap
C(W,R)\not=\varnothing\right\\}.$

If the parameters are provided by the context, we will simply write $K_{i}$
and $K_{o}$. Here, the subscripts $i/o$ stand for “inner/outer”.

These two types of sets are the central tool of our analysis. The intuition
behind their definition is that $K_{i}$ contains all dilations $h$ with the
property that the wavelets $\pi(y,h)\psi$ only “see” directions in the cone
$C(W,R)$, as long as ${\rm supp}\left(\smash{\widehat{\psi}}\right)\subset V$
holds. Here, we used the property ${\rm
supp}\left(\mathcal{F}\left[\pi\left(y,h\right)\psi\right]\right)\subset
h^{-T}{\rm supp}\left(\smash{\widehat{\psi}}\right)\subset h^{-T}V$ which is
immediate from equation (6). As a result, local regularity in these directions
should entail a decay estimate for the wavelet coefficients
$\left(W_{\psi}u\right)\left(y,h\right)$ with $h\in K_{i}$.

On the other hand, $K_{o}$ contains all those dilations that contribute to the
(formal) _wavelet reconstruction_

$\widehat{\varphi
u}\left(\xi\right)=\int_{\mathbb{R}^{d}}\int_{H}\left(W_{\psi}\varphi
u\right)\left(y,h\right)\cdot\left(\mathcal{F}\left[\pi\left(y,h\right)\psi\right]\right)\left(\xi\right)\,\frac{{\rm
d}h}{\left|\det\left(h\right)\right|}\,{\rm d}y$ (7)

of the frequency content of a (localized) tempered distribution $\varphi u$,
for $\xi\in C(W,R)$, again under the assumption ${\rm
supp}\left(\smash{\widehat{\psi}}\right)\subset V$. Thus, decay estimates for
wavelet coefficients with dilations in $h\in K_{o}$ should allow to predict
local regularity of $u$ in these directions.

Next, we show how the sets $K_{i}$ and $K_{o}$ are related to each other when
moving along the points of the orbit.

######  Lemma 2.3.

Let $\xi_{0},\xi_{1}\in\mathcal{O}\cap S^{d-1}$, such that
$h^{T}\xi_{0}=\xi_{1}$.

  1. (a)

For every $R_{1}>0$ and $\xi_{1}$-neighborhood $W_{1}$ in $S^{d-1}$, there
exists $R_{0}>0$ and a $\xi_{0}$-neighborhood $W_{0}$ such that

$h^{T}C(W_{0},R_{0})\subseteq C(W_{1},R_{1}).$

For all $R_{0},R_{1}>0$ and neighborhoods $W_{0},W_{1}$ satisfying the above
inclusion, we have

$K_{i}(W_{1},V,R_{1})\supseteq h^{-1}K_{i}(W_{0},V,R_{0}).$

  2. (b)

For every $R_{0}>0$ and $\xi_{0}$-neighborhood $W_{0}$ in $S^{d-1}$, there
exists $R_{1}>0$ and a $\xi_{1}$-neighborhood $W_{1}$ such that

$C(W_{1},R_{1})\subseteq h^{T}C(W_{0},R_{0}).$

For all $R_{0},R_{1}>0$ and neighborhoods $W_{0},W_{1}$ satisfying the above
inclusion, we have

$K_{o}(W_{1},V,R_{1})\subseteq h^{-1}K_{o}(W_{0},V,R_{0}).$

###### Proof.

To prove (a), let $W_{1},R_{1}$ be given. Pick $\delta>0$ such that for all
$v\in B_{\delta}(\xi_{1})$ one has $\frac{v}{|v|}\in W_{1}$ (see the proof of
[5, Lemma 4.5]). Let $R_{0}=R_{1}\cdot\|h^{-1}\|$ and
$W_{0}=(h^{-T}B_{\delta}(\xi_{1}))\cap S^{d-1}$, which is indeed a
neighborhood of $\xi_{0}$ in $S^{d-1}$. To prove that $R_{0},W_{0}$ are as
desired, let $x\in C(W_{0},R_{0})$. Hence $x=rw$, with $r>R_{0}$, $w\in
W_{0}$. Then $v=h^{T}w\in B_{\delta}(\xi_{1})$, and by choice of $\delta$, one
has $\frac{v}{|v|}\in W_{1}$. It follows that

$h^{T}x=rh^{T}w=r|v|\cdot\frac{v}{|v|}\in\mathbb{R}^{+}\cdot W_{1}~{}.$

Finally, by choice of $x$,

$R_{1}=R_{0}\|h^{-1}\|^{-1}<|x|\|h^{-1}\|^{-1}=|h^{-T}h^{T}x|\|h^{-1}\|^{-1}\leq|h^{T}x|~{},$

which establishes that $h^{T}x\in C(W_{1},R_{1})$. This proves the first
inclusion in (a). Next, assume $h^{T}C(W_{0},R_{0})\subseteq C(W_{1},R_{1})$,
and pick $g\in K_{i}(W_{0},V,R_{0})$. So we have

$g^{-T}V\subseteq C(W_{0},R_{0})\subseteq h^{-T}C(W_{1},R_{1}),$

and thus $h^{-1}g\in K_{i}(W_{1},V,R_{1})$.

To prove (b), let $W_{0},R_{0}$ be as given in the assumption. We apply (a)
for $h^{-T}\xi_{1}=\xi_{0}$ to find $R_{1},W_{1}$ such that
$h^{-T}C(W_{1},R_{1})\subseteq C(W_{0},R_{0})$. Multiplying both sides by
$h^{T}$ yields the desired inclusion. To prove the next inclusion, assume
$C(W_{1},R_{1})\subseteq h^{T}C(W_{0},R_{0})$, and let $g\in
K_{o}(W_{1},V,R_{1})$. By definition, this means $g^{-T}V\cap
C(W_{1},R_{1})\not=\varnothing$. As $C(W_{1},R_{1})\subseteq
h^{T}C(W_{0},R_{0})$, this implies

$g^{-T}V\cap h^{T}C(W_{0},R_{0})\not=\varnothing~{},$

or $(hg)^{-T}V\cap C(W_{0},R_{0})\not=\varnothing$, which entails $hg\in
K_{o}(W_{0},V,R_{0})$. ∎

We now come to an important technical assumption concerning the dual action.
Given a matrix $h$, we let $\|h\|$ denote the operator norm of the induced
linear map with respect to the euclidean norm.

######  Definition 2.4.

Let $\xi\in\mathcal{O}\cap S^{d-1}$ and $\varnothing\neq V\Subset\mathcal{O}$.
The dual action is called $V$-microlocally admissible in direction $\xi$ if
there exists a $\xi$-neighborhood $W_{0}\subset\mathcal{O}\cap S^{d-1}$ and
some $R_{0}>0$ such that the following hold:

  1. (a)

There exist $\alpha_{1}>0$ and $C>0$ such that

$\|h^{-1}\|\leq C\cdot\|h\|^{-\alpha_{1}}$

holds for all $h\in K_{o}(W_{0},V,R_{0})$.

  2. (b)

There exists $\alpha_{2}>0$ such that

$\int_{K_{o}(W_{0},V,R_{0})}\|h\|^{\alpha_{2}}\,{\rm d}h<\infty.$

The dual action is called microlocally admissible in direction $\xi$ if it is
$V$-microlocally admissible in direction $\xi$ for some $\varnothing\neq
V\Subset\mathcal{O}$. It is called globally $V$-microlocally admissible if
there is $\varnothing\neq V\Subset\mathcal{O}$ such that the dual action is
$V$-microlocally admissible in direction $\xi$ for all $\xi\in\mathcal{O}\cap
S^{d-1}$.

######  Remark 2.5.

We note that under the prevailing assumption that $\mathcal{O}$ is a single
orbit with compact stabilizers, microlocal admissibility in direction
$\xi\in\mathcal{O}$ already entails global microlocal admissibility. Under
further additional assumptions such as the cone approximation property,
condition (a) entails condition (b); see [5, Lemma 6.1].

The main purpose of microlocal admissibility is to establish a systematic
relationship between the norm of $h\in H$ and the norms of the frequencies
contained in the support of ${\mathcal{F}}(\pi(y,h)\psi)$. Such norm
estimates, which we list below, will be instrumental in our proof. See [5,
Lemma 2.8] for a proof of the following.

######  Lemma 2.6.

Assume that the closed subgroup $H\leq{\rm GL}\left(\mathbb{R}^{d}\right)$ is
irreducibly admissible, and consequently, the properties listed in Lemma 2.1
hold. Then $0\notin\mathcal{O}$. Furthermore, the following hold:

  1. (a)

Assume that $\varnothing\neq V\Subset\mathcal{O}$. Then there exists a
constant $C_{1}=C_{1}\left(V\right)>0$ such that, for all $h\in H$ and all
$\xi^{\prime}\in V$:

$\left|h^{-T}\xi^{\prime}\right|^{-1}\leq C_{1}\cdot\|h\|.$

  2. (b)

Assume that the dual action fulfills condition 2.4(a), for some
$\varnothing\not=V\Subset\mathcal{O}$, a suitable $\xi$-neighborhood
$W_{0}\subset S^{d-1}$ and some $R_{0}>0$. Then there exists $C_{2}>0$ such
that

$\|h\|\leq C_{2}\cdot\left|h^{-T}\xi^{\prime}\right|^{-\frac{1}{\alpha_{1}}}$

holds for all $h\in K_{o}(W_{0},V,R_{0})$ and $\xi^{\prime}\in V$, with
$\alpha_{1}$ as in Definition 2.4. Consequently, we have:

${\rm sup}_{h\in K_{o}(W_{0},V,R_{0})}\left\|h\right\|<\infty.$

  3. (c)

With $\alpha_{1}$ as in Definition 2.4, for all $h\in
K_{o}\left(W_{0},V,R_{0}\right)$

$\left|\det\left(h\right)\right|^{-1}\leq
C_{3}\cdot\left\|h\right\|^{-d\alpha_{1}}$ (8)

and

$1+\left\|h^{-1}\right\|\leq C_{4}\cdot\left\|h\right\|^{-\alpha_{1}},$ (9)

hold, for constants $C_{4}=C_{4}\left(W_{0},V,R_{0}\right)>0$ and
$C_{3}=C_{3}\left(W_{0},V,R_{0}\right)>0$.

##  3\. Criteria for $N$-regular directed points: Bandlimited wavelets

Let $\mathbb{N}_{0}=\mathbb{N}\cup\\{0\\}$. In the following, we use the
Schwartz norms

$\left|\psi\right|_{N}:=\max_{\alpha\in\mathbb{N}_{0}^{d},\,\left|\alpha\right|\leq
N}\sup_{z\in\mathbb{R}^{d}}(1+|z|)^{N}\left|\partial^{\alpha}\psi(z)\right|.$

Note that these norms are invariant under complex conjugation. Let
$u\in\mathcal{S}^{\prime}(\mathbb{R}^{d})$ denote a tempered distribution. We
say $u$ is of order $N(u)$ if for all $f\in\mathcal{S}(\mathbb{R}^{d})$, we
have

$|\langle u|f\rangle|\leq C|f|_{N(u)}~{}.$

It is well-known that every tempered distribution $u$ is of finite order. In
the next lemma, we collect various known decay behaviors which will be used in
our proofs later on. We refer to [5, Lemma 3.1].

######  Lemma 3.1.

Let $\psi,\varphi\in\mathcal{S}(\mathbb{R}^{d})$ and
$u\in\mathcal{S}^{\prime}(\mathbb{R}^{d})$.

  1. (a)

For all $N\in\mathbb{N}_{0}$ and $(x,h)\in G$, the following inequality holds:

$\left|\pi(x,h)\psi\right|_{N}\leq
C_{N}|\psi|_{N}\cdot\left|\det(h)\right|^{-1/2}(1+\|h^{-1}\|)^{N}\cdot\max\left\\{1,\|h\|^{N}\right\\}\cdot(1+|x|)^{N},$

with a constant $C_{N}$ independent of $\psi,x,h$.

  2. (b)

Let $N(u)$ denote the order of $u$. Then for all $\left(x,h\right)\in G$, the
following inequality holds:

$|W_{\psi}u(x,h)|\leq
C\cdot\left|\det(h)\right|^{-1/2}(1+\|h^{-1}\|)^{N(u)}\cdot\max\left\\{1,\|h\|^{N(u)}\right\\}\cdot(1+|x|)^{N(u)},$

where $C>0$ is a constant depending on $\psi$, $u$ and $N(u)$ but not on
$x,h$.

  3. (c)

For all $N\in\mathbb{N}$ and $(x,h)\in G$, we have

$|W_{\psi}\varphi(x,h)|\leq
C_{N}|\varphi|_{d+N+1}|\psi|_{N}\cdot\left|\det(h)\right|^{-1/2}(1+\|h^{-1}\|)^{N}\cdot\max\left\\{1,\|h\|^{N}\right\\}\cdot(1+|x|)^{-N}$

with $C_{N}$ independent of $\varphi,\psi,h,x$.

  4. (d)

Assume that the dual action is $V$-microlocally admissible at $\xi$ for some
$\varnothing\neq V\Subset\mathcal{O}$ and that
$\psi\in\mathcal{S}\left(\mathbb{R}^{d}\right)$ with ${\rm
supp}(\widehat{\psi})\subset V$. Choose $R_{0}>0$ and a $\xi$-neighborhood
$W_{0}\subset S^{d-1}$ as in Definition 2.4. Then

$|W_{\psi}\varphi(x,h)|\leq
C_{M,N,\psi,W_{0},V,R_{0}}\cdot|\varphi|_{M+N}\cdot\left|\det(h)\right|^{-1/2}\cdot(1+|x|)^{-N}\|h\|^{M}$

holds for all $x\in\mathbb{R}^{d}$, $h\in K_{o}\left(W_{0},V,R_{0}\right)$ and
$M,N\in\mathbb{N}$, where the constant $C_{M,N,\psi,W_{0},V,R_{0}}$ is
independent of $x,h$ and $\varphi$.

We next address how wavelet coefficient decay and regular directed points are
affected by certain multiplication operators. We refer to [5, Lemma 3.2] for
the proof of (i) and [5, Lemma 3.4] for the proof of (ii).

######  Lemma 3.2.

Let $u\in\mathcal{S}^{\prime}(\mathbb{R}^{d})$ and
$(x,\xi)\in\mathbb{R}^{d}\times S^{d-1}$. Let $\psi\in
C_{c}^{\infty}(\mathbb{R}^{d})$ be identically one in some neighborhood of
$x$. Then we have:

  * (i)

$(x,\xi)$ is an $N$-regular directed point of $u$ iff it is an $N$-regular
directed point of $\psi u$.

  * (ii)

Let $\varphi\in\mathcal{S}(\mathbb{R}^{d})$ be such that ${\rm
supp}(\widehat{\varphi})\subset V$ for some $\varnothing\neq
V\Subset\mathcal{O}$. Moreover, assume that the dual action is
$V$-microlocally admissible in direction $\xi$. Then for all $N\in\mathbb{N}$,
there exist constants $C_{N}>0$ and $\epsilon>0$, such that the estimate

$\left|W_{\varphi}u(y,h)-\left(W_{\varphi}(\psi u)\right)(y,h)\right|\leq
C_{N}\|h\|^{N}$

holds for all $y\in B_{\epsilon}(x)$ and all $h\in K_{o}(W_{0},V,R_{0})$, as
soon as $R_{0}>0$ and the $\xi$-neighborhood $W_{0}\subset S^{d-1}$ satisfy
part (a) of the definition of $V$-microlocal admissibility (Definition 2.4).

######  Remark 3.3.

We can now formulate wavelet criteria for $N$-regular directed points in the
case of bandlimited wavelets. Theorem 3.4 is an adaptation of [5, Theorem 3.5]
to the case of finite smoothness. There are two noteworthy gaps between the
necessary part (a) and the sufficient part (b): One concerns the set of
matrices $h\in H$ for which the norm estimate holds. For the necessary
condition the pertinent set is $K_{i}(W,V,R)$ (which may well be empty),
whereas the sufficient condition requires a decay estimate for the typically
larger set $K_{o}(W,V,R)$. This gap has already been observed for the criteria
in [5]. There exist criteria, the so-called (weak) cone approximation
condition(s), on the dual action that allow to determine when this gap can be
closed. These are fulfilled, in particular, for the shearlet dilation group
and its relatives. In Section 5, we prove Theorem 5.4, which allows to derive
a decay statement for sets of the type $K_{o}(W,V,R)$ from decay assumptions
for sets of the type $K_{i}(W^{\prime},V,R^{\prime})$. As a result, we are
able to close this gap in Corollary 5.6 without assuming the cone
approximation conditions. Instead, we required that our analyzing wavelet has
vanishing moments of sufficiently large order.

The second gap concerns the exponents in the decay estimates for wavelet
coefficients: For $N$-regular directed points $(x,\xi)$, the necessary
criterion notes a decay of order $N-\delta_{1}$, whereas the sufficient
criterion requires order $\alpha_{1}N+\delta_{2}$, with
$\alpha_{1},\delta_{1},\delta_{2}>0$. To our current knowledge, this gap can
not be closed (although possibly reduced); see the rather similar results for
shearlets obtained in [11]. We refer to subsection 6.1 for a more detailed
discussion.

######  Theorem 3.4.

Assume that the dual action is $V$-microlocally admissible in direction $\xi$,
with parameters $\alpha_{1},\alpha_{2}$ as in the definition. Let
$u\in\mathcal{S}^{\prime}(\mathbb{R}^{d})$ and
$(x,\xi)\in\mathbb{R}^{d}\times(S^{d-1}\cap\mathcal{O})$.

  1. (a)

If $(x,\xi)$ is an $N$-regular directed point of $u$, then there exists a
neighborhood $U$ of $x$, some $R>0$ and a $\xi$-neighborhood $W\subset
S^{d-1}\cap\mathcal{O}$ such that for _all_ admissible
$\psi\in\mathcal{S}(\mathbb{R}^{d})$ with ${\rm supp}(\widehat{\psi})\subset
V$, the following estimate holds:

$\exists\,C>0\,\forall\,y\in U\,\forall\,h\in
K_{i}(W,V,R)~{}:~{}|W_{\psi}u(y,h)|\leq C\cdot\|h\|^{N-\alpha_{1}d/2}\\!\\!.$

For each such $\psi$, we even have

$\qquad\qquad\exists\,C>0\,\forall\,y\in U\,\forall\,h\in
K_{i}(W,\,\widehat{\psi}^{-1}\left(\mathbb{C}\setminus\left\\{0\right\\}\right),\,R)~{}:~{}|W_{\psi}u(y,h)|\leq
C\cdot\left\|h\right\|^{N-\alpha_{1}d/2}\\!\\!.$

  2. (b)

Let $\psi\in\mathcal{S}(\mathbb{R}^{d})$ be admissible with ${\rm
supp}(\widehat{\psi})\subset V$. Assume that there exist a neighborhood $U$ of
$x$, a number $R>0$ and a $\xi$-neighborhood $W\subset S^{d-1}\cap\mathcal{O}$
such that

$\exists\,C>0\,\forall\,y\in U\,\forall\,h\in
K_{o}(W,V,R)~{}:~{}|W_{\psi}u(y,h)|\leq
C\cdot\left\|h\right\|^{\alpha_{1}N+\frac{3}{2}\alpha_{1}d+\alpha_{2}}$

Then $(x,\xi)$ is an $N$-regular directed point of $u$.

###### Proof.

Part (a) is provided by a verbatim reproduction of the proof of [5, Theorem
3.5(a)]. In a similar way, part (b) is obtained by a straightforward
adaptation of the argument for [5, Theorem 3.5(b)]. Since this argument is
more involved, we provide a somewhat more detailed sketch.

For the proof of part (b), observe that we may assume $u$ to be compactly
supported by Lemma 3.2. Let $\varepsilon_{1}>0$ be such that
$B_{\varepsilon_{1}}(x)\subseteq U$, and assume that $\varepsilon_{1}$
satisfies the statement in Lemma 3.2(ii). By assumption,

$|W_{\psi}u(y,h)|\leq C\|h\|^{\alpha_{1}(N+\frac{3}{2}d)+\alpha_{2}}$ (10)

holds for all $y\in B_{\varepsilon_{1}}(x)$ and $h\in K_{o}(W,V,R)$. With
$R_{0}>0$ and $W_{0}\subset S^{d-1}\cap\mathcal{O}$ as in Definition 2.4, we
may assume $W\subset W_{0}\subset\mathcal{O}$ and $R>R_{0}$. In particular,
this implies $C\left(W,R\right)\subset\mathcal{O}$, thanks to Lemma 2.1(a).

Let

$\eta:=\int_{\mathbb{R}^{d}}\int_{K_{o}(W,V,R)}W_{\psi}u(y,h)\cdot\pi(y,h)\psi\,\frac{{\rm
d}h}{|{\rm det}(h)|}\,{\rm d}y,$ (11)

where the integral defining $\eta$ is to be understood in the weak sense,
i.e., given $\varphi\in\mathcal{S}(\mathbb{R}^{d})$, we let

$\displaystyle\langle\eta\mid\varphi\rangle$
$\displaystyle=\int_{\mathbb{R}^{d}}\int_{\mathbb{R}^{d}}\int_{K_{o}(W,V,R)}W_{\psi}u(y,h)[\pi(y,h)\psi](z)\overline{\varphi(z)}\frac{{\rm
d}h}{|{\rm det}(h)|}\,{\rm d}y{\rm d}z$
$\displaystyle=\int_{\mathbb{R}^{d}}\int_{K_{o}\left(W,V,R\right)}W_{\psi}u\left(y,h\right)\cdot\overline{\left(W_{\psi}\varphi\right)\left(y,h\right)}\,\frac{{\rm
d}h}{|{\rm det}(h)|}\,{\rm d}y.$ (12)

Using parts (b) and (d) of Lemma 3.1 together with Lemma 2.6, it can be shown
that $\eta$ is a well-defined element of
$\mathcal{S}^{\prime}(\mathbb{R}^{d})$, and the integral in equation (12)
converges absolutely for all
$\varphi\in\mathcal{S}\left(\mathbb{R}^{d}\right)$. Next, we show that
$u=\eta$. Indeed, for the compactly supported tempered distribution $u$, its
Fourier transform $\widehat{u}$ can be represented as a continuous function on
${\mathbb{R}}^{d}$, which has an explicit formula on ${\mathcal{O}}$ given as
follows:

$\widehat{u}(\xi^{\prime})=\int_{{\mathbb{R}}^{d}}\int_{H}W_{\psi}u(y,h){\mathcal{F}}[\pi(y,h)\psi](\xi^{\prime})\frac{{\rm
d}h}{|\det h|}{\rm d}y,$

for all $\xi^{\prime}\in{\mathcal{O}}$. Since ${\mathcal{O}}$ is a co-null set
in $\mathbb{R}^{d}$, for every $\varphi\in{\mathcal{S}}({\mathbb{R}}^{d})$ we
have

$\displaystyle\langle u\mid\varphi\rangle$ $\displaystyle=$
$\displaystyle\langle\widehat{u}\mid\mathcal{F}^{-1}\varphi\rangle=\int_{{\mathcal{O}}}\widehat{u}(\xi^{\prime})\overline{\mathcal{F}^{-1}\varphi(\xi^{\prime})}{\rm
d}\xi^{\prime}$ $\displaystyle=$
$\displaystyle\int_{{\mathcal{O}}}\int_{{\mathbb{R}}^{d}}\int_{H}W_{\psi}u(y,h){\mathcal{F}}[\pi(y,h)\psi](\xi^{\prime})\overline{\mathcal{F}^{-1}\varphi(\xi^{\prime})}\frac{{\rm
d}h}{|\det h|}{\rm d}y{\rm d}\xi^{\prime}$ $\displaystyle=$
$\displaystyle\int_{{\mathcal{O}}}\int_{{\mathbb{R}}^{d}}\int_{K_{o}(W,V,R)}W_{\psi}u(y,h){\mathcal{F}}[\pi(y,h)\psi](\xi^{\prime})\overline{\mathcal{F}^{-1}\varphi(\xi^{\prime})}\frac{{\rm
d}h}{|\det h|}{\rm d}y{\rm d}\xi^{\prime}$ $\displaystyle=$
$\displaystyle\int_{{\mathbb{R}}^{d}}\int_{K_{o}(W,V,R)}W_{\psi}u(y,h)\left(\int_{{{\mathbb{R}}^{d}}}{\mathcal{F}}[\pi(y,h)\psi](\xi^{\prime})\overline{\mathcal{F}^{-1}\varphi(\xi^{\prime})}{\rm
d}\xi^{\prime}\right)\frac{{\rm d}h}{|\det h|}{\rm d}y$ $\displaystyle=$
$\displaystyle\int_{{\mathbb{R}}^{d}}\int_{K_{o}(W,V,R)}W_{\psi}u(y,h)\left(\int_{{{\mathbb{R}}^{d}}}[\pi(y,h)\psi](z)\overline{{\varphi(z)}}{\rm
d}z\right)\frac{{\rm d}h}{|\det h|}{\rm d}y$ $\displaystyle=$
$\displaystyle\int_{{{\mathbb{R}}^{d}}}\int_{{\mathbb{R}}^{d}}\int_{K_{o}(W,V,R)}W_{\psi}u(y,h)[\pi(y,h)\psi](z)\overline{\varphi(z)}\frac{{\rm
d}h}{|\det h|}{\rm d}y{\rm d}z$ $\displaystyle=$
$\displaystyle\langle\eta\mid\varphi\rangle.$

To obtain the desired result, we will show that in a suitable neighborhood of
$x$, the distribution $u$ is given by a $C^{N}$-function. Consider the
auxiliary function $\kappa$ obtained by pointwise evaluation of the integral
defining $\eta$ weakly in equation (11), i.e.

$\kappa:B_{\varepsilon_{1}/2}(x)\to\mathbb{C},\,z\mapsto\int_{\mathbb{R}^{d}}\int_{K_{o}(W,V,R)}W_{\psi}u(y,h)[\pi(y,h)\psi](z)\,\frac{{\rm
d}h}{\left|\det\left(h\right)\right|}\,{\rm d}y.$

Note that the convergence that is crucial to equate pointwise and weak
definitions is guaranteed by the assumptions on wavelet coefficient decay. We
then write $\kappa=\kappa_{1}+\kappa_{2}$, where

$\kappa_{1}(z):=\int_{B_{\varepsilon_{1}}(x)}\int_{K_{o}(W,V,R)}W_{\psi}u(y,h)[\pi(y,h)\psi](z)\,\frac{{\rm
d}h}{\left|\det\left(h\right)\right|}\,{\rm d}y.$

Note that $\kappa_{2}$ is obtained by integration over the complement of the
ball $B_{\varepsilon_{1}}(x)$, where we make no specific assumptions on the
wavelet coefficient decay. Using the proof of Theorem 3.5 of [5], we have that
$\kappa_{2}$ is $C^{\infty}$ inside $B_{\varepsilon_{1}/2}(x)$, and
$\kappa_{1}$ is $N$-fold differentiable as soon as the decay order $M$ of the
wavelet coefficients fulfills $M\geq\alpha_{1}(N+\frac{3}{2}d)+\alpha_{2}$.
Hence $\kappa$ is $C^{N}$, and for every
$\varphi\in\mathcal{S}(\mathbb{R}^{d})$ with ${\rm supp}(\varphi)\subset
B_{\epsilon_{1}/2}(x)$, we clearly have

$\langle\eta|\varphi\rangle=\int_{\mathbb{R}^{d}}\kappa(z)\overline{\varphi(z)}dz~{}.$

∎

##  4\. Towards an extension to compactly supported wavelets

As demonstrated in [11] for shearlets in two dimensions, it is possible to
sometimes replace the assumption that $\widehat{\psi}$ is compactly supported
by suitable finite vanishing moment conditions, and still obtain near
characterizations of $N$-regular directed points. This allows to use wavelets
that are compactly supported in space rather than in frequency. The chief
purpose of this section is to prepare the ground for the desired
generalization, by highlighting and clarifying various technical issues that
arise in the process.

The idea underlying the extension is reasonably simple: Assume that we have an
estimate

$|W_{\psi}u(y,h)|\leq C\|h\|^{N}~{},$ (13)

valid on a set $B_{\epsilon}(x)\times K_{i/o}(W,V,R)$, with a wavelet $\psi$
satisfying ${\rm supp}(\widehat{\psi})\subset V\Subset\mathcal{O}$. Assume
that $\eta\in\mathcal{S}(\mathbb{R}^{d})$ is a second admissible wavelet.
Under suitable convergence assumptions, it is possible to plug the inversion
formula

$\eta=\frac{1}{C_{\psi}}\int_{G}{W}_{\psi}\eta(x,h)\pi(x,h)\psi\text{\,d}(x,h)$

into the definition of the wavelet transform ${W}_{\eta}u$, and obtain that
the wavelet transforms ${W}_{\eta}u$ and ${W}_{\psi}u$ are related by a
convolution product:

${W}_{\eta}u=\frac{1}{C_{\psi}}{W}_{\psi}u\ast{W}_{\eta}\psi~{}.$

Subsection 4.2 will be devoted to formulating the explicit criteria needed for
the above identity to hold. Hence in order to transfer decay estimates from
${W}_{\psi}u$ to ${W}_{\eta}u$, we need to understand which conditions on the
cross-kernel ${W}_{\eta}\psi$ allow to transfer the wavelet coefficient decay
from ${W}_{\psi}u$ to ${W}_{\eta}u$, possibly on a slightly smaller set
$K^{\prime}$. Note also that the same reasoning, with ${W}_{\psi}\eta$
replacing ${W}_{\eta}\psi$, would also allow to transfer decay estimates for
${W}_{\eta}u$ to ${W}_{\psi}u$. This allows us to extend both necessary and
sufficient wavelet coefficient decay conditions for $N$-regular directed
points.

Our proof strategy also requires the use of methods to estimate and quantify
the decay of the cross-kernel $W_{\eta}\psi$; here we will take advantage of
previous work developed in [7, 8, 10]. However, one further aspect that needs
to be addressed in connection with vanishing moment conditions is the fact
that these conditions can be fulfilled by real-valued wavelets. Due to the
symmetry condition that real-valuedness of $\psi$ imposes on $\widehat{\psi}$,
these wavelets are generally unable to distinguish between directions
$\xi,-\xi$, i.e., our wavelet-based near characterizations will describe
unsigned $N$-regular directed points, in the sense of Definition 4.1 below. We
emphasize that this restriction is not due to a general shortcoming of our
approach, and that it is also present (rather more implicitly, but not less
stringently) in the precursor papers [15, 11]. We refer to Subsection 6.1 for
a more detailed explanation. But now we start the discussion of unsigned
regular directed point.

###  4.1. Unsigned regular directed points and real-valued distributions

As already mentioned, attempting to derive decay statements from vanishing
moment conditions opens the door to the use of real-valued wavelets. For these
wavelets $\psi$ however, one has
$\widehat{\psi}(-\xi)=\widehat{\psi}^{\ast}(\xi):=\overline{\widehat{\psi}(\xi)}$.
Hence by construction, such a wavelet will not be able to distinguish the
decay behaviour inside a cone $C$ containing $\xi$ and the cone $-C$,
containing $-\xi$. As a consequence, we will restrict attention to real-valued
wavelets in the following, and will pay the price that the behaviour in
directions $\xi$ and $-\xi$ is indistinguishable. The following definition
adapts the notion of $N$-regular directed points to this setting. While we
expect that these notions have already been studied in some detail elsewhere,
we have not been able to locate a convenient source.

######  Definition 4.1.

For $(x,\xi),(x^{\prime},\xi^{\prime})$ we write
$(x,\xi)\sim(x^{\prime},\xi^{\prime})$ iff $x=x^{\prime},\xi=\pm\xi^{\prime}$,
and use $\mathbb{R}^{d}\times S^{d-1}/\pm$ to denote the set of equivalence
classes.

Given $N\in\mathbb{N}$, $u\in{\mathcal{S}}^{\prime}(\mathbb{R}^{d})$ and
$(x,\pm\xi)\in\mathbb{R}^{d}\times S^{d-1}/\pm$, we call $(x,\pm\xi)$ an
unsigned $N$-regular directed point of $u$ if both $(x,\xi)$ and $(x,-\xi)$
are $N$-regular directed points of $u$. We call $(x,\pm\xi)$ unsigned regular
directed point if $(x,\pm\xi)$ is an unsigned $N$-regular directed point for
all $N$, or equivalently, if $(x,\xi)$ and $(x,-\xi)$ are regular directed
points of $u$.

It turns out that this notion is closely related to the $N$-regular directed
points of the real and imaginary parts of $u$. We first recall the necessary
terminology: Given $u\in{\mathcal{S}}^{\prime}(\mathbb{R}^{d})$, we define
$\overline{u}\in{\mathcal{S}}^{\prime}(\mathbb{R}^{d})$ as
$\langle\overline{u}|f\rangle=\overline{\langle u|\overline{f}\rangle}$;
recall that we use ${L}^{2}$-notation here. The real and imaginary parts of
$u$ are then defined in the usual manner as ${\rm
Re}(u)=\frac{1}{2}(u+\overline{u})$ and ${\rm
Im}(u)=\frac{1}{2i}(u-\overline{u})$.

It follows that $u={\rm Re}(u)+i{\rm Im}(u)$, and for real-valued functions
$f$, one has

$\langle{\rm Re}(u)|f\rangle={\rm Re}\langle u|f\rangle~{},~{}\langle{\rm
Im}(u)|f\rangle={\rm Im}\langle u|f\rangle~{}.$

We call $u$ real-valued if $u={\rm Re}(u)$. It is easy to see that a tempered
distribution is real-valued iff it maps real-valued Schwartz functions to real
values.

In order to express the just-described operations on tempered distributions in
Fourier domain, we need to extend the involution $f^{*}(x):=\overline{f(-x)}$
from functions to tempered distributions, by letting

$\langle u^{*}|f\rangle=\overline{\langle u|f^{*}\rangle}~{}.$

Part (b) of the following lemma makes clear that for real-valued
distributions, the distinction between signed and unsigned $N$-regular
directed points is moot, and part (c) relates unsigned $N$-regular directed
points of $u$ to $N$-regular directed points of ${\rm Re}(u)$ and ${\rm
Im}(u)$.

######  Lemma 4.2.

Let $u\in\mathcal{S}^{\prime}(\mathbb{R}^{d})$.

  1. (a)

$\widehat{\overline{u}}=(\widehat{u})^{\ast}$.

  2. (b)

If $u$ is real-valued, then its Fourier transform $\widehat{u}$ fulfills
$\widehat{u}=(\widehat{u})^{*}$. As a consequence, $(x,-\xi)$ is a regular
directed point of $u$ iff $(x,\xi)$ is.

  3. (c)

$(x,\pm\xi)$ is an unsigned $N$-regular directed point of $u$ iff $(x,\xi)$ is
an $N$-regular directed point of ${\rm Re}(u)$ and ${\rm Im}(u)$, or
equivalently, iff $(x,\pm\xi)$ is an unsigned $N$-regular directed point of
${\rm Re}(u)$ and ${\rm Im}(u)$.

###### Proof.

Part (a) is straightforward. Part (b) follows from the fact that the cut-out
function $\varphi$ in the definition of $N$-regular directed points can be
chosen real-valued, and then (a) applies to the product $\varphi u$, and
yields the desired statement.

For part (c), assume that $(x,\pm\xi)$ is an unsigned $N$-regular directed
point of $u$, i.e., we have decay of $\widehat{\varphi u}$ of order $N$ inside
$C\cup-C$, for a suitable cone $C$ in direction $\xi$. Again we choose the
cut-out function $\varphi$ to be real-valued, and then
$\varphi\overline{u}=\overline{\varphi u}$. So ${\rm Re}(\varphi
u)=\varphi({\rm Re}(u))$, and the same holds for the imaginary part as well.
Using this fact, together with part (a), we get

$\left(\varphi{\rm Re}(u)\right)^{\wedge}(\xi^{\prime})=\left({\rm Re}(\varphi
u)\right)^{\wedge}(\xi^{\prime})=\frac{1}{2}\left(\widehat{\varphi
u}+\widehat{\overline{\varphi
u}}\right)(\xi^{\prime})=\frac{1}{2}\left(\widehat{\varphi
u}(\xi^{\prime})+\overline{\widehat{\varphi u}(-\xi^{\prime})}\right).$

So, the same decay behaviour for $\left(\varphi{\rm Re}(u)\right)^{\wedge}$,
and similarly for $\left(\varphi{\rm Im}(u)\right)^{\wedge}$, holds in
$C\cup-C$. This establishes that $(x,\xi)$ and $(x,-\xi)$ are $N$-regular
directed points of ${\rm Re}(u)$ and ${\rm Im}(u)$; or equivalently, that
$(x,\pm\xi)$ is an unsigned $N$-regular directed point of ${\rm Re}(u)$ and
${\rm Im}(u)$.

The converse direction follows directly from part (b) and $u={\rm Re}(u)+i{\rm
Im}(u)$. ∎

The next somewhat technical notion formulates a fairly subtle structural
assumption on the dilation group $H$ that initially seems a bit opaque. We
will repeatedly comment on the role of this assumption to make its impact on
our arguments clear.

######  Definition 4.3.

We call $H_{0}<H$ a half-space adapted subgroup if it is an open subgroup with
the property that, for any $\xi_{0}\in\mathcal{O}$, one has
$\mathcal{O}=(H_{0}^{T}\xi_{0})\cup-(H_{0}^{T}\xi_{0})$, where the union is
disjoint.

As a pertinent class of examples with this property we mention the generalized
shearlet dilation groups featured in Section 6. On the other hand, it is clear
that not all admissible dilation groups have a half-space adapted subgroup.
Note that by definition $H_{0}$ is necessarily a proper open subgroup of index
2. The standard higher-dimensional wavelet dilation groups
$H=\mathbb{R}^{+}SO(\mathbb{R}^{d})$ are connected, and therefore do not
possess a half-space adapted subgroup.

The chief purpose of introducing the notion of half-space adapted subgroups is
that in certain contexts, the integration domain $H$ may be swapped for the
smaller subset $H_{0}$, and this will be instrumental in guaranteeing the
necessary decay condition. The following lemma spells this phenomenon out for
the inversion formula associated to real-valued wavelets.

######  Lemma 4.4.

Let $H_{0}<H$ denote a half-space adapted subgroup, and let $\psi$ denote a
real-valued admissible wavelet for $H$. Then, for all
$f\in{L}^{2}(\mathbb{R}^{d})$, one has

$f=\frac{2}{C_{\psi}}\int_{G_{0}}{W}_{\psi}f(x,h)\pi(x,h)\psi\text{\,d}(x,h)~{},$

where $G_{0}=\mathbb{R}^{d}\rtimes H_{0}$.

###### Proof.

Writing $\mathcal{O}^{+}=H_{0}^{T}\xi_{0}$, one has
$\mathcal{O}=\mathcal{O}^{+}\cup-\mathcal{O}^{+}$. The admissibility
assumption on $\psi$ entails that

$C_{\psi}=\int_{H}|\widehat{\psi}(h^{T}\xi_{0})|^{2}\text{\,d}h<\infty~{},$

and accordingly, that $\|{W}_{\psi}f\|_{{L}^{2}(G)}^{2}=C_{\psi}\|f\|^{2}$, as
well as

$f=\frac{1}{C_{\psi}}\int_{G}{W}_{\psi}f(x,h)\pi(x,h)\psi\text{\,d}(x,h)~{}.$

By assumptions on $\psi$ and $H_{0}$, we have

$\int_{H_{0}}|\widehat{\psi}(h^{T}\xi_{0})|^{2}\text{\,d}h=\int_{H_{0}}|\widehat{\psi}(-h^{T}\xi_{0})|^{2}\text{\,d}h=\frac{C_{\psi}}{2}~{}.$

Since $\mathcal{O}^{+}\cup(-\mathcal{O}^{+})=\mathcal{O}$ is conull, the
admissibility condition for non-irreducible quasi-regular representations (see
e.g. Theorem 1 of [9]) allows to conclude for the restriction of ${W}_{\psi}f$
to $G_{0}$ that
$\|{W}_{\psi}f\|_{{L}^{2}(G_{0})}^{2}=\frac{C_{\psi}}{2}\|f\|^{2}$, and
accordingly

$f=\frac{2}{C_{\psi}}\int_{G_{0}}{W}_{\psi}f(x,h)\pi(x,h)\psi\text{\,d}(x,h)~{}.$

∎

###  4.2. Vanishing moments and wavelet coefficient decay

Our methods and techniques build on previous work on wavelet coefficient decay
estimates from vanishing moment conditions. In this subsection, we shall
roughly sketch the pertinent results from [7, 8, 10].

The first question that arises concerns the precise nature of the vanishing
moment criteria. In higher dimensions, these depend on the dilation group, or
more precisely, on the open dual orbit $\mathcal{O}$:

######  Definition 4.5.

Let $r\in\mathbb{N}$ be given. We say $\psi\in\mathcal{S}(\mathbb{R}^{d})$ has
vanishing moments in $\mathcal{O}^{c}$ of order $r$ if all derivatives
$\partial^{\alpha}\widehat{\psi}$ of degree $|\alpha|<r$ are vanishing on
$\mathcal{O}^{c}$.

Clearly, any bandlimited wavelet, i.e., any Schwartz function $\psi$ with
$\widehat{\psi}\in C_{c}^{\infty}(\mathcal{O})$ has vanishing moments of all
orders. Note that the vanishing moment conditions are equivalent to

$\forall|\alpha|<r,\forall\xi\in\mathcal{O}^{c}~{}:~{}\int_{\mathbb{R}^{d}}x^{\alpha}\psi(x)e^{-2\pi
i\langle\xi,x\rangle}\text{\,d}x=0~{}.$

We next define an auxiliary function $A:\mathcal{O}\to\mathbb{R}^{+}$ that is
used in the decay estimates. Given any point $\xi\in\mathcal{O}$, let ${\rm
dist}(\xi,\mathcal{O}^{c})$ denote the distance of $\xi$ to the set
$\mathcal{O}^{c}$, and define

$A(\xi):=\min\left(\frac{{\rm
dist}(\xi,\mathcal{O}^{c})}{1+\sqrt{|\xi|^{2}-{\rm
dist}(\xi,\mathcal{O}^{c})^{2}}},\frac{1}{1+|\xi|}\right)~{}.$ (14)

By definition, $A$ is a continuous function with $A(\cdot)\leq 1$. If
$\eta\in\mathcal{O}^{c}$ denotes an element of minimal distance to $\xi$, the
fact that $\mathbb{R}^{+}\cdot\eta\subset\mathcal{O}^{c}$ then entails that
$\eta$ and $\xi-\eta$ are orthogonal with respect to the standard scalar
product on $\mathbb{R}^{d}$, and we obtain the more transparent expression

$A(\xi)=\min_{\eta\in\mathcal{O}^{c}}\left(\frac{|\xi-\eta|}{1+|\eta|},\frac{1}{1+|\xi|}\right)~{}.$
(15)

We define a further family of auxiliary functions
$\Phi_{\ell}:H\to\mathbb{R}^{+}\cup\\{\infty\\}$, for $\ell\in\mathbb{N}$, via

$\Phi_{\ell}(h)=\int_{\mathbb{R}^{d}}A(\xi)^{\ell}A(h^{T}\xi)^{\ell}\text{\,d}\xi~{}.$
(16)

The motivation for the definitions of $A$ and $\Phi_{\ell}$ is best explained
as follows: Two functions $\eta,\psi\in\mathcal{S}(\mathbb{R})$ are admissible
wavelets if and only if $\widehat{\eta}(0)=\widehat{\psi}(0)=0$. The usual
arguments relating smoothness of a function and the decay of its Fourier
transform (and vice versa), together with the convolution theorem, allow us to
estimate

$|{W}_{\psi}\eta(x,h)|=|\eta\ast(\pi(0,h)\psi^{*})(x)|\leq
C(1+|x|)^{-N}|h|^{1/2}\sum_{j=0}^{N}\left\|\partial^{j}\left(\widehat{\eta}\overline{\widehat{\psi}(h\cdot)}\right)\right\|_{1}~{}.$
(17)

Here the sum

$\sum_{j=0}^{N}\left\|\partial^{j}\left(\widehat{\eta}\overline{\widehat{\psi}(h\cdot)}\right)\right\|_{1}$

can be understood as a term measuring the overlap between $\widehat{\eta}$ and
the dilate of $\widehat{\psi}$, and it goes to zero as $h\to 0,h\to\infty$,
with a speed depending on the number of vanishing moments of $\eta$ and
$\psi$. The analog of this overlap term in the higher-dimensional setting is
essentially provided by the auxiliary function $\Phi_{\ell}$.

The following result guarantees that the set of compactly supported Schwartz
functions with vanishing moments of a given order is nonempty. It follows from
Lemma 4.1 from [7].

######  Lemma 4.6.

For any $r>0$ there exists a nonzero compactly supported
$\psi\in\mathcal{S}(\mathbb{R}^{d})$ having vanishing moments in
$\mathcal{O}^{c}$ of order $r$.

We next quote the following general estimate [8, Lemma 3.7], which can be
viewed as a generalization of (17):

######  Lemma 4.7.

Let $0<m<r$, and let $\eta,\psi\in\mathcal{S}(\mathbb{R}^{d})$ denote
functions with vanishing moments of order $r$ in $\mathcal{O}^{c}$. Then there
exists $C>0$ independent of $x,h,\eta,\psi$ such that

$\left|{W}_{\psi}\eta(x,h)\right|\leq
C|\widehat{\eta}|_{r}|\widehat{\psi}|_{r}(1+|x|)^{-m}|\det(h)|^{1/2}(1+\|h\|)^{m}\Phi_{r-m}(h)~{}.$

Next, we use the above lemma to obtain estimates on $W_{\psi}\eta$. Recall
that ${\mathcal{O}}$ is an open orbit in $\widehat{{\mathbb{R}}^{d}}$, with
the property that the stabilizer groups associated with the $H$-action on
${\mathcal{O}}$ are compact. This property allows us to transfer the Lebesgue
measure on $\widehat{{\mathbb{R}}^{d}}$ to the Haar measure on $H$, i.e. for
fixed $\xi_{0}\in\mathcal{O}$ and every $F\in C_{c}({\mathcal{O}})$ we have

$\int_{\mathcal{O}}F(\xi)\text{\,d}\xi=\int_{H}F(h^{T}\xi_{0})\frac{|\det(h)|}{\Delta_{H}(h)}\text{\,d}h.$

Fix an element $\xi_{0}\in\mathcal{O}$, and define $A_{H}:H\rightarrow[0,1]$
to be $A_{H}(h)=A(h^{T}\xi_{0})$. Our computations in the following
proposition are inspired by techniques developed in [10].

######  Proposition 4.8.

Assume that there exists a positive number $\ell_{1}\in{\mathbb{R}}$ and a
constant $C$ so that for every $h\in H$,

$\displaystyle\|h^{\pm 1}\|A_{H}(h)^{\ell_{1}}$ $\displaystyle\leq$
$\displaystyle C.$ (18)

Let $\beta_{1},\beta_{2},\beta_{3}$ be positive numbers with
$\beta_{1}\in\mathbb{N}$, and consider functions
$\psi_{1},\psi_{2}\in{\mathcal{S}}(\mathbb{R}^{d})$ with vanishing moments of
order $r$ in $\mathcal{O}^{c}$. If
$r>d/2+\ell_{1}({\beta_{1}+\beta_{2}}+\beta_{3})+\beta_{1}$ then for every
$(x,h)\in G$,

$\left|{W}_{\psi_{1}}\psi_{2}(x,h)\right|\leq
D(1+|x|)^{-\beta_{1}}(1+\|h\|)^{-\beta_{2}}(1+\|h^{-1}\|)^{-\beta_{3}},$

where the constant $D$ depends on $\psi_{1},\psi_{2}$,
$\beta_{1},\beta_{2},\beta_{3}$, and $r,\ell_{1}$, and does not depend on
$x,h$.

###### Proof.

Without loss of generality, we can assume that $C\geq 1$. By Lemma 4.7, we
have

$\left|{W}_{\psi_{1}}\psi_{2}(x,h)\right|\leq
C_{1}(1+|x|)^{-\beta_{1}}|\det(h)|^{1/2}(1+\|h\|)^{\beta_{1}}\Phi_{r-\beta_{1}}(h).$

Hence it remains to show that the map
$h\mapsto|\det(h)|^{1/2}(1+\|h\|)^{\beta_{1}+\beta_{2}}(1+\|h^{-1}\|)^{\beta_{3}}\Phi_{r-\beta_{1}}(h)$
is bounded. To do so, define
$m(h)=|\det(h)|^{1/2}(1+\|h\|)^{\beta_{1}+\beta_{2}}(1+\|h^{-1}\|)^{\beta_{3}}$,
and observe that $m$ is submultiplicative, as
$\beta_{1},\beta_{2},\beta_{3}>0$. Our goal is then to prove that
$m\Phi_{r-\beta_{1}}$ belongs to $L^{\infty}(H)$.

For $\ell\in\mathbb{N}$, using the submultiplicativity of $m$, we have

$\displaystyle m(h)\Phi_{\ell}(h)$ $\displaystyle=$ $\displaystyle
m(h)\int_{\mathbb{R}^{d}}A(\xi)^{\ell}A(h^{T}\xi)^{\ell}\text{\,d}\xi$
$\displaystyle=$ $\displaystyle
m(h)\int_{H}A(k^{T}\xi_{0})^{\ell}A(h^{T}k^{T}\xi_{0})^{\ell}\frac{|\det(k)|}{\Delta_{H}(k)}\text{\,d}k$
$\displaystyle=$ $\displaystyle
m(h)\int_{H}A_{H}(k)^{\ell}A_{H}(kh)^{\ell}\frac{|\det(k)|}{\Delta_{H}(k)}\text{\,d}k$
$\displaystyle=$ $\displaystyle
m(h)\int_{H}|\det(k)|^{-1}{A_{H}}(k^{-1})^{\ell}A_{H}(k^{-1}h)^{\ell}\text{\,d}k,$
$\displaystyle\leq$
$\displaystyle\int_{H}\left(|\det(k)|^{-1}m(k){A_{H}}(k^{-1})^{\ell}\right)\left(m(k^{-1}h)A_{H}(k^{-1}h)^{\ell}\right)\text{\,d}k$
$\displaystyle=$ $\displaystyle(B_{1}*B_{2})(h),$

where we use the auxiliary functions $B_{1},B_{2}:H\rightarrow[0,\infty)$
defined as $B_{1}(k)=|\det(k)|^{-1}m(k){A_{H}}(k^{-1})^{\ell}$ and
$B_{2}(k)=m(k)A_{H}(k)^{\ell}$. Now, it is only left to show that if
$\ell>\ell_{1}({\beta_{1}+\beta_{2}}+\beta_{3})+\frac{d}{2}$ then
$B_{1},\widetilde{B_{2}}\in{L}^{2}(H)$, where we use the notation
$\widetilde{B_{2}}(k)=B_{2}(k^{-1})$ whenever $k\in H$. Indeed, if that is the
case, we will have
$\|m\Phi_{\ell}\|_{\infty}\leq\|B_{1}\|_{2}\|\widetilde{B_{2}}\|_{2}<\infty$.

Note that $(1+\|h^{\pm 1}\|)A_{H}(h)^{\ell_{1}}=A_{H}(h)^{\ell_{1}}+\|h^{\pm
1}\|A_{H}(h)^{\ell_{1}}\leq 2C$. If
$\ell>d/2+\ell_{1}({\beta_{1}+\beta_{2}}+\beta_{3})$, we have

$\displaystyle\|B_{1}\|_{2}^{2}$ $\displaystyle=$
$\displaystyle\int_{H}|\det(k)|^{-2}m(k)^{2}{A_{H}}(k^{-1})^{2\ell}\text{\,d}k$
$\displaystyle=$
$\displaystyle\int_{H}|\det(k)|^{-2}|\det(k)|(1+\|k\|)^{2\beta_{1}+2\beta_{2}}(1+\|k^{-1}\|)^{2\beta_{3}}{A_{H}}(k^{-1})^{2\ell}\text{\,d}k$
$\displaystyle\leq$
$\displaystyle(2C)^{2(\beta_{1}+\beta_{2}+\beta_{3})}\int_{H}|\det(k)|^{-1}{A_{H}}(k^{-1})^{2\ell-2\ell_{1}({\beta_{1}+\beta_{2}}+\beta_{3})}\text{\,d}k$
$\displaystyle=$
$\displaystyle(2C)^{2(\beta_{1}+\beta_{2}+\beta_{3})}\int_{H}{A_{H}}(k)^{2\ell-2\ell_{1}({\beta_{1}+\beta_{2}}+\beta_{3})}\frac{|\det(k)|}{\Delta_{H}(k)}\text{\,d}k$
$\displaystyle=$
$\displaystyle(2C)^{2(\beta_{1}+\beta_{2}+\beta_{3})}\int_{\mathcal{O}}{A}(\xi)^{2\ell-2\ell_{1}({\beta_{1}+\beta_{2}}+\beta_{3})}\text{\,d}\xi$
$\displaystyle\leq$
$\displaystyle(2C)^{2(\beta_{1}+\beta_{2}+\beta_{3})}\int_{\mathcal{O}}\left(\frac{1}{1+|\xi|}\right)^{2\ell-2\ell_{1}({\beta_{1}+\beta_{2}}+\beta_{3})}\text{\,d}\xi,$

which is a bounded integral provided that
$2\ell-2\ell_{1}({\beta_{1}+\beta_{2}}+\beta_{3})>d$.

Also,

$\displaystyle\|\widetilde{B_{2}}\|_{2}^{2}$ $\displaystyle=$
$\displaystyle\int_{H}m(k^{-1})^{2}A_{H}(k^{-1})^{2\ell}\text{\,d}k$
$\displaystyle=$
$\displaystyle\int_{H}|\det(k)|(1+\|k\|)^{2(\beta_{1}+\beta_{2})}(1+\|k^{-1}\|)^{2\beta_{3}}A_{H}(k)^{2\ell}\frac{1}{\Delta_{H}(k)}\text{\,d}k$
$\displaystyle\leq$
$\displaystyle(2C)^{2(\beta_{1}+\beta_{2}+\beta_{3})}\int_{H}{A_{H}}(k)^{2\ell-2\ell_{1}({\beta_{1}+\beta_{2}}+\beta_{3})}\frac{|\det(k)|}{\Delta_{H}(k)}\text{\,d}k$
$\displaystyle=$
$\displaystyle(2C)^{2(\beta_{1}+\beta_{2}+\beta_{3})}\int_{\mathcal{O}}{A}(\xi)^{2\ell-2\ell_{1}({\beta_{1}+\beta_{2}}+\beta_{3})}\text{\,d}\xi$
$\displaystyle\leq$
$\displaystyle(2C)^{2(\beta_{1}+\beta_{2}+\beta_{3})}\int_{\mathcal{O}}\left(\frac{1}{1+|\xi|}\right)^{2\ell-2\ell_{1}({\beta_{1}+\beta_{2}}+\beta_{3})}\text{\,d}\xi,$

which is a bounded integral provided that
$2\ell-2\ell_{1}({\beta_{1}+\beta_{2}}+\beta_{3})>d$. Finally, letting
$D=C_{1}\|m\Phi_{r-\beta_{1}}\|_{\infty}$ finishes the proof. ∎

The estimates in the previous proposition motivate the introduction of a
further family of auxiliary functions.

######  Lemma 4.9.

For $s,t\in{\mathbb{R}}^{+}$ and $p\in{\mathbb{R}}$, let
$\Theta_{s,t}:H\rightarrow(0,\infty)$ be defined as

$\Theta_{s,t}(h)=(1+\|h\|)^{-s}(1+\|h^{-1}\|)^{-t}.$

If $s\geq\dim H+d+1$ and $t\geq\dim H+d+dp$ then
$|\det(\cdot)|^{-p}\Theta_{s,t}\in{L}^{1}(H)$.

###### Proof.

We first recall the inequalities $|\det(h)|\leq(1+\|h\|)^{d}$ and
$\Delta_{H}(h)\leq((1+\|h\|)(1+\|h^{-1}\|))^{\dim H}$. The first one is known
as Hadamard’s inequality; for the second one we refer to the proof of Lemma
2.8 of [10].

These inequalities allow to obtain the following useful estimate:

$\displaystyle|\det(h)|^{-p}\Theta_{s,t}(h)\frac{\Delta_{H}(h)}{|\det(h)|}$
$\displaystyle=$
$\displaystyle|\det(h^{-1})|^{p+1}\Theta_{s,t}(h)\Delta_{H}(h)$ (19)
$\displaystyle\leq$
$\displaystyle(1+\|h^{-1}\|)^{d(p+1)}(1+\|h\|)^{-s}(1+\|h^{-1}\|)^{-t}{((1+\|h\|)(1+\|h^{-1}\|))^{\dim
H}}$ $\displaystyle=$ $\displaystyle\Theta_{s-\dim H,t-\dim H-d-dp}(h)~{}.$

Fix an arbitrary $\xi_{0}\in\mathcal{O}$, and define
$\widetilde{\Theta}_{s,t}:H\rightarrow(0,\infty)$ as
$\widetilde{\Theta}_{s,t}(h)=\int_{H_{\xi_{0}}}\Theta_{s,t}(kh)\text{\,d}_{H_{\xi_{0}}}k$,
where $H_{\xi_{0}}$ is the stabilizer subgroup at $\xi_{0}$ with Haar measure
$\text{\,d}_{H_{\xi_{0}}}k$. Note that the integral defining
$\widetilde{\Theta}_{s,t}$ is bounded, as $H_{\xi_{0}}$ is compact. Next, we
observe that $\widetilde{\Theta}_{s,t}$ can be bounded from above and below by
multiples of ${\Theta}_{s,t}$, i.e., there exist positive constants
$a_{s,t},b_{s,t}$ such that

$a_{s,t}\Theta_{s,t}\leq\widetilde{\Theta}_{s,t}\leq b_{s,t}\Theta_{s,t}.$

Indeed, the inequality $\|gh\|\leq\|g\|~{}\|h\|$ entails
$\Theta_{s,t}(gh)\geq\Theta_{s,t}(g)\Theta_{s,t}(h)$. Hence we get

$\displaystyle\widetilde{\Theta}_{s,t}(h)=\int_{H_{\xi_{0}}}\Theta_{s,t}(kh)\text{\,d}_{H_{\xi_{0}}}k\geq\Theta_{s,t}(h)\int_{H_{\xi_{0}}}\Theta_{s,t}(k)\text{\,d}_{H_{\xi_{0}}}k=a_{s,t}\Theta_{s,t}(h),$

where $a_{s,t}:=\int_{H_{\xi_{0}}}\Theta_{s,t}(k)\text{\,d}_{H_{\xi_{0}}}k$.
On the other hand, the inequality $\|h\|\leq\|kh\|\|k^{-1}\|$ implies that
$(1+\|kh\|)^{-1}\leq(1+\|h\|)^{-1}(1+\|k^{-1}\|)$, which in turn implies that

$\displaystyle\int_{H_{\xi_{0}}}\Theta_{s,t}(kh)\text{\,d}_{H_{\xi_{0}}}k\leq\Theta_{s,t}(h)\int_{H_{\xi_{0}}}(1+\|k^{-1}\|)^{s}(1+\|k\|)^{t}\text{\,d}_{H_{\xi_{0}}}k=b_{s,t}\Theta_{s,t}(h),$

where
$b_{s,t}:=\int_{H_{\xi_{0}}}(1+\|k^{-1}\|)^{s}(1+\|k\|)^{t}\text{\,d}_{H_{\xi_{0}}}k$.
Note that both $a_{s,t},b_{s,t}$ are finite, as integrals of continuous
functions on a compact group, and that $a_{s,t}>0$.

The function $\widetilde{\Theta}_{s,t}$ can be regarded as a well-defined
function on $\mathcal{O}$, which we denote by $F_{s,t}$, in the following
manner:

$F_{s,t}:\mathcal{O}\rightarrow(0,\infty),\
F_{s,t}(h^{T}\xi_{0})=\widetilde{\Theta}_{s,t}(h),$

since $\widetilde{\Theta}_{s,t}(lh)=\widetilde{\Theta}_{s,t}(h)$ for every
$h\in H$ and $l\in H_{\xi_{0}}$. We will use $F_{s,t}$ to relate the 1-norm of
$\Theta_{s,t}$ to an integral on the orbit $\mathcal{O}$. First observe that
for $\xi=h^{T}\xi_{0}$, we clearly have $\frac{|\xi|}{|\xi_{0}|}\leq\|h\|$,
which implies that $(1+\|h\|)^{-1}\leq(1+\frac{|\xi|}{|\xi_{0}|})^{-1}$.
Hence, as $t,s\geq 0$ we have

$F_{s,t}(\xi)=F_{s,t}(h^{T}\xi_{0})=\widetilde{\Theta}_{s,t}(h)\leq
b_{s,t}(1+\|h\|)^{-s}(1+\|h^{-1}\|)^{-t}\leq
b_{s,t}\left(1+\frac{|\xi|}{|\xi_{0}|}\right)^{-s}~{}.$

Letting $s^{\prime}=s-\dim H$ and $t^{\prime}=t-d-dp-\dim H$, from the
inequality (19) we get

$\displaystyle\int_{H}|\det(h)|^{-p}\Theta_{s,t}(h)\text{\,d}h$
$\displaystyle\leq$
$\displaystyle\int_{H}\Theta_{s^{\prime},t^{\prime}}(h)\frac{|\det(h)|}{\Delta_{H}(h)}\text{\,d}h$
$\displaystyle\leq$
$\displaystyle\frac{1}{a_{s^{\prime},t^{\prime}}}\int_{\mathcal{O}}F_{s^{\prime},t^{\prime}}(\xi)\text{\,d}\xi$
$\displaystyle\leq$
$\displaystyle\frac{b_{s^{\prime},t^{\prime}}}{a_{s^{\prime},t^{\prime}}}\int_{\mathcal{O}}\left(1+\frac{|\xi|}{|\xi_{0}|}\right)^{-s^{\prime}}\text{\,d}\xi,$

which is a bounded integral if $s^{\prime}\geq d+1$ and $t^{\prime}\geq 0$. ∎

As a first application of these estimates, we can formulate explicit criteria
for the formula $W_{\eta}u=W_{\psi}u\ast W_{\eta}\psi$ to hold.

######  Proposition 4.10.

Assume that there exist positive numbers $\ell_{1},C\in{\mathbb{R}}^{+}$ so
that for every $h\in H$,

$\displaystyle\|h^{\pm 1}\|A_{H}(h)^{\ell_{1}}$ $\displaystyle\leq$
$\displaystyle C~{}.$ (20)

Let $u\in\mathcal{S}^{\prime}(\mathbb{R}^{d})$ denote a tempered distribution
of order $N(u)$. Let $\eta,\psi\in\mathcal{S}(\mathbb{R}^{d})$ denote
functions with vanishing moments of order $r$ in $\mathcal{O}^{c}$, where

$r>2d+N(u)+\ell_{1}(5+2\dim(H)+6d+3N(u))+2~{}.$

Then

$W_{\eta}u=\frac{1}{C_{\psi}}W_{\psi}u\ast W_{\eta}\psi$

holds with absolute convergence of the convolution integral. If $H_{0}<H$ is a
half-space adapted subgroup and $\psi$ is real-valued, convolution over $G$
can be replaced by convolution over $G_{0}=\mathbb{R}^{d}\rtimes H_{0}$, with
twice the normalization factor.

###### Proof.

We first recall the wavelet inversion formula

$\eta=\frac{1}{C_{\psi}}\int_{G}W_{\psi}\eta(x,h)\pi(x,h)\psi\text{\,d}(x,h)~{},$
(21)

which is valid in the weak ${L}^{2}$-sense. The proof is based on an
alternative interpretation of the right hand side as a Bochner integral
converging in $|\cdot|_{M}$, for a suitable choice of $M$. More precisely, we
let

$M=\left\\{\begin{array}[]{cc}N(u)+\frac{d}{2}+1&d\mbox{ even }\\\
N(u)+\frac{d+1}{2}&d\mbox{ odd }\end{array}\right.~{}.$

To see that the desired convergence holds, we introduce the quantities

$\beta_{1}=M+d+1~{},~{}\beta_{2}=2+\dim(H)+\frac{3}{2}d+N(u)~{},~{}\beta_{3}=1+\dim(H)+3d+N(u)~{},$
(22)

and observe that

$r-d/2-\ell_{1}(\beta_{1}+\beta_{2}+\beta_{3})-\beta_{1}>0~{}.$

This implies, for sufficiently small $\epsilon>0$ that

$r-d/2-\ell_{1}(\beta_{1}+\epsilon+\beta_{2}+\beta_{3})-\beta_{1}-\epsilon>0~{}.$
(23)

We can then estimate the norm of the integrand in (21) as follows:

$\displaystyle\left|(W_{\psi}\eta)(x,h)\pi(x,h)\psi\right|_{M}$
$\displaystyle\leq$ $\displaystyle
C_{M}|\psi|_{{M}}|W_{\psi}\eta(x,h)|(1+|x|)^{{M}}{|\det(h)|^{-1/2}}(1+\|h\|)^{{M}}(1+\|h^{-1}\|)^{{M}}$
$\displaystyle\leq$
$\displaystyle{C^{\prime}}(1+|x|)^{{{M}}-\beta_{1}-\epsilon}{|\det(h)|^{-1/2}}(1+\|h\|)^{{{M}}-\beta_{2}}(1+\|h^{-1}\|)^{{{M}}-\beta_{3}}$
$\displaystyle=$
$\displaystyle{C^{\prime}}(1+|x|)^{{{M}}-\beta_{1}-\epsilon}|\det(h)|^{-1/2}\Theta_{\beta_{2}-{{M}},\beta_{3}-{{M}}}(h)~{},$

where the constant $C^{\prime}$ does not depend on $x,h$. Here the first
inequality is due to Lemma 3.1(a), and the second one follows from Proposition
4.8, which is applicable thanks to (23). Recall that
$\text{\,d}(x,h)=\frac{\text{\,d}x\text{\,d}h}{|\det(h)|}$. Now Lemma 4.9 and
our choice of $\beta_{2},\beta_{3}$ imply (via $M\leq N(u)+1+\frac{d}{2}$)
that $|\det(h)|^{{-3/2}}\Theta_{\beta_{2}-M,\beta_{3}-M}$ is integrable over
$H$. Also, since $\beta_{1}+\epsilon-M>d$, the function
$(1+|x|)^{M-\beta_{1}-\epsilon}$ is integrable over ${\mathbb{R}}^{d}$, and
thus the right-hand side of (21) is a Bochner integral converging in the
completion $X$ of $\mathcal{S}(\mathbb{R}^{d})$ with respect to $|\cdot|_{M}$.
In fact, the inequality $d<2M$ entails $\|f\|_{2}\leq|f|_{M}$, which implies
that $X$ embeds continuously into ${L}^{2}(\mathbb{R}^{d})$. Thus the limit of
the Bochner integral coincides with the weak ${L}^{2}$-limit, i.e., with
$\eta$. So the equation given in (21) holds in $X$ as well. Since $N(u)\leq
M$, $u$ extends to a continuous functional on $X$, which entails that we can
interchange evaluation of $u$ and Bochner integration, giving for all
$(x^{\prime},h^{\prime})$

$\displaystyle W_{\eta}u(x^{\prime},h^{\prime})$ $\displaystyle=$
$\displaystyle\langle\pi(x^{\prime},h^{\prime})^{-1}u,\eta\rangle$
$\displaystyle=$
$\displaystyle\frac{1}{C_{\psi}}\int_{G}\overline{W_{\psi}\eta(x,h)}\langle\pi(x^{\prime},h^{\prime})^{-1}{u},\pi(x,h)\psi\rangle\text{\,d}(x,h)$
$\displaystyle=$ $\displaystyle\frac{1}{C_{\psi}}\int_{G}\langle
u,\pi((x^{\prime},h^{\prime})(x,h))\psi\rangle
W_{\eta}\psi((x,h)^{-1})\text{\,d}(x,h)$ $\displaystyle=$
$\displaystyle\frac{1}{C_{\psi}}\int_{G}W_{\psi}u(x,h)W_{\eta}\psi((x,h)^{-1}(x^{\prime},h^{\prime}))\text{\,d}(x,h)$
$\displaystyle=$ $\displaystyle\frac{1}{C_{\psi}}(W_{\psi}u\ast
W_{\eta}\psi)(x^{\prime},h^{\prime})~{}.$

Here absolute convergence of the convolution integral is guaranteed by Bochner
convergence.

The statement for real-valued wavelets $\psi$ then follows from Lemma 4.4. ∎

##  5\. Criteria for $N$-regular directed points: Compactly supported wavelets

After the groundwork laid in the previous section, we can now establish the
first major result of our paper, namely Theorem 5.4. It shows that the decay
order of wavelet coefficients is stable under exchange of wavelets, as long as
both wavelets in the exchange have a certain minimal number of vanishing
moments. As an auxiliary notion to guarantee this transfer, one further
somewhat technical condition needs to be formulated. The intuition behind this
definition and its necessity is discussed in Remark 5.5.

######  Definition 5.1.

Let $V\Subset\mathcal{O}$. We say that $H$ _fulfills the $V$-overlap control
condition with parameters $\gamma_{0}\geq 0,\gamma_{1},\gamma_{2}>0$_ in
direction $\xi\in\mathcal{O}\cap S^{d-1}$ if there exist

  * •

a half-space adapted subgroup $H_{0}<H$,

  * •

a neighborhood $W^{\prime}$ of $\xi$ in $S^{d-1}$ and $R^{\prime}>0$;

such that the following holds: For every pair
$(W^{\prime\prime},R^{\prime\prime})$ consisting of a $\xi$-neighborhood
$W^{\prime\prime}\subseteq W^{\prime}$ and $R^{\prime\prime}\geq R^{\prime}$
there exists a $\xi$-neighborhood $W$ and $R>0$ such that for all $L\geq 0$
and all

$s\geq\gamma_{0}+\gamma_{1}L~{},~{}t\geq\gamma_{0}+\gamma_{2}L$

the estimate

$\forall h\in K_{o}(W,V,R):\int_{H_{0}\setminus
h^{-1}K_{i}(W^{\prime\prime},V,R^{\prime\prime})}\Theta_{s,t}(g)\text{\,d}g\leq
C\|h\|^{L}$

holds with a constant $C$ independent of $h\in K_{o}(W,V,R)$.

Clearly, if $H$ satisfies the $V$-overlap control condition with $W^{\prime}$
and $R^{\prime}$ as in the above definition, then it would satisfy the same
condition if we replace $W^{\prime}$ by $\widetilde{W^{\prime}}\subseteq
W^{\prime}$, and $R^{\prime}$ by $\widetilde{R^{\prime}}\geq R^{\prime}$. We
next want to establish that the overlap condition only needs to be verified
for one direction. This argument is somewhat akin to the proof of the
analogous statement for the cone approximation property [5, Lemma 4.5].

######  Lemma 5.2.

If $H$ fulfills the $V$-overlap control condition with parameters
$\gamma_{0}\geq 0,\gamma_{1},\gamma_{2}>0$ at $\xi_{0}\in\mathcal{O}$, then it
fulfills this condition at all directions $\xi_{1}\in\mathcal{O}\cap S^{d-1}$
with the same parameters $\gamma_{0},\gamma_{1},\gamma_{2}$.

###### Proof.

Given $\xi_{1}\in\mathcal{O}\cap S^{d-1}$, we pick $h\in H$ with
$h^{T}\xi_{0}=\xi_{1}$. Assume that $W_{0}^{\prime}$ and $R_{0}^{\prime}$ are
given to guarantee the $V$-overlap control condition at $\xi_{0}$. By
appealing to Lemma 2.3(b) we find $W^{\prime}_{1},R^{\prime}_{1}$ so that
$K_{o}(W^{\prime}_{1},V,R^{\prime}_{1})\subseteq
h^{-1}K_{o}(W^{\prime}_{0},V,R^{\prime}_{0})$. Next, we show that
$W^{\prime}_{1}$ and $R^{\prime}_{1}$ satisfy the requirements of the
$V$-overlap control condition with parameters $\gamma_{1},\gamma_{2}>0$ in
direction $\xi_{1}$.

Let $W_{1}^{\prime\prime}\subset W_{1}^{\prime}$ and
$R_{1}^{\prime\prime}>R_{1}^{\prime}$ be given. Applying Lemma 2.3(a), we can
then find a $\xi_{0}$-neighborhood $\widetilde{W_{0}}$ and
$\widetilde{R_{0}}>0$ with

$K_{i}(W_{1}^{\prime\prime},V,R_{1}^{\prime\prime})\supseteq
h^{-1}K_{i}(\widetilde{W_{0}},V,\widetilde{R_{0}})~{}.$

Then $W_{0}^{\prime\prime}=W_{0}^{\prime}\cap\widetilde{W_{0}}\subset
W_{0}^{\prime}$ and
$R_{0}^{\prime\prime}=\max(R_{0}^{\prime},\widetilde{R_{0}})\geq
R_{0}^{\prime}$ fulfill

$h^{-1}K_{i}(W_{0}^{\prime},V,R_{0}^{\prime})\supseteq
h^{-1}K_{i}(W_{0}^{\prime\prime},V,R_{0}^{\prime\prime})~{},$

as well. By $V$-overlap control condition at $\xi_{0}$, given
$W_{0}^{\prime\prime},R_{0}^{\prime\prime}$, there exist a
$\xi_{0}$-neighborhood $W_{0}$ and $R_{0}>0$ such that for all $L\geq 0$ and
all $s\geq\gamma_{0}+\gamma_{1}L$ and $t\geq\gamma_{0}+\gamma_{2}L$ the
estimate

$\forall\,l\in K_{o}(W_{0},V,R_{0}):\int_{H_{0}\setminus
l^{-1}K_{i}(W_{0}^{\prime\prime},V,R_{0}^{\prime\prime})}\Theta_{s,t}(g)\text{\,d}g\leq
C\|l\|^{L}$ (24)

holds with a constant $C$ independent of $l\in K_{o}(W_{0},V,R_{0})$. Applying
Lemma 2.3(b) again, there exists a $\xi_{1}$-neighborhood $W_{1}$ and
$R_{1}>0$ such that $K_{o}(W_{1},V,R_{1})\subseteq
h^{-1}K_{o}(W_{0},V,R_{0})$.

Given $L$, pick $s\geq\gamma_{0}+\gamma_{1}L$ and
$t\geq\gamma_{0}+\gamma_{2}L$. Let $h_{1}\in K_{o}(W_{1},V,R_{1})$ be
arbitrary, and denote $hh_{1}=h_{0}$. Now
$h^{-1}K_{i}(W_{0}^{\prime\prime},V,R_{0}^{\prime\prime})\subseteq
K_{i}(W_{1}^{\prime\prime},V,R_{1}^{\prime\prime})$ implies
$h_{0}^{-1}K_{i}(W_{0}^{\prime\prime},V,R_{0}^{\prime\prime})\subseteq
h_{1}^{-1}K_{i}(W_{1}^{\prime\prime},V,R_{1}^{\prime\prime})$, and thus

$\int_{H_{0}\setminus
h_{1}^{-1}K_{i}(W_{1}^{\prime\prime},V,R_{1}^{\prime\prime})}\Theta_{s,t}(g)\text{\,d}g\leq\int_{H_{0}\setminus
h_{0}^{-1}K_{i}(W_{0}^{\prime\prime},V,R_{0}^{\prime\prime})}\Theta_{s,t}(g)\text{\,d}g~{}.$
(25)

Clearly, $h_{0}\in K_{o}(W_{0},V,R_{0})$, as $h_{1}\in K_{o}(W_{1},V,R_{1})$.
So combining (24) and (25), we get

$\int_{H_{0}\setminus
h_{1}^{-1}K_{i}(W_{1}^{\prime\prime},V,R_{1}^{\prime\prime})}\Theta_{s,t}(g)\text{\,d}g\leq
C\|h_{0}\|^{L}\leq\underbrace{C\|h\|^{L}}_{=C^{\prime}}\|h_{1}\|^{L}~{},$

which shows the desired property. ∎

The lemma shows that the $V$-overlap condition is a global property on
$\mathcal{O}\cap S^{d-1}$, once it is established for a single direction.
Accordingly, we can drop the reference to the direction $\xi$ in the
subsequent usage of that condition.

######  Remark 5.3.

The following theorem is central to our paper, as it weakens the requirements
on the analyzing wavelet from compact support in frequency to finite numbers
of vanishing moments.

But besides this rather direct benefit, the theorem is also of independent
interest for generalized wavelet analysis, since it expresses that, as soon as
the analyzing wavelet is chosen from a suitable class, a certain type of
wavelet coefficient decay is independent of the precise choice of wavelet. In
other words, this decay is recognized as an intrinsic property of the analyzed
function. This is akin to a fundamental observation from coorbit theory [2, 3,
4]. Recall that coorbit spaces quantify integrability properties of wavelet
coefficients, and thereby provide a global measure of wavelet coefficient
decay. An important foundational aspect of this theory is consistency, i.e.
the fact that (up to equivalence) this global decay behaviour is independent
of the analyzing wavelet, see [2]. The more recent paper [10] provides
guarantees for this independence in terms of vanishing moment conditions on
wavelets, for the wavelet transforms of the type studied in our paper.

In this context, Theorem 5.4 can be seen as an analog to the coorbit space
results in [10], by guaranteeing a similar type of consistency of wavelet
coefficient decay localized inside cone-affiliated sets, which holds for all
wavelets with sufficiently many vanishing moments.

######  Theorem 5.4.

Let $x_{0}\in{\mathbb{R}}^{d}$ and $\xi_{0}\in\mathcal{O}\cap S^{d-1}$ be
fixed. Assume that the dual action is $V$-microlocally admissible in direction
$\xi_{0}$, with associated $\xi_{0}$-neighborhood $W_{0}$ and $R_{0}>0$, and
with exponent $\alpha_{1}$. Let $N\in{\mathbb{N}}$ be fixed.

Let $u\in{\mathcal{S}}^{\prime}({\mathbb{R}}^{d})$ be a tempered distribution
of order $N(u)$. Let $\psi_{1},\psi_{2}\in{\mathcal{S}}({\mathbb{R}}^{d})$ be
wavelets of vanishing moments of order $r$ in $\mathcal{O}^{c}$, with
$\psi_{1}$ being real-valued. Assume that the parameters
$\beta_{1},\beta_{2},\beta_{3}$ and $r$ fulfill the following inequalities:

  * (i)

$\beta_{1}\geq N+N(u)+\alpha_{1}(N(u)+d/2)+d+1$,

  * (ii)

$\beta_{2}\geq\dim H+N+d+1$, and $\beta_{3}\geq\dim H+2d$ (which implies
$\Theta_{\beta_{2}-{N},\beta_{3}-d}\in{L}^{1}(H)$),

  * (iii)

$\beta_{1}\geq 2+\frac{3}{2}d+N(u)$, $\beta_{2}\geq
2+\dim(H)+\frac{3}{2}d+N(u)$ and $\beta_{3}\geq 1+\dim(H)+3d+N(u)$, (which
implies
$|\det(\cdot)|^{-p}\Theta_{\beta_{2}-N(u),\beta_{3}-N(u)}\in{L}^{1}(H)$ for
$p=\frac{1}{2},\frac{3}{2}$),

  * (iv)

$H$ fulfills the $V$-overlap control condition with parameters $\gamma_{0}\geq
0,\gamma_{1},\gamma_{2}>0$, and

$\displaystyle\beta_{2}$ $\displaystyle\geq$
$\displaystyle\gamma_{0}+\gamma_{1}\alpha_{1}\left(\frac{3}{2}d+N(u)\right)+N(u)+\gamma_{1}N$
$\displaystyle\beta_{3}$ $\displaystyle\geq$
$\displaystyle\gamma_{0}+\gamma_{2}\alpha_{1}\left(\frac{3}{2}d+N(u)\right)+N(u)+\frac{3}{2}d+\gamma_{2}N$

  * (v)

$r>d/2+\ell_{1}({\beta_{1}+\beta_{2}}+\beta_{3})+\beta_{1}$, where
$\ell_{1}>0$ satisfies (18) of Proposition 4.8.

Let $\epsilon>0$ be given, and assume that there exist suitable choices of
$\xi_{0}$-neighborhood $W^{\prime},R^{\prime}$ with $|W_{\psi_{1}}u(y,h)|\leq
C\|h\|^{N}$ for every $y\in B_{\epsilon}(x_{0})$ and every $h\in
K_{i}(W^{\prime},V,R^{\prime})$, where the constant $C$ is independent of
$h,y$. Then there exist $\xi_{0}$-neighborhood $W,R$ such that
$|W_{\psi_{2}}u(y,h)|\leq C^{\prime}\|h\|^{N}$ for every $y\in
B_{\epsilon/2}(x_{0})$ and every $h\in K_{o}(W,V,R)$, where the constant
$C^{\prime}$ is independent of the choice of $h,y$.

###### Proof.

Throughout this proof note that assumption $(v)$ guarantees that Proposition
4.8 is available to estimate ${W}_{\psi_{1}}\psi_{2}$.

Suppose that $W^{\prime},R^{\prime}$ are as provided by the premise of the
theorem, that is,

$|{W}_{\psi_{1}}u(y,g)|\leq C\|g\|^{N}\ \mbox{ for every }y\in
B_{\epsilon}(x_{0})\ \mbox{ and every }g\in K_{i}(W^{\prime},V,R^{\prime}),$
(26)

where the constant $C$ is independent of $y$ and $g$. We first note that the
premise remains true if we replace $W^{\prime}$ by sufficiently small
$W^{\prime\prime}\subset W^{\prime}$ and $R^{\prime}$ by sufficiently large
$R^{\prime\prime}>R^{\prime}$ to make the overlap control condition
applicable, and we choose $W,R$ as provided by that condition so that for
$s,t$ as in Definition 5.1, we have

$\exists C_{1}>0\ \forall h\in K_{o}(W,V,R):\int_{H_{0}\setminus
h^{-1}K_{i}(W^{\prime\prime},V,R^{\prime\prime})}\Theta_{s,t}(g)\text{\,d}g\leq
C_{1}\|h\|^{L}.$ (27)

Since replacing $W$ and $R$ by $W\cap W_{0}$ and $\max\\{R,R_{0}\\}$ does not
disturb the conclusion of the overlap control condition, we can also assume
that $W\subseteq W_{0}$ and $R\geq R_{0}$.

We start out by verifying that, under the assumptions of the theorem, the
convolution relation

$W_{\psi_{2}}u=\frac{2}{C_{\psi_{1}}}({W}_{\psi_{1}}u*{W}_{\psi_{2}}\psi_{1})$

holds with convolution taken over $\mathbb{R}^{d}\rtimes H_{0}$. This is done
most easily by comparing the assumptions (iii) and (v) with those of
Proposition 4.10, and with (22) and (23).

Throughout the following computations, let $h\in K_{o}(W,V,R)$, and denote
$K=K_{i}(W^{\prime\prime},V,R^{\prime\prime})$. We now split the convolution
integral into ${W}_{\psi_{2}}u(y,h)=\frac{2}{C_{\psi_{1}}}(I_{1}+I_{2})$,
where

$\displaystyle I_{1}$ $\displaystyle=$
$\displaystyle\int_{h^{-1}B_{\epsilon/2}(0)}\int_{H_{0}}{W}_{\psi_{1}}u(y+hx,hg){W}_{\psi_{2}}\psi_{1}(-g^{-1}x,g^{-1})\text{\,d}x\frac{\text{\,d}g}{|\det(g)|},$
$\displaystyle I_{2}$ $\displaystyle=$
$\displaystyle\int_{{\mathbb{R}}^{d}\setminus
h^{-1}B_{\epsilon/2}(0)}\int_{H_{0}}{W}_{\psi_{1}}u(y+hx,hg){W}_{\psi_{2}}\psi_{1}(-g^{-1}x,g^{-1})\text{\,d}x\frac{\text{\,d}g}{|\det(g)|}.$

Let $y\in B_{\epsilon/2}(x_{0})$ and $h\in K_{o}(W,V,R)$ be given. We prove
appropriate upper bounds for $I_{1}$ and $I_{2}$ separately.

$\displaystyle|I_{1}|$ $\displaystyle=$
$\displaystyle\left|\int_{h^{-1}B_{\epsilon/2}(0)}\int_{H_{0}}{W}_{\psi_{1}}u(y+hx,hg){W}_{\psi_{2}}\psi_{1}(-g^{-1}x,g^{-1})\text{\,d}x\frac{\text{\,d}g}{|\det(g)|}\right|$
$\displaystyle\leq$
$\displaystyle\int_{h^{-1}B_{\epsilon/2}(0)}\int_{H_{0}}\left|{W}_{\psi_{1}}u(y+hx,hg){W}_{\psi_{1}}\psi_{2}(x,g)\right|\text{\,d}x\frac{\text{\,d}g}{|\det(g)|}$
$\displaystyle=$ $\displaystyle I_{1}^{\prime}+I_{1}^{\prime\prime},$

where we split the integral into $I^{\prime}_{1}$ and $I^{\prime\prime}_{1}$
by taking the inner integral over $h^{-1}K$ and $H_{0}\setminus h^{-1}K$
respectively. The point of this separation of integrals is that for all
$(x,g)\in h^{-1}(B_{\epsilon/2}(0))\times h^{-1}K$, one has $(y+hx,hg)\in
B_{\epsilon}(x_{0})\times K$, and thus the decay assumptions on
$W_{\psi_{1}}u$ can be used to continue

$\displaystyle I^{\prime}_{1}$ $\displaystyle=$
$\displaystyle\int_{h^{-1}B_{\epsilon/2}(0)}\int_{h^{-1}K}\left|{W}_{\psi_{1}}u(y+hx,hg){W}_{\psi_{1}}\psi_{2}(x,g)\right|\text{\,d}x\frac{\text{\,d}g}{|\det(g)|}$
$\displaystyle\leq$ $\displaystyle
C\|h\|^{N}\int_{h^{-1}B_{\epsilon/2}(0)}\int_{h^{-1}K}{\|g\|^{N}}|{W}_{\psi_{1}}\psi_{2}(x,g)|\text{\,d}x\frac{\text{\,d}g}{|\det(g)|}$
$\displaystyle\leq$ $\displaystyle
CD\|h\|^{N}\int_{h^{-1}B_{\epsilon/2}(0)}\int_{h^{-1}K}(1+|x|)^{-\beta_{1}}(1+\|g\|)^{{N}-\beta_{2}}(1+\|g^{-1}\|)^{-\beta_{3}}\text{\,d}x\frac{\text{\,d}g}{|\det(g)|}$
$\displaystyle\leq$ $\displaystyle
CD\|h\|^{N}\left(\int_{h^{-1}B_{\epsilon/2}(0)}(1+|x|)^{-\beta_{1}}\text{\,d}x\right)\left(\int_{h^{-1}K}(1+\|g\|)^{{N}-\beta_{2}}{\|g^{-1}\|^{d}}(1+\|g^{-1}\|)^{-\beta_{3}}{\text{\,d}g}\right)$
$\displaystyle\leq$ $\displaystyle
CD\|h\|^{N}\left(\int_{{\mathbb{R}}^{d}}(1+|x|)^{-\beta_{1}}\text{\,d}x\right)\left(\int_{H}(1+\|g\|)^{-(\beta_{2}-{N})}(1+\|g^{-1}\|)^{-(\beta_{3}-d)}{\text{\,d}g}\right)$
$\displaystyle\leq$ $\displaystyle D_{1}\|h\|^{N},$

where the two integrals in the penultimate line are bounded, since
$\beta_{1}>d$ and $\Theta_{\beta_{2}-{N},\beta_{3}-d}$ belongs to ${L}^{1}(H)$
thanks to condition (ii). Also note that $D_{1}$ is a product of the constant
from Proposition 4.8, the constant $C$ from the assumption, and terms
depending on $\beta_{1},\beta_{2}$, and $\beta_{3}$.

To bound $I_{1}^{\prime\prime}$, we use (b) of Lemma 3.1 and Proposition 4.8.
To avoid clutter of notation, we will denote $N(u)$ by $N^{\prime}$.

$\displaystyle I^{\prime\prime}_{1}$ $\displaystyle=$
$\displaystyle\int_{h^{-1}B_{\epsilon/2}(0)}\int_{H_{0}\setminus
h^{-1}K}\left|{W}_{\psi_{1}}u(y+hx,hg){W}_{\psi_{1}}\psi_{2}(x,g)\right|\text{\,d}x\frac{\text{\,d}g}{|\det(g)|}$
$\displaystyle\leq$
$\displaystyle\int_{h^{-1}B_{\epsilon/2}(0)}\int_{H_{0}\setminus
h^{-1}K}C_{2}|\det(hg)|^{-1/2}(1+\|g^{-1}h^{-1}\|)^{N^{\prime}}\max\\{1,\|hg\|^{N^{\prime}}\\}(1+|y+hx|)^{N^{\prime}}$
$\displaystyle\qquad\qquad\qquad\quad\cdot(1+|x|)^{-\beta_{1}}(1+\|g\|)^{-\beta_{2}}(1+\|g^{-1}\|)^{-\beta_{3}}\text{\,d}x\frac{\text{\,d}g}{|\det(g)|}$
$\displaystyle=$ $\displaystyle C_{2}J_{1}J^{\prime}_{1},$

where $C_{2}$ is the product of the constant terms from part (b) of Lemma 3.1
and Proposition 4.8, and

$\displaystyle J_{1}$ $\displaystyle=$
$\displaystyle\int_{h^{-1}B_{\epsilon/2}(0)}(1+|x|)^{-\beta_{1}}(1+|y+hx|)^{N^{\prime}}\text{\,d}x,$
$\displaystyle J^{\prime}_{1}$ $\displaystyle=$
$\displaystyle\int_{H_{0}\setminus
h^{-1}K}|\det(hg)|^{-1/2}(1+\|g^{-1}h^{-1}\|)^{N^{\prime}}(1+\|hg\|)^{N^{\prime}}$
$\displaystyle(1+\|g\|)^{-\beta_{2}}(1+\|g^{-1}\|)^{-\beta_{3}}\frac{\text{\,d}g}{|\det(g)|}.$

It is easy to obtain a bound for $J_{1}$. Indeed, we note that
$|y+hx|\leq|y|+|hx|\leq|x_{0}|+\epsilon\leq 2|x_{0}|$, as $x\in
h^{-1}B_{\epsilon/2}(0)$, $y\in B_{\epsilon/2}(x_{0})$, and without loss of
generality we can assume $\epsilon\leq|x_{0}|$. So,

$J_{1}\leq(1+2|x_{0}|)^{N^{\prime}}|\det(h)|^{-1}\int_{B_{\epsilon/2}(0)}(1+|h^{-1}x|)^{-\beta_{1}}\text{\,d}x\leq(1+2|x_{0}|)^{N^{\prime}}|\det(h)|^{-1}\lambda_{{\mathbb{R}}^{d}}({B_{\epsilon/2}(0)}),$

where $\lambda_{{\mathbb{R}}^{d}}({B_{\epsilon/2}(0)})$ is the volume of the
$\frac{\epsilon}{2}$-ball.

We next estimate the various factors in the integrand of $J_{1}^{\prime}$ as
follows:

$\displaystyle\frac{|{\rm det}(hg)|^{-1/2}}{|{\rm det}(g)|}$
$\displaystyle\leq$ $\displaystyle|{\rm
det}(h)|^{-1/2}(1+\|g^{-1}\|)^{\frac{3}{2}d}~{},$
$\displaystyle(1+\|g^{-1}h^{-1}\|)^{N^{\prime}}$ $\displaystyle\leq$
$\displaystyle(1+\|h^{-1}\|)^{N^{\prime}}(1+\|g^{-1}\|)^{N^{\prime}}~{},$
$\displaystyle(1+\|hg\|)^{N^{\prime}}$ $\displaystyle\leq$
$\displaystyle(1+\|h\|)^{N^{\prime}}(1+\|g\|)^{N^{\prime}}~{}.$

Here we used Hadamard’s inequality to estimate the determinant against the
norm, and submultiplicativity of the norm. These estimates lead to

$\displaystyle{J^{\prime}_{1}}$ $\displaystyle\leq$
$\displaystyle|\det(h)|^{-1/2}(1+\|h\|)^{N^{\prime}}(1+\|h^{-1}\|)^{N^{\prime}}\int_{H_{0}\setminus
h^{-1}K}\Theta_{\beta_{2}-N^{\prime},\beta_{3}-N^{\prime}-\frac{3}{2}d}(g)dg$
$\displaystyle\leq$
$\displaystyle{C_{1}}|\det(h)|^{-1/2}(1+\|h\|)^{N^{\prime}}(1+\|h^{-1}\|)^{N^{\prime}}\|h\|^{L}~{}.$

Here the second inequality used the overlap control condition, applied with

${s}=\beta_{2}-N^{\prime}~{},~{}{t}=\beta_{3}-N^{\prime}-\frac{3}{2}d~{},~{}L=\alpha_{1}\left(\frac{3}{2}d+N^{\prime}\right)+N~{}.$

The applicability of this condition is guaranteed by assumption (iv), as well
as $h\in K_{o}(W,V,R)$, $K=K_{i}(W^{\prime\prime},V,R^{\prime\prime})$ and the
choice of $W,R$ depending on $W^{\prime\prime},R^{\prime\prime}$ according to
the mentioned condition (see (27)).

Using Hadamard’s inequality, followed by property (a) of microlocal
admissibility (Definition 2.4) for $h\in K_{o}(W,V,R)$, we have

$|\det(h^{-1})|^{3/2}\leq\|h^{-1}\|^{\frac{3}{2}d}\lesssim\|h\|^{-\frac{3}{2}\alpha_{1}d},$

which allows to estimate $I_{1}^{\prime\prime}$:

$\displaystyle I_{1}^{\prime\prime}$ $\displaystyle\leq$
$\displaystyle{C_{3}}|\det(h)|^{-3/2}(1+\|h\|)^{N^{\prime}}(1+\|h^{-1}\|)^{N^{\prime}}\|h\|^{N+\alpha_{1}(\frac{3}{2}d+N^{\prime})}$
$\displaystyle\leq$
$\displaystyle{C_{4}(1+\|h\|)^{N^{\prime}}(1+\|h^{-1}\|)^{N^{\prime}}\|h\|^{N+\alpha_{1}N^{\prime}}~{}.}$

Now, if $\|h^{-1}\|\leq 1$, then $1+\|h^{-1}\|\leq 2$; otherwise,
$1+\|h^{-1}\|\leq 2\|h^{-1}\|\lesssim\|h\|^{-\alpha_{1}}$ by microlocal
admissibility. In any case, we obtain

$(1+\|h^{-1}\|)^{N^{\prime}}\|h\|^{N+\alpha_{1}N^{\prime}}\leq
C_{5}\cdot\max\\{\|h\|^{N},\|h\|^{N+\alpha_{1}N^{\prime}}\\}~{}.$

Furthermore, Lemma 2.6 supplies that $\|h\|$ is bounded on $K_{o}(W,V,R)$,
whence we finally arrive at

$I_{1}^{\prime\prime}\leq C_{6}\|h\|^{N}~{}.$

This also concludes the estimate of $I_{1}$.

We now estimate $I_{2}$, as follows:

$\displaystyle|I_{2}|$ $\displaystyle\leq$
$\displaystyle\int_{{\mathbb{R}}^{d}\setminus
h^{-1}B_{\epsilon/2}(0)}\int_{H_{0}}\left|{W}_{\psi_{1}}u(y+hx,hg)\right|\left|{W}_{\psi_{2}}\psi_{1}(-g^{-1}x,g^{-1})\right|\text{\,d}x\frac{\text{\,d}g}{|\det(g)|}$
$\displaystyle=$ $\displaystyle\int_{{\mathbb{R}}^{d}\setminus
h^{-1}B_{\epsilon/2}(0)}\int_{H_{0}}\left|{W}_{\psi_{1}}u(y+hx,hg)\right|\left|{W}_{\psi_{1}}\psi_{2}(x,g)\right|\text{\,d}x\frac{\text{\,d}g}{|\det(g)|}$
$\displaystyle\leq$ $\displaystyle\int_{{\mathbb{R}}^{d}\setminus
h^{-1}B_{\epsilon/2}(0)}\int_{H_{0}}C_{2}|\det(hg)|^{-1/2}(1+\|g^{-1}h^{-1}\|)^{N^{\prime}}\max\\{1,\|hg\|^{N^{\prime}}\\}(1+|y+hx|)^{N^{\prime}}$
$\displaystyle\qquad\qquad\qquad\quad\cdot(1+|x|)^{-\beta_{1}}(1+\|g\|)^{-\beta_{2}}(1+\|g^{-1}\|)^{-\beta_{3}}\text{\,d}x\frac{\text{\,d}g}{|\det(g)|}$
$\displaystyle=$ $\displaystyle C_{2}I^{\prime}_{2}I^{\prime\prime}_{2},$

where $C_{2}$ is the product of the constant terms from part (b) of Lemma 3.1
and Proposition 4.8, and

$\displaystyle I^{\prime}_{2}$ $\displaystyle=$
$\displaystyle\int_{{\mathbb{R}}^{d}\setminus
h^{-1}B_{\epsilon/2}(0)}(1+|x|)^{-\beta_{1}}(1+|y+hx|)^{N^{\prime}}\text{\,d}x,$
$\displaystyle I^{\prime\prime}_{2}$ $\displaystyle=$
$\displaystyle\int_{H_{0}}|\det(hg)|^{-1/2}(1+\|g^{-1}h^{-1}\|)^{N^{\prime}}\max\\{1,\|hg\|^{N^{\prime}}\\}(1+\|g\|)^{-\beta_{2}}(1+\|g^{-1}\|)^{-\beta_{3}}\frac{\text{\,d}g}{|\det(g)|}.$

Note that for every $z\in{\mathbb{R}}^{d}$, we have

$(1+|z|)^{N^{\prime}}=(1+|h(h^{-1}z)|)^{N^{\prime}}\leq(1+\|h\||h^{-1}z|)^{N^{\prime}}\leq\max\\{1,\|h\|\\}^{N^{\prime}}(1+|h^{-1}z|)^{N^{\prime}},$

and since $\epsilon\leq|x_{0}|$, for every $x\in{\mathbb{R}}^{d}$

$(1+|y+x|)^{N^{\prime}}\leq(\frac{3}{2}|x_{0}|+1)^{N^{\prime}}(1+|x|)^{N^{\prime}}\
\mbox{ for }\ y\in B_{\epsilon/2}(x_{0}).$

So, letting $\omega_{d-1}$ denote the surface area of the $(d-1)$-sphere of
radius 1, we get

$\displaystyle I^{\prime}_{2}$ $\displaystyle=$
$\displaystyle\int_{{\mathbb{R}}^{d}\setminus
h^{-1}B_{\epsilon/2}(0)}(1+|x|)^{-\beta_{1}}(1+|y+hx|)^{N^{\prime}}\text{\,d}x$
$\displaystyle=$ $\displaystyle|\det(h)|^{-1}\int_{{\mathbb{R}}^{d}\setminus
B_{\epsilon/2}(0)}(1+|h^{-1}x|)^{-\beta_{1}}(1+|y+x|)^{N^{\prime}}\text{\,d}x$
$\displaystyle\leq$
$\displaystyle(\frac{3}{2}|x_{0}|+1)^{N^{\prime}}|\det(h)|^{-1}\int_{{\mathbb{R}}^{d}\setminus
B_{\epsilon/2}(0)}(1+|h^{-1}x|)^{-\beta_{1}}(1+|x|)^{N^{\prime}}\text{\,d}x$
$\displaystyle\leq$
$\displaystyle(\frac{3}{2}|x_{0}|+1)^{N^{\prime}}|\det(h)|^{-1}\max\\{1,\|h\|\\}^{N^{\prime}}\int_{{\mathbb{R}}^{d}\setminus
B_{\epsilon/2}(0)}(1+|h^{-1}x|)^{-\beta_{1}+N^{\prime}}\text{\,d}x$
$\displaystyle\leq$
$\displaystyle(\frac{3}{2}|x_{0}|+1)^{N^{\prime}}\max\\{1,\|h\|\\}^{N^{\prime}}\int_{{\mathbb{R}}^{d}\setminus
h^{-1}B_{\epsilon/2}(0)}(1+|x|)^{-\beta_{1}+N^{\prime}}\text{\,d}x$
$\displaystyle\leq$
$\displaystyle(\frac{3}{2}|x_{0}|+1)^{N^{\prime}}\max\\{1,\|h\|\\}^{N^{\prime}}\int_{{\mathbb{R}}^{d}\setminus
B_{\frac{\epsilon}{2\|h\|}}(0)}\frac{1}{(1+|x|)^{\beta_{1}-N^{\prime}}}\text{\,d}x$
$\displaystyle=$
$\displaystyle\omega_{d-1}(\frac{3}{2}|x_{0}|+1)^{N^{\prime}}\max\\{1,\|h\|\\}^{N^{\prime}}\int_{\frac{\epsilon}{2\|h\|}}^{\infty}\frac{r^{d-1}}{(1+r)^{\beta_{1}-N^{\prime}}}\text{\,d}r$
$\displaystyle\leq$
$\displaystyle\frac{\omega_{d-1}(\frac{3}{2}|x_{0}|+1)^{N^{\prime}}\max\\{1,\|h\|\\}^{N^{\prime}}}{{\beta_{1}-N^{\prime}-d}}(\frac{\epsilon}{2})^{d+N^{\prime}-\beta_{1}}\|h\|^{\beta_{1}-N^{\prime}-d}$
$\displaystyle\leq$ $\displaystyle
C_{6}\max\\{\|h\|^{\beta_{1}-N^{\prime}-d},\|h\|^{\beta_{1}-d}\\},$

where we used the fact that $B_{\frac{\epsilon}{2\|h\|}}(0)\subseteq
h^{-1}B_{\epsilon/2}(0)$, and $\beta_{1}-N^{\prime}>d$. Here, $C_{6}$ is a
constant that only depends on $x_{0},\beta_{1},N(u)$, and $d$.

Finally, using the submultiplicativity of $h\mapsto(1+\|h\|)^{N^{\prime}}$, we
have

$\displaystyle I^{\prime\prime}_{2}$ $\displaystyle=$
$\displaystyle\int_{{H_{0}}}|\det(hg)|^{-1/2}(1+\|g^{-1}h^{-1}\|)^{N^{\prime}}\max\\{1,\|hg\|^{N^{\prime}}\\}(1+\|g\|)^{-\beta_{2}}(1+\|g^{-1}\|)^{-\beta_{3}}\frac{\text{\,d}g}{|\det(g)|}$
$\displaystyle\leq$
$\displaystyle\frac{{(1+\|h\|)^{N^{\prime}}}(1+\|h^{-1}\|)^{N^{\prime}}}{|\det(h)|^{1/2}}\int_{{H_{0}}}|\det(g)|^{-1/2}(1+\|g^{-1}\|)^{N^{\prime}-\beta_{3}}(1+\|g\|)^{N^{\prime}-\beta_{2}}\frac{\text{\,d}g}{|\det(g)|}$
$\displaystyle\leq$
$\displaystyle{C_{7}}\|h\|^{-\alpha_{1}(N^{\prime}+d/2)}\||\det(\cdot)|^{{-3/2}}\Theta_{\beta_{2}-N^{\prime},\beta_{3}-N^{\prime}}\|_{{L}^{1}(H)},$

where in the last inequality, we used parts (b) and (c) of Lemma 2.6. Putting
all these together, we get

$\displaystyle|I_{2}|$ $\displaystyle\leq$ $\displaystyle
D_{2}\max\\{\|h\|^{\beta_{1}-N^{\prime}-d-\alpha_{1}(N^{\prime}+d/2)},\|h\|^{\beta_{1}-d-\alpha_{1}(N^{\prime}+d/2)}\\}$

where
$D_{2}=C_{2}C_{6}C_{7}\||\det(\cdot)|^{-3/2}\Theta_{\beta_{2}-N^{\prime},\beta_{3}-N^{\prime}}\|_{{L}^{1}(H)}$,
which is finite thanks to condition (iii). Using condition (i) on $\beta_{1}$
and the fact that $\|h\|$ is bounded on $K_{o}(W,V,R)$, we obtain $|I_{2}|\leq
D_{3}\|h\|^{N}$, where $D_{3}$ is clearly a uniform bound independent of the
choice of $h\in K_{o}(W,V,R)$ and $x\in B_{\epsilon/2}(0)$. ∎

######  Remark 5.5.

With hindsight on the proof of Theorem 5.4, we can provide some more intuition
regarding the overlap control condition. The initial idea of the proof is to
split the integration domain in the convolution formula

$W_{\psi_{2}}u=\frac{1}{C_{\psi_{1}}}({W}_{\psi_{1}}u*{W}_{\psi_{2}}\psi_{1})~{},$

into the part where the decay assumptions on ${W}_{\psi_{1}}u$ apply, and its
complement. On the latter, we can only combine moderate growth of
${W}_{\psi_{1}}u$ with rapid decay of ${W}_{\psi_{2}}\psi_{1}$ to achieve the
desired estimates. This rationale motivated the introduction of the auxiliary
functions $\Theta_{s,t}$ and the various related results. The overlap control
condition is intended to provide the right type of estimate that makes the
general approach work.

However, in the presence of a half-space adapted subgroup $H_{0}$ (e.g., in
the shearlet case), rapid decay of ${W}_{\psi_{2}}\psi_{1}$ will not be
enough, as long as we integrate over the whole group. To see this, recall that
$\mathcal{O}=(H_{0}^{T}\xi_{0})\cup-(H_{0}^{T}\xi_{0})$, and for simplicity
consider a direction $\xi\in H_{0}^{T}\xi_{0}$ in the proof of Theorem 5.14.
Assuming further that $V,W\subset H_{0}^{T}\xi_{0}$, it follows that
$K_{i/o}(W,V,R)\subset H_{0}$. As a consequence, it turns out that the part of
the integration domain where we solely rely on the decay estimates for
$W_{\psi_{2}}\psi_{1}$ to estimate $W_{\psi_{2}}u(x,h)$ contains the full
coset $H\setminus H_{0}$, independent of the choice of $h\in K_{o}(W,V,R)$. As
a consequence, the contribution of the integral over the coset is a constant.

Now it is important to remember that the right-hand side of the desired
inequality is a positive power of $h$, which becomes arbitrarily small. Hence
any nontrivial contribution from the coset $H\setminus H_{0}$ is essentially
fatal to our efforts. In particular, an analog of Definition 5.1 with the
whole group $H$ replacing $H_{0}$ would not make sense. This is why we replace
integration over $H$ by integration over $H_{0}$, as in our formulation of the
overlap control condition.

The only apparent alternative to this choice appears to make assumptions
guaranteeing that either the wavelets or the signal have Fourier transforms
which vanish on a half-space. However, these assumptions are incompatible with
compact support of the involved functions.

Note that the conclusion of the theorem allows to derive a decay statement for
_outer_ cone-affiliated sets of the type $K_{o}(W,V,R)$ from decay assumptions
for _inner_ cone-affiliated sets of the type $K_{i}(W^{\prime},V,R^{\prime})$.
Thus, as stated in the following corollary, Theorem 5.4 allows to close one of
the two gaps mentioned in Remark 3.3.

######  Corollary 5.6.

Let $u\in{\mathcal{S}}^{\prime}({\mathbb{R}}^{d})$ and
$(x,\xi)\in{\mathbb{R}}^{d}\times({\mathcal{O}}\cap S^{d-1})$. Assume that the
dual action is $V$-microlocally admissible in direction $\xi$ with parameters
$\alpha_{1},\alpha_{2}$. Also assume that $H$ fulfills the $V$-overlap control
condition, and there exists $\ell_{1}>0$ satisfying (18) of Proposition 4.8.
Then, we have

  1. (a)

If $(x,\xi)$ is an $N$-regular directed point of $u$, then there exists a
neighborhood $U$ of $x$, some $R>0$ and a $\xi$-neighborhood $W\subset
S^{d-1}$ such that for _all_ real-valued admissible
$\psi\in\mathcal{S}(\mathbb{R}^{d})$ with ${\rm supp}(\widehat{\psi})\subset
V$, the following estimate holds:

$\exists\,C>0\,\forall\,y\in U\,\forall\,h\in
K_{o}(W,V,R)~{}:~{}|W_{\psi}u(y,h)|\leq C\cdot\|h\|^{N-\alpha_{1}d/2}\\!\\!.$

  2. (b)

Let $\psi\in\mathcal{S}(\mathbb{R}^{d})$ be admissible with ${\rm
supp}(\widehat{\psi})\subset V$. Assume there exists a neighborhood $U$ of
$x$, some $R>0$ and a $\xi$-neighborhood $W\subset S^{d-1}$ such that

$\exists\,C>0\,\forall\,y\in U\,\forall\,h\in
K_{o}(W,V,R)~{}:~{}|W_{\psi}u(y,h)|\leq
C\cdot\left\|h\right\|^{\alpha_{1}N+\frac{3}{2}\alpha_{1}d+\alpha_{2}}$

Then $(x,\xi)$ is an $N$-regular directed point of $u$.

###### Proof.

Since $\psi$ is bandlimited, it satisfies vanishing moments condition for any
$r$. So, conditions (i), (ii), (iii) and (v) of Theorem 5.4 are automatically
satisfied for the case $\psi_{1}=\psi_{2}=\psi$. Combining Theorem 5.4 and
Theorem 3.4 finishes the proof of (a). Part (b) follows directly from Theorem
3.4. ∎

######  Remark 5.7.

Another condition that allows to close the gap between inner and outer cone
conditions is provided by the cone approximation property, which guarantees
inclusions of the type $K_{o}(W,V,R)\subset K_{i}(W^{\prime},V,R^{\prime})$,
for suitable choices of $W,R$ depending on $W^{\prime},R^{\prime}$.

It is currently not clear how the cone approximation property is related to
the assumptions underlying Theorem 5.8 and its corollary. We note that both
conditions are fulfilled for our chief class of examples, the shearlet
dilation groups covered in Section 6.

We now present a near characterization of unsigned regular directed points in
the case where the wavelet is real-valued and has sufficiently many vanishing
moments. Note that there exist compactly supported wavelets fulfilling these
conditions. Observe also that this theorem uses the same cone-related subsets
for both implications.

######  Theorem 5.8.

Fix $(x,\xi)\in\mathbb{R}^{d}\times(\mathcal{O}\cap S^{d-1})$. Let $V$ be a
relatively compact subset of $\mathcal{O}$, and assume that the dual action is
$V$-microlocally admissible in direction $\xi$, with associated parameters
$\alpha_{1},\alpha_{2}>0$. Assume that $H$ fulfills the $V$-overlap condition,
as well as inequality (18). Then there exist constants $s_{0},s_{1},s_{2}\geq
0$, depending only on $H$ and $V$, such that the following holds:

Let $u\in\mathcal{S}^{\prime}(\mathbb{R}^{d})$ denote a tempered distribution
of order $N(u)$. Let $\psi\in\mathcal{S}(\mathbb{R}^{d})$ denote a wavelet
with vanishing moments in $\mathcal{O}^{c}$ of order

$r>s_{0}+s_{1}N+s_{2}N(u)~{}.$

  1. (a)

If $(x,\pm\xi)$ is an unsigned $N$-regular directed point of $u$, then there
exists a neighborhood $U$ of $x$, some $R>0$ and a $\xi$-neighborhood
$W\subset S^{d-1}$ such that the following estimate holds:

$\exists\,C>0\,\forall\,y\in U\,\forall\,h\in K_{o}(W,V,R)\cup
K_{o}(-W,V,R)~{}:~{}|{W}_{\psi}u(y,h)|\leq
C\cdot\|h\|^{N-\alpha_{1}d/2}\\!\\!.$

  2. (b)

Let the wavelet $\psi$ be real-valued. Assume that $U$ is a neighborhood of
$x$, and that there are $R>0$ and a $\xi$-neighborhood $W\subset S^{d-1}$ such
that

$\exists\,C>0\,\forall\,y\in U\,\forall\,h\in K_{o}(W,V,R)\cup
K_{o}(-W,V,R)~{}:~{}|{W}_{\psi}u(y,h)|\leq
C\cdot\left\|h\right\|^{\alpha_{1}N+\frac{3}{2}\alpha_{1}d+\alpha_{2}}$

Then $(x,\pm\xi)$ is an unsigned $N$-regular directed point of $u$.

###### Proof.

Clearly, the lower bound imposed on $r$ via assumptions (i)-(v) of Theorem 5.4
can be guaranteed to hold by a lower estimate of the type

$r>s_{0}+s_{1}N+s_{2}N(u)~{},$

with suitably chosen $s_{0},s_{1},s_{2}$ depending only on $H$ and $V$.

For the proof of (a), pick a wavelet $\eta\in\mathcal{S}(\mathbb{R}^{d})$ that
fulfills ${\rm supp}(\widehat{\eta})\subset V_{0}\subset V$, w.l.o.g.
$V_{0}\cap-V_{0}=\varnothing$. The latter assumption guarantees that ${\rm
Re}(\eta),{\rm Im}(\eta)$ are both nonzero, with Fourier transforms supported
in $V_{0}\cup-V_{0}\subset\mathcal{O}$. Hence they are admissible wavelets as
well, with all moments in $\mathcal{O}^{c}$ vanishing. By Lemma 4.2,
$(x,\pm\xi)$ is an unsigned $N$-regular directed point of both ${\rm Re}(u)$
and ${\rm Im}(u)$. By Theorem 3.4, there exist a $\xi$-neighborhood
$W^{\prime}$ in $\mathcal{O}\cap S^{d-1}$, a neighborhood $U$ of $x$, and
$R^{\prime}>0$ such that

$|{W}_{\eta}({\rm Re}u)(y,h)|\leq
C\cdot\|h\|^{N-\alpha_{1}d/2}~{},~{}|{W}_{\eta}({\rm Im}u)(y,h)|\leq
C\cdot\|h\|^{N-\alpha_{1}d/2}$ (28)

holds for $h\in K_{i}(W^{\prime},V,R^{\prime})\cup
K_{i}(-W^{\prime},V,R^{\prime})$ and $y\in U$. Here, we used the fact that the
dual action is $V$-microlocal admissible in any direction, in particular in
the direction of $-\xi$ (see Remark 2.5). Writing

$\eta_{1}={\rm Re}(\eta),\eta_{2}={\rm Im}(\eta),$

we have that $W_{\eta}{\rm Re}(u)=W_{\eta_{1}}{\rm Re}(u)-iW_{\eta_{2}}{\rm
Re}(u)$, with $W_{\eta_{1}}{\rm Re}(u)$ and $W_{\eta_{2}}{\rm Re}(u)$ real-
valued. Hence the analog of (28) also holds for $W_{\eta_{i}}{\rm Re}(u)$, for
$i=1,2$.

Now Theorem 5.4, applied to $\eta_{1}$, allows to choose $W,R$ depending on
$W^{\prime},R^{\prime}$ in such a way that the same type of wavelet
coefficient decay holds for ${W}_{\psi}{\rm Re}(u)$ inside $K_{o}(W,V,R)\cup
K_{o}(-W,V,R)$. The analogous reasoning can be applied to ${W}_{\psi}{\rm
Im}(u)$, possibly at the expense of further restricting $W,R$. But then
$u={\rm Re}(u)+i{\rm Im}(u)$ implies the same decay for ${W}_{\psi}u$.

For the proof of part (b) we pick a wavelet
$\eta\in\mathcal{S}(\mathbb{R}^{d})$ that fulfills ${\rm
supp}(\widehat{\eta})\subset V\subseteq\mathcal{O}$. Then $\eta$ has vanishing
moments of all orders on $\mathcal{O}$. Since $\psi$ is real-valued, we can
use Theorem 5.4 to transfer this decay to ${W}_{\eta}(u)$ for all $h\in
K_{o}(W,V,R)\cup K_{o}(-W,V,R)$. Thus, Theorem 3.4 provides that both
$(x,\xi)$ and $(x,-\xi)$ are $N$-regular directed points of $u$. ∎

Note that the slight detour via ${\rm Re}(u)$ and ${\rm Im}(u)$ is
necessitated by the fact that for complex-valued wavelet $\eta$ and
distribution $u$, there is no simple estimate of $|W_{{\rm Re}(\psi)}u|$,
$|W_{{\rm Im}(\psi)}u|$ against $|W_{\psi}u|$ available, which would allow the
transfer of decay statements from one side to the other. The same observation
applies to $|W_{\psi}({\rm Re}(u))|$ and $|W_{\psi}({\rm Im}(u))|$.

As an easy application of the theorem, we note an adaptation of the main
result from [5], from bandlimited Schwartz wavelets to real Schwartz wavelets
with vanishing moments of arbitrary order.

######  Corollary 5.9.

Fix $(x,\xi)\in\mathbb{R}^{d}\times(\mathcal{O}\cap S^{d-1})$. Let $V$ be a
relatively compact subset of $\mathcal{O}$, and assume that the dual action is
$V$-microlocally admissible in direction $\xi$. Further assume that $H$
fulfills the $V$-overlap condition, as well as inequality (18). Let
$u\in\mathcal{S}^{\prime}(\mathbb{R}^{d})$, and let
$\psi\in\mathcal{S}(\mathbb{R}^{d})$ denote a real-valued wavelet with
vanishing moments in $\mathcal{O}^{c}$ of arbitrary order. Then the following
are equivalent:

  1. (a)

$(x,\pm\xi)$ is an unsigned regular directed point of $u$.

  2. (b)

There exists a neighborhood $U$ of $x$, some $R>0$ and a $\xi$-neighborhood
$W\subset S^{d-1}$ such that the following holds:

$\forall N>0\exists\,C>0\,\forall\,y\in U\,\forall\,h\in K_{o}(W,V,R)\cup
K_{o}(-W,V,R)~{}:~{}|{W}_{\psi}u(y,h)|\leq C\cdot\|h\|^{N}\\!\\!.$

##  6\. Shearlet dilation groups

Given that the lists of (partly implicit) assumptions of the Theorems 3.4 and
5.8 are somewhat long and technical, it is legitimate to ask whether there are
examples fulfilling all of them. It is the main purpose of this section to
demonstrate that there exists a fairly large example class for which all the
assumptions can be verified, with reasonably explicit expressions for the
involved constants.

We let $T(d,\mathbb{R})$ denote the matrix group of upper triangular matrices
of size $d$ with ones on the diagonal. Elements of $T(d,\mathbb{R})$ are
called unipotent. $\mathfrak{t}(d,\mathbb{R})$ refers to the Lie algebra of
$T(d,\mathbb{R})$, which consists of the upper triangular matrices with zero
diagonal.

######  Definition 6.1.

Let $H<{\rm GL}(d,\mathbb{R})$ denote an irreducibly admissible dilation
group. $H$ is called a generalized shearlet dilation group, if there exist two
closed subgroups $S,D<H$ with the following properties:

  1. (i)

$S$ is a connected abelian Lie subgroup of $T(d,\mathbb{R})$;

  2. (ii)

$D=\\{\exp(rY):r\in\mathbb{R}\\}$ is a one-parameter group, where $Y$ is a
diagonal matrix;

  3. (iii)

Every $h\in H$ can be written uniquely as $h=\pm ds$, with $d\in D$ and $s\in
S$.

$S$ is called the shearing subgroup of $H$, and $D$ is called the scaling
subgroup of $H$.

######  Remark 6.2.

The two-dimensional shearlet groups are given by

$H_{c}=\left\\{\pm\begin{pmatrix}a&t\\\
0&a^{c}\end{pmatrix}:a>0,t\in\mathbb{R}\right\\}<{\rm GL}(2,\mathbb{R})~{}.$

Here $c\in\mathbb{R}$ can be chosen arbitrarily.

######  Remark 6.3.

For dimension $d\geq 3$, emphasis of research so far has been focused on
standard and Toeplitz shearlet groups, which we define below. We remark that
with growing dimension $d$, the number of possible choices distinct from these
two classes of groups grows considerably.

  1. (i)

For $\lambda=(\lambda_{2},\ldots,\lambda_{d})\in\mathbb{R}^{d-1}$, we define
the standard shearlet group in $d$ dimensions $H^{\lambda}$ as the set

$\displaystyle\left\\{\epsilon\,\mathrm{diag}\left(a,a^{\lambda_{2}},\ldots,a^{\lambda_{d}}\right)\begin{pmatrix}1&t_{1}&\ldots&t_{d-1}\\\
&1&0\ldots&0\\\ &&\ddots&0\\\ &&&1\end{pmatrix}|\begin{array}[]{l}a>0,\\\
t_{i}\in\mathbb{R},\\\ \epsilon\in\left\\{\pm 1\right\\}\end{array}\right\\}.$

  2. (ii)

For $\delta\in\mathbb{R}$, we define the Toeplitz shearlet group in $d$
dimensions $H^{\delta}$ as the set

$\displaystyle\left\\{\epsilon\,\mathrm{diag}\left(a,a^{1-\delta},\ldots,a^{1-(d-1)\delta}\right)\cdot
T(1,t_{1},\ldots,t_{d-1})|\begin{array}[]{l}a>0,\\\ t_{i}\in{\mathbb{R}},\\\
\epsilon\in\\{\pm 1\\}\end{array}\right\\},$

where the matrix $T(1,t_{1},\ldots,t_{d-1})$ is defined by

$\displaystyle
T(1,t_{1},\ldots,t_{d-1}):=\begin{pmatrix}1&t_{1}&t_{2}&\ldots&t_{d-2}&t_{d-1}\\\
&1&t_{1}&t_{2}&\ldots&t_{d-2}\\\ &&\ddots&\ddots&\ddots&\vdots\\\
&&&1&t_{1}&t_{2}\\\ &&&&1&t_{1}\\\ &&&&&1\end{pmatrix}.$

Below we will establish that the results of the previous sections, in
particular Theorems 3.4 and 5.8, are applicable to shearlet groups from a
large example class, including the above-listed families, provided that the
scaling subgroup fulfills an additional condition. Thankfully, the majority of
the necessary technical conditions have already been verified in previous
papers. The following lemma already guarantees the applicability of Theorem
3.4.

######  Lemma 6.4.

Let $H$ denote a shearlet dilation group with scaling subgroup
$D=\exp(\mathbb{R}Y)$, where w.l.o.g. $Y={\rm
diag}(1,\lambda_{2},\ldots,\lambda_{d})$. Let

$\lambda_{\max}=\max\\{\lambda_{i}:2\leq i\leq
d\\}~{},~{}\lambda_{\min}=\min\\{\lambda_{i}:2\leq i\leq d\\}~{}.$

Assume that $0<\lambda_{\min}\leq\lambda_{\max}<1$. Then

  1. (a)

There exists an open $\varnothing\neq V\Subset H$ such that $H$ fulfills the
$V$-cone approximation property; that is, for every $\xi\in{\mathcal{O}}\cap
S^{d-1}$ we have the following: for all $\xi-$neighborhoods $W\subseteq
S^{d-1}$ and all $R>0$, there exists a $\xi$-neighborhood $W^{\prime}\subseteq
S^{d-1}$ and $R^{\prime}>0$ satisfying
$K_{o}(W^{\prime},V,R^{\prime})\subseteq K_{i}(W,V,R).$

  2. (b)

$H$ is microlocally admissible, with

$\alpha_{1}=\frac{1}{\lambda_{\min}}~{},~{}\alpha_{2}=\frac{d}{\lambda_{\min}}+\epsilon~{},$
(29)

where $\epsilon>0$ is arbitrary.

###### Proof.

The cone approximation property is shown in Proposition 30 of [1], and
microlocal admissibility is stated in Proposition 32 of that paper. The
constants are not explicitly stated in this result, but they can be found in
the proof: Part (a) of Definition 2.4 is shown to hold with
$\alpha_{1}=\frac{1}{\lambda_{\min}}$. For the existence of $\alpha_{2}$, the
reader is referred to Lemma 4.7 of [5], which appeals to the existence of
$\alpha_{1}$ and the cone approximation property. For an actual formula for
$\alpha_{2}$, the reader should consult the proofs of [5, Lemma 2.8 and Lemma
4.7] to find the condition $\alpha_{2}>d\alpha_{1}$. ∎

We next establish the condition allowing the application of the decay
estimates from Proposition 4.8. Before we formulate the statement, we cite a
statement concerning the structure of the shearing subgroup and its Lie
algebra.

######  Lemma 6.5 ([1] Lemma 5. and Lemma 6.).

Let $S$ be the shearing subgroup of a generalized shearlet dilation group
$H\subset\mathrm{GL}({\mathbb{R}}^{d})$. Then the following statements hold:

  1. (a)

There exists a unique basis $X_{2},\ldots,X_{d}$ of the Lie algebra
$\mathfrak{s}$ of $S$ with $X_{i}^{T}e_{1}=e_{i}$ for $2\leq i\leq d$, called
the canonical basis of $\mathfrak{s}$.

  2. (b)

We have $S=\\{I_{d}+X|\,X\in\mathfrak{s}\\}$.

  3. (c)

Let $\mathfrak{s}_{k}={\rm span}\\{X_{j}:j\geq k\\}$, for $k\in 2,\ldots,d$.
These are associative matrix algebras satisfying
$\mathfrak{s}_{k}\mathfrak{s}_{\ell}\subset\mathfrak{s}_{\max{\\{k,\ell\\}}+1}$,
where we write $\mathfrak{s}_{m}=\\{0\\}$ for $m>d$.

  4. (d)

$H$ is the inner semidirect product of the normal subgroup $S$ with the closed
subgroup $D\cup-D$.

Note that since $\mathfrak{s}$ consists of strictly upper triangular matrices,
it is nilpotent. We let $n(\mathfrak{s})$ denote the nilpotency degree of
$\mathfrak{s}$, i.e.

$n(\mathfrak{s})=\min\\{k\in\mathbb{N}:X^{k}=0\ \forall
X\in\mathfrak{s}\\}~{}.$

As will be seen more clearly in our computations below, $n(\mathfrak{s})$
enters quite naturally in computations with inverse matrices in $H$.

######  Lemma 6.6.

Let $H$ denote a shearlet dilation group with scaling subgroup
$D=\exp(\mathbb{R}Y)$, where w.l.o.g. $Y={\rm
diag}(1,\lambda_{2},\ldots,\lambda_{d})$. Let $\mathfrak{s}$ denote the Lie
algebra of the shearing subgroup, and $n(\mathfrak{s})$ its nilpotency degree.
Assume that $\lambda_{\min},\lambda_{\max}$, defined in Lemma 6.4, fulfill
$0<\lambda_{\min}\leq\lambda_{\max}<1$. Then

$\|h^{\pm 1}\|A_{H}(h)^{\ell_{1}}\leq 1$

holds for all $h\in H$, with

$\ell_{1}=n(\mathfrak{s})+1~{}.$ (30)

###### Proof.

We refer to [10], Theorem 4.10. Note that the cited theorem makes no
assumptions on $Y$, and gives the more general formula

$\ell_{1}=n(\mathfrak{s})-1+2\|Y\|~{}.$

Here we restrict attention to the case $0<\lambda_{\min}\leq\lambda_{\max}<1$,
which entails $\|Y\|=1$. ∎

Thus the only remaining obstacle to the applicability of Theorem 5.8 is the
overlap control condition from Definition 5.1. Unfortunately, this condition
requires somewhat lengthy computations, which we now prepare. Recall that we
need to estimate the integral

$\int_{H_{0}\setminus h^{-1}K_{i}(W,V,R)}\Theta_{s,t}(g)\text{\,d}g~{}.$

We will do this using fairly explicit computations in particular coordinates
for $H$. With respect to these coordinates, the following challenges present
themselves:

  1. (1)

Estimate norms of products and inverses in $H$;

  2. (2)

explicitly determine the integration domain $H_{0}\setminus
h^{-1}K_{i}(W,V,R)$ for certain $h$ in $H$;

  3. (3)

determine the Haar measure $\text{\,d}g$.

We start by introducing the coordinates we will use: Using the canonical basis
$X_{2},\ldots,X_{d}$ of $\mathfrak{s}$, we define, for $t\in\mathbb{R}^{d-1}$
and $a\not=0$,

$h(t,a)=\left(I_{d}+\sum_{i=2}^{d}t_{i}X_{i}\right)^{-1}{\rm sgn}(a){\rm
diag}(|a|,|a|^{\lambda_{2}},\ldots,|a|^{\lambda_{d}})\in H~{}.$

Here $I_{d}={\rm diag}(1,\ldots,1)$ denotes the $d\times d$ identity matrix.
It is easy to see that $(t,a)\ni\mathbb{R}^{d-1}\times\mathbb{R}^{\times}\to
H$ is a bijection, and that restricting this mapping to
$\mathbb{R}^{d-1}\times\\{1\\}$ yields a bijection onto $S$. The inversion of
the first factor is the cause of some awkwardness later on, but it will allow
a fairly simple description of cone-affiliated subset $K_{i/o}\subset H$.

Note that we can write

$I_{d}+\sum_{i=2}^{d}t_{i}X_{i}^{T}=\begin{pmatrix}1&0^{T}\\\
t&I_{d-1}+A(t)^{T}\end{pmatrix},$ (31)

with $A(t)$ being a $(d-1)\times(d-1)$ strictly upper-triangular matrix
satisfying

$\lVert A(t)\rVert\leq C|t|$ (32)

with a constant $C$ depending only on $H$.

We next describe group operations with respect to these coordinates:

######  Lemma 6.7.

Let $H$ denote a shearlet dilation group, with canonical basis
$X_{2},\ldots,X_{d}$ for $\mathfrak{s}$.

  1. (a)

There exists a bilinear map
$M:\mathbb{R}^{d-1}\times\mathbb{R}^{d-1}\to\mathbb{R}^{d-1}$ such that
$h(t,1)^{-1}h(t^{\prime},1)^{-1}=h(t+t^{\prime}+M(t,t^{\prime}),1)^{-1}$. For
all $i=2,\ldots,d$, the $i$th entry $M(t,t^{\prime})_{i}$ only depends on the
entries $t_{2},\ldots,t_{i-1},t_{2}^{\prime},\ldots,t_{i-1}^{\prime}$.

  2. (b)

There exists a map $B:\mathbb{R}^{d-1}\to\mathbb{R}^{d-1}$, polynomial of
degree $n(\mathfrak{s})-1$, such that for all $t\in\mathbb{R}^{d-1}$,

$h(t,1)^{-1}=h(-t+B(t),1)~{}.$

Here $B(t)_{i}$ only depends on the entries $t_{2},\ldots,t_{i-1}$.

  3. (c)

For all $a>0$ and $t=(t_{2},\ldots,t_{d})^{T}\in\mathbb{R}^{d-1}$, we have

$h(0,a)h(t,1)h(0,a)^{-1}=h((a^{1-\lambda_{2}}t_{2},\ldots,a^{1-\lambda_{d}}t_{d}),1)$

###### Proof.

For part (a), we have by definition

$\displaystyle h(t,1)^{-1}h(t^{\prime},1)^{-1}$ $\displaystyle=$
$\displaystyle\left(I_{d}+\sum_{i=2}^{d}t_{i}X_{i}\right)\left(I_{d}+\sum_{i=2}^{d}t_{i}^{\prime}X_{i}\right)$
$\displaystyle=$ $\displaystyle
I_{d}+\sum_{i=2}^{d}(t_{i}+t_{i}^{\prime})X_{i}+\sum_{i,j=2}^{d}t_{i}t_{j}^{\prime}X_{i}X_{j}$
$\displaystyle=$ $\displaystyle
I_{d}+\sum_{i=2}^{d}(t_{i}+t_{i}^{\prime})X_{i}+\sum_{i}M(t,t^{\prime})_{i}X_{i}~{},$

where $M(t,t^{\prime})_{i}$ denotes the coefficient of the $X_{i}$-coordinate
of $\sum_{\ell,k=2}^{d}t_{k}t_{\ell}^{\prime}X_{k}X_{\ell}$. Clearly this
coordinate depends bilinearly on the $t_{k},t_{\ell}^{\prime}$, and only on
those with $\ell,k<i$, by part (c) of Lemma 6.5.

The fact that $t\mapsto h(t,1)$ induces a bijection between $\mathbb{R}^{d-1}$
and $S$ entails the existence of a unique map $B^{\prime}$ with
$h(t,1)^{-1}=h(B^{\prime}(t),1)$. Furthermore, every unipotent matrix is
inverted by a finite Neumann series, and for elements $X\in\mathfrak{s}$, the
number of terms in the series is at most $n(\mathfrak{s})-1$. Now part (a) of
the current lemma implies that $B^{\prime}$ is a polynomial of degree $\leq
n(\mathfrak{s})-1$. Furthermore, part (c) of Lemma 6.5 implies that
$B^{\prime}(t)=-t+B(t)$, and $B$ has the described properties.

For part (c) we first note that by definition of the canonical basis, the
vector $t\in\mathbb{R}^{d-1}$ such that $h=h(t,1)^{-1}$ is just given by the
first row of $h$. Computing the conjugation action of the diagonal matrix on
the row entries then gives the equation

$h(0,a)h(t,1)^{-1}h(0,a)^{-1}=h((a^{1-\lambda_{2}}t_{2},\ldots,a^{1-\lambda_{d}}t_{d}),1)^{-1}~{}.$

Inverting both sides of the equation yields (c). ∎

######  Lemma 6.8.

In the $(t,a)$-coordinates, left Haar measure on $H$ is determined as

$\int_{H}f(h)\text{\,d}h=\int_{\mathbb{R}^{\times}}\int_{\mathbb{R}^{d-1}}f(h(t,a))\text{\,d}t\frac{\text{\,d}a}{|a|^{d-{\rm
tr}(Y)}}~{},$

where $\text{\,d}t$ is the Lebesgue measure of $\mathbb{R}^{d-1}$ and
$\text{\,d}a$ is the Haar measure of the multiplicative group
$\mathbb{R}^{\times}$.

###### Proof.

It is easy to see from part (a) of the previous lemma that Haar integration on
$S$ is given by

$\int_{\mathbb{R}^{d-1}}f(h(t,1)^{-1})\text{\,d}t~{}.$

Since $S$ is an abelian group, this implies that

$\int_{\mathbb{R}^{d-1}}f(h(t,1))\text{\,d}t$

yields the same Haar integral. Furthermore, one can use part (c) of the
previous lemma to determine the impact of the conjugation action of $D$ on
Haar measure of $S$. Now a standard formula for the Haar integral of
semidirect products, i.e. [14, 15.29], yields the desired result. ∎

We next provide approximate descriptions of the cone-affiliated sets with
respect to the coordinates that we just introduced. In order to do this, we
first parameterize the open orbit $\mathcal{O}$ by the global chart provided
by affine coordinates

$\Omega:\mathbb{R}^{\times}\times\mathbb{R}^{d-1}\to\mathcal{O}\qquad\Omega(\tau,v)=\tau(1,v^{T})^{T},$

and consider the corresponding diffeomorphism to its image

$\omega:\mathbb{R}^{d-1}\to
S^{d-1}\cap\mathcal{O},\qquad\omega(v)=\frac{(1,v^{T})^{T}}{\sqrt{1+|v|^{2}}}.$

Given $\epsilon>0$, we set

$W_{\epsilon}=\\{v\in\mathbb{R}^{d-1}:|v|<\epsilon\\}=B_{\epsilon}(0).$

Clearly, $\\{W_{\epsilon}:\epsilon>0\\}$ is a neighbourhood basis of the
origin in $\mathbb{R}^{d-1}$, and so $\\{\omega(W_{\epsilon}):\epsilon>0\\}$
is a neighbourhood basis of $\xi_{0}=(1,0,\ldots,0)\in
S^{d-1}\cap\mathcal{O}$. Furthermore, for fixed $0<\tau_{1}<\tau_{2}$ and
$\epsilon_{0}>0$ the set

$V=\Omega(\,(\tau_{1},\tau_{2})\times W_{\epsilon_{0}}\,)$

is an open subset with $V\Subset\mathcal{O}$.

######  Lemma 6.9.

Assume that $\lambda_{\min},\lambda_{\max}$, defined in Lemma 6.4, fulfill
$0<\lambda_{\min}\leq\lambda_{\max}<1$. Let
$V=\Omega(\,(\tau_{1},\tau_{2})\times W_{\epsilon_{0}}\,)$, $R>1$, where
$0<\tau_{1}<1<\tau_{2}$ and $W=\omega(W_{\epsilon})$, with $\epsilon<1$.
Suppose $R$ is sufficiently large, meaning,

$R>2\tau_{2}\max\left\\{1,(\frac{2\epsilon_{0}}{\epsilon})^{\frac{1}{1-\lambda_{\max}}},(2C\epsilon_{0})^{\frac{1}{1-\lambda_{\max}}}\right\\},$
(33)

with $C$ from (32). Then, the following inclusions hold:

$\left\\{h(t,a):|t|<\delta_{0},0<a<a_{0}\right\\}\subseteq
K_{i}(W,V,R)\subseteq
K_{o}(W,V,R)\subseteq\left\\{h(t,a):|t|<\delta_{1},0<a<a_{1}\right\\}~{},$

where $a_{0},a_{1},\delta_{0},\delta_{1}$ are given by

$a_{0}=\frac{\tau_{1}}{R}~{},~{}\delta_{0}=\frac{\epsilon}{3}$

and

$a_{1}=\frac{2\tau_{2}}{R}~{},~{}\delta_{1}=3\epsilon~{}.$

###### Proof.

Our argument is inspired by the proof of Proposition 30 in [1], so we will not
expand on its details here.

Given the definition of $V$, we have

$(h^{-1})^{T}(V)=\Omega\left((a^{-1}\tau_{1},a^{-1}\tau_{2})\times(t+(I_{d-1}+A(t)^{T})W_{\epsilon_{0}}^{a})\right),$
(34)

where $W_{\epsilon_{0}}^{a}:={\rm
diag}(a^{1-\lambda_{2}},\ldots,a^{1-\lambda_{d}})(W_{\epsilon_{0}})$. It is
also easy to see that for every $0<R$ and $0<\epsilon\leq 1$, we have

$(R,\infty)\times
W_{\epsilon}\subseteq\Omega^{-1}\left(C(\omega(W_{\epsilon}),R)\right)\subseteq(\frac{R}{2},\infty)\times
W_{\epsilon}.$ (35)

By (34) and (35), if $h=h(t,a)$ belongs to $K_{o}(W,V,R)$, then we must have

(i) $\displaystyle 0<a<\frac{2\tau_{2}}{R}$ (ii) $\displaystyle\exists\xi\in
B_{\epsilon_{0}}(0)\mbox{ s.t. }|t+(I_{d-1}+A(t)^{T}){\rm
diag}(a^{1-\lambda_{2}},\ldots,a^{1-\lambda_{d}})\xi|<\epsilon.$

Since $R>2\tau_{2}$, we have $a<1$, and consequently, we get

$|t|<\epsilon+(1+C|t|)a^{1-\lambda_{\max}}\epsilon_{0}\leq\epsilon+(1+C|t|)(\frac{2\tau_{2}}{R})^{1-\lambda_{\max}}\epsilon_{0}.$

The above inequality, together with
$R>\max\left\\{2\tau_{2}(\frac{2\epsilon_{0}}{\epsilon})^{\frac{1}{1-\lambda_{\max}}},2\tau_{2}(2C\epsilon_{0})^{\frac{1}{1-\lambda_{\max}}}\right\\}$,
gives $|t|<3\epsilon$.

For the other inclusion, note that by (35), $h\in K_{i}$ is guaranteed if
$\Omega^{-1}((h^{-1})^{T}(V))\subseteq(R,\infty)\times W_{\epsilon}$. For the
latter inclusion to hold, it is enough to have

(i) $\displaystyle 0<a<\frac{\tau_{1}}{R},$ (ii)
$\displaystyle|t|+(1+C|t|)a^{1-\lambda_{\max}}\epsilon_{0}<\epsilon.$

Combining (33) and $0<a<\frac{\tau_{1}}{R}$, one can verify that (ii) holds
whenever $|t|<\frac{\epsilon}{3}$. ∎

For estimates of the function $\Theta_{r,s}$, we need to estimate norms of
products of the type $h(t,a)^{-1}h(t^{\prime},a^{\prime})$. This is provided
by the next lemma:

######  Lemma 6.10.

Let $(t,a),(t^{\prime},a^{\prime})\in\mathbb{R}^{d-1}\times\mathbb{R}^{+}$.
Assume that $\lambda_{\min},\lambda_{\max}$, defined in Lemma 6.4, fulfill
$0<\lambda_{\min}\leq\lambda_{\max}<1$.

  1. (a)

Let $V=\Omega(\,(\tau_{1},\tau_{2})\times W_{\epsilon_{0}}\,)$, $R$ satisfying
(33), where $0<\tau_{1}<1<\tau_{2}$ and $W=\omega(W_{\epsilon})$, with
$\epsilon<1$. Then there exists a constant $C_{1}$ such that for all
$h=h(t,a)\in K_{o}(W,V,R)$, the following inequalities hold:

$a^{\lambda_{\min}}\leq\|h(t,a)\|\leq C_{1}a^{\lambda_{\min}}$

  2. (b)

$\|h(t,a)^{-1}h(t^{\prime},a^{\prime})\|\geq a^{-1}a^{\prime}$ .

  3. (c)

Assume that $a,a^{\prime}<1$ and $|t|<\delta_{1}$. There exist constants
$C_{\mathfrak{s}},C_{M}>0$ depending only on $H$ such that

$\|h(t,a)^{-1}h(t^{\prime},a^{\prime})\|\geq
a^{-1}(a^{\prime})^{\lambda_{\max}}\left(-\delta_{1}+(1-C_{M}\delta_{1})\left\\{\begin{array}[]{cc}\frac{|t^{\prime}|}{C_{\mathfrak{s}}}&|t^{\prime}|<C_{\mathfrak{s}}\\\
\frac{|t^{\prime}|^{1/(n-1)}}{C_{\mathfrak{s}}^{1/(n-1)}}&|t^{\prime}|\geq
C_{\mathfrak{s}}\end{array}~{}\right.\right)~{},$

where $n=n(\mathfrak{s})$.

###### Proof.

For the proof of $(a)$, we note that the diagonal entries of $h(t,a)$ are
$a,a^{\lambda_{2}},\ldots,a^{\lambda_{d}}$. Since $h\in K_{o}(W,V,R)$ and $R$
satisfies (33), by Lemma 6.9 we know $a<1$. So,

$\|h(t,a)\|\geq a^{\lambda_{\min}}~{}.$

For the upper bound, first note that by Lemma 6.9, we have $|t|<3\epsilon$.
Now, submultiplicativity of the norm yields

$\displaystyle\|h(t,a)\|$ $\displaystyle\leq$
$\displaystyle\left\|\left(I_{d}+\sum_{i=2}^{d}t_{i}X_{i}\right)^{-1}\right\|\left\|{\rm
diag}(a,a^{\lambda}_{2},\ldots,a^{\lambda_{d}})\right\|$ $\displaystyle\leq$
$\displaystyle C_{1}a^{\lambda_{\min}}~{},$

where $C_{1}$ is the maximum attained by the continuous map
$t\mapsto\left\|\left(I_{d}+\sum_{i=2}^{d}t_{i}X_{i}\right)^{-1}\right\|$ on
the compact set $\overline{B_{3\epsilon}(0)}$.

For part (b), we simply use that $a^{-1}a^{\prime}$ is the upper left entry of
$h(t,a)^{-1}h(t^{\prime},a^{\prime})$.

For part (c), we first compute

$\displaystyle h(t,1)^{-1}h(t^{\prime},1)$ $\displaystyle=$ $\displaystyle
h(t,1)^{-1}h(-t^{\prime}+B(t^{\prime}),1)^{-1}$ $\displaystyle=$
$\displaystyle
h(t-t^{\prime}+B(t^{\prime})+M(t,-t^{\prime}+B(t^{\prime})),1)^{-1}$
$\displaystyle=$ $\displaystyle
I_{d}+\sum_{i=2}^{d}t_{i}^{\prime\prime}X_{i}~{},$

with

$t^{\prime\prime}=t-t^{\prime}+B(t^{\prime})+M(t,-t^{\prime}+B(t^{\prime}))~{},$

where $B$ is the polynomial map from Lemma 6.7(b). Hence

$\displaystyle h(t,a)^{-1}h(t^{\prime},a^{\prime})$ $\displaystyle=$
$\displaystyle{\rm
diag}(a^{-1},a^{-\lambda_{2}},\ldots,a^{-\lambda_{d}})\left(I_{d}+\sum_{i=2}^{d}t_{i}^{\prime\prime}X_{i}\right){\rm
diag}(a^{\prime},(a^{\prime})^{\lambda_{2}},\ldots,(a^{\prime})^{\lambda_{d}})~{}.$

The norm of that product is greater or equal to the norm of its first row,
which is

$(a^{-1}a^{\prime},a^{-1}(a^{\prime})^{\lambda_{2}}t_{2}^{\prime\prime},\ldots,a^{-1}(a^{\prime})^{\lambda_{d}}t_{d}^{\prime\prime})~{},$

hence

$\left\|h(t,a)^{-1}h(t^{\prime},a^{\prime})\right\|\geq
a^{-1}(a^{\prime})^{\lambda_{\max}}|t^{\prime\prime}|~{}.$

For the estimate of $|t^{\prime\prime}|$, note that by Lemma 6.7(b),
$-t^{\prime}+B(t^{\prime})$ is a polynomial of order $n-1$. The polynomial has
zero constant term because $h(0,1)^{-1}=h(0,1)$, hence we have

$|-t^{\prime}+B(t^{\prime})|\leq\sum_{k=1}^{n-1}c_{k}|t^{\prime}|^{k}\leq
C_{\mathfrak{s}}\left\\{\begin{array}[]{cc}|t^{\prime}|&|t^{\prime}|<1\\\
|t^{\prime}|^{n-1}&|t^{\prime}|\geq 1\end{array},\right.$ (36)

with positive constants $c_{k},C_{\mathfrak{s}}$ depending on the generators
$X_{2},\ldots,X_{d}$ of the Lie algebra $\mathfrak{s}$. Since the inversion
map on $H$ is its own inverse, we have
$t^{\prime}=t^{\prime}-B(t^{\prime})+B(-t^{\prime}+B(t^{\prime}))$. This
implies, via (36), the estimate

$|t^{\prime}|\leq
C_{\mathfrak{s}}\left\\{\begin{array}[]{cc}|-t^{\prime}+B(t^{\prime})|&|-t^{\prime}+B(t^{\prime})|<1\\\
|-t^{\prime}+B(t^{\prime})|^{n-1}&|-t^{\prime}+B(t^{\prime})|\geq
1\end{array}~{}\right.~{}.$

Distinguishing the cases $|-t^{\prime}+B(t^{\prime})|\geq 1$ and
$|-t^{\prime}+B(t^{\prime})|<1$ allows to derive

$|-t^{\prime}+B(t^{\prime})|\geq\min\left\\{\frac{|t^{\prime}|^{1/(n-1)}}{C_{\mathfrak{s}}^{1/(n-1)}},\frac{|t^{\prime}|}{C_{\mathfrak{s}}}\right\\}~{},$

or equivalently,

$|-t^{\prime}+B(t^{\prime})|\geq\left\\{\begin{array}[]{cc}\frac{|t^{\prime}|}{C_{\mathfrak{s}}}&|t^{\prime}|<C_{\mathfrak{s}}\\\
\frac{|t^{\prime}|^{1/(n-1)}}{C_{\mathfrak{s}}^{1/(n-1)}}&|t^{\prime}|\geq
C_{\mathfrak{s}}\end{array}~{}\right.~{}.$

It follows, using $|t|<\delta_{1}$ and the constant $C_{M}$ bounding the
bilinear map $M$,

$|t^{\prime\prime}|\geq|-t^{\prime}+B(t^{\prime})|(1-C_{M}\delta_{1})-\delta_{1}\geq-\delta_{1}+(1-C_{M}\delta_{1})\left\\{\begin{array}[]{cc}\frac{|t^{\prime}|}{C_{\mathfrak{s}}}&|t^{\prime}|<C_{\mathfrak{s}}\\\
\frac{|t^{\prime}|^{1/(n-1)}}{C_{\mathfrak{s}}^{1/(n-1)}}&|t^{\prime}|\geq
C_{\mathfrak{s}}\end{array}~{}\right..$

∎

######  Lemma 6.11.

Let $H=DS\cup-DS$ denote a shearlet dilation group, with
$0<\lambda_{\min}\leq\lambda_{\max}<1$. Let $H_{0}=DS$. Then $H$ fulfills the
overlap control condition, with

$\gamma_{0}=2d+1~{},~{}\gamma_{1}=\frac{\lambda_{\min}}{1-\lambda_{\max}}~{},~{}\gamma_{2}=\frac{\lambda_{\min}\lambda_{\max}}{1-\lambda_{\max}}$

###### Proof.

By Lemma 5.2, it suffices to concentrate on the direction
$\xi_{0}=(1,0,\ldots,0)^{T}$. Let $V,W^{\prime},R^{\prime}$ be given as in
Lemma 6.9, i.e. $V=\Omega((\tau_{1},\tau_{2})\times W_{\epsilon_{0}})$,
$W^{\prime}=\omega(W_{\epsilon^{\prime}})$ with $\epsilon^{\prime}<1$, and
$R^{\prime}$ satisfying (33). In addition, assume that
$\epsilon^{\prime}<7C_{\mathfrak{s}}$, where $C_{\mathfrak{s}}$ denotes the
constant from Lemma 6.10(c). Let $W^{\prime\prime}\subset W^{\prime}$ and
$R^{\prime\prime}>R^{\prime}$ be arbitrary, and w.l.o.g. assume that
$W^{\prime\prime}=\omega(W_{\epsilon^{\prime\prime}})$ with
$\epsilon^{\prime\prime}\leq\epsilon^{\prime}$.

We pick $R=R^{\prime\prime}$ and $W=\omega(W_{\epsilon})$, where

$\epsilon<\min\left\\{\epsilon^{\prime\prime},\frac{1}{6C_{M}},\frac{\epsilon^{\prime\prime}}{21C_{\mathfrak{s}}}\right\\}~{}.$
(37)

We need to estimate, for $h\in K_{o}(W,V,R)$ the integral

$\int_{H_{0}\setminus
K_{i}(W^{\prime\prime},V,R^{\prime\prime})}\Theta_{r,s}(h^{-1}g)\text{\,d}g~{};$

this is the same integral as in the definition of the overlap control
condition, since $h\in K_{o}(W,V,R)\subseteq H_{0}$ and $\text{\,d}g$ is just
Haar integration. Using the parameterization $g=h(t^{\prime},a^{\prime})$, the
inclusion relations from Lemma 6.9 with