# Complements and Improvements Regarding Distributivity of the Product for
$\sigma$-Algebras with Respect to the Intersection

K.P.S. Bhaskara Rao1, Alexander Steinicke2

###### Abstract

We present a variety of refined conditions for $\sigma$ algebras $\mathcal{A}$
(on a set $X$), $\mathcal{F},\mathcal{G}$ (on a set $U$) such that the
distributivity equation

$(\mathcal{A}\otimes\mathcal{F})\cap(\mathcal{A}\otimes\mathcal{G})=\mathcal{A}\otimes\left(\mathcal{F}\cap\mathcal{G}\right),$

holds – or is violated.
The article generalizes the results in [16] and includes a positive result for
$\sigma$ algebras generated by at most countable partitions, was not covered
before. We also present a proof that counterexamples may be constructed
whenever $X$ is uncountable and there exist two $\sigma$-algebras on $X$ which
are both countably separated, but their intersection is not. We present
examples of such structures. In the last section, we extend [16, Theorem 2]
from analytic to the setting of Blackwell spaces.

Keywords: sigma algebra; intersection of sigma algebras; product sigma
algebras; counterexample for sigma algebras 11footnotetext: Department of
Computer Information Systems, Indiana University Northwest, Gary, IN, USA.
<EMAIL_ADDRESS>22footnotetext: Department of Mathematics and Information
Technology, Montanuniversitaet Leoben, Austria.
<EMAIL_ADDRESS>
##  1 Introduction

The question of distributivity of $\sigma$-algebras with respect to the
intersection has recently been studied in [16], motivated by questions from
stochastic analysis (therein and in [17]), but also coming from a question
about sequences of probability spaces in general stochastics [13,
15]333footnotetext: There was an erroneous statement in [16] regarding the
discussions in the references [13, 15]:
The assertion that $(\mu\otimes\nu)(K)=0$ for every
$K\in\bigcap{n\in\mathbb{N}}(\mathcal{A}n\otimes\mathcal{U})\setminus\biggl{(}\bigcap{n\in\mathbb{N}}\mathcal{A}n\biggr{)}\otimes\mathcal{U}$
has to be replaced with: For all
$K\in\bigcap{n\in\mathbb{N}}(\mathcal{A}n\otimes\mathcal{U})$ there is an
$L\in\biggl{(}\bigcap{n\in\mathbb{N}}\mathcal{A}n\biggr{)}\otimes\mathcal{U}$
such that $(\mu\otimes\nu)(L\triangle K)=0$.. While the problem can be
formulated quite simply using only well known basic constructions, its answer
still is nontrivial. The question, apart from the investigation in [16] has
not been addressed in various remarkable sources for results on
$\sigma$-algebras (sometimes referred to as Borel structures) exceeding
standard literature, such as Aumann [1], Basu [2], Bhaskara Rao and Bhaskara
Rao [3], Bhaskara Rao and Rao [4], Bhaskara Rao and Shortt [6], Blackwell [7],
Georgiou [8], Grzegorek [9], Rao [14] to mention an (incomplete) list of
contributions to the theory. A precise description of the problem is the
following.

Let $\mathcal{A}$ be a $\sigma$-algebra on a (nonempty) set $X$ and
$\mathcal{F},\mathcal{G}$ be two $\sigma$-algebras on a (nonempty) set $U$.
The product of the $\sigma$-algebras $\mathcal{C}$ on a set $Y$ and
$\mathcal{D}$ on a set $Z$ is denoted by $\mathcal{C}\otimes\mathcal{D}$ on
$Y\times Z$ and is defined as the smallest $\sigma$-algebra containing all
Cartesian products (or rectangles) $\\{C\times
D:C\in\mathcal{C},D\in\mathcal{D}\\}$. We ask, for which $\sigma$-algebras
$\mathcal{A},\mathcal{F},\mathcal{G}$ is

$\displaystyle(\mathcal{A}\otimes\mathcal{F})\cap(\mathcal{A}\otimes\mathcal{G})=\mathcal{A}\otimes\left(\mathcal{F}\cap\mathcal{G}\right),$
(1)

which means that ’$\otimes$’ is distributive with respect to ’$\cap$’.

The first trivial observation is the inclusion

$\displaystyle(\mathcal{A}\otimes\mathcal{F})\cap(\mathcal{A}\otimes\mathcal{G})\supseteq\mathcal{A}\otimes\left(\mathcal{F}\cap\mathcal{G}\right).$

Also rather easy to show is the relation

$\displaystyle(\mathcal{A}\otimes\mathcal{F})\vee(\mathcal{A}\otimes\mathcal{G})=\mathcal{A}\otimes\left(\mathcal{F}\vee\mathcal{G}\right),$

see e.g. [17, Proof of Lemma 3.2, Step 2]. The nontrivial results in [16] so
far pointed out a counterexample to (1) (Theorem 1) and showed that in case of
analytic measurable spaces $(X,\mathcal{M}),(U,\mathcal{U})$ with
$\mathcal{A}\subseteq\mathcal{M}$,
$\mathcal{F},\mathcal{G}\subseteq\mathcal{U}$, and countably generated
$\sigma$-Algebras
$\mathcal{F}\cap\mathcal{G},\mathcal{A},(\mathcal{A}\otimes\mathcal{F})\cap(\mathcal{A}\otimes\mathcal{G})$,
the equation (1) is equivalent to all atoms of
$(\mathcal{A}\otimes\mathcal{F})\cap(\mathcal{A}\otimes\mathcal{G})$ being
products (Theorem 2).

Here, we generalize these results in the following directions:

  * •

We show that the distributivity equation (1) holds if either one of the three
$\sigma$-algebras are given by a countable partition or if
$\mathcal{F}\cap\mathcal{G}$ is given by a countable partition. This is not
covered in [16].

  * •

We show that counterexamples are not limited to the special one from [16,
Theorem 1] but can be constructed whenever $X$ is not countable and
$\mathcal{F},\mathcal{G}$ are countably separated, but
$\mathcal{F}\cap\mathcal{G}$ is not.

  * •

We obtain a characterization of the distributivity equation (1) in terms of
its atoms, similar to [16, Theorem 2], but now for strongly Blackwell spaces
instead of analytic ones.

##  2 Distributivity for $\sigma$-algebras given by countable partitions

In this section we will treat the countable partition case, as the finite case
basically works in a similar, but easier way. We will write
$\sigma(\\{C_{1},C_{2},\dotsc\\}$ for the $\sigma$-algebra generated by
$\\{C_{1},C_{2},\dotsc\\}.$ Let $\mathbb{N}$ stand for the set of positive
integers. We will say that a $\sigma$-algebra $\mathcal{C}$ is given by a
countable partition if there are countably many(or finitely many) pairwise
disjoint nonempty sets $\\{C_{1},C_{2},\dotsc\\}$ in $\mathcal{C}$ such that
$\\{\bigcup_{i\in I}C_{i}:I\subset\mathbb{N}\\}=\mathcal{C}.$ Note that on a
countable set, every $\sigma$-algebra is given by a countable partition.
Observe also that if $\mathcal{C}$ is a $\sigma$-algebra given by a countable
partition $\\{C_{1},C_{2},\dotsc\\}$ and $\mathcal{D}$ is another
$\sigma$-algebra then every set in $\mathcal{C}\otimes\mathcal{D}$ is of the
form $\bigcup_{i\in\mathbb{N}}(C_{i}\times D_{i})$ for some sequence
$(D_{i})_{i\in\mathbb{N}}$ such that $D_{i}\in\mathcal{D}$ for all $i\geq 1.$
This representation is also unique. We will be using this result repeatedly in
some of the proofs below.

######  Theorem 2.1.

Let $\mathcal{A}$ be a $\sigma$-algebra given by a countable partition of $X$.
Let $\mathcal{F},\mathcal{G}$ be $\sigma$-algebras on $U$. Then equation (1)
is true.

###### Proof.

Let $\mathcal{A}$ be given by a countable partition
$\\{A_{1},A_{2},\dotsc\\}.$

We will take a general set
$B\in(\mathcal{A}\otimes\mathcal{F})\cap(\mathcal{A}\otimes\mathcal{G})$ and
show that $B\in\mathcal{A}\otimes\left(\mathcal{F}\cap\mathcal{G}\right).$ By
the above mentioned result, $B$ can be written as
$\bigcup_{i\in\mathbb{N}}(A_{i}\times F_{i})$ for some $F_{i}\in\mathcal{F}$
for every $i.$ Since $B\in\mathcal{A}\otimes\mathcal{G}$, by taking an $x$ in
$A_{i}$ and considering the section $B_{x}=F_{i}$, we see that
$F_{i}\in\mathcal{G}$ for every $i.$ This shows that
$B\in\mathcal{A}\otimes\left(\mathcal{F}\cap\mathcal{G}\right).$ ∎

The following result concerns the right factors.

######  Theorem 2.2.

If at least one of $\mathcal{F}$ and $\mathcal{G}$ is given by a countable
partition, equation (1) is true. More generally, if
$\mathcal{F}\cap\mathcal{G}$ is given by a countable partition, equation (1)
is true.

###### Proof.

Clearly, if $\mathcal{F}$ (or $\mathcal{G}$) is given by a countable
partition, $\mathcal{F}\cap\mathcal{G}$ is also given by a countable
partition. Hence it is sufficient to prove the second part of the statement of
the theorem. Let $\mathcal{F}\cap\mathcal{G}$ be given by a countable
partition $\\{H_{1},H_{2},\dotsc\\}.$

We will take a general set
$B\in(\mathcal{A}\otimes\mathcal{F})\cap(\mathcal{A}\otimes\mathcal{G})$ and
show that $B\in\mathcal{A}\otimes\left(\mathcal{F}\cap\mathcal{G}\right).$ Let
$x$ be a point in $X.$ Consider the $x$-section $B_{x}=\\{u:(x,u)\in B\\}.$
Then, $B_{x}\in\mathcal{F}\cap\mathcal{G}.$ Hence $B_{x}\supseteq H_{i}$ for
some $i$ if $B_{x}\neq\emptyset.$ Now, for a particular $i\in\mathbb{N},$ if
we define the set $A_{i}=\\{x:B_{x}\supseteq H_{i}\\}$ then,
$A_{i}\in\mathcal{A}$ and $B\supseteq A_{i}\times H_{i}.$ It easily follows
that $B=\bigcup_{i\in\mathbb{N}}(A_{i}\times H_{i}).$ Thus
$B\in\mathcal{A}\otimes\left(\mathcal{F}\cap\mathcal{G}\right).$

∎

Thus we have seen that if at least one of the three $\sigma$-algebras $\cal
A,\cal F,\cal G$ in equation (1) is given by a countable partition, (1) is
true. We will extend this result to the case of $\sigma$-algebras with the
property that every countably generated sub-$\sigma$-algebra is given by a
countable partition. The $\sigma$-algebra of countable and co-countable sets
on any infinite set is an example of such a $\sigma$-algebra (which itself is
not even countably generated). This $\sigma$-algebra is atomic. There are
examples of atomless $\sigma$-algebras (on any uncountable set) with this
property. See [5, Remark 5, p. 108] for examples of such $\sigma$-algebras.

######  Theorem 2.3.

If one of the three $\sigma$-algebras $\mathcal{A}$, $\mathcal{F}$ and
$\mathcal{G}$ in (1) is such that every countably generated
sub-$\sigma$-algebra is given by a countable partition, then (1) is true.
However, if $\mathcal{F}\cap\mathcal{G}$ has the property that every countably
generated sub-$\sigma$-algebra is given by a countable partition, but
$\mathcal{F}\cap\mathcal{G}$ itself is not given by a countable partition, (1)
need not be true.

###### Proof.

We will treat the case of $\mathcal{F}$ having the property that every
countably generated sub-$\sigma$-algebra is given by a countable partition
$\\{F_{1},F_{2},\dotsc\\}.$ Other cases can be covered using a similar
argument.

We will take a general set
$B\in(\mathcal{A}\otimes\mathcal{F})\cap(\mathcal{A}\otimes\mathcal{G})$ and
show that $B\in\mathcal{A}\otimes\left(\mathcal{F}\cap\mathcal{G}\right).$

We know that if a set $B\in\mathcal{C}\otimes\mathcal{D}$ there exists
$\\{C_{1},C_{2},\dotsc\\}$ in $\mathcal{C}$ and $\\{D_{1},D_{2},\dotsc\\}$ in
$\mathcal{D}$ such that $B\in\mathcal{C}_{0}\otimes\mathcal{D}_{0}$ where
$\mathcal{C}_{0}$ is the sub-$\sigma$-algebra generated by
$\\{C_{1},C_{2},\dotsc\\}$ and $\mathcal{D}_{0}$ is the sub-$\sigma$-algebra
generated by $\\{D_{1},D_{2},\dotsc\\}$. We will apply this result to
$\mathcal{A}\otimes\mathcal{F}.$

Since $B\in\mathcal{A}\otimes\mathcal{F}$ there exists
$\\{A_{1},A_{2},\dotsc\\}$ in $\mathcal{A}$ and $\\{F_{1},F_{2},\dotsc\\}$ in
$\mathcal{F}$ such that $B\in\mathcal{A}_{0}\otimes\mathcal{F}_{0}$ where
$\mathcal{A}_{0}$ is the sub-$\sigma$-algebra generated by
$\\{A_{1},A_{2},\dotsc\\}$ and $\mathcal{F}_{0}$ is the sub-$\sigma$-algebra
generated by $\\{F_{1},F_{2},\dotsc\\}.$ Since $\mathcal{F}_{0}$ is a
countably generated sub-$\sigma$-algebra it is given by a countable partition.
The previous theorem gives us that
$B\in\mathcal{A}_{0}\otimes\left(\mathcal{F}_{0}\cap\mathcal{G}_{0}\right).$
Hence $B\in\mathcal{A}\otimes\left(\mathcal{F}\cap\mathcal{G}\right).$

The example in [16, Theorem 1] has the property that
$\mathcal{F}\cap\mathcal{G}$ is the countable co-countable $\sigma$-algebra
and this $\sigma$-algebra has the property that every countably generated sub
$\sigma$-algebra is given by a countable partition. Thus we have concluded the
second statement of the theorem.

∎

In Theorem 3.6 below, we will show that the second part of the above theorem
is true in general.

We remark that, as a consequence of Theorem 2.2, if $\cal F\cap\cal G$ is the
trivial $\sigma$-algebra $\\{\emptyset,U\\}$ or a finite $\sigma$-algebra,
then also (1) is true.

##  3 Counterexamples for Uncountable Sets

In this section here, we first point out, that for $\sigma$-algebras
$\mathcal{F}$ and $\mathcal{G}$ on a set $U$, which are not given by countable
partitions, such that $\mathcal{F}\cap\mathcal{G}$ is infinite and neither
$\mathcal{F}\subseteq\mathcal{G}$ nor $\mathcal{G}\subseteq\mathcal{F}$ hold,
and for a $\sigma$-algebra $\mathcal{A}$, there are examples such that (1)
does not hold. We already saw classes of examples of $\sigma$-algebras such
that (1) holds.

Let us recall a basic definition first.

######  Definition 3.1.

Let $(Y,\mathcal{C})$ be a measurable space.

  1. (i)

The $\sigma$-algebra $C$ is called separated if there is a set $I$, together
with sets $\\{A_{i}:i\in I\\}\subseteq\mathcal{C}$, that separate the points
of $Y$, that is, for any two points $x,y\in Y$ there is an $i\in I$ such that
$x\in A_{i}$ and $y\notin A_{i}$. We call such a system of sets separator.

  2. (ii)

The $\sigma$-algebra $\mathcal{C}$ is called countably separated if
$\mathcal{C}$ contains a countable separator.

######  Remark 3.2.

  1. (i)

Note that if a countable separator exists for $(Y,\mathcal{C})$ then $\cal C$
contains all singletons: just write for $x\in Y$,

$\\{x\\}=\bigcap_{\begin{subarray}{c}i\geq 1\\\ x\in
A_{i}\end{subarray}}A_{i}\cap\bigcap_{\begin{subarray}{c}i\geq 1\\\ x\notin
A_{i}\end{subarray}}A_{i}^{c}.$

  2. (ii)

If $\mathcal{C}$ is separated and countably generated by the generator
$\\{A_{i}:i\geq 1\\}$, then this generator features as countable separator as
well: Assume the contrary, that there are no sets separating the points $x$
and $y\in Y$. That means that all sets $A_{i}$ contain either both, $x$ and
$y$, or none of them. The smallest $\sigma$-algebra that contains all the
$A_{i}$ and which does not separate $x$ and $y$ is the $\sigma$-algebra
generated by $\left\\{A_{i}:i\geq 1\right\\}$ and must thus be $\mathcal{C}$.
But $\mathcal{C}$ is separable. Hence $\left\\{A_{i}:i\geq 1\right\\}$ is a
separator.

######  Example 3.3.

  1. (a)

The Borel $\sigma$-algebra of any separable metric space is countably
generated and countably separated.

  2. (b)

The $\sigma$-algebra of Borel sets of $\mathbb{R}$ invariant of translation by
$1$ (or any other nonzero number) is countably generated but not countably
separated (any set containing $1$ also contains $\mathbb{Z}$).

  3. (c)

The $\sigma$-algebra generated by the analytic sets of $[0,1]$ is not
countably generated [4, p. 15] but countably separated (just take the
separator of the Borel sets).

  4. (d)

The $\sigma$-algebra of countable and co-countable sets on an uncountable set
$Y$ is neither countably generated nor countably separated.

The next theorem is a (substantial) extension of [16, Theorem 1] in the sense
that we provide classes of examples for which (1) does not hold.

######  Theorem 3.4.

Let $\mathcal{B}$ and $\mathcal{D}$ be any countably separated
$\sigma$-algebras on an uncountable set $X$, such that
$\mathcal{B}\cap\mathcal{D}=:\mathcal{C}$ is not countably separated.

Then,

$\displaystyle((\mathcal{D}\vee\mathcal{B})\otimes\mathcal{B})\cap((\mathcal{D}\vee\mathcal{B})\otimes\mathcal{D})\neq(\mathcal{D}\vee\mathcal{B})\otimes(\mathcal{D}\cap\mathcal{B}).$
(2)

###### Proof.

We will first show that the diagonal $\Delta=\\{(x,x):x\in X\\}$ is in in the
left hand side of (2), Let $\left\\{A_{i}:i\geq 1\right\\}$ be a countable
separator of $\mathcal{B}$. $\Delta$ can be expressed as

$\displaystyle\Delta=\bigcap_{i\geq 1}\big{(}(A_{i}\times
A_{i})\cup(A_{i}^{c}\times A_{i}^{c})\big{)}.$

This can be seen, as for all $x\in X$ and $i\geq 1$, $(x,x)$ is either
contained in $A_{i}\times A_{i}$ or $A_{i}^{c}\times A_{i}^{c}$. To exclude
any $(x,y)$ with $x\neq y$, take an $A_{i}$ separating $x$ and $y$, s.t. say
$x\in A_{i}$ and $y\in A_{i}^{c}$. Now $(x,y)\notin(A_{i}\times
A_{i})\cup(A_{i}^{c}\times A_{i}^{c})$. It follows that
$\Delta\in\mathcal{B}\otimes\mathcal{B}$. In the same way,
$\Delta\in\mathcal{D}\otimes\mathcal{D}$, showing that $\Delta$ is contained
in the left hand side of (2).

If for $\sigma$-algebras $\mathcal{H},\mathcal{I}$ on $X$, the diagonal
$\Delta$ is contained in $\mathcal{H}\otimes\mathcal{I}$, then $\mathcal{I}$
needs to be countably separated. This is a special case e.g. of [12,
Proposition 2.1] taking $f$ equal to the identity function.

Since $\mathcal{C}$ is not countably separated, it follows that $\Delta$
cannot be contained in the right hand side of (2). ∎

Two examples of pairs of $\sigma$-algebras satisfying the conditions of the
above theorem can be found in [4]. The first such example is from [4, p. 16]
and the second from [4, Proposition 57, p. 55].

######  Example 3.5.

  1. (a)

This example was first published in [1], following an idea from P. R. Halmos,
and was also used as a counterexample for (1) in [16].

The $\sigma$-algebra $\mathcal{D}$ is the preimage of the Borel sets
$\mathcal{B}$ on $X=[0,1]$ under a certain function $f\colon{[0,1]}\to{[0,1]}$
constructed as follows (see also [1], [14]):

Let $\omega_{\mathfrak{c}}$ be the first ordinal corresponding to the cardinal
$\mathfrak{c}$ of the continuum. Let
$(M_{\alpha})_{1\leq\alpha<\omega_{\mathfrak{c}}}$ be an enumeration of all
uncountable Borel subsets of $[0,1]$ with uncountable complement. Since all
uncountable Borel sets have cardinality $\mathfrak{c}$ (see e.g. [10, Theorem
13.6]) we can associate to each ordinal $\alpha<\omega_{\mathfrak{c}}$ a
triplet $(x_{\alpha},y_{\alpha},z_{\alpha})$ such that
$x_{\alpha},y_{\alpha}\in M_{\alpha}$, $z_{\alpha}\in[0,1]\setminus
M_{\alpha}$ and
$\\{x_{\alpha},y_{\alpha},z_{\alpha}\\}\cap\bigcup_{\beta<\alpha}\left\\{x_{\beta},y_{\beta},z_{\beta}\right\\}=\emptyset$
(as the cardinality of
$\bigcup_{\beta<\alpha}\left\\{x_{\beta},y_{\beta},z_{\beta}\right\\}$ is
strictly smaller than $\mathfrak{c}$). Define $f$ as the function that for
each $\alpha<\omega_{\mathfrak{c}}$ maps $x_{\alpha}\mapsto z_{\alpha}$,
$z_{\alpha}\mapsto x_{\alpha}$ and keeps $y_{\alpha}$ and all points outside
$\bigcup_{\beta<\omega_{\mathfrak{c}}}\left\\{x_{\beta},y_{\beta},z_{\beta}\right\\}$
fixed.

Finally, set

$\displaystyle\mathcal{D}:=\sigma(f)=\left\\{f^{-1}(B):B\in\mathcal{B}\right\\}.$

The intersection $\mathcal{B}\cap\mathcal{D}$ is the $\sigma$-algebra of
countable and co-countable sets.

  2. (b)

[4, Proposition 57] Let $f$ be a bijection of $[0,1]$ to some analytic, non-
Borel set $A\subseteq[0,1]$. Let $\mathcal{B}=\mathcal{B}([0,1])$, and let
$\mathcal{B}_{A}$ be the trace $\sigma$-algebra on $A$. Set
$\mathcal{D}=f^{-1}(\mathcal{B}_{A})$. Assume that
$\mathcal{B}\cap\mathcal{D}$ were countably separated. Then it contains a
separator $\\{A_{i}:i\geq 1\\}$ and $\sigma(A_{i}:i\geq 1)$ is countably
generated and countably separated. It follows through [4, Proposition 5] (the
Blackwell nature of $([0,1],\mathcal{B})$ and $([0,1],\mathcal{B})$) that
$\sigma(A_{i}:i\geq 1)=\mathcal{B}=\mathcal{D}$ and that $f$ were an
isomorphism $([0,1],\mathcal{B}([0,1]))\leftrightarrow(A,\mathcal{B}_{A})$.
This would imply that $A$ is a standard Borel set, which it is not. Hence
$\mathcal{B}\cap\mathcal{D}$ is not countably separated.

A consequence of Theorem 3.4 is that the second part of Theorem 2.3 is in fact
true in general if $\mathcal{F}$ and $\mathcal{G}$ are countably separated.

######  Theorem 3.6.

If $\mathcal{F}$ and $\mathcal{G}$ are countably separated and if
$\mathcal{F}\cap\mathcal{G}$ has the property that every countably generated
sub $\sigma$-algebra is given by a countable partition, but
$\mathcal{F}\cap\mathcal{G}$ itself is not given by a countable partition, (1)
is not true.

###### Proof.

If $\mathcal{F}\cap\mathcal{G}$ has the mentioned property, then, we show that
$\mathcal{F}\cap\mathcal{G}$ is not countably separated. If it is, let
$\\{A_{1},A_{2},\cdots\\}$ be a countable separator. Then
$\sigma(\\{A_{1},A_{2},\cdots\\})$ being a countably generated sub
$\sigma$-algebra is given by a countable partition. If
$\\{B_{1},B_{2},\cdots\\}$ is the countable partition, all the $B_{i}$s need
to be singleton sets. This implies that
$\sigma(\\{B_{1},B_{2},\cdots\\})=\sigma(\\{A_{1},A_{2},\cdots\\})=\mathcal{F}\cap\mathcal{G}$
and that $\mathcal{F}\cap\mathcal{G}$ is given by a countable partition and
this is a contradiction. Thus, $\mathcal{F}\cap\mathcal{G}$ is not countably
separated. From Theorem 3.4 the result follows. ∎

##  4 Distributivity and Blackwell spaces

######  Definition 4.1.

Let $(Y,\mathcal{C})$ be a measurable space.

  1. (i)

An atom $K\in\mathcal{C}$ is a nonempty set, such that no proper nonempty
subset is contained in $\cal C$. In other words, they are the minimal elements
with respect to ‘$\subseteq$’ in $\mathcal{C}$.

  2. (ii)

A $\sigma$-algebra $\mathcal{C}$ is called atomic if $Y$ is the union of the
atoms of $\cal C$.

This definition of an atom does not coincide with the similar notion from
measure theory, where it is a set of positive measure, such that all of its
proper subsets in the $\sigma$-algebra have zero measure. There are
$\sigma$-algebras containing atoms and atomless $\sigma$-algebras [4, Chapter
3]. Note that if two atoms of $\cal C$, when they exist, are disjoint.

######  Definition 4.2.

Let $(Y,\mathcal{C})$ be a measurable space.

  1. (i)

A subset of $[0,1]$ is called analytic if it is the image of a Polish space
under a continuous function.

  2. (ii)

We call $(Y,\mathcal{C})$ analytic if $Y$ is isomorphic (i.e. there is a
bijective, bimeasurable function) to an analytic subset of the unit interval
and $\mathcal{C}$ is countably generated and contains all singletons of $Y$.

We cite the following result of Blackwell [7] and Mackey [11], see also [4,
Chapter 2].

######  Lemma 4.3 ([7, Section 4], [11, Section 4], [4, Proposition 6]).

If $(Y,\mathcal{C})$ is an analytic space and $\mathcal{W},\mathcal{V}$ are
countably generated sub $\sigma$-algebras of $\mathcal{C}$ with the same
atoms, then $\mathcal{W}=\mathcal{V}$.

To generalize the situation of Lemma 4.3, we recall the definition of a
Blackwell space:

######  Definition 4.4.

Let $(Y,\mathcal{C})$ be a countably generated and separated (and thus
countably separated) measurable space.

  1. (i)

The space $(Y,\mathcal{C})$ is called Blackwell if the only separated
sub-$\sigma$-algebra of $\mathcal{C}$ is itself.

  2. (ii)

The space $(Y,\mathcal{C})$ is called strongly Blackwell if any two countably
generated sub-$\sigma$-algebras with the same atoms coincide.

For a thorough account on Blackwell spaces see [6]. Therein it is also shown
that under the continuum hypothesis (CH) there are Blackwell spaces which are
not strongly Blackwell. The same holds if Martin’s axiom and ($\neg$ CH) is
assumed.

To obtain a distributivity characterization in the sense of [16, Theorem 2] in
the case of Blackwell instead of analytic spaces, we formulate the following
Lemma (which has been shown in several places in the literature during proofs,
such as in [4] or [16]).

######  Lemma 4.5.

If $\cal B$ on $X$ and $\cal C$ on $Y$ are atomic $\sigma$\- algebras, then
$\cal B\otimes\cal C$ is atomic and atoms of $\cal B\otimes\cal C$ are
cartesian products.

###### Proof.

If $B\in\cal B$ and $C\in\cal C$ are atoms, to show that $B\times C$ is an
atom, observe that every rectangle in $\cal B\otimes\cal C$ either contains
$B\times C$ or is disjoint with $B\times C.$ On the other hand, $\\{A\in{\cal
B}\otimes{\cal C}:A\mbox{ either contains }B\times C\mbox{ or is disjoint with
}B\times C\\}$ is a $\sigma$-algebra. Hence every set $\cal B\times\cal C$
either contains $B\times C$ or is disjoint with $B\times C.$ But then the
union of all such sets (atoms) is equal to $X\times Y.$ Thus $\cal
B\otimes\cal C$ is atomic. ∎

######  Theorem 4.6.

Let $(X,\cal A),(U,\cal F)$ and $(U,\cal G)$ be measure spaces. In the
following, [(i) $\Rightarrow$ (ii)] is always true. The assertion [(ii)
$\Rightarrow$ (iii)] is true if $\cal A$ and $\cal F\cap\cal G$ are atomic.

(i)

$(\cal A\otimes\cal F)\cap(\cal A\otimes\cal G)=\cal A\otimes(\cal F\cap\cal
G)$

(ii)

$(\cal A\otimes\cal F)\cap(\cal A\otimes\cal G)\mbox{ and }\cal A\otimes(\cal
F\cap\cal G)\mbox{ have the same atoms}$

(iii)

$\mbox{ All atoms of }(\cal A\otimes\cal F)\cap(\cal A\otimes\cal G)\mbox{ are
cartesian products }$

If $(\cal A\otimes\cal F)\cap(\cal A\otimes\cal G)$ is strongly Blackwell (or
a sub $\sigma$-algebra of a strongly Blackwell space $(X,\cal M$)), $(\cal
A\otimes\cal F)\cap(\cal A\otimes\cal G)$ is countably generated and $\cal
A\otimes(\cal F\cap\cal G)$ is countably generated, then (iii) implies (i).

###### Proof.

The proof is a direct consequence of Lemma 4.5 and the definition of a
strongly Blackwell space. ∎

## References

  * [1] R. Aumann, Borel structures for function spaces, Ill. J. Math. 5, p. 614-635, 1961.
  * [2] D. Basu, Problems relating to the existence of maximal and minimal elements in some families of statistics (subfields), in Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Statistics, p. 41-50, 1967.
  * [3] M. Bhaskara Rao, K.P.S. Bhaskara Rao, Borel $\sigma$-algebra on ${[0,\Omega]}$, Bull. Acad. Polon. Sci. 26, p. 767-769, 1978.
  * [4] K.P.S. Bhaskara Rao, B.V. Rao, Borel Spaces, Warszawa: Instytut Matematyczny Polskiej Akademi Nauk, Dissertationes Mathematicae, Tom. CXC, http://eudml.org/doc/268562, 1981.
  * [5] K.P.S. Bhaskara Rao, B.V. Rao, Mixtures of Non-atomic Measures II , Colloquium Mathematicum, 33, 1, p. 105-112, 1975.
  * [6] K.P.S. Bhaskara Rao, R.M. Shortt, Borel Spaces II, Warszawa: Instytut Matematyczny Polskiej Akademi Nauk, Dissertationes Mathematicae, Tom. CCCLXXII, 1997.
  * [7] D. Blackwell, On a Class of Probability spaces, 3rd Berkeley Symposium 2, p. 1, 1956.
  * [8] D. Georgiou, I. Kougias, A. Megaritis, Borel Structures on the Set of Borel Mappings, Topology and its Applications, 159, 7, p. 1906-1915, 2012.
  * [9] E. Grzegorek, Remark on Some Borel Structures, Measure Theory Oberwolfach, p. 69-74, 1983.
  * [10] A. Kechris, Classical Descriptive Set Theory, Springer, New York, 1994.
  * [11] G.W. Mackey, Borel Structures in Groups and Their Duals, Transactions of the American Mathematical Society 85, p. 134-165, 1957.
  * [12] K. Musiał, Projective limits of perfect measure spaces, Fundamenta Mathematicae 110(3), 1980.
  * [13] W. Parry, Topics in Ergodics Theory, Cambridge University Press, 1981.
  * [14] B.V. Rao, On Borel Structures, Colloq. Math. 21, 199-204, 1970.
  * [15] Mathematics Stack Exchange, https://math.stackexchange.com/questions/92546/decreasing-sequence-of-product-sigma-algebras, viewed on July 30, 2022.
  * [16] A. Steinicke, (Non-)Distributivity of the Product for $\sigma$-Algebras with Respect to the Intersection, Archiv der Mathematik, 116(6), 667-675, 2021.
  * [17] A. Steinicke, Functionals of a Lévy Process on Canonical and Generic Probability Spaces, Journal of Theoretical Probability, 29, 2, p. 443-458, 2016.


11institutetext:  National-Regional Key Technology Engineering Laboratory for
Medical Ultrasound, School of Biomedical Engineering, Health Science Center,
Shenzhen University, China
11email<EMAIL_ADDRESS>22institutetext: Medical Ultrasound Image Computing
(MUSIC) Lab, Shenzhen University, China 33institutetext: Marshall Laboratory
of Biomedical Engineering, Shenzhen University, China

# Deep Motion Network for Freehand 3D Ultrasound Reconstruction

Mingyuan Luo Mingyuan Luo and Xin Yang contribute equally to this work.112233
Xin Yang0 112233 Hongzhang Wang 112233 Liwei Du 112233 Dong Ni(✉) 112233

###### Abstract

Freehand 3D ultrasound (US) has important clinical value due to its low cost
and unrestricted field of view. Recently deep learning algorithms have removed
its dependence on bulky and expensive external positioning devices. However,
improving reconstruction accuracy is still hampered by difficult elevational
displacement estimation and large cumulative drift. In this context, we
propose a novel deep motion network (MoNet) that integrates images and a
lightweight sensor known as the inertial measurement unit (IMU) from a
velocity perspective to alleviate the obstacles mentioned above. Our
contribution is two-fold. First, we introduce IMU acceleration for the first
time to estimate elevational displacements outside the plane. We propose a
temporal and multi-branch structure to mine the valuable information of low
signal-to-noise ratio (SNR) acceleration. Second, we propose a multi-modal
online self-supervised strategy that leverages IMU information as weak labels
for adaptive optimization to reduce drift errors and further ameliorate the
impacts of acceleration noise. Experiments show that our proposed method
achieves the superior reconstruction performance, exceeding state-of-the-art
methods across the board.

###### Keywords:

Inertial Measurement Unit Online Learning Freehand 3D Ultrasound

##  1 Introduction

Three-dimensional (3D) ultrasound (US) is widely used because of its intuitive
visuals, easy interaction and rich clinical information. Freehand 3D US offers
flexibility and simplicity over mechanical probes or electronic phased arrays
[5]. It typically reconstructs the volume by calculating the relative
positions of a series of US images. Recently, freehand reconstruction
techniques have moved away from complex and costly external positioning
systems previously used to obtain high-precision positions [12, 11, 2, 7].
However, improving reconstruction accuracy remains difficult as these
techniques only rely on images to infer relative positions. Specifically, the
difficulty in estimating elevational displacement between images and the
accumulation of drift errors make the reconstruction very challenging. In this
regard, lightweight sensors are expected to avoid the disadvantages of
traditional position sensors while improving the deep learning based
reconstruction performance, as illustrated in Fig. 1.

Figure 1: Pipeline of freehand 3D US reconstruction with a lightweight
inertial measurement unit (IMU) sensor.

Freehand 3D US reconstruction has been studied for over half a century [10].
Early solutions mainly relied on complex and expensive external positioning
systems to accurately calculate image locations [8, 10, 9]. The non-sensor
scheme is mainly speckle decorrelation [13], which uses speckle correlation
between adjacent images to estimate relative motion and decomposes it into in-
plane and out-of-plane parts. However, the reconstruction quality is
susceptible to scan rate and angle [9]. With the development of deep learning
technology [6], Prevost et al. [12] first used a convolutional neural network
(CNN) to estimate the relative motion of US images. Guo et al. [2] proposed a
deep contextual learning network (DCL-Net) to mine the correlation information
of US video clips for 3D reconstruction. Luo et al. [7] designed an online
learning framework (OLF) that improves reconstruction performance by utilizing
consistency constraints and shape priors. Although effective, the
reconstruction performance based on deep learning still faces challenges such
as difficult elevational displacement estimation and large cumulative drift.

Compared to complicated positioning systems, the lightweight sensor called
inertial measurement unit (IMU) is inexpensive and takes up less space (Fig.
1). IMU integrates triaxial accelerometer, gyroscope and magnetometer sensors
to measure an object’s triaxial orientation and acceleration [1]. Integrating
an IMU does not increase US scanning complexity. Prevost et al. [11] first
demonstrated the promise of boosting reconstruction performance by simply
concatenating the IMU orientation with the fully connected layer of neural
networks, due to the low signal-to-noise ratio (SNR) of the IMU acceleration.

In this study, we propose a novel lightweight sensor-based deep motion network
(MoNet) for freehand 3D US reconstruction. Our contribution is two-fold.
First, we equip a lightweight IMU sensor and exploit the IMU acceleration and
orientation for the first time to improve image-based reconstruction
performance. We propose a temporal and multi-branch structure from a velocity
perspective to mine the valuable information of low-SNR acceleration. Second,
we propose a multi-modal online self-supervised strategy that leverages IMU
information as weak label for adaptive optimization to reduce drift errors and
further amelioratethe impacts of acceleration noise. We thoroughly validate
the efficacy and generalizability of MoNet on the collected arm and carotid
scans. Experimental results show that MoNet achieves state-of-the-art
reconstruction performance by fully utilizing the IMU information and deeply
fusing it with image content.

##  2 Methodology

Figure 2: Overview of our lightweight sensor-based deep motion network
(MoNet).

We define the $N$-length scan as $I=\\{I_{i}|i=1,2,\cdots,N\\}$, and for image
$I_{i}$, the corresponding orientation and acceleration vector provided by the
IMU as $O_{i}=(O_{x},O_{y},O_{z})_{i}$ and $A_{i}=(A_{x},A_{y},A_{z})_{i}$,
respectively. The $O_{i}$ is based on the east-north-up coordinate system, and
the Euler angle $\Phi_{i}=(\Phi_{x},\Phi_{y},\Phi_{z})_{i}$ between the images
$I_{i}$ and $I_{i+1}$ can be calculated as

$\Phi_{i}=M^{-1}(M(O_{i})^{-1}*M(O_{i+1})),\quad i=1,2,\cdots,N-1,$ (1)

where $M(\cdot)$ converts the orientation vector into a $3\times 3$ rotation
matrix and $M^{-1}(\cdot)$ denotes the inverse operation of $M(\cdot)$. For
the $A_{i}$, we first subtract the component of the gravity direction $g_{i}$
computed from $O_{i}$. We also adjust the mean value of acceleration to zero
to reduce the influence of noise. The calculation can be expressed as

$A_{i}\leftarrow(A_{i}-g_{i})-\frac{1}{N}\sum_{i}{(A_{i}-g_{i})},\quad
i=1,2,\cdots,N.$ (2)

Fig. 2 illustrates the proposed lightweight sensor-based deep motion network
(MoNet). MoNet takes all adjacent images
$\\{(I_{i},I_{i+1})|i=1,2,\cdots,N-1\\}$, the acceleration
$\\{A_{i}|i=2,3,\cdots,N-1\\}$ and the Euler angle
$\\{\Phi_{i}|i=1,2,\cdots,N-1\\}$ as inputs to estimate the relative
transformation parameters $\theta=\\{\theta_{i}|i=1,2,\cdots,N-1\\}$, where
$\theta_{i}$ indicates 3 translations $t_{i}=(t_{x},t_{y},t_{z})_{i}$ and 3
rotation degrees $\phi_{i}=(\phi_{x},\phi_{y},\phi_{z})_{i}$, respectively. It
contains a temporal and multi-branch structure for mining valuable IMU
information. Furthermore, the multi-modal online self-supervised strategy is
used to improve estimation accuracy further.

###  2.1 Temporal and Multi-branch Structure for IMU Fusion

As shown in Fig. 2, we construct the MoNet using ResNet [3] and LSTM [4].
ResNet is a powerful feature extraction network that is commonly used to
design networks for various tasks. LSTM is used to process temporal
information, memorizing the knowledge of all historical images as contextual
information to help estimate future parameters.

Estimating out-of-plane elevational displacements from images alone is a huge
challenge. In this study, we introduce IMU acceleration for the first time to
address this challenge. We first argue that the output features of the ResNet
with adjacent images as input implicitly contain relative distance information
between images. Second, scanning speed can be represented by relative
distance, assuming a constant sampling time between adjacent images. Hence,
the IMU acceleration should be associated with the features containing
velocity information, allowing complementary learning through a multi-branch
structure implementation.

Within the multi-branch structure as shown in Fig. 2, we map the processed
acceleration to a high-dimensional space using multiple fully connected layers
and reshape it into 2D features for easy combination with image features. The
acceleration feature $f^{a}$ is then added to the implied velocity feature $f$
to construct the IMU velocity feature $f^{v}$, which can be expressed as

$\left\\{\begin{array}[]{lr}f^{v}_{1}=f_{1},&\\\
f^{v}_{i}=f_{i-1}+f^{a}_{i},&i=2,3,\cdots,N-1.\end{array}\right.$ (3)

Subsequently, to minimize the impact of acceleration noise on network
performance in short sequence intervals, we feed $f^{v}$ into a LSTM to
enhance the velocity features using the temporal context information. The LSTM
output is merged into the main branch and added to the ResNet output. By
fusing the IMU acceleration, the multi-branch structure provides a better
estimation of elevational displacements. Furthermore, the Euler angle
calculated from IMU orientation is fed into multiple fully connected layers
and concatenated to the LSTM input in the main branch to enhance the Euler
angle estimation in the relative transformation parameters.

In the training phase, the loss function of MoNet contains two items. The
first item minimizes the mean absolute error (MAE) between the estimated
transformation parameters $\hat{\theta}$ and ground truth $\theta$. The second
item is the Pearson correlation loss from [2], which aids in learning the
general trend of the scan.

$L=\|\hat{\theta}-\theta\|_{1}+(1-\frac{\textbf{Cov}(\hat{\theta},\theta)}{\sigma(\hat{\theta})\sigma(\theta)}),$
(4)

where $\left\|\cdot\right\|_{1}$ calculates L1 normalization, Cov denotes the
covariance, and $\sigma$ indicates the standard deviation.

###  2.2 Multi-modal Online Self-supervised Strategy

Most previous 3D reconstruction methods [12, 11, 2] relied solely on offline
network training and direct online inference. This strategy often fails to
handle scans with a different distribution than the training data. Online
self-supervised learning can address this challenge by leveraging adaptive
optimization and valuable prior knowledge. In this study, we propose a multi-
modal online self-supervised strategy that leverages IMU information as weak
labels for adaptive optimization to reduce drift errors and further ameliorate
the impacts of acceleration noise. Specifically, as shown in Fig. 2, the IMU
acceleration and Euler angle are used as weak labels for iterative
optimization after estimating the transformation parameters using the trained
MoNet. The estimated acceleration $\hat{A}$ at the centroid of each image is
calculated from the estimated translations $\hat{t}$ and scaled to match the
mean-zeroed IMU acceleration.

$\hat{A}_{i}=(\hat{t}^{-1}_{i-1}+\hat{t}_{i})-\frac{1}{N-2}\sum_{i}(\hat{t}^{-1}_{i-1}+\hat{t}_{i}),\quad
i=2,3,\cdots,N-1,$ (5)

where $\hat{t}^{-1}_{i-1}$ denotes the translations in the inversion of
$\hat{\theta}_{i-1}$. The optimization loss $L_{online}$ is split into two
components: acceleration and Euler angle. We measure the difference between
the estimated acceleration $\hat{A}$ and IMU acceleration $A$ with the Pearson
correlation loss. This exploits the correct trend of IMU acceleration over a
wide range of each scan to improve the MoNet estimation and ameliorate the
impacts of noise. Meanwhile, the MAE loss constrains the difference between
the estimated Euler angle $\hat{\phi}$ and IMU Euler angle $\Phi$.

$L_{online}=(1-\frac{\textbf{Cov}(\hat{A},A)}{\sigma(\hat{A})\sigma(A)})+\|\hat{\phi}-\Phi\|_{1}$
(6)

##  3 Experiments

##### Materials and Implementation.

We built a data acquisition system to acquire all US scans and corresponding
IMU data. The system consists of a portable US machine, an IMU sensor
(WT901C-232, WitMotion ShenZhen Co.,Ltd., China) and an electromagnetic (EM)
positioning transmitter/receiver. We acquired US images with a linear probe at
10 MHz and bound the IMU sensor to the probe to obtain the acceleration and
orientation information. We also connected the EM receiver to the probe and
utilized the EM positioning system to trace the scan route accurately. The US
image depth is set as 3.5 cm, the IMU measurement resolution is $5\times
10^{-4}$ g/LSB (acceleration) and 0.5 degree (orientation), and the EM
positioning resolution is 1.4 mm (position) and 0.5 degree (orientation). We
calibrated the entire system to acquire precise transformation parameters
while minimizing EM positioning interference with the IMU. As shown in Fig. 3,
we analyzed the acceleration and Euler angle calculated by IMU data and
compared them to the EM positioning. It can be observed that the acceleration
is much more noisy at a single point compared to the Euler angle, but has a
correct trend over a wide range.

Figure 3: Comparison of IMU data ($\Phi$ and $A$, blue line) and EM
positioning data ($\phi$ and $A$, red line). The abscissa of each subfigure
indicates the image index.

We constructed two datasets including arm and carotid using the acquisition
system for evaluation. The arm dataset contains 250 scans from 41 volunteers
with an average length of 94.83 mm. Scanning tactics included linear, curved,
fast-and-slow and loop scans to mimic complex real-world situations. The
carotid dataset contains 160 scans from 40 volunteers, with an average length
of 53.71 mm. Only linear scans were performed due to the narrow pathway and
the tissue’s deformable tendency. The size of all scanned images is $478\times
522$ pixel, and the image spacing is $0.075\times 0.075$ mm2. The local IRB
approves the collection and use of data.

We randomly divided the arm and carotid datasets into 196/54 and 136/24 scans
based on volunteer level to construct the training/test set. All images were
scaled down to 0.6 times their original size. To increase the network’s
robustness and prevent overfitting, we conducted random augmentation of each
scan, including subsequence intercepting, interval sampling, and sequence
inversion. Each training scan was randomly augmented to 40 sequences and
regenerated at each epoch to further improve the model generalizability. Each
test scan was randomly augmented to 10 fixed sequences to mimic complex real-
world situations. The Adam optimizer is used to optimize the MoNet. In the
training phase, the epoch and batch size are 200 and 1, respectively. The
learning rate is $10^{-4}$ and is cut in half every 30 epochs. During the
online learning phase, the iteration epoch and learning rate are 60 and
$2\times 10^{-6}$, respectively. All code was implemented in PyTorch and ran
on a RTX 3090 GPU.

##### Quantitative and Qualitative Analysis.

To demonstrate the efficacy of our method, we performed the comparison with
three SOTA approaches including CNN [11], DCL-Net [2], and OLF [7]. CNN used
images, calculated optical flow and IMU Euler angles as input, whereas DCL-Net
and OLF only used scanned images. Note that DCL-Net was modified to estimate
the transformation parameters of adjacent images rather than the average
parameters of several images to achieve better performance. Meanwhile, OLF’s
shape prior module was removed because muscles and vessels do not have
prominent shape features. In this study, six criteria, including final drift
rate (FDR), average drift rate (ADR), maximum drift (MD), sum of drift (SD),
symmetric Hausdorff distance (HD), and mean error of angle (EA) are used to
evaluate the performance (all criteria refer to [7] except EA). Furthermore,
our ablation experiments compare Backbone (ResNet+LSTM), Backbone+IMU, and
full MoNet (Backbone+IMU+Online) to validate the efficacy of IMU fusion and
online learning strategy.

Table 1 shows that our MoNet outperforms previous approaches on all metrics
significantly ($t$-test, $p<0.05$) for both arm and carotid scans. We note
that the CNN incorporating IMU Euler angles [11] outperforms other sensorless
approaches [2, 7] in terms of EA metric, verifying the effectiveness of IMU
integration. However, the simple combination of CNN and IMU results in the
lowest performance on other metrics. In addition, the OLF [7] and proposed
MoNet outperform other methods [11, 2] on most metrics, illustrating the
importance of online learning. The ablation experiments further demonstrate
that both IMU fusion and online learning strategy greatly improve the
reconstruction accuracy.

Table 1: The mean (std) results of different models on the arm and carotid scans. DCL: DCL-Net, Bk: Backbone. The best results are shown in blue. Models | FDR(%)$\downarrow$ | ADR(%)$\downarrow$ | MD(mm)$\downarrow$ | SD(mm)$\downarrow$ | HD(mm)$\downarrow$ | EA(deg)$\downarrow$
---|---|---|---|---|---|---
| Arm scans
CNN [11] | 31.84(18.35) | 37.58(18.02) | 27.31(19.34) | 765.02(721.00) | 26.45(19.42) | 1.96(1.70)
DCL [2] | 20.17(13.37) | 26.05(15.57) | 16.75(11.70) | 476.67(440.90) | 15.82(11.50) | 2.78(2.93)
OLF [7] | 15.00(13.09) | 20.91(12.23) | 12.69(8.30) | 374.07(350.84) | 11.92(8.02) | 2.38(2.25)
Bk | 16.42(14.24) | 22.40(16.13) | 13.10(10.15) | 394.90(427.16) | 11.92(9.22) | 2.29(2.50)
Bk+IMU | 14.05(10.36) | 20.12(12.63) | 11.56(7.54) | 352.54(334.08) | 10.76(7.20) | 1.75(1.57)
MoNet | 12.75(9.05) | 19.05(11.46) | 10.24(7.36) | 332.29(316.36) | 9.40(7.13) | 1.55(1.46)
| Carotid scans
CNN [11] | 31.88(15.76) | 39.71(14.88) | 17.63(9.88) | 493.71(449.95) | 16.66(9.92) | 2.31(1.79)
DCL [2] | 24.66(12.11) | 30.06(12.26) | 13.43(7.30) | 362.28(296.41) | 12.79(7.44) | 2.54(1.60)
OLF [7] | 20.08(14.72) | 29.21(14.99) | 11.01(6.76) | 326.91(317.36) | 10.38(6.53) | 2.59(1.58)
Bk | 20.55(18.73) | 30.74(19.09) | 11.06(7.84) | 318.38(306.10) | 10.03(6.73) | 2.61(1.72)
Bk+IMU | 17.78(11.50) | 27.47(13.05) | 9.73(4.83) | 285.99(239.24) | 9.14(4.83) | 2.18(1.43)
MoNet | 15.67(8.37) | 25.08(9.34) | 8.89(4.31) | 258.83(208.12) | 8.28(4.29) | 1.50(0.98)
Figure 4: Metric decline curves (with 95% confidence interval) for online
learning strategy. The abscissa and ordinate represent the number of
iterations and the value of metrics. Figure 5: Typical reconstruction cases on
arm (Row I) and carotid (Row II) scans.

We further illustrate the metric decline curves when using online learning
strategy in Fig. 4. For both arm and carotid, the online learning
substantially reduces all metrics and the most significant reductions are
12.64% of HD (arm) and 31.19% of EA (carotid). This further demonstrates the
efficacy of our proposed online learning strategy. Fig. 5 visualizes six
typical results to show the difference between our MoNet and other methods. It
can be observed that our proposed MoNet achieves the closest reconstruction
outcomes to ground truth compared to the Backbone or other methods.

##  4 Conclusion

We propose a lightweight sensor-based deep motion network (MoNet) to conduct
freehand 3D US reconstruction. For the first time, we propose a temporal and
multi-branch structure from a velocity perspective to mine the valuable
information of IMU low-SNR acceleration. We propose a multi-modal online self-
supervised strategy that leverages IMU information as weak labels for adaptive
optimization to reduce drift errors and further ameliorate the impacts of
acceleration noise. We build a data acquisition system and thoroughly validate
the system efficacy on the arm and carotid datasets. Experiments show that
MoNet achieves state-of-the-art reconstruction performance by fully utilizing
the IMU information and deeply fusing it with image content. Future research
will focus on extending this network to more anatomical structures.

#### Acknowledgements

This work was supported by the grant from National Natural Science Foundation
of China (Nos. 62171290, 62101343), Shenzhen-Hong Kong Joint Research Program
(No. SGDX20201103095613036), and Shenzhen Science and Technology Innovations
Committee (No. 20200812143441001).

## References

  * [1] Ahmad, N., Ghazilla, R.A.R., Khairi, N.M., Kasi, V.: Reviews on various inertial measurement unit (imu) sensor applications. International Journal of Signal Processing Systems 1(2), 256–262 (2013)
  * [2] Guo, H., Xu, S., Wood, B., Yan, P.: Sensorless freehand 3d ultrasound reconstruction via deep contextual learning. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 463–472. Springer (2020)
  * [3] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 770–778. IEEE (2016)
  * [4] Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Computation 9(8), 1735–1780 (1997)
  * [5] Huang, Q., Zeng, Z.: A review on real-time 3d ultrasound imaging technology. BioMed research international 2017 (2017)
  * [6] LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. nature 521(7553), 436–444 (2015)
  * [7] Luo, M., Yang, X., Huang, X., Huang, Y., Zou, Y., Hu, X., Ravikumar, N., Frangi, A.F., Ni, D.: Self context and shape prior for sensorless freehand 3d ultrasound reconstruction. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 201–210. Springer (2021)
  * [8] Mercier, L., LangØ, T., Lindseth, F., Collins, D.L.: A review of calibration techniques for freehand 3-d ultrasound systems. Ultrasound in Medicine & Biology 31, 449–471 (2005)
  * [9] Mohamed, F., Siang, C.V.: A survey on 3d ultrasound reconstruction techniques. Artificial Intelligence—Applications in Medicine and Biology (2019)
  * [10] Mozaffari, M.H., Lee, W.S.: Freehand 3-d ultrasound imaging: a systematic review. Ultrasound in medicine & biology 43(10), 2099–2124 (2017)
  * [11] Prevost, R., Salehi, M., Jagoda, S., Kumar, N., Sprung, J., Ladikos, A., Bauer, R., Zettinig, O., Wein, W.: 3d freehand ultrasound without external tracking using deep learning. Medical image analysis 48, 187–202 (2018)
  * [12] Prevost, R., Salehi, M., Sprung, J., Ladikos, A., Bauer, R., Wein, W.: Deep learning for sensorless 3d freehand ultrasound imaging. In: International conference on medical image computing and computer-assisted intervention. pp. 628–636. Springer (2017)
  * [13] Tuthill, T.A., Krücker, J., Fowlkes, J.B., Carson, P.L.: Automated three-dimensional us frame positioning computed from elevational speckle decorrelation. Radiology 209(2), 575–582 (1998)