# Viscoelastic response of an active particle under the action of magnetic
field

M Muhsin  Department of Physics, University of Kerala, Kariavattom,
Thiruvananthapuram-$695581$, India  F Adersh  Department of Physics,
University of Kerala, Kariavattom, Thiruvananthapuram-$695581$, India  M Sahoo
<EMAIL_ADDRESS>Department of Physics, University of Kerala, Kariavattom,
Thiruvananthapuram-$695581$, India

###### Abstract

We consider the dynamics of a charged inertial active Ornstein-Uhlenbeck
particle in a viscoelastic suspension under the action of an uniform magnetic
field. With the help of both numerical simulation and analytical framework, we
exactly investigate the viscoelastic response of the particle to the magnetic
field by means of particle trajectories, mean displacement, and mean square
displacement (MSD) calculations. The simulated particle trajectories and MSD
calculations reveal that the steady state response of a confined harmonic
particle to the magnetic field show interesting features due to the complex
interplay of underlying physical processes such as elastic dissipation and
active fluctuations in the medium. When the activity time scale of the
dynamics is larger than the elastic dissipation timescale (or vice-versa), the
steady state MSD shows an enhancement (or suppression) with increase in the
field strength. In both the cases, for very strong magnetic field the MSD
interestingly approaches the value at the equilibrium limit where the
generalized fluctuation dissipation relation is satisfied and the the particle
behaves like an inertial passive Brownian particle. Thus, for a finite
magnetic field, the system exhibits a re-entrant type transition from active
to passive and then to active behaviour with persistence of elastic
dissipation in the medium. However, for a free particle, the overall
displacement covered always gets suppressed with increase in the strength of
the magnetic field and finally approaches zero for a very strong magnetic
field.

##  I INTRODUCTION

In the recent years, the study of active matter has received considerable
attention in different branches of science [1, 2, 3, 4, 5]. The term “active
matter” refers to a class of soft-matter systems where its individual
constituents are capable of making autonomous movement by consuming energy
from the environment. The motion governed by these constituent particles is
called as active Brownian motion or self-propelled motion. Active matter
systems are immensely pursued both theoretically as well as experimentally [6,
7, 8, 9, 10, 11, 12, 13, 14, 1]. In particular, the biological active matter
is significant because of its complex dynamics in crowded and stochastic
environments [11, 12]. Similarly, the artificially synthesized active matter
systems like collection of Janus particles [15, 16], living crystals [17], and
swarming robots [18] have wide applications in the areas of robotics [19],
material science, and nano-medicines [20].

From the theoretical perspective, a variety of models have been used to
describe the dynamics of active matter. Among them, the active Brownian
particle (ABP) model [21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34]
and active Ornstein-Uhlenbeck particle (AOUP) model [35, 36, 37, 38, 39, 40,
41, 42, 43, 44, 45] are the two most successful ones in explaining many
interesting aspects of the self-propelling systems [46, 3, 47]. In most of
these studies, the particles are considered to be suspended in a Newtonian
bath with an instantaneous friction and with no memory effect of the medium or
environment. However, in many situations, the particles are exposed to non-
Newtonian environments like polymer solutions [48], crystalline domain [49],
and biological surroundings such as mucus [50], tissues etc with finite memory
effect of the medium or environment. The natural active systems are mostly
non-Newtonian and are often found to be viscoelastic in nature with a
transient elasticity in the medium. Such systems can be modeled using the
generalized Langevin equation which includes the delayed memory effects of the
medium in the viscous term. Therefore, it is crucial to study the viscoelastic
effects on the dynamics of such systems as the memory effect of the medium
cannot be ignored [51, 52, 53, 54, 55, 56].

Active Brownian particle in a Newtonian bath, when subjected to an external
magnetic field exhibits exotic transient and steady state properties [57, 58,
59, 60, 61, 62]. In our previous work, we observed that a harmonically
confined active Ornstein-Uhlenbeck particle (AOUP), while self-propelling in a
viscoelastic environment in the presence of a magnetic field features a novel
phase transition from diamagnetic to paramagnetic phase [62]. In the present
work, we employ this model to investigate the transport behaviour of an
inertial active OU particle in a viscoelastic suspension characterized by an
exponentially decaying memory kernel in the viscous drag. The particle is
charged and subjected to a constant magnetic field perpendicular to the plane
of its motion [63, 64, 65, 66, 67]. We simulate the dynamics and obtain
instantaneous trajectories of both free and harmonic particle. From the exact
solution of the dynamics, we derive the analytical results for the mean
displacement (MD) and MSD. Here, we consider the corresponding time scales of
activity or self propulsion and elastic dissipation not to be same so that the
generalized fluctuation dissipation relation(GFDR) breaks down. Depending on
the complex interplay of these two time scales, our analysis reveal that the
steady state response of a harmonically confined particle to the magnetic
field is interesting and it shows opposite behaviour in two different
circumstances. In particular, when the persistence duration of activity in the
medium dominates over elastic dissipation, the steady state MSD shows
enhancement and when the persistence of elastic dissipation dominates over
activity or self propulsion, it shows suppression with increase in strength of
magnetic field. Moreover, for a very strong magnetic field, the MSD approaches
the equilibrium value at which the GFDR holds good and becomes independent of
both these time scales, reflecting the passive nature of the particle.

In sec II, we introduce our model, define the physical parameters of interest
and explain the simulation details. In sec. III, we discuss our results and
findings both from simulation and analytical calculation, which are followed
by a summary in sec. IV.

##  II MODEL

Figure 1: Simulated trajectory of a free particle for two different values of
$\omega_{c}$ in (a) and (d), for two different values of $t_{c}^{\prime}$ in
(b) and (e), and for two different values of $t_{c}$ in (c) and (f),
respectively. The color map denotes the time evolution of trajectory.
$t_{c}=t_{c}^{\prime}$ is considered as unity in (a) and (d),
$\omega_{c}=t_{c}$ is considered as unity in (b) and (e), and
$\omega_{c}=t_{c}^{\prime}$ is considered as unity in (c) and (f),
respectively. The other common parameters are $\Gamma=1$, $v_{0}=1$, $z_{0}=0$
and $D=1$.

We consider the motion of an inertial active Ornstein-Uhlenbeck particle
suspended in a viscoelastic suspension. The particle is charged and
constrained to move in a two-dimensional(2D) harmonic potential
$U(x,y)=\frac{1}{2}k\left(x^{2}+y^{2}\right)$, with $k$ as the harmonic
constant. Its motion is coupled to the presence of a constant magnetic field
${\bf B}=B\hat{z}$ applied perpendicular to the plane of motion. The dynamics
is governed by the generalized Langevin’s equation [62, 56] of motion

$\begin{split}m\ddot{\bf
X}(t)=&-\int_{0}^{t}f\left(t-t^{\prime}\right){\bf\dot{X}}(t^{\prime})\,dt^{\prime}+\frac{|q|B}{c}\sigma\cdot{\bf\dot{X}}(t)\\\
&-k{\bf X}(t)+\sqrt{D}{\mathbf{\xi}}(t).\end{split}$ (1)

Here, ${\bf X}=\begin{pmatrix}x\\\ y\end{pmatrix}$ refers to the position, $m$
stands for the mass and $\ddot{\bf X}(t)$ is the acceleration of the particle.
The first term in right hand side of Eq. (1) is the drag force, that arises
due to the interaction of the particle with the surrounding medium. Because of
the elastic nature of the medium, the dynamics of the particle possess finite
memory and is non-Markovian. Hence the drag force at particular instant of
time not only depend on the velocity at that instant of time but also it
depends on the velocities at previous times. Therefore, the drag force at a
certain time $t$ is dependent on a friction kernel of the form

$f\left(t-t^{\prime}\right)=\frac{\gamma}{2t_{c}^{\prime}}e^{-\frac{t-t^{\prime}}{t_{c}^{\prime}}},\qquad
t\geq t^{\prime}$ (2)

with $\gamma$ being the friction or viscous coefficient of the medium. The
friction kernel gives maximum contribution to the instantaneous velocity, and
its contribution to the past velocities decays exponentially with a rate
$\frac{1}{t_{c}^{\prime}}$. $t_{c}^{{}^{\prime}}$ is the time up to which the
elastic dissipation sustains in the medium and for $t>t_{c}^{{}^{\prime}}$,
the viscous dissipation dominates. In $t_{c}^{\prime}\to 0$ limit, the
friction kernel becomes delta correlated and the system is left only with
viscous dissipation[61]. The friction kernel $f(t-t^{\prime})$ follows the
Maxwellian viscoelastic formalism as per which the elastic force of fluid
completely relax down to zero for sufficiently long interval of time and the
suspension becomes viscous in nature [62]. $\sigma[=\begin{pmatrix}0&-1\\\
1&0\end{pmatrix}]$ is an arbitrary matrix and
$\xi(t)[=\begin{pmatrix}\xi_{x}(t)\\\ \xi_{y}(t)\end{pmatrix}]$ represents the
Ornstein-Uhlenbeck noise followed by the moments

$\langle\xi_{\alpha}(t)\rangle=0,\qquad\langle\xi_{\alpha}(t)\xi_{\beta}(t^{\prime})\rangle=\frac{\delta_{\alpha\beta}}{2t_{c}}e^{-\frac{|t-t^{\prime}|}{t_{c}^{\prime}}}.$
(3)

Here, $(\alpha,\beta)\;\epsilon\;(X,Y)$ and $D$ is the strength of the noise.
$t_{c}$ represents the noise correlation time or activity time of the
dynamics. It is the time up to which the particle has a tendency to execute
self-propulsion in the medium along a specific direction even with random
collisions. The noise correlation in the dynamics decays exponentially with
the time constant $t_{c}$. For a finite $t_{c}$ value, the system is in
nonequilibrium even in the absence of memory kernel. However, the system
approaches thermal equilibrium only when both the activity time scale and
viscoelastic time scale are of equal order ($t_{c}=t_{c}^{{}^{\prime}}$) and
$D=2\gamma K_{B}T$. This is the condition at which the GFDR is validated.
Because of the linearity of the model, one can solve Eq. (1) using Laplace
transform method formalism. Introducing a complex number $z=x+iy$, Eq. (1) can
be written as

$\ddot{z}(t)+\frac{1}{m}\int_{0}^{t}f(t-t^{\prime})\dot{z}(t^{\prime})\,dt^{\prime}-i\omega_{c}\dot{z}(t)+\omega_{0}^{2}z(t)=\epsilon(t),$
(4)

where $\omega_{c}=\frac{|q|B}{mc}$, $\omega_{0}=\sqrt{\frac{k}{m}}$, and

$\epsilon(t)=\frac{\sqrt{D}}{m}[\xi_{x}(t)+i\xi_{y}(t)].$

The mean displacement (MD) can be obtained using the general definition as

$\langle R(t)\rangle=\langle z(t)-z(0)\rangle.$ (5)

Similarly, the MSD can be obtained as

$\langle R^{2}(t)\rangle=\langle|z(t)-z(0)|^{2}\rangle.$ (6)

Inorder to simulate the dynamics, Eq. (1) can be expressed in a Markovian form
by introducing the variable ${\bf U}$ such that

${\bf U}(t)=\int_{0}^{t}f\left(t-t^{\prime}\right){\bf
V}(t^{\prime})\,dt^{\prime}.$ (7)

Then Eq. (1) can be written as

$\displaystyle\dot{X}$ $\displaystyle=V$ (8) $\displaystyle m\dot{\bf V}$
$\displaystyle=-{\bf U}+\frac{|q|B}{c}\sigma\cdot{\bf V}-k{\bf
X}+\sqrt{D}{\mathbf{\xi}}$ (9) $\displaystyle\dot{{\bf U}}$
$\displaystyle=-\frac{{\bf
U}}{t_{c}^{\prime}}+\frac{\Gamma}{2t_{c}^{\prime}}{\bf V}$ (10)

This set of Markovian equations can be simulated using the Heun’s method
formalism[68]. The Ornstein-Uhlenbeck noise ($\xi$) along with the properties
given in Eq. (3) is implemented using Fox method approach as described in Ref.
[69]. The simulation is run for $10^{3}$s with a time step of $10^{-3}$s. The
steady state measurements are taken averaging over $10^{5}$ realizations and
after ignoring the initial transients of $10^{2}$s for each realization.

##  III RESULTS AND DISCUSSION

###  III.1 Free particle in the presence of magnetic field

For a free particle in the presence of magnetic field, the generalized
Langevin equation of motion [Eq. (4)] reduces to

$\ddot{z}(t)+\frac{1}{m}\int_{0}^{t}f(t-t^{\prime})\dot{z}(t^{\prime})\,dt^{\prime}-i\omega_{c}\dot{z}(t)+=\epsilon(t).$
(11)

By taking the initial conditions $z(0)=z_{0}$ and $\dot{z}(0)=v_{0}$, and
using the Laplace transform method the solution of Eq. (11) can be obtained as

$z(t)=z_{0}+\sum_{i=0}^{2}a_{i}\int_{0}^{t}e^{s_{i}(t-t^{\prime})}\xi(t^{\prime})\,dt^{\prime}+\sum_{i=0}^{2}a_{i}v_{0}e^{s_{i}t},$
(12)

where $s_{i}$’s are given by
$s_{0}=0$ and

$\begin{split}s_{1,2}=-\frac{1}{2}\biggr{[}+\frac{1}{t_{c}^{\prime}}-i\omega_{c}\pm\frac{\sqrt{1-t_{c}^{\prime}\left(2\Gamma+\omega_{c}\left(\omega_{c}t_{c}^{\prime}-2i\right)\right)}}{t_{c}^{\prime}}\biggr{]}.\end{split}$
(13)

Here, $\Gamma=\frac{\gamma}{m}$ and the coefficients $a_{i}$’s are

$\begin{split}&a_{0}=\frac{1}{s_{1}s_{2}t_{c}^{\prime}}\quad,a_{1}=\frac{s_{1}t_{c}^{\prime}+1}{s_{1}\left(s_{1}-s_{2}\right)t_{c}^{\prime}}\quad\text{and}\\\
&a_{2}=\frac{s_{2}t_{c}^{\prime}+1}{s_{2}\left(s_{2}-s_{1}\right)t_{c}^{\prime}}\end{split}$
(14)

In Fig.1, We have shown the simulated particle trajectories for three
different cases. Figs. 1 (a) and (d) represent the particle trajectories for a
low and high strengthen magnetic field ($\omega_{c}$). It is noticed that
under the influence of magnetic field the particle deviates from the motion
along x-direction and gets dragged towards y-direction and has a tendency to
make circular trajectories. When $\omega_{c}$ is large, these circular
trajectories get squeezed and particle suffers a kind of trapping or
suppression. As a consequence, the overall displacement of the particle
reduces. This can be interpreted from Fig. 1 (d). In Figs. 1(b) and (e), the
particle trajectories are plotted for a small and large value
$t_{c}^{{}^{\prime}}$. It is observed that with increase in
$t_{c}^{{}^{\prime}}$, the particle trajectories get smoother and the particle
makes more number of turns. At the same time on average it covers larger
distances before completing the turns. Because of which, the circular
trajectories get enhanced initially and subsequently reduces displaying a
spiral kind movement. Similarly, the simulated particle trajectories for a
small and large value of $t_{c}$ are shown in Figs. 1 (c) and (f),
respectively. With increase in $t_{c}$ value, the particle makes more directed
motion rather than making circular trajectories. At the same time, the
particle becomes more confined, that is why the overall displacement decreases
with longer persistent of activity in the medium.

Using Eq. (5), MD for a free particle in the presence of magnetic field is
calculated as

$\begin{split}\langle R(t)\rangle^{(f)}&=\langle z(t)-z(0)\rangle\\\
&=a_{0}v_{0}+a_{1}v_{0}e^{s_{1}t}+a_{2}v_{0}e^{s_{2}t}.\end{split}$ (15)

Figure 2: The 2D parametric plots of MD [Eq. (15)] is shown for different
values of $t_{c}^{\prime}$ in (a), for different values of $\omega_{c}$ in
(b), and for different values of $\Gamma$ in (c), respectively. The dashed
lines in (b) and (c) represent the 2D evolution of MD with $\omega_{c}$ and
$\Gamma$, respectively. Inset of (a) shows the evolution of MD for
$t_{c}^{\prime}=20$ along with the dotted circle representing MD in the
$t_{c}^{\prime}\to\infty$ limit. Other common parameters are $D=1$ and
$\omega_{c}=1$, $\Gamma=1$ in (a), $t_{c}^{\prime}=1$, $\Gamma=1$ in (b) and
$t_{c}^{\prime}=1$, $\omega_{c}=1$ in (c).

Expanding Eq. (15) around $t=0$ in terms of powers of $t$, we obtain MD in the
transient regime as

$\langle
R(t)\rangle^{(f)}=v_{0}t+\frac{i}{2}v_{0}\omega_{c}t^{2}+O\left(t^{3}\right).$
(16)

In the time asymptotic limit ($t\rightarrow\infty$) or at steady state,
$\langle R(t)\rangle$ reduces to

$\langle R\rangle^{(f)}_{st}=\lim_{t\to\infty}\langle
R(t)\rangle^{(f)}=\frac{2v_{0}\left(\Gamma+i2\omega_{c}\right)}{4\omega_{c}^{2}+\Gamma^{2}}.$
(17)

Hence, it is confirmed that the steady state MD has dependence neither on the
elastic dissipation time nor on noise correlation time or activity time,
rather it has dependence on the magnetic field as well as on viscosity of the
medium or inertial time scale. The 2D time evolution of MD [$\langle
y(t)\rangle$ vs $\langle x(t)\rangle$] for different values of
$t_{c}^{\prime}$, for different values of $\omega_{c}$, and for different
values of $\Gamma$, are shown in Figs. 2 (a), (b) and (c), respectively. For a
given set of parameters, the MD evolves with time and saturates to a constant
value in the long time limit, displaying a spira mirabilis kind trajectory
[55]. From Fig. 2 (a), it is noticed that MD has a dependence on
$t_{c}^{{}^{\prime}}$ in the transient regime whereas the steady state MD is
independent of $t_{c}^{{}^{\prime}}$. Further, it is noteworthy that with
increase in $t_{c}^{{}^{\prime}}$ value, the particle on average takes longer
time and covers larger distances before approaching the steady state at which
the MD is saturated to a constant value. This fact concludes a kind of delay
in the particle motion for approaching steady state because of the presence of
memory in the viscous kernel and it represents a memory induced delay in the
dynamics. Further, in $t_{c}^{\prime}\rightarrow 0$ limit, the $\langle
R(t)\rangle^{(f)}$ reduces to

$\langle R(t)\rangle^{(f)}=\frac{2v_{0}}{\Gamma-2i\
\omega_{c}}\left(1-e^{-\frac{\Gamma t}{2}+i\omega_{c}\ t}\right),$ (18)

Figure 3: Particle trajectories for two different $\omega_{c}$ values and the
corresponding steady state probability distribution functions are shown in
(A), (B) and (C) for three different parameter regimes $t_{c}<t_{c}^{\prime}$,
$t_{c}=t_{c}^{\prime}$ and $t_{c}>t_{c}^{\prime}$, respectively. Particle
trajectories in (a), (d) and (g) are for $\omega_{c}=0.01$ and in (b), (e) and
(h) for $\omega_{c}=5.0$, respectively. For taking the trajectories,
simulation is run for $t=100\;s$. The corresponding steady state position
distributions are shown in (c), (f) and (i). The other common parameters are
$\Gamma=1.0$, $m=1.0$, $\omega_{0}=1.0$ ,$v_{0}=1$, $z_{0}=0$, and $D=1$.

which represents a perfect spira mirabillis, familiar for active particles in
Newtonian fluids [55]. However, with increase in $t_{c}^{\prime}$ circular
motions are more stabilized and in $t_{c}^{\prime}\rightarrow\infty$ limit,
$\langle R(t)\rangle^{(f)}$ takes the form of a perfect circle of radius
$\frac{|v_{0}|^{2}}{\omega_{c}^{2}}$ centred at $\frac{iv_{0}}{\omega_{c}}$.
The same is reflected from the inset of Fig. 2 (a), where
$t_{c}^{{}^{\prime}}$ is taken as $20$. With increase in $\omega_{c}$ value,
the radius of the circular trajectories decreases and also centre of the
circle shifts towards zero value, reflecting kind of trapping of the particle
for high magnetic field [see Fig. 2 (b)].

The 2D evolution of steady state MD with $\omega_{c}$ and $\Gamma$ are shown
along with their corresponding parametric plots in Figs. 2 (b) and (c),
respectively. In the absence of magnetic field ($\omega_{c}=0$), the steady
state MD is simply $\frac{2v_{0}}{\Gamma}$. With increase in $\omega_{c}$, it
decreases in the x-direction and increases in the y-direction and finally
approaches to zero value for very large $\omega_{c}$ [Fig. 2 (b)]. Similarly,
for low value of $\Gamma$, where inertial influence is significant, the
particle on average takes longer time to approach steady state and covers
larger distances. However, the steady state is approached faster with increase
in $\Gamma$ values as expected. The steady state MD starts from the value
$\frac{iv_{0}}{\omega_{c}}$ for $\Gamma=0$ and approaches zero for very large
$\Gamma$ value [Fig. 2 (c)], where the inertial influence is negligibly small.
This provides a delay in the dynamical behaviour of the particle because of
inertia in the dynamics.

Using Eq. (6), we have exactly calculated the MSD in both transient and time
asymptotic limit. In the time asymptotic limit ($t\rightarrow\infty$) or at
steady state, the MSD is obtained as

$\langle
R^{2}\rangle^{(f)}_{st}=\frac{8Dt}{m^{2}\left(\Gamma^{2}+4\text{{\hbox{\omega_c}}}^{2}\right)}.$
(19)

From this expression, it is confirmed that $\langle R^{2}\rangle^{(f)}_{st}$
depends neither on $t_{c}^{{}^{\prime}}$ nor on $t_{c}$. Hence, the overall
distance covered by a free particle at steady state is completely independent
of the underlying physical processes of the medium. However, it has dependence
on $\omega_{c}$ and $\Gamma$. It varies inversely both with $\omega_{c}$ and
$\Gamma$. From the transient behaviour of MSD, it is confirmed that motion is
ballistic in the initial transient regimes as $\langle R^{2}\rangle\propto
t^{2}$ and it is diffusive in long time regimes since $\langle
R^{2}\rangle\propto t$. The intermediate time regimes of MSD represents kind
of oscillatory behaviour, indicating the anomalous diffusion of the particle.

###  III.2 Confined harmonic particle in the presence of magnetic field

For a harmonically confined particle, the solution of the dynamics [Eq. (4)]
is given by

$z(t)=\sum_{i=1}^{3}\left[b_{i}z_{0}e^{s_{i}t}+a_{i}v_{0}e^{s_{i}t}+a_{i}\int_{0}^{t}e^{s_{i}(t-t^{\prime})}\xi(t^{\prime})\,dt^{\prime}\right].$
(20)

Here, $s_{i}$’s are the solution of the cubic equation

$t_{c}^{\prime}s^{3}+(1-j\omega_{c}t_{c}^{\prime})s^{2}+\left(\frac{\Gamma}{2}-j\omega_{c}+\omega_{0}^{2}t_{c}^{\prime}\right)s+\omega_{0}^{2}=0.$
(21)

$a_{i}$’s and $b_{i}$’s are given by the matrix equations

$S\cdot A=\begin{pmatrix}0\\\ -1\\\
\frac{1}{t_{c}^{\prime}}\end{pmatrix}\quad\text{and}\quad S\cdot
B=\begin{pmatrix}1\\\ -\frac{1}{t_{c}^{\prime}}+j\omega_{c}\\\
\frac{\Gamma}{2t_{c}^{\prime}}-\frac{j\omega_{c}}{t_{c}^{\prime}}\end{pmatrix},$
(22)

with $A=\left[a_{i}\right]_{3\times 1}$ and $B=\left[b_{i}\right]_{3\times
1}$. The matrix $S$ is given by

$S=\begin{pmatrix}1&1&1\\\ s_{2}+s_{3}&s_{1}+s_{3}&s_{1}+s_{2}\\\
s_{2}s_{3}&s_{1}s_{3}&s_{1}s_{2}\end{pmatrix}.$ (23)

Figure 4: For a harmonically confined particle ($\omega_{0}=1$), the 2D
parametric plots of MD ($\langle y\rangle$ vs $\langle x\rangle$) [Eq. (24)]
are shown for $t_{c}^{\prime}=0.5$ in (a) and $t_{c}^{\prime}=5.0$ in (b), (c)
and (d), respectively. The color map shows the evolution of MD with time. The
other common parameters are $\Gamma=1.0$, $\omega_{c}=1.0$, $m=1.0$,
$v_{0}=1$, $z_{0}=0$ and $D=1$.

In order to understand the viscoelastic response of the particle to the
magnetic field, we have shown the simulated particle trajectories of the
dynamics [[Eq. (4)]] and the corresponding steady state position distributions
in Fig. 3. In harmonic confinement, the particle trajectories are circular and
show interesting features with magnetic field by the complex interplay between
the corresponding time scales of elastic dissipation and active fluctuations
in the medium. If the elastic dissipation persists for longer interval of time
as compared to the duration of activity or self propulsion, i.e., for
$t_{c}^{{}^{\prime}}>t_{c}$ [see Fig. 3 (a), (b) and (c)], the directed motion
gets suppressed with increase in $\omega_{c}$ and the particle becomes
incapable of taking advantage from the influence of magnetic field in making
circular trajectories. As a consequence, the circular trajectories get
suppressed and the particle gets more confined towards the mean position of
the well with increase in $\omega_{c}$ [Fig. 3 (b)]. The steady state position
distribution becomes sharp across the centre or mean position with increase in
$\omega_{c}$, deliberating the confinement of the particle with magnetic field
[see Fig. 3 (c)]. If the activity persists for longer duration as compared to
the elastic dissipation in the medium, i.e., for $t_{c}>t_{c}^{{}^{\prime}}$
[see Fig. 3 (g), (h) and (i)], the trajectories comparatively get enhanced
with increase in $\omega_{c}$. The particle as if moves away from the centre
initially and on average covers larger distances before coming back to the
mean position of the well. That is why, the steady state position distribution
gets broader and the peak of the distribution gets suppressed with increase in
$\omega_{c}$, reflected from Fig. 3 (i).

However, if both elastic dissipation and activity persists almost for equal
interval of time in the medium, i.e., for $t_{c}^{{}^{\prime}}=t_{c}$, the
steady state is not affected by the presence of magnetic field [see Fig. 3
(d), (e) and (f)] and that is why the distribution shows same behaviour for
both small and large $\omega_{c}$. In this limit, the generalized fluctuation
dissipation relation (GFDR) is satisfied and the system approaches
equilibrium. The MD [Eq. (5)] for a confined harmonic particle can be
calculated as

$\langle R(t)\rangle^{(h)}=\langle
z(t)-z(0)\rangle=\sum_{i=1}^{3}\left[b_{i}z_{0}e^{s_{i}t}+a_{i}v_{0}e^{s_{i}t}\right].$
(24)

Expanding MD around $t=0$, we get

$\langle R(t)\rangle^{(h)}=v_{0}t+\frac{1}{2}\left(-\frac{\Gamma
z_{0}}{2t_{c}^{\prime}}+iv_{0}\omega_{c}-z_{0}\omega_{0}^{2}\right)t^{2}+O\left(t^{3}\right).$
(25)

In the time asymptotic limit, $\langle R(t)\rangle^{(h)}$ approaches zero
value implying that the particle comes back to the mean position of the well.
In Fig. 4, we have plotted the 2D parametric plot for the time evolution of MD
for different values of $t_{c}^{{}^{\prime}}$. For $t_{c}^{{}^{\prime}}=0.5$,
the particle almost take $100$ secs to approach steady state and come back to
the mean position of the well as shown in Fig. 4(a). However, with increase in
$t_{c}^{{}^{\prime}}$, the particle motion gets delayed and $100secs$ is not
enough for the particle to come back to the mean position as in Fig. 4(b).
With further increase in time, the particle approaches towards steady state
[see Fig. 4(c)] and finally for very long time, the particle comes back
exactly to the mean position of the well [see Fig. 4(d)]. An interesting
feature of this behaviour is that the particle motion gets delayed due to the
longer persistence of elastic dissipation in the medium, confirming a kind of
delay in the dynamics due to presence of memory.

Using [Eq. (6)], the MSD for harmonically confined particle can be calculated
as

$\begin{split}\langle R^{2}(t)\rangle^{(h)}=&\left|\langle
R(t)\rangle^{(h)}\right|^{2}+\sum_{i=1}^{3}\sum_{j=1}^{3}\frac{a_{i}a_{j}^{*}D}{m^{2}}\Biggl{[}\frac{t_{c}e^{t\left(s^{*}_{j}-\frac{1}{t_{c}}\right)}}{\left(t_{c}s_{i}+1\right)\left(1-t_{c}s^{*}_{j}\right)}+\frac{t_{c}e^{t\left(s_{i}-\frac{1}{t_{c}}\right)}}{\left(1-t_{c}s_{i}\right)\left(t_{c}s^{*}_{j}+1\right)}\\\
&+\frac{t_{c}\left(s_{i}+s^{*}_{j}\right)-2}{\left(s_{i}+s^{*}_{j}\right)\left(t_{c}s_{i}-1\right)\left(t_{c}s^{*}_{j}-1\right)}+\frac{e^{t\left(s_{i}+s^{*}_{j}\right)}\left(t_{c}\left(s_{i}+s^{*}_{j}\right)+2\right)}{\left(s_{i}+s^{*}_{j}\right)\left(t_{c}s_{i}+1\right)\left(t_{c}s^{*}_{j}+1\right)}\Biggr{]}.\end{split}$
(26)

In Fig. 4, we have shown the plot of MSD as a function of $t$ for different
values of $\omega_{c}$ in three different regimes, i.e., for
$t_{c}<t_{c}^{\prime}$, $t_{c}=t_{c}^{\prime}$ and for $t_{c}>t_{c}^{\prime}$.
The MSD reflects the ballistic feature of the motion in the lower time regime
as $\langle R^{2}(t)\rangle^{(h)}\propto t^{2}$ and non-diffusive motion in
the long time limit since $\langle R^{2}(t)\rangle^{(h)}$ saturates to a
constant value. In the intermediate regime of time, $\langle
R^{2}(t)\rangle^{(h)}$ shows an oscillatory behaviour. The initial ballistic
regimes always get suppressed with increase in $\omega_{c}$ irrespective of
the value of $t_{c}$ and $t_{c}^{\prime}$. However, the steady state MSD with
$\omega_{c}$ shows rather complicated features due to the interplay between
the physical processes of elastic dissipation and active fluctuations in the
medium. It decreases with increase in $\omega_{c}$ when $t_{c}<t_{c}^{\prime}$
[Fig. 5(a)], becomes independent of $\omega_{c}$ when $t_{c}=t_{c}^{\prime}$
[Fig. 5(b)], and increases with $\omega_{c}$ when $t_{c}>t_{c}^{\prime}$ [Fig.
5(c)]. Moreover, for a very large $\omega_{c}$ value, MSD approaches to the
value in the equilibrium limit, where GFDR is satisfied, irrespective of both
these time scales $t_{c}$ and $t_{c}^{\prime}$[Fig. 5(d)] . In order to
understand these interesting feature of steady state MSD, we have exactly
calculated MSD in the stationary state. Because of the linearity of the model
and Gaussian nature of noise, the steady state probability distribution is
Gaussian and can be computed using the steady state correlation matrix method
[39]. By introducing a correlation matrix $\Xi$, the steady state probability
distribution can be expressed as

$P(w)=\frac{1}{(2\pi)^{4}\sqrt{|\Xi|}}\exp\left[{-\frac{1}{2}\left(w^{T}\Xi^{-1}w\right)}\right],$
(27)

with $w$ being a column vector. The exact form of $w$ is given in Appendix A.
The elements of $\Xi$ are given by

$\left[\Xi_{i,j}\right]=\langle w_{i}w_{j}\rangle-\langle w_{i}\rangle\langle
w_{j}\rangle.$ (28)

We can find the solution for $\Xi$ using correlation matrix method formalism
as described in Appendix A. From the elements of $\Xi$, the steady state MSD
($\langle R^{2}\rangle^{(h)}_{st}$) can be obtained as

$\langle R^{2}\rangle^{(h)}_{st}=\Xi_{1,1}+\Xi_{2,2}.$ (29)

By simplifying the above equation, $\langle R^{2}\rangle^{(h)}_{st}$ can be
expressed in the form

$\langle R^{2}\rangle^{(h)}_{st}=\langle R^{2}\rangle_{E}+\langle
R^{2}\rangle_{NE},$ (30)

where $\langle R^{2}\rangle_{E}$ and $\langle R^{2}\rangle_{NE}$ are the
equilibrium and nonequilibrium contribution of MSD. $\langle R^{2}\rangle_{E}$
is given by the expression

$\langle R^{2}\rangle_{E}=\frac{2D}{m^{2}\Gamma\omega_{0}^{2}}.$ (31)

For $D=2\gamma k_{B}T$, $\langle R^{2}\rangle_{E}$ represents the MSD of a
passive Brownian particle confined in a harmonic potential with $T$ as the
temperature of the bath. The expression for $\langle R^{2}\rangle_{NE}$ is
given by

$\langle
R^{2}\rangle_{NE}=\frac{4D(t_{c}^{\prime}-t_{c})(t_{c}+t_{c}^{\prime})^{2}\left[2t_{c}^{\prime}+t_{c}(2+t_{c}\Gamma+2t_{c}(t_{c}+t_{c}^{\prime})\omega_{0}^{2})\right]}{m^{2}\Gamma\left\\{\left[2t_{c}^{\prime}+t_{c}(2+t_{c}\Gamma+2t_{c}(t_{c}+t_{c}^{\prime})\omega_{0}^{2})\right]^{2}+4t_{c}^{2}(t_{c}+t_{c}^{\prime})^{2}\omega_{c}^{2}\right\\}}.$
(32)

From Eq. (32), it is confirmed that $\langle R^{2}\rangle_{NE}$ has a
dependence on both $t_{c}$ and $t_{c}^{{}^{\prime}}$. It is to be noted that
$\langle R^{2}\rangle_{NE}$ becomes zero when $t_{c}$ becomes equal to
$t_{c}^{{}^{\prime}}$. In this limit, we consider $D=2\gamma k_{B}T$ for GFDR
to be validated so that the system is in equilibrium. Also $\langle
R^{2}\rangle_{NE}$ vanishes in the presence of a very strong magnetic field
since $\lim_{\omega_{c}\to\infty}\langle R^{2}\rangle_{NE}=0$.

In both the above two cases, $\langle R^{2}\rangle^{(h)}_{st}$ simply becomes
$\langle R^{2}\rangle_{E}$ which is independent of both $t_{c}$ and
$t_{c}^{\prime}$. However, Eq. (30) indicate three interesting features of
$\langle R^{2}\rangle^{(h)}_{st}$ in the $t_{c}$-$t_{c}^{\prime}$ parameter
space, i.e., for $t_{c}<t_{c}^{\prime}$, $t_{c}=t_{c}^{\prime}$, and
$t_{c}>t_{c}^{\prime}$ regimes, respectively [Fig. 5]. For the parameter
regime with $t_{c}<t_{c}^{\prime}$, $\langle R^{2}\rangle_{NE}$ in Eq. (32) is
positive and hence $\langle R^{2}\rangle^{(h)}_{st}$ is always greater than
$\langle R^{2}\rangle_{E}$. Thus, for the regime with $t_{c}<t_{c}^{\prime}$,
since $\langle R^{2}\rangle_{NE}$ vanishes in $\omega_{c}\rightarrow\infty$
limit, the suppression of $\langle R^{2}\rangle^{(h)}_{st}$ with $\omega_{c}$
is obvious. That is why, it decreases with increase in $\omega_{c}$ and
approaches the equilibrium value in $\omega_{c}\rightarrow\infty$ limit. On
the other hand, for the parameter regime with $t_{c}>t_{c}^{\prime}$, $\langle
R^{2}\rangle_{NE}$ is negative and hence $\langle R^{2}\rangle^{(h)}_{st}$ is
less than its equilibrium value. Thus, enhancement of $\langle
R^{2}\rangle^{(h)}_{st}$ with $\omega_{c}$ is expected and that is what we
notice in Fig. 5(d) that $\langle R^{2}\rangle^{(h)}_{st}$ increases with
increase in $\omega_{c}$ and saturates at the equilibrium value in the
$\omega_{c}\rightarrow\infty$ limit. However, for the regime with
$t_{c}=t_{c}^{\prime}$, $\langle R^{2}\rangle_{NE}$ vanishes and the steady
state MSD is simply $\langle R^{2}\rangle_{E}$. It approaches the equilibrium
value for $D=2\gamma k_{B}T$ and becomes independent of $\omega_{c}$. In this
limit, the $\langle R^{2}\rangle^{(h)}_{st}$ is same as the case of a passive
Brownian particle. Hence, for a finite magnetic field, the system displays a
re-entrant type transition from active to passive and then to active behaviour
with the elastic dissipation time scale of the dynamics. Further in order to
understand the time dependence of $\langle R^{2}(t)\rangle^{(h)}$, we
introduce a parameter $\alpha$ such that $\langle R^{2}(t)\rangle\sim
t^{\alpha}$ and hence the time dependence of $\alpha$ can be expressed as

$\alpha=\frac{\partial\log\left(\langle R^{2}(t)\rangle\right)}{\partial\log
t}.$ (33)

The variation of $\alpha$ with $t$ is shown in Fig. 5(e). $\alpha=2$ confirms
the ballistic motion for initial time regimes and $\alpha=0$ implies the non-
diffusive motion for long time regimes. The oscillatory behaviour of $\alpha$
with $t$ in the intermediate time regimes indicate the anomalous diffusion of
the particle

Figure 5: For a harmonically confined particle ($\omega_{0}=1$), MSD as a
function of $t$ [Eq. (26)] for different values of $\omega_{c}$ are shown for
$t_{c}<t_{c}^{\prime}$ ($t_{c}=0.5$,$t_{c}^{\prime}=2$) in (a), for
$t_{c}=t_{c}^{\prime}=1$ in (b) and for $t_{c}>t_{c}^{\prime}$ ($t_{c}=2$,
$t_{c}^{\prime}=0.5$) in (c), respectively. The steady state MSD [$\langle
R\rangle^{(h)}_{st}$] [Eq. (30)] is plotted as a function of $\omega_{c}$ in
(d). The time variation of exponent ($\alpha$) is shown in (e) for
$t_{c}=t_{c}^{\prime}=1$. The other common parameters are $m=1.0$,
$\gamma=1.0$, $v_{0}=1$, $z_{0}=0$ and $D=1$.

##  IV CONCLUSION

In this work, we have investigated the dynamics of an inertial charged active
particle suspended in a viscoelastic medium with an exponentially decaying
viscous kernel. The particle is further subjected to an external magnetic
field of constant magnitude and in a direction perpendicular to the plane of
its motion. The dynamics of the particle is modelled using the generalized
Langevin equation with an exponentially correlated noise. We consider the
elastic dissipation time scale and the noise correlation time scales are not
of equal order. As a consequence, the GFDR is violated. By simulating the
generalized Langevin equation and with the help of analytical calculations, we
have exactly explored the transient as well as steady state response of both
free and harmonically confined particle to an uniform magnetic field. One
aspect of the transient dynamics is that the instantaneous mean displacement
has dependence on the elastic dissipation time scale for both free and
harmonic particles. The particle on average takes longer time to approach
steady state because of the presence of memory in the viscous kernel. This is
confirmed from the simulated particle trajectories as well as from the
parametric plots or mean displacement calculations.

However, the steady state response of a harmonically confined particle to a
magnetic field shows interesting behaviour by complex interplay between the
elastic dissipation time scale and activity or self propulsion time scale of
the dynamics. In particular, if the elastic dissipation persists for longer
interval of time as compared to the self propulsion or activity in the medium,
the steady state MSD decreases with increase in magnetic field. On the other
hand, if the self propulsion persists for longer time as compared to the
elastic dissipation, the steady state MSD increases with increase in the
magnetic field strength. In both the situations, for a very strong magnetic
field the steady state MSD becomes independent of magnetic field and
approaches the value at the equilibrium limit at which the GFDR is validated
and the particle behaves like an inertial passive Brownian particle.
Therefore, for a finite magnetic field, the system passes through a memory
induced re-entrant type transition from active to passive and then to active
behaviour. For a free particle, the steady state MSD always get reduced with
increase in strength of magnetic field and approaches zero for a very strong
magnetic field, reflecting field induced trapping of the particle. Finally, we
believe that all of our aforementioned results are amenable for experimental
verification. Our results are further important for controlling the self-
propulsion or active dynamics of a particle by fine tuning the external
magnetic field along with the physical properties of the suspension.

##  V Acknowledgement

M.S. acknowledges the start-up grant from UGC Faculty recharge program, Govt.
of India for financial support.

## Appendix A

For a harmonically confined particle, the dynamics [Eq. (1)] can be expressed
in the form of a vector equation

$\dot{w}=A\cdot w+B\cdot\eta,$ (34)

with $w$ being the column vector

$w=\begin{pmatrix}x&y&v_{x}&v_{y}&u_{x}&u_{y}&\xi_{x}&\xi_{y}\end{pmatrix}^{T}.$
(35)

Here, $v_{x}$ and $v_{y}$ are the $x$ and $y$ components of velocity. The
variables $u_{x}$ and $u_{y}$ are given by

$\begin{pmatrix}u_{x}\\\
u_{y}\end{pmatrix}=\int_{0}^{t}\Gamma\left(t-t^{\prime}\right){\bf\dot{X}}(t^{\prime})\,dt^{\prime}.\vspace{0.5em}$
(36)

The matrices $A$ and $B$ in Eq. (34) are coefficient matrix and noise matrix,
respectively. Using the correlation matrix method formalism, by introducing
correlation matrix $\Xi$ as defined in Eq. (28), the Eq. (34) can take a new
form

$A\cdot\Xi+\Xi\cdot A^{T}+B\cdot B^{T}=0.$ (37)

By solving Eq. (37), one can get the correlation matrix $\Xi$ with which the
probability distribution [Eq. (27)] can also be calculated. From the elements
of the correlation matrix $\Xi$, the steady state MSD is calculated.

## References

  * Bechinger  _et al._ [2016] C. Bechinger, R. Di Leonardo, H. Löwen, C. Reichhardt, G. Volpe, and G. Volpe, Active particles in complex and crowded environments, Rev. Mod. Phys. 88, 045006 (2016).
  * Ramaswamy [2017] S. Ramaswamy, Active matter, J. Stat. Mech. 2017, 054002 (2017).
  * Gompper  _et al._ [2020] G. Gompper, R. G. Winkler, T. Speck, A. Solon, C. Nardini, F. Peruani, H. Löwen, R. Golestanian, U. B. Kaupp, L. Alvarez, T. Kiørboe, E. Lauga, W. C. K. Poon, A. DeSimone, S. Muiños-Landin, A. Fischer, N. A. Söker, F. Cichos, R. Kapral, and e. a. P Gaspard, The 2020 motile active matter roadmap, J. Phys.: Condens. Matter 32, 193001 (2020).
  * Pietzonka [2021] P. Pietzonka, The oddity of active matter, Nat. Phys. 17, 1193 (2021).
  * De Magistris and Marenduzzo [2015] G. De Magistris and D. Marenduzzo, An introduction to the physics of active matter, Physica A 418, 65 (2015).
  * Dauchot and Démery [2019] O. Dauchot and V. Démery, Dynamics of a self-propelled particle in a harmonic trap, Phys. Rev. Lett. 122, 068002 (2019).
  * Bashirzadeh and Liu [2019] Y. Bashirzadeh and A. P. Liu, Encapsulation of the cytoskeleton: towards mimicking the mechanics of a cell, Soft Matter 15, 8425 (2019).
  * Scholz  _et al._ [2018a] C. Scholz, M. Engel, and T. Pöschel, Rotating robots move collectively and self-organize, Nat. Commun. 9, 931 (2018a).
  * Andac  _et al._ [2019] T. Andac, P. Weigmann, S. K. P. Velu, E. Pinçe, G. Volpe, G. Volpe, and A. Callegari, Active matter alters the growth dynamics of coffee rings, Soft Matter 15, 1488 (2019).
  * Makarchuk  _et al._ [2019] S. Makarchuk, V. C. Braz, N. A. M. Araújo, L. Ciric, and G. Volpe, Enhanced propagation of motile bacteria on surfaces due to forward scattering, Nat. Commun. 10, 4110 (2019).
  * Taktikos  _et al._ [2014] J. Taktikos, H. Stark, and V. Zaburdaev, How the motility pattern of bacteria affects their dispersal and chemotaxis, PLOS ONE 8, 1 (2014).
  * Aranson [2022] I. S. Aranson, Bacterial active matter, Rep. Prog. Phys. 85, 076601 (2022).
  * Needleman and Dogic [2017] D. Needleman and Z. Dogic, Active matter at the interface between materials science and cell biology, Nat. Rev. Mater. 2, 17048 (2017).
  * Vernerey  _et al._ [2019] F. Vernerey, E. Benet, L. Blue, A. Fajrial, S. Lalitha Sridhar, J. Lum, G. Shakya, K. Song, A. Thomas, and M. Borden, Biological active matter aggregates: Inspiration for smart colloidal materials, Adv. Colloid Interface Sci. 263, 38 (2019).
  * Walther and Müller [2013] A. Walther and A. H. E. Müller, Janus particles: Synthesis, self-assembly, physical properties, and applications, Chem. Rev. 113, 5194 (2013).
  * Howse  _et al._ [2007] J. R. Howse, R. A. Jones, A. J. Ryan, T. Gough, R. Vafabakhsh, and R. Golestanian, Self-motile colloidal particles: from directed propulsion to random walk, Phys. Rev. Lett. 99, 048102 (2007).
  * Palacci  _et al._ [2013] J. Palacci, S. Sacanna, A. P. Steinberg, D. J. Pine, and P. M. Chaikin, Living crystals of light-activated colloidal surfers, Science 339, 936 (2013).
  * Wang  _et al._ [2021] G. Wang, T. V. Phan, S. Li, M. Wombacher, J. Qu, Y. Peng, G. Chen, D. I. Goldman, S. A. Levin, R. H. Austin, and L. Liu, Emergent field-driven robot swarm states, Phys. Rev. Lett. 126, 108002 (2021).
  * Aguilar  _et al._ [2016] J. Aguilar, T. Zhang, F. Qian, M. Kingsbury, B. McInroe, N. Mazouchova, C. Li, R. Maladen, C. Gong, M. Travers, R. L. Hatton, H. Choset, P. B. Umbanhowar, and D. I. Goldman, A review on locomotion robophysics: the study of movement at the intersection of robotics, soft matter and dynamical systems, Rep. Prog. Phys. 79, 110001 (2016).
  * Ghosh  _et al._ [2020] A. Ghosh, W. Xu, N. Gupta, and D. H. Gracias, Active matter therapeutics, Nano Today 31, 100836 (2020).
  * ten Hagen  _et al._ [2009] B. ten Hagen, S. van Teeffelen, and H. Lowen, Non-gaussian behaviour of a self-propelled particle on a substrate, Condens. Matter Phys. 12, 725 (2009).
  * ten Hagen  _et al._ [2011] B. ten Hagen, S. van Teeffelen, and H. Löwen, Brownian motion of a self-propelled particle, J. Phys.: Condens. Matter 23, 194119 (2011).
  * Cates and Tailleur [2013] M. E. Cates and J. Tailleur, When are active brownian particles and run-and-tumble particles equivalent? consequences for motility-induced phase separation, Euro. Phys. Lett. 101, 20010 (2013).
  * Malakar  _et al._ [2020] K. Malakar, A. Das, A. Kundu, K. V. Kumar, and A. Dhar, Steady state of an active brownian particle in a two-dimensional harmonic trap, Phys. Rev. E 101, 022610 (2020).
  * Buttinoni  _et al._ [2013] I. Buttinoni, J. Bialké, F. Kümmel, H. Löwen, C. Bechinger, and T. Speck, Dynamical clustering and phase separation in suspensions of self-propelled colloidal particles, Phys. Rev. Lett. 110, 238301 (2013).
  * Fily and Marchetti [2012] Y. Fily and M. C. Marchetti, Athermal phase separation of self-propelled particles with no alignment, Phys. Rev. Lett. 108, 235702 (2012).
  * Stenhammar  _et al._ [2014] J. Stenhammar, D. Marenduzzo, R. J. Allen, and M. E. Cates, Phase behaviour of active brownian particles: the role of dimensionality, Soft Matter 10, 1489 (2014).
  * Bialké  _et al._ [2015] J. Bialké, J. T. Siebert, H. Löwen, and T. Speck, Negative interfacial tension in phase-separated active brownian particles, Phys. Rev. Lett. 115, 098301 (2015).
  * Solon  _et al._ [2015] A. P. Solon, J. Stenhammar, R. Wittkowski, M. Kardar, Y. Kafri, M. E. Cates, and J. Tailleur, Pressure and phase equilibria in interacting active brownian spheres, Phys. Rev. Lett. 114, 198301 (2015).
  * Caprini  _et al._ [2021] L. Caprini, C. Maggi, and U. Marini Bettolo Marconi, Collective effects in confined active brownian particles, J. Chem. Phys. 154, 244901 (2021).
  * Caprini  _et al._ [2020] L. Caprini, U. M. B. Marconi, C. Maggi, M. Paoluzzi, and A. Puglisi, Hidden velocity ordering in dense suspensions of self-propelled disks, Phys. Rev. Res. 2, 023321 (2020).
  * van Roon  _et al._ [2022] D. M. van Roon, G. Volpe, M. M. Telo da Gama, and N. A. M. Araújo, The role of disorder in the motion of chiral active particles in the presence of obstacles, Soft Matter 18, 6899 (2022).
  * Scholz  _et al._ [2018b] C. Scholz, S. Jahanshahi, A. Ldov, and H. Löwen, Inertial delay of self-propelled particles, Nat. Commun. 9, 5156 (2018b).
  * Mandal  _et al._ [2019] S. Mandal, B. Liebchen, and H. Löwen, Motility-induced temperature difference in coexisting phases, Phys. Rev. Lett. 123, 228001 (2019).
  * Lehle and Peinke [2018] B. Lehle and J. Peinke, Analyzing a stochastic process driven by ornstein-uhlenbeck noise, Phys. Rev. E 97, 012113 (2018).
  * Bonilla [2019] L. L. Bonilla, Active ornstein-uhlenbeck particles, Phys. Rev. E 100, 022601 (2019).
  * Martin  _et al._ [2021] D. Martin, J. O’Byrne, M. E. Cates, É. Fodor, C. Nardini, J. Tailleur, and F. van Wijland, Statistical mechanics of active ornstein-uhlenbeck particles, Phys. Rev. E 103, 032607 (2021).
  * Caprini  _et al._ [2019] L. Caprini, U. Marini Bettolo Marconi, A. Puglisi, and A. Vulpiani, Active escape dynamics: The effect of persistence on barrier crossing, J. Chem. Phys. 150, 024902 (2019).
  * Caprini and Marini Bettolo Marconi [2021] L. Caprini and U. Marini Bettolo Marconi, Inertial self-propelled particles, J. Chem. Phys. 154, 024902 (2021).
  * Dabelow  _et al._ [2019] L. Dabelow, S. Bo, and R. Eichhorn, Irreversibility in active matter systems: Fluctuation theorem and mutual information, Phys. Rev. X 9, 021009 (2019).
  * Berthier  _et al._ [2019] L. Berthier, E. Flenner, and G. Szamel, Glassy dynamics in dense systems of active particles, J. Chem. Phys. 150, 200901 (2019).
  * Wittmann  _et al._ [2018] R. Wittmann, J. M. Brader, A. Sharma, and U. M. B. Marconi, Effective equilibrium states in mixtures of active particles driven by colored noise, Phys. Rev. E 97, 012601 (2018).
  * Fily [2019] Y. Fily, Self-propelled particle in a nonconvex external potential: Persistent limit in one dimension, J. Chem. Phys. 150, 174906 (2019).
  * Mandal  _et al._ [2017] D. Mandal, K. Klymko, and M. R. DeWeese, Entropy production and fluctuation theorems for active matter, Phys. Rev. Lett. 119, 258001 (2017).
  * Fodor  _et al._ [2016] É. Fodor, C. Nardini, M. E. Cates, J. Tailleur, P. Visco, and F. van Wijland, How far from equilibrium is active matter?, Phys. Rev. Lett. 117, 038103 (2016).
  * Marini Bettolo Marconi and Maggi [2015] U. Marini Bettolo Marconi and C. Maggi, Towards a statistical mechanical theory of active fluids, Soft Matter 11, 8768 (2015).
  * Cates and Tailleur [2015] M. E. Cates and J. Tailleur, Motility-induced phase separation, Annu. Rev. Condens. Matter Phys. 6, 219 (2015).
  * Martinez  _et al._ [2014] V. A. Martinez, J. Schwarz-Linek, M. Reufer, L. G. Wilson, A. N. Morozov, and W. C. K. Poon, Flagellated bacterial motility in polymer solutions, PNAS 111, 17771 (2014).
  * Zhou  _et al._ [2014] S. Zhou, A. Sokolov, O. D. Lavrentovich, and I. S. Aranson, Living liquid crystals, PNAS 111, 1265 (2014).
  * Sleigh  _et al._ [1988] M. A. Sleigh, J. R. Blake, and N. Liron, The propulsion of mucus by cilia, Rev. Respir. Dis. 137, 726 (1988).
  * Patteson  _et al._ [2015] A. E. Patteson, A. Gopinath, M. Goulian, and P. E. Arratia, Running and tumbling with e. coli in polymeric solutions, Sci. Rep. 5, 15761 (2015).
  * Schamel  _et al._ [2014] D. Schamel, A. G. Mark, J. G. Gibbs, C. Miksch, K. I. Morozov, A. M. Leshansky, and P. Fischer, Nanopropellers and their actuation in complex viscoelastic media, ACS Nano 8, 8794 (2014).
  * Gomez-Solano  _et al._ [2016] J. R. Gomez-Solano, A. Blokhuis, and C. Bechinger, Dynamics of self-propelled janus particles in viscoelastic fluids, Phys. Rev. Lett. 116, 138301 (2016).
  * Narinder  _et al._ [2018] N. Narinder, C. Bechinger, and J. R. Gomez-Solano, Memory-induced transition from a persistent random walk to circular motion for achiral microswimmers, Phys. Rev. Lett. 121, 078003 (2018).
  * Sprenger  _et al._ [2022] A. R. Sprenger, C. Bair, and H. Löwen, Active brownian motion with memory delay induced by a viscoelastic medium, Phys. Rev. E 105, 044610 (2022).
  * Sevilla  _et al._ [2019] F. J. Sevilla, R. F. Rodríguez, and J. R. Gomez-Solano, Generalized ornstein-uhlenbeck model for active motion, Phys. Rev. E 100, 032123 (2019).
  * Sandoval  _et al._ [2016] M. Sandoval, R. Velasco, and J. Jiménez-Aquino, Magnetic field effect on charged brownian swimmers, Physica A 442, 321 (2016).
  * Fan  _et al._ [2017] W.-T. L. Fan, O. S. Pak, and M. Sandoval, Ellipsoidal brownian self-driven particles in a magnetic field, Phys. Rev. E 95, 032605 (2017).
  * Vuijk  _et al._ [2020] H. D. Vuijk, J. U. Sommer, H. Merlitz, J. M. Brader, and A. Sharma, Lorentz forces induce inhomogeneity and flux in active systems, Phys. Rev. Research 2, 013320 (2020).
  * Abdoli and Sharma [2021] I. Abdoli and A. Sharma, Stochastic resetting of active brownian particles with lorentz force, Soft Matter 17, 1307 (2021).
  * Muhsin and Sahoo [2022] M. Muhsin and M. Sahoo, Inertial active ornstein-uhlenbeck particle in the presence of a magnetic field, Phys. Rev. E 106, 014605 (2022).
  * Muhsin  _et al._ [2021] M. Muhsin, M. Sahoo, and A. Saha, Orbital magnetism of an active particle in viscoelastic suspension, Phys. Rev. E 104, 034613 (2021).
  * Jayannavar and Sahoo [2007] A. M. Jayannavar and M. Sahoo, Charged particle in a magnetic field: Jarzynski equality, Phys. Rev. E 75, 032102 (2007).
  * Kumar [2012a] N. Kumar, Classical orbital magnetic moment in a dissipative stochastic system, Phys. Rev. E 85, 011114 (2012a).
  * Kumar [2012b] N. Kumar, Erratum: Classical orbital magnetic moment in a dissipative stochastic system [phys. rev. e 85, 011114 (2012)], Phys. Rev. E 86, 029902 (2012b).
  * Paraan  _et al._ [2008] F. N. C. Paraan, M. P. Solon, and J. P. Esguerra, Brownian motion of a charged particle driven internally by correlated noise, Phys. Rev. E 77, 022101 (2008).
  * Lisy and Tothova [2013] V. Lisy and J. Tothova, Brownian motion of charged particles driven by correlated noise in magnetic field, Transp. Theory. Stat. Phys. 42, 365 (2013).
  * Burrage  _et al._ [2007] K. Burrage, I. Lenane, and G. Lythe, Numerical methods for second‐order stochastic differential equations, SIAM J. Sci. Comput. 29, 245 (2007).
  * Fox  _et al._ [1988] R. F. Fox, I. R. Gatland, R. Roy, and G. Vemuri, Fast, accurate algorithm for numerical simulation of exponentially correlated colored noise, Phys. Rev. A 38, 5938 (1988).


# Efficient 3D Object Reconstruction using Visual Transformers

Rohan Agarwal
<EMAIL_ADDRESS>Wei Zhou
<EMAIL_ADDRESS>Xiaofeng Wu
<EMAIL_ADDRESS>Yuhan Li
<EMAIL_ADDRESS>Georgia Institute of Technology
North Ave NW, Atlanta, GA 30332

##  1 Abstract

Reconstructing a 3D object from a 2D image is a well-researched vision
problem, with many kinds of deep learning techniques having been tried. Most
commonly, 3D convolutional approaches are used, though previous work has shown
state-of-the-art methods using 2D convolutions that are also significantly
more efficient to train. With the recent rise of transformers for vision
tasks, often outperforming convolutional methods, along with some earlier
attempts to use transformers for 3D object reconstruction, we set out to use
visual transformers in place of convolutions in existing efficient, high-
performing techniques for 3D object reconstruction in order to achieve
superior results on the task. Using a transformer-based encoder and decoder to
predict 3D structure from 2D images, we achieve accuracy similar or superior
to the baseline approach. This study serves as evidence for the potential of
visual transformers in the task of 3D object reconstruction.

##  2 Introduction

3D reconstruction from 2D image is a process of transforming a 2D image into a
3D model represented by a point cloud, voxel mesh, or polygon mesh. It has
been a challenging task, yet beneficial for many technologies and industries
such as computer vision, animation, medical imaging, video game design,
virtual reality and other computer graphics related industries. Improvements
in accuracy and efficiency would be highly valuable.

The typical approach for 3D object reconstruction involves using 3D
convolutional neural networks. However, these methods are computationally
inefficient, and superior accuracy and efficiency has been achieved through
using 2D convolution [2]. Meanwhile, in the field of computer vision in
general, visual transformers have become increasingly popular, and on many
vision tasks, outperform traditional convolutional architectures [1].

Thus, we set out to combine the benefits of 2D convolutional approaches on the
task of 3D object reconstruction with the recent improvements of visual
transformers. We attempt to enable superior performance and efficiency for 3D
object reconstruction to make it more useful and practical in the numerous
applications discussed above.

##  3 Related Works

###  3.1 3D Object Reconstruction

Previous works have tried many approaches for 3D object reconstruction.
Typically, just as 2D convolutions are used in most image-based tasks, 3D
convolutions are used in some form for this task. However, this requires
coarser voxel representations of objects (similar to pixels in 2D) and is
generally inefficient in both time and memory [2]. It has been demonstrated by
Lin, Kong, and Lucey (2018) that 2D convolutions can more efficiently achieve
state-of-the-art results for this task [2]. Their work predicted the 3D
structure of an object using images from multiple viewpoints, generated
through a 2D-convolution-based structure generator. They also presented a more
efficient optimization method called ”pseudo-rendering,” which optimizes using
2D projection error. This approach also allows using point cloud
representations instead of voxel representations.

There are other methods, such as those taking advantage of depth prediction,
as presented by Yin, et. al. (2021) [3]. Their work trains a depth prediction
model and a point cloud generation model separately. After combining the two
in a single pipeline, the approach is able to outperform prior methods in 3D
scene reconstruction.

###  3.2 Visual Transformers

As discussed by Lahoud, et. al. (2022), visual transformers are creating a new
state-of-the-art in some vision tasks, both 2D and 3D, just as they have done
in language [1]. They often outperform convolution-based approaches. There is
also significant previous work on using transformers on both point-cloud and
voxel representations (in addition to 2D pixel images), for object
classification, object detection, segmentation, point cloud completion, pose
estimation, and even 3D reconstruction to some extent.

There are mainly two design principles related to building visual
transformers. One principle is to get rid of convolutions entirely and replace
it with self attention, like ViT [4] which embeds each 16*16 patch of the
image as a vector and uses the encoder from the original transformer [5].
Another principle is to design a hybrid model with both convolutional layer
and self attention like in MaxViT [6]. We have tried both architecture designs
for our encoder and we will provide more detail in section 4.3.1.

##  4 Methods

Due to its efficiency and impressive results, and a focus on object
reconstruction instead of scene reconstruction, we mostly model our approach
after Lin, Kong, and Lucey (2018) [2]. The model uses a convolution encoder
and decoder for feature extraction and viewpoint generation. This resembles
the architecture of visual transformers, which we wish to replace the
convolutional sections of the model with due to their potential to lead to
superior results as shown by recent research in the field.

Our model consists of a visual transformer for converting an image to 8 images
with depth and mask information at 8 different viewpoints. The projections are
used to form a point cloud. The point cloud generation is then rendered to
different projections at novel viewpoints and are compared with the ground
truth projections to obtain loss. It is an efficient and reasonable loss
function as we believe a good 3D reconstruction should produce a similar
projection in different viewpoints compared to the ground truth.

###  4.1 Data

We used the dataset provided by Lin, Kong, and Lucey (2018) [2] at the
following website: https://chenhsuanlin.bitbucket.io/3D-point-cloud-
generation/. This dataset contains around 3000 data points of RGB images,
depth information, and mask information. It also contains several categories
of common objects, such as chairs. The depth and mask information, used as the
ground truth information, was generated by rendering the corresponding CAD
models at random viewpoints. Each image is 128x128 pixels.

###  4.2 Preliminary Methods

####  4.2.1 Transformer-Based Structure Generator

In the original structure generator, we replaced its encoder with a pretrained
visual transformer 1. The outputs of the encoder are images with size
224$\times$224 pixels. To further shrink down the width and height, we passed
it through two 2D convolutional blocks and five fully connected blocks.

For the decoder part, we use four 2D deconvolutional blocks and one 2D
convolutional block and bias term to upsample the image to the preferred size
of 128$\times$128\. The final output is an image with 16 channels, in which 8
channels represent the depth channel from 8 different viewpoints and the other
8 channels represent the mask from 8 different viewpoints representing whether
each pixel should be kept during projection to 3D. Within each block, we add a
batch normalization and a ReLU activation function to normalize the input for
the next block and add nonlinearity to the model.

Another modification we have made is in the type of outputs from the model.
The baseline model’s output consisted of an XY coordinate matrix corresponding
to depth information and mask. However, we noticed that the XY coordinate
matrix is exactly the same for every image and thus we deemed it unnecessary
to require the model to predict. Hence, our model only outputs depth
information and mask. Thus, we deduced this might be why our model’s result is
much more stable than the baseline model when given the same image from
different viewpoints as shown in Fig. 3.

During training time, we froze all the parameters within the pretrained visual
transformers except for the embedding layer and the final attention layer. The
2D convolutional and deconvolutional blocks as well as the fully connected
blocks are all set to trainable. The training time of our model is much faster
than the baseline model since our model consists of 894,128 trainable
parameters while the baseline model consists of 28 million trainable
parameters.

Figure 1: Architecture of the original visual transformer and the transformer
encoder used in it from the paper ”An Image is Worth 16x16 Words: Transformers
for Image Recognition at Scale” by Dosovitskiy et al. [4]

####  4.2.2 Optimization

The optimization method we are using is same one as used by Lin, Kong, and
Lucey (2018) [2]. To avoid computationally expensive and intractable direct
distance calculations between the 3D point cloud and ground truth label, we
are using joint 2D projection with pseudo-rendering. To be specific, we
project canonical 3D points gained after the structure generator to 8
different preset viewpoints using rigid 3D transformation with upsampling and
max pooling on inverse-depth to solve collision (same pixel contains more than
one point in the point cloud). This gives us 8 images, where each pixel
contains information on whether any point cloud is projected onto it by a
binary mask. Then the loss is calculated as the cross-entropy loss of the mask
and element-wise $L_{1}$ loss of the depth between 8 images of the ground
truth model and the same viewpoints’ 8 images of point clouds. We find this
part very useful in our training because it can greatly reduce the generated
point clouds from 30833.049 average number of points to 19565.377 average
number of points for our transformer-based structure generator.

###  4.3 Final Approach

To improve results further, several further changes were made to the
architecture from the preliminary method–mainly, replacing more convolutional
parts of the architecture with more transformers (see Fig. 2).

####  4.3.1 Encoder

In theory, convolutional layer carries more inductive bias towards images and
thus it’s able to learn good features with less data compared to self
attention layer. Therefore, we experimented using a hybrid design of
convolutional layer and self attention layer like MaxViT described in [6], as
well as Swin transformer described in [7]. However, after experiments, the
original pretrained visual transformer is able to achieve the best results and
thus we choose to use it as our model’s encoder. The pretrained visual
transformer from [4] was kept as is for the encoder part of the structure
generator, but the images were resized to 224$\times$224 pixels before being
passed into the encoder. After the encoder, a MLP block is used to reduce the
sequence length of the encoder’s output from 179 to 64 so that it can be later
interpreted in the decoder head as a feature map with height and width of 8.

####  4.3.2 Decoder

The most significant change from the preliminary method was replacing the
decoder of the original structure generator with four layers of a transformer
decoder layer in [5] with 30 million parameters, instead of the 2D
convolutional layers from before. The transformer decoder at each timestep
will generate a text-like output with shape batch size*sequence
length*dimension. This is incompatible with the required output, which
consists of depth and mask feature map with shape batch size*width*height.

Therefore, to address for this semantic gap, we add a decoder head with the
target of correctly interpreting the transformer decoder’s output as feature
maps. During forward pass, the transformer decoder’s output is reshaped from
batch size*sequence length*dimension to batch size*channels*width*height,
where dimension is interpreted as channels and width and height is the square
root of sequence length. Our final design choice for the decoder output head
is a 2D convolution with a 1$\times$1 kernel to reduce channels from 768 to
512 and a pixel shuffle layer that reshape a tensor of
(channel*$r^{2}$,height,width) to (channel,height*$r$,width*$r$). This design
choice is adopted from the visual transformer’s decoder for masked image
modeling. We also tried adding complexities to the decoder head by using
layers of deconvolutional blocks, but there is no significant improvement in
performance during experiments.

The output of the transformer decoder plus the decoder head should be a depth
(1,128,128) and mask (1,128,128) matrix concatenated together (2,128,128) for
each of the 8 points of view. So in total our model’s output should be of
shape (8,2,128,128). We believe that the depth and mask information from one
point of view should be conditioned on the depth and mask information from the
other points of view. Thus, the decoder’s forward function is designed to be
autoregressive. The decoder’s hidden state is the encoder’s last hidden state
(after the linear block). The decoder’s output should be the depth and mask
information for 8 different views, so at time $0$, we use a tensor of all ones
as the input to signal the decoder to output the first view. At time $t$, we
use the decoder’s output from time $t-1$ (i.e., the depth and mask info from
view $t-1$) as inputs. We iterate 8 times to generate the output for all 8
views.

Figure 2: Overview of the architecture of our new model.

####  4.3.3 Training Procedure

To train our model, we froze the encoder’s first six transformer layers
(requires_grad=False) and set its last six transformer layers to trainable.
The linear block, decoder, and decoder head are all trained from scratch for a
total of 76 million trainable parameters.

We first trained for 80 epochs without joint 2D optimization to save time due
to our limited computational resources. This can be done by comparing the
outputted point cloud in eval mode with the ground truth point cloud for an
error metric. The pretrained transformer is given a smaller learning rate
since we should only have to finetune it. We use a cosine annealing learning
rate scheduler.

After that, we further train the model with joint 2D optimization for 20
epochs. The learning rate is the same for the whole network and we again use a
cosine annealing learning rate scheduler.

##  5 Results

###  5.1 Preliminary Results

We used the code provided at the same website [2], unmodified, to get baseline
results. After making our modifications to the architecture we achieved the
below preliminary results in Tables 1, 2. The 3D error measures the average
point-wise 3D Euclidean distance between two 3D point cloud models. This
metric is defined bidirectionally as the distance from the predicted point
cloud to the ground truth CAD model and vice versa.

Method | Generated Points
---|---
Baseline | 18895.68
Our Method | 19565.38
Table 1: The average number of 3D points generated by each method to form the point cloud after 20 training epochs. Our method is able to generate slightly more dense result than baseline. | 3D error metric |
---|---|---
Method | Pred $\rightarrow$ GT | GT $\rightarrow$ Pred
Baseline | 5.682 | 7.214
Our Method | 4.777 | 5.15
Table 2: The average 3D test error (scaled by 100) after 20 training epochs
from both methods. Prediction to GT (ground truth) error measures 3D shape
similarity and GT to prediction error measures surface coverage. Our method
outperforms the baseline method in both 3D shape similarity and surface
coverage.

The visual comparison of the point clouds we produced is shown in Fig. 3.
While the result for the baseline model changed to a great extent when the
perspective of images changed, the result of our preliminary model changes
significantly less, which shows more robustness of our model. To be more
specific, when the perspective of the image does not give too much information
about the object’s shape (when the back of the chair takes up most part of the
image or when the front of the chair takes up most part of the image) it is
difficult to get the depth information of the input. Compare to the
preliminary model, the baseline one is more vulnerable when inputs are tricky.
Another thing to notice: due to limited computing power, we only trained 20
epochs on both model’s structure generator which resulted in a relatively poor
point cloud of the model. However, the differences between two models are
shown clearly.

Figure 3: This figure shows the 3D cloud point result by 10 epochs of our
preliminary result and baseline result using the work by Lin, Kong, and Lucey
(2018) [2]. The input is an image from one of multiples perspectives of the
same chair. When giving an input of the back of the chair or when the chair
only shows two of the four legs, our model shows a more stable result than the
baseline.

###  5.2 Final Results

After the final changes to the model architecture and a more extensive
training process with more epochs and across all 6678 objects provided in the
dataset, as described in Section 4.3, we achieved new results, shown in Tables
3, 4, 5, 6.

Method | Generated Points, No Opt.
---|---
Baseline | 32126.88
Our Method | 31219.99
Table 3: The average number of 3D points generated by each method to form the point cloud after following the procedure described in Section 4.3.3, but before training with joint 2D optimization. Method | Generated Points, Opt.
---|---
Baseline | 25383.11
Our Method | 15614.25
Table 4: The average number of 3D points generated by each method to form the
point cloud after following the procedure described in Section 4.3.3
completely.

Looking at number of generated points, our method is effective at reducing
that number compared to the baseline, especially when using joint 2D
optimization.

| 3D Error Metric, No Opt. |
---|---|---
Method | Pred $\rightarrow$ GT | GT $\rightarrow$ Pred
Baseline | 7.513 | 3.564
Our Method | 7.823 | 1.825
Table 5: The average 3D test error (scaled by 100) after following the procedure described in Section 4.3.3, but before training with joint 2D optimization. Prediction to GT (ground truth) error measures 3D shape similarity and GT to prediction error measures surface coverage. | 3D Error Metric, Opt. |
---|---|---
Method | Pred $\rightarrow$ GT | GT $\rightarrow$ Pred
Baseline | 4.651 | 5.187
Our Method | 3.980 | 2.946
Table 6: The average 3D test error (scaled by 100) after following the
procedure described in Section 4.3.3 completely. Prediction to GT (ground
truth) error measures 3D shape similarity and GT to prediction error measures
surface coverage.

Looking at the errors, our method outperforms the baseline method, with the
complete training method using joint 2D optimization. Without joint 2D
optimization, our model is similar to or better than the baseline, depending
on the metric. In some metrics, namely surface coverage, our model without
joint 2D optimization outperforms the baseline even without joint 2D
optimization, which would allow for significant savings in efficiency.

We present the outputted point clouds from the baseline and our model below in
Figures 4, 5, 6, 7, 8. Neither model fully reconstructs the object, but that
is to be expected due to the short training time of 80-100 epochs. This is a
limitation of this study due to low computational resources. Regardless, it is
qualitatively clear that both models are approximating the shape of the
original model well, yet still produce very different outputs. It is
interesting to consider how the choice of model changes the ”approach” it
takes to forming these point clouds over many iterations.

Though the baseline (Figure 6) has a clearer ”seat” than our method (Figure
8), our method more closely approximates the squared shape of the original
model in Fig. 4 than the baseline result. Without joint 2D optimization, our
model (Fig. 7 is much closer to the correct shape of the chair than the
baseline in Fig. 5; this is an avenue for potential efficiency gains if joint
2D optimization could be fully eliminated in future work. Even still, it is
hard to definitely make a comparison qualitatively due to the low training
time. Yet along with the lower errors, it is evident that our model using
primarily visual transformers outperforms or at least matches the baseline
model that uses primarily 2D convolution.

Figure 4: This figure shows the ground truth model given to get outputs shown
in Figures 5, 6, 7, 8. Figure 5: This figure shows the 3D point cloud results
from the baseline method without joint 2D optimization. Figure 6: This figure
shows the 3D point cloud results from the baseline method with joint 2D
optimization. Figure 7: This figure shows the 3D point cloud results from our
method without joint 2D optimization. Figure 8: This figure shows the 3D point
cloud results from our method with joint 2D optimization.

Below we present several more examples of point cloud outputs from our model
on other 3D objects that demonstrate its ability to approximate the shape,
despite a short training process (Figures 9, 10, 11).

Figure 9: This figure shows our model’s output on another object.

Figure 10: This figure shows our model’s output on another object.

Figure 11: This figure shows our model’s output on another object.

##  6 Conclusion

Overall, this study shows the potential of visual transformers in
accomplishing efficient 3D object reconstruction. By taking an approach to
this task with state-of-the-art results and high efficiency [2], and replacing
the convolutional portions of the technique with visual transformers, we are
able to achieve similar results in all cases and better in some cases. While
we provide a new potential method for 3D object reconstruction applications,
more importantly to this area of research, this work provides evidence that
visual transformers can effectively tackle the problem of 3D object
reconstruction from a single image. It shows that visual transformers should
be explored further for this specific task of 3D object reconstructions, other
variations of the task (such as starting with images from multiple
viewpoints), and more generally, for other 2D and 3D vision tasks.

Due to limited computational resources, this work should ideally be repeated
with more time so that we can make more thorough comparisons and more
interesting qualitative comparisons (as the generated point clouds were not
very precise for either method with such short training time). The original
paper for our baseline method used 1000 epochs to generate results [2], but we
were only able to train for 100 epochs. Another limit we found was how our
model addresses 3D geometric information. Based on our results, our model can
generate 3D shapes with high surface coverage but low 3D similarity to the
ground truth. We attempted developing loss function using point-wise distance
between vertices of polygons and generated point cloud, which is more
efficient than using two point clouds, but our hardware memory was a limit and
requires much more tuning when implement it. Thus, it is worth keep trying to
incorporate more 3D geometric information when computing loss in the future.
Other future work could explore more custom-designed transformer architectures
for improvements. It should explore how visual transformers can improve other
approaches to this problem, such as those more popular techniques based on 3D
convolution (especially since visual transformers have been used directly on
3D before [1]), depth recognition, and more.

## References

  * [1]

[1] J. Lahoud, J. Cao, F. S. Khan, H. Cholakkal, R. M. Anwer, S. Khan, and
M.-H. Yang, “3D vision with transformers: A survey,” arXiv.org, 08-Aug-2022.
[Online]. Available: https://arxiv.org/abs/2208.04309. [Accessed:
31-Oct-2022].

[2] C.-H. Lin, C. Kong, and S. Lucey, “Learning efficient point cloud
generation for dense 3D object reconstruction,” Proceedings of the AAAI
Conference on Artificial Intelligence, vol. 32, no. 1, 2018.

[3] W. Yin, J. Zhang, O. Wang, S. Niklaus, L. Mai, S. Chen, and C. Shen,
“Learning to recover 3D scene shape from a single image,” 2021 IEEE/CVF
Conference on Computer Vision and Pattern Recognition (CVPR), 2021.

[4] A. Dosovitskiy, L. Beyer, A. Kolesnikov., et al. 2021. ”An image is worth
16x16 words: Transformers for image recognition at scale.” arXiv.org.
[Online]. Available: https://arxiv.org/abs/2010.11929v2. [Accessed:
30-Nov-2022].

[5] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L.
Kaiser, and I. Polosukhin, “Attention is all you need,” arXiv.org,
06-Dec-2017. [Online]. Available: https://arxiv.org/abs/1706.03762. [Accessed:
05-Dec-2022].

[6] Z. Tu, H. Talebi, H. Zhang, F. Yang, P. Milanfar, A. Bovik, and Y. Li,
“MaxViT: Multi-axis vision transformer,” arXiv.org, 09-Sep-2022. [Online].
Available:https://arxiv.org/abs/2204.01697. [Accessed: 05-Dec-2022].

[7] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin
Transformer: Hierarchical vision transformer using shifted windows,”
arXiv.org, 17-Aug-2021. [Online]. Available: https://arxiv.org/abs/2103.14030.
[Accessed: 05-Dec-2022].