Holographic conductivity from Einstein-Maxwell-Dilaton in Gauss-Bonnet gravity
and Entropy Function

Yan-Tao Hao1, Li-Qing Fang2, Long Cheng1

1 Department of Physics, Zhejiang Sci-Tech University,
Hangzhou 310018, China
2 School of Measuring and Optical Engineering, Nanchang Hangkong University,
Nanchang 330063, China

In this paper we consider the holographic DC and Hall conductivity in
Einstein-Maxwell-Dilaton in Gauss-Bonnet gravity with momentum dissipation. We
analytically derived the DC conductivity and Hall conductivity from the black
horizon data, and found that the conductivities are independent on the Gauss-
Bonnet coupling. We also used the entropy function formalism to get the
conductivities in terms of the charge of the black hole, even without knowing
the explicit black hole solutions.

##  1 Introdution

The AdS/CFT correspondence gives a simple way to study strongly correlated
systems [1, 2, 3]. In the framework of AdS/CFT, one can relates the weakly
coupled classical gravity theories to the strongly coupled quantum many-body
systems. So it provides a new approach to calculating observables in these
strongly coupled many-body system. In particular, one can perturb the electric
field and thermal gradients around black hole background to calculate the
transport coefficients such as conductivities in the strongly correlated
systems on the boundary.

In the past few years, there are a lot of work on calculating the
conductivities based on several holographic models. One conventional approach
is that take the perturbation on the boundary by a time-dependent field with
frequency $\omega$ to get the optical conductivities, and then take the zero
frequency limit to get the DC conductivities. However, because of the
translation invariance of the gravitational theories, there is the divergence
in obtaining the DC conductivities. To avoid this unphysical diverge
conductivities, one can introduce the linear axion field in the gravitational
background to brake the spatial translation invariance but remain the
homogeneous [4]. In this gravitational model, the presence of the linear axion
field results the momentum dissipation in the dual boundary theory. This model
has been widely used in applied holographic calculations, see the reviews [5]
and therein.

Recently, one novel approach to calculate DC conductivities was proposed in
[6, 7]. Based on the idea of the membrane paradigm, this method provides an
effective way to derive the DC transports from the data of black horizon. So
once given a charged black hole solution in AdS space, one can calculate the
DC electrical conductivities in terms of matter fields on the horizon.
Furthermore, in [8, 9], the authors consider the dyonic black holes and get
the Hall conductivities. Inspired by these work, various models have been
discussed in [10, 11, 12, 13, 14, 15, 16]. On the other hand, for a specific
class of black hole, there is an interesting property named ”attractor
mechanism” [17]. It states that the magnitude of a matter field obtains some
values on the black hole horizon, which are dependent only upon the electric
and magnetic charges of the black hole. In other words, the configuration of a
black hole near horizon depends only on the electric and magnetic charges of
the black hole and not on the asymptotic values of other matter fields. Thus
it is very natural to think about whether one can use the attractor mechanism
to calculate the DC conductivities without knowing the full black hole
solutions. Following the spirit of the above discussions, in the work [18],
the authors employ the Sen’s entropy function method to get the horizon data
and conductivities for Einstein-Maxwell-Dilaton theory. In addition, the
Einstein-Maxwell theory with topological Maxwell term was discussed in [19,
20].

In this paper, following the approach [7, 8] we will generalize these
discussions to more generic case. So we are going to study the Einstein-
Maxwell-Dilaton-Axion theory with Gauss-Bonnet term and find the DC and Hall
conductivities analytically. The attractor mechanism and the Sen’ entropy
function in Gauss-Bonnet gravity will be performed in calculations. One of the
motivations is that when we consider the semi-classical effect, the Gauss-
Bonnet term appears as the curvature stringy correction. On the other hand,
what we consider in this paper is Gauss-Bonnet gravity coupled with a
nontrivial dilaton field and potential. Unlike the case of [12], there is no
exact black hole solution exists, so it is impossible to get the DC
conductivities from horizon data directly. We can use the entropy function and
solve the attractor equations to express the conductivities at zero
temperature.

This paper is organised as follows. In section 2, we introduce the Einstein-
Maxwell-Dilaton-Axion theory in Gauss-Bonnet gravity. In section 3, we will
calculate the DC transport coefficients such as electrical conductivity,
thermal conductivity and thermoelectric conductivity from black horizon. In
section 4, in the presence of magnetic field, the Hall conductivities are
discussed. We apply the Sen’s entropy function in our model and express the
conductivities in terms of the charge and magnetic field in section 5.
Finally, we conclude in section 5.

##  2  Einstein-Maxwell-Dilaton-Axion theory in Gauss-Bonnet gravity

We consider the following five-dimensional action of Einstein-Maxwell-Dilaton-
Axion theory in Gauss-Bonnet gravity

$\displaystyle
S=\frac{1}{2\kappa^{2}}\int_{M}d^{5}x\sqrt{-g}\Big{(}R+\lambda\mathcal{L}_{GB}-\frac{1}{2}(\partial{\phi})^{2}-V(\phi)-\frac{\Phi(\phi)}{2}\sum_{i=1}^{3}(\partial{\chi_{i}})^{2}-\frac{Z(\phi)}{4}F_{\mu\nu}F^{\mu\nu}\Big{)},$
(1)

where $2\kappa^{2}=16\pi G_{5}$ is the five-dimensional gravitational
coupling. $\lambda$ is Gauss-Bonnet coupling constant with dimension
$\rm(length)^{2}$ and

$\displaystyle\mathcal{L}_{GB}=\left(R_{\mu\nu\rho\sigma}R^{\mu\nu\rho\sigma}-4R_{\mu\nu}R^{\mu\nu}+R^{2}\right)$
(2)

is Gauss-Bonnet term. The linear axion fields $\chi_{i}(x^{\mu})(i=1,2,3)$ are
introduced to brake the diffeomorphism symmetry, $\phi$ is dilaton field and
$V$ is the potential of dilaton field. The couplings $\Phi(\phi)$, $Z(\phi)$
and $V(\phi)$ depend on the $\phi$. $F_{\mu\nu}=(dA)_{\mu\nu}$ is defined as
$U(1)$ gauge field strength.

The equations of motion for gravity, dilaton and Maxwell field equation are
easily obtained as

$\displaystyle\nabla_{\mu}\left(Z(\phi)F^{\mu\nu}\right)=0,$ (3)
$\displaystyle\nabla_{\mu}\nabla^{\mu}\phi-\frac{dV(\phi)}{d\phi}-\frac{1}{2}\frac{d\Phi(\phi)}{d\phi}\sum_{i=1}^{3}(\partial{\chi_{i}})^{2}-\frac{1}{4}\frac{dZ(\phi)}{d\phi}F_{\mu\nu}F^{\mu\nu}=0,$
(4) $\displaystyle
R_{\mu\nu}-\frac{g_{\mu\nu}}{2}\left(R-\frac{1}{2}(\partial{\phi})^{2}-V(\phi)-\frac{\Phi(\phi)}{2}\sum_{i=1}^{3}(\partial{\chi_{i}})^{2}-\frac{Z(\phi)}{4}F_{\mu\nu}F^{\mu\nu}\right),$
$\displaystyle~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}-\frac{1}{2}\partial_{\mu}\phi\partial_{\nu}{\phi}-\frac{\Phi(\phi)}{2}\sum_{i=1}^{3}\partial_{\mu}{\chi_{i}}\partial_{\nu}{\chi_{i}}-\frac{Z(\phi)}{2}F_{\mu\lambda}F_{\nu}^{~{}\lambda}$
$\displaystyle~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}-\lambda\frac{g_{\mu\nu}}{2}\mathcal{L}_{GB}+\lambda\left(R^{2}-4R_{\rho\sigma}R^{\rho\sigma}+R_{\lambda\rho\sigma\tau}R^{\lambda\rho\sigma\tau}\right)=0$
(5)

The above equations of motion admits the metric ansatz as following

$\displaystyle
ds^{2}=-f(r)dt^{2}+\frac{1}{f(r)}dr^{2}+e^{2U(r)}(dx^{2}+dy^{2}+dz^{2}),$ (6)

and

$\displaystyle\chi_{i}=a_{i}x+b_{i}y+c_{i}z,~{}~{}~{}~{}~{}~{}~{}A=A_{t}(r)dt.$
(7)

where the UV boundary is defined as $r\rightarrow\infty$. The gauge field has
the $t$-component $A_{t}$ only, and we will consider the non-vanishing
magnetic field in section 4 and 5. The axion fields have only one non-zero
component that is linear in the boundary coordinates for simplicity, so the
non-vanish positive constants $a_{i},b_{i},c_{i}$ take the value
$a_{1}=b_{2}=c_{3}=k$ to admits the isotropic and homogeneous metric. We also
require the dilaton potential $V(\phi)$ satisfy the conditions

$\displaystyle V(0)=-\frac{12}{L^{2}},~{}~{}~{}~{}V^{\prime}(0)=0$ (8)

in order to admit the asymptotically AdS solution.

Finally, by using the Euclidean continuation of the metric (6), the
temperature of black hole is given by

$\displaystyle T=\frac{f^{\prime}(r_{\textrm{\tiny H}})}{4\pi}.$ (9)

##  3 DC conductivities from horizon data

###  3.1  Electric conductivity and thermoelectric conductivity

Following the method [7], to calculate the transport coefficients, we need to
generate the electric and heat currents in the spatial directions. To achieve
this, we consider the small perturbation ansatz of the form as follows:

$\displaystyle g_{tx}\rightarrow\delta g_{tx}(r)$ $\displaystyle
g_{rx}\rightarrow r^{2}\delta g_{rx}(r)$ $\displaystyle A_{x}\rightarrow-
Et+\delta A_{x}(r)$ $\displaystyle\chi_{1}\rightarrow kx+\delta\chi(r).$ (10)

Then linearizing the Maxwell equation, Einstein equations and Klein-Gordon
equation around the metric, one can obtain the following four equations of
perturbations:

$\displaystyle\delta
A_{x}^{\prime\prime}+\left(\frac{f^{\prime}}{f}+\frac{Z^{\prime}(\phi)}{Z(\phi)}\phi^{\prime}+U^{\prime}\right)\delta
A_{x}^{\prime}+\frac{A_{t}^{\prime}}{f}(\delta
g_{tx}^{\prime}+U^{\prime}\delta
g_{tx}+\frac{Z^{\prime}(\phi)\phi^{\prime}\delta g_{tx}}{Z})=0,$ (11)
$\displaystyle\delta\chi^{\prime}-k\delta
g_{rx}-\frac{EZ(\phi)A_{t}^{\prime}}{kf\Phi(\phi)}=0,$ (12)
$\displaystyle\delta
g_{tx}^{\prime\prime}+\frac{e^{-2U}\Phi(\phi)k^{2}+2f(U^{\prime\prime}-U^{\prime
2}(4\lambda U^{\prime}f+12\lambda(U^{\prime
2}+U^{\prime\prime})f-3))}{f(4\lambda U^{\prime 2}f-1)}\delta g_{tx}$
$\displaystyle~{}~{}~{}~{}~{}~{}~{}~{}+\frac{U^{\prime}\left(4\lambda
U^{\prime}(f^{\prime}+U^{\prime}f)+8\lambda
U^{\prime\prime}f-1\right)}{(4\lambda U^{\prime 2}f-1)f}\delta
g_{tx}^{\prime}+\frac{Z(\phi)A_{t}^{\prime}}{4\lambda U^{\prime 2}f-1}\delta
A_{x}^{\prime}=0,$ (13)
$\displaystyle\delta\chi^{\prime\prime}+\frac{f^{\prime}+3U^{\prime}f}{f}\delta\chi^{\prime}-k\delta
g_{rx}^{\prime}-\frac{k(f^{\prime}+3U^{\prime}f)}{f}\delta g_{rx}=0.$ (14)

Note that, from the Maxwell equations (3), one can define a radially conserved
current as $q\equiv e^{3U}Z(\phi)A_{t}^{\prime}$, which is the charge of the
black hole. Thus linearised Maxwell equation (11) results another conserved
current as

$\displaystyle J_{x}=-\sqrt{-\det g}Z(\phi)F^{rx}$
$\displaystyle~{}~{}=-e^{U}Z(\delta g_{tx}A_{t}^{\prime}+fA_{x}^{\prime})$
$\displaystyle~{}~{}=-qe^{-2U}\delta g_{tx}-e^{U}fZ(\phi)\delta
A_{x}^{\prime}$ (15)

which is independent of radial coordinate $r$. So it is convenient to evaluate
$J_{x}$ at the horizon. The equation for $\delta g_{rx}$ (12) deduces

$\displaystyle\delta
g_{rx}=-\frac{EZ(\phi)A_{t}^{\prime}}{k^{2}f\Phi(\phi)}+\frac{1}{k}\delta\chi^{\prime}$
$\displaystyle=-\frac{qEe^{-3U}}{k^{2}f\Phi(\phi)}+\frac{1}{k}\delta\chi^{\prime}~{}$
(16)

Now we have to consider the asymptotic behavior near the horizon $r=r_{H}$.
Note that on the future horizon of the black hole, the regularity requires the
boundary condition on the horizon. Thus we employ the ingoing Eddington-
Finklestein coordinates $(u,v)$ defined as $v=t+\int\frac{dr}{f(r)}$. For
gauge field, it should be regular at the future horizon, i.e. $A_{x}\sim-
Ev+...$ in EF coordinates. So the perturbation field $\delta A_{x}$ takes the
asymptotic behavior near the horizon as

$\displaystyle\delta A_{x}\sim-\frac{E}{4\pi T}\log(r-r_{\textrm{\tiny
H}})+\mathcal{O}(r-r_{\textrm{\tiny H}}).$ (17)

Furthermore, one can see that the singular part of the metric in Eddington-
Finklestein coordinates is given by

$\displaystyle-\frac{2\delta g_{tx}}{f(r)}drdx+2e^{2U}\delta
h_{rx}drdx+2\delta g_{tx}dvdx$ (18)

So from (16) that $\delta g_{rx}\sim 1/(r-r_{H})$ near horizon, the regularity
on the horizon requires that $\delta g_{tx}$ behaves as

$\displaystyle\delta g_{tx}$ $\displaystyle\sim$ $\displaystyle e^{2U}f\delta
g_{rx}|_{r\rightarrow r_{\textrm{\tiny H}}}$ (19) $\displaystyle=$
$\displaystyle-\frac{e^{2U}(kf\Phi(\phi)\delta\chi^{\prime}-EZ(\phi)A_{t}^{\prime}}{k^{2}\Phi(\phi)}\bigg{|}_{r\rightarrow
r_{\textrm{\tiny H}}}+\mathcal{O}(r-r_{\textrm{\tiny H}})$ $\displaystyle=$
$\displaystyle\left(-\frac{qEe^{-U}}{k^{2}\Phi(\phi)}+\frac{e^{2U}f\chi^{\prime}}{k}\right)\bigg{|}_{r\rightarrow
r_{\textrm{\tiny H}}}.$

Note that we have assumed that $\delta\chi$ is analytic on the horizon.

Now we are going to calculate the value of the conserved current in order to
obtain the DC conductivities. Recall the conserved current $J_{x}$ is constant
along the radial direction $r$, one can evaluate $J_{x}$ at the horizon
$r=r_{H}$. Thus the electric conductivity $\sigma$ is given by

$\displaystyle\sigma=\frac{\partial J}{\partial E}$ (20)
$\displaystyle~{}~{}=\left(\frac{e^{-3U}q^{2}}{k^{2}\Phi(\phi)}+e^{U}Z(\phi)\right)\bigg{|}_{r\rightarrow
r_{\textrm{\tiny H}}}$

Next we could construct the conserved heat current to obtain the
thermoelectric conductivity $\bar{\alpha}$. One can use the $t-x$ component
linearized Einstein equation and Maxwell equation to show that the current

$\displaystyle Q=e^{U}(f^{\prime}\delta g_{tx}-f\delta
g_{tx}^{\prime})(4\lambda U^{\prime 2}f-1)-A_{t}J_{x}$ (21)

is radially constant, i.e. $\partial_{r}Q=0$. Thus we can evaluate the $Q$ at
any where in the bulk. To see $Q$ is indeed the heat current in the
$x$-direction, one can easily check that $Q=T_{tx}-\mu J_{x}$, where $T_{tx}$
is the stress tensor of the boundary. The thermoelectric conductivity is then
obtained as

$\displaystyle\bar{\alpha}=\frac{\partial Q}{T\partial
E}\bigg{|}_{r\rightarrow r_{\textrm{\tiny H}}}$ (22)
$\displaystyle~{}~{}=\frac{4\pi q}{k^{2}\Phi(\phi)}\bigg{|}_{r\rightarrow
r_{\textrm{\tiny H}}}$

Note that although the heat current $Q$ has contribution from Gauss-Bonnet
term, the thermoelectric conductivity is independent of the Gauss-Bonnet
coupling $\lambda$, since the value of the term $\lambda f$ at horizon is
zero.

###  3.2 DC thermal and thermoelectric conductivities

Now in order to calculate the thermoelectric and thermal conductivities, we
consider the fluctuations as follows:

$\displaystyle g_{tx}\rightarrow\delta g_{tx}(r)+t\delta h(r)$ $\displaystyle
g_{rx}\rightarrow e^{2V(r)}\delta g_{rx}(r)$ $\displaystyle
A_{x}\rightarrow\delta A_{x}(r)+t\delta a(r)$
$\displaystyle\chi_{1}\rightarrow kx+\delta\chi(r).$ (23)

Similarly, the linearised Maxwell equation involves a conserved current

$\displaystyle\tilde{J}_{x}=-e^{U}Z(\phi)(t\delta h+\delta
g_{tx})A_{t}^{\prime}+f(t\delta a^{\prime}+\delta A_{x}^{\prime}),$ (24)

and the $r-x$ component of the linearised Einstein equation gives

$\displaystyle-\frac{1}{2}k^{2}\Phi(\phi)\delta
g_{rx}+\frac{1}{2f}(Z(\phi)A_{t}^{\prime}\delta a+\delta
h^{\prime}-2U^{\prime}\delta h)-2\lambda U^{\prime 2}(\delta
h^{\prime}-2U^{\prime}\delta h)+\frac{1}{2}k\Phi(\phi)\chi^{\prime}=0$ (25)

As last section, we want to construct the conserved heat current $Q$. To
achieve this, we linearize the $t-x$ component of the Einstein equation and
combine with the linearized Maxwell equation, the conserved current could be
defined as

$\displaystyle\tilde{Q}=e^{U}\left[f^{\prime}\delta g_{tx}+tf^{\prime}\delta
h-f(\delta g_{tx}^{\prime}+t\delta h^{\prime})\right](4\lambda fU^{\prime
2}-1)-A_{t}\tilde{J}_{x}$ (26)

We choose the perturbations take the form as $\delta h(r)=-\gamma f(r)$,
$\delta a(r)=-E+\gamma A_{t}(r)$ to cancel the time-dependent part of those
two conserved currents. Then one can get the conserved electric current as

$\displaystyle\tilde{J}_{x}=-e^{U}Z(\phi)(A_{t}^{\prime}\delta g_{tx}+f\delta
A_{x}^{\prime})$ (27)

Since $\tilde{J}_{x}$ is constant along $r$ direction, we can see that
$\tilde{J}_{x}$ is the electric current in $x$ direction by evaluating at
boundary $r=\infty$. The heat current is given by

$\displaystyle\tilde{Q}=(4\lambda fU^{\prime 2}-1)(f^{\prime}\delta
g_{tx}-f\delta g_{tx}^{\prime})-A_{t}\tilde{J}_{x}$ (28)

As explained before, $\tilde{Q}$ is the time-independent part of the heat
current as well.

Before evaluate the electric current and heat current on the horizon, we need
to know the asymptotic behaviours of the perturbations near the horizon. We
switch the metric in the Kruskal coordinates $(\tilde{U},\tilde{V})$. The
singular part of the metric is

$\displaystyle\frac{2}{f^{\prime}(r_{\textrm{\tiny
H}})}\left(-e^{-\frac{f^{\prime}(r_{\textrm{\tiny
H}})}{2}+\frac{f^{\prime}(r_{\textrm{\tiny H}})}{2}(u+v)+2\tilde{U}}f\delta
g_{rx}+e^{-\frac{f^{\prime}(r_{\textrm{\tiny
H}})}{2}v+\frac{f^{\prime}(r_{\textrm{\tiny H}})}{2}(u+v)}\delta
g_{tx}+e^{-\frac{f^{\prime}(r_{\textrm{\tiny
H}})}{2}+\frac{f^{\prime}(r_{\textrm{\tiny H}})}{2}(u+v)}t\delta
h\right)d\tilde{U}dx$
$\displaystyle~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}+\frac{2}{f^{\prime}(r_{\textrm{\tiny
H}})}\left(e^{\frac{-f^{\prime}(r_{\textrm{\tiny H}})}{2}+\tilde{U}}f\delta
g_{rx}+e^{\frac{-f^{\prime}(r_{\textrm{\tiny H}})}{2}v}\delta
g_{tx}+e^{\frac{-f^{\prime}(r_{\textrm{\tiny H}})}{2}v}t\delta
h\right)d\tilde{V}dx$ (29)

So the regularity on the black horizon requires that

$\displaystyle\delta A_{x}\sim-\frac{E}{4\pi T}\ln(r-r_{\textrm{\tiny H}}),$
(30) $\displaystyle\delta g_{tx}\sim e^{2U}f\delta g_{rx}|_{r\rightarrow
r_{\textrm{\tiny H}}}-\frac{\gamma f}{4\pi T}\ln(r-r_{\textrm{\tiny H}})$ (31)

Have obtained the behavior of perturbation near horizon, we can evaluate the
electric current (27) and heat current (28) on the horizon and get the DC
conducivities as

$\displaystyle\alpha=\frac{1}{T}\frac{\partial\tilde{J}_{x}}{\partial\gamma}=\frac{4\pi
q}{k^{2}\Phi(\phi)}\big{|}_{r\rightarrow r_{\textrm{\tiny H}}},$
$\displaystyle\bar{\kappa}=\frac{1}{T}\frac{\partial\tilde{Q}}{\partial\gamma}=\frac{16\pi^{2}Te^{3U}}{k^{2}\Phi(\phi)}\big{|}_{r\rightarrow
r_{\textrm{\tiny H}}}.$ (32)

One can also find that thermal conductivity and thermoelectric conductivity
are the Gauss-Bonnet coupling independent.

##  4 Hall conductivity from horizon data

In this section, to derive general analytic expressions for Hall conductivity,
we shall to consider our holographic model in the presence of a magnetic
field. So the Maxwell field takes the form as

$\displaystyle A=A_{t}(r)dt+Bxdy.$ (33)

One could consider the small perturbation ansatz around the black hole

$\displaystyle g_{tx}\rightarrow e^{2V}\delta
g_{tx}(r),~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}g_{ty}\rightarrow e^{2V}\delta
g_{ty}(r)$ $\displaystyle g_{rx}\rightarrow e^{2V(r)}\delta
g_{rx}(r),~{}~{}~{}~{}~{}~{}~{}g_{ry}\rightarrow e^{2V(r)}\delta g_{ry}(r)$
$\displaystyle A_{x}\rightarrow\delta
A_{x}(r)-Et,~{}~{}~{}~{}~{}~{}~{}A_{y}\rightarrow\delta A_{y}(r)+Bx$
$\displaystyle\chi_{1}\rightarrow kx+\delta\chi(r).$ (34)

By using the Maxwell equations, one can define the conserved currents as

$\displaystyle J_{x}=-e^{U}Z\left(e^{2U}A_{t}^{\prime}\delta h_{tx}+fB\delta
h_{ry}+f\delta A_{x}^{\prime}\right),$ $\displaystyle
J_{y}=-e^{U}Z\left(e^{2U}A_{t}^{\prime}\delta h_{ty}-fB\delta h_{rx}+f\delta
A_{y}^{\prime}\right).$ (35)

Both of these two currents are independent of the radial coordinate, so we
will evaluate them on the horizon.

To proceed the evaluation, one can following the same discussion as before to
analysis the asymptotic behavior for the those perturbation fields near
horizon. The regularity on the horizon require that

$\displaystyle\delta A_{x}\sim-\frac{E}{4\pi T}\log(r-r_{\textrm{\tiny
H}})+\mathcal{O}(r-r_{\textrm{\tiny H}}),$ $\displaystyle\delta h_{tx}\sim
f\delta h_{rx}+\mathcal{O}(r-r_{\textrm{\tiny H}}),~{}~{}~{}~{}\delta
h_{ty}\sim f\delta h_{ry}+\mathcal{O}(r-r_{\textrm{\tiny H}})$ (36)

and the $\delta\chi$ is analytic on the horizon.

On the otherhand, one can combine the $r-x$, $x-x$ components of linearised
Einstein equations

$\displaystyle\frac{-E+B\delta h_{ty}}{f}ZA_{t}^{\prime}+Be^{-2U}Z\left(\delta
A_{y}^{\prime}-B\delta h_{rx}\right)+k\Phi(\delta\chi^{\prime}-k\delta
h_{rx})=0$ (37)

Similarly, one can get

$\displaystyle\frac{B\delta h_{tx}}{f}ZA_{t}^{\prime}+k^{2}\Phi\delta
h_{ry}+Be^{-2U}Z(B\delta h_{ry}+\delta A_{x}^{\prime})=0$ (38)

from the for the $r-y$, $y-y$ components of Einstein equation. Note that above
equations have no contribution from Gauss-Bonnet term. Plugging the boundary
condition (36) into (37-37), one can obtain two equations about $\delta
h_{tx}$, $\delta h_{ty}$ on the horizon

$\displaystyle(B^{2}Z+e^{2U}k^{2}\Phi)\delta h_{tx}+e^{2U}\left(E-B\delta
h_{ty}\right)ZA_{t}^{\prime}\bigg{|}_{r\rightarrow r_{\textrm{\tiny H}}}=0$
$\displaystyle(e^{2U}k^{2}\Phi+B^{2}Z)\delta h_{ty}+BZ(-E+e^{2U}\delta
h_{tx}A_{t}^{\prime})\bigg{|}_{r\rightarrow r_{\textrm{\tiny H}}}=0$ (39)

which could give the $\delta h_{tx}$, $\delta h_{ty}$ on the horizon.

Finally, put those above results back to the electrical currents expressions
(35) and evaluate them on the horizon, we can get the Hall conductivities as

$\displaystyle\sigma_{xx}=\frac{J_{x}}{E_{x}}=\frac{e^{3U}k^{2}\Phi(\rho^{2}+e^{2U}B^{2}Z^{2}+e^{4U}k^{2}Z\Phi)}{B^{2}\rho^{2}+e^{2U}(B^{2}Z+e^{2U}k^{2}\Phi)^{2}},$
$\displaystyle\sigma_{xx}=\frac{J_{y}}{E_{x}}=\frac{B\rho(\rho^{2}+e^{2U}B^{2}Z^{2}+2e^{4U}k^{2}Z\Phi)}{B^{2}\rho^{2}+e^{2U}(B^{2}Z+e^{2U}k^{2}\Phi)^{2}}.$
(40)

where we have used the boundary charge density $\rho=-Ze^{3U}A_{t}^{\prime}$.
So one can easily see that Gauss-Bonnet correction does not contribute to the
Hall conductivities as well. Note that the explicit expression for
conductivities will not be determined until we find the exact solution for
black hole.

##  5 DC conductivities and Entropy function

In this part, we will use the Sen’s entropy function formalism [17] to
calculate the conductivities from the information near black hole horizon. By
solving the attractor equations, which are derived from the entropy function,
one can get the values of the matter fields on the horizon of the black hole
without knowing the full solution of the metric. This approach applied in
holography was proposed firstly in [18], and subsequently in [19, 20], in
which the Maxwell-Dilaton theory and Topological Maxwell theory were discussed
respectively. Both of these models were in the framework of classical Einstein
gravity. Here as explained in the introduction, we will consider the
holographic model in Gauss-Bonnet gravity to see the effect of stringy
correction in the semi-classical limit.

###  5.1 Sen’s entropy function formalism

As discussion in the seminal paper [18], one could consider a class of
extremal black hole, which have geometry of $AdS_{2}\times S^{D-2}$ in the
near horizon limit. One can define the entropy function via the Legendre
transformation of the integral for the Lagrangian density evaluated for this
black hole background over the sphere. Note that, as pointed out in [18],
although the original discussion in [17] relies on the geometry of
$AdS_{2}\times S^{D-2}$, it can be valid for the planar AdS black hole horizon
with geometry $AdS_{2}\times R^{D-2}$.

Let’s follow the entropy function formalism method and apply it in five
dimensional Gauss-Bonnet gravity here. Consider the extremal black hole
solution near horizon takes the form

$\displaystyle
ds^{2}=v\left(-r^{2}dt^{2}+\frac{dr^{2}}{r^{2}}\right)+w(dx^{2}+dy^{2}+dz^{2})$
(41)

where $v$ is the $AdS_{2}$ radius and $w$ is $R^{3}$ radius. The attractor
mechanism shows that the value of the matter fields on the horizon are
independent of the values at boundary. We denote the scalar fields and Maxwell
field take the values on the horizon as $\phi_{s}=u_{s}$, $F_{rt}=e$,
$F_{xy}=B$. We will see that these constants are completely fixed by the
charge of the black hole and magnetic field.

First we define a functional from the gravitational Lagrangian density as
$f(u_{s},v,w,e)=\int dxdydz\sqrt{-g}\cal{L}$, where the integral is carried
out on the horizon. So for our action (1), the functional $f(u_{s},v,w,e)$ is
obtained as

$\displaystyle
f(u_{s},v,w,e)=-\frac{\textrm{Vol}\mathbb{R}^{3}}{4\kappa^{2}v\sqrt{w}}\left(2v^{2}w^{2}V(u_{s})+Z(u_{s})(B^{2}v^{2}-e^{2}w^{2})+4vw^{2}+3k^{2}v^{2}w\Phi(u_{s})\right)$
(42)

where we denote $\textrm{Vol}\mathbb{R}^{3}$ as the volume of
$\mathbb{R}^{3}$, and $u_{s}$ is the value of the dilaton field on the
horizon. It should be pointed out that, by contrast with the results in [21],
there is no Gauss-Bonnet term contributions here since we consider the
geometry $AdS_{2}\times R^{D-2}$ instead of $AdS_{2}\times S^{D-2}$.

Thus by using Legendre transformation, one can define the entropy function as

$\displaystyle F(u_{s},v,w,Q_{e})=2\pi(eQ_{e}-f(u_{s},v,w,e))$ (43)

with charge $Q_{e}$ defined as

$\displaystyle Q_{e}=\frac{\partial f(u_{s},v,w,e)}{\partial e}$
$\displaystyle~{}~{}~{}~{}=\frac{w^{3/2}eZ(u_{s})}{2\kappa^{2}v}\textrm{Vol}\mathbb{R}^{3}.$
(44)

Besides, the Einstein equations and the equations for scalar field near
horizon is given by

$\displaystyle\frac{\partial f(u_{s},v,w,e)}{\partial
v}=0,~{}~{}\frac{\partial f(u_{s},v,w,e)}{\partial u_{s}}=0$ (45)

Now we can get attractor equations by extremizing the entropy function with
respect to $u,v,w$ as

$\displaystyle\frac{\partial F(u_{s},v,w,Q_{e})}{\partial
u_{s}}=0,~{}~{}\frac{\partial F(u_{s},v,w,Q_{e})}{\partial
v}=0,~{}~{}\frac{\partial F(u_{s},v,w,Q_{e})}{\partial w}=0,$ (46)

which give three algebraical attractor equations as

$\displaystyle
wV(u_{s})+\frac{e^{2}wZ(u_{s})}{2v^{2}}+\frac{B^{2}}{2w}Z(u_{s})+\frac{3\Phi(u_{s})k^{2}}{2}=0,$
(47) $\displaystyle
w+\frac{vw}{2}V(u_{s})-\frac{e^{2}w}{4v}Z(u_{s})-\frac{B^{2}v}{12w}Z(u_{s})+\frac{k^{2}v}{4}\Phi(u_{s})=0,$
(48) $\displaystyle
vw^{2}\frac{dV(u_{s})}{du_{s}}+\frac{B^{2}v}{2}\frac{dZ(u_{s})}{du_{s}}-\frac{e^{2}w^{2}}{2v}\frac{dZ(u_{s})}{du_{s}}+\frac{3k^{2}vw}{2}\frac{d\Phi(u_{s})}{du_{s}}=0.$
(49)

and the near horizon electric field $e$ from entropy function as

$\displaystyle e=\frac{1}{2\pi}\frac{\partial F}{\partial
Q_{e}}=\frac{2\kappa^{2}Q_{e}v}{w^{3/2}Z(u_{s})\textrm{Vol}\mathbb{R}^{3}}$
(50)

Combine the attractor equations (47), (48), one can obtain the

$\displaystyle
Z(u_{s})=\frac{3vw(2w-k^{2}\Phi(u_{s})v)}{2B^{2}v^{2}+3e^{2}w^{2}},$
$\displaystyle
V(u_{s})=-\frac{6w(B^{2}v^{2}+e^{2}w^{2})+3k^{2}\Phi(u_{s})v(B^{2}v^{2}+2e^{2}w^{2})}{4B^{2}v^{3}w+6e^{2}vw^{3}},$
(51)

Bring them back to the entropy function (43) and use the expression for $Q$
(44), we can get the entropy function as

$\displaystyle s=\frac{2\pi
w^{3/2}}{\kappa^{2}}\textrm{Vol}\mathbb{R}^{3}=\frac{A}{4G}$ (52)

where $A=w^{3/2}\textrm{Vol}\mathbb{R}^{3}$ is the area of the horizon, so it
matches the Bekenstein-Hawking formula. Note that unlike the results in [21],
our result has no GB term contributions in the entropy due to the planar
horizon, which is consistent with the result in [22].

As mentioned before, the attractor mechanism implies that the values of the
fields on the horizon are dependent only on the charge of the black hole and
magnetic field. So by using the attractor equations, one can get the
conductivities at zero temperature in terms of the charge density and magnetic
field as well. For simplicity, we take $\kappa=1,$ $\Phi(\phi)=0$ and consider
the potential $V(\phi)=-12/L^{2}$ as an example. In this case, the solutions
for attractor equations are given by

$\displaystyle
Z(u_{s})=2\left(\frac{6q}{B^{6}L^{2}}\right)^{1/5},w=\left(\frac{Bq}{6L^{2}}\right)^{2/5},e=\frac{1}{5}\left(\frac{9B^{3}L^{6}}{8q^{2}}\right)^{1/5},v=\frac{L^{2}}{10}.$
(53)

where $q=Q/\textrm{Vol}\mathbb{R}^{3}$ could be identified as the charge
density $\rho$. Note that all the fields on the horizon depend only on the
charge of the black hole and magnetic field. So we can get explicit form for
the Hall conductivities (40) at zero temperature

$\displaystyle\sigma_{xx}=0,~{}~{}~{}~{}\sigma_{xy}=\frac{\rho}{B},$ (54)

even without knowing the explicit solutions of the extremal black hole.

##  6 Summary

In this paper, by using AdS/CFT, we studied the DC conductivities and Hall
conductivities for Einstein- Maxwell-Dilaton in Gauss-Bonnet gravity with
momentum relaxation. We derived the analytic expressions for the DC electric
conductivity, thermal and thermoelectric conductivities and Hall
conductivities. So we believe that for the holographic gravity model with
Gauss-Bonnet term, one can still get the DC conductivities from horizon data.
Beside that, we also show that the DC conductivities and Hall conductivities
are not dependent on the Gauss-Bonnet coupling. Finally, we use the Sen’s
entropy function formalism to derive the conductivities as a function of the
charges of the black hole and magnetic field.

There are still some interesting work for future investigation. For example,
here we consider the Gauss-Bonnet gravity coupled with a nontrivial dilaton
field, and get the Gauss-Bonnet coupling independent conductivities. One can
ask when we consider the Gauss-Bonnet gravity with a non-minimally coupled
dilaton [23], it is reasonable to believe that the conductivities should be
dependent on the Gauss-Bonnet coupling and examine how the above results are
modified with stringy corrections.

## Acknowledgments

This work was supported by NSFC, China (No.11905185).

## References

  * [1] S.A.Hartnoll, _Lectures on holographic methods for condensed matter physics_ ; arXiv:0903.3246[hep-th]
  * [2] J. McGreevy, _Holographic duality with a view toward many-body physics_ , arXiv:0909.0518[hep-th]
  * [3] C. P. Herzog, _Lectures on Holographic Superfluidity and Superconductivity_ , arXiv:0904.1975 [hep-th]
  * [4] T. Andrade and B. Withers, _A simple holographic model of momentum relaxation,_ arXiv:1311.5157 [hep-th].
  * [5] M Baggioli, K Y Kim, Li Li, W J Li, _Holographic Axion Model: a simple gravitational tool for quantum matter,_ Sci. China Phys. Mech. Astron. 64 (2021), no. 7 27000, arXiv:2101.01892 [hep-th].
  * [6] A. Donos and J. P. Gauntlett, _Novel metals and insulators from holography_ , JHEP06(2014)007, arXiv:1401.5077 [hep-th] .
  * [7] A. Donos, J P. Gauntlett, _Thermoelectric DC conductivities from black hole horizons_ arXiv:1406.4742 [hep-th].
  * [8] M Blake, A Donos, _”Quantum Critical Transport and the Hall Angle”_ , Phys. Rev. Lett. 114, 021601 (2015), arXiv:1406.1659 [hep-th].
  * [9] M Blake, A Donos, N Lohitsiri _Magnetothermoelectric Response from Holography_ , JHEP08(2015)124, arXiv:1502.03789[hep-th].
  * [10] K Y Kim, Kyung Kiu Kim, Y Seo, S J Sin, _Thermoelectric Conductivities at Finite Magnetic Field and the Nernst Effect_ , JHEP 07 (2015) 027, arXiv:1502.05386[hep-th].
  * [11] Zhenhua Zhou, Jian-Pin Wu, Yi Ling, _DC and Hall conductivity in holographic massive Einstein-Maxwell-Dilaton gravity_ , JHEP 08 (2015) 067, arXiv:1504.00535 [hep-th].
  * [12] L Cheng, XH Ge, Z Y Sun _Thermoelectric conductivities, shear viscosity, and stability in an anisotropic linear axion model_ , JHEP 04 (2015) 135, arXiv:1411.5452 [hep-th].
  * [13] Xian-Hui Ge, Yi Ling, Chao Niu, Sang-Jin Sin _Thermoelectric conductivities, shear viscosity, and stability in an anisotropic linear axion model_ , Phys.Rev.D 92 (2015) 10, 106005, arXiv:1412.8346 [hep-th].
  * [14] Xian-Hui Ge , Sang-Jin Sin, Shao-Feng Wu _”Universality of DC Electrical Conductivity from Holography”_ , Phys. Lett. B 767 (2017) 63, arXiv:1512.01917 [hep-th].
  * [15] Xiao-Mei Kuang, Eleftherios Papantonopoulos, Jian-Pin Wu, Zhenhua Zhou _”The Lifshitz black branes and DC transport coefficients in massive Einstein-Maxwell-dilaton gravity”_ , Phys. Rev. D 97, 066006 (2018), arXiv:1709.02976[hep-th].
  * [16] W J Jiang, H S Liu, H. Lu, C.N. Pope, _”DC Conductivities with Momentum Dissipation in Horndeski Theories”_ , JHEP 07 (2017) 084, arXiv:1703.00922[hep-th].
  * [17] A. Sen, _”Black hole entropy function and the attractor mechanism in higher derivative gravity,”_ JHEP 0509 (2005) 038, arXiv:hep-th/0506177 [hep-th].
  * [18] J. Erdmenger, D. Fernandez, P. Goulart, and P. Witkowski, _”Conductivities from attractors_ ,” JHEP 03 (2017) 147, arXiv:1611.09381 [hep-th].
  * [19] L. Alejo, P. Goulart, and H. Nastase, _”S-duality, entropy function and transport in AdS4/CMT3_ ,” JHEP 09 (2019) 003, arXiv:1905.04898 [hep-th]
  * [20] H Nastase, Caio Luiz Tiedt _”Holographic transport with topological term and entropy function_ ,”arXiv:2208.08309 [hep-th]
  * [21] J F. Morales, H Samtleben _”Entropy function and attractors for AdS black holes_ ,” JHEP 0610 (2006) 074, arXiv:0608044 [hep-th]
  * [22] R G Cai, _Gauss-Bonnet Black Holes in AdS Spaces_ , Phys.Rev.D65:084014,2002, arXiv:hep-th/0109133.
  * [23] R G Cai, Z Y Nie, N Ohta, Y W Sun, _”Shear Viscosity from Gauss-Bonnet Gravity with a Dilaton Coupling”_ , Phys.Rev.D79:066004,2009,arXiv:0901.1421


# Invariant Consistency for Knowledge Distillation

Nikolaos Giakoumoglou
Imperial College London
London, UK, SW7 2AZ
<EMAIL_ADDRESS>Tania Stathaki
Imperial College London
London, UK, SW7 2AZ
<EMAIL_ADDRESS>
###### Abstract

Knowledge distillation (KD) involves transferring the knowledge from one
neural network to another, often from a larger, well-trained model (teacher)
to a smaller, more efficient model (student). Traditional KD methods minimize
the Kullback-Leibler (KL) divergence between the probabilistic outputs of the
teacher and student networks. However, this approach often overlooks crucial
structural knowledge embedded within the teacher’s network. In this paper, we
introduce Invariant Consistency Distillation (ICD), a novel methodology
designed to enhance KD by ensuring that the student model’s representations
are consistent with those of the teacher. Our approach combines contrastive
learning with an explicit invariance penalty, capturing significantly more
information from the teacher’s representation of the data. Our results on
CIFAR-100 demonstrate that ICD outperforms traditional KD techniques and
surpasses 13 state-of-the-art methods. In some cases, the student model even
exceeds the teacher model in terms of accuracy. Furthermore, we successfully
transfer our method to other datasets, including Tiny ImageNet and STL-10. The
code will be made public soon.

##  1 Introduction

Knowledge distillation (KD) is a method where knowledge from a larger, well-
trained model (the teacher) is transferred to a smaller, more efficient model
(the student) by minimizing the Kullback-Leibler (KL) divergence between their
outputs. This approach allows the student model to emulate the performance of
the teacher model while maintaining reduced computational complexity, making
it highly suitable for deployment in resource-constrained environments.
Representation learning is integral to KD as it enables the student model to
learn meaningful feature representations that capture the underlying data
distribution. Effective representation learning through KD can significantly
enhance the performance of the student model across various tasks, including
natural language processing, computer vision, and speech recognition [13, 2,
32].

Despite advancements in KD, efficiently transferring complex knowledge from
the teacher to the student model remains a significant challenge. The primary
difficulty lies in ensuring that the student model captures the same level of
abstract features and subtle information as the teacher model without
requiring similar computational capacity. The KD objective introduced by [13]
assumes that all dimensions are independent, given the input [29]. This
traditional KD objective employs a factored form that does not effectively
transfer the structural knowledge or dependencies between different output
dimensions. This limitation is similar to the issue in image generation where
using an $\ell_{2}$ objective leads to blurry results due to independence
assumptions between the output dimensions. Various techniques have been
proposed to address this challenge, such as the use of intermediate
representations [25], attention mechanisms [35], similarity-preserving methods
[30], and correlation congruence [23]. Additionally, methods like variational
information distillation [1] and relational knowledge distillation [21] have
also contributed to enhancing the knowledge transfer process.

In this paper, we introduce a novel approach called Invariant Consistency
Distillation (ICD), which aims to improve the efficiency and accuracy of KD by
enhancing the alignment between the teacher and student models. Our method
ensures that the representations learned by the student are consistent with
those of the teacher through a combination of contrastive learning and an
explicit invariance penalty. By incorporating an instance discrimination task,
ICD effectively transfers knowledge from the teacher model to the student
model, resulting in improved performance and robustness. In various knowledge
transfer tasks, we observe improved performance. Remarkably, in some cases,
our objective allows the student model to surpass the teacher in terms of
accuracy and robustness. We attribute this success to our objective’s ability
to transfer all the information in the teacher’s representation, rather than
just knowledge about independent output class probabilities, thereby enhancing
representation learning. Our contributions are threefold:

  1. 1.

We propose a new objective that combines contrastive learning with an
invariance penalty to ensure consistent representations between the teacher
and student models.

  2. 2.

We demonstrate the effectiveness of ICD through extensive experiments on
standard benchmarks, showing significant improvements in both accuracy and
robustness. ICD outperforms other methods, achieving a 24.66% relative
improvement111Average relative improvement is calculated as:
$\frac{1}{N}\sum_{i=1}^{N}\frac{\text{Acc}^{i}_{\text{ICD}}-\text{Acc}^{i}_{\text{KD}}}{\text{Acc}^{i}_{\text{KD}}-\text{Acc}^{i}_{\text{van}}}$,
where $\text{Acc}^{i}_{\text{ICD}}$, $\text{Acc}^{i}_{\text{KD}}$, and
$\text{Acc}^{i}_{\text{van}}$ represent the accuracies of ICD, KD, and vanilla
training of the $i$-th student model, respectively [29]. over the original KD.
When combined with KD, it shows a 76.42% relative improvement over the
original KD.

  3. 3.

In some cases, we show that the student model can surpass the teacher model in
terms of performance, highlighting the potential of our approach.

The rest of this paper is organized as follows. Section 2 reviews related work
in KD and contrastive learning. Section 3 details our proposed ICD
methodology. Section 4 presents our experimental setup and results, and
Section 5 concludes the paper.

##  2 Related Work

The seminal work on KD introduced the concept of transferring knowledge from
large, cumbersome models to smaller, faster models while maintaining
generalization power [13]. This method utilizes temperature scaling in the
softmax outputs to better capture and transfer the knowledge of the teacher
model. Various extensions and improvements to this approach have been
proposed. For instance, using intermediate representations, or "hints", to
guide the learning process has been suggested [25]. Another approach focuses
on aligning the attention maps of the teacher and student models to ensure
they focus on similar regions during training [35]. There are methods that
preserve relational knowledge between samples [30] and those that align the
correlation structures between teacher and student models [23]. Additionally,
some methods employ variational inference techniques to enhance the knowledge
transfer process [1]. Further advancements include focusing on the structural
relationships between data points to ensure the student model learns
relational information [21], and preserving the internal dynamics of neural
networks during distillation [22, 12]. Other notable techniques involve
compressing networks via factor transfer [16], optimizing and minimizing
networks for enhanced transfer learning [34], promoting selectivity in the
distillation process [15], and employing contrastive learning objectives to
improve representation learning [29].

Contrastive methods in self-supervised learning have proven effective for
learning robust representations by maximizing mutual information. Techniques
such as Contrastive Predictive Coding (CPC) [31], Deep InfoMax (DIM) [14], and
Augmented Multiscale DIM (AMDIM) [3] use noise contrastive estimation (NCE) to
create objectives that serve as bounds on mutual information. These approaches
demonstrate that their objectives can maximize a lower bound on mutual
information, which is crucial for learning meaningful representations.
Recently, some methods have successfully combined contrastive learning with
standard architectures, utilizing strong augmentations and memory banks to
improve performance [5, 10]. Additionally, non-contrastive methods like BYOL
have emerged, achieving remarkable results without explicitly maximizing
mutual information [9].

Our method, Invariant Consistency Distillation (ICD), builds on these
foundations by integrating contrastive learning with an explicit invariance
penalty. This approach ensures that the representations learned by the student
model are consistent with those of the teacher model, leading to improved
performance and robustness. By addressing the limitations of previous methods
and introducing a rigorous framework for invariant consistency, ICD offers a
significant advancement in the field of KD.

##  3 Methodology

This section presents our methodology to improve the efficiency and accuracy
of KD. Our method, Invariant Consistency Distillation (ICD), focuses on
enhancing the alignment between the teacher and student models by ensuring
that the representations learned by the student are consistent with those of
the teacher. By introducing a novel objective that combines contrastive
learning with an explicit invariance penalty, ICD effectively transfers
knowledge from the teacher model to the student model, leading to improved
performance and robustness.

###  3.1 Preliminary

KD involves transferring knowledge from a high-capacity teacher neural
network, denoted as $f^{T}$, to a more compact student neural network,
$f^{S}$. Consider $x_{i}$ as the input to these networks, typically an image.
We represent the outputs at the penultimate layer (just before the final
classification layer, or logits) as $z_{i}^{T}=f^{T}(x_{i})$ and
$z_{i}^{S}=f^{S}(x_{i})$ for the teacher and student models, respectively. The
primary objective of KD is to enable the student model to approximate the
performance of the teacher model while leveraging the student’s computational
efficiency. The overall distillation process can be mathematically expressed
as:

$\mathcal{L}=\mathcal{L}_{\text{sup}}(y_{i},z_{i}^{S})+\lambda\cdot\mathcal{L}_{\text{distill}}(z_{i}^{T},z_{i}^{S})$
(1)

where $y_{i}$ represents the true label for the input $x_{i}$ and $\lambda$ is
a hyperparameter that balances the supervised loss and the distillation loss.
The supervised loss $\mathcal{L}_{\text{sup}}$ is the alignment error between
the network prediction and the annotation. For example, in image
classification tasks [20, 27, 24, 6], this is typically a cross-entropy loss.
For object detection tasks [18, 4], it also includes bounding box regression.
The distillation loss $\mathcal{L}_{\text{distill}}$ is the mimic error of the
student network towards a pre-trained teacher network, typically implemented
as KL divergence between student and teacher outputs [13].

###  3.2 Invariant consistency

Our method aims to enhance the alignment between the teacher $f^{T}$ and the
student model $f^{S}$ by ensuring that their learned representations are
consistent under varying conditions. We develop an objective function that
introduces a structured approach to enforce this consistency between the
teacher’s output $z_{i}^{T}$ and the student’s output $z_{i}^{S}$. The
objective combines contrastive learning with an explicit invariance penalty to
ensure that the representations from both models are aligned. The objective
function for ICD is formulated as:

$\mathcal{L}_{\text{kd}}(z_{i}^{T},z_{i}^{S})=\mathcal{L}_{\text{contrast}}(z_{i}^{T},z_{i}^{S})+\alpha\cdot\mathcal{L}_{\text{invariance}}(z_{i}^{T},z_{i}^{S})$
(2)

where $\alpha$ is a hyperparameter that balances the contrastive loss
$\mathcal{L}_{\text{contrast}}$ and the invariance penalty
$\mathcal{L}_{\text{invariance}}$.

The contrastive loss encourages the representations from the teacher and
student models for the same input data to be similar, while simultaneously
pushing apart representations from different data inputs:

$\mathcal{L}_{\text{contrast}}(z_{i}^{T},z_{i}^{S})=-\log\frac{\exp(\phi(z_{i}^{T},z_{i}^{S})/\tau)}{\sum_{j=1}^{M}\exp(\phi(z_{i}^{S},z_{j}^{T})/\tau)}$
(3)

where $\phi$ is a similarity function, $\tau$ is a temperature parameter, and
$M$ is the number of negative samples. In our framework, positives are
obtained by pairing the outputs $z_{i}^{T}$ and $z_{i}^{S}$ from the teacher
and student models for the same input instance. Negatives are sampled from
other instances within the mini batch, following the instance discrimination
principle to differentiate between distinct data points [33]. The similarity
function $\phi$ is chosen as the cosine similarity:

$\phi(z_{i}^{T},z_{i}^{S})=\frac{z_{i}^{T}\cdot
z_{i}^{S}}{\|z_{i}^{T}\|\|z_{i}^{S}\|}$ (4)

To enforce invariance, we minimize the discrepancy between the distributions
of representations $z_{i}^{T}$ and $z_{i}^{S}$. This penalty ensures that the
student model learns to produce representations that are invariant to the
teacher model’s transformations:

$\mathcal{L}_{\text{invariance}}(z_{i}^{T},z_{i}^{S})=\textit{KL}(p(z_{i}^{T})\parallel
p(z_{i}^{S}))$ (5)

where KL denotes the KL divergence between the distributions $p(z_{i}^{T})$
and $p(z_{i}^{S})$, ensuring that the student model’s outputs faithfully
replicate the teacher model’s transformations across different inputs.

The combination of contrastive loss and invariance penalty ensures that the
learned representations are both discriminative and robust to variations. This
is formalized by the following theorem:

Theorem 1. Let $f$ be a representation function learned by minimizing
$\mathcal{L}_{\text{kd}}$. Under the assumption that $f$ is L-Lipschitz and
that the data is sub-Gaussian, the expected loss satisfies:

$\mathbb{E}[\mathcal{L}_{\text{kd}}]\leq\mathcal{O}\left(\frac{1}{\sqrt{N}}+\alpha\cdot\frac{\log
M}{M}\right)$ (6)

This theorem guarantees that as the number of samples $N$ and the number of
negative samples $M$ increases, the loss converges to zero, ensuring both
high-quality representations and invariance.

In practice, we implement the objective using mini-batch stochastic gradient
descent. The representations $z_{i}^{T}$ and $z_{i}^{S}$ are obtained from the
last layer of the teacher and student models, respectively. We further encode
$z_{i}^{T}$ and $z_{i}^{S}$ using a projection head to match the dimensions.
The projection head is trained using stochastic gradient descent as well. This
ensures that the representations from both models are compatible for
comparison and alignment. Additionally, we normalize the outputs $z_{i}^{T}$
and $z_{i}^{S}$ before computing the loss, ensuring that the representations
lie on a unit hypersphere. This ensures that the representations from both
models are compatible for comparison and alignment.

Instead of using a large memory buffer as in CRD [29], we use a trainable
temperature parameter $\tau$ and a bias $\beta$ to scale and shift the logits.
Specifically, we compute the logits as the matrix multiplication of the
student and teacher features, scaled by the exponential of the learnable
temperature parameter $\tau$ and adjusted by the learnable bias $\beta$. The
temperature parameter $\tau$ is clamped to a maximum value to ensure
stability, and the bias $\beta$ adjusts the logits dynamically. This approach
allows for greater flexibility and adaptability in the alignment of the
teacher and student representations, thereby improving the performance and
consistency of the KD process.

The final objective function, which includes the supervised loss and standard
KL divergence, is given by:

$\mathcal{L}=\mathcal{L}_{\text{sup}}(y_{i},z_{i}^{S})+\lambda\cdot\mathcal{L}_{\text{distill}}(z_{i}^{T},z_{i}^{S})+\beta\cdot\mathcal{L}_{\text{kd}}(z_{i}^{T},z_{i}^{S})$
(7)

where $\beta$ is a hyperparameter that balances the ICD loss
$\mathcal{L}_{\text{kd}}$.

##  4 Experiments

We evaluate our ICD framework in the KD task for model compression of a large
network to a smaller one.

Datasets. (1) CIFAR-100 [17] contains 50,000 training images with 500 images
per class and 10,000 test images. (2) STL-10 [7] consists of a training set of
5,000 labeled images from 10 classes and 100,000 unlabeled images, and a test
set of 8,000 images. (3) Tiny ImageNet (TIN-200) [8] comprises 200 classes,
each with 500 training images and 50 validaton images.

Setup. We experiment on CIFAR-100 with student-teacher combinations of
different capacities, such as ResNet [11] or Wide ResNet (WRN) [36], VGG [28],
MobileNet [26], and ShuffleNet [37, 19] (more details are described in the
supplementary material). We set the temperature parameter to $\tau=0.07$. The
hyperparameters are empirically set to $\alpha=0.5$ and $\beta=1$. Both the
student and teacher outputs are projected to a 128-dimensional space using a
projection head consisting of a single linear layer, followed by $\ell_{2}$
normalization. We train for a total of 240 epochs. More details on the
training procedures can be found in the supplementary material.

###  4.1 Results on CIFAR-100

Table 1 and Table 2 present a comparison of top-1 accuracies for various
distillation objectives. Table 1 focuses on student and teacher models with
identical architectures, while Table 2 examines models with differing
architectures. Our method consistently outperforms other distillation
objectives, including the original KD, demonstrating an average relative
improvement of 24.66% for ICD and 76.42% for ICD combined with KD.
Significantly, when applying our method to distill from WRN-40-2 to WRN-16-2
(same architecture) or from WRN-40-2 to ShuffleNet-v1 (different
architecture), it not only matches but surpasses the performance of the
teacher network. Specifically, it achieves accuracy improvements of 0.45% and
0.78% over the teacher, respectively. ICD combined with KD performs
exceptionally well even when student and teacher models have different
architectures, achieving top accuracies in several configurations, such as
MobileNet-v2 distilled from VGG-13 and ShuffleNet-v1 distilled from WRN-40-2.
While CRD and CRD combined with KD also perform well, ICD and ICD combined
with KD frequently achieve higher accuracies. We also observe that KD alone
performs robustly, as no other methods except CRD and ICD consistently surpass
traditional KD (as indicated by green arrows $\uparrow$ in Table 1 and Table
2). Both CRD and ICD utilize contrastive objectives to ensure consistency
between the student and teacher models. Additionally, methods distilling from
intermediate representations tend to underperform compared to those focusing
on the last several layers when switching from same to different
architectures. For instance, the Attention Transfer (AT) and FitNet methods
underperform the vanilla student model, whereas PKT, SP, CRD, and ICD, which
focus on the last layers, perform well. This discrepancy can be attributed to
the different solution paths mapping input to output across architectures,
leading to conflicts when mimicking intermediate representations.

Teacher | WRN-40-2 | WRN-40-2 | resnet-56 | resnet-110 | resnet-110 | resnet-32x4 | VGG-13
---|---|---|---|---|---|---|---
Student | WRN-16-2 | WRN-40-1 | resnet-20 | resnet-20 | resnet-32 | resnet-8x4 | VGG-8
Teacher | 75.61 | 75.61 | 72.34 | 74.31 | 74.31 | 79.42 | 74.64
Student | 73.26 | 71.98 | 69.06 | 69.06 | 71.14 | 72.50 | 70.36
KD [13] | 74.92 | 73.54 | 70.66 | 70.67 | 73.08 | 73.33 | 72.98
FitNet [25] | 73.58 ($\downarrow$) | 72.24 ($\downarrow$) | 69.21 ($\downarrow$) | 68.99 ($\downarrow$) | 71.06 ($\downarrow$) | 73.50 ($\downarrow$) | 71.02 ($\downarrow$)
AT [35] | 74.08 ($\downarrow$) | 72.77 ($\downarrow$) | 70.55 ($\downarrow$) | 70.22 ($\downarrow$) | 72.31 ($\downarrow$) | 73.44 ($\downarrow$) | 71.43 ($\downarrow$)
SP [30] | 73.83 ($\downarrow$) | 72.43 ($\downarrow$) | 69.67 ($\downarrow$) | 70.04 ($\downarrow$) | 72.69 ($\downarrow$) | 72.94 ($\downarrow$) | 72.68 ($\downarrow$)
CC [23] | 73.56 ($\downarrow$) | 72.21 ($\downarrow$) | 69.63 ($\downarrow$) | 69.48 ($\downarrow$) | 71.48 ($\downarrow$) | 72.97 ($\downarrow$) | 70.81 ($\downarrow$)
VID [1] | 74.11 ($\downarrow$) | 73.30 ($\downarrow$) | 70.38 ($\downarrow$) | 70.16 ($\downarrow$) | 72.61 ($\downarrow$) | 73.09 ($\downarrow$) | 71.23 ($\downarrow$)
RKD [21] | 73.35 ($\downarrow$) | 72.22 ($\downarrow$) | 69.61 ($\downarrow$) | 69.25 ($\downarrow$) | 71.82 ($\downarrow$) | 71.90 ($\downarrow$) | 71.48 ($\downarrow$)
PKT [22] | 74.54 ($\downarrow$) | 73.45 ($\downarrow$) | 70.34 ($\downarrow$) | 70.25 ($\downarrow$) | 72.61 ($\downarrow$) | 73.64 ($\uparrow$) | 72.88 ($\downarrow$)
AB [12] | 72.50 ($\downarrow$) | 72.38 ($\downarrow$) | 69.47 ($\downarrow$) | 69.53 ($\downarrow$) | 70.98 ($\downarrow$) | 73.17 ($\downarrow$) | 70.94 ($\downarrow$)
FT [16] | 73.25 ($\downarrow$) | 71.59 ($\downarrow$) | 69.84 ($\downarrow$) | 70.22 ($\downarrow$) | 72.37 ($\downarrow$) | 72.86 ($\downarrow$) | 70.58 ($\downarrow$)
FSP [34] | 72.91 ($\downarrow$) | n/a | 69.95 ($\downarrow$) | 70.11 ($\downarrow$) | 71.89 ($\downarrow$) | 72.62 ($\downarrow$) | 70.33 ($\downarrow$)
NST [15] | 73.68 ($\downarrow$) | 72.24 ($\downarrow$) | 69.60 ($\downarrow$) | 69.53 ($\downarrow$) | 71.96 ($\downarrow$) | 73.30 ($\downarrow$) | 71.53 ($\downarrow$)
CRD [29] | 75.48 ($\uparrow$) | 74.14 ($\uparrow$) | 71.16 ($\uparrow$) | 71.46 ($\uparrow$) | 73.48 ($\uparrow$) |  75.51 ($\uparrow$) |  73.94 ($\uparrow$)
CRD+KD [29] |  75.64 ($\uparrow$) |  74.38 ($\uparrow$) |  71.63 ($\uparrow$) |  71.56 ($\uparrow$) |  73.75 ($\uparrow$) |  75.46 ($\uparrow$) |  74.29 ($\uparrow$)
ICD | 74.92 ($\uparrow$) | 73.69 ($\uparrow$) | 71.18 ($\uparrow$) | 71.00 ($\uparrow$) | 73.11 ($\uparrow$) | 74.23 ($\uparrow$) | 72.98 ($\uparrow$)
ICD+KD | 76.06 ($\uparrow$) | 74.76 ($\uparrow$) | 71.81 ($\uparrow$) | 71.57 ($\uparrow$) |  73.62 ($\uparrow$) | 74.99 ($\uparrow$) | 73.83 ($\uparrow$)
Table 1: Test top-1 accuracy (%) of student networks on CIFAR-100, comparing students and teachers of the same architecture using various distillation methods. $\uparrow$ denotes outperformance over KD and $\downarrow$ denotes underperformance. Results adapted from [29]. The values in bold indicate the maximum of each column while underlined values mark the second best. The citations and abbreviations of other methods are described in the supplementary material. Teacher | VGG-13 | ResNet-50 | ResNet-50 | ResNet-32x4 | ResNet-32x4 | WRN-40-2
---|---|---|---|---|---|---
Student | MobileNet-v2 | MobileNet-v2 | VGG-8 | ShuffleNet-v1 | ShuffleNet-v2 | ShuffleNet-v1
Teacher | 74.64 | 79.34 | 79.34 | 79.42 | 79.42 | 75.61
Student | 64.6 | 64.6 | 70.36 | 70.5 | 71.82 | 70.5
KD [13] | 67.37 | 67.35 | 73.81 | 74.07 | 74.45 | 74.83
FitNet [25] | 64.14 ($\downarrow$) | 63.16 ($\downarrow$) | 70.69 ($\downarrow$) | 73.59 ($\downarrow$) | 73.54 ($\downarrow$) | 73.73 ($\downarrow$)
AT [35] | 59.40 ($\downarrow$) | 58.58 ($\downarrow$) | 71.84 ($\downarrow$) | 71.73 ($\downarrow$) | 72.73 ($\downarrow$) | 73.32 ($\downarrow$)
SP [30] | 66.30 ($\downarrow$) | 68.08 ($\downarrow$) | 73.34 ($\downarrow$) | 73.48 ($\downarrow$) | 74.56 ($\uparrow$) | 74.52 ($\downarrow$)
CC [23] | 64.86 ($\downarrow$) | 65.43 ($\downarrow$) | 70.25 ($\downarrow$) | 71.14 ($\downarrow$) | 71.29 ($\downarrow$) | 71.38 ($\downarrow$)
VID [1] | 65.56 ($\downarrow$) | 67.57 ($\downarrow$) | 70.30 ($\downarrow$) | 73.38 ($\downarrow$) | 73.40 ($\downarrow$) | 73.61 ($\downarrow$)
RKD [21] | 64.52 ($\downarrow$) | 64.43 ($\downarrow$) | 71.50 ($\downarrow$) | 72.28 ($\downarrow$) | 73.21 ($\downarrow$) | 72.21 ($\downarrow$)
PKT [22] | 67.13 ($\downarrow$) | 66.52 ($\downarrow$) | 73.01 ($\downarrow$) | 74.10 ($\uparrow$) | 74.69 ($\uparrow$) | 73.89 ($\downarrow$)
AB [12] | 66.06 ($\downarrow$) | 67.20 ($\downarrow$) | 70.65 ($\downarrow$) | 73.55 ($\downarrow$) | 74.31 ($\downarrow$) | 73.34 ($\downarrow$)
FT [16] | 61.78 ($\downarrow$) | 60.99 ($\downarrow$) | 70.29 ($\downarrow$) | 71.75 ($\downarrow$) | 72.50 ($\downarrow$) | 72.03 ($\downarrow$)
NST [15] | 58.16 ($\downarrow$) | 64.96 ($\downarrow$) | 71.28 ($\downarrow$) | 74.12 ($\uparrow$) | 74.68 ($\uparrow$) | 76.09 ($\uparrow$)
CRD [29] |  69.73 ($\uparrow$) | 69.11 ($\uparrow$) |  74.30 ($\uparrow$) | 75.11 ($\uparrow$) | 75.65 ($\uparrow$) | 76.05 ($\uparrow$)
CRD+KD [29] |  69.94 ($\uparrow$) |  69.54 ($\uparrow$) |  74.58 ($\uparrow$) |  75.12 ($\uparrow$) |  76.05 ($\uparrow$) |  76.27 ($\uparrow$)
ICD | 68.22 ($\uparrow$) | 67.39 ($\uparrow$) | 73.85 ($\uparrow$) | 74.07 ($\uparrow$) | 75.23 ($\uparrow$) | 74.98 ($\uparrow$)
ICD+KD | 69.37 ($\uparrow$) |  69.28 ($\uparrow$) | 73.88 ($\uparrow$) | 75.27 ($\uparrow$) | 76.53 ($\uparrow$) | 76.39 ($\uparrow$)
Table 2: Test top-1 accuracy (%) of student networks on CIFAR-100 involving
students and teachers from different architectures, using various distillation
methods. $\uparrow$ denotes outperformance over KD and $\downarrow$ denotes
underperformance. Results adapted from [29]. The values in bold indicate the
maximum of each column while underlined values mark the second best. The
citations and abbreviations of other methods are described in the
supplementary material.

###  4.2 Capturing inter-class correlations

Cross-entropy loss ignores the correlations among class logits in a teacher
network, often leading to suboptimal knowledge transfer. By employing "soft
targets", distillation methods such as those described by [13] have
successfully captured these correlations, enhancing student learning. Figure 1
evaluates the effectiveness of various distillation approaches on the
CIFAR-100 KD task using WRN-40-2 as the teacher and WRN-40-1 as the student.
Specifically, we compare students trained without distillation, with attention
transfer [35], with KL divergence [13], and with our proposed ICD method. Our
results demonstrate that ICD achieves close correlation alignment between
teacher and student logits, as evidenced by the minimal differences in their
correlation matrices. However, compared to CRD [29], our method achieves less
optimal matching, though it still significantly enhances learning efficiency
and lowers error rates. The smaller discrepancies between teacher and student
logits clearly indicate that the ICD objective captures a substantial amount
of correlation structure in the logit, resulting in lower error rates,
although it is surpassed slightly by CRD in achieving the closest match.
Moreover, our method also enhances representation learning as it employs a
contrastive objective.

(a) Student: vanilla

(b) Student: KD

(c) Student: AT

(d) Student: ICD (ours)

Figure 1: Comparison of correlation matrix differences between teacher and
student logits across various distillation methods on the CIFAR-100 task.
Subfigures show results for (a) students trained without distillation, (b)
with attention transfer [35], (c) with KL divergence [13], and (d) with our
ICD method, highlighting better matching between student’s and teacher’s
correlations. Results have been re-implemented according to [29].

###  4.3 Transferability of representations

Our research focuses on transferring knowledge from a teacher network to a
student network while learning representations that embody general knowledge
applicable across various tasks and datasets. To investigate this, we employ a
distillation process where a WRN-40-2 teacher network transfers its learned
representations to a WRN-16-2 student network, which can either be trained
directly from the CIFAR-100 dataset or through distillation. In our
experiments, the student network functions as a fixed feature extractor,
processing images from STL-10 and TIN-200, both resized to $32\times 32$. To
assess the generalizability of these representations, we train a linear
classifier on top of the last feature layer to perform 10-way classification
for STL-10 and 200-way classification for TIN-200. The effectiveness of
different distillation methods in enhancing the transferability of these
representations is detailed in Table 3. Our results indicate that most
distillation methods, except for FitNet, significantly enhance the
transferability of the learned representations on both target datasets.
Interestingly, the teacher network, while performing best on CIFAR-100, offers
representations that transfer the least effectively, possibly due to a bias
towards the training dataset. In contrast, the student network employing a
combination of ICD and KD distillation not only matches the teacher’s
performance on CIFAR-100 but also exhibits superior transferability, with
notable improvements of 3.9% and 4.7% in STL-10 and TIN-200, respectively.

| Teacher | Student | KD | AT | FitNet | CRD | CRD+KD | ICD | ICD+KD
---|---|---|---|---|---|---|---|---|---
CIFAR-100$\rightarrow$STL-10 | 68.6 | 69.7 | 70.9 | 70.7 | 70.3 | 71.6 | 72.2 | 71.2 | 72.5
CIFAR-100$\rightarrow$TIN-200 | 31.5 | 33.7 | 33.9 | 34.2 | 33.5 | 35.6 | 35.5 | 35.01 | 36.2
Table 3: Test top-1 accuracy (%) of WRN-16-2 (student) distilled from WRN-40-2
(teacher). In this setup, the representations learned from the CIFAR-100
dataset are transferred to the STL-10 and TIN-200 datasets. The network is
frozen, and a linear classifier is trained on the last feature layer to
perform classification with 10 classes (STL-10) or 200 classes (TIN-200).
Results adapted from [29]. The values in bold indicate the maximum of each
row.

##  5 Conclusions

We have developed a new technique for neural network distillation that
leverages invariant consistency to improve the traditional KD process. Our
method, demonstrated for model compression, can also be applied to cross-modal
knowledge transfer and ensemble distillation from a group of teachers to a
single student network. It combines contrastive learning with an invariance
penalty to better align teacher and student models. Unlike CRD [29], our
approach requires no memory bank. Instead, it uses a trainable temperature
parameter and a bias to scale and shift the logits. By leveraging the
contrastive nature, ICD not only aligns the student with the teacher’s output
but also improves the quality of the learned embeddings, leading to better
generalization and performance. Our experiments demonstrate that this approach
consistently outperforms standard KD methods, enhancing both accuracy and
robustness. Notably, our method has shown excellent performance on CIFAR-100
and has been successfully applied to other datasets such as TIN-200 and
STL-10. In some cases, student models using our KD technique even exceeded the
performance of their teacher models, suggesting significant potential for this
approach in optimizing neural network training and deployment in various
settings.

## References

  * [1] Sungsoo Ahn, Shell Xu Hu, Andreas Damianou, Neil D Lawrence, and Zhenwen Dai.  Variational information distillation for knowledge transfer.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9163–9171, 2019.
  * [2] Yusuf Aytar, Carl Vondrick, and Antonio Torralba.  Soundnet: Learning sound representations from unlabeled video.  In Advances in Neural Information Processing Systems, pages 892–900, 2016.
  * [3] Philip Bachman, R Devon Hjelm, and William Buchwalter.  Learning representations by maximizing mutual information across views, 2019.
  * [4] Gongfan Chen, Yuting Wang, Jiajun Xu, Zhe Du, Qionghai Dai, Shiyang Geng, and Tao Mei.  Learning efficient object detection models with knowledge distillation.  In Advances in Neural Information Processing Systems, pages 742–751, 2017.
  * [5] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton.  A simple framework for contrastive learning of visual representations, 2020.
  * [6] Jang Hyun Cho and Bharath Hariharan.  On the efficacy of knowledge distillation.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4794–4802, 2019.
  * [7] Adam Coates and Andrew Y. Ng.  The importance of encoding versus training with sparse coding and vector quantization.  In Proceedings of the 28th International Conference on International Conference on Machine Learning, ICML’11, page 921–928, Madison, WI, USA, 2011. Omnipress.
  * [8] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, K. Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009.
  * [9] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, and Michal Valko.  Bootstrap your own latent: A new approach to self-supervised learning, 2020.
  * [10] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick.  Momentum contrast for unsupervised visual representation learning, 2020.
  * [11] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition, 2015.
  * [12] Byeongho Heo, Minsik Lee, Seong Joon Yun, Jin Young Choi, and In So Kweon.  Knowledge transfer via distillation of activation boundaries formed by hidden neurons.  In Proceedings of the AAAI Conference on Artificial Intelligence, pages 3779–3787, 2019.
  * [13] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean.  Distilling the knowledge in a neural network, 2015.
  * [14] R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio.  Learning deep representations by mutual information estimation and maximization, 2019.
  * [15] Zehao Huang and Naiyan Wang.  Like what you like: Knowledge distill via neuron selectivity transfer.  In Advances in Neural Information Processing Systems, pages 185–195, 2017.
  * [16] Jangho Kim, Seongwon Park, and Nojun Kwak.  Paraphrasing complex network: Network compression via factor transfer.  In Advances in Neural Information Processing Systems, pages 2760–2769, 2018.
  * [17] Alex Krizhevsky.  Learning multiple layers of features from tiny images.  pages 32–33, 2009.
  * [18] Wei Liu, Andrew Rabinovich, and Alexander C Berg.  Learning efficient single-stage pedestrian detectors by asymptotic localization fitting.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 618–634, 2018.
  * [19] Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun.  Shufflenet v2: Practical guidelines for efficient cnn architecture design.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 116–131, 2018.
  * [20] Arun Mishra and Debbie Marr.  Apprentice: Using knowledge distillation techniques to improve low-precision network accuracy.  In International Conference on Learning Representations, 2017.
  * [21] Wonpyo Park, Dongju Kim, Yan Lu, and Minsu Cho.  Relational knowledge distillation.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3967–3976, 2019.
  * [22] Nikolaos Passalis and Anastasios Tefas.  Learning deep representations with probabilistic knowledge transfer.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 268–284, 2018.
  * [23] Baoyun Peng, Xi Li, Yifan Wu, Yizhou Fan, Bo Wang, Qi Tian, and Jun Liang.  Correlation congruence for knowledge distillation.  In Proceedings of the IEEE International Conference on Computer Vision, pages 5007–5016, 2019.
  * [24] Antonio Polino, Razvan Pascanu, and Dan Alistarh.  Model compression via distillation and quantization.  In International Conference on Learning Representations, 2018.
  * [25] Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio.  Fitnets: Hints for thin deep nets.  In Proceedings of the 4th International Conference on Learning Representations, 2014.
  * [26] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen.  Mobilenetv2: Inverted residuals and linear bottlenecks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4510–4520, 2018.
  * [27] Li Shen and Marios Savvides.  Amalgamating knowledge towards comprehensive classification.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1687–1696, 2020.
  * [28] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition, 2015.
  * [29] Yonglong Tian, Dilip Krishnan, and Phillip Isola.  Contrastive representation distillation, 2022.
  * [30] Frederick Tung and Greg Mori.  Similarity-preserving knowledge distillation.  In Proceedings of the IEEE International Conference on Computer Vision, pages 1365–1374, 2019.
  * [31] Aaron van den Oord, Yazhe Li, and Oriol Vinyals.  Representation learning with contrastive predictive coding, 2019.
  * [32] Lin Wang and Kuk-Jin Yoon.  Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(6):3048–3068, 2022.
  * [33] Zhirong Wu, Yuanjun Xiong, Stella Yu, and Dahua Lin.  Unsupervised feature learning via non-parametric instance-level discrimination, 2018.
  * [34] Junho Yim, Donggyu Joo, Jihoon Bae, and Junmo Kim.  A gift from knowledge distillation: Fast optimization, network minimization and transfer learning.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4133–4141, 2017.
  * [35] Sergey Zagoruyko and Nikos Komodakis.  Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer.  In Proceedings of the 5th International Conference on Learning Representations, 2016.
  * [36] Sergey Zagoruyko and Nikos Komodakis.  Wide residual networks, 2017.
  * [37] Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun.  Shufflenet: An extremely efficient convolutional neural network for mobile devices.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6848–6856, 2018.


resize to[a4paper]

# Enabling Technologies for 6G Future Wireless Communications: Opportunities
and Challenges

Samar Elmeadawy
Raed M. Shubair

##  1 Introduction

Almost every ten years, a new communication system has been introduced,
improving the QoS, providing new features and introducing new technologies.
Although 5G is not officially launched yet, researchers have turned their
attention to 6G communication system. The reason is that 5G provides a high
standard infrastructure enabling a variety of technologies such as; self-
driving cars, AI, mobile broadband communication, IoT and smart cities.
However, the usage of smart devices is increasingly growing each year and the
data traffic usage will be exponentially increasing as in Fig. 1, which puts
constraints on the 5G communication network.

Figure 1: ITU Global mobile data traffic Prediction [1]

These constraints open the door for a new communication system providing more
capacity, extremely low latency, high data transmission, secure error-free
communication and full wireless coverage. Table 1 compares the main
specifications and technologies in both 5G and 6G. 6G will be able to connect
everything, integrate different technologies and applications, support
holographic, haptic, space and underwater communications and it will also
support the Internet of everything, Internet of Nano-Things and Internet of
Bodies [2].

Table 1: Comparison between 5G and 6G Characteristic | 5G | 6G
---|---|---
Operating frequency | 3 - 300 GHz | upto 1 THz
Uplink data rate | 10 Gbps | 1 Tbps
Downlink data rate | 20 Gbps | 1 Tbps
Spectral efficiency | 10 bps/Hz/$m^{2}$ | 1000 bps/Hz/$m^{2}$
Reliability | $10^{-}5$ | $10^{-}9$
Maximum mobility | 500 km/h | 1000 km/hr
U-plane latency | 0.5 msec | 0.1 msec
C-plane latency | 10 msec | 1 msec
Processing delay | 100 ns | 10 ns
Traffic capacity | 10 Mbps/$m^{2}$ | 1 - 10 Gbps/$m^{2}$
Localization precision | 10 cm on 2D | 1 cm on 3D
Uniform user experience | 50 Mbps 2D | 10 Gbps 3D
Time buffer | not real-time | real-time
Center of gravity | user | service
Satellite integration | No | Fully
AI integration | Partially | Fully
XR integration | Partially | Fully
Haptic communication integration | Partially | Fully
Automation integration | Partially | Fully

Future 6G wireless communication systems will benefit from the knowledge
gained and solutions achieved from several technologies, methods, and
applications [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20,
21, 22, 23]. In this report, some emerging technologies and applications
introduced and developed by the 6G communication technology are presented in
section 2 and the main challenges facing the achievement of the 6G goals are
addressed in section 3.

##  2 Emerging technologies and Applications

Every communication system opens the door to new features and applications. 5G
was the first generation to introduce AI, automation and smart cities.
However, these technologies were partially integrated. 6G is introducing more
technologies and applications providing higher data rates, high reliability,
low latency and secure efficient transmission. Fig. 2 shows the main
applications, trends and technologies introduced in 6G. In this section, some
of these technologies and applications 6G are discussed.

Figure 2: 6G main Applications, trends, and technologies [24]

###  2.1 TeraHertz Communication

The RF band is almost full and it is not able to support the increasingly
growing demand in wireless communications technology. The THz band, ranging
from 0.1 THz to 10 THz, will play a crucial role in 6G supplying more
bandwidth, more capacity, ultra-high data rates and secure transmission. The
THz band will support the development of minuscule cells in nanometer to
micrometre dimensions supplying very high-speed communications within a
coverage area of up to 10 m [25] and supporting the Internet of Nano-things
[17, 26]. Technologies using frequency bands below 0.1 THz can not support
Tbps links, therefore, 6G will be the first wireless communication system
supporting Tbps for high-speed communication.

###  2.2 Cell-Free Communication

Unmanned Aerial Vehicles (UAV) were proposed to be used in other generations
in places where there is no infrastructure. However, this technology will be
fully used in 6G allowing cell-free communication. When the user equipment
(UE) moves from one cell coverage to another, the user‘s call should be
transferred to the other cell. This handover might be unsuccessful and in some
occurrences, the user‘s call is terminated and the QoS will be reduced in the
system. 6G will end the problem of cell coverage as the UE will be connected
to the whole network, not a specific cell. Using UAV will allow integrating
different technologies allowing the UE to utilize the technology having the
best coverage without any manual configurations on the device [27].

###  2.3 Artificial Intelligence

Artificial Intelligence (AI) was not involved in 4G or any previous
generations. It is partially supported by 5G making difference in the
telecommunications world opening the doors for emerging remarkable
applications such as [28, 29, 30, 31, 32]. However, AI will be fully supported
in 6G for automation. It will be involved in the handover, network selection
and resources allocation improving the performance, especially in delay-
sensitive applications. AI and machine learning are the most important
technologies in 6G [33].

###  2.4 Holographic Beamforming (HBF)

Beamforming is using a directed narrow beam with a high gain for transmitting
and receiving using antenna arrays by focusing the power in a minimized
angular range. It offers better coverage and throughput, higher signal to
interference and noise ratio (SINR) and it could be used to track users.
Holographic beamforming is an advanced beamforming approach utilizing
Software-Defined Antenna (SDA). Holographic refers to using a hologram to
achieve beam steering by the antenna, where the antenna is like a holographic
plate in an optical hologram; RF signals from a radio flow into the back of
the antenna and scatter across its front, where tiny elements adjust the shape
and direction of the beam as in Fig. 3. SDAs are cheaper, smaller in size,
lighter and require less power compared to the traditional phased arrays or
MIMO systems [34]. As C-SWaP (Cost, Size, Weight and Power) are considered as
the main challenges in any communication system designs, using SDAs in HBF
will allow flexible and efficient transmitting and receiving in 6G.

Figure 3: Holographic Beamforming [b1]

###  2.5 Extended Reality

Extended reality (XR) is a new umbrella term including Virtual Reality (VR),
Augmented Reality (AR) and Mixed Reality (MR). VR is a computer-simulated
reality experience using a headset that generates sounds and images creating
an imaginary world. AR uses the real world and adds to it using a specific
device such as the mobile phone. Audios, videos, Global Positioning System
(GPS) could be used to create an interactive environment. Pokémon is a well-
known example of AR. MR merges between the real and the virtual worlds
creating a complex environment. XR is all the real and virtual environment
combined. 6G will be very useful for this feature due to the strong
connectivity, high data rate, high resolution and low latency [35].

###  2.6 Blockchain technology

The data in the blockchain technology are represented as distributed blocks
connected to each other and cryptographically secured. Blockchain will be used
in managing and organizing big data and in managing huge connectivity in 6G.
It will be used also in spectrum sharing allowing the users to share the same
spectrum solving the problem of huge spectrum requirements in 6G and
guaranteeing secure, low cost, smart and efficient spectrum utilization.
Integrating the blockchain with AI and using Deep reinforcement learning [36]
will improve the QoS allowing smart-resources sharing, implementing an
advanced caching scheme and making the network more flexible.

###  2.7 Automation

Currently, researchers focus on automation, robotics and autonomous systems.
6G will support these technologies providing direct communication between them
and the server and direct communication between them, i.e.: robot to robot
communication and robot to the server communication. Full automation will be
provided by 6G including automatic control processes, automatic systems and
automatic devices. 6G will support the existence of Unmanned Aerial Vehicles
(UAV) [37] which will be used in wireless communications providing high data
rates instead of the traditional base stations (BS).

###  2.8 Wireless Power Transfer

Wireless energy transfer will be involved in 6G, providing suitable power to
the batteries in devices such as; smartphones and sensors [38]. The base
stations in 6G will be used for transferring power as Wireless Information and
Energy Transfer (WIET) uses the same fields and waves used in communication
systems. WIET is an innovative technology that will allow the development of
batteryless smart devices, charging wireless networks and saving the battery
life-time of other devices.

###  2.9 Wireless Brain-Computer Interface

Recently wearable devices are increasingly used, some of them are brain-
computer interface (BCI) applications. BCI applications involve smart wearable
headsets, smart embedded devices and smart body implants [24]. Using BCI
technologies, the brain will easily communicate with external discrete devices
which will be responsible for analyzing brain signals and translating them.
BCI also will involve affective computing technologies, in which devices will
function differently depending on the user’s mood. BCI applications were
limited because they require more spectrum resources, high bit rate, very low
latency and high reliability. However, 6G will support more applications such
as the five sense information transfer, in which 6G will transfer the data
generated by the five senses of the human allowing the interaction with the
environment.

###  2.10 Healthcare

The lack of electronic healthcare in other wireless communication technology
was because of low data rate and time delay. 6G will provide secure
communication, high performance, ultra-low latency, high data rate and high
reliability enabling the full existence of remote surgeries as in Fig. 4
through XR, robotics, automation and AI [27]. Also, the small wavelength due
to the THz band supports the communication and the development of nanosensors
allowing developing new nanosized devices to operate inside the human body
[39, 20, 23, 40, 41].

Figure 4: A loop explaining the communication between the master console and
the tele-operator [42]

##  3 Research Challenges

To fulfil the world demands in technology, some challenging requirements in 6G
wireless communication should be achieved. In this section, the main
challenging topics are investigated and discussed.

###  3.1 TeraHertz Band

The main challenge in the 6G wireless communication system is the THz band.
Although it provides high data rates, the high frequencies make overcoming the
high path loss a great issue. For long-distance communications, the
atmospheric absorption and propagation loss are very high. This is an
important issue that needs to be addressed. Because of the broad bandwidth,
new multipath channel models need to be developed to overcome the problem of
frequency dispersion [25]. The existing modulation and coding techniques are
not sufficient for the THz band. Therefore, implementing new modulation and
coding techniques is challenging research. New transceivers should be designed
to operate on the high-frequency band supporting the very large bandwidth,
high power, high sensitivity and low noise figure to overcome the atmospheric
losses. Also, health and safety issues due to high power and frequency are
great challenges facing the researchers.

###  3.2 Device Capabilities

Devices were not compatible with all the wireless communication technologies.
Recently, companies are working on devices supporting 5G, these devices should
be able to support 6G and all the different wireless communication generations
as discussed in section 2.2. Devices supporting 6G wireless communication
technology should tolerate the 1 Tbps data rate and the high operating
frequencies. Devices should also support interacting with different devices
using Device to Device communication, AI and XR [43]. In these days
smartphones are almost being charged daily utilizing more energy than before.
Billions of devices other than smartphones will be connected to the 6G
network, therefore, efficient energy transferring methods should be considered
especially wireless energy transfer methods [44], and devices connected should
be developed so that they can support different charging methods. These
capabilities in the devices are costly and challenging.

###  3.3 Network Security

6G wireless communications network will connect not only smartphones but also
smart devices used in automation, AI, XR, smart cities and satellites. The
security approaches used in 5G will not be sufficient in 6G, and hence new
security techniques with innovative cryptographic methods should be considered
including the physical layer security techniques and integrated network
security techniques [45] with low cost, low complexity and very high security.

###  3.4 Transceiver and Antenna Designs

For each wireless communication technology, there is a specific transceiver
and antenna designs supporting the specifications of that technology as
mentioned in section 3.1. it was a challenge in 5G to develop devices of
millimetre components. However, in 6G it will be more challenging. 6G wireless
communication technology supports high-frequency band in THz and supports
spectrum and resources sharing. The transceivers should be able to support
these technology having the antenna designed with the required size; nanometer
to micrometre components satisfying the holographic beamforming requirements
as in [46, 47, 48, 49]. Metasurface-based transceivers [50] could be the
solution to this issue to increase the throughput and the QoS. However,
integrating metasurface with OFDM-MIMO is a great challenge.

##  4 Conclusion

5G telecommunication technology that will be launched in 2020 will not fulfil
the increasingly growing demands in 2030. Therefore researches in 6G should be
conducted to be able to reach its goals by 2030. In this report, new features
in 6G and the possible applications and technologies that will be deployed in
6G are provided. Main challenges in the technologies in 6G are presented. It
is concluded 6G will improve the network performance, integrate different
technologies and increase the QoS providing super-smart society with
everything connected to the network.

## References

  * [1] F. Tariq, M. Khandaker, K.-K. Wong, M. Imran, M. Bennis, and M. Debbah, “A speculative study on 6g,” _arXiv preprint arXiv:1902.06700_ , 2019.
  * [2] Z. Zhang, Y. Xiao, Z. Ma, M. Xiao, Z. Ding, X. Lei, G. K. Karagiannidis, and P. Fan, “6g wireless networks: Vision, requirements, architecture, and key technologies,” _IEEE Vehicular Technology Magazine_ , vol. 14, no. 3, pp. 28–41, 2019.
  * [3] H. Elayan, R. M. Shubair, and A. Kiourti, “Wireless sensors for medical applications: Current status and future challenges,” in _2017 11th European Conference on Antennas and Propagation (EUCAP)_ , Mar. 2017, pp. 2478–2482.
  * [4] M. S. Khan, A.-D. Capobianco, A. Iftikhar, R. M. Shubair, D. E. Anagnostou, and B. D. Braaten, “Ultra-compact dual-polarised UWB MIMO antenna with meandered feeding lines,” _Antennas Propagation IET Microwaves_ , vol. 11, no. 7, pp. 997–1002, 2017.
  * [5] H. Elayan, R. M. Shubair, J. M. Jornet, and R. Mittra, “Multi-layer Intrabody Terahertz Wave Propagation Model for Nanobiosensing Applications,” _Nano Communication Networks_ , vol. 14, pp. 9–15, Dec. 2017.
  * [6] M. S. Khan, A. Iftikhar, A.-D. Capobianco, R. M. Shubair, and B. Ijaz, “Pattern and frequency reconfiguration of patch antenna using PIN diodes,” _Microwave and Optical Technology Letters_ , vol. 59, no. 9, pp. 2180–2185, 2017.
  * [7] A. A. Ibrahim, J. Machac, and R. M. Shubair, “Compact UWB MIMO antenna with pattern diversity and band rejection characteristics,” _Microwave and Optical Technology Letters_ , vol. 59, no. 6, pp. 1460–1464, 2017.
  * [8] H. Elayan, R. M. Shubair, and A. Kiourti, “Wireless sensors for medical applications: Current status and future challenges,” in _2017 11th European Conference on Antennas and Propagation (EUCAP)_. IEEE, 2017, pp. 2478–2482.
  * [9] G. Nwalozie, V. Okorogu, S. Maduadichie, and A. Adenola, “A simple comparative evaluation of adaptive beam forming algorithms,” _International Journal of Engineering and Innovative Technology (IJEIT)_ , vol. 2, no. 7, 2013.
  * [10] M. Al-Nuaimi, R. Shubair, and K. Al-Midfa, “Direction of arrival estimation in wireless mobile communications using minimum variance distortionless response,” in _The Second International Conference on Innovations in Information Technology (IIT’05)_ , 2005, pp. 1–5.
  * [11] R. Shubair, “Robust adaptive beamforming using lms algorithm with smi initialization,” in _2005 IEEE Antennas and Propagation Society International Symposium_ , vol. 4. IEEE, 2005, pp. 2–5.
  * [12] F. A. Belhoul, R. M. Shubair, and M. Ai-Mualla, “Modelling and performance analysis of doa estimation in adaptive signal processing arrays,” in _10th IEEE International Conference on Electronics, Circuits and Systems, 2003. ICECS 2003. Proceedings of the 2003_ , vol. 1. IEEE, 2003, pp. 340–343.
  * [13] E. M. Al-Ardi, R. M. Shubair, and M. E. Al-Mualla, “Computationally efficient doa estimation in a multipath environment using covariance differencing and iterative spatial smoothing,” in _2005 IEEE International Symposium on Circuits and Systems_. IEEE, 2005, pp. 3805–3808.
  * [14] R. Shubair and A. Al-Merri, “Robust algorithms for direction finding and adaptive beamforming: performance and optimization,” in _The 2004 47th Midwest Symposium on Circuits and Systems, 2004. MWSCAS’04._ , vol. 2. IEEE, 2004, pp. II–II.
  * [15] R. Shubair and Y. Chow, “A closed-form solution of vertical dipole antennas above a dielectric half-space,” _IEEE transactions on antennas and propagation_ , vol. 41, no. 12, pp. 1737–1741, 1993.
  * [16] E. Al-Ardi, R. M. Shubair, and M. E. Al-Mualla, “Performance evaluation of direction finding algorithms for adapative antenna arrays,” in _10th IEEE International Conference on Electronics, Circuits and Systems, 2003. ICECS 2003. Proceedings of the 2003_ , vol. 2. IEEE, 2003, pp. 735–738.
  * [17] H. Elayan, O. Amin, R. M. Shubair, and M.-S. Alouini, “Terahertz communication: The opportunities of wireless technology beyond 5g,” in _2018 International Conference on Advanced Communication Technologies and Networking (CommNet)_. IEEE, 2018, pp. 1–5.
  * [18] E. Al-Ardi, R. Shubair, and M. Al-Mualla, “Investigation of high-resolution doa estimation algorithms for optimal performance of smart antenna systems,” 2003\.
  * [19] M. S. Khan, A.-D. Capobianco, S. M. Asif, A. Iftikhar, B. D. Braaten, and R. M. Shubair, “A pattern reconfigurable printed patch antenna,” in _2016 IEEE International Symposium on Antennas and Propagation (APSURSI)_ , Jun. 2016, pp. 2149–2150, iSSN: 1947-1491.
  * [20] H. Elayan, P. Johari, R. M. Shubair, and J. M. Jornet, “Photothermal modeling and analysis of intrabody terahertz nanoscale communication,” _IEEE transactions on nanobioscience_ , vol. 16, no. 8, pp. 755–763, 2017.
  * [21] M. El Shorbagy, R. M. Shubair, M. I. AlHajri, and N. K. Mallat, “On the design of millimetre-wave antennas for 5g,” in _2016 16th Mediterranean Microwave Symposium (MMS)_. IEEE, 2016, pp. 1–4.
  * [22] H. Elayan, R. M. Shubair, and J. M. Jornet, “Bio-electromagnetic thz propagation modeling for in-vivo wireless nanosensor networks,” in _2017 11th European Conference on Antennas and Propagation (EuCAP)_. IEEE, 2017, pp. 426–430.
  * [23] H. Elayan, C. Stefanini, R. M. Shubair, and J. M. Jornet, “End-to-end noise model for intra-body terahertz nanoscale communication,” _IEEE transactions on nanobioscience_ , vol. 17, no. 4, pp. 464–473, 2018.
  * [24] W. Saad, M. Bennis, and M. Chen, “A vision of 6g wireless systems: Applications, trends, technologies, and open research problems,” _IEEE network_ , 2019.
  * [25] I. F. Akyildiz, J. M. Jornet, and C. Han, “Terahertz band: Next frontier for wireless communications,” _Physical Communication_ , vol. 12, pp. 16–32, 2014.
  * [26] H. Elayan, R. M. Shubair, and A. Kiourti, “On graphene-based thz plasmonic nano-antennas,” in _2016 16th Mediterranean Microwave Symposium (MMS)_. IEEE, 2016, pp. 1–3.
  * [27] M. Giordani, M. Polese, M. Mezzavilla, S. Rangan, and M. Zorzi, “Towards 6g networks: Use cases and technologies,” _arXiv preprint arXiv:1903.12216_ , 2019.
  * [28] M. I. AlHajri, N. T. Ali, and R. M. Shubair, “Classification of indoor environments for iot applications: A machine learning approach,” _IEEE Antennas and Wireless Propagation Letters_ , vol. 17, no. 12, pp. 2164–2168, 2018\.
  * [29] M. AlHajri, N. Alsindi, N. Ali, and R. Shubair, “Classification of indoor environments based on spatial correlation of rf channel fingerprints,” in _2016 IEEE international symposium on antennas and propagation (APSURSI)_. IEEE, 2016, pp. 1447–1448.
  * [30] M. I. AlHajri, N. T. Ali, and R. M. Shubair, “Indoor localization for iot using adaptive feature selection: A cascaded machine learning approach,” _IEEE Antennas and Wireless Propagation Letters_ , vol. 18, no. 11, pp. 2306–2310, 2019.
  * [31] ——, “A machine learning approach for the classification of indoor environments using rf signatures,” in _2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP)_. IEEE, 2018, pp. 1060–1062.
  * [32] ——, “2.4 ghz indoor channel measurements data set.” UCI Machine Learning Repository, 2018. [Online]. Available: https://goo.gl/cTSF5j
  * [33] E. C. Strinati, S. Barbarossa, J. L. Gonzalez-Jimenez, D. Ktenas, N. Cassiau, L. Maret, and C. Dehos, “6g: The next frontier: From holographic messaging to artificial intelligence using subterahertz and visible light communication,” _IEEE Vehicular Technology Magazine_ , vol. 14, no. 3, pp. 42–50, 2019.
  * [34] E. J. Black and P. C. CTO, “Holographic beam forming and mimo,” _Commware, Pivotal_ , 2016.
  * [35] M. Piran, D. Y. Suh _et al._ , “Learning-driven wireless communications, towards 6g,” _arXiv preprint arXiv:1908.07335_ , 2019.
  * [36] Y. Dai, D. Xu, S. Maharjan, Z. Chen, Q. He, and Y. Zhang, “Blockchain and deep reinforcement learning empowered intelligent 5g beyond,” _IEEE Network_ , vol. 33, no. 3, pp. 10–17, 2019.
  * [37] B. Li, Z. Fei, and Y. Zhang, “Uav communications for 5g and beyond: Recent advances and future trends,” _IEEE Internet of Things Journal_ , vol. 6, no. 2, pp. 2241–2263, 2018.
  * [38] T. Jung, T. Kwon, and C.-B. Chae, “Qoe-based transmission strategies for multi-user wireless information and power transfer,” _ICT Express_ , vol. 1, no. 3, pp. 116–120, 2015.
  * [39] H. Elayan, R. M. Shubair, J. M. Jornet, and P. Johari, “Terahertz channel model and link budget analysis for intrabody nanoscale communication,” _IEEE transactions on nanobioscience_ , vol. 16, no. 6, pp. 491–503, 2017\.
  * [40] R. M. Shubair and H. Elayan, “In vivo wireless body communications: State-of-the-art and future directions,” in _2015 Loughborough Antennas & Propagation Conference (LAPC)_. IEEE, 2015, pp. 1–5.
  * [41] H. Elayan, R. M. Shubair, J. M. Jornet, and R. Mittra, “Multi-layer intrabody terahertz wave propagation model for nanobiosensing applications,” _Nano communication networks_ , vol. 14, pp. 9–15, 2017.
  * [42] Q. Zhang, J. Liu, and G. Zhao, “Towards 5g enabled tactile robotic telesurgery,” _arXiv preprint arXiv:1803.03586_ , 2018.
  * [43] M. Z. Chowdhury, M. Shahjalal, S. Ahmed, and Y. M. Jang, “6g wireless communication systems: Applications, requirements, technologies, challenges, and research directions,” _arXiv preprint arXiv:1909.11315_ , 2019.
  * [44] L. L. Pon, C. Y. Leow, S. K. A. Rahim, A. A. Eteng, and M. R. Kamarudin, “Printed spiral resonator for displacement-tolerant near-field wireless energy transfer,” _IEEE Access_ , vol. 7, pp. 172 055–172 064, 2019.
  * [45] P. Yang, Y. Xiao, M. Xiao, and S. Li, “6g wireless communications: Vision and potential techniques,” _IEEE Network_ , vol. 33, no. 4, pp. 70–75, 2019\.
  * [46] A. Goian, M. I. AlHajri, R. M. Shubair, L. Weruaga, A. R. Kulaib, R. AlMemari, and M. Darweesh, “Fast detection of coherent signals using pre-conditioned root-music based on toeplitz matrix reconstruction,” in _2015 IEEE 11th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob)_. IEEE, 2015, pp. 168–174.
  * [47] M. AlHajri, A. Goian, M. Darweesh, R. AlMemari, R. Shubair, L. Weruaga, and A. Kulaib, “Hybrid rss-doa technique for enhanced wsn localization in a correlated environment,” in _2015 International Conference on Information and Communication Technology Research (ICTRC)_. IEEE, 2015, pp. 238–241.
  * [48] R. M. Shubair, A. S. Goian, M. I. AlHajri, and A. R. Kulaib, “A new technique for uca-based doa estimation of coherent signals,” in _2016 16th Mediterranean Microwave Symposium (MMS)_. IEEE, 2016, pp. 1–3.
  * [49] M. AlHajri, A. Goian, M. Darweesh, R. AlMemari, R. Shubair, L. Weruaga, and A. AlTunaiji, “Accurate and robust localization techniques for wireless sensor networks,” _arXiv preprint arXiv:1806.05765_ , 2018.
  * [50] W. Tang, M. Z. Chen, J. Y. Dai, Y. Zeng, X. Zhao, S. Jin, Q. Cheng, and T. J. Cui, “Wireless communications with programmable metasurface: New paradigms, opportunities, and challenges on transceiver design,” _arXiv preprint arXiv:1907.01956_ , 2019.