††thanks: The first author is the corresponding author, supported by Swami
Vivekananda Merit Cum Means Scholarship, Government of West Bengal, India.

#  $\ast$-$\eta$-Ricci-Yamabe solitons on $\alpha$-Cosymplectic manifolds with
a quarter-symmetric metric connection

Soumendu Roy  Department of Mathematics,Jadavpur University, Kolkata-700032,
India<EMAIL_ADDRESS>Santu Dey  Department of Mathematics, Bidhan
Chandra College, Asansol - 4, West Bengal-713304 , India
<EMAIL_ADDRESS>Arindam Bhattacharyya  Department of
Mathematics,Jadavpur University, Kolkata-700032, India
<EMAIL_ADDRESS>Mohd. Danish Siddiqi  Department of Mathematics,
Faculty of Science, Jazan University, 82715, Jazan, Kingdom of Saudi Arabia.
<EMAIL_ADDRESS>
###### Abstract.

The goal of the present paper is to deliberate certain types of metric such as
$*$-$\eta$-Ricci-Yamabe soliton on $\alpha$-Cosymplectic manifolds with
respect to quarter-symmetric metric connection. Further, we have proved some
curvature properties of $\alpha$-Cosymplectic manifolds admitting quarter-
symmetric metric connection. Here, we have shown the characteristics of the
soliton when the manifold satisfies quarter-symmetric metric connection on
$\alpha$-Cosymplectic manifolds. Later, we have acquired Laplace equation from
$*$-$\eta$-Ricci-Yamabe soliton equation when the potential vector field $\xi$
of the soliton is of gradient type in terms of quarter-symmetric metric
connection. Next, we have developed the nature of the soliton when the vector
field is conformal killing admitting quarter-symmetric metric connection.
Finally, we present an example of a 5-dimensional $\alpha$-cosymplectic metric
as a $*$-$\eta$-Ricci-Yamabe soliton with respect to a quarter-symmetric
metric connection to prove our results.

###### Key words and phrases:

Ricci-Yamabe soliton, $*$-$\eta$-Ricci-Yamabe soliton, conformal Killing
vector field, $\alpha$-Cosymplectic manifolds

###### 1991 Mathematics Subject Classification:

53C15, 53C25, 53C44

##  1\. Introduction

The concept of Ricci flow, which is an evolution equation for metrics on a
Riemannian manifold, was introduced by R. S. Hamilton [12] in the year 1982. A
self-similar solution to the Ricci flow [12], [32] is called a Ricci soliton
[13] if it moves only by a one parameter family of diffeomorphism and scaling.
After the introction of Ricci flow, to construct Yamabe metrics on compact
Riemannian manifolds, Hamilton [13] developed the idea of Yamabe flow.
In 2-dimension the Yamabe flow is equivalent to the Ricci flow [12]. But in
dimension $>2$ the Yamabe and Ricci flows do not agree, since the Yamabe flow
preserves the conformal class of the metric but the Ricci flow does not in
general.
Since the introduction of these geometric flows, the respective solitons and
their generalizations have been a great centre of attention of many geometers
viz. [3, 5, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 29, 30] who have
provided new approaches to understand the geometry of different kinds of
Riemannian manifold.
Recently in 2019, S. Güler and M. Crasmareanu [10] introduced a new geometric
flow which is a scalar combination of Ricci and Yamabe flow under the name
Ricci-Yamabe map. This flow is also known as Ricci-Yamabe flow of the type
$(\rho,q)$.
Let $M$ be a Riemannian manifold of dimension $n$ and $T^{s}_{2}(M)$ be the
linear space of its symmetric tensor fields of (0, 2)-type and
$Riem(M)\subsetneqq T^{s}_{2}(M)$ be the infinite space of its Riemannian
metrics. In [10], the authors have stated the following definition:

######  Definition 1.1.

[10] A Riemannian flow on $M$ is a smooth map:

$g:I\subseteq\mathbb{R}\rightarrow Riem(M),$

where $I$ is a given open interval. We can call it also as time-dependent (or
non-stationary) Riemannian metric.

######  Definition 1.2.

[10] The map $RY^{(\rho,q,g)}:I\rightarrow T^{s}_{2}(M)$ given by:

$RY^{(\rho,q,g)}:=\frac{\partial}{\partial t}g(t)+2\rho S(t)+qr(t)g(t),$

is called the $(\rho,q)$-Ricci-Yamabe map of the Riemannian flow of the
Riemannian flow $(M^{n},g)$, where $\rho,q$ are some scalars. If
$RY^{(\rho,q,g)}\equiv 0$, then $g(\cdot)$ will be called an $(\rho,q)$-Ricci-
Yamabe flow.
Also in [10], the authors characterized that the $(\rho,q)$-Ricci-Yamabe flow
is said to be,
$\bullet$ Ricci flow [12] if $\rho=1$, $q=0$.
$\bullet$ Yamabe flow [13] if $\rho=0$, $q=1$.
$\bullet$ Einstein flow [4] if $\rho=1$, $q=-1$.

A soliton to the Ricci-Yamabe flow is called Ricci-Yamabe solitons[28] if it
moves only by one parameter group of diffeomorphism and scaling. The metric of
the Riemannain manifold $(M^{n},g)$, $n>2$ is said to admit $(\rho,q)$–Ricci-
Yamabe soliton or simply Ricci-Yamabe soliton (RYS) $(g,V,\Lambda,\rho,q)$ if
it satisfies the equation,

$\pounds_{V}g+2\rho S=[2\Lambda-qr]g,$ (1.1)

where $\pounds_{V}g$ denotes the Lie derivative of the metric $g$ along the
vector field $V$, $S$ is the Ricci tensor, $r$ is the scalar curvature and
$\Lambda,\rho,q$ are real scalars.

Moreover the Ricci-Yamabe soliton is said to be expanding, steady or shrinking
according as $\Lambda$ is negative, zero, positive respectively. Also if
$\Lambda,\rho,q$ become smooth function then (1.1) is called almost Ricci-
Yamabe soliton.
In 2020, Mohd. Danish Siddiqi et al. [28] introduced a new generalization of
Ricci-Yamabe soliton, namely $\eta$-Ricci-Yamabe soliton, which is given by,

$\pounds_{V}g+2\rho S+[2\Lambda-qr]g+2\mu\eta\otimes\eta=0,$ (1.2)

where $\mu$ is a constant and $\eta$ is a 1-form on $M$.
Recently $\ast$-$\eta$ Ricci soliton, a generalization of $\eta$-Ricci soliton
[5], has been defined by S. Dey and S. Roy [24], which can be given as,

$\pounds_{\xi}g+2S^{\ast}+2\Lambda g+2\mu\eta\otimes\eta=0,$ (1.3)

where $S^{\ast}$ is the $*$-Ricci tensor, which was first introduced by
Tachibana [31] on almost Hermitian manifolds and then studied by Hamada [14]
on real hypersurfaces of non-flat complex space forms.
Motivating from the above generalizations, we now introduce the notion of
$\ast$-$\eta$-Ricci-Yamabe soliton as:

######  Definition 1.3.

A Riemannian manifold $(M,g)$ of dimension $n$ is said to admit
$\ast$-$\eta$-Ricci-Yamabe soliton if

$\pounds_{\xi}g+2\rho S^{\ast}+[2\Lambda-qr^{\ast}]g+2\mu\eta\otimes\eta=0,$
(1.4)

where $r^{\ast}=\operatorname{Tr}(S^{\ast})$, is the $\ast$-scalar curvature.

Moreover, in the above equation, if the vector field $\xi$ is the gradient of
a smooth function $f$ (denoted by $Df$, $D$ denotes the gradient operator)
then the equation (1.4) is called gradient $*$-$\eta$-Ricci-Yamabe soliton and
it is defined as:

$Hessf+\rho
S^{\ast}+\big{[}\Lambda-\frac{q{r^{\ast}}}{2}\big{]}g+\mu\eta\otimes\eta=0,$
(1.5)

where $Hessf$ is the Hessian of the smooth function $f$.

On the other hand, A linear connection $\tilde{\nabla}$ on an $n$-dimensional
Riemannian manifold $(M,g)$ is called a quarter-symmetric connection [8, 6] if
its torsion tensor of the connection $\tilde{\nabla}$,

$T(X,Y)=\tilde{\nabla}_{X}Y-\tilde{\nabla}_{Y}X-[X,Y]$ (1.6)

satisfies,

$T(X,Y)=\eta(Y)\phi X-\eta(X)\phi Y,$ (1.7)

where $\eta$ is a differentiable 1-form and $\phi$ is a (1, 1) tensor field.
If moreover the connection $\tilde{\nabla}$ satisfies,

$(\tilde{\nabla}_{X}g)(Y,Z)=0,$

for ali vector fields $X,Y,Z$ on $(M,g)$, then it is called a quarter-
symmetric metric connection [8, 6].
If $\phi X$ is replaced by $X$, then the connection is called a semi-symmetric
metric connection [33].
Based on the above facts and discussions in the research of contact geometry,
a natuaral question arises
Are there contact metric almost manifolds, whose metrics are $*$-$\eta$-Ricci-
Yamabe soliton?
In some next sections, we show that indeed the answer to this question is in
affirmative. This paper is organized as follows: After introduction, in
section 2, we have discussed some basic formulas of contact geometry and
curvature properties of $\alpha$-cosymplectic manifolds. In Section 3, we have
proved some characteristics of $n$-dimensional $\alpha$-cosymplectic manifold
admitting quarter-symmetric metric connection. Section 4 deals with
$*$-$\eta$-Ricci-Yamabe soliton in terms of quarter-symmetric metric
connection on $\alpha$-cosymplectic manifolds. In this section, we have
established the relationship between soliton constans $\Lambda$ and $\mu$.
Section 4 is also devoted to form the Laplace equation from $\eta$-Ricci-
Yamabe soliton equation with respect to quarter-symmetric metric connection
when the potential vector field $\xi$ of the soliton is of gradient type. We
have also constructed some applications on Laplace equation and proved some
theorems on harmonic aspect of $\ast$-$\eta$-Ricci-Yamabe soliton on
$n$-dimensional $\alpha$-cosymplectic manifold with a quarter-symmetric metric
connection in section 5 and 6 respectively. We have next considered the
potential vector field $V$ of the solition as conformal Killing vector field
to characterize the vector field to accessorize the nature of this soliton on
this manifold with quarter-symmetric metric connection. In section 7, we have
constructed an example to illustrate the existence of $*$-$\eta$-Ricci-Yamabe
soliton on $5$-dimensional $\alpha$-cosymplectic manifold with restect to
quarter-symmetric metric connection.

##  2\. Preliminaries

Let $M$ be a (2$n$+1)dimensional connected almost contact metric manifold with
an almost contact metric structure $(\phi,\xi,\eta,g)$ where $\phi$ is a
$(1,1)$ tensor field, $\xi$ is a vector field, $\eta$ is a 1-form and $g$ is
the compatible Riemannian metric such that [2],

$\phi^{2}(X)=-X+\eta(X)\xi,\eta(\xi)=1,\eta\circ\phi=0,\phi\xi=0,$ (2.1)
$g(\phi X,\phi Y)=g(X,Y)-\eta(X)\eta(Y),$ (2.2)
$g(X,\phi Y)=-g(\phi X,Y),$ (2.3)
$g(X,\xi)=\eta(X),$ (2.4)

for all vector fields $X,Y\in\chi(M),$ where $\chi(M)$ denotes the collection
of all smooth vector fields of $M$.
On this manifold $M$, the 2-form $\Phi$ is defined as,

$\Phi(X,Y)=g(\phi X,Y),$ (2.5)

for all vector fields $X,Y\in\chi(M).$
An almost contact metric manifold $(M,\phi,\xi,\eta,g)$ is said to be almost
cosymplectic [9, 15] if $d\eta=0$ and $d\Phi=0$, where $d$ is the exterior
differential operator. An almost contact manifold $(M,\phi,\xi,\eta,g)$ is
said to be normal if the Nijenhuis torsion,

$N_{\phi}(X,Y)=[\phi X,\phi Y-\phi[\phi X,Y]-\phi[X,\phi
Y]+\phi^{2}[X,Y]+2d\eta(X,Y)\xi,$

vanishes for any vector fields $X$ and $Y$ . A normal almost cosymplectic
manifold is called a cosymplectic manifold [9, 15].
An almost contact metric manifold $M$ is said to be almost $\alpha$-Kenmotsu
if $d\eta=0$ and $d\Phi=2\alpha\eta\wedge\Phi$, $\alpha$ being a non-zero real
constant.
In 2005, a combination of almost $\alpha$-Kenmotsu and almost cosymplectic
manifolds was developed by Kim and Pak [16], into a new class called almost
$\alpha$-cosymplectic manifold, where $\alpha$ is a scalar. An almost
$\alpha$-cosymplectic manifold can be defined as [11, 15],

$d\eta=0,\quad d\Phi=2\alpha\eta\wedge\Phi,$

for any real number $\alpha$.
A normal almost $\alpha$-cosymplectic manifold is called an
$\alpha$-cosymplectic manifold. An $\alpha$-cosymplectic manifold is either
cosymplectic under the condition $\alpha=0$ or $\alpha$-Kenmotsu under the
condition $\alpha\neq 0$, for $\alpha\in\mathbb{R}$ [15].
In an $\alpha$-cosymplectic manifold, we have [11, 15],

$(\nabla_{X}\phi)Y=\alpha(g(\phi X,Y)\xi,\eta(Y)\phi X).$ (2.6)

Let $M$ be an $n$-dimensional $\alpha$-cosymplectic manifold. Then from (2.6),
we have,

$\nabla_{X}\xi=-\alpha\phi^{2}X=\alpha[X-\eta(X)\xi],$ (2.7)

where $\nabla$ is the Levi-Civita connection associated with $g$.
On an $n$-dimensional $\alpha$-cosymplectic manifold $M$, the following
relations hold [15],

$R(X,Y)\xi=\alpha^{2}[\eta(X)Y-\eta(Y)X],$ (2.8)
$R(\xi,X)Y=\alpha^{2}[\eta(Y)X-g(X,Y)\xi],$ (2.9)
$R(\xi,X)\xi=\alpha^{2}[X-\eta(X)\xi],$ (2.10)

e

$\eta(R(X,Y)Z)=\alpha^{2}[\eta(Y)g(X,Z)-\eta(X)g(Y,Z)],$ (2.11)
$S(X,\xi)=-\alpha^{2}(n-1)\eta(X),$ (2.12)

for all vector fields $X,Y,Z$ on $M$, where $R$ is the Riemannian curvature
tensor and $S$ is the Ricci tensor of $M$.
In paper [15], Lemma 2.2, the authors have proved that the $\ast$-Ricci tensor
on an $n$-dimensional $\alpha$-cosymplectic manifold is given by,

$S^{\ast}(Y,Z)=S(Y,Z)+\alpha^{2}(n-2)g(Y,Z)+\alpha^{2}\eta(Y)\eta(Z),$ (2.13)

for any vector field $Y,Z$ on $M$, where $S$ and $S^{\ast}$ are the Ricci
tensor and the $\ast$-Ricci tensor of type $(0,2)$, respectively on $M$.
Taking $Y=Z=e_{i}$, where $e_{i}^{\prime}s$ are the orthonormal basis of
$T_{p}(M)$ for $i=1,2,...n$, we get,

$r^{\ast}=r+\alpha^{2}(n-1)^{2},$ (2.14)

where $r^{\ast}=\operatorname{Tr}(S^{\ast})$, is the $\ast$-scalar curvature
and $r$ is the scalar curvature.

##  3\. On an $n$-dimensional $\alpha$-cosymplectic manifold with a quarter-
symmetric metric connection

Let $\tilde{\nabla}$ be a linear connection and $\nabla$ be a Levi-Civita
connection of an almost contact metric manifold $M$ such that,

$\tilde{\nabla}_{X}Y=\nabla_{X}Y+H(X,Y),$ (3.1)

where $H$ is a tensor of type $(1,1)$. For $\tilde{\nabla}$ to be a quarter-
symmetric metric connection on $M$, we have [8],

$H(X,Y)=\frac{1}{2}\big{[}T(X,Y)+T^{\prime}(X,Y)+T^{\prime}(Y,X)\big{]},$
(3.2)

where

$g(T^{\prime}(X,Y),Z)=g(T(Z,X),Y).$ (3.3)

Now from (1.7) and (3.3), we get,

$T^{\prime}(X,Y)=g(\phi Y,X)\xi-\eta(X)\phi Y.$ (3.4)

Using (1.6), (3.4) and (3.2), we obtain,

$H(X,Y)=-\eta(X)\phi Y.$ (3.5)

Hence a quarter-symmetric metric connection $\tilde{\nabla}$ in a
$\alpha$-cosymplectic manifold is given by,

$\tilde{\nabla}_{X}Y=\nabla_{X}Y-\eta(X)\phi Y.$ (3.6)

Let $R$ and $\tilde{R}$ be the curvature tensors of $\nabla$ and
$\tilde{\nabla}$ of a $\alpha$-cosymplectic manifold, respectively.
Then using (3.6) and (2.7), we have,

$\tilde{R}(X,Y)Z=R(X,Y)Z+\eta(X)(\nabla_{Y}\phi)Z-\eta(Y)(\nabla_{X}\phi)Z,$
(3.7)

which in view of (2.6), gives,

$\displaystyle\tilde{R}(X,Y)Z$ $\displaystyle=$ $\displaystyle
R(X,Y)Z+\alpha\eta(X)g(\phi Y,Z)\xi-\alpha\eta(Y)g(\phi X,Z)\xi$ (3.8)
$\displaystyle-$ $\displaystyle\alpha\eta(X)\eta(Z)\phi
Y+\alpha\eta(Y)\eta(Z)\phi X.$

A relation between the curvature tensor of M with respect to the quarter-
symmetric metric connection $\tilde{\nabla}$ and Levi-Civita connection
$\nabla$ is given by the equation (3.8).
Taking inner product of (3.8) with $W$, we get,

$\displaystyle\tilde{R}(X,Y,Z,W)$ $\displaystyle=$ $\displaystyle
R(X,Y,Z,W)+\alpha\eta(X)\eta(W)g(\phi Y,Z)$ (3.9) $\displaystyle-$
$\displaystyle\alpha\eta(Y)\eta(W)g(\phi X,Z)-\alpha\eta(X)\eta(Z)g(\phi Y,W)$
$\displaystyle+$ $\displaystyle\alpha\eta(Y)\eta(Z)g(\phi X,W).$

Contracting (3.9) over $X$ and $W$, we obtain,

$\tilde{S}(Y,Z)=S(Y,Z)+\alpha g(\phi Y,Z),$ (3.10)

where $\tilde{S}$ and $S$ are the Ricci tensors of the connections
$\tilde{\nabla}$ and $\nabla$, respectively.
So in a $\alpha$-cosymplectic manifold, the Ricci tensor of the quarter-
symmetric metric connection is not symmetric.
Again, contracting (3.10) over $Y,Z$, it gives,

$\tilde{r}=r.$ (3.11)

where $\tilde{r}$ and $r$ are the scalar curvatures of the connections
$\tilde{\nabla}$ and $\nabla$, respectively.
So we have the following theorem:

######  Theorem 3.1.

For a $\alpha$-cosymplectic manifold $M$ with the quarter-symmetric metric
connection $\tilde{\nabla}$
(i) The Riemannian curvature tensor $\tilde{R}$ is given by (3.8),
(ii) The Ricci tensor $\tilde{S}$ is given by (3.10),
(iii) $\tilde{R}(X,Y,Z,W)+\tilde{R}(X,Y,W,Z)=0$,
(iv) $\tilde{R}(X,Y,Z,W)+\tilde{R}(Y,X,Z,W)=0$,
(v) $\tilde{S}(Y,\xi)=S(Y,\xi)=-\alpha^{2}(n-1)\eta(Y)$,
(vi) The Ricci tensor $\tilde{S}$ is not symmetric,
(vii) $\tilde{r}=r$.

##  4\. Main Results

Let us consider an $n$-dimensional $\alpha$-cosymplectic manifold $M$,
admitting a $\ast$-$\eta$-Ricci-Yamabe soliton.
Then from (1.4), we get,

$(\pounds_{\xi}g)(Y,Z)+2\rho S^{\ast}(Y,Z)+[2\Lambda-
qr^{\ast}]g(Y,Z)+2\mu\eta(Y)\eta(Z)=0,$ (4.1)

for any vector fields $Y,Z$ on $M$.
Using (2.13) and (2.14), the above equation takes the form,

$(\pounds_{\xi}g)(Y,Z)+2\rho
S(Y,Z)+\big{[}2\Lambda+2\alpha^{2}\rho(n-2)-qr-q\alpha^{2}(n-1)^{2}\big{]}g(Y,Z)\\\
+\big{[}2\mu+2\alpha^{2}\rho\big{]}\eta(Y)\eta(Z)=0.$ (4.2)

We consider $\tilde{\pounds}_{V}$ is the Lie derivative along $V$ with respect
to the quarter-symmetric metric connection $\tilde{\nabla}$.
Now if an $n$-dimensional $\alpha$-cosymplectic manifold $M$ admits a
$\ast$-$\eta$-Ricci-Yamabe soliton with respect to the quarter-symmetric
metric connection $\tilde{\nabla}$, then from (4.2), we have,

$(\tilde{\pounds}_{\xi}g)(Y,Z)+2\rho\tilde{S}(Y,Z)+\big{[}2\Lambda+2\alpha^{2}\rho(n-2)-q\tilde{r}-q\alpha^{2}(n-1)^{2}\big{]}g(Y,Z)\\\
+\big{[}2\mu+2\alpha^{2}\rho\big{]}\eta(Y)\eta(Z)=0.$ (4.3)

In view of (3.6), we obtain,

$\displaystyle(\tilde{\pounds}_{V}g)(X,Y)$ $\displaystyle=$ $\displaystyle
g(\tilde{\nabla}_{X}V,Y)+g(X,\tilde{\nabla}_{Y}V)$ $\displaystyle=$
$\displaystyle(\pounds_{V}g)(X,Y)-\eta(X)g(\phi V,Y)-\eta(Y)g(X,\phi V).$

Then using (2.1), we get,

$(\tilde{\pounds}_{\xi}g)(X,Y)=(\pounds_{\xi}g)(X,Y),$ (4.5)

for any vector fields $X,Y$ on $M$.
From (4.5), (3.10) and (3.11), (4.3) becomes,

$\Big{[}(\pounds_{\xi}g)(Y,Z)+2\rho
S(Y,Z)+\big{[}2\Lambda+2\alpha^{2}\rho(n-2)-qr-q\alpha^{2}(n-1)^{2}\big{]}g(Y,Z)\\\
+\big{[}2\mu+2\alpha^{2}\rho\big{]}\eta(Y)\eta(Z)\Big{]}+2\alpha\rho g(\phi
Y,Z)=0.\\\ $ (4.6)

Now in an $n$-dimensional $\alpha$-cosymplectic manifold, from (2.7), we
obtain,

$(\pounds_{\xi}g)(Y,Z)=g(\nabla_{Y}\xi,Z)+g(Y,\nabla_{Z}\xi)=2\alpha[g(Y,Z)-\eta(Y)\eta(Z)].$
(4.7)

Putting this value in (4.6) and then taking $Z=\xi$, we get,

$2\rho
S(Y,\xi)+\big{[}2\Lambda+2\alpha^{2}\rho(n-2)-qr-q\alpha^{2}(n-1)^{2}\big{]}\eta(Y)+\big{[}2\mu+2\alpha^{2}\rho\big{]}\eta(Y)=0.$
(4.8)

Using (2.12), we find,

$\Lambda+\mu=\frac{qr}{2}+\frac{q\alpha^{2}(n-1)^{2}}{2}.$ (4.9)

Hence we can state,

######  Theorem 4.1.

If the metric of an $n$-dimensional $\alpha$-cosymplectic manifold admits a
$\ast$-$\eta$-Ricci-Yamabe soliton with respect to a quarter-symmetric metric
connection $\tilde{\nabla}$, then the soliton constants $\Lambda$ and $\mu$
are related by the equation (4.9).

Again using (4.2), (4.6) takes the form,

$\alpha\rho g(\phi Y,Z)=0.$ (4.10)

for any vector fields $Y,Z$ on $M$.
Then $\alpha=0$, since in a $\ast$-$\eta$-Ricci-Yamabe soliton $\rho$ can not
be zero.
So as $\alpha=0$, then $M$ reduces to an $n$-dimensional cosymplectic
manifold.
This gives,

######  Corollary 4.2.

If the metric of an $n$-dimensional $\alpha$-cosymplectic manifold admits a
$\ast$-$\eta$-Ricci-Yamabe soliton with respect to the Levi-Civita connection
$\nabla$. Then the metric admits the soliton with respect to a quarter-
symmetric metric connection $\tilde{\nabla}$ iff the manifold becomes an
$n$-dimensional cosymplectic manifold.

Taking $Y=Z=e_{i}$, where $e_{i}^{\prime}s$ are the orthonormal basis of
$T_{p}(M)$ for $i=1,2,...n$, we get,

$div\xi+\rho
r+n\Big{[}\Lambda+\alpha^{2}\rho(n-2)-\frac{qr}{2}-\frac{q\alpha^{2}(n-1)^{2}}{2}\Big{]}+\mu+\alpha^{2}\rho=0,$
(4.11)

where $div\xi$ is the divergence of the vector field $\xi$.
Replacing the value of $\mu$ from (4.9) in the above equation, $\Lambda$ takes
the form,

$\Lambda=\frac{qr}{2}-\frac{q\alpha^{2}(n-1)^{2}}{2}-\alpha^{2}\rho(n-1)-\frac{div\xi+\rho
r}{n-1}.$ (4.12)

And in view of (4.12), (4.9) becomes,

$\mu=\alpha^{2}\rho(n-1)+\frac{div\xi+\rho r}{n-1}.$ (4.13)

Hence we can state,

######  Corollary 4.3.

If the metric of an $n$-dimensional $\alpha$-cosymplectic manifold admits a
$\ast$-$\eta$-Ricci-Yamabe soliton with respect to a quarter-symmetric metric
connection $\tilde{\nabla}$. Then the soliton constants $\Lambda$ and $\mu$
takes the form of
$\frac{qr}{2}-\frac{q\alpha^{2}(n-1)^{2}}{2}-\alpha^{2}\rho(n-1)-\frac{div\xi+\rho
r}{n-1}$ and $\alpha^{2}\rho(n-1)+\frac{div\xi+\rho r}{n-1}$ respectively,
where $div\xi$ is the divergence of the vector field $\xi$.

If $\xi=grad(f)$, where $grad(f)$ is the gradient of a smooth function $f$,
then from (4.11), we have the following,

######  Theorem 4.4.

Let an $n$-dimensional $\alpha$-cosymplectic manifold admit a
$\ast$-$\eta$-Ricci-Yamabe soliton with respect to a quarter-symmetric metric
connection $\tilde{\nabla}$. If the vector field $\xi$, associated with the
soliton is of the form $grad(f)$, where $f$ is a smooth function. Then the
Laplacian equation satisfied by $f$ becomes:

$\Delta(f)=-n\Big{[}\Lambda+\alpha^{2}\rho(n-2)-\frac{qr}{2}-\frac{q\alpha^{2}(n-1)^{2}}{2}\Big{]}-\mu-\rho(\alpha^{2}+r).$
(4.14)

##  5\. Some Applications

As an application, we obtain the following results for $\ast$-$\eta$-Ricci
soliton, $\ast$-$\eta$-Yamabe soliton, and $\ast$-$\eta$-Einstein soliton
($\rho=1,q=0$, $\rho=0,q=1$, and $\rho=1,q=-1$ ( cf. [12, 13, 4])).

######  Theorem 5.1.

Let an $n$-dimensional $\alpha$-cosymplectic manifold admit a
$\ast$-$\eta$-Ricci soliton with respect to a quarter-symmetric metric
connection $\tilde{\nabla}$. If the vector field $\xi$, associated with the
soliton is of the form $grad(f)$, where $f$ is a smooth function. Then the
Laplacian equation satisfied by $f$ becomes:

$\Delta(f)=-n\Big{[}\Lambda+\alpha^{2}(n-2)\Big{]}-\mu-(\alpha^{2}+r).$ (5.1)

######  Theorem 5.2.

Let an $n$-dimensional $\alpha$-cosymplectic manifold admit a
$\ast$-$\eta$-Yamabe soliton with respect to a quarter-symmetric metric
connection $\tilde{\nabla}$. If the vector field $\xi$, associated with the
soliton is of the form $grad(f)$, where $f$ is a smooth function. Then the
Laplacian equation satisfied by $f$ becomes:

$\Delta(f)=-n\Big{[}\Lambda-\frac{r}{2}-\frac{\alpha^{2}(n-1)^{2}}{2}\Big{]}-\mu.$
(5.2)

######  Theorem 5.3.

Let an $n$-dimensional $\alpha$-cosymplectic manifold admit a
$\ast$-$\eta$-Einstein soliton with respect to a quarter-symmetric metric
connection $\tilde{\nabla}$. If the vector field $\xi$, associated with the
soliton is of the form $grad(f)$, where $f$ is a smooth function. Then the
Laplacian equation satisfied by $f$ becomes:

$\Delta(f)=-n\Big{[}\Lambda+\alpha^{2}(n-2)+\frac{r}{2}+\frac{\alpha^{2}(n-1)^{2}}{2}\Big{]}-\mu-(\alpha^{2}+r).$
(5.3)

##  6\. Harmonic aspect of $\ast$-$\eta$-Ricci-Yamabe soliton on
$n$-dimensional $\alpha$-cosymplectic manifold with a quarter-symmetric metric
connection

This section is based on the fact that a function
$f:{M}\longrightarrow\mathbb{R}$ is said to be harmonic if $\Delta f=0$, where
$\Delta$ is the Lalplacian operator on M [34]. Therefore, let us considering
the fact that the vector field $\xi$ is a gradient of a harmonic function $f$,
then from Theorem (4.4), we turn up the following conclusions:

######  Theorem 6.1.

Let an $n$-dimensional $\alpha$-cosymplectic manifold $M$ admit a
$\ast$-$\eta$-Ricci-Yamabe soliton with respect to a quarter-symmetric metric
connection $\tilde{\nabla}$ and the vector field $\xi$, associated with the
soliton is of the form $grad(f)$, if $f$ is a harmonic function on ${M}$. Then
the $\ast$-$\eta$-Ricci-Yamabe soliton is expanding, steady and shrinking
according as

  1. (1)

$\frac{q}{2}\left\\{r+\alpha^{2}(n-1)^{2}\right\\}>\left\\{\mu+\rho(\alpha^{2}+r)+\alpha^{2}\rho(n-2)\right\\}$,

  2. (2)

$\frac{q}{2}\left\\{r+\alpha^{2}(n-1)^{2}\right\\}=\left\\{\mu+\rho(\alpha^{2}+r)+\alpha^{2}\rho(n-2)\right\\}$
and

  3. (3)

$\frac{q}{2}\left\\{r+\alpha^{2}(n-1)^{2}\right\\}<\left\\{\mu+\rho(\alpha^{2}+r)+\alpha^{2}\rho(n-2)\right\\}$
respectively.

###### Proof.

Form equation (4.14) we can easily obtain the desired result. ∎

######  Theorem 6.2.

Let an $n$-dimensional $\alpha$-cosymplectic manifold $M$ admit a
$\ast$-$\eta$-Ricci soliton with respect to a quarter-symmetric metric
connection $\tilde{\nabla}$ and the vector field $\xi$, associated with the
soliton is of the form $grad(f)$, if $f$ is a harmonic function on ${M}$. Then
the $\ast$-$\eta$-Ricci soliton is shrinking.

######  Theorem 6.3.

Let an $n$-dimensional $\alpha$-cosymplectic manifold $M$ admit a
$\ast$-$\eta$-Yamabe soliton with respect to a quarter-symmetric metric
connection $\tilde{\nabla}$ and the vector field $\xi$, associated with the
soliton is of the form $grad(f)$, if $f$ is a harmonic function on ${M}$. Then
the $\ast$-$\eta$-Yamabe soliton is expanding, steady and shrinking according
as

  1. (1)

$\frac{1}{2}\left\\{r+\alpha^{2}(n-1)^{2}\right\\}>\mu$,

  2. (2)

$\frac{1}{2}\left\\{r+\alpha^{2}(n-1)^{2}\right\\}=\mu$ and

  3. (3)

$\frac{1}{2}\left\\{r+\alpha^{2}(n-1)^{2}\right\\}<\mu$ respectively.

###### Proof.

Form equation (4.14) we can easily obtain the desired result. ∎

######  Theorem 6.4.

Let an $n$-dimensional $\alpha$-cosymplectic manifold $M$ admit a
$\ast$-$\eta$-Einstein soliton with respect to a quarter-symmetric metric
connection $\tilde{\nabla}$ and the vector field $\xi$, associated with the
soliton is of the form $grad(f)$, if $f$ is a harmonic function on ${M}$. Then
the $\ast$-$\eta$-Einstein soliton is also shrinking.

###### Proof.

Form equation (4.14) we can easily obtain the desired result. ∎

######  Definition 6.5.

A vector field $V$ is said to be a conformal Killing vector field iff the
following relation holds:

$(\pounds_{V}g)(Y,Z)=2\theta g(Y,Z),$ (6.1)

where $\theta$ is some function of the co-ordinates(conformal scalar).
Moreover if $\theta$ is not constant the conformal Killing vector field $V$ is
said to be proper. Also when $\theta$ is constant, $V$ is called homothetic
vector field and when the constant $\theta$ becomes non zero, $V$ is said to
be proper homothetic vector field. If $\theta=0$ in the above equation, then
$V$ is called Killing vector field.

Let us consider an $n$-dimensional $\alpha$-cosymplectic manifold $M$,
admitting a $\ast$-$\eta$-Ricci-Yamabe soliton, where $\xi=V$.
Then using (1.4), (2.13) and (2.14), we have,

$(\pounds_{V}g)(Y,Z)+2\rho
S(Y,Z)+\big{[}2\Lambda+2\alpha^{2}\rho(n-2)-qr-q\alpha^{2}(n-1)^{2}\big{]}g(Y,Z)\\\
+\big{[}2\mu+2\alpha^{2}\rho\big{]}\eta(Y)\eta(Z)=0.$ (6.2)

Now if an $n$-dimensional $\alpha$-cosymplectic manifold $M$ admits a
$\ast$-$\eta$-Ricci-Yamabe soliton $(g,V,\Lambda,\mu,\rho,q)$ with respect to
the quarter-symmetric metric connection $\tilde{\nabla}$, then from (6.2), we
have,

$(\tilde{\pounds}_{V}g)(Y,Z)+2\rho\tilde{S}(Y,Z)+\big{[}2\Lambda+2\alpha^{2}\rho(n-2)-q\tilde{r}-q\alpha^{2}(n-1)^{2}\big{]}g(Y,Z)\\\
+\big{[}2\mu+2\alpha^{2}\rho\big{]}\eta(Y)\eta(Z)=0.$ (6.3)

In the view of (4), (3.10), (3.11), the above equation becomes,

$(\pounds_{V}g)(Y,Z)-\eta(Y)g(\phi V,Z)-\eta(Z)g(Y,\phi V)+2\rho[S(Y,Z)+\alpha
g(\phi Y,Z)]\\\
+\big{[}2\Lambda+2\alpha^{2}\rho(n-2)-qr-q\alpha^{2}(n-1)^{2}\big{]}g(Y,Z)+\big{[}2\mu+2\alpha^{2}\rho\big{]}\eta(Y)\eta(Z)=0.$
(6.4)

Using (6.1), the equation (6.4) takes the form,

$\big{[}2\theta+2\Lambda+2\alpha^{2}\rho(n-2)-qr-q\alpha^{2}(n-1)^{2}\big{]}g(Y,Z)+\big{[}2\mu+2\alpha^{2}\rho\big{]}\eta(Y)\eta(Z)\\\
+2\rho[S(Y,Z)+\alpha g(\phi Y,Z)]-\eta(Y)g(\phi V,Z)-\eta(Z)g(Y,\phi V)=0.$
(6.5)

Now, taking $Z=\xi$ and using (2.12), we get,

$g(Y,\phi V)=\big{[}2\theta+2\Lambda+2\mu-
qr-q\alpha^{2}(n-1)^{2}\big{]}\eta(Y),$ (6.6)

which can be written as,

$\phi V=\big{[}2\theta+2\Lambda+2\mu-qr-q\alpha^{2}(n-1)^{2}\big{]}\xi.$ (6.7)

######  Theorem 6.6.

Let an $n$-dimensional $\alpha$-cosymplectic manifold admit a
$\ast$-$\eta$-Ricci-Yamabe soliton $(g,V,\Lambda,\mu,\rho,q)$ with respect to
a quarter-symmetric metric connection $\tilde{\nabla}$. If $V$ is a conformal
Killing vector field, then (6.6) holds.

######  Remark 6.7.

For particular values of $\rho$ and $q$, we can also obtain the similar
results like Theorem (6.6) for $\ast$-$\eta$-Ricci soliton,
$\ast$-$\eta$-Yamabe soliton, and $\ast$-$\eta$-Einstein soliton,
respectively.

##  7\. Example of a 5-dimensional $\alpha$-cosymplectic metric as a
$\ast$-$\eta$-Ricci-Yamabe soliton with respect to a quarter-symmetric metric
connection

We consider the 5-dimensional manifold $M=\\{(x,y,z,u,v)\in\mathbb{R}^{5}\\}$,
where $(x,y,z,u,v)$ are standard coordinates in $\mathbb{R}^{5}$. Let
${e_{1},e_{2},e_{3},e_{4},e_{5}}$ be a linearly independent frame field on M
given by,

$e_{1}=e^{\alpha v}\frac{\partial}{\partial x},\quad e_{2}=e^{\alpha
v}\frac{\partial}{\partial y},\quad e_{3}=e^{\alpha v}\frac{\partial}{\partial
z},\quad e_{4}=e^{\alpha v}\frac{\partial}{\partial u},\quad
e_{5}=-\frac{\partial}{\partial v}.$

Let $g$ be the Riemannian metric defined by,

$g(e_{i},e_{j})=0,i\neq j,i,j=1,2,3,4,5,$
$g(e_{1},e_{1})=g(e_{2},e_{2})=g(e_{3},e_{3})=g(e_{4},e_{4})=g(e_{5},e_{5})=1.$

Let $\eta$ be the 1-form defined by $\eta(Z)=g(Z,e_{5})$, for any
$Z\in\chi(M)$,where $\chi(M)$ is the set of all differentiable vector fields
on $M$ and $\phi$ be the (1, 1)-tensor field defined by,

$\phi e_{1}=-e_{2},\quad\phi e_{2}=e_{1},\quad\phi e_{3}=-e_{4},\quad\phi
e_{4}=e_{3},\quad\phi e_{5}=0.$

Then, using the linearity of $\phi$ and $g$, we have
$\eta(e_{5})=1,\phi^{2}(Z)=-Z+\eta(Z)e_{5}$ and $g(\phi Z,\phi
U)=g(Z,U)-\eta(Z)\eta(U)$, for any $Z,U\in\chi(M)$. Thus $e_{5}=\xi$.
Let $\nabla$ be the Levi-Civita connection with respect to the Riemannian
metric $g$. Then we have,
$[e_{1},e_{5}]=\alpha e_{1},[e_{2},e_{5}]=\alpha e_{2},[e_{3},e_{5}]=\alpha
e_{3},[e_{4},e_{5}]=\alpha e_{4}$, and $[e_{i},e_{j}]=0$ for others $i,j$.
The connection $\nabla$ of the metric $g$ is given by,

$\displaystyle 2g(\nabla_{X}Y,Z)$ $\displaystyle=$ $\displaystyle
Xg(Y,Z)+Yg(Z,X)-Zg(X,Y)$ $\displaystyle-$ $\displaystyle
g(X,[Y,Z])-g(Y,[X,Z])+g(Z,[X,Y]),$

which is known as Koszul’s formula.
Using Koszul’s formula, we can easily calculate,

$\nabla_{e_{1}}e_{1}=-\alpha
e_{5},\nabla_{e_{1}}e_{2}=0,\nabla_{e_{1}}e_{3}=0,\nabla_{e_{1}}e_{4}=0,\nabla_{e_{1}}e_{5}=\alpha
e_{1},$ $\nabla_{e_{2}}e_{1}=0,\nabla_{e_{2}}e_{2}=-\alpha
e_{5},\nabla_{e_{2}}e_{3}=0,\nabla_{e_{2}}e_{4}=0,\nabla_{e_{2}}e_{5}=\alpha
e_{2},$
$\nabla_{e_{3}}e_{1}=0,\nabla_{e_{3}}e_{2}=0,\nabla_{e_{3}}e_{3}=-\alpha
e_{5},\nabla_{e_{3}}e_{4}=0,\nabla_{e_{3}}e_{5}=\alpha e_{3},$
$\nabla_{e_{4}}e_{1}=0,\nabla_{e_{4}}e_{2}=0,\nabla_{e_{4}}e_{3}=0,\nabla_{e_{4}}e_{4}=-\alpha
e_{5},\nabla_{e_{4}}e_{5}=\alpha e_{4},$
$\nabla_{e_{5}}e_{1}=\nabla_{e_{5}}e_{2}=\nabla_{e_{5}}e_{3}=\nabla_{e_{5}}e_{4}=\nabla_{e_{5}}e_{5}=0.$

It can be easily verified that for $e_{5}=\xi$, the manifold satisfies,
$\nabla_{X}\xi=\alpha[X-\eta(X)\xi]$ and $(\nabla_{X}\phi)Y=\alpha[g(\phi
X,Y)\xi-\eta(Y)\phi X]$.
Thus the manifold $M$ is an $\alpha$-cosymplectic manifold.
Also, the Riemannian curvature tensor $R$ is given by,

$R(X,Y)Z=\nabla_{X}\nabla_{Y}Z-\nabla_{Y}\nabla_{X}Z-\nabla_{[X,Y]}Z.$

Hence,

$R(e_{1},e_{2})e_{2}=R(e_{1},e_{3})e_{3}=R(e_{1},e_{4})e_{4}=R(e_{1},e_{5})e_{5}=-\alpha^{2}e_{1},$
$R(e_{1},e_{2})e_{1}=\alpha^{2}e_{2},R(e_{1},e_{3})e_{1}=R(e_{2},e_{3})e_{2}=R(e_{5},e_{3})e_{5}=\alpha^{2}e_{3}$
$R(e_{2},e_{3})e_{3}=R(e_{2},e_{4})e_{4}=R(e_{2},e_{5})e_{5}=-\alpha^{2}e_{2},R(e_{3},e_{4})e_{4}=-\alpha^{2}e_{3},$
$R((e_{1},e_{5})e_{1}=R(e_{2},e_{5})e_{2}=R(e_{3},e_{5})e_{3}=R(e_{4},e_{5})e_{4}=\alpha^{2}e_{5},$
$R(e_{1},e_{4})e_{1}=R(e_{2},e_{4})e_{2}=R(e_{3},e_{4})e_{3}=R(e_{5},e_{4})e_{5}=\alpha^{2}e_{4}.$

From the above relations, we have,

$S(e_{1},e_{1})=S(e_{2},e_{2})=S(e_{3},e_{3})=S(e_{4},e_{4})=S(e_{5},e_{5})=-4\alpha^{2}.$

Hence $r=-20\alpha^{2}$.
Let $\tilde{\nabla}$ be a quarter-symmetric metric connection on $M$.
Then using (3.6), we get,

$\tilde{\nabla}_{e_{1}}e_{1}=-\alpha
e_{5},\tilde{\nabla}_{e_{1}}e_{2}=0,\tilde{\nabla}_{e_{1}}e_{3}=0,\tilde{\nabla}_{e_{1}}e_{4}=0,\tilde{\nabla}_{e_{1}}e_{5}=\alpha
e_{1},$ $\tilde{\nabla}_{e_{2}}e_{1}=0,\tilde{\nabla}_{e_{2}}e_{2}=-\alpha
e_{5},\tilde{\nabla}_{e_{2}}e_{3}=0,\tilde{\nabla}_{e_{2}}e_{4}=0,\tilde{\nabla}_{e_{2}}e_{5}=\alpha
e_{2},$
$\tilde{\nabla}_{e_{3}}e_{1}=0,\tilde{\nabla}_{e_{3}}e_{2}=0,\tilde{\nabla}_{e_{3}}e_{3}=-\alpha
e_{5},\tilde{\nabla}_{e_{3}}e_{4}=0,\tilde{\nabla}_{e_{3}}e_{5}=\alpha e_{3},$
$\tilde{\nabla}_{e_{4}}e_{1}=0,\tilde{\nabla}_{e_{4}}e_{2}=0,\tilde{\nabla}_{e_{4}}e_{3}=0,\tilde{\nabla}_{e_{4}}e_{4}=-\alpha
e_{5},\tilde{\nabla}_{e_{4}}e_{5}=\alpha e_{4},$
$\tilde{\nabla}_{e_{5}}e_{1}=e_{2},\tilde{\nabla}_{e_{5}}e_{2}=-e_{1},\tilde{\nabla}_{e_{5}}e_{3}=e_{4},\tilde{\nabla}_{e_{5}}e_{4}=-e_{3},\tilde{\nabla}_{e_{5}}e_{5}=0.$

Then,

$\tilde{R}(e_{1},e_{2})e_{2}=\tilde{R}(e_{1},e_{3})e_{3}=\tilde{R}(e_{1},e_{4})e_{4}=-\alpha^{2}e_{1},\tilde{R}(e_{1},e_{5})e_{5}=-\alpha^{2}e_{1}-\alpha
e_{2},$
$\tilde{R}(e_{1},e_{2})e_{1}=\alpha^{2}e_{2},\tilde{R}(e_{1},e_{3})e_{1}=\tilde{R}(e_{2},e_{3})e_{2}=\alpha^{2}e_{3},\tilde{R}(e_{5},e_{3})e_{5}=\alpha^{2}e_{3}+\alpha
e_{4}$
$\tilde{R}(e_{2},e_{3})e_{3}=\tilde{R}(e_{2},e_{4})e_{4}=-\alpha^{2}e_{2},\tilde{R}(e_{2},e_{5})e_{5}=-\alpha^{2}e_{2}+\alpha
e_{1},\tilde{R}(e_{3},e_{4})e_{4}=-\alpha^{2}e_{3},$
$\tilde{R}((e_{1},e_{5})e_{1}=\tilde{R}(e_{2},e_{5})e_{2}=\tilde{R}(e_{3},e_{5})e_{3}=\tilde{R}(e_{4},e_{5})e_{4}=\alpha^{2}e_{5},$
$\tilde{R}(e_{1},e_{4})e_{1}=\tilde{R}(e_{2},e_{4})e_{2}=\tilde{R}(e_{3},e_{4})e_{3}=\alpha^{2}e_{4},\tilde{R}(e_{5},e_{4})e_{5}=\alpha^{2}e_{4}-\alpha
e_{3}.$

The above expressions of $\tilde{R}$ satisfy our result (3.8) of Theorem
(3.1).
Also from the above relations, we obtain,

$\tilde{S}(e_{1},e_{1})=\tilde{S}(e_{2},e_{2})=\tilde{S}(e_{3},e_{3})=\tilde{S}(e_{4},e_{4})=\tilde{S}(e_{5},e_{5})=-4\alpha^{2}$
(7.1)

which satisfy our result (3.10) of Theorem (3.1).
Hence,

$\tilde{r}=-20\alpha^{2}=r,$ (7.2)

which satisfies our result (3.11) of Theorem (3.1).
Now putting $X=Y=e_{5}$ in (4.5) and using (4.8), we get,

$(\tilde{\pounds}_{\xi}g)(e_{5},e_{5})=0.$ (7.3)

Then taking $X=Y=e_{5}$ in (4.3) and using (7.1), (7.2), (7.3), we obtain,

$-4\alpha^{2}\rho+\big{[}\Lambda+3\alpha^{2}\rho+2\alpha^{2}q\big{]}+\big{[}\mu+\alpha^{2}\rho\big{]}=0,$
(7.4)

which gives,

$\Lambda+\mu=-2\alpha^{2}q.$

Hence the above equation proves that, $\Lambda$ and $\mu$ satisfies our result
(4.9) of Theorem (4.1) for $n=5$ and so $g$ defines a $\ast$-$\eta$-Ricci-
Yamabe soliton on the 5-dimensional $\alpha$-cosymplectic manifold with
respect to a quarter-symmetric metric connection $\tilde{\nabla}$.

##  8\. Conclusion $\&$ Remarks

The effect of $\ast$-$\eta$-Ricci-Yamabe soliton have been studied within the
framework of $\alpha$-cosymplectic manifold. Here we have characterized
$\alpha$-cosymplectic manifold, which admits $\ast$-$\eta$-Ricci-Yamabe
soliton, in terms of quarter-symmetric metric connection. Also there is
further scope of research in this direction of various types of complex
manifolds like Kaehler manifolds, para-Kaehler manifolds, Hopf manifolds etc.
There are some questions arise from our article to study further research.
(i) Are Theorem 4.1 true if we ignore quarter-symmetric connection on
$\alpha$-cosymplectic manifold?
(ii) Are Theorem 4.4 true without assuming the potential vector field of the
soliton is of gradient type?
(iii) If the conformal vector field is not killing, then is Theorem 6.6 true?
(iv) What are the results of the our paper also in true Trans-Sasakian
manifolds, Co-kähler manifold or para-contact geometry?

## References

  * [1] E. Barbosa and E. Ribeiro Jr., On conformal solutions of the Yamabe flow, Arch. Math., 101 (2013), 79–89.
  * [2] D. E. Blair, Contact Manifolds in Riemannian Geometry, Lecture Notes in Math., 509, Springer-Verlag, Berlin, 1976.
  * [3] H. D. Cao, X. Sun and Y. Zhang, On the structure of gradient Yamabe solitons, arXiv:1108.6316v2 [math.DG] (2011).
  * [4] G. Catino and L. Mazzieri, Gradient Einstein solitons, Nonlinear Anal., 132 (2016), 66–94.
  * [5] J. T. Cho and M. Kimura,Ricci solitons and real hypersurfaces in a complex space form, Tohoku Mathematical Journal, Second Series, 61(2) (2009), 205-212.
  * [6] U. C. De and J. Sengupta, Quater-symmetric metric connection on a sasakian manifold, Commun. Fac. Sci. Univ. Ank. Series Al, 49 (2000), 7-13.
  * [7] A. Ghosh, Yamabe soliton and Quasi Yamabe soliton on Kenmotsu manifold, Mathematica Slovaca, 70(1) (2020), 151-160.
  * [8] S. Golab, On Semi-symmetric and quarter-symmetric connections, Tensor, N.S., 29(1975), 249-254.
  * [9] S. I. Goldberg and K. Yano, Integrability of almost cosymplectic structure, Pac. J. Math., 31 (1969), 373-381.
  * [10] S. Güler and M. Crasmareanu, Ricci-Yamabe maps for Riemannian flows and their volume variation and volume entropy, Turk. J. Math., 43 (2019), 2361-2641.
  * [11] H. Ozturk, C. Murathan, N. Aktan and A. T. Vanli, Almost $\alpha$-cosymplectic $f$-manifolds, Ann. Alexandru Ioan Cuza Univ.-Math., 60(1) (2014), 211-226.
  * [12] R. S. Hamilton, Three Manifold with positive Ricci curvature, J.Differential Geom., 17(2) (1982), 255-306.
  * [13] R. S. Hamilton, The Ricci flow on surfaces, Contemporary Mathematics, 71 (1988), 237-261.
  * [14] T. Hamada, Real hypersurfaces of complex space forms in terms of Ricci $*$-tensor, Tokyo J.Math., 25 (2002), 473-483.
  * [15] A. Haseeb, D. G. Prakasha and H. Harish, $\ast$-Conformal $\eta$-Ricci Soliton on $\alpha$-Cosymplectic manifolds, Int. J. Analy. Appli., 19(2)(2021), 165-179.
  * [16] T. W. Kim and H. K. Pak, Canonical foliations of certain classes of almost contact metric structures, Acta Math. Sin., 21(4) (2005), 841-846.
  * [17] S. Roy and A. Bhattacharyya, Conformal Ricci solitons on 3-dimensional trans-Sasakian manifold, Jordan Journal of Mathematics and Statistics , 13(1) (2020), 89-109.
  * [18] S. Roy, S. Dey, A. Bhattacharyya and S. K. Hui, $*$-Conformal $\eta$-Ricci Soliton on Sasakian manifold, accepted for publication in Asian-European Journal of Mathematics, https://doi.org/10.1142/S1793557122500358.
  * [19] S. Roy, S. Dey and A. Bhattacharyya, Yamabe Solitons on $(LCS)_{n}$-manifolds, Journal of Dynamical Systems $\&$ Geometric Theories, 18(2) (2020), 261-279.
  * [20] S. Roy, S. Dey and A. Bhattacharyya, Some results on $\eta$-Yamabe Solitons in 3-dimensional trans-Sasakian manifold, arXiv:2001.09271v2 [math.DG] (2020).
  * [21] S. Roy, S. Dey and A. Bhattacharyya, Conformal Einstein soliton within the framework of para-Kähler manifold, Differential Geometry-Dynamical Systems, 23 (2021), 235-243.
  * [22] S. Roy, S. Dey and A. Bhattacharyya, A Kenmotsu metric as a conformal $\eta$-Einstein soliton, Carpathian Mathematical Publications, 13(1) (2021), 110-118, https://doi.org/10.15330/cmp.13.1.110-118.
  * [23] S. Roy, S. Dey and A. Bhattacharyya, Conformal Yamabe soliton and $*$-Yamabe soliton with torse forming potential vector field, accepted for publication in Matemati$\check{c}$ki Vesnik (2020), arXiv:2105.13885v1 [math.DG].
  * [24] S. Dey and S. Roy, $*$-$\eta$-Ricci Soliton within the framework of Sasakian manifold, Journal of Dynamical Systems $\&$ Geometric Theories, 18(2) (2020), 163-181.
  * [25] R. Sharma, Almost Ricci solitons and K-contact geometry, Monatsh Math , 175(4) (2014), 621-628.
  * [26] R. Sharma, A 3-dimensional Sasakian metric as a Yamabe Soliton, Int. J. Geom. Methods Mod. Phys., 9(4) (2012), 1220003 (5 pages).
  * [27] M. D. Siddiqi, Conformal $\eta$-Ricci solitons in $\delta$\- Lorentzian Trans Sasakian manifolds, International Journal of Maps in Mathematics, 1(1) (2018), 15-34.
  * [28] M. D. Siddiqi and M. A. Akyol, $\eta$-Ricci-Yamabe Soliton on Riemannian Submersions from Riemannian manifolds, arXiv:2004.14124v1 [math.DG](2020).
  * [29] M. D. Siddiqi, S. A. Siddqui, Conformal Ricci soliton and Geometrical structure in a perfect fluid spacetime, Int. J. Geom. Methods Mod. Phys, (2020) 2050083 (18 pages) https://doi.org/10.1142/S0219887820500838.
  * [30] A. Singh, S. Kishor, Some types of $\eta$-Ricci Solitons on Lorentzian para-Sasakian manifolds, Facta Universitatis (NIŠ)
  * [31] S. Tachibana, On almost-analytic vectors in almost K$\ddot{a}$hlerian manifolds, Tohoku Math.J, 11(2) (1959), 247-265.
  * [32] P. Topping, Lecture on the Ricci Flow, Cambridge University Press(2006).
  * [33] K. Yano, On semi-symmetric connection, Rev. Roum. Math. Pures Appl., 15 (1970), 1570–1586.
  * [34] S.T. Yau, Harmonic functions on complete Riemannian manifolds, Commu. Pure. Appl. Math. 28, (1975), 201-228.


# FedUV: Uniformity and Variance for Heterogeneous Federated Learning 111To be
presented at CVPR 2024

Ha Min Son
University of California, Davis
Moon Hyun Kim
Hippo T&C
Tai-Myoung Chung
Hippo T&C
Chao Huang
University of California, Davis
Xin Liu222Correspondence to<EMAIL_ADDRESS>
University of California, Davis

###### Abstract

Federated learning is a promising framework to train neural networks with
widely distributed data. However, performance degrades heavily with
heterogeneously distributed data. Recent work has shown this is due to the
final layer of the network being most prone to local bias, some finding
success freezing the final layer as an orthogonal classifier. We investigate
the training dynamics of the classifier by applying SVD to the weights
motivated by the observation that freezing weights results in constant
singular values. We find that there are differences when training in IID and
non-IID settings. Based on this finding, we introduce two regularization terms
for local training to continuously emulate IID settings: (1) variance in the
dimension-wise probability distribution of the classifier and (2)
hyperspherical uniformity of representations of the encoder. These
regularizations promote local models to act as if it were in an IID setting
regardless of the local data distribution, thus offsetting proneness to bias
while being flexible to the data. On extensive experiments in both label-shift
and feature-shift settings, we verify that our method achieves highest
performance by a large margin especially in highly non-IID cases in addition
to being scalable to larger models and datasets.

##  1 Introduction

Federated Learning (FL) [31] is a distributed learning framework that allows
the training of deep neural networks with data in decentralized locations. FL
is especially appealing because it achieves similar performance to centralized
training in specific settings while also negating the cost of collecting data
into a centralized location and allowing effective parallelization across
devices [42]. However, because devices participating in FL use locally
collected data for training, it is realistic to expect that the collective
data across all devices are not independent and identically distributed (non-
IID). This severely degrades overall performance. Local devices optimize their
copy of the model towards local optima. In non-IID settings, however, it is
likely that local optima across devices are in disagreement with the true
optimum of the IID setting. Thus, the direction of the gradient causes
dissonance and degraded performance. This phenomena is referred to as client
drift in the FL literature.

To combat client drift, a number of work [20, 1, 19] introduced the idea of
using the global model as a basis for regularization. More recently, some work
showed it may be effective to focus on certain layers which are more prone to
bias. In particular, [29] found that the final layer, referred to as the
classifier, is the most biased in non-IID settings. They introduce a method to
calibrate the classifier to offset this bias. Other work [32] used
augmentation techniques [49] to regularize the classifier, while another work
[34] randomly initialized and froze the weights of the classifier. Since
random vectors in high dimension space are likely to be orthogonal, the output
of the penultimate layer, referred to as the representations or activations of
the encoder, are trained to fit these orthogonal classifiers, thus offsetting
bias.

However, we note previous work have approached the non-IID problem by
regularizing local models to be less biased, rather than directly regularizing
local models to emulate the IID setting. In addition, many work have not been
tested in a feature-shift setting, often only using the Dirichlet distribution
to simulate a label-shift setting. Furthermore, while providing important
insight for FL, these work generally do not take into account efficiency and
scalability. State-of-the-art methods such as FedProx [20], SCAFFOLD [15], and
FedDyn [1] require distances calculations of weights between all layers of
multiple models at every batch. MOON [19] and FedAlign [32] require extra
forward passes at every batch. Computation and memory constraints certainly
become issues as the size of models and datasets grow.

Figure 1: Singular values of the weights of the classifier (final layer)
trained on CIFAR-100. Training setup is specified in Section 4.1

Motivation. In most classification problems, it is standard to minimize a
Cross-Entropy loss function [9] to fit a neural network’s probability
distribution to the sampled probability distribution. When the sampled class
distribution is imbalanced, such as in a non-IID environment, the classifier
becomes biased towards some subspace regardless of the features extracted by
the encoder. Motivated by the success of freezing the classifier as orthogonal
vectors [34], we use the singular value decomposition to gain insight into the
training dynamics of the classifier. We train a Resnet-18 [10] on CIFAR-100
[16] and take the singular value decomposition of the weight matrix
($\mathbf{W}=\mathbf{U}\mathbf{\Sigma}\mathbf{V^{\top}}$,
$\mathbf{\Sigma}=diag(\mathbf{\sigma_{1},...,\sigma_{n}})$) as shown in Fig.
1. The singular values of the classifier decrease in a non-IID setting,
showing there is a difference in the training dynamic between IID and non-IID
settings.

In light of this, we propose an approach to the non-IID problem in which we
directly promote the emulation of an IID setting. There is a nuanced
difference from previous work which regularize to decrease bias. Previous work
use the global model as a source of regularization under the intuition that
the global model is less biased than any single local model. However, this may
not be optimal in every setting as the aggregate of biased local model may
still be biased. We instead focus on regularizing local models to emulate the
IID setting without the use of the global model. Namely, we present FedUV, a
method that promotes IID emulation by inducing representation hyperspherical
Uniformity and classifier Varaiance during local training. FedUV penalizes
classifier probability distributions that are biased towards a subset of
classes, thus promoting probability distributions in non-IID settings to match
IID settings. This promotes local classifiers to be unbiased in their
predictions rather than becoming biased towards local labels. Furthermore, we
penalize representations that are not thoroughly spread across the
hypersphere. There are two reasons for this. First, this encourages the
encoder to not become biased towards local features. Second, expanding the
feature space allows the classifier to expand its variance in more directions.
We show both regularizations are essential for improved performance. FedUV
achieves state-of-the-art performance on various standard label-shift
benchmark in addition to various feature-shift benchmarks while being
extremely simple, efficient, and scalable.

Our contributions are summarized as follows. First, we verify that singular
values of the classifier decreases as a consequence of non-IID environments in
FL (Fig. 1). Second, we present two regularization terms, representation
hyperspherical uniformity and classifier variance, to prevent this
degradation. These regularization terms are simple yet effective. Third, we
show that FedUV achieves state-of-the-art performance not only on label shift
non-IID settings but also on feature shift non-IID settings. Unlike previous
work, FedUV does not use the global model as a source of regularization, which
requires extra forward-pass, requires extra memory, or requires matrix
comparison between weights of each layer, thus being more efficient.

##  2 Related Work

###  2.1 Federated Learning (FL)

The basic FL algorithm is FedAvg [31]. This algorithm progresses with the
repetition of four steps. First, (step 1) a server broadcasts its global
model. Second, (step 2) clients receive and train the model using their own
local data. Third, (step 3) clients upload their trained local model. Fourth,
(step 4) the server aggregates local models to create the next generational
global model.

There are a wide range of recent work in the field of FL such as client
selection [33, 39], data sharing [50], privacy [3, 14], communication
efficiency [46, 48, 7], medical applications [47], and knowledge distillation
[52, 38]. As we cannot hope to cover all work on FL, we mainly focus on work
that address the performance degradation in non-IID settings. Work in this
domain can generally be divided into two groups: 1) papers that focus on the
aggregation process (step 4), and 2) efforts that focus on the local training
process (step 2).

###  2.2 Aggregation Regularization

Work that focus on aggregation generally do not modify the local training
scheme of FedAvg. Clients train their local model on local data using a SGD
based optimizer. To address the non-IID problem, they change how the weights
of the local model are aggregated. FedAvgM [11] applied a momentum term when
aggregating and FedNova [44] normalized local models before averaging. Other
work in Personalized FL [30, 37] used hypernetworks to generate the parameter
of other client layers. PFNM [51] and FedMA [43] introduced a Bayesian method
to align potentially mixed neurons due to the permutation invariance of neural
networks. PAN [22] improved efficiency by encoding position into neurons.
FedDF [23] utilized generated and unlabelled data. CCVR [29] noted the largest
bias exists in the classifier (the last layer) than any other layer. They used
a Gaussian mixture model to create artificial data and calibrated the
classifier.

###  2.3 Local Training Regularization

Work that focus on local training generally do not modify the aggregation
scheme of FedAvg. To address the non-IID problem, they instead penalize local
clients based on different criteria. As our work also belongs to this group,
we discuss them in more detail.

FedProx [20] added a ${L_{2}}$ regularizer between local model weights and
global model weights that prevent local model weights from becoming dissimilar
to the global model. FedDyn [1] added an inner product regularization term
between the current round local model and previous round local model noting
that regularizing solely based on the global model may cause convergence
problems for local models. MOON [19] added a contrastive loss regularization
term noting there should be a balance between the representational similarity
between global model, the current round local model, and previous round local
model.

These methods use the global model as a source of regularization. The
intuition is that the global model is less biased than any single local model.
However, there are two large issues concerning this approach. First, the
global model can also be biased if the local models are biased, since it is an
averaged model. This bias may slow convergence and not improve performance.
Second, efficiency and scalability becomes an issue. FedProx and FedDyn
require the calculation of the ${L_{2}}$ distance between global model and
local model, and MOON requires three forward passes at every batch.

More recently, FedAlign [32] noted that it is possible to use only the local
model for effective regularization. Local models employ GradAug [49] to
regularize itself rather than relying on the global model. However, FedAlign
still suffers in efficiency due to its reliance on an additional forward pass
and estimation of second order information. FedBABU [34], on the other hand,
randomly initializes and freezes the classifier to prevent bias. While this
work can be categorized as Personalized Federated Learning, we include FedBABU
in our discussion due to its focus on the classifier. We further explore these
ideas in Section 4.2.

(a) Cross-entropy loss aligns the representations

(b) Hyperspherical Uniformity regularizes the representations (penultimate
layer output)

(c) Variance regularizes the classifier (final layer output)

Figure 2: Uniformity regularization is applied to the penultimate layer and
variance regularization is applied to the output layer.

##  3 Proposed Method — FedUV

The motivation of FedUV comes from two main observations. First, the
similarity between local models decreases further in later layers of the model
[29], with the classifier (the final layer) being most biased towards local
data. Second, as shown in Fig. 1, the singular values of the classifier of the
global model degenerate in non-IID settings. Our goal then is to prevent this
degeneration, promoting local models to emulate the IID setting regardless of
local data distribution.

We achieve this by penalizing the classifier if it does not act as it would in
an IID setting. We use a hinge loss to regularize the classifier if the class-
wise variance of the probability distribution does not exceed a constant
threshold $c$, which is set to be the class-wise variance of the ground truth
labels in a batch with balanced classes. However, since the classifier acts as
a linear hyperplane finding a separation of the representations in high
dimensional space, increase in classifier variance may be limited by the
representational feature imbalance in feature-shift non-IID settings. We
introduce an additional regularization term to spread the representations of
the encoder throughout the hypershpere. Fig. 2 is a visualization of the FedUV
regularization scheme in $R^{2}$ (a PyTorch-like pseudocode is included in
Appendix 1). We first promote hyperspherical uniformity in the representations
of the encoder (row vectors of the penultimate layer output) then promote the
class-wise variance of the probability distribution (column vectors of the
last layer output). We show that both regularization terms are important to
increasing the performance in label-shift and feature-shift settings.

Our overall loss function is shown in Eq. 1. Each client uses the cross-
entropy loss as well as two regularization terms. The $\mu$ and $\lambda$
terms control the strength of each regularization. These parameters will be
discussed in Section 4.1. We now discuss each regularization term in more
detail.

$\mathcal{L}=\mathcal{L}_{CE}(f_{\theta}(X),Y)+\mu\mathcal{L}_{U}(g_{\theta}(X))+\lambda\mathcal{L}_{V}(f_{\theta}(X))$
(1)

###  3.1 Classifier Variance

Our goal is to encourage the classifier in any setting to emulate a classifier
in an IID setting. We specifically focus on the classifier as it is the layer
of the model most prone to bias in non-IID settings. We do this by inducing
the class-wise variance of the classifier probability distribution to emulate
an IID scenario.

Given a mini-batch $(X,Y)$ and a model $f_{\theta}$ that outputs a prediction
$f_{\theta}(X)=\hat{Y}$, we create a probability distribution on our mini-
batch $\hat{P}:=Softmax(\hat{Y})$. $\hat{P}$ is the probability distribution
matrix of the entire mini-batch, where the row vectors are probabilities of a
single sample and the column vectors are probabilities of a single class
across the entire mini-batch. We define the variance regularization term
$\mathcal{L}_{V}$ as the hinge loss of the square root of the variance
(standard deviation) along the column vectors of the probability matrix as
seen in Eq. 2.

$\begin{multlined}\mathcal{L}_{V}(\hat{Y})=\frac{1}{D}\sum_{j=1}^{D}max(0,c-\sqrt{Var(\hat{P}_{j})})\end{multlined}\mathcal{L}_{V}(\hat{Y})=\frac{1}{D}\sum_{j=1}^{D}max(0,c-\sqrt{Var(\hat{P}_{j})})$
(2)

Here, $D$ is the number of classes, $c$ is a constant that represents the
variance in an IID setting, and $\hat{P}_{j}$ represents the column vectors of
the probability matrix. We set $c$ to be the average class-wise standard
deviation of $\mathbf{A}$ as shown in Eq. 3. Here, $\mathbf{A}$ is a $D$x$D$
identity matrix ($diag(1_{1},...,1_{D})$) where $D$ is the number of classes.
This represents the ideal probability distribution of a balanced mini-batch
where all classes are included.

$\begin{multlined}c:=\frac{1}{D}\sum_{j}^{D}{\sqrt{Var(\mathbf{A}_{j})}}\end{multlined}c:=\frac{1}{D}\sum_{j}^{D}{\sqrt{Var(\mathbf{A}_{j})}}$
(3)

In essence, we ask that the predicted probability distribution in any setting
match the probability distribution of an ideal IID setting where each mini-
batch contains a balanced ratio of the included classes. Furthermore, we
purposefully use a hinge loss to prevent the variance in a few classes from
overpowering the remaining classes.

###  3.2 Hyperspherical Uniformity

Promoting class-wise variance in the classifier may be difficult in scenarios
where the representations are clustered in a small area. For instance, in a
feature-shift non-IID setting a model may biased towards a subset of the
global data manifold. This may also occur in scenarios where a client holds
data from only two classes. There is no incentive for the model to spread the
representations to allow larger separation with a linear hyperplane. We thus
induce hyperspherical uniformity on the representations of the encoder (the
output of the penultimate layer). We use the RBF kernel for its connection
with the unit hypersphere [5], and its success in various deep learning
applications [25, 6]. Namely, we calculate the average pairwise Gaussian
energy between the row vectors of the representations as seen in Eq. 4.

$\begin{multlined}\mathcal{L}_{U}(g_{\theta}(X))=\mathop{\mathbb{E}}_{x,y\in
g_{\theta}(X)}\left[\exp{(-\frac{\lVert
x-y\rVert^{2}_{2}}{2\sigma})}\right]\end{multlined}\mathcal{L}_{U}(g_{\theta}(X))=\mathop{\mathbb{E}}_{x,y\in
g_{\theta}(X)}\left[\exp{(-\frac{\lVert x-y\rVert^{2}_{2}}{2\sigma})}\right]$
(4)

$g_{\theta}(X)$ is the representation matrix of the encoder where $x$ and $y$
represent the row vectors in the representations. $\sigma$ is a hyperparameter
that controls the sensitivity to small distances. This is a hyperparameter
that can be tuned to each setting to improve performance. We set $\sigma$ as
the median value of the squared pairwise distances to ensure hypershperical
uniformity is induced in any setting.

##  4 Experiments and Results

| STL-10 | CIFAR-100 | Tiny ImageNet | PACS | HAM10000 | Office-Home
---|---|---|---|---|---|---
Method |  $\alpha$=0.01 |  $\alpha$=1.0 |  $\alpha$=0.01 |  $\alpha$=1.0 |  $\alpha$=0.01 |  $\alpha$=1.0 |  |  |
FedAvg | 27.6$\pm$1.6 | 68.5$\pm$0.7 | 51.6$\pm$1.7 | 58.3$\pm$1.2 | 37.5$\pm$0.9 | 40.2$\pm$1.2 | 61.9$\pm$0.6 | 73.7$\pm$0.3 | 42.2$\pm$1.1
FedProx | 26.5$\pm$1.5 | 67.9$\pm$0.8 | 50.2$\pm$1.1 | 59.0$\pm$1.0 | 37.1$\pm$1.1 | 39.9$\pm$1.0 | 59.1$\pm$0.8 | 73.5$\pm$0.3 | 41.9$\pm$0.6
MOON | 26.0$\pm$1.3 | 70.0$\pm$0.6 | 48.2$\pm$1.2 | 60.5$\pm$0.7 | 37.6$\pm$1.8 | 42.4$\pm$0.9 | 62.8$\pm$1.6 | 72.9$\pm$0.4 | 41.9$\pm$0.4
Freeze | 23.7$\pm$1.5 | 72.1$\pm$0.6 | 51.1$\pm$1.5 | 59.3$\pm$1.8 | 38.9$\pm$1.9 | 42.6$\pm$1.6 | 61.6$\pm$0.5 | 73.2$\pm$0.3 | 41.3$\pm$1.8
FedUV | 30.4$\pm$1.4 | 68.5$\pm$0.6 | 55.7$\pm$1.0 | 59.1$\pm$0.9 | 40.3$\pm$0.8 | 43.2$\pm$1.5 | 65.9$\pm$0.9 | 73.9$\pm$0.5 | 45.4$\pm$1.0
Table 1: Test accuracies with $\alpha\in\\{0.01,1.0\\}$ on STL-10, CIFAR-100,
and Tiny ImageNet and PACS, HAM10000, and Office-Home. We bold the highest
performing method and underline the second highest performing method.

###  4.1 Experimental Setup

We compare FedUV with the state-of-the-art FL algorithms: FedAvg [31], FedProx
[20], and MOON [19]. We also include a method Freeze which freezes the final
layer. This is similar to FedBABU [34] without personalization. We use the
PyTorch [35] library and follow the official implementations for all available
work.

Label-shift Datasets. STL-10 [4] is a dataset with 10 classes with balanced
5,000 training and 8,000 validation samples. CIFAR-100 [16] is a dataset with
100 classes with 60,000 training and 10,000 validation samples. Tiny ImageNet
[18] is a dataset downsampled from the ImageNet [36] dataset with 200 classes
with balanced 100,000 training and 10,000 validation samples. Basic data
augmentation, random cropping and horizontal flipping, is used consistently
throughout all datasets and methods. As with many previous work [19, 34, 32],
we use the Dirichlet distribution to simulate label-shift non-IID settings.
The $\alpha$ term controls the extent of non-IIDness with $\alpha=0$ being
most non-IID and $\alpha=\infty$ being IID. We use $\alpha\in\\{0.01,1.0\\}$.
We split the training datasets into a 90-10 training-validation dataset and
use the original validation set as a test set.

Feature-shift Dataests. A more recent work [21] has shown it is important also
to focus specifically on feature-shift environments. To simulate feature-
shift, we use 3 datasets which includes data from various domains. PACS [17]
is a dataset with 7 classes in 9,991 images from four domains: Photo (1,670
images), Art painting (2,048 images), Cartoon (2,344 images), and Sketch
(3,929 images). We randomly sampled an equal 280 images per domain (40 images
per class in each domain) for our testing dataset (1,120 images in total).
HAM10000 [40] is a dataset with 7 classes in 10,015 images from four domains.
The domains are represented by different institutions which collected
dermoscopic (skin disease) images. Each domain has 3363, 2259, 439, 3954
images respectively. Due to the imbalance in data, we took roughly 20% from
each domain for 1955 images for our testing set. Office-Home [41] is a dataset
with 65 classes in 15,588 images from 4 domains: Art (2,427 images), Clipart
(4,365 images), Product (4,439 images), and Photo (4,357 images). We randomly
sampled an equal 455 images per domain (7 images per class in each domain) for
our testing dataset (1,820 images in total). We resize images from these
datasets into 96x96x3 images and apply basic data augmentation (random
cropping and horizontal flipping).

A detailed overview of the distributions of the datasets are provided in
Appendix 2.

Models. We use 3 different models across the different datasets. Note that we
also add a non-linear projector following [19] due to reduced performance
without it. This is simply two fully-connected layers followed by a batch norm
and ReLu activation. For STL-10 and PACS, we used a small CNN model (Appendix
3) with a projector with 256 neurons. For CIFAR-100 and PACS, we used a
ResNet-18 [10] model, removing the initial max-pooling layer due to the small
image size and adding the projector with 512 neurons. For Tiny ImageNet and
Office-Home, we used a ResNet-50 model, removing the initial max-pooling layer
and adding the projector with 2048 neurons. On all datasets, we report the
results of a single aggregated global model.

Hyperparameters. Our default testing environment for our ablation is done on
STL-10 ($\alpha=0.01$) with a small CNN (Appendix 3) and Office-Home with a
ResNet-50, local epoch $E=10$, number of clients $\kappa=10$ for STL-10 and
$\kappa=4$ for Office-Home (for each of the domains), participation rate
$\rho=1.0$. The total aggregation rounds is chosen based on performance of the
training-validation split on FedAvg. We set total aggregation rounds, $R$, as
100, 60, 40 for STL-10, CIFAR-100, Tiny ImageNet, and 60, 60, 40 for PACS,
HAM10000, and Office-Home, respectively. All reported accuracy are averaged
over 3 runs. Changes from these default settings are mentioned clearly. We use
the Cross-entropy loss and SGD with learning rate of 0.01, momentum 0.9, and
weight decay of 1e-5.

FedProx, MOON, and FedUV are all regularization methods, thus adds an extra
parameter to balance the cross-entropy loss and the strength of
regularization. We tune this parameter on the CIFAR-10 [16] dataset, since
tuning this parameter in every setting of each dataset is not viable option
for FL as resources are limited. For FedProx, we use $\mu=0.01$, for MOON,
$\mu=1.0$. For FedUV, we use fixed hyperparameters $\lambda=\frac{D}{4}$,
where $D$ is the number of classes, and $\mu=0.5$ for all datasets. We report
the hyperparameter tuning results in the appendix (Appendix 4). All
experiments were conducted on a single RTX A5000 and one AMD EPYC 7763
processor.

###  4.2 Results

Data Heterogeneity. We study how data heterogeneity changes performance of the
global model. We test the performance for $\alpha\in\\{0.01,1\\}$ for label-
shift datasets, and use the defined domains for feature-shift datasets.
Results are shown in Table 1.

Across most settings, FedUV outperforms other methods. Focusing first on the
label-shift non-IID settings, we see that in the extreme $\alpha=0.01$
setting, FedUV performs best. FedProx and MOON peform worse than FedAvg as
they base their regularization on the global model which may also be biased
since it is averaging extremely biased local models. Freeze also does not
perform well, as the frozen classifier causes the encoder to have difficulty
learning in extreme settings. By emulating an IID setting, FedUV outperforms
these methods.

In the less extreme setting of $\alpha=1.0$ other regularization methods
generally perform well. On STL-10, Freeze performs best by a large margin. For
less extreme label shift and small number of classes, a frozen classifier is
beneficial as this prevents the classifier from being biased, which has been
established as a desirable property. MOON performs best on CIFAR-100, and also
improves performance compared to FedAvg on the Tiny ImageNet dataset. However,
we find that Freeze and FedUV achieve the better performance on Tiny ImageNet
most likely because each client holds more data as the dataset is much larger.
When data is plentiful, it is likely that focusing specifically on the
classifier is most beneficial.

In the feature-shift setting, we find that baseline regularization methods
generally does not help performance. This could be because these work have
focused only on label-shift settings. Freeze simply freezes the final layer,
while MOON and FedProx use the global model as a source of regularization to
mitigate the bias of individual clients. However, because FedUV simulates an
IID setting rather than mitigating the effects of label-shift by relying on
the global model as a source of regularization, performance is improved not
only in label-shift settings but also feature-shift settings. As will be shown
in the ablation study, the Uniformity regularizer is indeed an important
factor to prevent this performance degradation.

| STL-10 ($\alpha=0.01$) | Office-Home
---|---|---
Method | p=0.1 | p=0.5 | p=1.0 | p=0.25 | p=0.5 | p=1.0
FedAvg | 17.4$\pm$3.4 | 21.5$\pm$1.8 | 27.6$\pm$1.6 | 39.1$\pm$2.5 | 41.9$\pm$1.4 | 42.2$\pm$1.1
FedProx | 19.3$\pm$2.9 | 22.0$\pm$1.3 | 26.5$\pm$1.5 | 39.6$\pm$2.1 | 40.8$\pm$1.7 | 41.9$\pm$0.6
MOON | 20.5$\pm$3.7 | 23.6$\pm$1.7 | 26.0$\pm$1.3 | 40.3$\pm$2.7 | 40.5$\pm$1.6 | 41.9$\pm$0.4
Freeze | 21.5$\pm$3.6 | 23.7$\pm$1.6 | 23.7$\pm$1.5 | 37.6$\pm$3.9 | 39.5$\pm$1.1 | 41.3$\pm$1.8
FedUV | 24.9$\pm$3.1 | 30.0$\pm$1.1 | 30.4$\pm$1.4 | 41.7$\pm$2.4 | 44.8$\pm$1.3 | 45.4$\pm$1.0
Table 2: Accuracy across client participation rate

(a) STL-10

(b) CIFAR-100

(c) PACS

(d) Office-Home

Figure 3: The training loss for FedAvg and FedUV across different settings

Client participation. In real-world FL applications, it is realistic to expect
that only a subset of all clients participate in an aggregation round. The
server must triggers the aggregation process even if some clients do not
participate. We study the effect of ratio of client participation, $\rho$,
where $\rho=0.1$ means 10% of clients participate in the aggregation round.
Across these settings, we use 100 aggregation rounds for STL-10 and 60 for
Office-Home. The results are shown in Table 2.

The advantages of FedUV are clear even when the ratio of participating clients
is low. Indeed, when $\rho=0.1$ for STL-10, FedUV performs the best by a large
margin. This increase in performance occurs owing to the focus of FedUV to
emulate the IID. The second best method is Freeze. Intuitively, since the
classifier is the layer that is most prone to bias, this is magnified in
scenarios where client participation is low, as bias of the averaged global
model is further increased. By fixing the classifier as in Freeze, we limit
this bias and increase performance. However, as seen with the performance
increase of FedUV, it is far more beneficial to emulate an IID setting. It is
also interesting to see that when $\rho=0.25$ for Office-Home (1 client
participating), FedProx and MOON improves performance. When the server
aggregates weights from only 1 domain, it may be beneficial to use the global
model as regularization. This improvement does not continue for $\rho=0.5$ (2
clients participating), suggesting that the average of two domains is more
beneficial than the use of the global model for regularization.

| STL-10 ($\alpha=0.01$) | Office-Home
---|---|---
Method | $E=10$ | $E=20$ | $E=40$ | $E=10$ | $E=20$ | $E=40$
FedAvg | 27.6$\pm$1.6 | 29.8$\pm$1.1 | 29.1$\pm$0.3 | 42.2$\pm$1.1 | 43.5$\pm$1.0 | 35.1$\pm$0.7
FedProx | 26.5$\pm$1.5 | 28.6$\pm$0.8 | 27.4$\pm$0.6 | 41.9$\pm$0.6 | 43.3$\pm$0.5 | 35.7$\pm$0.5
MOON | 26.0$\pm$1.3 | 28.9$\pm$0.7 | 27.4$\pm$0.4 | 41.9$\pm$0.4 | 42.0$\pm$0.4 | 41.7$\pm$0.5
Freeze | 23.7$\pm$1.5 | 29.9$\pm$0.9 | 27.2$\pm$0.7 | 41.3$\pm$1.8 | 41.3$\pm$0.8 | 35.4$\pm$0.9
FedUV | 30.4$\pm$1.4 | 31.5$\pm$1.2 | 25.9$\pm$0.9 | 45.4$\pm$1.0 | 46.1$\pm$0.7 | 35.9$\pm$0.8
Table 3: Accuracy across different number of local epochs

Number of local epochs We study the effect of number of local epochs, $E$,
where $E=40$ means each clients trains for 40 epochs during a single
aggregation round. Results are shown in Table 3. It would be desirable to
achieve high performance with a low number of local epochs thereby reducing
the burden of less powerful edge devices. We find that when $E=20$ performance
generally improves. However, when the number of local epochs increase to 40,
the accuracy of all methods decrease, suggesting overfitting occurs with a
high number of local epochs. The performance of FedUV significantly degraded
when local epoch is high. This suggests high local epochs cause local models
to converge to local optima, harming performance. On the other hand, MOON and
FedProx did not show a large drop in performance. This can be explained by the
fact that these methods rely on the global model for regularization. Because
of this, local models are less likely to converge to local optima as compared
to FedUV.

| STL | CIFAR | Tiny | PACS | HAM | Office
---|---|---|---|---|---|---
Method |  |  |  |  |  |
FedAvg | 27.6$\pm$1.6 | 51.6$\pm$1.7 | 37.5$\pm$0.9 | 61.9$\pm$0.6 | 73.3$\pm$0.3 | 42.2$\pm$1.1
Freeze | 23.7$\pm$1.5 | 54.1$\pm$1.5 | 38.9$\pm$1.9 | 61.6$\pm$0.5 | 73.2$\pm$0.3 | 41.3$\pm$1.8
Only $L_{V}$ | 29.5$\pm$1.4 | 54.0$\pm$1.1 | 38.8$\pm$1.5 | 60.1$\pm$0.3 | 72.5$\pm$0.4 | 41.8$\pm$1.1
Only $L_{U}$ | 24.1$\pm$1.3 | 52.8$\pm$1.2 | 37.9$\pm$1.7 | 63.7$\pm$0.3 | 73.4$\pm$0.2 | 43.5$\pm$1.3
FedUV | 30.4$\pm$1.4 | 55.7$\pm$1.0 | 40.3$\pm$0.9 | 65.9$\pm$0.9 | 73.9$\pm$0.5 | 45.4$\pm$1.0
Table 4: FedUV ablation study

Ablation Study. We study the effect of hyperspherical uniformity and
classifier variance separately. We also compare Freeze which creates an
orthogonal classifier by randomly initializing and freezing the weights.
Results are shown in Table 4. For the label-shift datasets (STL-10, CIFAR-100,
Tiny ImageNet), we find that hyperspherical uniformity regularization, Only
$L_{U}$, does not improve performance significantly, whereas classifier
variance regularization, Only $L_{V}$, performs much better than Freeze on the
STL-10 dataset while being competitive in other datasets. These results
suggests that label-shift settings, focusing on the final layer is more
important than focusing on the feature representations of the encoder, and
that emulating an IID label distribution is more desirable than a fixed
classifier.

Furthermore, for the feature-shift datasets (PACS, HAM10000, Office-Home), we
find that varaince regularization does not improve performance significantly,
whereas uniformity regularization improves performance compared to both Only
$L_{V}$ and Freeze. This suggests that in feature-shift settings, when clients
have vastly differing feature distributions, it is beneficial to encourage
features to be more spread. Intuitively, this prevents any local encoder from
heavy bias towards a subspace, which is an aspect other regularization methods
do not incorporate.

Overall, these results suggests that, in label-shift situations, focusing
specifically on the classifier through variance regularization is more
important, while in feature-shift scenarios, focusing specifically on the
feature space representations of the encoder through hyperspherical uniformity
is more important. However, both regularization terms are important to emulate
the IID setting regardless of data distribution. The combined approach in
FedUV shows further improved performance across these various settings.

Convergence. We study the convergence of FedUV. Graphs of the training loss
are shown in Fig. 3. FedUV shows quicker convergence when compared to FedAvg,
often reaching a lower loss at earlier training iterations. This is intuitive
considering the performance improvement FedUV provies over FedAvg. We also
find that in the extreme case of STL-10 $\alpha=0.01$, the loss curve is much
smoother for FedUV when compared to FedAvg. This shows FedUV stabilizes
training.

##  5 Discussion

Figure 4: t-SNE on the features (output of the penultimate layer) of STL-10
(Class #1). Experiment setup is equivalent to main text. Clients and class
were selected at random.

###  5.1 Preventing Classifier Bias

Our method of emulating the IID setting by encouraging classifier variance is
not the only method to prevent classifier bias. When focusing on label-shift,
many loss function weighting and specialized sampling techniques have been
considered [13]. There have also been interesting applications in
representation learning that use latent space representations to sample
training data such that the predicted probability distribution is uniform [2].
These are interesting directions to explore for addressing the non-IID
problem. The biggest contraint using these approaches in FL however, is
privacy. The data and class distributions cannot be known beforehand, thus
cannot be set to match each client. In addition, parameters that control
sampling or loss weighting may introduce security vulnerabilities.

Another straightforward method would be to randomly initialize and freeze the
classifier. Freezing the classifier would also prevent biases forming in the
probability distribution of the classifier. Indeed, this is the intuition
behind FedBABU [34] and the Freeze method we have used as our baseline, which
initializes and freezes the classifier to orthogonal vectors. Freezing weights
also prevents the degeneration of singular values in non-IID settings, a
phenomenon shown in Fig. 1.

However, it may not be optimal to separate classes with orthogonal vectors in
every setting. A number of work [28, 25] have shown that when the number of
neurons exceeds the dimensions of data manifold, promoting orthogonality is
problematic. This problem arises when there is a large number of classes with
data that lives in a lower subspace. Another alternative would be to
regularize the weights directly. One could maximize the spectral norm, the
largest singular value, or minimize the Frobenius norm, the ${L_{2}}$ vector-
norm of the singular values. These methods may also bridge the difference in
singular values between IID and non-IID while preventing classifier basis.
These methods differ from FedUV as we regularize in the representation space
(the outputs of the model) rather than in the weight space. The main reason is
efficiency. We re-use the representations obtained in the forward-pass of the
model without directly accessing the weight parameters.

###  5.2 Hyperspherical Uniformity

The goals of hypershperical uniformity in FedUV are to mitigate
representational bias and to create separation between representations such
that the classifier has more directions to increase its variance. Note that
this regularizer focuses on feature distributions, not class distributions.
This is shown in Figure 4. Features are aligned regardless of class
distribution. Empirically, we have shown that this regularization is
beneficial in feature-shift settings. There are alternatives to this, such as
spreading representations along a hypercube. However, hyperspherical
uniformity is gaining popularity in the field of deep learning due to its
desirable properties [25, 6, 24, 26, 27]. While more traditional
regularization approaches such as the $L_{2}$ regularization have seen success
in various applications in the wider field of machine learning and statistics
[12, 8], neural networks are special due to their high over-parameterization.
Using $L_{2}$ regularization essentially focuses only on individual weights
rather than taking into account the interactions between these over-
parameterized neurons. Hyperspherical uniformity regularizes based on these
interactions and also has clear geometric properties. We point readers to [28]
for a recent in-depth discussion on its development in deep learning.

Furthermore, we choose the RBF kernel for its efficiency. In particular, the
kernel method allows us to first calculate the the pairwise ${L_{2}}$ norms of
the row vectors before using the kernel function. It also allows for easy
differentiation on automatic differentiation software, which facilitate the
training of neural networks.

| STL | CIFAR | Tiny | PACS | HAM | Office
---|---|---|---|---|---|---
Method | (10-Layer) | (ResNet-18) | (ResNet-50) | (10-Layer) | (ResNet-18) | (ResNet-50)
FedAvg | 26 | 57 | 728 | 47 | 131 | 359
FedProx | 27 | 67 | 823 | 48 | 135 | 371
MOON | 37 | 88 | 1186 | 70 | 145 | 493
Freeze | 26 | 56 | 728 | 47 | 129 | 351
FedUV | 27 | 58 | 755 | 48 | 132 | 362
Table 5: Average time (in seconds) per aggregation round.

###  5.3 Convergence of FedUV

We have shown in Fig. 3 that in our extensive testing environments, FedUV
converges stably. In the general case, we know that the Uniformity regularizer
converges weakly to the uniform distribution, since we are minimizing the
average pairwise gaussian potential between sample points. As the number of
samples approaches infinity, the loss will converge weakly to the uniform
distribution according to Wang and Isola [45]. For the Variance regularizer,
however, the hinge loss of the regularizer is not smooth. Because of this, it
is non-trivial to prove convergence. This may become much simpler when
replaced with a smooth surrogate loss. We leave the proof for the convergence
of this regularizer for future work.

###  5.4 Efficiency and Scalability

One of the most important aspects in FL is efficiency. Edge devices cannot be
expected to have powerful hardware that allows heavy computation in real-world
scenarios. We unfortunately find that many regularization methods are not very
scalable. While this is not a problem for small datasets and models, as
dataset size and model size both increase at a rapid rate, these models become
more computationally expensive.

We compare the wall-clock time of different methods as shown in Table 5. Note
that the time extended is per aggregation round with 10 local epochs. FedProx
requires $L_{2}$ norm calculations between each layer of local model and
global model at each batch, while MOON requires 3 forward passes at each batch
as well as the storage of 3 full models. Though this may not be a problem with
small models, FedProx and MOON are not very scalable methods as the size of
models increases. FedUV is a simple yet efficient regularization technique as
it only requires few matrix operations with no weight parameter access, extra
memory, or extra forward passes. The most expensive operation is the pairwise
distance calculation which is negligible even with less powerful hardware.

##  6 Conclusion

In this work, we introduced a new approach to address the non-IID problem in
FL by encouraging local models to emulate the IID setting. Rather than
regularizing to reduce bias, we emulate the IID setting by promoting variance
in the predicted dimension-wise probability distribution. We also promote
hyperspherical uniformity on the representations of the encoder to allow the
classifier to increase variance in more directions. Overall, our method
improves performance by a large margin throughout our extensive experiments
while being the most efficient and scalable among regularization methods.

## References

  * Acar et al. [2021] Durmus Alp Emre Acar, Yue Zhao, Ramon Matas Navarro, Matthew Mattina, Paul N Whatmough, and Venkatesh Saligrama.  Federated learning based on dynamic regularization.  _arXiv preprint arXiv:2111.04263_ , 2021.
  * Amini et al. [2019] Alexander Amini, Ava P Soleimany, Wilko Schwarting, Sangeeta N Bhatia, and Daniela Rus.  Uncovering and mitigating algorithmic bias through learned latent structure.  In _Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society_ , pages 289–295, 2019.
  * Cheng et al. [2022] Anda Cheng, Peisong Wang, Xi Sheryl Zhang, and Jian Cheng.  Differentially private federated learning with local regularization and sparsification.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 10122–10131, 2022.
  * Coates et al. [2011] Adam Coates, Andrew Ng, and Honglak Lee.  An analysis of single-layer networks in unsupervised feature learning.  In _Proceedings of the fourteenth international conference on artificial intelligence and statistics_ , pages 215–223. JMLR Workshop and Conference Proceedings, 2011.
  * Cohn and Kumar [2007] Henry Cohn and Abhinav Kumar.  Universally optimal distribution of points on spheres.  _Journal of the American Mathematical Society_ , 20(1):99–148, 2007.
  * Davidson et al. [2018] Tim R Davidson, Luca Falorsi, Nicola De Cao, Thomas Kipf, and Jakub M Tomczak.  Hyperspherical variational auto-encoders.  _arXiv preprint arXiv:1804.00891_ , 2018.
  * Diao et al. [2020] Enmao Diao, Jie Ding, and Vahid Tarokh.  Heterofl: Computation and communication efficient federated learning for heterogeneous clients.  _arXiv preprint arXiv:2010.01264_ , 2020.
  * Gatys et al. [2016] Leon A Gatys, Alexander S Ecker, and Matthias Bethge.  Image style transfer using convolutional neural networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 2414–2423, 2016.
  * Good [1992] Irving John Good.  Rational decisions.  In _Breakthroughs in statistics_ , pages 365–377. Springer, 1992.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 770–778, 2016.
  * Hsu et al. [2019] Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown.  Measuring the effects of non-identical data distribution for federated visual classification.  _arXiv preprint arXiv:1909.06335_ , 2019.
  * Johnson et al. [2016] Justin Johnson, Alexandre Alahi, and Li Fei-Fei.  Perceptual losses for real-time style transfer and super-resolution.  In _European conference on computer vision_ , pages 694–711. Springer, 2016.
  * Johnson and Khoshgoftaar [2019] Justin M Johnson and Taghi M Khoshgoftaar.  Survey on deep learning with class imbalance.  _Journal of Big Data_ , 6(1):1–54, 2019.
  * Karimireddy et al. [2020a] Sai Praneeth Karimireddy, Lie He, and Martin Jaggi.  Byzantine-robust learning on heterogeneous datasets via bucketing.  _arXiv preprint arXiv:2006.09365_ , 2020a.
  * Karimireddy et al. [2020b] Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh.  Scaffold: Stochastic controlled averaging for federated learning.  In _International Conference on Machine Learning_ , pages 5132–5143. PMLR, 2020b.
  * Krizhevsky [2009] Alex Krizhevsky.  Learning multiple layers of features from tiny images, 2009.
  * Li et al. [2017] Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales.  Deeper, broader and artier domain generalization.  In _Proceedings of the IEEE international conference on computer vision_ , pages 5542–5550, 2017.
  * Li et al. [2014] Fei-Fei Li, Andrej Karpathy, and Justin Johnson.  Tiny imagenet, 2014.
  * Li et al. [2021a] Qinbin Li, Bingsheng He, and Dawn Song.  Model-contrastive federated learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 10713–10722, 2021a.
  * Li et al. [2020] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith.  Federated optimization in heterogeneous networks.  _Proceedings of Machine Learning and Systems_ , 2:429–450, 2020.
  * Li et al. [2021b] Xiaoxiao Li, Meirui Jiang, Xiaofei Zhang, Michael Kamp, and Qi Dou.  Fedbn: Federated learning on non-iid features via local batch normalization.  _arXiv preprint arXiv:2102.07623_ , 2021b.
  * Li et al. [2022] Xin-Chun Li, Yi-Chu Xu, Shaoming Song, Bingshuai Li, Yinchuan Li, Yunfeng Shao, and De-Chuan Zhan.  Federated learning with position-aware neurons.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 10082–10091, 2022.
  * Lin et al. [2020] Tao Lin, Lingjing Kong, Sebastian U Stich, and Martin Jaggi.  Ensemble distillation for robust model fusion in federated learning.  _Advances in Neural Information Processing Systems_ , 33:2351–2363, 2020.
  * Liu et al. [2017] Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song.  Sphereface: Deep hypersphere embedding for face recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 212–220, 2017.
  * Liu et al. [2018a] Weiyang Liu, Rongmei Lin, Zhen Liu, Lixin Liu, Zhiding Yu, Bo Dai, and Le Song.  Learning towards minimum hyperspherical energy.  _Advances in neural information processing systems_ , 31, 2018a.
  * Liu et al. [2018b] Weiyang Liu, Zhen Liu, Zhiding Yu, Bo Dai, Rongmei Lin, Yisen Wang, James M Rehg, and Le Song.  Decoupled networks.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 2771–2779, 2018b.
  * Liu et al. [2019] Weiyang Liu, Zhen Liu, James M Rehg, and Le Song.  Neural similarity learning.  _Advances in Neural Information Processing Systems_ , 32, 2019.
  * Liu et al. [2021] Weiyang Liu, Rongmei Lin, Zhen Liu, Li Xiong, Bernhard Schölkopf, and Adrian Weller.  Learning with hyperspherical uniformity.  In _International Conference On Artificial Intelligence and Statistics_ , pages 1180–1188. PMLR, 2021.
  * Luo et al. [2021] Mi Luo, Fei Chen, Dapeng Hu, Yifan Zhang, Jian Liang, and Jiashi Feng.  No fear of heterogeneity: Classifier calibration for federated learning with non-iid data.  _Advances in Neural Information Processing Systems_ , 34:5972–5984, 2021.
  * Ma et al. [2022] Xiaosong Ma, Jie Zhang, Song Guo, and Wenchao Xu.  Layer-wised model aggregation for personalized federated learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 10092–10101, 2022.
  * McMahan et al. [2017] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas.  Communication-efficient learning of deep networks from decentralized data.  In _Artificial intelligence and statistics_ , pages 1273–1282. PMLR, 2017.
  * Mendieta et al. [2022] Matias Mendieta, Taojiannan Yang, Pu Wang, Minwoo Lee, Zhengming Ding, and Chen Chen.  Local learning matters: Rethinking data heterogeneity in federated learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 8397–8406, 2022.
  * Nagalapatti et al. [2022] Lokesh Nagalapatti, Ruhi Sharma Mittal, and Ramasuri Narayanam.  Is your data relevant?: Dynamic selection of relevant data for federated learning.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , 2022.
  * Oh et al. [2021] Jaehoon Oh, Sangmook Kim, and Se-Young Yun.  Fedbabu: Towards enhanced representation for federated image classification.  _arXiv preprint arXiv:2106.06042_ , 2021.
  * Paszke et al. [2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala.  Pytorch: An imperative style, high-performance deep learning library.  In _Advances in Neural Information Processing Systems 32_ , pages 8024–8035. Curran Associates, Inc., 2019.
  * Russakovsky et al. [2015] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei.  ImageNet Large Scale Visual Recognition Challenge.  _International Journal of Computer Vision (IJCV)_ , 115(3):211–252, 2015.
  * Shamsian et al. [2021] Aviv Shamsian, Aviv Navon, Ethan Fetaya, and Gal Chechik.  Personalized federated learning using hypernetworks.  In _International Conference on Machine Learning_ , pages 9489–9502. PMLR, 2021.
  * Shen et al. [2022] Yiqing Shen, Yuyin Zhou, and Lequan Yu.  Cd2-pfed: Cyclic distillation-guided channel decoupling for model personalization in federated learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 10041–10050, 2022.
  * Tang et al. [2022] Minxue Tang, Xuefei Ning, Yitu Wang, Jingwei Sun, Yu Wang, Hai Li, and Yiran Chen.  Fedcor: Correlation-based active client selection strategy for heterogeneous federated learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 10102–10111, 2022.
  * Tschandl et al. [2018] Philipp Tschandl, Cliff Rosendahl, and Harald Kittler.  The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions.  _Scientific data_ , 5(1):1–9, 2018.
  * Venkateswara et al. [2017] Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan.  Deep hashing network for unsupervised domain adaptation.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 5018–5027, 2017.
  * Verbraeken et al. [2019] Joost Verbraeken, Matthijs Wolting, Jonathan Katzy, Jeroen Kloppenburg, Tim Verbelen, and Jan S. Rellermeyer.  A survey on distributed machine learning, 2019.
  * Wang et al. [2020a] Hongyi Wang, Mikhail Yurochkin, Yuekai Sun, Dimitris Papailiopoulos, and Yasaman Khazaeni.  Federated learning with matched averaging.  _arXiv preprint arXiv:2002.06440_ , 2020a.
  * Wang et al. [2020b] Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H Vincent Poor.  Tackling the objective inconsistency problem in heterogeneous federated optimization.  _Advances in neural information processing systems_ , 33:7611–7623, 2020b.
  * Wang and Isola [2020] Tongzhou Wang and Phillip Isola.  Understanding contrastive representation learning through alignment and uniformity on the hypersphere.  In _International Conference on Machine Learning_ , pages 9929–9939. PMLR, 2020.
  * Wu et al. [2022] Donglei Wu, Xiangyu Zou, Shuyu Zhang, Haoyu Jin, Wen Xia, and Binxing Fang.  Smartidx: Reducing communication cost in federated learning by exploiting the cnns structures.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , pages 4254–4262, 2022.
  * Xu et al. [2022] An Xu, Wenqi Li, Pengfei Guo, Dong Yang, Holger R Roth, Ali Hatamizadeh, Can Zhao, Daguang Xu, Heng Huang, and Ziyue Xu.  Closing the generalization gap of cross-silo federated medical image segmentation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 20866–20875, 2022.
  * Yang et al. [2021] Haibo Yang, Minghong Fang, and Jia Liu.  Achieving linear speedup with partial worker participation in non-iid federated learning.  _arXiv preprint arXiv:2101.11203_ , 2021.
  * Yang et al. [2020] Taojiannan Yang, Sijie Zhu, and Chen Chen.  Gradaug: A new regularization method for deep neural networks.  _Advances in Neural Information Processing Systems_ , 33:14207–14218, 2020.
  * Yoon et al. [2021] Tehrim Yoon, Sumin Shin, Sung Ju Hwang, and Eunho Yang.  Fedmix: Approximation of mixup under mean augmented federated learning.  _arXiv preprint arXiv:2107.00233_ , 2021.
  * Yurochkin et al. [2019] Mikhail Yurochkin, Mayank Agarwal, Soumya Ghosh, Kristjan Greenewald, Nghia Hoang, and Yasaman Khazaeni.  Bayesian nonparametric federated learning of neural networks.  In _International Conference on Machine Learning_ , pages 7252–7261. PMLR, 2019.
  * Zhang et al. [2022] Lin Zhang, Li Shen, Liang Ding, Dacheng Tao, and Ling-Yu Duan.  Fine-tuning global model via data-free knowledge distillation for non-iid federated learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 10174–10183, 2022.