# Effect of Earth-Moon’s gravity on TianQin’s range acceleration noise. II.
Impact of orbit selection

Chengjian Luo  Xuefeng Zhang<EMAIL_ADDRESS>MOE Key Laboratory of
TianQin Mission, TianQin Research Center for Gravitational Physics & School of
Physics and Astronomy, Frontiers Science Center for TianQin, Gravitational
Wave Research Center of CNSA, Sun Yat-sen University (Zhuhai Campus), Zhuhai
519082, China.

###### Abstract

The paper is a sequel to our previous work (Zhang et al. Phys. Rev. D 103,
062001 (2021)). For proposed geocentric space-based gravitational wave
detectors such as TianQin, gLISA, and GADFLI, the gravity-field disturbances,
i.e., the so called “orbital noise”, from the Earth-Moon system on the
sensitive intersatellite laser interferometric measurements should be
carefully evaluated and taken into account in the concept studies. Based on
TianQin, we investigate how the effect, in terms of frequency spectra, varies
with different choices of orbital orientations and radii through single-
variable studies, and present the corresponding roll-off frequencies that may
set the lower bounds of the targeted detection bands. The results, including
the special cases of geostationary orbits (gLISA/GADFLI) and repeat orbits,
can provide a useful input to orbit and constellation design for future
geocentric missions.

††preprint: APS/123-QED

##  I Introduction

In ground-based gravitational wave (GW) detectors such as LIGO [1], Virgo [2],
and KAGRA [3], Newtonian (or gravity-gradient) noise due to nearby terrestrial
gravity fluctuations constitutes a prominent obstacle to sensitivity
improvement below $\sim 10$ Hz [4]. This difficulty on the ground, in addition
to seismic noise and the need of long arms (typically $\sim 10^{6}$ km), has
motivated space-based detection in order to access sub-Hertz GW signals [5].

The space-based GW detection missions can be roughly categorized into
heliocentric concepts, featured by LISA [6], and geocentric concepts, featured
by TianQin [7]. For geocentric missions, though the issue of gravity-field
disturbances is expected to be less severe, it may still require careful
assessment due to the relative proximity of the satellites to the Earth and
the Moon. The goal is to ascertain that there can be a proper separation in
the frequencies of gravity-field disturbances and the targeted sensitive
frequency bands near mHz, which is prerequisite to a successful space-based GW
detection. Unlike ground-based detectors, armlengths of space-based detectors
are generally not fixed primarily due to gravitational perturbations to
orbital motion. Varying armlengths also induces Doppler shifts ($\sim 5$ MHz)
in the beat notes of the inter-satellite heterodyne laser interferometric
measurements. The frequency shifts may generate phase ramp-up, onto which GW-
induced phase fluctuations are superimposed. The former, containing gravity-
field information, should be removed or separated with caution in the pre-data
processing stage on the ground. Thereby an accurate simulation of the orbital
Doppler shifts would be needed to verify the phase measurement accuracy and
time-delay interferometry [8], at least for TianQin.

The nominal orbit design for TianQin and other geocentric missions should take
into account and trade off between a variety of factors [9] such as science
objectives, space environment, science payloads, etc. For example, in [10],
the impact of orbit selection on the constellation stability of TianQin has
been studied, and the permitted ranges of the orbital elements that can meet
the payload requirements were identified. In a similar manner, one should also
investigate the impact of orbit selection on gravity-field disturbances so as
to facilitate trade studies in the orbit design and help seek potential
remedies if needed.

In the previous paper [11], we have shown that, with the $1\times 10^{5}$ km
orbital radius and a vertical orbital plane relative to the ecliptic for
TianQin, the Earth-Moon’s gravity-field disturbances are dominating in lower
frequencies, and roll off rapidly at $1\times 10^{-4}$ Hz. The effect does not
enter the detection frequency band of $10^{-4}-1$ Hz where the intersatellite
residual acceleration noise requirement of $\sqrt{2}\times 10^{-15}$
m/s2/Hz1/2 is imposed, and hence presents no showstopper to the mission. For
comparison, the gravity mapping mission GRACE-FO has a much higher roll-off
frequency at $4\times 10^{-2}$ Hz [12], owing to its low altitude of $\sim
500$ km above the ground, at which the science instrument can be sensitive to
the Earth’s spherical harmonic gravity field to $>150$th degree [13]. Thereby,
a separation of $1\times 10^{5}$ km from the Earth is considered large enough
for TianQin to push the gravity-field disturbances out of the detection band.

Nevertheless, these two isolated cases does not tell us much about other
possible options in selecting orbital orientations and radii. It begs the
questions on how the gravity-field effect changes with different orbit
selections, and how one chooses a safe distance from gravity-field noise
sources, particularly, the Earth. In this work, we intend to resolve these
problems by evaluating the effect with respect to various inclinations,
longitudes of ascending nodes, and orbital radii. The results may provide a
useful reference to general orbit and constellation design for geocentric
missions, such as GEOGRAWI/gLISA [14], GADFLI [15], B-DECIGO [16], etc.

This is our sixth paper of the concept study series on TianQin’s orbit and
constellation [17, 10, 11, 18, 19]. For progress on other space environmental
effects on TianQin, one may refer to [20, 21, 22, 23, 24]. This article is
structured as follows. Section II describes the observable and force models
used in the simulations. The results are shown in Section III, which also
include geostationary orbits, and repeat orbits and resonant orbits are
deferred to the Appendices. Finally, we reach the conclusions in Section IV.

##  II Simulation Setup

Following [11], we adopt the range acceleration $\ddot{\rho}$ as the
observable, which is simply the second time derivative of the range $\rho$
between two satellites. The criteria is to compare the amplitude spectral
densities (ASD) of $\ddot{\rho}$ with the residual acceleration noise
requirements between two free-floating test masses, which is generally in the
order of $10^{-15}$ m/s2/Hz1/2 at mHz. To overcome the insufficiency of double
precision, we again use the TQPOP (TianQin Quadruple Precision Orbit
Propagator) program [11] to propagate pure-gravity orbits in quadruple
precision and with a constant step size of 50 seconds for 90 days. Here the
step size is a balanced choice made among the factors of frequency band
coverage, numerical accuracy, and computational run-time [11]. Like TianQin,
all constellations are assumed to be equilateral triangles barring small
deviations, and the three satellites moving in the same nominal circular
orbits with 120∘ phase separation (see, e.g., Table 1). The spectral results
are not sensitive to the specific year chosen. Without loss of generality, the
initial epoch assumes 06 Jun. 2004, 00:00:00 UTC as in [11].

Table 1:  The nominal orbital elements of the TianQin constellation in the J2000-based Earth-centered ecliptic (EarthMJ2000Ec) coordinate system . Here $a$ denotes the semimajor axis, $e$ the eccentricity, $i$ the inclination, $\Omega$ the longitude of ascending node, $\omega$ the argument of periapsis, and $\Delta\nu$ the difference in the true anomaly. | $a$ | $e$ | $i$ | $\Omega$ | $\omega$ | $\Delta\nu$
---|---|---|---|---|---|---
SC1,2,3 |  $1\times 10^{5}$ km | $0^{\circ}$ | $94.7^{\circ}$ | $210.4^{\circ}$ | $0^{\circ}$ | $120^{\circ}$

The detailed force models implemented in TQPOP are summarized in Table 2.
Further description of them can be found in [11]. In particular, the program
offers multiple options in modeling solar system ephemeris, Earth’s precession
and nutation, Earth’s static gravity field, etc. The flexibility helps cross-
checking model errors, which have been shown not to alter the spectral results
for TianQin [11]. In the following simulations, the more recent options will
be used (except for the ocean tides). A new addition to TQPOP lately includes
two ocean tide models FES2014b [25] and EOT11a [26]. Based on the results of
the test runs, all three models generate spectra consistent with one another
in low frequencies. However, since FES2014b contains more high-frequency tidal
components (e.g. M6, M8), we will use FES2014b in the following simulation.

Table 2:  The list of force models implemented in TQPOP. If multiple options are given, the first one is used in the simulations.  Models | Specifications
---|---
Solar system ephemeris | JPL DE430 [27]
| JPL DE405 [28]
Earth’s precession & nutation | IAU 2006/2000A [29]
| IAU 1976/1980 [30]
Earth’s polar motion | EOP 14 C04 [31]
Earth’s static gravity field | EGM2008 ($n=12$) [32]
| EGM96 ($n=12$) [33]
Solid Earth tides | IERS (2010) [29]
Ocean tides | FES2004 ($n=10$) [34]
| FES2014b ($n=10$) [25]
| EOT11a ($n=10$) [26]
Solid Earth pole tide | IERS (2010) [29]
Ocean pole tide | Desai (2003) [29]
Atmospheric tides | Biancale & Bode (2003) [35]
Moon’s libration | JPL DE430 [27]
| JPL DE405 [28]
Moon’s static gravity field | GL0660B ($n=7$) [36]
| LP165P ($n=7$) [37]
Sun’s orientation | IAU [38], Table 1
Sun’s $J_{2}$ | IAU [38], Table 1
relativistic effect | post-Newtonian [30]

##  III Simulation Results

We can assess the impact of orbit selection on the gravity-field effect from
the changes of the amplitudes $A_{kl}^{\pm}$ and frequencies $f_{kl}^{\pm}$ of
spectral peaks in the range acceleration ASD curve. The frequencies
$f_{kl}^{\pm}$ can be generally expressed as

$\displaystyle f_{kl}^{\pm}=k\cdot f_{orb}\pm l\cdot f_{gra},$ (1)

where $f_{orb}$ is the satellites’ orbital frequency, and $f_{gra}$ is the
intrinsic frequencies of gravity-field disturbances, and $k$, $l$ are
integers.

The goal is to find how the gravity-field effect and the roll-off frequency
vary with orbital elements. Here the roll-off frequency refers to where the
range acceleration ASD curve intersects the acceleration noise requirement.

The study cases involve three orbital elements, i.e., the inclination $i$, the
longitude of ascending node $\Omega$, and the semimajor axis $a$. Starting
from the nominal orbit of TianQin given in Table 1, we vary one single element
at a time to make the trends more prominent. Besides, we discuss the
particular case of geostationary orbits in Section III.3.

###  III.1 Impact of orbital orientations

Orbital inclinations $i$ and longitudes of ascending nodes $\Omega$ determine
orbital orientations. For single-variable studies, we vary $i$ and $\Omega$
separately in the EarthMJ2000Ec coordinate system.

However, due to the existence of obliquity of the ecliptic, the influence of
the Earth’s static gravity field and its variation with orbital inclinations
cannot be fully demonstrated in the EarthMJ2000Ec coordinate system by
numerical method. Fortunately, from the previous work [11] (Sec. IVB), we have
learned that the total gravity-field effect in terms of the range-acceleration
ASD can be roughly divided into two primary parts. The high-frequency part
(above $\sim 5\times 10^{-5}$ Hz) is dominated by the non-spherical gravity
field of the Earth, and the low-frequency part (below $\sim 5\times 10^{-5}$
Hz) is dominated by the Sun and the Moon. Therefore, based on symmetry
considerations, we discuss the impact of the Earth’s static gravity field with
orbital orientations varying in the geocentric equatorial (EarthMJ2000Eq)
coordinate system to supplement the effects of all the gravity-field
disturbances in the EarthMJ2000Ec coordinate system. Here we mention that
there is a $5^{\circ}$ difference between the Moon’s orbital plane and the
ecliptic plane, but it does not contribute significantly to the results.

Table 3:  The orbital elements with different inclinations given in the EarthMJ2000Ec coordinate system at the epoch 06 Jun. 2004, 00:00:00 UTC.  $a$ | $i$ | $\Omega$
---|---|---
$1\times 10^{5}$ km | 0∘, 23.5∘, 45∘, 90∘, 135∘, 180∘ | 210.4∘
Table 4:  The orbital elements with different longitudes of ascending node given in the EarthMJ2000Ec coordinate system at the epoch 06 Jun. 2004, 00:00:00 UTC.  $a$ | $i$ | $\Omega$
---|---|---
$1\times 10^{5}$ km | 94.7∘ | 30.4∘, 120.4∘, 210.4∘, 300.4∘

In simulation, setting the orbital radius to a fixed value of $1\times 10^{5}$
km, we vary the inclination from $i=0^{\circ}$ to $180^{\circ}$ by $\Delta
i=45^{\circ}$ intervals (see Table 3), where $i=23.5^{\circ}$ corresponds to
the orbit near the equatorial plane. Then we take four distinct values of
longitudes of ascending nodes $\Omega$ from $30.4^{\circ}$ to $300.4^{\circ}$
by $\Delta\Omega=90^{\circ}$ intervals (Table 4) in the EarthMJ2000Ec
coordinate system. Besides, to supplement the above cases, we vary the
inclination from $i_{eq}=0^{\circ}$ to $180^{\circ}$ by $\Delta
i_{eq}=30^{\circ}$ intervals (see Table 5) in the EarthMJ2000Eq coordinate
system. Due to rotational symmetry, the variation of the Earth’s static
gravity-field effect with different $\Omega_{eq}$ is insignificant, which is
also confirmed by our simulations. A few observations can be summarized as
follows.

First, the low-frequency peaks of Fig. 1 show that $A_{kl}^{-}$($l\neq 0$)
falls and $A_{kl}^{+}$($l\neq 0$) rises as $i$ increases, which, for example,
can be read from Fig. 2. Moreover, the peaks $k=1$, $l=0$ corresponding to the
orbital frequencies coincide with different $i$’s in the plot. The difference
in the amplitudes $A_{10}$ has been discussed in [10].

Figure 1:  The range acceleration ASD between SC1 and SC2 calculated with the orbital parameters in Table 3, for 90 days with a constant step size of 50 seconds.  Table 5:  The orbital elements with different inclinations given in the EarthMJ2000Eq coordinate system at the epoch 06 Jun. 2004, 00:00:00 UTC, assuming circular orbits and $\Delta\nu=120^{\circ}$ for three satellites. Note that $i=94.7^{\circ}$, $\Omega=210.4^{\circ}$ (cf. Table 1) correspond to $i_{eq}=74.5^{\circ}$, $\Omega_{eq}=211.6^{\circ}$.  $a$ | $i_{eq}$ | $\Omega_{eq}$
---|---|---
$1\times 10^{5}$ km | 0∘, 30∘, 60∘, 90∘, 120∘, 150∘, 180∘ | 211.6∘
Figure 2:  Examples of low-frequency peaks of Fig. 1 and Eq. 1, where
$f_{22}^{\pm}=2f_{orb}\pm 2f_{moon}$. The plot shows that the amplitude
$A_{22}^{-}$ falls and $A_{22}^{+}$ rises as the inclination increases in
geocentric ecliptic coordinate system.

Figure 3:  The range acceleration ASD of the Earth’s static gravity field
between SC1 and SC2 calculated with the orbital parameters in Table 5, for 90
days with a constant step size of 50 seconds.

Second, for high-frequency components, Fig. 1 shows the ASD curves extend to
higher frequencies monotonically as $i$ increases from $45^{\circ}$. Besides,
Fig. 3 demonstrates that the ASD curves of the Earth’s static gravity-field
effect extend to higher frequencies as $i_{eq}$ increases. When $0^{\circ}\leq
i\leq 45^{\circ}$ in the EarthMJ2000Ec coordinate system and $i_{eq}\geq
150^{\circ}$ in the EarthMJ2000Eq coordinate system, the roll-off frequency is
insensitive to changes in inclinations, and hence it does not affect the range
of the detection frequency band significantly.

Third, when the satellites are running on the equatorial plane
($i_{eq}=0^{\circ}$ and $180^{\circ}$), Fig. 3 also shows the curves’ trough
values are obviously lower than other cases. This may be attributed to that
the satellites experience less “raggedness” of the Earth’s gravity field when
orbiting in the equatorial plane (see also Fig. 7).

Fourth, compared with the case of orbital inclinations, the gravity-field
effect is less sensitive to changes of the longitude of ascending node
$\Omega$, as shown in Fig. 4. In high frequencies, the differences come from
that the Earth’s static gravity field varies with $\Omega$ in the
EarthMJ2000Ec coordinate system because of the obliquity of the ecliptic. In
lower frequencies, the differences are partly due to the $5^{\circ}$ mismatch
between the Moon’s orbital plane and the ecliptic plane. The situation is
somewhat similar to that of constellation stability [10], which is related to
an approximate rotational symmetry about the ecliptic pole.

Figure 4:  The range acceleration ASD between SC1 and SC2 calculated with the
orbital parameters in Table 4, for 90 days with a constant step size of 50
seconds.

###  III.2 Impact of orbital radii

Based on TianQin, we examine the orbital radius ranging from $4\times 10^{4}$
km to $1.2\times 10^{5}$ km by $\Delta a=2\times 10^{4}$ km intervals. For
special cases of repeat orbits and resonant orbits, see the appendices. The
orbital elements are given in Table 6.

Table 6:  The orbital elements with different orbital radii given in the EarthMJ2000Ec coordinate system at the epoch 06 Jun. 2004, 00:00:00 UTC.  $a/10^{4}$ km | $i/^{\circ}$ | $\Omega/^{\circ}$
---|---|---
4, 6, 8, 10, 12 | 94.7 | 210.4

It should be noted that simulations with smaller orbital radii require higher
degrees and orders in the Earth’s gravity-field model. With estimated
gravitational acceleration from different degrees, it is sufficient to set
$n=16$ for $a=4\times 10^{4}$ km, $n=13$ for $a=6\times 10^{4}$ km, and $n=12$
for $a\geq 8\times 10^{4}$ km [11], so as to meet an accuracy below $10^{-16}$
m/s2/Hz1/2. The corresponding results of the range acceleration ASD are
presented in Fig. 5.

Figure 5:  The range acceleration ASD with different orbital radii from Table
6 and for 90 days with a constant step size of 50 seconds.

First, with the orbital radius increasing, the ASD curves appear to shift to
low frequencies overall. Moreover, one can read out the roll-off frequencies
from Fig. 5, and the values are listed in Table 7 and shown in Fig. 6.

Figure 6:  The relation between roll-off frequency and orbital radius of Fig.
5

Second, one can fit the ASD curves of the range acceleration near the roll-off
frequencies $f_{r}$ by

$\displaystyle\widetilde{RA}$ $\displaystyle=$ $\displaystyle\sqrt{2}\times
10^{-15}\times\left(\frac{f_{r}}{f}\right)^{E_{f}}{\rm m/s^{2}/Hz^{1/2}},$ (2)

where $f$ is the Fourier frequency, and $E_{f}$ the power index of the ratio.
The fitted values of $E_{f}$ are given in Table 7. We mention that $E_{f}$
roughly changes with $a^{2}$ and $f_{r}$ with $1/a^{2}$.

Table 7: The numerical values of the roll-off frequency $f_{r}$ and the index $E_{f}$. $a$/km |  $f_{r}$/$10^{-4}$ Hz | $E_{f}$
---|---|---
40000 | 2.30 | 21.0
60000 | 1.46 | 22.2
80000 | 1.20 | 23.8
100000 | 1.00 | 26.0
120000 | 0.93 | 28.6

###  III.3 Geostationary orbits for gLISA/GADFLI

In this subsection, we examine geostationary orbits proposed by gLISA/GADFLI.
The initial orbital elements are given in Table 8, which have been optimized
for 90 days to stabilize the constellation. Here to focus on the gravity-field
effect, we ignore station-keeping maneuvers that may be needed roughly every
two weeks [14]. The corresponding range acceleration ASD is presented in Fig.
7, which incorporates the cases with and without the Earth’s non-spherical
gravity field in the simulation.

Compared with Fig. 5, the plot shows a rather distinct feature in that the
number of peaks is significantly reduced. This is due to the fact that by
placing the satellites at the same location above the Earth’s surface, the
Earth’s gravity experienced by the satellites becomes localized and nearly
static. Therefore, the range acceleration ASD is instead dominated by the Moon
and the Sun’s gravity, with the high-frequency components from the Earth’s
non-spherical gravity field being suppressed. This is indicated by the
overlapping of the blue and green curves in Fig. 7, where the latter is
calculated without the Earth’s non-spherical gravity field in orbit
propagation. Moreover, the ASD curve intersects with the noise requirement
$3\sqrt{2}\times 10^{-15}$ m/s2/Hz1/2 at $1.69\times 10^{-4}$ Hz, which is
much lower than the proposed detection frequency band $10^{-2}-1$ Hz [14].

Table 8:  The initial orbital elements of gLISA given in the EarthMJ2000Eq coordinate system at the epoch 06 Jun. 2004, 00:00:00 UTC.  Satellite | $a$ | $e$ | $i$ | $\Omega$ | $\omega$ | $\nu^{ini}$
---|---|---|---|---|---|---
SC1 | 42158.65 km |  |  |  |  | $210.0^{\circ}$
SC2 | 42164.90 km | $0^{\circ}$ | $0^{\circ}$ | $0^{\circ}$ | $0^{\circ}$ | $330.8^{\circ}$
SC3 | 42171.00 km |  |  |  |  | $\phantom{0}91.3^{\circ}$
Figure 7:  The range acceleration ASD between two satellites of gLISA/GADFLI,
using the orbital parameters in Table 8, and calculated for 90 days with a
constant step size of 50 seconds.

##  IV Conclusion and Discussion

In this paper, we have simulated and analyzed the gravity-field effect of the
Earth-Moon system on inter-satellite range acceleration of geocentric
detectors in circular orbits with various orbital orientations and radii. It
can help fill the blank in the concept studies of geocentric missions (e.g.,
[39]) that might have been overlooked. As the previous work has shown [11],
one common behavior is that the “orbital noise” dominates in the neighborhood
of the orbital frequencies (e.g., $3\times 10^{-6}$ Hz for TianQin), and then
falls off rapidly below residual acceleration noise requirements ($\sim
10^{-15}$ m/s2/Hz1/2) at specific higher frequencies. Besides, three main
conclusions can be drawn here.

First, for a fixed orbital radius $1\times 10^{5}$ km and the longitude of
ascending node $210.4^{\circ}$, the roll-off frequency extends higher
frequencies with the inclination increasing when the orbital inclination is
greater than $45^{\circ}$ in the EarthMJ2000Ec coordinate system. When the
inclination is between $0^{\circ}$ and $45^{\circ}$ in the EarthMJ2000Ec
coordinate system or above $150^{\circ}$ in the EarthMJ2000Eq coordinate
system, the variation of roll-off frequency would not affect the choice of the
detection frequency band significantly. For a given inclination, altering the
longitude of ascending node has only a small impact.

Second, as the orbital radius increases, the gravity-field effect shifts to
lower frequencies so that the detection frequency band can be extended
accordingly (cf. Fig. 5).

Third, the gravity-field effect for gLISA in geostationary orbits intersects
with the acceleration noise requirement $3\sqrt{2}\times 10^{-15}$ m/s2/Hz1/2
at $1.69\times 10^{-4}$ Hz. It allows for sufficient clearance from the
targeted detection band of $10^{-2}-1$ Hz [14], and hence presents no
showstopper to the mission.

It should be pointed out that the results appear to favor small inclinations
(see Section III.1). However, for TianQin, the inclination selection is
largely driven by other more pressing technical demands. These include
sciencecraft thermal control [40, 20] and the need of lowering eclipse
occurrence [18], etc., which instead favor a $90^{\circ}$ inclination. With a
vertical orbital plane relative to the ecliptic, setting the radius equal to
or greater than $1\times 10^{5}$ km is sufficient for TianQin to meet the
requirement of extending the detection band to as low as $1\times 10^{-4}$ Hz.
Meanwhile, the upper limit of the orbital radius is likely to be set by the
constellation stability requirement [10]. For future work, it would be
desirable to develop an analytic model of the orbital dynamics to account for
the range acceleration ASD at greater depths.

###### Acknowledgements.

The authors thank Yuzhou Fang, Lei Jiao, Bobing Ye, Jun Luo, and the anonymous
referee for helpful discussions and comments. The work is supported by the
National Key R&D Program of China (No. 2020YFC2201202). X.Z. is supported by
NSFC Grant No. 11805287.

*

##  Appendix A A. Repeat orbits

A repeat orbit has the property that the satellite covers the same ground
track periodically. Such orbits may have certain engineering benefits. With
the orbital plane facing J0806, we set the orbital periods from one day to
four days, and give the initial orbital radii in Table 9. The corresponding
results of the range acceleration ASD are presented in Fig. 8. Notably, the
ASD of the geosynchronous orbit ($T=1$ day) bears great resemblance to Fig. 7.

Table 9: The initial optimized orbital elements of different repeat orbits for the detectors in the EarthMJ2000Ec coordinate system at the epoch 06 Jun. 2004, 00:00:00 UTC. $T$/day | SC1/km | SC2/km | SC3/km
---|---|---|---
1 | 42164.14 | 42162.22 | 42163.60
2 | 66934.50 | 66936.30 | 66932.00
3 | 87714.00 | 87719.70 | 87710.00
4 | 106265.00 | 106278.50 | 106261.00
Figure 8: The range acceleration ASD of different repeat orbital radii between
SC1 and SC2, using orbital parameters in Table 9, for 90 days with a constant
step size of 50 seconds.

##  Appendix B B. Resonant orbits

Although for TianQin, the work of [10] has ruled out the orbits resonant with
the Moon due to their inferior constellation stability performance. However,
the results are still included for the sake of completeness.

Table 10: The initial orbital elements of different orbital resonance for the detectors in the EarthMJ2000Ec coordinate system at the epoch 06 Jun. 2004, 00:00:00 UTC. $T_{moon}$/$T_{orb}$ | SC1/km | SC2/km | SC3/km
---|---|---|---
8 | 95796.617 | 95805.000 | 95791.000
7 | 104715.621 | 104728.000 | 104710.500
6 | 116049.338 | 116070.000 | 116040.000
Figure 9:  The range acceleration ASD of different orbital resonance altitude
between SC1 and SC2, using the orbital parameters in Table 10, for 90 days
with a constant step size of 50 seconds.

We consider orbital period ratios $T_{moon}/T_{orb}=6,7,8$. Table 10 shows the
corresponding orbital radii, and the other orbital elements are the same as
Table 6. The results are presented in Fig. 9. Compared with non-resonant
orbits, the peaks corresponding to the Earth-Moon coupling are enhanced. On
the whole, the plot show a similar trend as Fig. 5.

## References

  * Abramovici  _et al._ [1992] A. Abramovici _et al._ , LIGO: the laser interferometer gravitational-wave observatory, Science 256(5055), 325–333 (1992).
  * Caron  _et al._ [1997] B. Caron _et al._ , The VIRGO interferometer for gravitational wave detection, Nucl. Phys. B, Proc. Suppl. 54, 167 (1997).
  * Somiya [2012] K. Somiya, Detector configuration of KAGRA - the japanese cryogenic gravitational-wave detector, Class. Quantum Gravity 29, 124007 (2012).
  * Harms [2019] J. Harms, Terrestrial gravity fluctuations, Living Rev. Relativ. 22, 6 (2019).
  * Hild [2012] S. Hild, Beyond the second generation of laser interferometric gravitational wave observatories, Class. Quantum Grav. 29, 124006 (2012).
  * Folkner  _et al._ [1997] W. M. Folkner, F. Hechler, T. H. Sweetser, M. A. Vincent, and P. L. Bender, LISA orbit selection and stability, Class. Quantum Grav. 14, 1405 (1997).
  * Luo  _et al._ [2016] J. Luo _et al._ , TianQin: a space-borne gravitational wave detector, Class. Quantum Grav. 33, 035010 (2016).
  * Tinto and Dhurandhar [2021] M. Tinto and S. V. Dhurandhar, Time-delay interferometry, Living Rev Relativ 24, 1 (2021).
  * Zhang  _et al._ [2021a] X. Zhang, B. Ye, Z. Tan, H. Yuan, C. Luo, L. Jiao, D. Gu, Y. Ding, and J. Mei, Orbit and constellation design for TianQin: progress review, Acta Scientiarum Naturalium Universitatis Sunyatseni 60(1-2), 123 (2021a).
  * Tan  _et al._ [2020] Z. Tan, B. Ye, and X. Zhang, Impact of orbital orientations and radii on TianQin constellation stability, Int. J. Mod. Phys. D 29, 2050056 (2020).
  * Zhang  _et al._ [2021b] X. Zhang, C. Luo, L. Jiao, B. Ye, H. Yuan, L. Cai, D. Gu, J. Mei, and J. Luo, Effect of Earth-Moon’s gravity on TianQin’s range acceleration noise, Phys. Rev. D 103, 062001 (2021b).
  * Abich  _et al._ [2019] K. Abich _et al._ , In-orbit performance of the GRACE Follow-on laser ranging interferometer, Phys. Rev. Lett. 123, 031101 (2019).
  * Kornfeld  _et al._ [2019] R. P. Kornfeld, B. W. Arnold, M. A. Gross, N. T. Dahya, W. M. Klipstein, P. F. Gath, and S. Bettadpur, Grace-fo: The gravity recovery and climate experiment follow-on mission, Journal of Spacecraft and Rockets 56, 931 (2019), https://doi.org/10.2514/1.A34326 .
  * Tinto  _et al._ [2015] M. Tinto, D. DeBra, S. Buchman, and S. Tilley, gLISA: geosynchronous laser interferometer space antenna concepts with off-the-shelf satellites, Rev. Sci. Instrum. 86, 014501 (2015).
  * McWilliams [2011] S. T. McWilliams, Geostationary Antenna for Disturbance-Free Laser Interferometry (GADFLI), arXiv:1111.3708 [astro-ph.IM] (2011).
  * Kawamura  _et al._ [2018] S. Kawamura _et al._ , Space gravitational-wave antennas DECIGO and B-DECIGO, Int. J. Mod. Phys. D 27, 1845001 (2018).
  * Ye  _et al._ [2019] B. Ye, X. Zhang, M. Zhou, Y. Wang, H. Yuan, D. Gu, Y. Ding, J. Zhang, J. Mei, and J. Luo, Optimizing orbits for TianQin, Int. J. Mod. Phys. D 28, 1950121 (2019).
  * Ye  _et al._ [2021] B. Ye, X. Zhang, Y. Ding, and Y. Meng, Eclipse avoidance in TianQin orbit selection, Phys. Rev. D 103, 042007 (2021).
  * Zhou  _et al._ [2021] M. Zhou, X. Hu, B. Ye, S. Hu, D. Zhu, X. Zhang, W. Su, , and Y. Wang, Orbital effects on time delay interferometry for TianQin, Phys. Rev. D 103, 103026 (2021).
  * Chen  _et al._ [2021] H. Chen, C. Ling, X. Zhang, X. Zhao, M. Li, and Y. Ding, Thermal environment analysis for TianQin, Class. Quantum Grav. 38, 155015 (2021).
  * Lu  _et al._ [2021] L.-F. Lu, W. Su, X. Zhang, Z.-G. He, H.-Z. Duan, Y.-Z. Jiang, and H.-C. Yeh, Effects of the Space Plasma Density Oscillation on the Interspacecraft Laser Ranging for TianQin Gravitational Wave Observatory, JGR: Space Physics 126, e2020JA028579 (2021).
  * Su  _et al._ [2021] W. Su, Y. Wang, C. Zhou, L. Lu, Z.-B. Zhou, T. Li, T. Shi, X.-C. Hu, M.-Y. Zhou, M. Wang, H.-C. Yeh, H. Wang, and P. Chen, Analyses of laser propagation noises for TianQin gravitational wave observatory based on the global magnetosphere MHD simulations, ApJ 914, 139 (2021).
  * LU  _et al._ [2020] L. LU, Y. LIU, H. DUAN, Y. JIANG, and H.-C. YEH, Numerical simulations of the wavefront distortion of inter-spacecraft laser beams caused by solar wind and magnetospheric plasmas, Plasma Sci. Technol. 22, 115301 (2020).
  * Su  _et al._ [2020] W. Su, Y. Wang, Z.-B. Zhou, Y.-Z. Bai, Y. Guo, C. Zhou, T. Lee, M. Wang, M.-Y. Zhou, T. Shi, H. Yin, and B.-T. Zhang, Analyses of residual accelerations for TianQin based on the global MHD simulation, Class. Quant. Grav. 37, 185017 (2020).
  * [25] https://grace.obs-mip.fr/dealiasing_and_tides/ocean-tides/ .
  * Savcenko and Bosch [2012] R. Savcenko and W. Bosch,  _EOT11a - empirical ocean tide model from multi-mission satellite altimetry_ , Tech. Rep. (Deutsches Geodätisches Forschungsinstitut (DGFI), 2012).
  * Folkner  _et al._ [2014] W. M. Folkner, J. G. Williams, D. H. Boggs, R. S. Park, and P. Kuchynka,  _The planetary and lunar ephemerides DE430 and DE431_ , IPN Progress Report 42-196 (Jet Propulsion Laboratory, 2014).
  * Standish [1998] E. Standish,  _JPL Planetary and Lunar Ephemerides, DE405/LE405_ , Tech. Rep. 312.F-98-048 (JPL IOM, 1998).
  * Petit and Luzum [2010] G. Petit and B. Luzum,  _IERS Conventions (2010)_ , Technical Report 36 (BUREAU INTERNATIONAL DES POIDS ET MESURES SEVRES (FRANCE), 2010).
  * McCarthy [1996] D. D. McCarthy,  _IERS Conventions (1996)_ , Technical Report 21 (Central Bureau of IERS - Observatoire de Paris, 1996).
  * [31] http://hpiers.obspm.fr/iers/eop/eopc04/ .
  * Pavlis  _et al._ [2012] N. K. Pavlis, S. A. Holmes, S. C. Kenyon, and J. K. Factor, The development and evaluation of the Earth Gravitational Model 2008 (EGM2008), J. Geophys. Res. 117, B04406 (2012).
  * Lemoine  _et al._ [1998] F. G. Lemoine, S. C. Kenyon, J. K. Factor, R. Trimmer, N. K. Pavlis, D. S. Chinn, C. M. Cox, S. M. Klosko, S. B. Luthcke, M. H. Torrence, Y. M. Wang, R. G. Williamson, E. C. Pavlis, R. H. Rapp, and T. R. Olson,  _The Development of the Joint NASA GSFC and NIMA Geopotential Model EGM96_ , Tech. Rep. (NASA Goddard Space Flight Center, 1998).
  * Lyard  _et al._ [2006] F. Lyard, F. Lefevre, T. Letellier, and O. Francis, Modelling the global ocean tides: modern insights from FES2004, Ocean Dyn. 56, 394 (2006).
  * Biancale and Bode [2006] R. Biancale and A. Bode,  _Mean annual and seasonal atmospheric tide models based on 3-hourly and 6-hourly ECMWF surface pressure data_ , STR 06/01 (GeoForschungsZentrum Potsdam, 2006).
  * Konopliv  _et al._ [2013] A. S. Konopliv, R. S. Park, D.-N. Yuan, S. W. Asmar, M. M. Watkins, J. G. Williams, E. Fahnestock, G. Kruizinga, M. Paik, D. Strekalov, N. Harvey, D. E. Smith, and M. T. Zuber, The JPL lunar gravity field to spherical harmonic degree 660 from the GRAIL primary mission, J. Geophys. Res. Planets 118, 1 (2013).
  * Konopliv  _et al._ [2001] A. Konopliv, S. Asmar, E. Carranza, W. Sjogren, and D. Yuan, Recent gravity models as a result of the lunar prospector mission, Icarus 150, 1 (2001).
  * Archinal  _et al._ [2011] B. A. Archinal, M. F. A’Hearn, E. Bowell, A. Conrad, G. J. Consolmagno, R. Courtin, T. Fukushima, D. Hestroffer, J. L. Hilton, G. A. Krasinsky, G. Neumann, J. Oberst, P. K. Seidelmann, P. Stooke, D. J. Tholen, P. C. Thomas, and I. P. Williams, Report of the IAU working group on cartographic coordinates and rotational elements: 2009, Celest. Mech. Dyn. Astr. 109, 101 (2011).
  * Anderson  _et al._ [2012] K. Anderson, R. Stebbins, R. Weiss, and E. Wright,  _NASA Gravitational-wave mission concept study final report_ (2012).
  * Zhang  _et al._ [2018] X. Zhang, H. Li, and J. Mei, Thermal stability estimation of TianQin satellites based on LISA-like thermal design concept (2018), internal technical report (in Chinese).


#  Concept Matching: Clustering-based Federated Continual Learning

Xiaopeng Jiang (xj8@njit.edu), Cristian Borcea<EMAIL_ADDRESS>
Department of Computer Science, New Jersey Institute of Technology, Newark,
NJ, USA

###### Abstract

Federated Continual Learning (FCL) has emerged as a promising paradigm that
combines Federated Learning (FL) and Continual Learning (CL). To achieve good
model accuracy, FCL needs to tackle catastrophic forgetting due to concept
drift over time in CL, and to overcome the potential interference among
clients in FL. We propose Concept Matching (CM), a clustering-based framework
for FCL to address these challenges. The CM framework groups the client models
into concept model clusters, and then builds different global models to
capture different concepts in FL over time. In each round, the server sends
the global concept models to the clients. To avoid catastrophic forgetting,
each client selects the concept model best-matching the concept of the current
data for further fine-tuning. To avoid interference among client models with
different concepts, the server clusters the models representing the same
concept, aggregates the model weights in each cluster, and updates the global
concept model with the cluster model of the same concept. Since the server
does not know the concepts captured by the aggregated cluster models, we
propose a novel server concept matching algorithm that effectively updates a
global concept model with a matching cluster model. The CM framework provides
flexibility to use different clustering, aggregation, and concept matching
algorithms. The evaluation demonstrates that CM outperforms state-of-the-art
systems and scales well with the number of clients and the model size.

##  1 Introduction

As a privacy-preserving deep learning (DL) paradigm, Federated Learning (FL)
(McMahan et al., 2017) attracts significant interest. However, most of the
current FL research assumes the data have been collected before training, and
the data at clients do not change over the training rounds. This is not
necessarily the case in many applications, especially on IoT and mobile
devices where it is difficult to train with the entire dataset on-device at
every round due to their resource constraints. The data not only accumulate
over time, but also change their distributions. The data distribution change,
also referred to as concept drift, makes DL models obsolete over time. For
example, a user sleep quality prediction model trained on the data collected
during their routine life will not work well when the users experience changes
in their sleep patterns due to stress, illness, or travel. This effect of
dynamic change in data is being actively studied by the Continual Learning
(CL) community in centralized settings. However, CL research in FL settings is
still in its infancy.

Federated Continual Learning (FCL) performs FL under the CL dynamic data
scenarios. There are two main challenges in FCL. One, inherited from CL, is
catastrophic forgetting (French, 1999). Due to concept drift, the model
forgets previously learned knowledge as it learns new information over time.
For example, in human activity recognition (HAR) (Jiang et al., 2022), the
concepts can be the subsets of activities, the locations of the activities, or
the health status of the user. FCL imposes privacy constraints on the top of
CL, which escalates this challenge. Even if the clients are aware of concept
drift (e.g., sedentary vs. active lifestyle in HAR), they may not want to
reveal it to the FL server due to privacy concerns. The second challenge is
that the FL clients with different data concepts may potentially interfere
with each other, because the data in FL is typically non independently or
identically distributed (non-iid). The interference will sabotage the efforts
of clients’ training during aggregation and lead to underperforming global
models. CL amplifies this interference in FL, because the union of the clients
data may also be distributed differently over time.

An efficient FCL framework shall tackle these challenges to achieve good model
performance. So far, no existing system has achieved this goal under realistic
assumptions. While several works (Yoon et al., 2021; Casado et al., 2022; Guo
et al., 2021; Zhang et al., 2023; Qi et al., 2023; Dong et al., 2022; Ma et
al., 2022) have recently targeted FCL, their applicability is limited due to
unrealistic assumptions (i.e., the server knows the concept drift from the
clients or the classes to learn do not change over time), or the interference
among the clients is not handled.

This paper proposes Concept Matching (CM), the first framework for FCL to
alleviate the two challenges and achieve good model performance. Intuitively,
if we can separate the client models based on the concepts of the data, and
train different concept models specifically to learn each concept in the FL
training rounds, catastrophic forgetting and the interference among clients
can be greatly diminished. This process has to be performed under the FL
assumption that the server cannot access any raw data. The CM framework
achieves these goals through clustering and concept matching in FL. At every
CM training round, to avoid interference among the clients, the server
clusters the client models representing the same concept and aggregates them.
To mitigate catastrophic forgetting, different concept models are trained for
each concept through concept matching which occurs differently at the server
and the clients. The server concept matching is to match and update the
concept model of the previous round with an aggregated cluster model. We
propose a novel distance-based concept matching algorithm for the server
concept matching. This algorithm matches an aggregated cluster model with a
concept model close in distance, and aligns them to update the concept model
in the appropriate gradient descent direction. The client concept matching is
to test the concept models from the previous round on the current local data,
and to select the one with the lowest loss as the best match. The CM framework
does not require the clients to have any knowledge about the concepts.
Furthermore, the server does not need any additional information when compared
to vanilla FL (i.e., it only requires the model weights from the clients). The
CM framework provides flexibility to use a variety of clustering, aggregation,
and concept matching algorithms. The framework can evolve as new algorithms
are proposed for different applications and models. We proved theoretically
that during the iterations of gradient descent, the distance between the
current model and the model from the previous round gradually diminishes. Our
server concept matching algorithm applies this theorem and achieves up to 100%
accuracy. Furthermore, we experimentally demonstrated the superior performance
of CM over the baselines (Yoon et al., 2021; French, 1999), the flexibility of
using different algorithms, and its scalability with the number of clients and
the model size.

##  2 CM Overview

CM is an effective learning framework for FCL, where each client encounters a
stream of data with different concepts over time (Figure 1a). A concept infers
a function from training examples of its inputs and outputs (Mitchell, 1997),
such as a set of classes in image classification or an accent in speech
recognition. The clients may or may not be aware of the concepts of the data.
In FCL, it is infeasible for the clients to store all the data (e.g., limited
storage on IoT devices) or the system cannot wait for all data to be
accumulated before the training starts (Lomonaco and Maltoni, 2017). The
system has to consume the data promptly to train one or multiple models
working well for every concept. As regular FL, the server in FCL only receives
and aggregates the model weights from the clients, without accessing any
additional information.

(a) FCL scenario

(b) CM training process overview

Figure 1: CM training for an FCL scenario

There are two learning scenarios (Kim et al., 2022; Maltoni and Lomonaco,
2019) in FCL: class-incremental and task-incremental. Class-incremental
requires the system to learn different classes over time. The changes of the
sets of classes in data causes concept drift, but the system is not given any
additional information to identify the concept drift. Task-incremental focuses
on learning different tasks over time, such as different languages in speech
recognition. This scenario can also consider the tasks to be different sets of
classes. In this case, the difference from class-incremental is that the tasks
(e.g., sedentary activities vs. physical exercises in HAR) are known, and the
concept drift is recognized by the model. In centralized CL, the task IDs can
be used to help both training and inference. However, in FCL, the clients can
utilize the task IDs, but they cannot share them with the server due to
privacy concerns.

The FCL learning scenarios are negatively impacted by catastrophic forgetting
due to concept drift over the training rounds and the interference among the
clients with different concepts of data. CM is the first learning framework to
alleviate these challenges and effectively train accurate models in FCL under
realistic assumptions. The main idea of CM is to separate the client models
representing the same concept, and the server and the clients collaboratively
train different concept models for each concept by matching the concept
concealed in the data and the model. In the CM framework, the clients do not
send any addition information beyond the model weights to the server, and thus
the framework works for both class-incremental and task-incremental scenarios.

Figure 1b illustrates the CM training in one FCL round. In the bootstrapping
phase, the system administrator at the server-side determines the input and
output of the model, the estimated number of concepts in the data, and designs
the neural network. In the initialization phase (orange box), the server
transmits the global concept models to all clients. To avoid catastrophic
forgetting, in the client operation phase (green box), each client utilizes
its data in the current round to perform concept matching and chooses a best-
matching model for further fine-tuning. Subsequently, the updated models are
sent back to the server. To avoid interference among client models with
different concepts, in the server operation phase (blue box), the server
clusters the models with the same concept and aggregates them. Due to the fact
that the union of the clients data in each round may not encompass all
concepts, the number of clusters may not correspond to the number of concepts.
The server does not know the concept captured by an aggregated cluster model.
Thus, the server must perform concept matching to associate the aggregated
cluster models with the global concept models of the previous round, and only
update the relevant concept models for the next round.

The number of concepts is typically a small constant estimated from the
semantics of the application, and the system administrator can select its
value using domain knowledge. For example, a HAR model can use the locations
of the user activities as different concepts, such as home, workplace, park,
etc. Image data, on the other hand, can use the number of categories as the
number of concepts. If an unexpected new concept occurs, by design, the system
will automatically treat the new concept as the most similar existing concept.

The CM framework uses different concept matching algorithms at the server and
the clients. In the server concept matching algorithm, for each global concept
model, the server maintains a distance record between the current model and
the model of the previous round. The server then matches the cluster
aggregated model with a global concept model such that their distance is
smaller than the record. This is to ensure the global concept model will be
updated in the correct gradient descent direction. If there are multiple
qualifying global concept models, the server selects the one closest in
distance to the cluster model. It then updates the global concept model with
the matching cluster model. For the client concept matching, an efficient
algorithm tests the local data with the concept models and chooses the model
with the lowest loss.

##  3 Related Work

Most of the works on generic FL focus on system design (Jiang et al., 2022;
McMahan et al., 2017; Beutel et al., 2020), model performance (Zhang et al.,
2022; Gao et al., 2022; Jiang et al., 2023; Huang et al., 2022), privacy
(Geiping et al., 2020; Wei et al., 2020), and communication and computation
overhead (Jiang and Borcea, 2023; Wu et al., 2020; Luping et al., 2019). This
paper does not focus on these generic FL challenges. Relevant for this paper
are the works (Ouyang et al., 2022; Ghosh et al., 2020) on clustering the
client models to improve the model accuracy. These clustering approaches in FL
assume the number of client groups is a constant throughout the training, and
cannot be applied directly in CL scenarios. All the works mentioned here
assume the training data for the clients do not change over time, which limits
the applicability of FL. Our work, on the other hand, focuses on making FL
work well in the presence of dynamic changes of the concepts in data.

Continual Learning (CL), also known as lifelong learning or incremental
learning, allows an AI model to learn continuously over time from a stream of
data, while avoiding catastrophic forgetting. Recent works addressing CL can
be categorized into three families (De Lange et al., 2021): replay (Rolnick et
al., 2019; Isele and Cosgun, 2018), regularization (Ahn et al., 2019; Aljundi
et al., 2019) and parameter isolation (Mallya and Lazebnik, 2018; Serra et
al., 2018). These techniques do not address additional challenges from FL. In
addition to its distributed nature, FL also introduces privacy restrictions.
For example, FL clients shall not share their task IDs with the server in the
task-incremental CL scenario. In addition, even if the clients can learn new
concepts well without forgetting the previous ones, the aggregation may
sabotage the efforts of the clients when their learning paths diverge due to
non-iid data. This phenomenon has been demonstrated experimentally with image
data in a recent work (Yoon et al., 2021). Expanding CL to FL, our work
adheres to the FL requirement that the server only accesses the client model
weights, and it handles the interference among the clients in FL.

Federated Continual Learning (FCL) is a newly introduced research area that
combines FL and CL. FedWeIT (Yoon et al., 2021) and CFeD (Ma et al., 2022)
only work in the task-incremental scenario, and they requires the server to
know the task IDs from the clients. CDA-FedAvg (Casado et al., 2022) only
considers the context information (i.e. the positions of sensors in HAR) as
the concepts, and it is not proven to work with concept drift caused by
different sets of classes. Other works (Guo et al., 2021; Dong et al., 2022;
Qi et al., 2023; Zhang et al., 2023) do not address the potential interference
among the clients. Unlike these works, our work tackles catastrophic
forgetting and the interference among the clients under more realistic
assumptions, such as the clients do not share any additional information with
the server beyond the model weights, and the classes can change over time.

##  4 CM Framework

This section describes the problem definition, the CM learning framework,
several design choices, and the concept matching algorithms.

###  4.1 Problem Definition

For each client $n\in\\{1,2,...,N\\}$, the data arrives in a streaming fashion
as a (possible infinite) sequence of learning experiences
$\mathcal{S}_{n}={e_{n}^{1},e_{n}^{2},...,e_{n}^{t}}$. Without loss of
generality, each experience $e_{n}^{t}$ consists a batch of samples
$\mathcal{D}_{n}^{t}$, where the $i$-th sample is a tuple
⟨$x_{i},y_{i}$⟩${}_{n}^{t}$ of input and target respectively. Let
$\mathcal{C}=\\{C_{1},C_{2},...,C_{k}\\}$ be the set of $K$ concepts hidden in
entire dataset $\mathcal{D}$. Each concept $C_{k}$ is associated with a
probability distribution $P_{k}(X,Y)$, where $X$ denotes the input space and
$Y$ denotes the label space. A batch of client samples follows one of the
distributions $\mathcal{D}_{n}^{t}\sim P_{k}(X,Y)$, which may or may not be
explicitly known by the client.

The goal of the CM framework is to learn a set of models
$\\{w_{k}\\}_{k=1}^{K}$, and each model $w_{k}$ can perform well for its
corresponding concept $C_{k}$. The problem can be formulated as the Eq. 1,
where $L$ is the loss function, $\mathcal{D}_{n}$ is the entire stream of data
on client $n$.

$\arg\min_{\\{w_{k}\\}_{k=1}^{K}}\sum_{k=1}^{K}\sum_{n=1}^{N}L(w_{k},\mathcal{D}_{n})$
(1)

###  4.2 Learning Framework for FCL

To begin the learning process, the system administrator at the server side
determines the number of concepts $K$ and designs the model. The server
initializes the weights of $K$ global concept models randomly and sends them
to the clients. In every round, each client $n$ receives the weights of global
concept models $W^{t-1}=(w_{1},w_{2},...,w_{K})^{t-1}$ from the server. To
avoid catastrophic forgetting caused by training a model with the data of
different concepts, the clients perform concept matching with the local data
of current round to select the best-matching global concept model as Eq. 2.

$k^{*}=ClientConceptMatch(W^{t-1},\mathcal{D}_{n}^{t})$ (2)

Next, the client fine-tunes the best-matching global concept model weights
$w_{k^{*}}$ with the local data, and produces a new local model with weights
$\theta^{t}_{n}$ and sends it the server. After receiving the client models
with weights $\\{\theta^{t}_{n}\\}_{n=1}^{N}$, to separate the clients models
with the different concepts, the server clusters the client models into a set
of clusters of size $J$, denoted as $\Omega^{t}$ (Eq. 3). Since the union of
clients data per round may not cover all concepts, $J$ and $K$ are usually
different.

$\displaystyle\Omega^{t}=Cluster(\\{\theta^{t}_{n}\\}_{n=1}^{N})$ (3)
$\displaystyle\Theta^{t}=Aggregate(\Omega^{t})$ (4)

Then, the server produces aggregated cluster models with weights
$\Theta^{t}=(w_{1},w_{2},...,w_{J})^{t}$ (Eq. 4). The server does not know the
concept in a aggregated cluster model or which global concept model to update.
Therefore, it need match the aggregated cluster models $\Theta^{t}$ with the
global concept models $W^{t-1}$, and only update the global concept models
with data encountered in this round as Eq. 5.

$W^{t}=ServerConceptMatch(\Theta^{t},W^{t-1})$ (5)

The pseudo-code of the CM framework is described in Appendix A.

###  4.3 Design Discussion

The CM framework provides the flexibility to use different clustering,
aggregation, and concept matching algorithms. It can evolve, as new algorithms
are proposed for different applications and models. The aggregation algorithms
in FL are orthogonal to the CM framework, and any of them can be employed to
further mitigate non-iid. We evaluate the CM framework with multiple
clustering algorithms in section 5.2, and demonstrates that CM works well with
classic clustering algorithms, such as kmean, agglomerative, and BIRCH. Some
clustering algorithms, such as DBSCAN (Ester et al., 1996) and OPTICS (Ankerst
et al., 1999), do not require the number of clusters to be known. As a future
work, the CM framework may be extended to work in such a scenario. In
addition, dimension reduction techniques (Carreira-Perpinán, 1997) can also be
applied in conjunction with clustering algorithms to mitigate the curse of
dimensionality.

The CM framework is designed to be compatible with both task-incremental and
class-incremental scenarios, because the clients do not need to possess any
prior understanding of concept drift in data, including task IDs. In the task-
incremental scenario where the clients know the task IDs and the concept drift
due to the transition of different tasks, the clients do not have to perform
concept matching every round. The clients can maintain a mapping between the
global concept model ID and the task ID. After the client encounters all
tasks, it can just use the mapping to match the data of a task with the global
concept model. In the class-incremental scenario, the clients have to perform
concept matching every round.

In terms of privacy, the CM framework is the same as vanilla FL (i.e., the
clients send only their model weights to the server). In terms of
communication, the clients send a single model trained with the local data in
the same way as vanilla FL, but the server sends multiple concept models to
the clients. The number of concepts is usually a small constant under the
control of the system administrator, and the concept models designed in CM can
be smaller than the single model in vanilla FL, because each model only learns
a single concept. Nevertheless, it is essential to balance the trade-off
between model performance and communication overhead, by taking into account
the available system resources. Thus, to further improve privacy protection
and communication efficiency, CM can use existing privacy protection
(Mothukuri et al., 2021) or communication reduction (Deng et al., 2020)
techniques for FL.

###  4.4 Concept Matching Algorithms

The concept matching algorithms at the client and the server collaboratively
and iteratively update each global concept model with the information learnt
from the data of a matching concept, and achieve up to 100% accuracy. Two
algorithms are connected through client training, and server clustering and
aggregation. The main novelty of this distributed approach lies in the server
concept matching algorithm, which ensures the model updates in the correct
gradient descent direction.

Server Concept Matching. After clustering, the groups of the client models
fine-tuned with the data of different concepts are unordered, and the number
of clusters may not be the same as the total number of concepts because the
union of the clients’ data in the current round may not cover all concepts.
The server does not know how to update the global concept models without the
matching between the aggregated cluster models and the global concept models
from previous round.

To resolve this challenge, we propose a novel distance-based server concept
matching algorithm. This algorithm not only updates a global concept model
with a close cluster model in distance, but also ensures the update in the
correct gradient descent direction. Our algorithm can use different distance
metrics. For a normal size neural network such as LeNet (LeCun et al., 1998),
Manhattan or Euclidean distance can be employed for their low computational
complexity. For larger neural networks, the dimension reduction techniques
(Carreira-Perpinán, 1997) can be incorporated to mitigate the curse of
dimensionality.

The pseudo-code of server concept matching algorithm is shown in Algorithm 1.
The algorithm requires a global record of the distance between each global
concept model and the corresponding previous global concept model (line 3).
For each aggregated cluster model (line 4), the algorithm tracks its best-
matching candidate (line 5) and its distance from the best-matching candidate
(line 6). Each aggregated cluster model is compared with each global concept
model (line 7) by computing their distance (line 8). If their distance is
smaller than both the global distance record of the corresponding concept $k$
and the distance from previous matching candidate (line 9), we consider them a
better match (line 10) and update the distance between the cluster model and
its matching candidate (line 11). After checking all the global concept
models, the algorithm updates the best-matching global concept model with the
aggregated cluster model (line 12), and also updates the global distance
record with the distance between the best-matching pair (line 13).

This algorithm is theoretically grounded. Intuitively, in a gradient descent-
based learning algorithm, as the learning curve becomes flatter over
iterations, the learning progress slows down. Therefore, the distance between
the current model and the model of the previous iteration becomes smaller.
Theorem 1 formulates this intuition, and Algorithm 1 utilizes this theorem. By
tracking and comparing with the distance record, Algorithm 1 updates each
concept model with a matching cluster model only when their distance becomes
smaller than the current distance. The proof of Theorem 1 demonstrates
theoretically that Algorithm 1 updates the global concept model with a
matching cluster model in the correct gradient descent path. This allows the
concept models to learn over time without interference from other concepts.

Algorithm 1 Server Concept Matching Pseudo-code

1:procedure ServerConceptMatch($W,\Theta$)

2: // Executed at Server

3: require $distRecord$ of size $K$ as the global record of distance between
each global concept model and the corresponding previous global concept model

4: for each aggregated cluster model $w_{j}\in\Theta$ do

5: $candidate\leftarrow null$

6: $candidateDist\leftarrow\infty$

7: for each global concept model $w_{k}\in W$ do

8: $tmpDist\leftarrow Distance(w_{j},w_{k})$

9: if $tmpDist<distList[k]$ and $tmpDist<candidateDist$ then

10: $candidate\leftarrow k$

11: $candidateDist\leftarrow tmpDist$

12: $W[candidate]\leftarrow w_{j}$

13: $distRecord[candidate]\leftarrow candidateDist$

14: return $W$

######  Assumption 1.

Differentiability: The loss function $L(w)$, used to optimize a neural
network, is differentiable with respect to the model parameters $w$.

######  Assumption 2.

Lipschitz continuity: The gradient of the loss function $\nabla L(w)$ is
Lipschitz continuous with a positive constant L. By Lipschitz continuity
definition, for any two points $w^{1}$ and $w^{2}$, the following inequality
holds $\lVert\nabla L(w^{1})-\nabla L(w^{2})\rVert\leq\text{L}\lVert
w^{1}-w^{2}\rVert$, where $\lVert.\rVert$ denotes the norm.

######  Theorem 1.

Given a loss function $L(w)$ under assumptions 1 and 2, $w$ is updated with
gradient descent $w^{t+1}=w^{t}-\eta\nabla L(w^{t})$, where $t$ is the
iteration number, $\eta$ is the learning rate, and $\nabla L(w^{t})$ is the
gradient of the loss function with respect to $w^{t}$, the following
inequality holds $\lVert w^{t+1}-w^{t}\rVert<\lVert w^{t}-w^{t-1}\rVert$.

Theorem 1 is under Assumption 1 Differentiability and Assumption 2 Lipschitz
continuity. The proof of the theorem is in Appendix B. Without requiring
strong assumptions, such as convexity, Assumption 1 and 2 can be applied to
most loss functions for neural networks. In the theorem, gradient descent is
also the prevalent algorithm to update neural networks. Therefore, this
theorem can be applied to the general optimization process of most neural
networks.

Client Concept Matching. The clients receive the global concept models from
the server every round, and each model shall learn the data distribution of
each concept. The clients need to select one of the global concept models, and
fine-tune it with the data of the current round. Since the clients may or may
not be aware of the concepts, they shall perform client concept matching to
match the concept of the current data with a global concept model.

At round $t$, a client $n$ can test the global concept models of the previous
round $\\{w_{k}^{t-1}\\}_{1}^{K}$ on its current local data
$\mathcal{D}_{n}^{t}$, and select the $k^{*}$-th concept model with the
smallest loss for further fine-tuning as Eq. 6. Since the data do not
accumulate over a given limit in FCL, testing the models is an effective
method to select the best-matching global concept model without significant
overhead.

$k^{*}=\arg\min_{k}L(w_{k}^{t-1},\mathcal{D}_{n}^{t})$ (6)

##  5 Evaluation

The evaluation has five main goals: (i) Investigate CM effectiveness; (ii)
Compare with the baselines; (iii) Quantify the performance of different
clustering algorithms; (iv) Quantify the concept matching accuracy; (v)
Investigate the scalabilitiy in terms of the number of clients and the model
size. In the Appendix C.6, we demonstrate CM has good resilience when
configured with numbers of concepts that are different from the ground truth.
We also evaluate CM on a Qualcomm QCS605 IoT device, and demonstrate its
feasibility in real-world and its low overhead in terms of the client end-to-
end operation time.

###  5.1 Experimental Setup

Similar to a recent FCL work (Yoon et al., 2021), we evaluate CM with a
“super” dataset, which consists of six frequently used image datasets: SVHN,
FaceScrub, MNIST, Fashion-MNIST, Not-MNIST, and TrafficSigns. To simulate
different concepts, the “super” dataset is split into five concepts. The
training, test, and validation split follows 7:2:1. More dataset details are
described in Appendix C.1.

Non-overlapping chunks from the five concept datasets are further distributed
randomly to the clients. At every round, the local data across clients are
non-IID. Each client encounters one of the five local concept datasets
randomly, and uses a cyclic sliding window of 320 samples in the encountered
concept dataset. Unless otherwise specified, CM is tested with 20 clients (all
clients participate in each training round), kmean clustering algorithm,
FedAvg aggregation algorithm, and Manhattan distance for the server concept
matching algorithm. To compare with the baseline fairly, we use the same CNN-
based model as (Yoon et al., 2021), detailed in Appendix C.2. The system runs
100 rounds of training for each experiment. More details of Python libraries
used, computing equipment and hyper-parameters are described in Appendix C.3.

To quantify the overall concept matching accuracy, we evaluate under the
class-incremental scenario, and assume the clients are not aware of the
concepts and perform the client concept matching every round. In task-
incremental scenarios, the difference is that the clients do not have to
perform concept matching every round, because they know the task IDs and the
concept drift due to the transition of different tasks. For the same
experimental setting, task-incremental training would achieve the same model
performance with lower computation overhead at the clients.

###  5.2 Results

Effectiveness of CM in FCL vs. Vanilla FL.  Figure 2 demonstrates the
effectiveness of CM, as the volatile learning curves of vanilla FL prominently
illustrate the catastrophic forgetting due to the concept drift and the
potential interference among clients. CM can effectively eliminate these
negative effects with smooth learning curves for all concepts. The superior
performance of CM is more apparent for difficult concepts (RBG images with 50
classes from FaceScrub in Concepts 2 and 3) than in easy concepts (BW images
with 30 classes from mixed MNIST in Concept 4). Compared with vanilla FL, the
concept model accuracy improvement is up to 17.5%, and the weighted average
over the number of samples for all the concepts is also improved from 86.2% to
90.3%. Considering the complexity of the super dataset, this improvement is
significant for image classification.

$0$$20$$40$$60$$80$$100$$0.2$$0.4$$0.6$$0.8$RoundConcept 1CMVanilla FL

$0$$20$$40$$60$$80$$100$$0$$0.2$$0.4$$0.6$$0.8$RoundConcept 2CMVanilla FL

$0$$20$$40$$60$$80$$100$$0$$0.2$$0.4$$0.6$$0.8$RoundConcept 3CMVanilla FL

$0$$20$$40$$60$$80$$100$$0.5$$0.6$$0.7$$0.8$$0.9$RoundConcept 4CMVanilla FL

$0$$20$$40$$60$$80$$100$$0.4$$0.6$$0.8$RoundConcept 5CMVanilla FL

Figure 2: CM vs. vanilla FL: Test set accuracy over communication rounds

Comparison with FCL baselines. Table 1 shows the results of the comparison,
which is performed with the same experimental setup. EWC (French, 1999) is a
commonly employed baseline in CL. We apply it on the clients’ training, and
then the server aggregates with FedAvg. FedWeIT (Yoon et al., 2021) is a
recently published work for FCL. CM outperforms the baselines and achieves
90.3% accuracy. While EWC performs reasonably well (86.7%), FedWeIT does not
perform well under more realistic assumptions and in a larger scale experiment
than its original evaluation (i.e., 5 clients per round, 5 classes to train
per client, and non-overlapping classes over clients). Since FedWeIT applies a
completely different design when learning information across tasks or
concepts, its inferior performance may be partially due to the sparse
parameters employed, which fail to separate different concepts and fully
capture the complex information (i.e. up to 50 classes) in the concepts.

Table 1: Test set accuracy (%) comparison with baselines

Concept |  1 |  2 |  3 |  4 |  5 |  avg
---|---|---|---|---|---|---
EWC | 82.3 | 74.5 | 72.3 | 92.1 | 85.9 | 86.7
FedWeIT | 61.0 | 62.0 | 66.8 | 72.6 | 70.7 | 68.3
CM | 85.4 | 85.2 | 86.5 | 93.3 | 92.4 | 90.3

Performance of clustering algorithms. Table 2 shows the results for 5
clustering algorithms whose parameters are detailed in Appendix C.5. A perfect
clustering can group all client models correctly with the same concept.
Adjusted Rand Index (ARI) is a commonly used metric for clustering algorithms:
1.0 stands for perfect matching. Table 2 also shows the minimum ARI, as the
worst clustering performance over 100 rounds. The results show that CM with
BIRCH performs best, as it achieves up to 96 rounds of perfect clustering out
of 100 and 0.994 average ARI. Furthermore, all algorithms perform reasonably
well and achieve over 88.4% average model accuracy.

Table 2: Clustering performance with 100 rounds training

| Kmean | Agglomerative | BIRCH |  DBSCAN |  OPTICS
---|---|---|---|---|---
Rounds with perfect clustering | 91 | 93 | 96 | 66 | 50
ARI (average) | 0.988 | 0.989 | 0.994 | 0.972 | 0.888
ARI (minimum) | 0.713 | 0.704 | 0.771 | 0.700 | 0.478
Model accuracy % (average) | 90.3 | 90.1 | 90.4 | 88.5 | 88.4

Concept matching accuracy. The concept matching accuracy is defined as the
percentage of correct concept matching over the entire training process (i.e.,
the collaborative client/server matching). Table 3 shows the concept matching
accuracy with a variety of clustering algorithms and distance metrics. To
quantify the concept matching accuracy separately with minimum influence from
the clustering errors, we use the cluster ID that has the most number of
clients with the given data concept, instead of the actual cluster ID for the
mapping of a given client data concept. The results demonstrate that CM
achieves up to 100% concept matching accuracy. Table 2 and 3 demonstrate that
CM provides the flexibility to use different clustering algorithms and
distance metrics, as it performs well with all of them. As neural networks
grow in size and complexity, the curse of dimensionality may manifest itself
and requires advanced clustering algorithms and distance metrics.

Table 3: Concept matching accuracy (%) with 100 training rounds

| Kmean | Agglomerative | BIRCH |  DBSCAN |  OPTICS
---|---|---|---|---|---
Manhattan | 100 | 100 | 100 | 97.4 | 93.4
Euclidean | 100 | 99.8 | 100 | 90.4 | 94.9
Chebyshev | 98.6 | 99.9 | 100 | 97.7 | 93.6

Table 4: CM performance vs. number of clients |  ARI (average) |  CM accuracy % |  Model accuracy %
---|---|---|---
20 | 0.988 | 100 | 90.3
40 | 0.983 | 99.7 | 95.3
80 | 1.000 | 100 | 95.4

Table 5: CM performance vs. model size |  ARI (average) |  CM accuracy % |  Model accuracy %
---|---|---|---
-20% | 0.983 | 100 | 90.0
original | 0.988 | 100 | 90.3
+20% | 0.999 | 100 | 90.4

Concept matching scalability. Table 5 shows CM performs well as the number of
clients increases. Let us note that with 80 clients, both the clustering and
the CM algorithms perform perfectly. This is because the clustering algorithms
generally perform better with larger number of samples. CM enjoys this benefit
and may achieve better model performance (up to 95.4%) with a larger number of
clients. We further test CM with 20% increase or decrease in the size of CNN
layer channels and dense layer neurons. A larger model can further stress-test
CM, and a smaller model can reduce the communication overhead for CM. To avoid
overfitting the model under the given dataset, we could not further downsize
the model or split the data into more clients. Table 5 shows the performance
metrics of CM, and there is a small improvement (90.4%) in accuracy when using
a larger model. Tables 5 and 5 show that both the clustering and the concept
matching achieve near flawless performance in all the settings.

##  6 Conclusion

Concept Matching (CM) is a novel FCL framework to alleviate catastrophic
forgetting and interference among clients by training different models for
different concepts concealed in the data. To avoid interference among clients,
CM uses a clustering algorithm to group the client models with the same
concept. To mitigate catastrophic forgetting, the server and the clients run
concept matching algorithms that collaboratively train and update each concept
model with the matching data of the same concept. Also, the server concept
matching algorithm ensures the updating of the concept model in the correct
gradient descent direction. CM achieves higher model accuracy than state-of-
the-art systems, and works regardless of whether the clients are aware of the
concepts or not. Our extensive evaluation also demonstrates that CM performs
well with a variety of clustering algorithms and distance metrics, and scales
well with the number of clients and the model size.

## References

  * Ahn et al. [2019] Hongjoon Ahn, Sungmin Cha, Donggyu Lee, and Taesup Moon.  Uncertainty-based continual learning with adaptive regularization.  _Advances in neural information processing systems_ , 32, 2019.
  * Aljundi et al. [2019] Rahaf Aljundi, Klaas Kelchtermans, and Tinne Tuytelaars.  Task-free continual learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 11254–11263, 2019.
  * Ankerst et al. [1999] Mihael Ankerst, Markus M Breunig, Hans-Peter Kriegel, and Jörg Sander.  Optics: Ordering points to identify the clustering structure.  _ACM Sigmod record_ , 28(2):49–60, 1999.
  * Beutel et al. [2020] Daniel J Beutel, Taner Topal, Akhil Mathur, Xinchi Qiu, Javier Fernandez-Marques, Yan Gao, Lorenzo Sani, Kwing Hei Li, Titouan Parcollet, Pedro Porto Buarque de Gusmão, et al.  Flower: A friendly federated learning research framework.  _arXiv preprint arXiv:2007.14390_ , 2020.
  * Carreira-Perpinán [1997] Miguel A Carreira-Perpinán.  A review of dimension reduction techniques.  _Department of Computer Science. University of Sheffield. Tech. Rep. CS-96-09_ , 9:1–69, 1997.
  * Casado et al. [2022] Fernando E Casado, Dylan Lema, Marcos F Criado, Roberto Iglesias, Carlos V Regueiro, and Senén Barro.  Concept drift detection and adaptation for federated and continual learning.  _Multimedia Tools and Applications_ , pages 1–23, 2022.
  * De Lange et al. [2021] Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars.  A continual learning survey: Defying forgetting in classification tasks.  _IEEE transactions on pattern analysis and machine intelligence_ , 44(7):3366–3385, 2021.
  * Deng et al. [2020] Lei Deng, Guoqi Li, Song Han, Luping Shi, and Yuan Xie.  Model compression and hardware acceleration for neural networks: A comprehensive survey.  _Proceedings of the IEEE_ , 108(4):485–532, 2020.
  * Dong et al. [2022] Jiahua Dong, Lixu Wang, Zhen Fang, Gan Sun, Shichao Xu, Xiao Wang, and Qi Zhu.  Federated class-incremental learning.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 10164–10173, 2022.
  * Ester et al. [1996] Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, et al.  A density-based algorithm for discovering clusters in large spatial databases with noise.  In _kdd_ , volume 96, pages 226–231, 1996.
  * French [1999] Robert M French.  Catastrophic forgetting in connectionist networks.  _Trends in cognitive sciences_ , 3(4):128–135, 1999.
  * Gao et al. [2022] Liang Gao, Huazhu Fu, Li Li, Yingwen Chen, Ming Xu, and Cheng-Zhong Xu.  Feddc: Federated learning with non-iid data via local drift decoupling and correction.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 10112–10121, 2022.
  * Geiping et al. [2020] Jonas Geiping, Hartmut Bauermeister, Hannah Dröge, and Michael Moeller.  Inverting gradients-how easy is it to break privacy in federated learning?  _Advances in Neural Information Processing Systems_ , 33:16937–16947, 2020.
  * Ghosh et al. [2020] Avishek Ghosh, Jichan Chung, Dong Yin, and Kannan Ramchandran.  An efficient framework for clustered federated learning.  _Advances in Neural Information Processing Systems_ , 33:19586–19597, 2020.
  * Guo et al. [2021] Yongxin Guo, Tao Lin, and Xiaoying Tang.  Towards federated learning on time-evolving heterogeneous data.  _arXiv preprint arXiv:2112.13246_ , 2021.
  * Huang et al. [2022] Wenke Huang, Mang Ye, and Bo Du.  Learn from others and be yourself in heterogeneous federated learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 10143–10153, 2022.
  * Isele and Cosgun [2018] David Isele and Akansel Cosgun.  Selective experience replay for lifelong learning.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 32, 2018.
  * Jiang and Borcea [2023] Xiaopeng Jiang and Cristian Borcea.  Complement sparsification: Low-overhead model pruning for federated learning.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , 2023.
  * Jiang et al. [2022] Xiaopeng Jiang, Han Hu, Thinh On, Phung Lai, Vijaya Datta Mayyuri, An Chen, Devu M Shila, Adriaan Larmuseau, Ruoming Jin, Cristian Borcea, et al.  Flsys: Toward an open ecosystem for federated learning mobile apps.  _IEEE Transactions on Mobile Computing_ , 2022.
  * Jiang et al. [2023] Xiaopeng Jiang, Thinh On, NhatHai Phan, Hessamaldin Mohammadi, Vijaya Datta Mayyuri, An Chen, Ruoming Jin, and Cristian Borcea.  Zone-based federated learning for mobile sensing data.  In _2023 IEEE International Conference on Pervasive Computing and Communications (PerCom)_ , pages 141–148. IEEE, 2023.
  * Kim et al. [2022] Gyuhak Kim, Sepideh Esmaeilpour, Changnan Xiao, and Bing Liu.  Continual learning based on ood detection and task masking.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 3856–3866, 2022.
  * LeCun et al. [1998] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner.  Gradient-based learning applied to document recognition.  _Proceedings of the IEEE_ , 86(11):2278–2324, 1998.
  * Lomonaco and Maltoni [2017] Vincenzo Lomonaco and Davide Maltoni.  Core50: a new dataset and benchmark for continuous object recognition.  In _Conference on Robot Learning_ , pages 17–26. PMLR, 2017.
  * Luping et al. [2019] WANG Luping, WANG Wei, and LI Bo.  Cmfl: Mitigating communication overhead for federated learning.  In _2019 IEEE 39th international conference on distributed computing systems (ICDCS)_ , pages 954–964. IEEE, 2019.
  * Ma et al. [2022] Yuhang Ma, Zhongle Xie, Jue Wang, Ke Chen, and Lidan Shou.  Continual federated learning based on knowledge distillation.  In _Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence_ , volume 3, 2022.
  * Mai et al. [2021] Zheda Mai, Ruiwen Li, Hyunwoo Kim, and Scott Sanner.  Supervised contrastive replay: Revisiting the nearest class mean classifier in online class-incremental continual learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 3589–3599, 2021.
  * Mai et al. [2022] Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyunwoo Kim, and Scott Sanner.  Online continual learning in image classification: An empirical survey.  _Neurocomputing_ , 469:28–51, 2022.
  * Mallya and Lazebnik [2018] Arun Mallya and Svetlana Lazebnik.  Packnet: Adding multiple tasks to a single network by iterative pruning.  In _Proceedings of the IEEE conference on Computer Vision and Pattern Recognition_ , pages 7765–7773, 2018.
  * Maltoni and Lomonaco [2019] Davide Maltoni and Vincenzo Lomonaco.  Continuous learning in single-incremental-task scenarios.  _Neural Networks_ , 116:56–73, 2019.
  * McMahan et al. [2017] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas.  Communication-efficient learning of deep networks from decentralized data.  In _Artificial intelligence and statistics_ , pages 1273–1282. PMLR, 2017.
  * Mitchell [1997] Tom Michael Mitchell.  _Machine learning_.  McGraw-hill New York, 1997.
  * Mothukuri et al. [2021] Viraaji Mothukuri, Reza M Parizi, Seyedamin Pouriyeh, Yan Huang, Ali Dehghantanha, and Gautam Srivastava.  A survey on security and privacy of federated learning.  _Future Generation Computer Systems_ , 115:619–640, 2021.
  * Ouyang et al. [2022] Xiaomin Ouyang, Zhiyuan Xie, Jiayu Zhou, Guoliang Xing, and Jianwei Huang.  Clusterfl: A clustering-based federated learning system for human activity recognition.  _ACM Transactions on Sensor Networks_ , 19(1):1–32, 2022.
  * Qi et al. [2023] Daiqing Qi, Handong Zhao, and Sheng Li.  Better generative replay for continual federated learning.  In _The Eleventh International Conference on Learning Representations_ , 2023.
  * Rolnick et al. [2019] David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne.  Experience replay for continual learning.  _Advances in Neural Information Processing Systems_ , 32, 2019.
  * Serra et al. [2018] Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou.  Overcoming catastrophic forgetting with hard attention to the task.  In _International Conference on Machine Learning_ , pages 4548–4557. PMLR, 2018.
  * Shim et al. [2021] Dongsub Shim, Zheda Mai, Jihwan Jeong, Scott Sanner, Hyunwoo Kim, and Jongseong Jang.  Online class-incremental continual learning with adversarial shapley value.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 35, pages 9630–9638, 2021.
  * Wei et al. [2020] Kang Wei, Jun Li, Ming Ding, Chuan Ma, Howard H Yang, Farhad Farokhi, Shi Jin, Tony QS Quek, and H Vincent Poor.  Federated learning with differential privacy: Algorithms and performance analysis.  _IEEE Transactions on Information Forensics and Security_ , 15:3454–3469, 2020.
  * Wu et al. [2020] Wentai Wu, Ligang He, Weiwei Lin, Rui Mao, Carsten Maple, and Stephen Jarvis.  Safa: A semi-asynchronous protocol for fast federated learning with low overhead.  _IEEE Transactions on Computers_ , 70(5):655–668, 2020.
  * Yoon et al. [2021] Jaehong Yoon, Wonyong Jeong, Giwoong Lee, Eunho Yang, and Sung Ju Hwang.  Federated continual learning with weighted inter-client transfer.  In _International Conference on Machine Learning_ , pages 12073–12086. PMLR, 2021.
  * Zhang et al. [2023] Jie Zhang, Chen Chen, Weiming Zhuang, and Lingjuan Lv.  Addressing catastrophic forgetting in federated class-continual learning.  _arXiv preprint arXiv:2303.06937_ , 2023.
  * Zhang et al. [2022] Lin Zhang, Li Shen, Liang Ding, Dacheng Tao, and Ling-Yu Duan.  Fine-tuning global model via data-free knowledge distillation for non-iid federated learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 10174–10183, 2022.

##  Appendix A Algorithmic Description of Concept Matching Framework

Algorithm 2 describes the pseudo-code of the CM framework. CM executes as a
multi-round, iterative FL cycle (lines 3-9). At each round (line 3), each
client operates in parallel (line 5), including clients concept matching (line
13), local model updates with batches of data (line 14-16), and returning the
client model weights to the server (line 18). Then, server performs clustering
(line 7), aggregation (line 8), and server concept matching to update the
global concept models with the aggregated cluster models (line 9).

Algorithm 2 Concept Matching Framework Pseudo-code

1:procedure ServerExecute:

2: initialize $W^{0}=(w_{1},w_{2},...,w_{K})^{0}$ randomly, total number of
rounds $T$

3: for $t=1$ to $T$ do

4: // Update Done at Clients and Returned to Server

5: for each client $n$ do // In Parallel

6: $\theta_{n}$ = n.ClientUpdate($W^{t-1}$)

7: $\Omega^{t}=Cluster(\\{\theta^{t}_{n}\\}_{n=1}^{N})$

8: $\Theta^{t}=Aggregate(\Omega^{t})$

9: $W^{t}=ServerConceptMatch(\Theta^{t},W^{t-1})$

10:

11:procedure ClientUpdate($W$)

12: // Executed at Clients

13: require step size hyperparameter $\eta$, local dataset of current round
$\mathcal{D}$

14: $k^{*}=ClientConceptMatch(W,\mathcal{D})$

15: $x_{n}\leftarrow$ $\mathcal{D}$ divided into minibatches

16: for each batch $b\in x_{n}$ do

17: $\theta_{n}=w_{k^{*}}-\eta\nabla L(w_{k^{*}},b)$

18: // Results Returned to Server

19: return $\theta_{n}$

##  Appendix B Theorem Proof

######  Assumption 1.

Differentiability: The loss function $L(w)$, used to optimize a neural
network, is differentiable with respect to the model parameters $w$.

######  Assumption 2.

Lipschitz continuity: The gradient of the loss function $\nabla L(w)$ is
Lipschitz continuous with a positive constant L. By Lipschitz continuity
definition, for any two points $w^{1}$ and $w^{2}$, the following inequality
holds $\lVert\nabla L(w^{1})-\nabla L(w^{2})\rVert\leq\text{L}\lVert
w^{1}-w^{2}\rVert$, where $\lVert.\rVert$ denotes the norm.

######  Lemma 1.

Given a loss function $L(w)$ under assumption 1 and 2, $w$ is updated with
gradient descent $w^{t+1}=w^{t}-\eta\nabla L(w^{t})$, where $t$ is the
iteration number, $\eta$ is the learning rate, and $\nabla L(w^{t})$ is the
gradient of the loss function with respect to $w^{t}$, the following
inequality holds $\lVert\nabla L(w^{t+1})\rVert<\lVert\nabla L(w^{t})\rVert$.

###### Proof.

To prove $\lVert\nabla L(w^{t+1})\rVert<\lVert\nabla L(w^{t})\rVert$, we can
use the assumption 2. Let L be the Lipschitz constant. Then, we have
$\lVert\nabla L(w^{t+1})-\nabla L(w^{t})\rVert\leq\text{L}\lVert
w^{t+1}-w^{t}\rVert$.

Now, using the reverse triangle inequality, we can write $\lVert\nabla
L(w^{t+1})\rVert-\lVert\nabla L(w^{t})\rVert\leq\lVert\nabla L(w^{t+1})-\nabla
L(w^{t})\rVert$. Substituting the previous inequality, we get $\lVert\nabla
L(w^{t+1})\rVert-\lVert\nabla L(w^{t})\rVert\leq\text{L}\lVert
w^{t+1}-w^{t}\rVert$.

Using the gradient descent update $w^{t+1}=w^{t}-\eta\nabla L(w^{t})$, we can
write $\lVert\nabla L(w^{t+1})\rVert-\lVert\nabla
L(w^{t})\rVert\leq\text{L}\eta\lVert\nabla L(w^{t})\rVert$. Rearranging the
terms, we get $\lVert\nabla L(w^{t+1})\rVert\leq(1-\text{L}\eta)\lVert\nabla
L(w^{t})\rVert$.

Since L and $\eta$ are both positive, we have $1-\text{L}\eta<1$. Therefore,
we can conclude that $\lVert\nabla L(w^{t+1})\rVert<\lVert\nabla
L(w^{t})\rVert$. This completes the proof. ∎

######  Theorem 1.

Given a loss function $L(w)$ under assumptions 1 and 2, $w$ is updated with
gradient descent $w^{t+1}=w^{t}-\eta\nabla L(w^{t})$, where $t$ is the
iteration number, $\eta$ is the learning rate, and $\nabla L(w^{t})$ is the
gradient of the loss function with respect to $w^{t}$, the following
inequality holds $\lVert w^{t+1}-w^{t}\rVert<\lVert w^{t}-w^{t-1}\rVert$.

###### Proof.

From the inequality in lemma 1, $\lVert\nabla L(w^{t})\rVert<\lVert\nabla
L(w^{t-1})\rVert$, using the gradient descent update $w^{t+1}=w^{t}-\eta\nabla
L(w^{t})$, we write $\lVert w^{t+1}-w^{t}\rVert<\lVert w^{t}-w^{t-1}\rVert$.
This completes the proof. ∎

##  Appendix C Evaluation Details

###  C.1 Dataset

We evaluate CM with a “super” dataset, similar to a recent FCL work [Yoon et
al., 2021], which consists of six frequently used image datasets: SVHN,
FaceScrub, MNIST, Fashion-MNIST, Not-MNIST, and TrafficSigns. To simulate
different concepts, the “super” dataset is splitted into five concepts. As
shown in Table 6, the data in the five concepts differ across a wide spectrum
of classes and number of samples. The original FaceScrub dataset has 100
classes. In order to stress test CM, it is splitted into Concept 2 and 3 with
50 different classes each, as we aim to verify whether CM can differentiate
them successfully. Since MNIST datasets are easy to learn, we mix the three
MNIST datasets together to make it more difficult. The training, test, and
validation split of the data follows 7:2:1.

Table 6: Dataset details for each concept

Concept |  1 |  2 |  3 |  4 |  5
---|---|---|---|---|---
Dataset | SVHN | FaceScrub0 | FaceScrub1 | MNIST, Fashion-MNIST, Not-MNIST | TrafficSigns
No. Classes | 10 | 50 | 50 | 30 | 43
No. Samples | 88300 | 9898 | 9899 | 138197 | 45956

###  C.2 Model

To compare with the baseline fairly, we use the same CNN-based image
classification model as [Yoon et al., 2021]. We believe this model is ideal in
size to learn from the dataset. This model uses two convolutional layers and
three dense layers to classify an image input (32*32*3) into one of the 183
classes. The two convolutional layers have 20 and 50 channels, with 5 by 5
filters, stride of 1, and ReLU activation. A 3 by 3 max pooling with stride of
2 follows them. Then, the flattened tensor is fed into three dense layers of
800, 500 and 183 neurons respectively, with ReLU and Softmax activation.

As it is common practice in class-incremental CL, the model follows the
single-head evaluation setup [Shim et al., 2021, Mai et al., 2021, 2022],
where it has one output head to classify all labels. This setup is ideal for
clients with constrained resource capacity in FL, because the clients do not
have to spend computation resources on expanding or selecting the output head.
Let us note that using the total number of labels as the model output size
does not mean we have to know the entire label space or even the label space
size, because we can use any output size not smaller than the upper bound of
the number of labels. It makes no difference in the training and testing
accuracy when experimenting with a given dataset, because the weights
associated with unencountered output neurons will not be updated in the
backward pass.

###  C.3 Experimental Settings

We implement CM with TensorFlow and scikit-learn. The experiments are
conducted on a Ubuntu Linux cluster (Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
with 512GB memory, 2 NVIDIA P100-SXM2 GPUs with 16GB total memory). Non-
overlapping chunks from the five concept datasets are further distributed
randomly to the clients. At every round, the local data across clients are
non-IID. Each client encounters one of the five local concept datasets
randomly, and uses a cyclic sliding window of 320 samples in the encountered
concept dataset. Unless otherwise specified, CM is tested with 20 clients (all
clients participate in each training round), kmean clustering algorithm,
FedAvg aggregation algorithm, and Manhattan distance for the server CM
algorithm. For the local training, we use the Adam optimizer with learning
rate of 0.001, weight initializer of HeUniform, and batch size of 64. Each
client maintains and uses a single Adam optimizer throughout the training for
all concepts. We train 15 epochs every round, and use early stopping with the
patience value as 3. We test CM with different hyper-parameters, and only
present the results with the hyper-parameters that lead to the best results.
The system runs 100 rounds of training for each experiment.

To quantify the overall concept matching accuracy, we evaluate under the
class-incremental scenario, and assume the clients are not aware of the
concepts and perform the client concept matching every round. In task-
incremental scenarios, the difference is that the clients do not have to
perform concept matching every round, because they know the task IDs and the
concept drift due to the transition of different tasks. For the same
experimental setting, task-incremental training would achieve the same model
performance with lower computation overhead at the clients.

###  C.4 IoT Device Setup

CM is evaluated on a Qualcomm QCS605 IoT device. This device is equipped with
Snapdragon™ QCS605 64-bit ARM v8-compliant octa-core CPU up to 2.5 GHz, Adreno
615 GPU, 8G RAM, and 16 GB eMMC 5.1 onboard storage. We choose it because its
specifications are ideal for AIoT cameras and image-based applications. The
device is connected to Internet through WiFi with a bandwidth of 300 Mbps. We
re-use CM simulation Python code on the device. Since the device does not
support native Linux and its the operating system is rooted Android 8.1, we
need to run a Linux distribution on the Linux kernel of Android for easy
package management and better support. We achieve this goal with two open
source projects: termux-app and ubuntu-in-termux. Termux-app is an Android
application for terminal application and Linux environment. It provides some
basic Linux commands and packages, but is not on par with a mature Linux
distribution, such as Ubuntu. Ubuntu-in-termux bridges the gap. Through it, we
are able to install well-maintained Python environments and libraries to
execute training on-device. The Python training process can be observed
directly under adb shell, which does not include the overhead of Termux-app or
Ubuntu-in-termux.

###  C.5 Parameters for Clustering Algorithms

Kmean, agglomerative, and BIRCH require the number of clusters as a parameter.
Unless otherwise specified, we set this parameter to be 5. DBSCAN and OPTICS
require some threshold values tuned for the data as parameters instead of the
number of clusters. We adhere to the convention when selecting their
parameters. For DBSCAN, there are two main parameters: $min\\_samples$ and
$\epsilon$. $min\\_samples$ is the fewest number of points required to form a
cluster. We adjust it to be 3, which is a lower value than the average number
of clients per concept (20/5). $\epsilon$ is the maximum distance between two
points while the two points can still belong to the same cluster. To choose
$\epsilon$ , we firstly calculate the average distance between each point (the
model weights of a client) and its 3 ($min\\_samples$) nearest neighbors, and
then we sort distance values in the ascending order and plot them. We choose
$\epsilon$ to be 20 as the point of maximum curvature in the plot. Similarly,
we set $min\\_samples$ parameter to be 3 for OPTICS. The other parameters for
these clustering algorithms are the default values in scikit-learn.

###  C.6 Additional Results

Learning curves for scalability. Figure 3 shows the learning curves as the
number of clients increases. The results show that CM can always learn
smoothly. Figure 4 demonstrates CM can also learn smoothly with 20 clients as
the model size increases or decreases 20%. To stress test CM, we further
investigate the model performance when training 80 clients with the network
size increased and decreased 20%. The learning curves in Figure 5 demonstrate
CM’s smooth learning progress, as it achieves 94.3% and 94.9% average
accuracy, respectively.

$0$$20$$40$$60$$80$$100$$0.7$$0.75$$0.8$$0.85$$0.9$$0.95$RoundConcept 180
clients40 clients20 clients

$0$$20$$40$$60$$80$$100$$0$$0.2$$0.4$$0.6$$0.8$$1$RoundConcept 280 clients40
clients20 clients

$0$$20$$40$$60$$80$$100$$0$$0.2$$0.4$$0.6$$0.8$$1$RoundConcept 380 clients40
clients20 clients

$0$$20$$40$$60$$80$$100$$0.86$$0.88$$0.9$$0.92$$0.94$$0.96$RoundConcept 480
clients40 clients20 clients

$0$$20$$40$$60$$80$$100$$0.5$$0.6$$0.7$$0.8$$0.9$$1$RoundConcept 580 clients40
clients20 clients

Figure 3: Test set accuracy vs. communication rounds as number of clients
increases

$0$$20$$40$$60$$80$$100$$0.7$$0.75$$0.8$$0.85$RoundConcept 1+20%original-20%

$0$$20$$40$$60$$80$$100$$0.4$$0.5$$0.6$$0.7$$0.8$RoundConcept
2+20%original-20%

$0$$20$$40$$60$$80$$100$$0.2$$0.4$$0.6$$0.8$RoundConcept 3+20%original-20%

$0$$20$$40$$60$$80$$100$$0.86$$0.88$$0.9$$0.92$RoundConcept 4+20%original-20%

$0$$20$$40$$60$$80$$100$$0.5$$0.6$$0.7$$0.8$$0.9$RoundConcept
5+20%original-20%

Figure 4: Test set accuracy vs. communication rounds for training 20 clients
with different model size

$0$$20$$40$$60$$80$$100$$0.7$$0.75$$0.8$$0.85$$0.9$$0.95$RoundConcept 180
clients, +20%80 clients, -20%

$0$$20$$40$$60$$80$$100$$0$$0.2$$0.4$$0.6$$0.8$$1$RoundConcept 280 clients,
+20%80 clients, -20%

$0$$20$$40$$60$$80$$100$$0$$0.2$$0.4$$0.6$$0.8$$1$RoundConcept 380 clients,
+20%80 clients, -20%

$0$$20$$40$$60$$80$$100$$0.86$$0.88$$0.9$$0.92$$0.94$$0.96$RoundConcept 480
clients, +20%80 clients, -20%

$0$$20$$40$$60$$80$$100$$0.5$$0.6$$0.7$$0.8$$0.9$$1$RoundConcept 580 clients,
+20%80 clients, -20%

Figure 5: Test set accuracy vs. communication rounds for training 80 clients
with different model size

Resilience to number of concepts different from the ground truth. CM requires
an estimated number of concepts configured in the bootstrapping phrase. In
case the system administrator fails to estimate the number of concept
correctly, we experiment with the number of concepts from 3 to 7 under the
same experimental settings (i.e., 5 concepts is the ground truth). As shown in
Figure 6, when the configured number of concepts is higher (6 and 7) than the
ground truth, 5 concept models (out of 6 or 7) learn the corresponding
concepts smoothly. The extra concept models in the system do not effect the
smooth learning progress, and the average model accuracy achieves 90.5% and
90.0% respectively. When the number of concepts is smaller (4 and 3) than the
ground truth, the system starts to treat the similar concepts (e.g., two
FaceScrub concepts) as one. Although the model accuracy on the affected
concepts (2, 3, and 5) exhibit minor fluctuations during the training, the
smooth learning progress for the other concepts (1 and 4) is not affected.
Nevertheless, the average model accuracy achieves 89.5% and 88.9%
respectively, and beats vanilla FL (86.7%). These results demonstrate CM has
good resilience in terms of the number of concepts configured in the system.
Furthermore, they suggest it is better if system administrators over-estimate
the number of concepts, because the performance is still very good in this
case.

$0$$20$$40$$60$$80$$100$$0.2$$0.4$$0.6$$0.8$RoundConcept 17 concepts6
concepts5 concepts4 concepts3 concepts

$0$$20$$40$$60$$80$$100$$0$$0.2$$0.4$$0.6$$0.8$RoundConcept 27 concepts6
concepts5 concepts4 concepts3 concepts

$0$$20$$40$$60$$80$$100$$0$$0.2$$0.4$$0.6$$0.8$RoundConcept 37 concepts6
concepts5 concepts4 concepts3 concepts

$0$$20$$40$$60$$80$$100$$0.5$$0.6$$0.7$$0.8$$0.9$RoundConcept 47 concepts6
concepts5 concepts4 concepts3 concepts

$0$$20$$40$$60$$80$$100$$0.4$$0.6$$0.8$RoundConcept 57 concepts6 concepts5
concepts4 concepts3 concepts

Figure 6: Test set accuracy with different number of concepts over
communication rounds Table 7: Client operation time (second) on real IoT
device

| Receiving model | Concept Matching | Training |  Sending model |  Total
---|---|---|---|---|---
Vanilla FL | 0.67 | N/A | 85.27 | 0.67 | 86.61
CM | 3.35 | 8.52 | 85.27 | 0.67 | 97.81

Client operation overhead. Designed for mobile or IoT devices, CM is evaluated
on a real IoT device in terms of the client end-to-end operation time.
Compared with vanilla FL, the client operation overhead comes from receiving
multiple concept models from the server and the client concept matching. Table
7 shows the breakdown of client operation time on the Qualcomm QCS605 IoT
device in one round. We assume the worst-case scenario that the clients do not
know the concepts, and perform concept matching every round. Compared with a
multi-epoch training process over the entire experienced data, the concept
matching can be achieved by testing only a portion of the data. The
communication time is calculated as sending or receiving the model(s) of size
25.2MB over 300 Mbps WiFi network. The total client end-to-end operation time
in one round is 97.81 seconds, which is feasible for a real-world deployment.
Overall, the total CM operation time has a low overhead (11%) over vanilla FL
operations on the IoT device. We believe the improvement in performance
achieved by CM is worth this overhead cost.