\begin{subtable}[h]{0.5\textwidth}
        \centering
        \resizebox{\textwidth}{!}{
        \begin{tabular}{ccccccc}
\toprule
 & \multirow{2}{*}{Setting} & \multicolumn{4}{c}{Unseen client}                                         & \multirow{2}{*}{Average} \\ \cline{3-6}
                          &                          & P              & A             & C              & \multicolumn{1}{c}{S} &                          \\ \hline
& FedAvg~\cite{mcmahan2017communication} & 95.51          & 82.23          & 78.20          & 73.56                   & 82.37                    \\ \cline{2-7}
&  Single (K=1)           & 95.75&87.5&74.66&76.56&83.62\\
& Single (K=2) &\textbf{96.77}&86.23&75.73&80.12&84.71\\
& Single (K=3)            & 96.65&86.63&74.53&81.85&84.84\\
& Overall (K=1)           & 95.69&86.67&75.85&77.37&83.90\\
&Overall (K=2) &96.41&\textbf{88.72}&78.03&80.91&86.02\\
& Overall (K=3)           & 96.65&88.33&\textbf{78.20}&\textbf{82.90}&\textbf{86.52}\\\hline            
\end{tabular}}
       \caption{Control experiment for CCST. \vspace{-20pt}}
       
       \label{tab:ablation}
    \end{subtable}
    \textbf{(a)} We propose a simple yet effective framework named cross-client style transfer (CCST).
    Our approach achieves new state-of-the-art generalization performance in FL setting on two standard DG benchmarks (PACS~\cite{PACS}, OfficeHome~\cite{officehome}) and a large-scale medical image dataset (Camelyon17~\cite{camelyon17}).
    \textbf{(b)}  Two types of styles with corresponding sharing mechanisms are proposed, named \textit{overall domain style} and \textit{single image style}, which can be chosen according to different circumstances. The diversity level of our method is also flexible to be adjusted.
    \textbf{(c)} The proposed method is orthogonal to many other SOTA DG methods. Therefore, our method can be readily applied to those DG methods to have a further performance boost. We also study the effectiveness of several SOTA DG methods when they are applied in the FL setting for image recognition.
    \textbf{(d)} We give an intuitive (Section \ref{sec:discussion}) and experimental analysis (Section~\ref{sec:privacy}) on the privacy-preserving performance of our style vectors to demonstrate that one can hardly reconstruct the original images merely from the style vectors using the generator from a SOTA GAN \cite{liu2021towards} in FL setting.

\textbf{Neural style transfer.}
\label{style_transfer}
Neural style transfer (NST) aims to transfer the style of an image to another content image with its semantic structure reserved. The development of NST has roughly gone through three stages: per-style-per-model (PSPM), multiple-style-per-model (MSPM) and arbitrary-style-per-model (ASPM) methods \cite{StyleTransferReview}. PSPM methods \cite{Gatys_2016_CVPR,Johnson2016PerceptualLF,Ulyanov2016TextureNF} can only transfer a single style for each trained model. MSPM methods \cite{dumoulin2016learned,stylebank,Zhang2018MultistyleGN,Li2017DiversIfiedTS}  are able to transfer multiple styles with a single trained model. However, PSPM and MSPM are expensive to deploy when too many styles are required to be transferred in our setting. ASPM \cite{chen2016fast,huang2017adain,Ghiasi2017,Li2017} can transfer arbitrary styles to any content images and is often faster than PSPM and MSPM, which is more suitable for our scenario. 

    where $\{i_1,\hdots,i_J\}$ are randomly sampled image indices from client $C_n$. Sharing single image styles consumes relatively low computation but can lead to high communication costs for uploading multiple styles.
    \textbf{Overall domain style.} Domain style is the domain-level channel wise mean and standard variance, which considers all the images (pixels) in a client. Formally, assume client $C_n$ has $M$ training images with corresponding VGG features $\{F_1 ^ {C_n}, F_2 ^ {C_n}, ..., F_M ^ {C_n}\}$, the overall style  $S _ {overall} ^ {C_n}$ of this client is:

\begin{equation}
    \begin{aligned}
     &S _ {overall} ^ {C_n} = (\mu(F_{all} ^ {C_n}), \sigma(F_{all} ^ {C_n})), \\
     &F_{all} ^ {C_n} = Stack(F_1 ^ {C_n}, F_2 ^ {C_n}, ..., F_M ^ {C_n}).
     \end{aligned}
    \end{equation}

\subsubsection{Local style transfer.} 
When client $C_n$ receives the style bank $\mathbf{B}$, the local data can be augmented by transferring styles in $\mathbf{B}$ to existing images, which introduces styles of other domains into this client.
A hyperparameter $K \in \{1,2,...,N\}$ called \textit{augmentation level}, is set to choose $K$ styles from style bank $\mathbf{B}$ for the augmentation of each image, indicating the diversity of final augmented data set. Suppose the size of the original dataset is $d$, then after cross-client style transfer, the size of the augmented data set will become $d \times K$. 

\section{Experiments}
\subsection{Datasets}
We evaluate our method on two standard domain generalization datasets (PACS~\cite{li2017deeper}, Office-Home~\cite{venkateswara2017deep}) that consist of various image styles as domains and a real-world medical image dataset (Camelyon17~\cite{camelyon17}). Specifically, PACS is a 7-class image recognition benchmark including 9,991 images with four different image style domains, including photo, art, cartoon, and sketch. Office-Home is another image recognition dataset that includes 15,588 images of 65 classes from four different domains (art, clipart, product, and real-world). Camelyon17 is a public tumor classification dataset, which has histology images from 5 hospitals. 

\subsection{Experimental Settings}
\textbf{Experiment setup.} We take each domain as a single client and conduct the leave-one-domain-out experiments on PACS and Office-Home datasets. Specifically, we select one client as the target test domain and train our model on the other clients. For the medical dataset, following the setting of source/target domains in literature~\cite{camelyon17,koh2021wilds}, we apply the leave-one-domain-out setting to hospital 4 and hospital 5. For the PACS dataset, we follow the JiGen~\cite{carlucci2019domain} to split 90\% data of each client as the training set and 10\% of that as the validation set for source clients, while for unseen target clients, the entire data is used for testing. For OfficeHome and Camelyon17, which have more data samples, the ratio between train and validation set is 4:1 for each source client, and 20\% data is utilized as the test set on the unseen target client. We compare our method with \textbf{FedDG}~\cite{liu2021feddg}, which aims to solve DG problems in federated learning for medical image segmentation. We also test the performance of three centralized DG methods under FL setting (FedAvg), including \textbf{JiGen}~\cite{carlucci2019domain}, \textbf{RSC}~\cite{huang2020self} and \textbf{MixStyle}~\cite{zhou2021domain}. For \textbf{COPA}, due to its re-designed layers of ResNet18 and unknown train-validate-test split, we copy the results for reference only. We regard every single client as a centralized dataset and apply these methods locally in FL. We report the test accuracy on each unseen client by choosing the best validation model.
\\\textbf{Implementation details.} We utilize the pre-trained AdaIN~\cite{huang2017adain} to perform style transfer. Following \cite{huang2020self}, we choose ResNet~\cite{he2016deep} pre-trained on ImageNet as our backbone for PACS and Office-Home datasets. For the Camelyon17 dataset, we follow \cite{harmofl} to use the DenseNet121~\cite{huang2017densely}. We use FedAvg~\cite{mcmahan2017communication} as our FL framework and train the model using SGD optimizer with $1e^{-3}$ learning rate for 500 communication rounds with one local update epoch on the PACS and Office-Home dataset. For Camelyon17, we train 100 communication rounds considering its large data amount.
The JiGen and RSC can be directly integrated into the FedAvg without further modifications. We adapt the MixStyle into an intra-client version that shuffles styles inside each batch of data to fit the federated setting. 
All hyper-parameters of compared methods are chosen based on corresponding papers. We follow the standard training procedure of FedAvg~\cite{mcmahan2017communication} for federated training.
The value of M in  is $\lceil dataset\_size / 32\rceil$ in our experiment. 
The framework is implemented with PyTorch and is trained on a single NVIDIA RTX 2080 Ti GPU. 

\textbf{Control experiments on CCST.}
We conduct control experiments to investigate two types of image style with different augmentation levels. In Table \ref{tab:ablation}, \textit{Single} and \textit{Overall} represents single image style and overall domain style mentioned in Section \ref{sec:method} respectively. Different augmentation level $K$ indicates the intensity of augmentation.  We evaluate the four settings on the PACS benchmark with ResNet50. 
For each kind of style, larger augmentation level K leads to a better performance. It is worth mentioning that the performance achieved by K=2 is similar with K=3, which indicates that our method can already achieve good performance with a relatively large K. 
For different types of style, overall domain style shows more improvement than single image style because the overall style is able to represent a more general and accurate domain statistics, while single image styles may differ a lot due to randomness.
{"value": "95.51", "type": "Result", "training data/set": "PACS", "test data/set": "PACS P", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "FedAvg", "model settings": {"xx": "yy"}} {"value": "82.23", "type": "Result", "training data/set": "PACS", "test data/set": "PACS A", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "FedAvg", "model settings": {"xx": "yy"}} {"value": "78.20", "type": "Result", "training data/set": "PACS", "test data/set": "PACS C", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "FedAvg", "model settings": {"xx": "yy"}} {"value": "73.56", "type": "Result", "training data/set": "PACS", "test data/set": "PACS S", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "FedAvg", "model settings": {"xx": "yy"}} {"value": "82.37", "type": "Result", "training data/set": "PACS", "test data/set": "PACS", "task": "image recognition", "metric": "Average accuracy", "experimental settings": {"xx": "yy"}, "model": "FedAvg", "model settings": {"xx": "yy"}} {"value": "95.75", "type": "Result", "training data/set": "PACS", "test data/set": "PACS P", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Single", "model settings": {"K": "1"}} {"value": "87.5", "type": "Result", "training data/set": "PACS", "test data/set": "PACS A", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Single", "model settings": {"K": "1"}} {"value": "74.66", "type": "Result", "training data/set": "PACS", "test data/set": "PACS C", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Single", "model settings": {"K": "1"}} {"value": "76.56", "type": "Result", "training data/set": "PACS", "test data/set": "PACS S", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Single", "model settings": {"K": "1"}} {"value": "83.62", "type": "Result", "training data/set": "PACS", "test data/set": "PACS", "task": "image recognition", "metric": "Average accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Single", "model settings": {"K": "1"}} {"value": "96.77", "type": "Result", "training data/set": "PACS", "test data/set": "PACS P", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Single", "model settings": {"K": "2"}} {"value": "86.23", "type": "Result", "training data/set": "PACS", "test data/set": "PACS A", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Single", "model settings": {"K": "2"}} {"value": "75.73", "type": "Result", "training data/set": "PACS", "test data/set": "PACS C", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Single", "model settings": {"K": "2"}} {"value": "80.12", "type": "Result", "training data/set": "PACS", "test data/set": "PACS S", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Single", "model settings": {"K": "2"}} {"value": "84.71", "type": "Result", "training data/set": "PACS", "test data/set": "PACS", "task": "image recognition", "metric": "Average accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Single", "model settings": {"K": "2"}} {"value": "96.65", "type": "Result", "training data/set": "PACS", "test data/set": "PACS P", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Single", "model settings": {"K": "3"}} {"value": "86.63", "type": "Result", "training data/set": "PACS", "test data/set": "PACS A", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Single", "model settings": {"K": "3"}} {"value": "74.53", "type": "Result", "training data/set": "PACS", "test data/set": "PACS C", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Single", "model settings": {"K": "3"}} {"value": "81.85", "type": "Result", "training data/set": "PACS", "test data/set": "PACS S", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Single", "model settings": {"K": "3"}} {"value": "84.84", "type": "Result", "training data/set": "PACS", "test data/set": "PACS", "task": "image recognition", "metric": "Average accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Single", "model settings": {"K": "3"}} {"value": "95.69", "type": "Result", "training data/set": "PACS", "test data/set": "PACS P", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Overall", "model settings": {"K": "1"}} {"value": "86.67", "type": "Result", "training data/set": "PACS", "test data/set": "PACS A", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Overall", "model settings": {"K": "1"}} {"value": "75.85", "type": "Result", "training data/set": "PACS", "test data/set": "PACS C", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Overall", "model settings": {"K": "1"}} {"value": "77.37", "type": "Result", "training data/set": "PACS", "test data/set": "PACS S", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Overall", "model settings": {"K": "1"}} {"value": "83.90", "type": "Result", "training data/set": "PACS", "test data/set": "PACS", "task": "image recognition", "metric": "Average accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Overall", "model settings": {"K": "1"}} {"value": "96.41", "type": "Result", "training data/set": "PACS", "test data/set": "PACS P", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Overall", "model settings": {"K": "2"}} {"value": "88.72", "type": "Result", "training data/set": "PACS", "test data/set": "PACS A", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Overall", "model settings": {"K": "2"}} {"value": "78.03", "type": "Result", "training data/set": "PACS", "test data/set": "PACS C", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Overall", "model settings": {"K": "2"}} {"value": "80.91", "type": "Result", "training data/set": "PACS", "test data/set": "PACS S", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Overall", "model settings": {"K": "2"}} {"value": "86.02", "type": "Result", "training data/set": "PACS", "test data/set": "PACS", "task": "image recognition", "metric": "Average accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Overall", "model settings": {"K": "2"}} {"value": "96.65", "type": "Result", "training data/set": "PACS", "test data/set": "PACS P", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Overall", "model settings": {"K": "3"}} {"value": "88.33", "type": "Result", "training data/set": "PACS", "test data/set": "PACS A", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Overall", "model settings": {"K": "3"}} {"value": "78.20", "type": "Result", "training data/set": "PACS", "test data/set": "PACS C", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Overall", "model settings": {"K": "3"}} {"value": "82.90", "type": "Result", "training data/set": "PACS", "test data/set": "PACS S", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Overall", "model settings": {"K": "3"}} {"value": "86.52", "type": "Result", "training data/set": "PACS", "test data/set": "PACS", "task": "image recognition", "metric": "Average accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST Overall", "model settings": {"K": "3"}}