\begin{subtable}[h]{0.49\textwidth}
        \centering
        \resizebox{\textwidth}{!}{
        \begin{tabular}{cccccc}
\toprule
\multirow{2}{*}{Setting} & \multicolumn{4}{c}{Unseen client}                                         & \multirow{2}{*}{Average} \\ \cline{2-5}
                                              & P              & A             & C              & \multicolumn{1}{c}{S} &                          \\ \hline
EoA~\cite{arpit2021ensembledg} & {98.00} & {90.50} & {83.40} & {82.50} & {88.60} \\\hline
Single (K=1) & 97.78& 89.55& 84.51& 82.79& 88.66 \\
Single (K=2) &97.54& 89.40& 84.43& 85.42& 89.20\\
Single (K=3) & 98.08& 89.75& 86.05& 86.03& 89.98 \\
Overall (K=1) & 97.78& \textbf{90.92}& 86.01& 83.66& 89.59 \\
Overall (K=2) & \textbf{98.38}& 90.72& 86.47& 85.62& 90.30 \\
Overall (K=3)  & 98.14& 90.87& \textbf{86.77}& \textbf{86.10}& \textbf{90.47}  \\ \hline
\end{tabular}}
        \caption{CCST+Tent~\cite{wang2021tent}. \vspace{-20pt}}

        \label{tab:add_tent}
     \end{subtable}
    \textbf{(a)} We propose a simple yet effective framework named cross-client style transfer (CCST).
    Our approach achieves new state-of-the-art generalization performance in FL setting on two standard DG benchmarks (PACS~\cite{PACS}, OfficeHome~\cite{officehome}) and a large-scale medical image dataset (Camelyon17~\cite{camelyon17}).
    \textbf{(b)}  Two types of styles with corresponding sharing mechanisms are proposed, named \textit{overall domain style} and \textit{single image style}, which can be chosen according to different circumstances. The diversity level of our method is also flexible to be adjusted.
    \textbf{(c)} The proposed method is orthogonal to many other SOTA DG methods. Therefore, our method can be readily applied to those DG methods to have a further performance boost. We also study the effectiveness of several SOTA DG methods when they are applied in the FL setting for image recognition.
    \textbf{(d)} We give an intuitive (Section \ref{sec:discussion}) and experimental analysis (Section~\ref{sec:privacy}) on the privacy-preserving performance of our style vectors to demonstrate that one can hardly reconstruct the original images merely from the style vectors using the generator from a SOTA GAN \cite{liu2021towards} in FL setting.

\section{Related Work}
\textbf{Domain generalization.}
 Domain generalization is a popular research field that aims to learn a model from multiple source domains such that the model can generalize on the unseen target domain. Many works are proposed towards solving the domain shifts from various directions under the centralized data setting. Those methods can be divided into three categories~\cite{wang2021generalizing}, including manipulating data to enrich data diversity~\cite{jackson2019style,xu2021fourier,shankar2018generalizing,zhou2021domain}, learning domain-invariant representations or disentangling domain-shared and specific features to enhance the generalization ability of model~\cite{arjovsky2019invariant,piratla2020efficient,carlucci2019domain,zhao2020domain} and exploiting general learning strategies to promote generalizing capability~\cite{li2019episodic,huang2020self,dou2019domain,du2020learning}.

\textbf{Neural style transfer.}
\label{style_transfer}
Neural style transfer (NST) aims to transfer the style of an image to another content image with its semantic structure reserved. The development of NST has roughly gone through three stages: per-style-per-model (PSPM), multiple-style-per-model (MSPM) and arbitrary-style-per-model (ASPM) methods \cite{StyleTransferReview}. PSPM methods \cite{Gatys_2016_CVPR,Johnson2016PerceptualLF,Ulyanov2016TextureNF} can only transfer a single style for each trained model. MSPM methods \cite{dumoulin2016learned,stylebank,Zhang2018MultistyleGN,Li2017DiversIfiedTS}  are able to transfer multiple styles with a single trained model. However, PSPM and MSPM are expensive to deploy when too many styles are required to be transferred in our setting. ASPM \cite{chen2016fast,huang2017adain,Ghiasi2017,Li2017} can transfer arbitrary styles to any content images and is often faster than PSPM and MSPM, which is more suitable for our scenario. 

\begin{equation}
    \begin{aligned}
     &S _ {overall} ^ {C_n} = (\mu(F_{all} ^ {C_n}), \sigma(F_{all} ^ {C_n})), \\
     &F_{all} ^ {C_n} = Stack(F_1 ^ {C_n}, F_2 ^ {C_n}, ..., F_M ^ {C_n}).
     \end{aligned}
    \end{equation}

\subsection{Experimental Settings}
\textbf{Experiment setup.} We take each domain as a single client and conduct the leave-one-domain-out experiments on PACS and Office-Home datasets. Specifically, we select one client as the target test domain and train our model on the other clients. For the medical dataset, following the setting of source/target domains in literature~\cite{camelyon17,koh2021wilds}, we apply the leave-one-domain-out setting to hospital 4 and hospital 5. For the PACS dataset, we follow the JiGen~\cite{carlucci2019domain} to split 90\% data of each client as the training set and 10\% of that as the validation set for source clients, while for unseen target clients, the entire data is used for testing. For OfficeHome and Camelyon17, which have more data samples, the ratio between train and validation set is 4:1 for each source client, and 20\% data is utilized as the test set on the unseen target client. We compare our method with \textbf{FedDG}~\cite{liu2021feddg}, which aims to solve DG problems in federated learning for medical image segmentation. We also test the performance of three centralized DG methods under FL setting (FedAvg), including \textbf{JiGen}~\cite{carlucci2019domain}, \textbf{RSC}~\cite{huang2020self} and \textbf{MixStyle}~\cite{zhou2021domain}. For \textbf{COPA}, due to its re-designed layers of ResNet18 and unknown train-validate-test split, we copy the results for reference only. We regard every single client as a centralized dataset and apply these methods locally in FL. We report the test accuracy on each unseen client by choosing the best validation model.
\\\textbf{Implementation details.} We utilize the pre-trained AdaIN~\cite{huang2017adain} to perform style transfer. Following \cite{huang2020self}, we choose ResNet~\cite{he2016deep} pre-trained on ImageNet as our backbone for PACS and Office-Home datasets. For the Camelyon17 dataset, we follow \cite{harmofl} to use the DenseNet121~\cite{huang2017densely}. We use FedAvg~\cite{mcmahan2017communication} as our FL framework and train the model using SGD optimizer with $1e^{-3}$ learning rate for 500 communication rounds with one local update epoch on the PACS and Office-Home dataset. For Camelyon17, we train 100 communication rounds considering its large data amount.
The JiGen and RSC can be directly integrated into the FedAvg without further modifications. We adapt the MixStyle into an intra-client version that shuffles styles inside each batch of data to fit the federated setting. 
All hyper-parameters of compared methods are chosen based on corresponding papers. We follow the standard training procedure of FedAvg~\cite{mcmahan2017communication} for federated training.
The value of M in  is $\lceil dataset\_size / 32\rceil$ in our experiment. 
The framework is implemented with PyTorch and is trained on a single NVIDIA RTX 2080 Ti GPU. 

\subsection{Results}
\textbf{Comparison with state-of-the-arts.} We compare our approach with three centralized DG methods and a federated DG method for on standard DG benchmarks PACS and OfficeHome as well as a real-world medical image dataset Camelyon17. 
Table~\ref{table:all_results} presents the quantitative results of the image recognition task for different target clients on both PACS and Office-Home datasets. Each single letter column shows the test accuracy of the global model with the best validation accuracy on an unseen client. 
Although all DG methods can have better performance based on FedAvg, our approach demonstrates a significant boost over others on both datasets. On the PACS benchmark, our method achieves the average accuracy of 86.52\% , which is 3.47\% better than the second best method FedDG. Especially for the unseen client S (Sketch), CCST outperforms other methods by more than 7\%. When photo is the target domain, all the methods perform similarly because we start training based on the ImageNet-pretrained model, which already has very high performance on photo images.
Besides the PACS benchmark, the performance of our approach on the Office-Home dataset also has consistent results. Specifically, CCST outperforms other methods on average with a testing accuracy of 63.56\%. 
Due to the small discrepancy in domain styles, all those domain generalization methods bring smaller improvements (less than 1\%) than on the PACS. Overall, CCST outperforms other DG methods by a large margin.
Figure~\ref{fig:camelyon17_results} shows the results on the Camelyon17 dataset, our method outperforms other DG methods both when hospital 4 and hospital 5 as the target client. Some DG method, such as JiGen, is even harmful when applied in FL setting on the Camelyon17 dataset.   

\textbf{Control experiments on CCST.}
We conduct control experiments to investigate two types of image style with different augmentation levels. In Table \ref{tab:ablation}, \textit{Single} and \textit{Overall} represents single image style and overall domain style mentioned in Section \ref{sec:method} respectively. Different augmentation level $K$ indicates the intensity of augmentation.  We evaluate the four settings on the PACS benchmark with ResNet50. 
For each kind of style, larger augmentation level K leads to a better performance. It is worth mentioning that the performance achieved by K=2 is similar with K=3, which indicates that our method can already achieve good performance with a relatively large K. 
For different types of style, overall domain style shows more improvement than single image style because the overall style is able to represent a more general and accurate domain statistics, while single image styles may differ a lot due to randomness.

\textbf{Orthogonality.}
Our method is orthogonal to many other DG methods and can lead to additive performance via combined utilization. As many traditional domain generalization methods require centralized data and need to make use of various styles to achieve a domain robust model, CCST can serve as an initial step to benefit traditional DG methods with diversified styles. As shown in Figure~\ref{fig:extra_boost}, we plot the average test accuracy of FedDG and three centralized DG methods on the PACS benchmark before and after applying our CCST. From the average accuracy, we can see all DG methods benefit a further boost with the help of cross-client style transfer (CCST). Interestingly, we find the performance when tested on sketch client (S) gains the largest improvement with CCST.
Besides, we also extend our method with Tent~\cite{wang2021tent} which uses entropy to update parameters in batch normalization layers at test time. As shown in Table~\ref{tab:add_tent}, with this method combined in federated setting, our method surpasses the state-of-the-art DG method EoA~\cite{arpit2021ensembledg} on the PACS benchmark in centralized setting with an average test accuracy of 90.47\%. 
{"value": "98.00", "type": "Result", "training data/set": "PACS", "test data/set": "PACS P", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "EoA", "model settings": {"xx": "yy"}} {"value": "90.50", "type": "Result", "training data/set": "PACS", "test data/set": "PACS A", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "EoA", "model settings": {"xx": "yy"}} {"value": "83.40", "type": "Result", "training data/set": "PACS", "test data/set": "PACS C", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "EoA", "model settings": {"xx": "yy"}} {"value": "82.50", "type": "Result", "training data/set": "PACS", "test data/set": "PACS S", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "EoA", "model settings": {"xx": "yy"}} {"value": "88.60", "type": "Result", "training data/set": "PACS", "test data/set": "PACS", "task": "image recognition", "metric": "Average accuracy", "experimental settings": {"xx": "yy"}, "model": "EoA", "model settings": {"xx": "yy"}} {"value": "97.78", "type": "Result", "training data/set": "PACS", "test data/set": "PACS P", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Single", "model settings": {"K": "1"}} {"value": "89.55", "type": "Result", "training data/set": "PACS", "test data/set": "PACS A", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Single", "model settings": {"K": "1"}} {"value": "84.51", "type": "Result", "training data/set": "PACS", "test data/set": "PACS C", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Single", "model settings": {"K": "1"}} {"value": "82.79", "type": "Result", "training data/set": "PACS", "test data/set": "PACS S", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Single", "model settings": {"K": "1"}} {"value": "88.66", "type": "Result", "training data/set": "PACS", "test data/set": "PACS", "task": "image recognition", "metric": "Average accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Single", "model settings": {"K": "1"}} {"value": "97.54", "type": "Result", "training data/set": "PACS", "test data/set": "PACS P", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Single", "model settings": {"K": "2"}} {"value": "89.40", "type": "Result", "training data/set": "PACS", "test data/set": "PACS A", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Single", "model settings": {"K": "2"}} {"value": "84.43", "type": "Result", "training data/set": "PACS", "test data/set": "PACS C", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Single", "model settings": {"K": "2"}} {"value": "85.42", "type": "Result", "training data/set": "PACS", "test data/set": "PACS S", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Single", "model settings": {"K": "2"}} {"value": "89.20", "type": "Result", "training data/set": "PACS", "test data/set": "PACS", "task": "image recognition", "metric": "Average accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Single", "model settings": {"K": "2"}} {"value": "98.08", "type": "Result", "training data/set": "PACS", "test data/set": "PACS P", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Single", "model settings": {"K": "3"}} {"value": "89.75", "type": "Result", "training data/set": "PACS", "test data/set": "PACS A", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Single", "model settings": {"K": "3"}} {"value": "86.05", "type": "Result", "training data/set": "PACS", "test data/set": "PACS C", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Single", "model settings": {"K": "3"}} {"value": "86.03", "type": "Result", "training data/set": "PACS", "test data/set": "PACS S", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Single", "model settings": {"K": "3"}} {"value": "89.98", "type": "Result", "training data/set": "PACS", "test data/set": "PACS", "task": "image recognition", "metric": "Average accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Single", "model settings": {"K": "3"}} {"value": "97.78", "type": "Result", "training data/set": "PACS", "test data/set": "PACS P", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Overall", "model settings": {"K": "1"}} {"value": "90.92", "type": "Result", "training data/set": "PACS", "test data/set": "PACS A", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Overall", "model settings": {"K": "1"}} {"value": "86.01", "type": "Result", "training data/set": "PACS", "test data/set": "PACS C", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Overall", "model settings": {"K": "1"}} {"value": "83.66", "type": "Result", "training data/set": "PACS", "test data/set": "PACS S", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Overall", "model settings": {"K": "1"}} {"value": "89.59", "type": "Result", "training data/set": "PACS", "test data/set": "PACS", "task": "image recognition", "metric": "Average accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Overall", "model settings": {"K": "1"}} {"value": "98.38", "type": "Result", "training data/set": "PACS", "test data/set": "PACS P", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Overall", "model settings": {"K": "2"}} {"value": "90.72", "type": "Result", "training data/set": "PACS", "test data/set": "PACS A", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Overall", "model settings": {"K": "2"}} {"value": "86.47", "type": "Result", "training data/set": "PACS", "test data/set": "PACS C", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Overall", "model settings": {"K": "2"}} {"value": "85.62", "type": "Result", "training data/set": "PACS", "test data/set": "PACS S", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Overall", "model settings": {"K": "2"}} {"value": "90.30", "type": "Result", "training data/set": "PACS", "test data/set": "PACS", "task": "image recognition", "metric": "Average accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Overall", "model settings": {"K": "2"}} {"value": "98.14", "type": "Result", "training data/set": "PACS", "test data/set": "PACS P", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Overall", "model settings": {"K": "3"}} {"value": "90.87", "type": "Result", "training data/set": "PACS", "test data/set": "PACS A", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Overall", "model settings": {"K": "3"}} {"value": "86.77", "type": "Result", "training data/set": "PACS", "test data/set": "PACS C", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Overall", "model settings": {"K": "3"}} {"value": "86.10", "type": "Result", "training data/set": "PACS", "test data/set": "PACS S", "task": "image recognition", "metric": "accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Overall", "model settings": {"K": "3"}} {"value": "90.47", "type": "Result", "training data/set": "PACS", "test data/set": "PACS", "task": "image recognition", "metric": "Average accuracy", "experimental settings": {"xx": "yy"}, "model": "CCST+Tent Overall", "model settings": {"K": "3"}}