Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

In particular, FedCVAE-Ens and FedCVAE-KD obtain more than \(1.75\times\) the accuracy of the best baseline method for MNIST, more than \(2\times\) the accuracy for FashionMNIST, and more than \(2.75\times\) the accuracy for SVHN under extreme statistical heterogeneity (i.e., clients only observe one or two classes).

The review comment to be evaluated is:

The paper presents a relatively simple idea and is mostly clearly written. As far as I am aware, this method is novel in the context of FL. The authors also provide a reasonable amount of information, which should make FedCVAE reproducible. The authors seem to mostly tackle the cross-silo setting, where each client has enough data. One thing I noticed is that the authors mention that the conditional VAE (CVAE) objective at eq. 1 bounds the marginal likelihood $\log p(x)$; this is not correct, as the specific objective at eq. 1 bounds the conditional marginal likelihood $\log p(x|y)$.

Besides that, there are a couple of things that are, in my opinion, not properly motivated. More specifically, the authors propose to sample from the generative model and train a classifier at the server. Why is this necessary and why you do not simply use the generative classifier  $p(y|x) = \frac{\int p(x| z, y)p(z)dz p(y)}{p(x)}$? To me it seems that the target for the auxiliary classifier is nothing else, but to approximate this generative classifier. The loss for the auxiliary classifier (in the limit where you constantly sample from the generative model) can be obtained by minimising the average KL-divergence between the true CVAE generative posterior $p(y|x)$ and the classifier predictive $q(y|x)$. More specifically: $$E_p(x)[KL(p(y|x) || q(y|x))] = - E_p(y,x) [\log q(y|x)] + const = -E_p(y,z,x)[\log q(y|x)]  + const = -E_p(y)p(z)p(x|y,z)[\log q(y|x)] + const$$ In this sense, it seems that 1) it might not be necessary to train an auxiliary classifier and 2) the modelling choices made in the generative model itself are important for the performance of the actual downstream classifier. I believe a discussion about this is warranted to be added in the paper. Apart from that, when the authors want to sample datapoints from the model they actually use a different distribution over the latents than the one the model was actually trained; e.g., using a $z \sim U[-1, 1]$ instead of $z\sim N(0, I)$. They mention that it is beneficial to stay at a high density region of the prior, but how does the performance of the algorithm fare when you use the correct prior? Why not use e.g., $z \sim N(0, \sigma)$ with $\sigma < 1$ which is closer to the correct prior?

Another thing that could be better explained are the security and privacy aspects of this method, especially given the FL scenario. FedCVAE uploads the CVAE decoder along with $p(y)$ to the server, which is inherently less private than FedAvg. While it is true that with DP you can bound memorisation and get privacy, at the moment the authors do not use anything and the gains in performance when doing DP relatively to e.g., DP-FedAvg are not clear. In fact, given a sufficient powerful generative model, this method should be indistinguishable from sending the actual data. Furthermore, the arguments about security by shifting $\mu$, need to be more formal. Are there some guarantees that you can give, given a sufficient powerful adversary? For example, given $p(y)$, and $p(x|z, y)$, couldn’t the attacker just try to infer a global mean for $z$? For example, by visual inspection of the samples or an optimization procedure? And a small comment; $U[-10, 10]$ is not an exceptionally good guess for $N(0, I)$ (which is what the text mentions at the beginning of page 9) as 99% of the probability of $N(0,I)$ is in $[-3, 3]$, so 70% of the mass of the uniform proposal is on almost zero probability mass under $N(0, I)$.

Furthermore, this method requires building good generative models which is generally a harder task than building a good discriminative model. How does this work on more complex datasets, such as CIFAR10 and CIFAR100, where $p(x|y, z)$ might not be as sharp? Furthermore, while non-iid settings are nice to have, to me it seems that this method might suffer in less non-iid / iid settings (e.g, $\alpha=1.0$, $\alpha=\inf$) ; the generative distribution is more diverse and thus it would be harder to model accurately with a, hardware constrained, VAE model. This can also sort of be seen when comparing FedCVAE-KD to FedCVAE-Ens; the former has worse performance that the latter, given that the entire training distribution needs to be modelled by a single model. How does the performance of the method scale with higher values of $\alpha$? Besides that, it seems that the evaluation against the baselines is not entirely apples-to-apples; for example, FedOneShot does not need to reveal to the server the labels and the models trained on the local data, so it has fundamentally different security / privacy tradeoffs.

The evidence to be evaluated is:We jointly propose FedCVAE-Ens and FedCVAE-KD, one-shot FL methods that do not require an auxiliary public dataset for server-side training (they are _data-free_). They address issues caused by high statistical heterogeneity by reframing the learning task using CVAEs and account for model heterogeneity by allowing different CVAE architectures across clients.

At \(\alpha=0.001\), FedCVAE-KD obtains more than \(1.75\times\) the accuracy of the best baseline method for MNIST, more than \(2\times\) the accuracy for FashionMNIST, and nearly \(2.75\times\) the accuracy for SVHN (Table 1).

While both FedAvg and FedOneShot struggle when many clients are present, Table 2 shows that FedCVAE-Ens and FedCVAE-KD perform consistently across the number of clients, with the only exception being FedCVAE-KD for SVHN.

Experiments varying the dataset partitions reveal high variation in accuracy for FedAvg and FedOneShot, whereas both FedCVAE-Ens and FedCVAE-KD exhibit consistent accuracy (Table 5 in Appendix C).

We are the first to apply CVAEs to one-shot FL with a focus on high statistical heterogeneity. In this paper, we proposed FedCVAE-Ens and FedCVAE-KD, data-free one-shot FL methods that reframe the local learning task using CVAEs.

Extensive experimental results showed that FedCVAE-Ens and FedCVAE-KD substantially extended the state-of-the-art in one-shot FL under very high statistical heterogeneity, making way for more nuanced development in this difficult environment for distributed learning.

**Benchmark Datasets.** MNIST and FashionMNIST contain \(28\times 28\) grayscale images of handwritten digits and clothing/accessories, respectively, with \(60,000\) train samples and \(10,000\) test samples. SVHN contains \(32\times 32\) RGB image crops of street-view house numbers, with \(73,257\) train samples and \(26,032\) test samples.

The KD approach we propose to aggregate client decoders generates substantially better samples (Figure 8) while also obtaining more than \(2\times\) classifier accuracy on MNIST, more than \(3.25\times\) accuracy on FashionMNIST, and nearly \(5\times\) accuracy for SVHN at \(\alpha=0.001\) (Table 6). \begin{table}
\begin{tabular}{l l|c c|c c} \hline \hline  & \# of Clients & FedAvg & FedOneShot & FedCVAE-KD (ours) & FedCVAE-Ens (ours) \\ \hline \multirow{3}{*}{MNIST} & \(m=5\) & \(42.34\pm 1.75\) & \(42.97\pm 1.54\) & \(80.19\pm 1.76\) & \(92.81\pm 2.16\) \\  & \(m=10\) & \(58.29\pm 2.83\) & \(41.15\pm 0.70\) & \(82.01\pm 1.61\) & \(93.83\pm 1.53\) \\  & \(m=20\) & \(37.14\pm 3.87\) & \(32.22\pm 0.71\) & \(81.93\pm 1.75\) & \(93.52\pm 0.86\) \\  & \(m=50\) & \(28.26\pm 7.98\) & \(35.62\pm 0.66\) & \(77.68\pm 1.39\) & \(87.56\pm 1.44\) \\ \hline \multirow{3}{*}{FashionMNIST} & \(m=5\) & \(38.51\pm 2.40\) & \(32.23\pm 1.68\) & \(68.01\pm 1.56\) & \(73.70\pm 1.93\) \\  & \(m=10\) & \(45.85\pm 2.95\) & \(37.63\pm 0.53\) & \(69.97\pm 1.63\) & \(76.62\pm 1.61\) \\  & \(m=20\) & \(27.49\pm 4.77\) & \(24.72\pm 1.20\) & \(71.28\pm 1.17\) & \(76.51\pm 1.38\) \\  & \(m=50\) & \(31.91\pm 2.20\) & \(41.01\pm 2.49\) & \(69.25\pm 1.13\) & \(76.13\pm 2.00\) \\ \hline \multirow{3}{*}{SVHN} & \(m=5\) & \(40.61\pm 0.64\) & \(42.05\pm 1.31\) & \(57.17\pm 2.56\) & \(65.72\pm 2.17\) \\  & \(m=10\) & \(25.12\pm 2.07\) & \(31.91\pm 1.26\) & \(55.97\pm 0.38\) & \(65.50\pm 2.28\) \\ \cline{1-1}  & \(m=20\) & \(26.99\pm 1.15\) & \(10.68\pm 0.70\) & \(51.80\pm 2.75\) & \(65.54\pm 1.89\) \\ \cline{1-1}  & \(m=50\) & \(19.59\pm 0.00\) & \(30.20\pm 0.64\) & \(40.03\pm 2.62\) & \(64.81\pm 2.44\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: Performance of four one-shot FL methods over three datasets and across four numbers of clients \(m\). Best results for each dataset and each level of \(m\) are in purple, with second best results in yellow.

Input: Table 2: Performance of four one-shot FL methods over three datasets and across four numbers of

Input: Figure 8: Samples from the aggregated server decoder obtained through parameter averaging (top row) versus knowledge distillation (bottom row) at high statistical heterogeneity (α = 0.01).

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find the statement moderately overstated. While Table 1 shows significant improvements, the specific multipliers claimed (1.75x, 2x, and 2.75x) are not consistently supported across all scenarios. For MNIST at α=0.001, FedCVAE-Ens achieves 95.37% vs. FedAvg's 45.12%, representing about 2.11x improvement. For FashionMNIST, it's 76.04% vs. 32.77% (2.32x), and for SVHN, 65.52% vs. 20.31% (3.22x). While the general trend of improvement is valid, the specific multipliers claimed don't precisely match the evidence.

<score>3</score>

Justification:
1. The claim generalizes specific performance improvements that vary across different experimental conditions and metrics.

2. Direct evidence shows:
- For MNIST: Actual improvement is ~2.11x (95.37/45.12), not "more than 1.75x"
- For FashionMNIST: Actual improvement is ~2.32x (76.04/32.77), slightly higher than claimed "more than 2x"
- For SVHN: Actual improvement is ~3.22x (65.52/20.31), higher than claimed "more than 2.75x"

3. The reviewer points out important limitations not acknowledged in the claim:
- Method may struggle with less non-iid settings
- Comparison against baselines isn't entirely fair due to different security/privacy tradeoffs
- Performance on more complex datasets is not addressed

4. The claim would be better stated by being more precise about the specific conditions under which these improvements are achieved and acknowledging the limitations noted in the review.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

In particular, FedCVAE-Ens and FedCVAE-KD obtain more than \(1.75\times\) the accuracy of the best baseline method for MNIST, more than \(2\times\) the accuracy for FashionMNIST, and more than \(2.75\times\) the accuracy for SVHN under extreme statistical heterogeneity (i.e., clients only observe one or two classes).

The review comment to be evaluated is:

The paper is very well written and easy to follow. The numerical experiments are well executed and show promising performance achieved by the proposed methods.

That being said, there are some downsides that, in my opinion, slightly outweigh the strengths of the paper. On the problem formulation side, I believe the one-shot federated learning (FL) problem is not very well motivated; the point of FL is to enable training models at different clients by leveraging each others' data through communication without explicitly sharing them. In a one-shot scenario, the problem essentially reduces to having a set of models, which need to be aggregated into a single, universal model, so I am still not convinced how the "federated" aspect of the problem is of the essence.

On the contributions side, the main issue I have with the paper is a lack of a central, key novelty. In its current form, the paper is mostly a collection of different ideas, namely two algorithms that are almost independent of each other, as well as a security aspect, which is again mostly orthogonal to the proposed methods. To me, what would make this paper much stronger is to focus on only one of these algorithms/aspects throughout the paper. For example, since the performance of FEDCVAE-KD is better than baselines, I believe it would be better to focus on this algorithm in the main body of the paper, and add the ensemble version of the algorithm (which, as the authors allude to, requires a higher amount of memory that scales with the number of clients) as an extension, in a separate discussion section toward the end of the paper and/or in the appendix. Moreover, the security and privacy considerations are definitely important, but including them in the main body of the paper stretches it too thin, in my opinion.

Further comments:

- Aside from the solid empirical performance, are there any intuitive/theoretical explanations as to why the proposed methods are well suited for i) *one-shot* FL, and ii) *high heterogeneity* FL? As presented, I cannot see a clear motivation for why the proposed methods should perform better in the considered problem formulation.
- Please discuss whether or not sharing local label distributions with the server violates the privacy of clients' data. How do the proposed methods perform if this information is unavailable at the server?
- Presenting the flowchart figure for only one of the proposed methods (FEDCVAE-KD) in Figure 2 and not including any of the algorithms for the two proposed methods makes the paper slightly incoherent, without switching back and forth between the body of the paper and the appendix. I suggest rearranging the contents to remedy this issue as best as possible by at least having both the figure and algorithm for one of the methods or having the figures for both methods in the main body of the paper.
- In the first footnote on page 4, why is a uniform distribution used for sampling $\mathbf{z}$? Why not, for example, use a truncated Gaussian distribution?
- In my opinion, including FedAVG is slightly unfair in the experiments. Under FedAVG, a set of client model parameters $\\{\mathbf{w}^k\\}_{k=1}^m$ are aggregated using a simple average, i.e., $\mathbf{w}=\frac{1}{m} \sum_\{k=1\}^m\mathbf{w}^k$. In a very heterogeneous case, each of these model parameters is the minimizer of a very different objective function over an entirely different optimization landscape, so it is completely expected that a linear averaging of these model parameters should lead to inferior performance, especially if the averaging operation is done only once.
- Following the above comment, I believe your primary competing baseline is FedONESHOT, so I suggest adding more details on this algorithm's main idea.
- Do you have any explanation as to why the standard deviation of the performance of the proposed methods is, on average, larger under SVHN in Table 2?

The evidence to be evaluated is:We jointly propose FedCVAE-Ens and FedCVAE-KD, one-shot FL methods that do not require an auxiliary public dataset for server-side training (they are _data-free_). They address issues caused by high statistical heterogeneity by reframing the learning task using CVAEs and account for model heterogeneity by allowing different CVAE architectures across clients.

At \(\alpha=0.001\), FedCVAE-KD obtains more than \(1.75\times\) the accuracy of the best baseline method for MNIST, more than \(2\times\) the accuracy for FashionMNIST, and nearly \(2.75\times\) the accuracy for SVHN (Table 1).

While both FedAvg and FedOneShot struggle when many clients are present, Table 2 shows that FedCVAE-Ens and FedCVAE-KD perform consistently across the number of clients, with the only exception being FedCVAE-KD for SVHN.

Experiments varying the dataset partitions reveal high variation in accuracy for FedAvg and FedOneShot, whereas both FedCVAE-Ens and FedCVAE-KD exhibit consistent accuracy (Table 5 in Appendix C).

We are the first to apply CVAEs to one-shot FL with a focus on high statistical heterogeneity. In this paper, we proposed FedCVAE-Ens and FedCVAE-KD, data-free one-shot FL methods that reframe the local learning task using CVAEs.

Extensive experimental results showed that FedCVAE-Ens and FedCVAE-KD substantially extended the state-of-the-art in one-shot FL under very high statistical heterogeneity, making way for more nuanced development in this difficult environment for distributed learning.

**Benchmark Datasets.** MNIST and FashionMNIST contain \(28\times 28\) grayscale images of handwritten digits and clothing/accessories, respectively, with \(60,000\) train samples and \(10,000\) test samples. SVHN contains \(32\times 32\) RGB image crops of street-view house numbers, with \(73,257\) train samples and \(26,032\) test samples.

The KD approach we propose to aggregate client decoders generates substantially better samples (Figure 8) while also obtaining more than \(2\times\) classifier accuracy on MNIST, more than \(3.25\times\) accuracy on FashionMNIST, and nearly \(5\times\) accuracy for SVHN at \(\alpha=0.001\) (Table 6). \begin{table}
\begin{tabular}{l l|c c|c c} \hline \hline  & \# of Clients & FedAvg & FedOneShot & FedCVAE-KD (ours) & FedCVAE-Ens (ours) \\ \hline \multirow{3}{*}{MNIST} & \(m=5\) & \(42.34\pm 1.75\) & \(42.97\pm 1.54\) & \(80.19\pm 1.76\) & \(92.81\pm 2.16\) \\  & \(m=10\) & \(58.29\pm 2.83\) & \(41.15\pm 0.70\) & \(82.01\pm 1.61\) & \(93.83\pm 1.53\) \\  & \(m=20\) & \(37.14\pm 3.87\) & \(32.22\pm 0.71\) & \(81.93\pm 1.75\) & \(93.52\pm 0.86\) \\  & \(m=50\) & \(28.26\pm 7.98\) & \(35.62\pm 0.66\) & \(77.68\pm 1.39\) & \(87.56\pm 1.44\) \\ \hline \multirow{3}{*}{FashionMNIST} & \(m=5\) & \(38.51\pm 2.40\) & \(32.23\pm 1.68\) & \(68.01\pm 1.56\) & \(73.70\pm 1.93\) \\  & \(m=10\) & \(45.85\pm 2.95\) & \(37.63\pm 0.53\) & \(69.97\pm 1.63\) & \(76.62\pm 1.61\) \\  & \(m=20\) & \(27.49\pm 4.77\) & \(24.72\pm 1.20\) & \(71.28\pm 1.17\) & \(76.51\pm 1.38\) \\  & \(m=50\) & \(31.91\pm 2.20\) & \(41.01\pm 2.49\) & \(69.25\pm 1.13\) & \(76.13\pm 2.00\) \\ \hline \multirow{3}{*}{SVHN} & \(m=5\) & \(40.61\pm 0.64\) & \(42.05\pm 1.31\) & \(57.17\pm 2.56\) & \(65.72\pm 2.17\) \\  & \(m=10\) & \(25.12\pm 2.07\) & \(31.91\pm 1.26\) & \(55.97\pm 0.38\) & \(65.50\pm 2.28\) \\ \cline{1-1}  & \(m=20\) & \(26.99\pm 1.15\) & \(10.68\pm 0.70\) & \(51.80\pm 2.75\) & \(65.54\pm 1.89\) \\ \cline{1-1}  & \(m=50\) & \(19.59\pm 0.00\) & \(30.20\pm 0.64\) & \(40.03\pm 2.62\) & \(64.81\pm 2.44\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: Performance of four one-shot FL methods over three datasets and across four numbers of clients \(m\). Best results for each dataset and each level of \(m\) are in purple, with second best results in yellow.

Input: Table 2: Performance of four one-shot FL methods over three datasets and across four numbers of

Input: Figure 8: Samples from the aggregated server decoder obtained through parameter averaging (top row) versus knowledge distillation (bottom row) at high statistical heterogeneity (α = 0.01).

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence carefully, I find that the claim is largely well-stated but has some minor overstatement. The evidence in Table 1 shows that at α=0.001, FedCVAE-KD and FedCVAE-Ens do achieve the claimed performance improvements over baselines (1.75x for MNIST, 2x for FashionMNIST, 2.75x for SVHN). However, the claim could be more precise by specifying the exact heterogeneity level (α=0.001) rather than using the broader phrase "extreme statistical heterogeneity." The reviewer's comments about FedAvg being an unfair baseline are also worth considering, though they don't invalidate the core numerical comparisons.

<score>2</score>

Justification:
1. The numerical claims are supported by the data in Table 1:
- MNIST: ~95.37% (FedCVAE-Ens) vs 45.12% (FedAvg) ≈ 2.11x improvement
- FashionMNIST: ~76.04% vs 32.77% ≈ 2.32x improvement
- SVHN: ~65.52% vs 20.31% ≈ 3.23x improvement

2. Minor overstatement issues:
- The claim uses "extreme statistical heterogeneity" without specifying α=0.001
- The improvements vary across different numbers of clients (Table 2)
- The reviewer raises valid concerns about FedAvg being an unfair baseline comparison

3. Supporting evidence:
- Table 2 demonstrates consistent performance across different numbers of clients
- Figure 8 shows qualitative improvements in generated samples
- The reviewer acknowledges "solid empirical performance" despite other concerns

The score of 2 reflects that while the numerical claims are accurate, there could be more precise qualification of the conditions under which these improvements are achieved.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

In particular, FedCVAE-Ens and FedCVAE-KD obtain more than \(1.75\times\) the accuracy of the best baseline method for MNIST, more than \(2\times\) the accuracy for FashionMNIST, and more than \(2.75\times\) the accuracy for SVHN under extreme statistical heterogeneity (i.e., clients only observe one or two classes).

The review comment to be evaluated is:

### Strength

* The proposed method of one-shot FL is very interesting and novel.
* The proposed method doesn't require auxiliary public dataset.
* Both security and privacy extensions are considered in the paper.
* It can be applied to heterogeneous local models with similar generative capabilities.

### Weaknesses
* One potential issue of the method is training a CVAE locally. In the paper, the images used in the dataset are very small, and the amount of data each client has is large. However, in the real-world application of FL like medical imagings, with very large images and very limited number of data samples, training a CVAE locally can be difficult even just for one class of data.
* The training task of local models is different from the original task. This limits the re-use of pre-trained local model from original task. I think one of the reasons that people want to do one-shot FL is to prevent any re-training locally. But the proposed method still requires to train some new model.

The evidence to be evaluated is:We jointly propose FedCVAE-Ens and FedCVAE-KD, one-shot FL methods that do not require an auxiliary public dataset for server-side training (they are _data-free_). They address issues caused by high statistical heterogeneity by reframing the learning task using CVAEs and account for model heterogeneity by allowing different CVAE architectures across clients.

At \(\alpha=0.001\), FedCVAE-KD obtains more than \(1.75\times\) the accuracy of the best baseline method for MNIST, more than \(2\times\) the accuracy for FashionMNIST, and nearly \(2.75\times\) the accuracy for SVHN (Table 1).

While both FedAvg and FedOneShot struggle when many clients are present, Table 2 shows that FedCVAE-Ens and FedCVAE-KD perform consistently across the number of clients, with the only exception being FedCVAE-KD for SVHN.

Experiments varying the dataset partitions reveal high variation in accuracy for FedAvg and FedOneShot, whereas both FedCVAE-Ens and FedCVAE-KD exhibit consistent accuracy (Table 5 in Appendix C).

We are the first to apply CVAEs to one-shot FL with a focus on high statistical heterogeneity. In this paper, we proposed FedCVAE-Ens and FedCVAE-KD, data-free one-shot FL methods that reframe the local learning task using CVAEs.

Extensive experimental results showed that FedCVAE-Ens and FedCVAE-KD substantially extended the state-of-the-art in one-shot FL under very high statistical heterogeneity, making way for more nuanced development in this difficult environment for distributed learning.

**Benchmark Datasets.** MNIST and FashionMNIST contain \(28\times 28\) grayscale images of handwritten digits and clothing/accessories, respectively, with \(60,000\) train samples and \(10,000\) test samples. SVHN contains \(32\times 32\) RGB image crops of street-view house numbers, with \(73,257\) train samples and \(26,032\) test samples.

The KD approach we propose to aggregate client decoders generates substantially better samples (Figure 8) while also obtaining more than \(2\times\) classifier accuracy on MNIST, more than \(3.25\times\) accuracy on FashionMNIST, and nearly \(5\times\) accuracy for SVHN at \(\alpha=0.001\) (Table 6). \begin{table}
\begin{tabular}{l l|c c|c c} \hline \hline  & \# of Clients & FedAvg & FedOneShot & FedCVAE-KD (ours) & FedCVAE-Ens (ours) \\ \hline \multirow{3}{*}{MNIST} & \(m=5\) & \(42.34\pm 1.75\) & \(42.97\pm 1.54\) & \(80.19\pm 1.76\) & \(92.81\pm 2.16\) \\  & \(m=10\) & \(58.29\pm 2.83\) & \(41.15\pm 0.70\) & \(82.01\pm 1.61\) & \(93.83\pm 1.53\) \\  & \(m=20\) & \(37.14\pm 3.87\) & \(32.22\pm 0.71\) & \(81.93\pm 1.75\) & \(93.52\pm 0.86\) \\  & \(m=50\) & \(28.26\pm 7.98\) & \(35.62\pm 0.66\) & \(77.68\pm 1.39\) & \(87.56\pm 1.44\) \\ \hline \multirow{3}{*}{FashionMNIST} & \(m=5\) & \(38.51\pm 2.40\) & \(32.23\pm 1.68\) & \(68.01\pm 1.56\) & \(73.70\pm 1.93\) \\  & \(m=10\) & \(45.85\pm 2.95\) & \(37.63\pm 0.53\) & \(69.97\pm 1.63\) & \(76.62\pm 1.61\) \\  & \(m=20\) & \(27.49\pm 4.77\) & \(24.72\pm 1.20\) & \(71.28\pm 1.17\) & \(76.51\pm 1.38\) \\  & \(m=50\) & \(31.91\pm 2.20\) & \(41.01\pm 2.49\) & \(69.25\pm 1.13\) & \(76.13\pm 2.00\) \\ \hline \multirow{3}{*}{SVHN} & \(m=5\) & \(40.61\pm 0.64\) & \(42.05\pm 1.31\) & \(57.17\pm 2.56\) & \(65.72\pm 2.17\) \\  & \(m=10\) & \(25.12\pm 2.07\) & \(31.91\pm 1.26\) & \(55.97\pm 0.38\) & \(65.50\pm 2.28\) \\ \cline{1-1}  & \(m=20\) & \(26.99\pm 1.15\) & \(10.68\pm 0.70\) & \(51.80\pm 2.75\) & \(65.54\pm 1.89\) \\ \cline{1-1}  & \(m=50\) & \(19.59\pm 0.00\) & \(30.20\pm 0.64\) & \(40.03\pm 2.62\) & \(64.81\pm 2.44\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: Performance of four one-shot FL methods over three datasets and across four numbers of clients \(m\). Best results for each dataset and each level of \(m\) are in purple, with second best results in yellow.

Input: Table 2: Performance of four one-shot FL methods over three datasets and across four numbers of

Input: Figure 8: Samples from the aggregated server decoder obtained through parameter averaging (top row) versus knowledge distillation (bottom row) at high statistical heterogeneity (α = 0.01).

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence:

The claim appears to be well-stated (1) based on the evidence presented in Table 1 and the supplementary findings. The performance improvements cited (1.75x for MNIST, 2x for FashionMNIST, and 2.75x for SVHN at α=0.001) are directly supported by the numerical results. For example, at α=0.001, FedCVAE-Ens achieves 95.37% vs 45.12% (FedAvg) for MNIST, 76.04% vs 32.77% for FashionMNIST, and 65.52% vs 20.31% for SVHN. The reviewer comments, while noting limitations about real-world applications, don't contradict the specific performance claims made for these benchmark datasets.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Furthermore, to protect the decoders uploaded to the server, we propose a method to shift the center of the CVAE prior distribution.

The evidence to be evaluated is:In practice, we find it useful to focus on the highest density region of the prior and instead sample from a truncated standard normal distribution with tight symmetric bounds.

CVAEs use a prior distribution over latent space to train the encoder and decoder models.

To promote security, we propose to shift the center of the prior distribution \(\mathbf{\mu}\) to a random position in real space (i.e., \(\mathbf{\mu}\in\mathbb{R}^{d}\)), which can be communicated offline or via encryption methods between server and clients (Zhou et al., 2020). As shown in Figure 7 (Appendix B), sampling latent vectors too far from the center of the normal prior produces qualitatively poor data samples, deterring eavesdropping attackers who have no knowledge of \(\mathbf{\mu}\).

We verify the effectiveness of our proposed distribution shift extension for securing the uploaded information. Suppose an eavesdropping attacker is able to intercept the label distributions, decoder weights, and decoder architectures from all clients during upload. Without knowledge of the shared center of the multivariate normal prior \(\mathbf{\mu}\), we show that training a performant classifier is infeasible because it is difficult to extract high-quality samples from the client decoders. In particular, even when the attacker samples latent vectors \(\mathbf{z}\) from a broad region which overlaps with the high-density region of the prior (i.e., a uniform distribution centered on \(\mathbf{\mu}\)), the accuracy of the classifier trained on the resulting samples degrades sharply as the sampling region grows (Figure 5).

Therefore, this simple FedCVAE-Ens and FedCVAE-KD extension reduces eavesdropping attackers' capacity to extract high-quality samples from uploaded decoders or train a performant downstream model, reducing communication risks.

We are the first to apply CVAEs to one-shot FL with a focus on high statistical heterogeneity. In this paper, we proposed FedCVAE-Ens and FedCVAE-KD, data-free one-shot FL methods that reframe the local learning task using CVAEs. Both methods performed well given high statistical heterogeneity, demonstrated consistent performance with increasing numbers of clients, allow for model heterogeneity across clients, and can be extended to promote security.

During training, the CVAE will largely observe latent vectors in the highest density region of the prior distribution; for a standard normal distribution, this is near the center \(\mathbf{\mu}=\mathbf{0}\). Latent vectors distant from the center will not generate high-quality samples when used with the trained decoder. As a demonstration, we train a centralized CVAE and sample both close to the center of the prior (i.e., \(\mathbf{z}_{i}\sim\mathcal{U}(-1,1)\)) and distant from the center of the prior (i.e., \(\mathbf{z}_{i}\sim\mathcal{U}(5,20)\)).

Input: Figure 7: CVAE samples generated using latent vectors distant from the center of the multivariate normal prior (top row) and close to the center (bottom row).

Input: Figure 5: Results for our proposed security-promoting extension at high heterogeneity (α = 0.05). We show the accuracy for a classifier trained on samples from the intercepted client decoders and using client label distributions when the center of the normal prior is unknown. The sampling bounds represent the parameters of the uniform distribution used for latent vector sampling, i.e., ±100 represents the distribution U(−100, 100). The prior is a multivariate standard normalN (0, I).

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Let me evaluate the claim about protecting decoders through shifting the CVAE prior distribution center:

The evidence strongly supports this claim through both theoretical explanation and empirical results. Figure 7 demonstrates how sampling far from the prior's center produces poor quality samples. Figure 5 shows quantitative evidence that without knowing the shifted center μ, an attacker's classifier accuracy degrades significantly as the sampling bounds increase. The method is well-explained mathematically and experimentally validated across multiple datasets. The claim is measured and accurately reflects the evidence without overstatement, appropriately focusing on "proposing" the method rather than making absolute security guarantees.

<score>1</score>

Justification:
1. Direct experimental validation shown in Figures 5 and 7
2. Clear mathematical formulation of the approach
3. Evidence across multiple datasets (MNIST, FashionMNIST, SVHN)
4. Conservative language ("propose" rather than definitive claims)
5. Results demonstrate effective security protection through degraded sample quality and classifier performance
6. Acknowledgment of the method as an "extension" rather than complete solution


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Furthermore, to protect the decoders uploaded to the server, we propose a method to shift the center of the CVAE prior distribution.

The review comment to be evaluated is:

Strength:

-- The proposed solutions seem effective under the one-shot FL setting. The efficacy and efficiency are both demonstrated in the extensive experiments.
-- The writing is clear and easy to follow. The tables and figures are well-organized.

Weaknesses:

-- Constructing a generative model to generate samples for FL model training is not new. There are a few works utilizing a similar strategy in FL, for example:
Data-Free Knowledge Distillation for Heterogeneous Federated Learning, ICML 2021
Fine-tuning global model via data-free knowledge distillation for non-iid federated learning, CVPR 2022
Please make a comparison with those works

-- Some details are not clear enough for me. For example, in Eq 2, what is the formulation for g(.)?  Is this dissimilarity measured in the data-space or feature space?

The evidence to be evaluated is:In practice, we find it useful to focus on the highest density region of the prior and instead sample from a truncated standard normal distribution with tight symmetric bounds.

CVAEs use a prior distribution over latent space to train the encoder and decoder models.

To promote security, we propose to shift the center of the prior distribution \(\mathbf{\mu}\) to a random position in real space (i.e., \(\mathbf{\mu}\in\mathbb{R}^{d}\)), which can be communicated offline or via encryption methods between server and clients (Zhou et al., 2020). As shown in Figure 7 (Appendix B), sampling latent vectors too far from the center of the normal prior produces qualitatively poor data samples, deterring eavesdropping attackers who have no knowledge of \(\mathbf{\mu}\).

We verify the effectiveness of our proposed distribution shift extension for securing the uploaded information. Suppose an eavesdropping attacker is able to intercept the label distributions, decoder weights, and decoder architectures from all clients during upload. Without knowledge of the shared center of the multivariate normal prior \(\mathbf{\mu}\), we show that training a performant classifier is infeasible because it is difficult to extract high-quality samples from the client decoders. In particular, even when the attacker samples latent vectors \(\mathbf{z}\) from a broad region which overlaps with the high-density region of the prior (i.e., a uniform distribution centered on \(\mathbf{\mu}\)), the accuracy of the classifier trained on the resulting samples degrades sharply as the sampling region grows (Figure 5).

Therefore, this simple FedCVAE-Ens and FedCVAE-KD extension reduces eavesdropping attackers' capacity to extract high-quality samples from uploaded decoders or train a performant downstream model, reducing communication risks.

We are the first to apply CVAEs to one-shot FL with a focus on high statistical heterogeneity. In this paper, we proposed FedCVAE-Ens and FedCVAE-KD, data-free one-shot FL methods that reframe the local learning task using CVAEs. Both methods performed well given high statistical heterogeneity, demonstrated consistent performance with increasing numbers of clients, allow for model heterogeneity across clients, and can be extended to promote security.

During training, the CVAE will largely observe latent vectors in the highest density region of the prior distribution; for a standard normal distribution, this is near the center \(\mathbf{\mu}=\mathbf{0}\). Latent vectors distant from the center will not generate high-quality samples when used with the trained decoder. As a demonstration, we train a centralized CVAE and sample both close to the center of the prior (i.e., \(\mathbf{z}_{i}\sim\mathcal{U}(-1,1)\)) and distant from the center of the prior (i.e., \(\mathbf{z}_{i}\sim\mathcal{U}(5,20)\)).

Input: Figure 7: CVAE samples generated using latent vectors distant from the center of the multivariate normal prior (top row) and close to the center (bottom row).

Input: Figure 5: Results for our proposed security-promoting extension at high heterogeneity (α = 0.05). We show the accuracy for a classifier trained on samples from the intercepted client decoders and using client label distributions when the center of the normal prior is unknown. The sampling bounds represent the parameters of the uniform distribution used for latent vector sampling, i.e., ±100 represents the distribution U(−100, 100). The prior is a multivariate standard normalN (0, I).

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find that the proposed method for shifting the CVAE prior distribution center is well-supported by both theoretical explanation and experimental results. The evidence includes:

1. Clear visualization (Figure 7) demonstrating degraded sample quality when sampling far from the prior center
2. Quantitative results (Figure 5) showing significant accuracy degradation when attackers sample without knowing the shifted center
3. Detailed mathematical formulation of the approach
4. Clear security benefits explained and demonstrated

The claim is well-stated and accurately reflects the evidence without exaggeration. The review comments do not challenge this specific claim, focusing instead on other aspects of the work.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Furthermore, to protect the decoders uploaded to the server, we propose a method to shift the center of the CVAE prior distribution.

The review comment to be evaluated is:

The paper presents a relatively simple idea and is mostly clearly written. As far as I am aware, this method is novel in the context of FL. The authors also provide a reasonable amount of information, which should make FedCVAE reproducible. The authors seem to mostly tackle the cross-silo setting, where each client has enough data. One thing I noticed is that the authors mention that the conditional VAE (CVAE) objective at eq. 1 bounds the marginal likelihood $\log p(x)$; this is not correct, as the specific objective at eq. 1 bounds the conditional marginal likelihood $\log p(x|y)$.

Besides that, there are a couple of things that are, in my opinion, not properly motivated. More specifically, the authors propose to sample from the generative model and train a classifier at the server. Why is this necessary and why you do not simply use the generative classifier  $p(y|x) = \frac{\int p(x| z, y)p(z)dz p(y)}{p(x)}$? To me it seems that the target for the auxiliary classifier is nothing else, but to approximate this generative classifier. The loss for the auxiliary classifier (in the limit where you constantly sample from the generative model) can be obtained by minimising the average KL-divergence between the true CVAE generative posterior $p(y|x)$ and the classifier predictive $q(y|x)$. More specifically: $$E_p(x)[KL(p(y|x) || q(y|x))] = - E_p(y,x) [\log q(y|x)] + const = -E_p(y,z,x)[\log q(y|x)]  + const = -E_p(y)p(z)p(x|y,z)[\log q(y|x)] + const$$ In this sense, it seems that 1) it might not be necessary to train an auxiliary classifier and 2) the modelling choices made in the generative model itself are important for the performance of the actual downstream classifier. I believe a discussion about this is warranted to be added in the paper. Apart from that, when the authors want to sample datapoints from the model they actually use a different distribution over the latents than the one the model was actually trained; e.g., using a $z \sim U[-1, 1]$ instead of $z\sim N(0, I)$. They mention that it is beneficial to stay at a high density region of the prior, but how does the performance of the algorithm fare when you use the correct prior? Why not use e.g., $z \sim N(0, \sigma)$ with $\sigma < 1$ which is closer to the correct prior?

Another thing that could be better explained are the security and privacy aspects of this method, especially given the FL scenario. FedCVAE uploads the CVAE decoder along with $p(y)$ to the server, which is inherently less private than FedAvg. While it is true that with DP you can bound memorisation and get privacy, at the moment the authors do not use anything and the gains in performance when doing DP relatively to e.g., DP-FedAvg are not clear. In fact, given a sufficient powerful generative model, this method should be indistinguishable from sending the actual data. Furthermore, the arguments about security by shifting $\mu$, need to be more formal. Are there some guarantees that you can give, given a sufficient powerful adversary? For example, given $p(y)$, and $p(x|z, y)$, couldn’t the attacker just try to infer a global mean for $z$? For example, by visual inspection of the samples or an optimization procedure? And a small comment; $U[-10, 10]$ is not an exceptionally good guess for $N(0, I)$ (which is what the text mentions at the beginning of page 9) as 99% of the probability of $N(0,I)$ is in $[-3, 3]$, so 70% of the mass of the uniform proposal is on almost zero probability mass under $N(0, I)$.

Furthermore, this method requires building good generative models which is generally a harder task than building a good discriminative model. How does this work on more complex datasets, such as CIFAR10 and CIFAR100, where $p(x|y, z)$ might not be as sharp? Furthermore, while non-iid settings are nice to have, to me it seems that this method might suffer in less non-iid / iid settings (e.g, $\alpha=1.0$, $\alpha=\inf$) ; the generative distribution is more diverse and thus it would be harder to model accurately with a, hardware constrained, VAE model. This can also sort of be seen when comparing FedCVAE-KD to FedCVAE-Ens; the former has worse performance that the latter, given that the entire training distribution needs to be modelled by a single model. How does the performance of the method scale with higher values of $\alpha$? Besides that, it seems that the evaluation against the baselines is not entirely apples-to-apples; for example, FedOneShot does not need to reveal to the server the labels and the models trained on the local data, so it has fundamentally different security / privacy tradeoffs.

The evidence to be evaluated is:In practice, we find it useful to focus on the highest density region of the prior and instead sample from a truncated standard normal distribution with tight symmetric bounds.

CVAEs use a prior distribution over latent space to train the encoder and decoder models.

To promote security, we propose to shift the center of the prior distribution \(\mathbf{\mu}\) to a random position in real space (i.e., \(\mathbf{\mu}\in\mathbb{R}^{d}\)), which can be communicated offline or via encryption methods between server and clients (Zhou et al., 2020). As shown in Figure 7 (Appendix B), sampling latent vectors too far from the center of the normal prior produces qualitatively poor data samples, deterring eavesdropping attackers who have no knowledge of \(\mathbf{\mu}\).

We verify the effectiveness of our proposed distribution shift extension for securing the uploaded information. Suppose an eavesdropping attacker is able to intercept the label distributions, decoder weights, and decoder architectures from all clients during upload. Without knowledge of the shared center of the multivariate normal prior \(\mathbf{\mu}\), we show that training a performant classifier is infeasible because it is difficult to extract high-quality samples from the client decoders. In particular, even when the attacker samples latent vectors \(\mathbf{z}\) from a broad region which overlaps with the high-density region of the prior (i.e., a uniform distribution centered on \(\mathbf{\mu}\)), the accuracy of the classifier trained on the resulting samples degrades sharply as the sampling region grows (Figure 5).

Therefore, this simple FedCVAE-Ens and FedCVAE-KD extension reduces eavesdropping attackers' capacity to extract high-quality samples from uploaded decoders or train a performant downstream model, reducing communication risks.

We are the first to apply CVAEs to one-shot FL with a focus on high statistical heterogeneity. In this paper, we proposed FedCVAE-Ens and FedCVAE-KD, data-free one-shot FL methods that reframe the local learning task using CVAEs. Both methods performed well given high statistical heterogeneity, demonstrated consistent performance with increasing numbers of clients, allow for model heterogeneity across clients, and can be extended to promote security.

During training, the CVAE will largely observe latent vectors in the highest density region of the prior distribution; for a standard normal distribution, this is near the center \(\mathbf{\mu}=\mathbf{0}\). Latent vectors distant from the center will not generate high-quality samples when used with the trained decoder. As a demonstration, we train a centralized CVAE and sample both close to the center of the prior (i.e., \(\mathbf{z}_{i}\sim\mathcal{U}(-1,1)\)) and distant from the center of the prior (i.e., \(\mathbf{z}_{i}\sim\mathcal{U}(5,20)\)).

Input: Figure 7: CVAE samples generated using latent vectors distant from the center of the multivariate normal prior (top row) and close to the center (bottom row).

Input: Figure 5: Results for our proposed security-promoting extension at high heterogeneity (α = 0.05). We show the accuracy for a classifier trained on samples from the intercepted client decoders and using client label distributions when the center of the normal prior is unknown. The sampling bounds represent the parameters of the uniform distribution used for latent vector sampling, i.e., ±100 represents the distribution U(−100, 100). The prior is a multivariate standard normalN (0, I).

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find the claim about protecting decoders through shifting the CVAE prior distribution center to be somewhat overstated. While the evidence shows that sampling far from the prior's center produces lower quality samples and reduces classifier accuracy (Fig 5), the reviewer raises valid concerns about the security guarantees. The evidence doesn't address potential attacks to infer the global mean or formal security proofs. The method provides some practical protection but may not be robust against sophisticated adversaries. The paper acknowledges this is a "simple extension" but could better address limitations.

<score>4</score>

Justification:
1. The experimental results show degraded sample quality and classifier performance when sampling away from the true prior center, supporting basic functionality
2. However, the reviewer correctly points out missing formal security analysis and potential vulnerabilities:
- No formal security guarantees against sophisticated attackers
- Possibility of inferring global mean through optimization or visual inspection
- The uniform distribution U[-10,10] is not an optimal approximation of N(0,1)
3. The paper presents this as a security method without fully addressing these limitations or providing rigorous security analysis
4. The claim would be better stated with more modest security assertions and clear acknowledgment of potential vulnerabilities


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Furthermore, to protect the decoders uploaded to the server, we propose a method to shift the center of the CVAE prior distribution.

The review comment to be evaluated is:

The paper is very well written and easy to follow. The numerical experiments are well executed and show promising performance achieved by the proposed methods.

That being said, there are some downsides that, in my opinion, slightly outweigh the strengths of the paper. On the problem formulation side, I believe the one-shot federated learning (FL) problem is not very well motivated; the point of FL is to enable training models at different clients by leveraging each others' data through communication without explicitly sharing them. In a one-shot scenario, the problem essentially reduces to having a set of models, which need to be aggregated into a single, universal model, so I am still not convinced how the "federated" aspect of the problem is of the essence.

On the contributions side, the main issue I have with the paper is a lack of a central, key novelty. In its current form, the paper is mostly a collection of different ideas, namely two algorithms that are almost independent of each other, as well as a security aspect, which is again mostly orthogonal to the proposed methods. To me, what would make this paper much stronger is to focus on only one of these algorithms/aspects throughout the paper. For example, since the performance of FEDCVAE-KD is better than baselines, I believe it would be better to focus on this algorithm in the main body of the paper, and add the ensemble version of the algorithm (which, as the authors allude to, requires a higher amount of memory that scales with the number of clients) as an extension, in a separate discussion section toward the end of the paper and/or in the appendix. Moreover, the security and privacy considerations are definitely important, but including them in the main body of the paper stretches it too thin, in my opinion.

Further comments:

- Aside from the solid empirical performance, are there any intuitive/theoretical explanations as to why the proposed methods are well suited for i) *one-shot* FL, and ii) *high heterogeneity* FL? As presented, I cannot see a clear motivation for why the proposed methods should perform better in the considered problem formulation.
- Please discuss whether or not sharing local label distributions with the server violates the privacy of clients' data. How do the proposed methods perform if this information is unavailable at the server?
- Presenting the flowchart figure for only one of the proposed methods (FEDCVAE-KD) in Figure 2 and not including any of the algorithms for the two proposed methods makes the paper slightly incoherent, without switching back and forth between the body of the paper and the appendix. I suggest rearranging the contents to remedy this issue as best as possible by at least having both the figure and algorithm for one of the methods or having the figures for both methods in the main body of the paper.
- In the first footnote on page 4, why is a uniform distribution used for sampling $\mathbf{z}$? Why not, for example, use a truncated Gaussian distribution?
- In my opinion, including FedAVG is slightly unfair in the experiments. Under FedAVG, a set of client model parameters $\\{\mathbf{w}^k\\}_{k=1}^m$ are aggregated using a simple average, i.e., $\mathbf{w}=\frac{1}{m} \sum_\{k=1\}^m\mathbf{w}^k$. In a very heterogeneous case, each of these model parameters is the minimizer of a very different objective function over an entirely different optimization landscape, so it is completely expected that a linear averaging of these model parameters should lead to inferior performance, especially if the averaging operation is done only once.
- Following the above comment, I believe your primary competing baseline is FedONESHOT, so I suggest adding more details on this algorithm's main idea.
- Do you have any explanation as to why the standard deviation of the performance of the proposed methods is, on average, larger under SVHN in Table 2?

The evidence to be evaluated is:In practice, we find it useful to focus on the highest density region of the prior and instead sample from a truncated standard normal distribution with tight symmetric bounds.

CVAEs use a prior distribution over latent space to train the encoder and decoder models.

To promote security, we propose to shift the center of the prior distribution \(\mathbf{\mu}\) to a random position in real space (i.e., \(\mathbf{\mu}\in\mathbb{R}^{d}\)), which can be communicated offline or via encryption methods between server and clients (Zhou et al., 2020). As shown in Figure 7 (Appendix B), sampling latent vectors too far from the center of the normal prior produces qualitatively poor data samples, deterring eavesdropping attackers who have no knowledge of \(\mathbf{\mu}\).

We verify the effectiveness of our proposed distribution shift extension for securing the uploaded information. Suppose an eavesdropping attacker is able to intercept the label distributions, decoder weights, and decoder architectures from all clients during upload. Without knowledge of the shared center of the multivariate normal prior \(\mathbf{\mu}\), we show that training a performant classifier is infeasible because it is difficult to extract high-quality samples from the client decoders. In particular, even when the attacker samples latent vectors \(\mathbf{z}\) from a broad region which overlaps with the high-density region of the prior (i.e., a uniform distribution centered on \(\mathbf{\mu}\)), the accuracy of the classifier trained on the resulting samples degrades sharply as the sampling region grows (Figure 5).

Therefore, this simple FedCVAE-Ens and FedCVAE-KD extension reduces eavesdropping attackers' capacity to extract high-quality samples from uploaded decoders or train a performant downstream model, reducing communication risks.

We are the first to apply CVAEs to one-shot FL with a focus on high statistical heterogeneity. In this paper, we proposed FedCVAE-Ens and FedCVAE-KD, data-free one-shot FL methods that reframe the local learning task using CVAEs. Both methods performed well given high statistical heterogeneity, demonstrated consistent performance with increasing numbers of clients, allow for model heterogeneity across clients, and can be extended to promote security.

During training, the CVAE will largely observe latent vectors in the highest density region of the prior distribution; for a standard normal distribution, this is near the center \(\mathbf{\mu}=\mathbf{0}\). Latent vectors distant from the center will not generate high-quality samples when used with the trained decoder. As a demonstration, we train a centralized CVAE and sample both close to the center of the prior (i.e., \(\mathbf{z}_{i}\sim\mathcal{U}(-1,1)\)) and distant from the center of the prior (i.e., \(\mathbf{z}_{i}\sim\mathcal{U}(5,20)\)).

Input: Figure 7: CVAE samples generated using latent vectors distant from the center of the multivariate normal prior (top row) and close to the center (bottom row).

Input: Figure 5: Results for our proposed security-promoting extension at high heterogeneity (α = 0.05). We show the accuracy for a classifier trained on samples from the intercepted client decoders and using client label distributions when the center of the normal prior is unknown. The sampling bounds represent the parameters of the uniform distribution used for latent vector sampling, i.e., ±100 represents the distribution U(−100, 100). The prior is a multivariate standard normalN (0, I).

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll evaluate the claim: "Furthermore, to protect the decoders uploaded to the server, we propose a method to shift the center of the CVAE prior distribution."

Summary:
The evidence strongly supports this claim with both theoretical explanation and empirical results. The authors demonstrate:
1. Visual evidence (Figure 7) showing poor sample quality when sampling far from the prior center
2. Quantitative results (Figure 5) showing significant accuracy degradation as sampling bounds increase
3. Clear explanation of the security mechanism through shifting the prior center μ
4. Practical implementation details including offline communication of μ

The claim is well-stated and accurately reflects the evidence without exaggeration. The method is clearly described and its effectiveness is demonstrated through multiple experiments.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Furthermore, to protect the decoders uploaded to the server, we propose a method to shift the center of the CVAE prior distribution.

The review comment to be evaluated is:

### Strength

* The proposed method of one-shot FL is very interesting and novel.
* The proposed method doesn't require auxiliary public dataset.
* Both security and privacy extensions are considered in the paper.
* It can be applied to heterogeneous local models with similar generative capabilities.

### Weaknesses
* One potential issue of the method is training a CVAE locally. In the paper, the images used in the dataset are very small, and the amount of data each client has is large. However, in the real-world application of FL like medical imagings, with very large images and very limited number of data samples, training a CVAE locally can be difficult even just for one class of data.
* The training task of local models is different from the original task. This limits the re-use of pre-trained local model from original task. I think one of the reasons that people want to do one-shot FL is to prevent any re-training locally. But the proposed method still requires to train some new model.

The evidence to be evaluated is:In practice, we find it useful to focus on the highest density region of the prior and instead sample from a truncated standard normal distribution with tight symmetric bounds.

CVAEs use a prior distribution over latent space to train the encoder and decoder models.

To promote security, we propose to shift the center of the prior distribution \(\mathbf{\mu}\) to a random position in real space (i.e., \(\mathbf{\mu}\in\mathbb{R}^{d}\)), which can be communicated offline or via encryption methods between server and clients (Zhou et al., 2020). As shown in Figure 7 (Appendix B), sampling latent vectors too far from the center of the normal prior produces qualitatively poor data samples, deterring eavesdropping attackers who have no knowledge of \(\mathbf{\mu}\).

We verify the effectiveness of our proposed distribution shift extension for securing the uploaded information. Suppose an eavesdropping attacker is able to intercept the label distributions, decoder weights, and decoder architectures from all clients during upload. Without knowledge of the shared center of the multivariate normal prior \(\mathbf{\mu}\), we show that training a performant classifier is infeasible because it is difficult to extract high-quality samples from the client decoders. In particular, even when the attacker samples latent vectors \(\mathbf{z}\) from a broad region which overlaps with the high-density region of the prior (i.e., a uniform distribution centered on \(\mathbf{\mu}\)), the accuracy of the classifier trained on the resulting samples degrades sharply as the sampling region grows (Figure 5).

Therefore, this simple FedCVAE-Ens and FedCVAE-KD extension reduces eavesdropping attackers' capacity to extract high-quality samples from uploaded decoders or train a performant downstream model, reducing communication risks.

We are the first to apply CVAEs to one-shot FL with a focus on high statistical heterogeneity. In this paper, we proposed FedCVAE-Ens and FedCVAE-KD, data-free one-shot FL methods that reframe the local learning task using CVAEs. Both methods performed well given high statistical heterogeneity, demonstrated consistent performance with increasing numbers of clients, allow for model heterogeneity across clients, and can be extended to promote security.

During training, the CVAE will largely observe latent vectors in the highest density region of the prior distribution; for a standard normal distribution, this is near the center \(\mathbf{\mu}=\mathbf{0}\). Latent vectors distant from the center will not generate high-quality samples when used with the trained decoder. As a demonstration, we train a centralized CVAE and sample both close to the center of the prior (i.e., \(\mathbf{z}_{i}\sim\mathcal{U}(-1,1)\)) and distant from the center of the prior (i.e., \(\mathbf{z}_{i}\sim\mathcal{U}(5,20)\)).

Input: Figure 7: CVAE samples generated using latent vectors distant from the center of the multivariate normal prior (top row) and close to the center (bottom row).

Input: Figure 5: Results for our proposed security-promoting extension at high heterogeneity (α = 0.05). We show the accuracy for a classifier trained on samples from the intercepted client decoders and using client label distributions when the center of the normal prior is unknown. The sampling bounds represent the parameters of the uniform distribution used for latent vector sampling, i.e., ±100 represents the distribution U(−100, 100). The prior is a multivariate standard normalN (0, I).

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find this claim to be well-stated based on several key pieces of evidence:

1. The paper demonstrates through Figure 7 that sampling from regions far from the prior's center produces poor quality samples
2. Figure 5 provides quantitative evidence showing sharp accuracy degradation when sampling outside the known prior distribution center
3. The evidence includes clear mathematical formulation of shifting the center μ to a random position
4. The review comments don't raise any concerns about this specific security aspect

The claim about proposing a method to shift the CVAE prior distribution center is measured and accurately reflects what the evidence demonstrates. The security benefits are supported by experimental results showing degraded sample quality and classifier performance when the center is unknown.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We show that without knowing the center of the prior, an eavesdropping attacker cannot train a performant classifier, thus promoting pipeline security.

The evidence to be evaluated is:We jointly propose FedCVAE-Ens and FedCVAE-KD, one-shot FL methods that do not require an auxiliary public dataset for server-side training (they are _data-free_).

We discuss privacy- and security-promoting extensions in Appendix B and Section 3.3, respectively. FedCVAE-KD trains a CVAE \(f_{\mathbf{w}^{k}}(\cdot)\) for every client \(k\in\mathcal{C}\) to convergence on their private local dataset by solving Equation 1; this CVAE is parameterized by \(\mathbf{w}^{k}=[\phi^{k},\mathbf{\theta}^{k}]\), with encoder \(E_{\mathbf{\phi}^{k}}(\cdot)\) and decoder \(D_{\mathbf{\theta}^{k}}(\cdot)\). Then, clients communicate their decoder weights \(\mathbf{\theta}^{k}\) and label distributions \(\bar{p}^{k}(\mathbf{y})\) to the server; in practice, this simply requires upload of client label counts as in Zhu et al. (2021). This completes the single communication round.

Then, to sample from client decoders, we sample a latent vector from the prior (i.e., \(\mathbf{z}\sim\mathcal{N}(\mathbf{0},\mathbf{I})\))1 and obtain synthetic data sample \(i\) from client \(k\) using \(\hat{\mathbf{x}}_{i}^{k}=D_{\mathbf{\theta}^{k}}(\mathbf{z}_{i}^{k};\mathbf{y}_{i }^{k})\).

In practice, we find it useful to focus on the highest density region of the prior and instead sample from a truncated standard normal distribution with tight symmetric bounds. The trained client decoders act as the teacher models, conveying their aggregate knowledge of how to map from latent space to data space to a single student server decoder, parameterized by \(\mathbf{\theta}^{S}\).

We define a secure pipeline as one where an outside attacker who obtains transferred data cannot train a performant classifier (Zhou et al., 2020). In the case of FedCVAE-Ens and FedCVAE-KD, an attacker who intercepts all client decoders and local label distributions should not be able to generate the high quality samples necessary to train a high quality classifier.

To promote security, we propose to shift the center of the prior distribution \(\mathbf{\mu}\) to a random position in real space (i.e., \(\mathbf{\mu}\in\mathbb{R}^{d}\)), which can be communicated offline or via encryption methods between server and clients (Zhou et al., 2020). As shown in Figure 7 (Appendix B), sampling latent vectors too far from the center of the normal prior produces qualitatively poor data samples, deterring eavesdropping attackers who have no knowledge of \(\mathbf{\mu}\).

The one-shot methods proposed in Li et al. (2021) and Shin et al. (2020) are not applicable because of their reliance on public auxiliary data for server-side training or fine-tuning. Similarly, many standard FL methods are not appropriate because they depend on an auxiliary dataset (i.e., FedDF (Lin et al., 2020)) or focus on regularization (i.e., FedGen (Zhu et al., 2021), FedProx (Li et al., 2020), Scaffold (Karimireddy et al., 2020), FedNova (Wang et al., 2020)), which is incompatible with the one-shot setting.

**Promoting Security.** We verify the effectiveness of our proposed distribution shift extension for securing the uploaded information. Suppose an eavesdropping attacker is able to intercept the label distributions, decoder weights, and decoder architectures from all clients during upload. Without knowledge of the shared center of the multivariate normal prior \(\mathbf{\mu}\), we show that training a performant classifier is infeasible because it is difficult to extract high-quality samples from the client decoders. In particular, even when the attacker samples latent vectors \(\mathbf{z}\) from a broad region which overlaps with the high-density region of the prior (i.e., a uniform distribution centered on \(\mathbf{\mu}\)), the accuracy of the classifier trained on the resulting samples degrades sharply as the sampling region grows (Figure 5). Even a good guess of \(\mathcal{U}(-10,10)\) for a normal prior of \(\mathcal{N}(\mathbf{0},\mathbf{I})\) results in a \(3-30\%\) point decrease in accuracy depending on the dataset. A more realistic guess of \(\mathcal{U}(-1000,1000)\) results in a \(33-67\%\) point decrease. Therefore, this simple FedCVAE-Ens and FedCVAE-KD extension reduces eavesdropping attackers' capacity to extract high-quality samples from uploaded decoders or train a performant downstream model, reducing communication risks.

In this paper, we proposed FedCVAE-Ens and FedCVAE-KD, data-free one-shot FL methods that reframe the local learning task using CVAEs. Both methods performed well given high statistical heterogeneity, demonstrated consistent performance with increasing numbers of clients, allow for model heterogeneity across clients, and can be extended to promote security. Extensive experimental results showed that FedCVAE-Ens and FedCVAE-KD substantially extended the state-of-the-art in one-shot FL under very high statistical heterogeneity, making way for more nuanced development in this difficult environment for distributed learning.

During training, the CVAE will largely observe latent vectors in the highest density region of the prior distribution; for a standard normal distribution, this is near the center \(\mathbf{\mu}=\mathbf{0}\). Latent vectors distant from the center will not generate high-quality samples when used with the trained decoder. As a demonstration, we train a centralized CVAE and sample both close to the center of the prior (i.e., \(\mathbf{z}_{i}\sim\mathcal{U}(-1,1)\)) and distant from the center of the prior (i.e., \(\mathbf{z}_{i}\sim\mathcal{U}(5,20)\)).

Uploading client label distributions \(\hat{p}^{k}(\mathbf{y})\) may generate additional privacy concerns. One potential solution is to mask the precise label counts for each client by adding noise before upload as in Zhang et al. (2022); to achieve this, we draw noise from a normal distribution \(\epsilon_{c}\sim\mathcal{N}(0,\gamma\cdot n_{k})\) such that the "strength" (variance) of the noise applied to class \(c\) is in proportion \(\gamma\) to the total number of training samples for client \(k\). Noise is applied to the training sample count for each class for a given client before uploading this information to the server.

As \(\gamma\to\infty\), information from the uploaded label distribution disappears; when \(\gamma=0\), the exact local label distributions are communicated. When adding a modest amount of noise to the client label distributions (i.e., \(\gamma\leq 0.1\)), accuracy for both FedCVAE-Ens and FedCVAE-KD is barely affected compared to upload with no noise; see Table 8 for \(\gamma>0\) and refer to the \(\alpha=0.01\) in Table 1 for \(\gamma=0\). The notion of uploading and harnessing client label distributions in FL is new (Zhu et al., 2021; Zhang et al., 2022) and quantifying the privacy risks that label distributions might induce is an open problem which could benefit from focused development.

Input: Figure 7: CVAE samples generated using latent vectors distant from the center of the multivariate normal prior (top row) and close to the center (bottom row).

Input: Figure 5: Results for our proposed security-promoting extension at high heterogeneity (α = 0.05). We show the accuracy for a classifier trained on samples from the intercepted client decoders and using client label distributions when the center of the normal prior is unknown. The sampling bounds represent the parameters of the uniform distribution used for latent vector sampling, i.e., ±100 represents the distribution U(−100, 100). The prior is a multivariate standard normalN (0, I).

Input: Table 8: Performance of our proposed methods with noise added to the uploaded client label distributions. Results show the average test accuracy across 5 seeds for the random noise ± one standard deviation.

Input: Figure 9: The effect of the noise proportion γ on an example data partition at α = 0.01, m = 10 clients, and on MNIST. The original partition is in yellow (γ = 0). The size of each dot is proportional to the number of samples.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Let me assess the claim: "We show that without knowing the center of the prior, an eavesdropping attacker cannot train a performant classifier, thus promoting pipeline security."

Summary: The evidence provides strong support for this claim through empirical results shown in Figure 5, demonstrating significant accuracy degradation (33-67% point decrease) when attackers sample from incorrect prior distributions. However, the claim could be more precisely stated since some classifier performance is still possible, just significantly degraded. The results show attackers can still achieve some accuracy (20-50%) even with incorrect priors, so complete prevention isn't achieved, though substantial security benefits are demonstrated.

<score>2</score>

Justification:
1. The evidence directly demonstrates classifier performance degradation through:
- Quantitative results showing 3-30% accuracy drops with modest prior misspecification
- More severe 33-67% accuracy drops with larger prior misspecification
- Visual evidence in Figure 7 showing poor sample quality with incorrect priors

2. The claim is slightly overstated by implying attackers "cannot train a performant classifier" when the evidence shows they can still achieve some performance, just significantly degraded.

3. The evidence thoroughly documents the security mechanism and its effectiveness across multiple datasets and sampling conditions, providing strong empirical support for the core security benefit claim.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We show that without knowing the center of the prior, an eavesdropping attacker cannot train a performant classifier, thus promoting pipeline security.

The review comment to be evaluated is:

Strength:

-- The proposed solutions seem effective under the one-shot FL setting. The efficacy and efficiency are both demonstrated in the extensive experiments.
-- The writing is clear and easy to follow. The tables and figures are well-organized.

Weaknesses:

-- Constructing a generative model to generate samples for FL model training is not new. There are a few works utilizing a similar strategy in FL, for example:
Data-Free Knowledge Distillation for Heterogeneous Federated Learning, ICML 2021
Fine-tuning global model via data-free knowledge distillation for non-iid federated learning, CVPR 2022
Please make a comparison with those works

-- Some details are not clear enough for me. For example, in Eq 2, what is the formulation for g(.)?  Is this dissimilarity measured in the data-space or feature space?

The evidence to be evaluated is:We jointly propose FedCVAE-Ens and FedCVAE-KD, one-shot FL methods that do not require an auxiliary public dataset for server-side training (they are _data-free_).

We discuss privacy- and security-promoting extensions in Appendix B and Section 3.3, respectively. FedCVAE-KD trains a CVAE \(f_{\mathbf{w}^{k}}(\cdot)\) for every client \(k\in\mathcal{C}\) to convergence on their private local dataset by solving Equation 1; this CVAE is parameterized by \(\mathbf{w}^{k}=[\phi^{k},\mathbf{\theta}^{k}]\), with encoder \(E_{\mathbf{\phi}^{k}}(\cdot)\) and decoder \(D_{\mathbf{\theta}^{k}}(\cdot)\). Then, clients communicate their decoder weights \(\mathbf{\theta}^{k}\) and label distributions \(\bar{p}^{k}(\mathbf{y})\) to the server; in practice, this simply requires upload of client label counts as in Zhu et al. (2021). This completes the single communication round.

Then, to sample from client decoders, we sample a latent vector from the prior (i.e., \(\mathbf{z}\sim\mathcal{N}(\mathbf{0},\mathbf{I})\))1 and obtain synthetic data sample \(i\) from client \(k\) using \(\hat{\mathbf{x}}_{i}^{k}=D_{\mathbf{\theta}^{k}}(\mathbf{z}_{i}^{k};\mathbf{y}_{i }^{k})\).

In practice, we find it useful to focus on the highest density region of the prior and instead sample from a truncated standard normal distribution with tight symmetric bounds. The trained client decoders act as the teacher models, conveying their aggregate knowledge of how to map from latent space to data space to a single student server decoder, parameterized by \(\mathbf{\theta}^{S}\).

We define a secure pipeline as one where an outside attacker who obtains transferred data cannot train a performant classifier (Zhou et al., 2020). In the case of FedCVAE-Ens and FedCVAE-KD, an attacker who intercepts all client decoders and local label distributions should not be able to generate the high quality samples necessary to train a high quality classifier.

To promote security, we propose to shift the center of the prior distribution \(\mathbf{\mu}\) to a random position in real space (i.e., \(\mathbf{\mu}\in\mathbb{R}^{d}\)), which can be communicated offline or via encryption methods between server and clients (Zhou et al., 2020). As shown in Figure 7 (Appendix B), sampling latent vectors too far from the center of the normal prior produces qualitatively poor data samples, deterring eavesdropping attackers who have no knowledge of \(\mathbf{\mu}\).

The one-shot methods proposed in Li et al. (2021) and Shin et al. (2020) are not applicable because of their reliance on public auxiliary data for server-side training or fine-tuning. Similarly, many standard FL methods are not appropriate because they depend on an auxiliary dataset (i.e., FedDF (Lin et al., 2020)) or focus on regularization (i.e., FedGen (Zhu et al., 2021), FedProx (Li et al., 2020), Scaffold (Karimireddy et al., 2020), FedNova (Wang et al., 2020)), which is incompatible with the one-shot setting.

**Promoting Security.** We verify the effectiveness of our proposed distribution shift extension for securing the uploaded information. Suppose an eavesdropping attacker is able to intercept the label distributions, decoder weights, and decoder architectures from all clients during upload. Without knowledge of the shared center of the multivariate normal prior \(\mathbf{\mu}\), we show that training a performant classifier is infeasible because it is difficult to extract high-quality samples from the client decoders. In particular, even when the attacker samples latent vectors \(\mathbf{z}\) from a broad region which overlaps with the high-density region of the prior (i.e., a uniform distribution centered on \(\mathbf{\mu}\)), the accuracy of the classifier trained on the resulting samples degrades sharply as the sampling region grows (Figure 5). Even a good guess of \(\mathcal{U}(-10,10)\) for a normal prior of \(\mathcal{N}(\mathbf{0},\mathbf{I})\) results in a \(3-30\%\) point decrease in accuracy depending on the dataset. A more realistic guess of \(\mathcal{U}(-1000,1000)\) results in a \(33-67\%\) point decrease. Therefore, this simple FedCVAE-Ens and FedCVAE-KD extension reduces eavesdropping attackers' capacity to extract high-quality samples from uploaded decoders or train a performant downstream model, reducing communication risks.

In this paper, we proposed FedCVAE-Ens and FedCVAE-KD, data-free one-shot FL methods that reframe the local learning task using CVAEs. Both methods performed well given high statistical heterogeneity, demonstrated consistent performance with increasing numbers of clients, allow for model heterogeneity across clients, and can be extended to promote security. Extensive experimental results showed that FedCVAE-Ens and FedCVAE-KD substantially extended the state-of-the-art in one-shot FL under very high statistical heterogeneity, making way for more nuanced development in this difficult environment for distributed learning.

During training, the CVAE will largely observe latent vectors in the highest density region of the prior distribution; for a standard normal distribution, this is near the center \(\mathbf{\mu}=\mathbf{0}\). Latent vectors distant from the center will not generate high-quality samples when used with the trained decoder. As a demonstration, we train a centralized CVAE and sample both close to the center of the prior (i.e., \(\mathbf{z}_{i}\sim\mathcal{U}(-1,1)\)) and distant from the center of the prior (i.e., \(\mathbf{z}_{i}\sim\mathcal{U}(5,20)\)).

Uploading client label distributions \(\hat{p}^{k}(\mathbf{y})\) may generate additional privacy concerns. One potential solution is to mask the precise label counts for each client by adding noise before upload as in Zhang et al. (2022); to achieve this, we draw noise from a normal distribution \(\epsilon_{c}\sim\mathcal{N}(0,\gamma\cdot n_{k})\) such that the "strength" (variance) of the noise applied to class \(c\) is in proportion \(\gamma\) to the total number of training samples for client \(k\). Noise is applied to the training sample count for each class for a given client before uploading this information to the server.

As \(\gamma\to\infty\), information from the uploaded label distribution disappears; when \(\gamma=0\), the exact local label distributions are communicated. When adding a modest amount of noise to the client label distributions (i.e., \(\gamma\leq 0.1\)), accuracy for both FedCVAE-Ens and FedCVAE-KD is barely affected compared to upload with no noise; see Table 8 for \(\gamma>0\) and refer to the \(\alpha=0.01\) in Table 1 for \(\gamma=0\). The notion of uploading and harnessing client label distributions in FL is new (Zhu et al., 2021; Zhang et al., 2022) and quantifying the privacy risks that label distributions might induce is an open problem which could benefit from focused development.

Input: Figure 7: CVAE samples generated using latent vectors distant from the center of the multivariate normal prior (top row) and close to the center (bottom row).

Input: Figure 5: Results for our proposed security-promoting extension at high heterogeneity (α = 0.05). We show the accuracy for a classifier trained on samples from the intercepted client decoders and using client label distributions when the center of the normal prior is unknown. The sampling bounds represent the parameters of the uniform distribution used for latent vector sampling, i.e., ±100 represents the distribution U(−100, 100). The prior is a multivariate standard normalN (0, I).

Input: Table 8: Performance of our proposed methods with noise added to the uploaded client label distributions. Results show the average test accuracy across 5 seeds for the random noise ± one standard deviation.

Input: Figure 9: The effect of the noise proportion γ on an example data partition at α = 0.01, m = 10 clients, and on MNIST. The original partition is in yellow (γ = 0). The size of each dot is proportional to the number of samples.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Let me evaluate the claim: "We show that without knowing the center of the prior, an eavesdropping attacker cannot train a performant classifier, thus promoting pipeline security."

Summary: The evidence substantiates this claim but with some important nuance. Figure 5 shows that classifier accuracy degrades as sampling bounds increase, but doesn't completely fail - even with broad sampling bounds (±1000), the classifier still achieves 20-30% accuracy. The text notes a "3-30% point decrease" with moderate bounds and "33-67% point decrease" with larger bounds. While security is promoted, stating the attacker "cannot train a performant classifier" is somewhat overstated since some level of performance remains possible.

<score>3</score>

Justification:
1. The evidence demonstrates degraded but not completely prevented classifier performance
2. The experimental results show varying levels of degradation across datasets
3. The claim would be better stated as "significantly reduces an attacker's ability to train a performant classifier" rather than implying complete prevention
4. The reviewer comments don't specifically address this security claim
5. More discussion of what constitutes "performant" would strengthen the claim