Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Our extensive experiments show that across a variety of natural conditions in twelve distinct datasets, classifiers trained with our algorithms significantly outperform classifiers trained via ERM, adversarial training, and domain adaptation techniques.

The review comment to be evaluated is:

This paper proposes a model-based framework for improving the robustness of image classifiers to average-case corruptions of varying severity. The proposed framework can be thought of as adversarial training where the perturbation is replaced by a function that transforms the image according to a specific corruption. A nuisance parameter controls the instantiation and severity of the corruption that is applied to the input. The paper compares baselines to different versions of this general model-based framework with experiments on several datasets.

Pros:
- Focusing on average-case corruptions is an important and underexplored problem
- Using image translation to learn the corruptions is an interesting proposal

Cons:
- There isn't much novelty in the proposed technique. Adversarial training with 'natural' corruptions has been done several times before.
- There isn't any discussion of how the nuisance parameter is used. I suspect it isn't used for ImageNet-C experiments, though I may be wrong. In this case, the method becomes data augmentation with image translation, which is not very novel either.
- It sounds like the ImageNet-C experiments use the corruptions to train the image translation network. Even though these experiments use held-out images, the corruption types themselves are not being held out, which goes against the recommended methodology for that dataset.

Typo: "ImageNet-c" should be "ImageNet-C"

___________________________________________________________________________________

Update after author feedback:

I thank the authors for improving my understanding of the paper. I feel better about it after reading it again. One of the most interesting findings--that a translation network trained on one domain/set of classes generalizes to another--needs more discussion. Using this phenomenon to improve robustness is a good idea, and the MRT/MDA/MAT methods explored in this work are nice choices for this investigation. However, I agree with the other reviewers that it would be nice to include more baselines from other work where appropriate.

The results in Table 1 are also very interesting and deserve more discussion--perhaps an analysis of whether a translation network trained on weak corruptions can generalize to create something akin to the ground-truth stronger corruptions, as the results in the table for MRT imply.

Overall, I think this paper has some strong experiments and investigates a good idea, but the claims of a "paradigm shift" are overly grandiose, and some of the most interesting experiments could use more analysis. Additionally, the writing clarity and presentation could be improved. My initial understanding of the paper was flawed in some places, so I'll raise my score to 5.

The evidence to be evaluated is:We present experiments in five different settings over twelve distinct datasets to demonstrate the broad applicability of MBRDL. First, in Sections 5.1, 5.2, we show that our algorithms are the first to consistently provide out-of-distribution robustness across a range of challenging corruptions, including shifts in brightness, contrast, snow, fog, frost, and haze on CURE-TSR, ImageNet, and ImageNet-c. In Section 5.3 we curate several new datasets containing simultaneous sources of natural variation, and we then show that models of natural variation can be composed to provide robustness against these simultaneous shifts. In Section 5.4 we show that models of natural variation trained on a fixed dataset can be reused to provide robustness on datasets entirely unseen during training. Finally, in Section 5.5 we assume access to unlabeled data corresponding to a fixed domain shift, and we compare our algorithms to suitable baselines, including domain adaptation methods.

In such cases, we show that our algorithms can be used to provide significant out-of-distribution robustness against data with high levels of natural variation by training on data with relatively low levels of the same source of natural variation. To do so, we use data from the CURE-TSR dataset (Hemel et al., 2019), which contains images of street signs divided into subsets according to various sources of natural variation and corresponding severity levels.

Thus, for each row of Table 1 we use unlabeled data from levels 0 and 2 to learn a model of natural variation corresponding to a given source of natural variation in CURE-TSR.

Note that while this is an unfair comparison for our methods, given that the model-based algorithms are not given access to labeled level 2 data, our algorithms still outperform the baselines by as much as 20 percentage points on level 5 data. To demonstrate the scalability of our approach, we perform experiments on ImageNet (Deng et al., 2009) and the recently-curated ImageNet-c dataset (Hendrycks and Dietterich, 2019). ImageNet-c contains images from the ImageNet test set that are corrupted according to artificial transformations, such as snow, rain, and fog, and are labeled from 1-5 depending on the severity of the corruption. For numerous challenging corruptions, we train models to map from the classes 0-9 of ImageNet to the corresponding classes of ImageNet-c. We then train all networks on classes 10-59 of ImageNet, and test on the corresponding classes for various subsets of ImageNet-c. Note that in this setting, the ImageNet classes used to train the model of natural variation are disjoint from those that are used to train the classifier, so many techniques, including most domain adaptation methods, do not apply; to offer a point of comparison, we include the accuracies of classifiers trained using AugMix, which is a recently proposed method that adds known transformations to the data (Hendrycks et al., 2019).

To address this robustness challenge, Because we learn models of natural variation offline before training a classifier, our paradigm can be applied to domains that are _entirely unseen_ while training the model. In particular, we show that models can be reused on similar yet unseen datasets to provide robustness against a common source of natural variation.

Whereas many techniques, including most domain adaptation methods, do not apply in this scenario, in the MBRDL paradigm, we can simply learn a model corresponding to the changes in lighting for the European street signs and then apply this model to the dataset of the American signs. Table 4 shows several experiments of this stripe in which a model \(G\) is learned on one dataset \(\mathcal{D}_{1}\) and then applied on another \(\mathcal{D}_{2}\); we improve robustness on unseen domains by up to 40 percentage points.

We then train classifiers using our algorithms, as well with ERM and PGD, using data from domain \(A\) and test on data from the test set for domain \(B\). Furthermore, we compare to ADDA, which is a well-known domain adaptation method (Tzeng et al., 2017). In every scenario, our model-based algorithms significantly outperform the baselines, often by 10-20 percentage points.

Unlike these approaches, our solution does not assume access to unlabeled data from a fixed shift and can be applied to datasets that are entirely unseen during training.

In our work, we propose a more general framework and three novel robust training algorithms that can exploit any suitable generative network, and we show improvements on more challenging, naturally-occurring shifts across twelve distinct datasets.

In this paper, we proposed a novel _model-based robust training paradigm_ for deep learning that provides robustness with respect to models of natural variation.

In our experiments, we show that our paradigm can provide significant out-of-distribution robustness on many challenging distributional shifts. Furthermore, our paradigm can provide robustness against multiple simultaneous distribution shifts and on domains that are entirely unseen while training the model, and shows significant out-of-distribution robustness as datasets become more challenging.

Input: Table 1: Out-of-distribution robustness. In each experiment, we train a model of natural variation to map from challenge-level 0 to challenge-level 2 data from different subsets of CURE-TSR. We then perform model-based training using challenge-level 0 data and test on challenge-levels 3-5.

Input: Table 4: Transferability of model-based robustness. In each experiment, we train a model of natural variation on a given training dataset D1. Then, we use this model to perform model-based training on a new dataset D1 entirely unseen during the training of the model.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find the claim somewhat overstated for several reasons:

1. While the evidence shows improvements over baselines in many cases, the claim of "significantly outperforming" across "all natural conditions" is too strong. Some improvements are modest, and there are cases where performance is similar or worse (e.g., Table 1 snow level 5 shows 70.7% vs baseline 60.9%).

2. The review points out important limitations not acknowledged in the claim:
- Limited novelty in the technique
- Methodological concerns with ImageNet-C experiments
- Need for more baselines
- Claims of "paradigm shift" being "overly grandiose"

3. The evidence primarily compares against ERM and PGD, with limited comparison to domain adaptation techniques, making the broad claim about outperforming "domain adaptation techniques" not fully supported.

<score>4</score>

Justification:
- The results do show improvements in many scenarios, but the claim overstates the consistency and magnitude of improvements
- Language like "significantly outperform" and "across a variety of natural conditions" implies universal superiority not fully supported by the evidence
- Lack of acknowledgment of limitations and methodological concerns raised in the review
- Limited comparison against the full range of claimed baseline techniques

Direct citations:
- Review notes "claims of a 'paradigm shift' are overly grandiose"
- Table 4 shows mixed results, with some modest improvements and some cases of similar performance
- Review mentions "It would be nice to include more baselines from other work where appropriate"


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Our extensive experiments show that across a variety of natural conditions in twelve distinct datasets, classifiers trained with our algorithms significantly outperform classifiers trained via ERM, adversarial training, and domain adaptation techniques.

The review comment to be evaluated is:

The paper looks at the idea of building models of natural variation of an input and then using these models to develop robust training algorithms that are less susceptible to outliers ( testable for worst-cases via adversarial attacks ).

Strength:

+ The paper addresses a very important topic of adversarial robustness of DNN models and is accompanied by diligent evaluation over different datasets.

Weakness:

- The paper adopts an approach which is very reminiscent of Ajil Jalal, Andrew Ilyas, Constantinos Daskalakis, and Alexandros G Dimakis. The robust manifold defense: Adversarial training using generative models. arXiv preprint arXiv:1712.09196, 2017. The key difference seems to be adoption of the idea of using auxiliary transformations (called natural perturbations) which is also very well-studied in literature, for e.g. see https://openaccess.thecvf.com/content_CVPR_2020/papers/Zhang_Auxiliary_Training_Towards_Accurate_and_Robust_Models_CVPR_2020_paper.pdf  So, the approach presented here is quite incremental for a premier venue such as ICLR.

- The attack in Jalal et. al. paper can be launched against this approach once the attacker also access to this generative/natural model (which would be easy to build for an attacker)

- The reviewer will strongly recommend reviewing the advices in https://arxiv.org/abs/1902.06705 for writing papers on defense and how to self-evaluate its robustness by suitably designing the attacker. If the defense approach uses some background/auxiliary knowledge, one must consider the attacker with this knowledge if it is accessible to the attacker.

Questions to authors:

- Can authors explain why an attacker can't build similar natural model to defeat the proposed defense?
- Any clarification on incremental novelty from  Jalal et. al. and Zhang et. al. (and references therein) would be also useful.

After author's rebuttal:

"The other paper that the review points to ([4]) addresses a similar setting to our paper, but the approach is completely different. ... Moreover, [4] was published within three months of the ICLR submission deadline, meaning that it is essentially concurrent with our work."

Yes, the reviewer concurs that a work published so close to deadline should be treated as concurrent and will raise the score.

The evidence to be evaluated is:We present experiments in five different settings over twelve distinct datasets to demonstrate the broad applicability of MBRDL. First, in Sections 5.1, 5.2, we show that our algorithms are the first to consistently provide out-of-distribution robustness across a range of challenging corruptions, including shifts in brightness, contrast, snow, fog, frost, and haze on CURE-TSR, ImageNet, and ImageNet-c. In Section 5.3 we curate several new datasets containing simultaneous sources of natural variation, and we then show that models of natural variation can be composed to provide robustness against these simultaneous shifts. In Section 5.4 we show that models of natural variation trained on a fixed dataset can be reused to provide robustness on datasets entirely unseen during training. Finally, in Section 5.5 we assume access to unlabeled data corresponding to a fixed domain shift, and we compare our algorithms to suitable baselines, including domain adaptation methods.

In such cases, we show that our algorithms can be used to provide significant out-of-distribution robustness against data with high levels of natural variation by training on data with relatively low levels of the same source of natural variation. To do so, we use data from the CURE-TSR dataset (Hemel et al., 2019), which contains images of street signs divided into subsets according to various sources of natural variation and corresponding severity levels.

Thus, for each row of Table 1 we use unlabeled data from levels 0 and 2 to learn a model of natural variation corresponding to a given source of natural variation in CURE-TSR.

Note that while this is an unfair comparison for our methods, given that the model-based algorithms are not given access to labeled level 2 data, our algorithms still outperform the baselines by as much as 20 percentage points on level 5 data. To demonstrate the scalability of our approach, we perform experiments on ImageNet (Deng et al., 2009) and the recently-curated ImageNet-c dataset (Hendrycks and Dietterich, 2019). ImageNet-c contains images from the ImageNet test set that are corrupted according to artificial transformations, such as snow, rain, and fog, and are labeled from 1-5 depending on the severity of the corruption. For numerous challenging corruptions, we train models to map from the classes 0-9 of ImageNet to the corresponding classes of ImageNet-c. We then train all networks on classes 10-59 of ImageNet, and test on the corresponding classes for various subsets of ImageNet-c. Note that in this setting, the ImageNet classes used to train the model of natural variation are disjoint from those that are used to train the classifier, so many techniques, including most domain adaptation methods, do not apply; to offer a point of comparison, we include the accuracies of classifiers trained using AugMix, which is a recently proposed method that adds known transformations to the data (Hendrycks et al., 2019).

To address this robustness challenge, Because we learn models of natural variation offline before training a classifier, our paradigm can be applied to domains that are _entirely unseen_ while training the model. In particular, we show that models can be reused on similar yet unseen datasets to provide robustness against a common source of natural variation.

Whereas many techniques, including most domain adaptation methods, do not apply in this scenario, in the MBRDL paradigm, we can simply learn a model corresponding to the changes in lighting for the European street signs and then apply this model to the dataset of the American signs. Table 4 shows several experiments of this stripe in which a model \(G\) is learned on one dataset \(\mathcal{D}_{1}\) and then applied on another \(\mathcal{D}_{2}\); we improve robustness on unseen domains by up to 40 percentage points.

We then train classifiers using our algorithms, as well with ERM and PGD, using data from domain \(A\) and test on data from the test set for domain \(B\). Furthermore, we compare to ADDA, which is a well-known domain adaptation method (Tzeng et al., 2017). In every scenario, our model-based algorithms significantly outperform the baselines, often by 10-20 percentage points.

Unlike these approaches, our solution does not assume access to unlabeled data from a fixed shift and can be applied to datasets that are entirely unseen during training.

In our work, we propose a more general framework and three novel robust training algorithms that can exploit any suitable generative network, and we show improvements on more challenging, naturally-occurring shifts across twelve distinct datasets.

In this paper, we proposed a novel _model-based robust training paradigm_ for deep learning that provides robustness with respect to models of natural variation.

In our experiments, we show that our paradigm can provide significant out-of-distribution robustness on many challenging distributional shifts. Furthermore, our paradigm can provide robustness against multiple simultaneous distribution shifts and on domains that are entirely unseen while training the model, and shows significant out-of-distribution robustness as datasets become more challenging.

Input: Table 1: Out-of-distribution robustness. In each experiment, we train a model of natural variation to map from challenge-level 0 to challenge-level 2 data from different subsets of CURE-TSR. We then perform model-based training using challenge-level 0 data and test on challenge-levels 3-5.

Input: Table 4: Transferability of model-based robustness. In each experiment, we train a model of natural variation on a given training dataset D1. Then, we use this model to perform model-based training on a new dataset D1 entirely unseen during the training of the model.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After careful review, I find the claim moderately overstated. While the evidence shows strong performance across multiple datasets and conditions, there are several key considerations:

1. The claim states "significantly outperform" across "all" conditions, but the tables show mixed results - in some cases the improvements are modest or comparable to other methods (e.g., in Table 4, MRT performs similarly to MDA in several cases).

2. The reviewer points out important limitations regarding the novelty and potential vulnerabilities of the approach, particularly noting that an attacker could potentially defeat the defense by building a similar natural model.

3. The evidence demonstrates good performance but not universal superiority across "all" conditions as claimed.

4. Some of the baseline comparisons exclude certain methods due to setup limitations, making the comprehensive comparison implied in the claim somewhat overstated.

<score>3</score>

Justification:
- The claim overstates the universality and significance of the improvements
- The evidence shows good but not universal superiority
- Reviewer highlights important limitations not acknowledged in the claim
- Results are strong but more nuanced than the absolute language suggests
- Some experimental constraints limit the comprehensiveness of comparisons


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Specifically, when training on ImageNet and testing on various subsets of ImageNet-c, our algorithms improve over baseline methods by up to 30 percentage points in top-1 accuracy.

The evidence to be evaluated is:First, in Sections 5.1, 5.2, we show that our algorithms are the first to consistently provide out-of-distribution robustness across a range of challenging corruptions, including shifts in brightness, contrast, snow, fog, frost, and haze on CURE-TSR, ImageNet, and ImageNet-c.

Note that while this is an unfair comparison for our methods, given that the model-based algorithms are not given access to labeled level 2 data, our algorithms still outperform the baselines by as much as 20 percentage points on level 5 data. To demonstrate the scalability of our approach, we perform experiments on ImageNet (Deng et al., 2009) and the recently-curated ImageNet-c dataset (Hendrycks and Dietterich, 2019). ImageNet-c contains images from the ImageNet test set that are corrupted according to artificial transformations, such as snow, rain, and fog, and are labeled from 1-5 depending on the severity of the corruption. For numerous challenging corruptions, we train models to map from the classes 0-9 of ImageNet to the corresponding classes of ImageNet-c. We then train all networks on classes 10-59 of ImageNet, and test on the corresponding classes for various subsets of ImageNet-c.

we improve robustness on unseen domains by up to 40 percentage points.

In every scenario, our model-based algorithms significantly outperform the baselines, often by 10-20 percentage points.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll analyze the claim: "when training on ImageNet and testing on various subsets of ImageNet-c, our algorithms improve over baseline methods by up to 30 percentage points in top-1 accuracy."

Analysis:
The evidence provides several performance improvements across different scenarios:
- "up to 20 percentage points on level 5 data"
- "improve robustness... by up to 40 percentage points"
- "outperform the baselines, often by 10-20 percentage points"

The claim appears to be reasonably well-stated, as the stated 30 percentage point improvement falls within the range of improvements mentioned in the evidence (10-40 percentage points). While the evidence shows varying levels of improvement across different scenarios, the claim's use of "up to" appropriately indicates this is a maximum rather than typical improvement. The claim specifically refers to ImageNet/ImageNet-c testing, which matches the context provided in the evidence.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Specifically, when training on ImageNet and testing on various subsets of ImageNet-c, our algorithms improve over baseline methods by up to 30 percentage points in top-1 accuracy.

The review comment to be evaluated is:

# Summary
This paper proposes a “paradigm shift” for augmenting datasets when training CNN-based image classifiers. On one side, traditional augmentations include blur, Gaussian noise, color distortions. On the other side, methods like adversarial training consider augmentations under norm bounds in the image space. The proposed method, instead, uses models of natural variation to augment the images. Increased out-of-domain accuracy is shown on ImageNet-C and several slices of the CURE-TSR dataset.

Central question: How to shift from perturbation-based robustness analysis to model-based robustness analysis?

# Strong and Weak points

## Strong points

  * This paper introduces a “paradigm shift” from perturbation based to model-based augmentation, when training CNN based image classifiers. The existing perturbations only reach so far, and using models of natural variation provides a new avenue for augmenting data sets.
  * The method is presented in clear pseudo code, and detailed in the 36 page counting appendix.

## Weak points

  * The paper claims a “paradigm shift”, but the proposed method of augmenting data has been used since 2012,  [1], and is used ever since [2]. Tables 2 and 3 provide key results for this method, but both were obtained with Model-based Data Augmentation, which is the common method before the "paradigm shift".
  * Results outlined in Section 5.2 and Table 2 concerns scenarios where the test-time perturbation is known at training time. Table 2 then compares MDA to ERM using this a-priori knowledge, and shows improved accuracy. However, the comparison is unbalanced, as ERM does not benefit from the prior knowledge. If Table 2 would be about comparing methods under the assumption that the test-time perturbation would be known at training time, then the ERM should be trained with the same data augmentation.
  * The paper misses comparisons against previous published work. Although Table 2 and 4 compare against another method, these results were not obtained by the respective authors, thus introducing a bias. The related work, Section 6, cites as many as 9 papers that addressed the same problem as this paper. The argument for the proposed method could be stronger when comparing to results from any of these papers.
Moreover, Table 3 reports results on ImageNet-C, which multiple other research papers have published results on. See for example, [3] and [4]. Likewise, Table 4 reports results on MNIST-C, which have been reported in [5] and [6]. Omitting those comparisons hampers the argument for the proposed method.

[1] Krizhevsky, 2012
[2] Szegedy, 2015
[3] https://paperswithcode.com/sota/domain-generalization-on-imagenet-c
[4] https://arxiv.org/pdf/1903.12261.pdf
[5] https://proceedings.icml.cc/paper/2020/file/20546457187cf3d52ea86538403e47cc-Paper.pdf
[6] https://arxiv.org/pdf/1906.02337.pdf

# Statement

Recommendation: Reject
Reason:
Although this paper provides four tables of extensive evaluation of the method, none of the numbers were obtained from previous literature. The related work section cites 9 papers on the same problem, but none were compared with in the results section. (See concrete details above)

# Clarifying questions

  * How to quantify the increase in computational complexity for the inner maximization in Equation (2)? As I understand, even evaluating one point requires a forward pass of the variation-model and the classifier-model. In case of iterative optimization, this would also require two backward passes(see line 5 in Algorithm (1)).
If the proposed method requires significantly more compute, the comparison should be made with a method that has access to the same amount of compute (for example a larger model, or longer trained model).
  * Line 6 of Algorithm (1). I miss the definition of set $\Delta$ and the projection operator in the text nor the appendix. Please explain?
  * Section 5.2 speaks of ImageNet “classes”. I am confused if these “classes” are referring to the ImageNet classes (1000 in total) or the severity levels in ImageNet-C (75 in total). Could you please clarify?
  Moreover, this argument implicitly assumes that the test-time perturbation is known at training-time. Could you name a scenario where this assumption is justified?
  * To what extent is the accuracy on the clean data preserved by the proposed method? Table 3, for example, shows an increase in top-1 accuracy on ImageNet data under perturbation. However, I wonder if this model preserves accuracy on the clean data. We know from existing literature [1] that a trade off exists for adversarial training between accuracy on perturbed and clean data. Providing the clean accuracy would provide evidence that this method compares favourably on this trade off.

[1] Zhang, Hongyang, et al. "Theoretically principled trade-off between robustness and accuracy." arXiv preprint arXiv:1901.08573 (2019).

# Minor feedback

This minor feedback is not part of the assessment.

  * “arguably more common” -> “arguably more important”
  * “x with a corresponding label y” -> “x with a corresponding prediction y”
  * Abbreviation “MBRDL” appears in Section 5 without any explanation or definition.
  * Abbreviation “CURE-TSR” appears in Section 5.0 without any explanation or definition.
  * The citation for CURE in Section 5.1 is wrong. The bibliography refers to “Temel, Dogancan, Min-Hung Chen, and Ghassan AlRegib. "Traffic sign detection under challenging conditions: A deeper look into performance variations and spectral characteristics." IEEE Transactions on Intelligent Transportation Systems (2019).”, but the CURE TSR dataset was published in “Temel, Dogancan, et al. "CURE-TSR: Challenging unreal and real environments for traffic sign recognition." arXiv preprint arXiv:1712.02463 (2017).”
  * Caption of Table 4: “ Then, we use this model to perform model-based training on a new dataset D1” -> “ Then, we use this model to perform model-based testing on a new dataset D2”?

The evidence to be evaluated is:First, in Sections 5.1, 5.2, we show that our algorithms are the first to consistently provide out-of-distribution robustness across a range of challenging corruptions, including shifts in brightness, contrast, snow, fog, frost, and haze on CURE-TSR, ImageNet, and ImageNet-c.

Note that while this is an unfair comparison for our methods, given that the model-based algorithms are not given access to labeled level 2 data, our algorithms still outperform the baselines by as much as 20 percentage points on level 5 data. To demonstrate the scalability of our approach, we perform experiments on ImageNet (Deng et al., 2009) and the recently-curated ImageNet-c dataset (Hendrycks and Dietterich, 2019). ImageNet-c contains images from the ImageNet test set that are corrupted according to artificial transformations, such as snow, rain, and fog, and are labeled from 1-5 depending on the severity of the corruption. For numerous challenging corruptions, we train models to map from the classes 0-9 of ImageNet to the corresponding classes of ImageNet-c. We then train all networks on classes 10-59 of ImageNet, and test on the corresponding classes for various subsets of ImageNet-c.

we improve robustness on unseen domains by up to 40 percentage points.

In every scenario, our model-based algorithms significantly outperform the baselines, often by 10-20 percentage points.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll analyze the claim: "Specifically, when training on ImageNet and testing on various subsets of ImageNet-c, our algorithms improve over baseline methods by up to 30 percentage points in top-1 accuracy."

Analysis:
The evidence shows varying improvement claims:
- "up to 20 percentage points on level 5 data"
- "up to 40 percentage points" on unseen domains
- "often by 10-20 percentage points"

The reviewer points out critical issues:
- Missing comparisons with published work on ImageNet-C
- Unbalanced comparisons where baseline methods weren't given the same advantages
- Test-time perturbation knowledge giving unfair advantages

The claim appears to be in the middle range of the reported improvements (30% vs 20-40% in evidence), but given the reviewer's concerns about unbalanced comparisons and missing benchmarks against published work, this claim seems moderately overstated.

<score>3</score>

Justification:
1. The 30% improvement claim falls within the range shown in evidence, but may be based on unbalanced comparisons
2. The reviewer highlights lack of comparison with published literature on ImageNet-C, weakening the claim's credibility
3. The experimental setup appears to give unfair advantages to the proposed method
4. The claim doesn't acknowledge these limitations or comparative context


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Specifically, when training on ImageNet and testing on various subsets of ImageNet-c, our algorithms improve over baseline methods by up to 30 percentage points in top-1 accuracy.

The review comment to be evaluated is:

The paper extends current adversarial learning approaches beyond imperceptible L_p norm perturbations. The proposed approach can handle many models of natural variation, such as a change in brightness. The main idea behind the approach is to use unsupervised approaches such as GANs to model the natural variation. Given this model of natural variation, the paper replaces the adversarial learning objective of finding the worst example in an L_p norm ball around a data point to finding the worst example based on the model of natural variation. This is expensive, so the paper also proposes more computationally efficient approaches based on data augmentation. The experimental results demonstrate that the proposed approach performs well on a variety of tasks.

Strengths:

1. The question of training robust neural networks is clearly timely and well-motivated. A valid raised criticism of the current adversarial examples literature is that it is too tied to imperceptible or L_p norm perturbations, and this paper tries to go beyond them.
2. There is a bit of a lack of relevant methods to compare against, but the experiments still show that the proposed approach does well. I also appreciated the comparisons to domain adaption, and the multiple experimental setups generally.

Weaknesses:

1. Though the paper suggests that the approach is a “paradigm shift” in robust deep learning, it appears to be more or less an extension of the current adversarial examples literature. The adversarial training approach has been quite successful in defending against a fixed adversary model, from various L_p norms to geometric transformations. This paper is essentially defining a new adversary model, given by natural variations. In fact it is simpler than this, since the goal is to only do well on "random variations" in the predefined test set, i.e. there is no adversary which will try and attack the network by using the class of variations. There is a novelty here though that the variation model is learnt rather than pre-specified. But given that learning transformations such as change in brightness etc. is quite straightforward for unsupervised models such as GANs, this is not too much to ask for. Therefore, I find it a bit unsurprising that the proposed method works well, given the success of adversarial training and GANs.
2. Related to the above, in my view one main drawback of the current adversarial learning literature is that it is not difficult to get robustness to one pre-specified model of perturbation/variation, but in reality the space of possible variations is quite large. It is good that the proposed approach does seem to work on two simultaneous shifts, but the approach still seems restricted and not a substantial change in the current understanding. Note that some of the compared approaches are different in this regard: AugMix does not tailor the model to a particular variation (it explicitly excludes operations which overlap with ImageNet-C corruptions) and domain adaptation can handle quite large changes in the data distribution (which are not just transformations of the original data points).
3. Perhaps less importantly, in the experiments the proposed MDA approach based on data augmentation does almost as well as the other ones, and data augmentation using unsupervised models is already studied in the literature.

Overall, this is not a bad paper and I am not completely opposed to acceptance, but I am not sure I can argue for it.

Other less important comments:

1. Since there is a lack of baselines to compare against, the paper could benefit from a few ablation studies. For example, is there any difference on using other unsupervised approaches?
2. I think it would be better to have a concise list of a few major contributions in Section 1, rather than the current rather long list.
3. It would be nice to have representative images of the models of variation, perhaps in an appendix.

------Updates after author response------

I thank the authors for the response and it helps clarify some points. However, I am still not unconvinced that the paper is a significant departure from current work on adversarial robustness (see weakness 1 and 2 above). I think it would be much more interesting if the approach could yield robustness against a wider/different class of shifts compared to what it was adversarially trained against. Therefore, I am keeping my score at 5 and will not advocate for acceptance, though I am not completely opposed to it.

The evidence to be evaluated is:First, in Sections 5.1, 5.2, we show that our algorithms are the first to consistently provide out-of-distribution robustness across a range of challenging corruptions, including shifts in brightness, contrast, snow, fog, frost, and haze on CURE-TSR, ImageNet, and ImageNet-c.

Note that while this is an unfair comparison for our methods, given that the model-based algorithms are not given access to labeled level 2 data, our algorithms still outperform the baselines by as much as 20 percentage points on level 5 data. To demonstrate the scalability of our approach, we perform experiments on ImageNet (Deng et al., 2009) and the recently-curated ImageNet-c dataset (Hendrycks and Dietterich, 2019). ImageNet-c contains images from the ImageNet test set that are corrupted according to artificial transformations, such as snow, rain, and fog, and are labeled from 1-5 depending on the severity of the corruption. For numerous challenging corruptions, we train models to map from the classes 0-9 of ImageNet to the corresponding classes of ImageNet-c. We then train all networks on classes 10-59 of ImageNet, and test on the corresponding classes for various subsets of ImageNet-c.

we improve robustness on unseen domains by up to 40 percentage points.

In every scenario, our model-based algorithms significantly outperform the baselines, often by 10-20 percentage points.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my review of the claim, evidence, and reviewer comments:

The claim states improvements "up to 30 percentage points" in top-1 accuracy, but the evidence presents varying improvement ranges:
- "up to 20 percentage points on level 5 data"
- "up to 40 percentage points" for unseen domains
- "often by 10-20 percentage points"

While the evidence shows significant improvements, the specific "30 percentage points" claim appears cherry-picked and isn't directly supported by the cited results. Additionally, the reviewer points out that the lack of baselines and ablation studies weakens the comparative claims. The claim would be better stated by using the consistently observed "10-20 percentage points" improvement range.

<score>3</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Specifically, when training on ImageNet and testing on various subsets of ImageNet-c, our algorithms improve over baseline methods by up to 30 percentage points in top-1 accuracy.

The review comment to be evaluated is:

This paper proposes a model-based framework for improving the robustness of image classifiers to average-case corruptions of varying severity. The proposed framework can be thought of as adversarial training where the perturbation is replaced by a function that transforms the image according to a specific corruption. A nuisance parameter controls the instantiation and severity of the corruption that is applied to the input. The paper compares baselines to different versions of this general model-based framework with experiments on several datasets.

Pros:
- Focusing on average-case corruptions is an important and underexplored problem
- Using image translation to learn the corruptions is an interesting proposal

Cons:
- There isn't much novelty in the proposed technique. Adversarial training with 'natural' corruptions has been done several times before.
- There isn't any discussion of how the nuisance parameter is used. I suspect it isn't used for ImageNet-C experiments, though I may be wrong. In this case, the method becomes data augmentation with image translation, which is not very novel either.
- It sounds like the ImageNet-C experiments use the corruptions to train the image translation network. Even though these experiments use held-out images, the corruption types themselves are not being held out, which goes against the recommended methodology for that dataset.

Typo: "ImageNet-c" should be "ImageNet-C"

___________________________________________________________________________________

Update after author feedback:

I thank the authors for improving my understanding of the paper. I feel better about it after reading it again. One of the most interesting findings--that a translation network trained on one domain/set of classes generalizes to another--needs more discussion. Using this phenomenon to improve robustness is a good idea, and the MRT/MDA/MAT methods explored in this work are nice choices for this investigation. However, I agree with the other reviewers that it would be nice to include more baselines from other work where appropriate.

The results in Table 1 are also very interesting and deserve more discussion--perhaps an analysis of whether a translation network trained on weak corruptions can generalize to create something akin to the ground-truth stronger corruptions, as the results in the table for MRT imply.

Overall, I think this paper has some strong experiments and investigates a good idea, but the claims of a "paradigm shift" are overly grandiose, and some of the most interesting experiments could use more analysis. Additionally, the writing clarity and presentation could be improved. My initial understanding of the paper was flawed in some places, so I'll raise my score to 5.

The evidence to be evaluated is:First, in Sections 5.1, 5.2, we show that our algorithms are the first to consistently provide out-of-distribution robustness across a range of challenging corruptions, including shifts in brightness, contrast, snow, fog, frost, and haze on CURE-TSR, ImageNet, and ImageNet-c.

Note that while this is an unfair comparison for our methods, given that the model-based algorithms are not given access to labeled level 2 data, our algorithms still outperform the baselines by as much as 20 percentage points on level 5 data. To demonstrate the scalability of our approach, we perform experiments on ImageNet (Deng et al., 2009) and the recently-curated ImageNet-c dataset (Hendrycks and Dietterich, 2019). ImageNet-c contains images from the ImageNet test set that are corrupted according to artificial transformations, such as snow, rain, and fog, and are labeled from 1-5 depending on the severity of the corruption. For numerous challenging corruptions, we train models to map from the classes 0-9 of ImageNet to the corresponding classes of ImageNet-c. We then train all networks on classes 10-59 of ImageNet, and test on the corresponding classes for various subsets of ImageNet-c.

we improve robustness on unseen domains by up to 40 percentage points.

In every scenario, our model-based algorithms significantly outperform the baselines, often by 10-20 percentage points.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll assess the claim: "Specifically, when training on ImageNet and testing on various subsets of ImageNet-c, our algorithms improve over baseline methods by up to 30 percentage points in top-1 accuracy."

1. Summary of Evaluation:
The claim appears to be overstated based on the available evidence. While the evidence shows significant improvements, it specifically mentions improvements of "10-20 percentage points" and "up to 40 percentage points" in different contexts. The claimed "30 percentage points" improvement specifically for ImageNet-C testing is not directly supported by the provided evidence. Additionally, the reviewer points out concerns about the methodology used for ImageNet-C experiments, suggesting potential limitations in how the results should be interpreted.

The evidence shows:
- "outperform the baselines by as much as 20 percentage points on level 5 data"
- "improve robustness on unseen domains by up to 40 percentage points"
- "outperform the baselines, often by 10-20 percentage points"

None of these specifically support the 30 percentage point claim for ImageNet-C testing.

<score>4</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Specifically, when training on ImageNet and testing on various subsets of ImageNet-c, our algorithms improve over baseline methods by up to 30 percentage points in top-1 accuracy.

The review comment to be evaluated is:

The paper looks at the idea of building models of natural variation of an input and then using these models to develop robust training algorithms that are less susceptible to outliers ( testable for worst-cases via adversarial attacks ).

Strength:

+ The paper addresses a very important topic of adversarial robustness of DNN models and is accompanied by diligent evaluation over different datasets.

Weakness:

- The paper adopts an approach which is very reminiscent of Ajil Jalal, Andrew Ilyas, Constantinos Daskalakis, and Alexandros G Dimakis. The robust manifold defense: Adversarial training using generative models. arXiv preprint arXiv:1712.09196, 2017. The key difference seems to be adoption of the idea of using auxiliary transformations (called natural perturbations) which is also very well-studied in literature, for e.g. see https://openaccess.thecvf.com/content_CVPR_2020/papers/Zhang_Auxiliary_Training_Towards_Accurate_and_Robust_Models_CVPR_2020_paper.pdf  So, the approach presented here is quite incremental for a premier venue such as ICLR.

- The attack in Jalal et. al. paper can be launched against this approach once the attacker also access to this generative/natural model (which would be easy to build for an attacker)

- The reviewer will strongly recommend reviewing the advices in https://arxiv.org/abs/1902.06705 for writing papers on defense and how to self-evaluate its robustness by suitably designing the attacker. If the defense approach uses some background/auxiliary knowledge, one must consider the attacker with this knowledge if it is accessible to the attacker.

Questions to authors:

- Can authors explain why an attacker can't build similar natural model to defeat the proposed defense?
- Any clarification on incremental novelty from  Jalal et. al. and Zhang et. al. (and references therein) would be also useful.

After author's rebuttal:

"The other paper that the review points to ([4]) addresses a similar setting to our paper, but the approach is completely different. ... Moreover, [4] was published within three months of the ICLR submission deadline, meaning that it is essentially concurrent with our work."

Yes, the reviewer concurs that a work published so close to deadline should be treated as concurrent and will raise the score.

The evidence to be evaluated is:First, in Sections 5.1, 5.2, we show that our algorithms are the first to consistently provide out-of-distribution robustness across a range of challenging corruptions, including shifts in brightness, contrast, snow, fog, frost, and haze on CURE-TSR, ImageNet, and ImageNet-c.

Note that while this is an unfair comparison for our methods, given that the model-based algorithms are not given access to labeled level 2 data, our algorithms still outperform the baselines by as much as 20 percentage points on level 5 data. To demonstrate the scalability of our approach, we perform experiments on ImageNet (Deng et al., 2009) and the recently-curated ImageNet-c dataset (Hendrycks and Dietterich, 2019). ImageNet-c contains images from the ImageNet test set that are corrupted according to artificial transformations, such as snow, rain, and fog, and are labeled from 1-5 depending on the severity of the corruption. For numerous challenging corruptions, we train models to map from the classes 0-9 of ImageNet to the corresponding classes of ImageNet-c. We then train all networks on classes 10-59 of ImageNet, and test on the corresponding classes for various subsets of ImageNet-c.

we improve robustness on unseen domains by up to 40 percentage points.

In every scenario, our model-based algorithms significantly outperform the baselines, often by 10-20 percentage points.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Let me assess the claim: "When training on ImageNet and testing on various subsets of ImageNet-c, our algorithms improve over baseline methods by up to 30 percentage points in top-1 accuracy."

Review of Evidence & Claim:
The evidence shows varying improvements:
- "up to 20 percentage points on level 5 data"
- "improve robustness... by up to 40 percentage points"
- "outperform the baselines, often by 10-20 percentage points"

While the evidence demonstrates significant improvements over baselines, the specific claim of "30 percentage points" improvement in top-1 accuracy is not directly supported by any of the cited evidence. The evidence shows improvements ranging from 10-40 percentage points in different contexts, but doesn't specifically validate the 30 percentage point claim for ImageNet to ImageNet-c testing.

The reviewer comments don't specifically address the accuracy claims, so they don't impact this assessment.

<score>3</score>

Justification:
The claim receives a moderate overstatement score of 3 because:
1. The evidence shows improvements of varying magnitudes (10-40 percentage points)
2. The specific "30 percentage points" figure isn't directly supported by the presented evidence
3. While the general trend of significant improvement is supported, the precise magnitude claimed needs stronger direct evidence
4. The claim could be more accurately stated using the actual ranges demonstrated in the evidence


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Further, we show that our methods provide robustness (1) against natural, out-of-distribution data, (2) against multiple simultaneous distributional shifts, and (3) to domains entirely unseen during training.

The evidence to be evaluated is:Notably, DL is known to be fragile to many forms of _natural variation_, which cannot be described by small perturbations \(x\mapsto x+\delta\). In image classification, such natural variation includes changes in weather or background color (Eykholt et al., 2018; Hendrycks et al., 2019; Hosseini and Poovendran, 2018), spatial transformations such as rotation or scaling (Xiao et al., 2018; Karanakis et al., 2016), and sensor-based attacks (Kurakin et al., 2016). Because such transformations frequently arise in safety-critical domains, it is critically important for the DL community to develop algorithms that are robust against out-of-distribution, natural variation in data. In this paper, we specifically address this challenge by proposing a principled, optimization-based framework which can be used in general settings to provide robustness against arbitrary sources of natural variation. Underlying the task of improving the robustness of neural networks against natural, out-of-distribution data are two fundamental challenges. Firstly, unlike in the adversarial robustness community, in real-world, safety-critical environments, data can vary in unknown and highly nonlinear ways. Thus, the first step toward building a robust training procedure must be to design a mechanism that accurately describes how data varies in such environments.

In order to effectively model sources of natural variation in a domain-agnostic setting, we will abstractly define _models of natural variation_. Concretely, a model of natural variation \(G(x,\delta)\) is a map that describes how an input datum \(x\) can be naturally varied by a _nuisance parameter_\(\delta\) resulting in a new datum \(x^{\prime}:=G(x,\delta)\). Ideally, for a fixed datum \(x\), varying the nuisance parameter \(\delta\) should vary the severity of the natural conditions in the generated datum \(x^{\prime}\).

In the remainder of this subsection, we consider cases in which (1) a model \(G\) is known a priori, and (2) a model \(G\) is unknown and therefore must be learned offline from data.

In prior work, this idea has been used to train classifiers to be robust to rotation and scaling (Engstrom et al., 2017)(Kamath et al., 2020).

In many situations, models natural variation are not known a priori or are too costly to obtain. For example, consider Figure 1 in which a model \(G(x,\delta)\) takes an image \(x\) of a street sign in sunny weather and maps it to an image \(x^{\prime}:=G(x,\delta)\) in snowy weather. Even though there is a relationship between the two images, obtaining a model \(G\) relating these two domains is extremely challenging if we resort to geometric structure. For such problems we advocate for _learning_ the model \(G\) from data. To do so, we assume that we have access to two unpaired domains \(A\) and \(B\) that are drawn from a common distribution. Domain \(A\) contains the original data, such as the images with sunny weather, and domain \(B\) contains naturally transformed data, such as images with snow. Ideally, a model of natural variation should learn to map images from domain \(A\) to corresponding images with different levels of natural variation captured by the images of domain \(B\).

By training the network to correctly classify this worst-case data, the goal is to become invariant to the model \(G(x,\delta)\) for any \(\delta\in\Delta\). We now assume that we have access to a suitable model of natural variation \(G(x,\delta)\) and shift our attention toward exploiting \(G\) in the development of novel robust training algorithms.

Specifically, we propose three algorithmic variants: _Model-based Adversarial Training_ (MAT), _Model-based Robust Training_ (MRT), and _Model-based Data Augmentation_ (MDA).

First, in Sections 5.1, 5.2, we show that our algorithms are the first to consistently provide out-of-distribution robustness across a range of challenging corruptions, including shifts in brightness, contrast, snow, fog, frost, and haze on CURE-TSR, ImageNet, and ImageNet-c. In Section 5.3 we curate several new datasets containing simultaneous sources of natural variation, and we then show that models of natural variation can be composed to provide robustness against these simultaneous shifts. In Section 5.4 we show that models of natural variation trained on a fixed dataset can be reused to provide robustness on datasets entirely unseen during training.

In such cases, we show that our algorithms can be used to provide significant out-of-distribution robustness against data with high levels of natural variation by training on data with relatively low levels of the same source of natural variation. To do so, we use data from the CURE-TSR dataset (Hemel et al., 2019), which contains images of street signs divided into subsets according to various sources of natural variation and corresponding severity levels. For example, for images in the "snow" subset, level 0 corresponds to no snow, whereas level 5 corresponds to a full blizzard. Thus, for each row of Table 1 we use unlabeled data from levels 0 and 2 to learn a model of natural variation corresponding to a given source of natural variation in CURE-TSR.

To demonstrate the scalability of our approach, we perform experiments on ImageNet (Deng et al., 2009) and the recently-curated ImageNet-c dataset (Hendrycks and Dietterich, 2019). ImageNet-c contains images from the ImageNet test set that are corrupted according to artificial transformations, such as snow, rain, and fog, and are labeled from 1-5 depending on the severity of the corruption. For numerous challenging corruptions, we train models to map from the classes 0-9 of ImageNet to the corresponding classes of ImageNet-c. We then train all networks on classes 10-59 of ImageNet, and test on the corresponding classes for various subsets of ImageNet-c. Note that in this setting, the ImageNet classes used to train the model of natural variation are disjoint from those that are used to train the classifier, so many techniques, including most domain adaptation methods, do not apply; to offer a point of comparison, we include the accuracies of classifiers trained using AugMix, which is a recently proposed method that adds known transformations to the data (Hendrycks et al., 2019). In practice, it is common to encounter multiple simultaneous distributional shifts. For example, in image classification, there may be shifts in both brightness and contrast; yet while there may be examples corresponding to shifts in either brightness or contrast in the training data, there may not be any examples of both shifts occurring simultaneously. To address this robustness challenge, Because we learn models of natural variation offline before training a classifier, our paradigm can be applied to domains that are _entirely unseen_ while training the model. In particular, we show that models can be reused on similar yet unseen datasets to provide robustness against a common source of natural variation.

However, one might wish to provide robustness against the same shift from daytime to nighttime on a new dataset of American street signs without access to any nighttime images in this new dataset. Whereas many techniques, including most domain adaptation methods, do not apply in this scenario, in the MBRDL paradigm, we can simply learn a model corresponding to the changes in lighting for the European street signs and then apply this model to the dataset of the American signs. Table 4 shows several experiments of this stripe in which a model \(G\) is learned on one dataset \(\mathcal{D}_{1}\) and then applied on another \(\mathcal{D}_{2}\); we improve robustness on unseen domains by up to 40 percentage points.

In every scenario, our model-based algorithms significantly outperform the baselines, often by 10-20 percentage points.

Aside from the algorithms we introduced in Section2 we are not aware of any other algorithms that can be used to address out-of-distribution robustness across the diverse array of tasks presented in the previous section.

In the domain adaptation literature, various methods have been proposed which rely on the restrictive assumption that unlabeled data corresponding to a fixed distributional shift is available during training (Tzeng et al., 2017; Ajakan et al., 2014; Ganin and Lempitsky, 2015). Unlike these approaches, our solution does not assume access to unlabeled data from a fixed shift and can be applied to datasets that are entirely unseen during training.

On the other hand, our approach is broadly applicable to arbitrary and challenging sources of natural variation.

In our work, we propose a more general framework and three novel robust training algorithms that can exploit any suitable generative network, and we show improvements on more challenging, naturally-occurring shifts across twelve distinct datasets.

In our framework we improve robustness against more challenging, natural shifts by learning from _unpaired_ datasets and we do not rely on class-conditioning to generate realistic images.

In our experiments, we show that our paradigm can provide significant out-of-distribution robustness on many challenging distributional shifts. Furthermore, our paradigm can provide robustness against multiple simultaneous distribution shifts and on domains that are entirely unseen while training the model, and shows significant out-of-distribution robustness as datasets become more challenging.

On robustness and transferability of convolutional neural networks.

Input: Table 1: Out-of-distribution robustness. In each experiment, we train a model of natural variation to map from challenge-level 0 to challenge-level 2 data from different subsets of CURE-TSR. We then perform model-based training using challenge-level 0 data and test on challenge-levels 3-5.

Input: Figure 1: A new notion of robustness. Past work has focused on perturbation-based adversarial examples, such as Figure 1a. In this paper, we focus on robustness with respect to natural variation, shown in Figure 1b, which often does not obey perceptual or norm-bounded constraints.

Input: Table 4: Transferability of model-based robustness. In each experiment, we train a model of natural variation on a given training dataset D1. Then, we use this model to perform model-based training on a new dataset D1 entirely unseen during the training of the model.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After careful review, the claim appears to be generally well-supported by the evidence, but with some overstatement. While the paper demonstrates robustness against natural out-of-distribution data (showing improvements across multiple datasets and conditions) and multiple simultaneous shifts (like combined brightness and contrast changes), the claim about robustness to "domains entirely unseen during training" is somewhat overstated. Table 4 shows mixed results for unseen domains - some show strong improvements (Q-MNIST: 98.0%) while others show modest gains (CURE brightness: 47.5%). The claim would be more accurate if it acknowledged these variations in performance and didn't imply uniform success across all scenarios.

<score>3</score>

Justification:
1. Natural OOD robustness is well-supported by Table 1 results showing consistent improvements across snow, haze, decolorization and rain conditions.

2. Multiple simultaneous shifts capability is demonstrated, though with less extensive evidence than single-shift cases.

3. The "entirely unseen domains" claim overstates the consistency of results:
- Some transfers work very well (MNIST→Q-MNIST: 98.0%)
- Others show modest improvements (ImageNet→CURE snow: 62.2%)
- Some show limited gains (ImageNet→CURE brightness: 47.5%)

The paper would be more accurate acknowledging this variability rather than making a broad claim implying uniform success across all scenarios.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Further, we show that our methods provide robustness (1) against natural, out-of-distribution data, (2) against multiple simultaneous distributional shifts, and (3) to domains entirely unseen during training.

The review comment to be evaluated is:

# Summary
This paper proposes a “paradigm shift” for augmenting datasets when training CNN-based image classifiers. On one side, traditional augmentations include blur, Gaussian noise, color distortions. On the other side, methods like adversarial training consider augmentations under norm bounds in the image space. The proposed method, instead, uses models of natural variation to augment the images. Increased out-of-domain accuracy is shown on ImageNet-C and several slices of the CURE-TSR dataset.

Central question: How to shift from perturbation-based robustness analysis to model-based robustness analysis?

# Strong and Weak points

## Strong points

  * This paper introduces a “paradigm shift” from perturbation based to model-based augmentation, when training CNN based image classifiers. The existing perturbations only reach so far, and using models of natural variation provides a new avenue for augmenting data sets.
  * The method is presented in clear pseudo code, and detailed in the 36 page counting appendix.

## Weak points

  * The paper claims a “paradigm shift”, but the proposed method of augmenting data has been used since 2012,  [1], and is used ever since [2]. Tables 2 and 3 provide key results for this method, but both were obtained with Model-based Data Augmentation, which is the common method before the "paradigm shift".
  * Results outlined in Section 5.2 and Table 2 concerns scenarios where the test-time perturbation is known at training time. Table 2 then compares MDA to ERM using this a-priori knowledge, and shows improved accuracy. However, the comparison is unbalanced, as ERM does not benefit from the prior knowledge. If Table 2 would be about comparing methods under the assumption that the test-time perturbation would be known at training time, then the ERM should be trained with the same data augmentation.
  * The paper misses comparisons against previous published work. Although Table 2 and 4 compare against another method, these results were not obtained by the respective authors, thus introducing a bias. The related work, Section 6, cites as many as 9 papers that addressed the same problem as this paper. The argument for the proposed method could be stronger when comparing to results from any of these papers.
Moreover, Table 3 reports results on ImageNet-C, which multiple other research papers have published results on. See for example, [3] and [4]. Likewise, Table 4 reports results on MNIST-C, which have been reported in [5] and [6]. Omitting those comparisons hampers the argument for the proposed method.

[1] Krizhevsky, 2012
[2] Szegedy, 2015
[3] https://paperswithcode.com/sota/domain-generalization-on-imagenet-c
[4] https://arxiv.org/pdf/1903.12261.pdf
[5] https://proceedings.icml.cc/paper/2020/file/20546457187cf3d52ea86538403e47cc-Paper.pdf
[6] https://arxiv.org/pdf/1906.02337.pdf

# Statement

Recommendation: Reject
Reason:
Although this paper provides four tables of extensive evaluation of the method, none of the numbers were obtained from previous literature. The related work section cites 9 papers on the same problem, but none were compared with in the results section. (See concrete details above)

# Clarifying questions

  * How to quantify the increase in computational complexity for the inner maximization in Equation (2)? As I understand, even evaluating one point requires a forward pass of the variation-model and the classifier-model. In case of iterative optimization, this would also require two backward passes(see line 5 in Algorithm (1)).
If the proposed method requires significantly more compute, the comparison should be made with a method that has access to the same amount of compute (for example a larger model, or longer trained model).
  * Line 6 of Algorithm (1). I miss the definition of set $\Delta$ and the projection operator in the text nor the appendix. Please explain?
  * Section 5.2 speaks of ImageNet “classes”. I am confused if these “classes” are referring to the ImageNet classes (1000 in total) or the severity levels in ImageNet-C (75 in total). Could you please clarify?
  Moreover, this argument implicitly assumes that the test-time perturbation is known at training-time. Could you name a scenario where this assumption is justified?
  * To what extent is the accuracy on the clean data preserved by the proposed method? Table 3, for example, shows an increase in top-1 accuracy on ImageNet data under perturbation. However, I wonder if this model preserves accuracy on the clean data. We know from existing literature [1] that a trade off exists for adversarial training between accuracy on perturbed and clean data. Providing the clean accuracy would provide evidence that this method compares favourably on this trade off.

[1] Zhang, Hongyang, et al. "Theoretically principled trade-off between robustness and accuracy." arXiv preprint arXiv:1901.08573 (2019).

# Minor feedback

This minor feedback is not part of the assessment.

  * “arguably more common” -> “arguably more important”
  * “x with a corresponding label y” -> “x with a corresponding prediction y”
  * Abbreviation “MBRDL” appears in Section 5 without any explanation or definition.
  * Abbreviation “CURE-TSR” appears in Section 5.0 without any explanation or definition.
  * The citation for CURE in Section 5.1 is wrong. The bibliography refers to “Temel, Dogancan, Min-Hung Chen, and Ghassan AlRegib. "Traffic sign detection under challenging conditions: A deeper look into performance variations and spectral characteristics." IEEE Transactions on Intelligent Transportation Systems (2019).”, but the CURE TSR dataset was published in “Temel, Dogancan, et al. "CURE-TSR: Challenging unreal and real environments for traffic sign recognition." arXiv preprint arXiv:1712.02463 (2017).”
  * Caption of Table 4: “ Then, we use this model to perform model-based training on a new dataset D1” -> “ Then, we use this model to perform model-based testing on a new dataset D2”?

The evidence to be evaluated is:Notably, DL is known to be fragile to many forms of _natural variation_, which cannot be described by small perturbations \(x\mapsto x+\delta\). In image classification, such natural variation includes changes in weather or background color (Eykholt et al., 2018; Hendrycks et al., 2019; Hosseini and Poovendran, 2018), spatial transformations such as rotation or scaling (Xiao et al., 2018; Karanakis et al., 2016), and sensor-based attacks (Kurakin et al., 2016). Because such transformations frequently arise in safety-critical domains, it is critically important for the DL community to develop algorithms that are robust against out-of-distribution, natural variation in data. In this paper, we specifically address this challenge by proposing a principled, optimization-based framework which can be used in general settings to provide robustness against arbitrary sources of natural variation. Underlying the task of improving the robustness of neural networks against natural, out-of-distribution data are two fundamental challenges. Firstly, unlike in the adversarial robustness community, in real-world, safety-critical environments, data can vary in unknown and highly nonlinear ways. Thus, the first step toward building a robust training procedure must be to design a mechanism that accurately describes how data varies in such environments.

In order to effectively model sources of natural variation in a domain-agnostic setting, we will abstractly define _models of natural variation_. Concretely, a model of natural variation \(G(x,\delta)\) is a map that describes how an input datum \(x\) can be naturally varied by a _nuisance parameter_\(\delta\) resulting in a new datum \(x^{\prime}:=G(x,\delta)\). Ideally, for a fixed datum \(x\), varying the nuisance parameter \(\delta\) should vary the severity of the natural conditions in the generated datum \(x^{\prime}\).

In the remainder of this subsection, we consider cases in which (1) a model \(G\) is known a priori, and (2) a model \(G\) is unknown and therefore must be learned offline from data.

In prior work, this idea has been used to train classifiers to be robust to rotation and scaling (Engstrom et al., 2017)(Kamath et al., 2020).

In many situations, models natural variation are not known a priori or are too costly to obtain. For example, consider Figure 1 in which a model \(G(x,\delta)\) takes an image \(x\) of a street sign in sunny weather and maps it to an image \(x^{\prime}:=G(x,\delta)\) in snowy weather. Even though there is a relationship between the two images, obtaining a model \(G\) relating these two domains is extremely challenging if we resort to geometric structure. For such problems we advocate for _learning_ the model \(G\) from data. To do so, we assume that we have access to two unpaired domains \(A\) and \(B\) that are drawn from a common distribution. Domain \(A\) contains the original data, such as the images with sunny weather, and domain \(B\) contains naturally transformed data, such as images with snow. Ideally, a model of natural variation should learn to map images from domain \(A\) to corresponding images with different levels of natural variation captured by the images of domain \(B\).

By training the network to correctly classify this worst-case data, the goal is to become invariant to the model \(G(x,\delta)\) for any \(\delta\in\Delta\). We now assume that we have access to a suitable model of natural variation \(G(x,\delta)\) and shift our attention toward exploiting \(G\) in the development of novel robust training algorithms.

Specifically, we propose three algorithmic variants: _Model-based Adversarial Training_ (MAT), _Model-based Robust Training_ (MRT), and _Model-based Data Augmentation_ (MDA).

First, in Sections 5.1, 5.2, we show that our algorithms are the first to consistently provide out-of-distribution robustness across a range of challenging corruptions, including shifts in brightness, contrast, snow, fog, frost, and haze on CURE-TSR, ImageNet, and ImageNet-c. In Section 5.3 we curate several new datasets containing simultaneous sources of natural variation, and we then show that models of natural variation can be composed to provide robustness against these simultaneous shifts. In Section 5.4 we show that models of natural variation trained on a fixed dataset can be reused to provide robustness on datasets entirely unseen during training.

In such cases, we show that our algorithms can be used to provide significant out-of-distribution robustness against data with high levels of natural variation by training on data with relatively low levels of the same source of natural variation. To do so, we use data from the CURE-TSR dataset (Hemel et al., 2019), which contains images of street signs divided into subsets according to various sources of natural variation and corresponding severity levels. For example, for images in the "snow" subset, level 0 corresponds to no snow, whereas level 5 corresponds to a full blizzard. Thus, for each row of Table 1 we use unlabeled data from levels 0 and 2 to learn a model of natural variation corresponding to a given source of natural variation in CURE-TSR.

To demonstrate the scalability of our approach, we perform experiments on ImageNet (Deng et al., 2009) and the recently-curated ImageNet-c dataset (Hendrycks and Dietterich, 2019). ImageNet-c contains images from the ImageNet test set that are corrupted according to artificial transformations, such as snow, rain, and fog, and are labeled from 1-5 depending on the severity of the corruption. For numerous challenging corruptions, we train models to map from the classes 0-9 of ImageNet to the corresponding classes of ImageNet-c. We then train all networks on classes 10-59 of ImageNet, and test on the corresponding classes for various subsets of ImageNet-c. Note that in this setting, the ImageNet classes used to train the model of natural variation are disjoint from those that are used to train the classifier, so many techniques, including most domain adaptation methods, do not apply; to offer a point of comparison, we include the accuracies of classifiers trained using AugMix, which is a recently proposed method that adds known transformations to the data (Hendrycks et al., 2019). In practice, it is common to encounter multiple simultaneous distributional shifts. For example, in image classification, there may be shifts in both brightness and contrast; yet while there may be examples corresponding to shifts in either brightness or contrast in the training data, there may not be any examples of both shifts occurring simultaneously. To address this robustness challenge, Because we learn models of natural variation offline before training a classifier, our paradigm can be applied to domains that are _entirely unseen_ while training the model. In particular, we show that models can be reused on similar yet unseen datasets to provide robustness against a common source of natural variation.

However, one might wish to provide robustness against the same shift from daytime to nighttime on a new dataset of American street signs without access to any nighttime images in this new dataset. Whereas many techniques, including most domain adaptation methods, do not apply in this scenario, in the MBRDL paradigm, we can simply learn a model corresponding to the changes in lighting for the European street signs and then apply this model to the dataset of the American signs. Table 4 shows several experiments of this stripe in which a model \(G\) is learned on one dataset \(\mathcal{D}_{1}\) and then applied on another \(\mathcal{D}_{2}\); we improve robustness on unseen domains by up to 40 percentage points.

In every scenario, our model-based algorithms significantly outperform the baselines, often by 10-20 percentage points.

Aside from the algorithms we introduced in Section2 we are not aware of any other algorithms that can be used to address out-of-distribution robustness across the diverse array of tasks presented in the previous section.

In the domain adaptation literature, various methods have been proposed which rely on the restrictive assumption that unlabeled data corresponding to a fixed distributional shift is available during training (Tzeng et al., 2017; Ajakan et al., 2014; Ganin and Lempitsky, 2015). Unlike these approaches, our solution does not assume access to unlabeled data from a fixed shift and can be applied to datasets that are entirely unseen during training.

On the other hand, our approach is broadly applicable to arbitrary and challenging sources of natural variation.

In our work, we propose a more general framework and three novel robust training algorithms that can exploit any suitable generative network, and we show improvements on more challenging, naturally-occurring shifts across twelve distinct datasets.

In our framework we improve robustness against more challenging, natural shifts by learning from _unpaired_ datasets and we do not rely on class-conditioning to generate realistic images.

In our experiments, we show that our paradigm can provide significant out-of-distribution robustness on many challenging distributional shifts. Furthermore, our paradigm can provide robustness against multiple simultaneous distribution shifts and on domains that are entirely unseen while training the model, and shows significant out-of-distribution robustness as datasets become more challenging.

On robustness and transferability of convolutional neural networks.

Input: Table 1: Out-of-distribution robustness. In each experiment, we train a model of natural variation to map from challenge-level 0 to challenge-level 2 data from different subsets of CURE-TSR. We then perform model-based training using challenge-level 0 data and test on challenge-levels 3-5.

Input: Figure 1: A new notion of robustness. Past work has focused on perturbation-based adversarial examples, such as Figure 1a. In this paper, we focus on robustness with respect to natural variation, shown in Figure 1b, which often does not obey perceptual or norm-bounded constraints.

Input: Table 4: Transferability of model-based robustness. In each experiment, we train a model of natural variation on a given training dataset D1. Then, we use this model to perform model-based training on a new dataset D1 entirely unseen during the training of the model.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence and claim, I find that the claim regarding robustness against (1) natural out-of-distribution data, (2) multiple simultaneous shifts, and (3) unseen domains during training is partially supported but overstated.

The evidence shows improvements in specific scenarios, but has important limitations noted in the review:
- No comparisons against published work or state-of-the-art methods
- Test-time perturbation knowledge at training time creates unbalanced comparisons
- Results are limited to specific datasets and conditions

While Tables 1 and 4 show improvements over baselines, claiming broad robustness "against natural, out-of-distribution data" overstates the scope and generalizability of the results. The claim would be better stated with more specific scope and acknowledgment of limitations.

<score>4</score>

Justification:
1. The paper shows positive results but overstates their breadth/generalizability
2. Reviewer points out missing comparisons to relevant prior work
3. Some experimental comparisons are unbalanced
4. Claims of "paradigm shift" are overstated given similar prior approaches
5. Results are promising but more limited than the broad claim suggests

The claim would be better stated by:
- Specifying the exact scenarios where robustness was demonstrated
- Acknowledging limitations and scope
- Making more measured statements about generalizability
- Including comparisons to state-of-the-art methods