Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To address this gap, we propose a paradigm shift from perturbation-based adversarial robustness to _model-based robust deep learning_.

The review comment to be evaluated is:

# Summary
This paper proposes a “paradigm shift” for augmenting datasets when training CNN-based image classifiers. On one side, traditional augmentations include blur, Gaussian noise, color distortions. On the other side, methods like adversarial training consider augmentations under norm bounds in the image space. The proposed method, instead, uses models of natural variation to augment the images. Increased out-of-domain accuracy is shown on ImageNet-C and several slices of the CURE-TSR dataset.

Central question: How to shift from perturbation-based robustness analysis to model-based robustness analysis?

# Strong and Weak points

## Strong points

  * This paper introduces a “paradigm shift” from perturbation based to model-based augmentation, when training CNN based image classifiers. The existing perturbations only reach so far, and using models of natural variation provides a new avenue for augmenting data sets.
  * The method is presented in clear pseudo code, and detailed in the 36 page counting appendix.

## Weak points

  * The paper claims a “paradigm shift”, but the proposed method of augmenting data has been used since 2012,  [1], and is used ever since [2]. Tables 2 and 3 provide key results for this method, but both were obtained with Model-based Data Augmentation, which is the common method before the "paradigm shift".
  * Results outlined in Section 5.2 and Table 2 concerns scenarios where the test-time perturbation is known at training time. Table 2 then compares MDA to ERM using this a-priori knowledge, and shows improved accuracy. However, the comparison is unbalanced, as ERM does not benefit from the prior knowledge. If Table 2 would be about comparing methods under the assumption that the test-time perturbation would be known at training time, then the ERM should be trained with the same data augmentation.
  * The paper misses comparisons against previous published work. Although Table 2 and 4 compare against another method, these results were not obtained by the respective authors, thus introducing a bias. The related work, Section 6, cites as many as 9 papers that addressed the same problem as this paper. The argument for the proposed method could be stronger when comparing to results from any of these papers.
Moreover, Table 3 reports results on ImageNet-C, which multiple other research papers have published results on. See for example, [3] and [4]. Likewise, Table 4 reports results on MNIST-C, which have been reported in [5] and [6]. Omitting those comparisons hampers the argument for the proposed method.

[1] Krizhevsky, 2012
[2] Szegedy, 2015
[3] https://paperswithcode.com/sota/domain-generalization-on-imagenet-c
[4] https://arxiv.org/pdf/1903.12261.pdf
[5] https://proceedings.icml.cc/paper/2020/file/20546457187cf3d52ea86538403e47cc-Paper.pdf
[6] https://arxiv.org/pdf/1906.02337.pdf

# Statement

Recommendation: Reject
Reason:
Although this paper provides four tables of extensive evaluation of the method, none of the numbers were obtained from previous literature. The related work section cites 9 papers on the same problem, but none were compared with in the results section. (See concrete details above)

# Clarifying questions

  * How to quantify the increase in computational complexity for the inner maximization in Equation (2)? As I understand, even evaluating one point requires a forward pass of the variation-model and the classifier-model. In case of iterative optimization, this would also require two backward passes(see line 5 in Algorithm (1)).
If the proposed method requires significantly more compute, the comparison should be made with a method that has access to the same amount of compute (for example a larger model, or longer trained model).
  * Line 6 of Algorithm (1). I miss the definition of set $\Delta$ and the projection operator in the text nor the appendix. Please explain?
  * Section 5.2 speaks of ImageNet “classes”. I am confused if these “classes” are referring to the ImageNet classes (1000 in total) or the severity levels in ImageNet-C (75 in total). Could you please clarify?
  Moreover, this argument implicitly assumes that the test-time perturbation is known at training-time. Could you name a scenario where this assumption is justified?
  * To what extent is the accuracy on the clean data preserved by the proposed method? Table 3, for example, shows an increase in top-1 accuracy on ImageNet data under perturbation. However, I wonder if this model preserves accuracy on the clean data. We know from existing literature [1] that a trade off exists for adversarial training between accuracy on perturbed and clean data. Providing the clean accuracy would provide evidence that this method compares favourably on this trade off.

[1] Zhang, Hongyang, et al. "Theoretically principled trade-off between robustness and accuracy." arXiv preprint arXiv:1901.08573 (2019).

# Minor feedback

This minor feedback is not part of the assessment.

  * “arguably more common” -> “arguably more important”
  * “x with a corresponding label y” -> “x with a corresponding prediction y”
  * Abbreviation “MBRDL” appears in Section 5 without any explanation or definition.
  * Abbreviation “CURE-TSR” appears in Section 5.0 without any explanation or definition.
  * The citation for CURE in Section 5.1 is wrong. The bibliography refers to “Temel, Dogancan, Min-Hung Chen, and Ghassan AlRegib. "Traffic sign detection under challenging conditions: A deeper look into performance variations and spectral characteristics." IEEE Transactions on Intelligent Transportation Systems (2019).”, but the CURE TSR dataset was published in “Temel, Dogancan, et al. "CURE-TSR: Challenging unreal and real environments for traffic sign recognition." arXiv preprint arXiv:1712.02463 (2017).”
  * Caption of Table 4: “ Then, we use this model to perform model-based training on a new dataset D1” -> “ Then, we use this model to perform model-based testing on a new dataset D2”?

The evidence to be evaluated is:However, neural networks trained using ERM are known to be susceptible to _adversarial attacks_. This means that given a datum \(x\) with a corresponding label \(y\), one can find another datum \(x^{\text{adv}}\) such that \(x\) is close to \(x^{\text{adv}}\) in a given Euclidean norm and \(x^{\text{adv}}\) is predicted by the learned classifier as belonging to a different class \(c\neq y\). If such a datum \(x^{\text{adv}}\) exists, it is called an _adversarial example_.

The dominant paradigm toward improving robustness against adversarial examples relies on a robust optimization perspective wherein neural networks are trained to correctly classify worst-case perturbations of data (Madry et al., 2017; Wong and Kolter, 2017).

\[\operatorname*{arg\,min}_{w}\mathbb{E}_{(x,y)\sim\mathcal{D}}\Big{[}\max_{ \delta\in\Delta}\ell(x+\delta,y;w)\Big{]} \tag{1}\]

Despite remarkable progress toward improving the robustness of DL against norm-bounded perturbations, there are significant limitations to adversarial training. Notably, DL is known to be fragile to many forms of _natural variation_, which cannot be described by small perturbations \(x\mapsto x+\delta\). In image classification, such natural variation includes changes in weather or background color (Eykholt et al., 2018; Hendrycks et al., 2019; Hosseini and Poovendran, 2018), spatial transformations such as rotation or scaling (Xiao et al., 2018; Karanakis et al., 2016), and sensor-based attacks (Kurakin et al., 2016). Because such transformations frequently arise in safety-critical domains, it is critically important for the DL community to develop algorithms that are robust against out-of-distribution, natural variation in data. In this paper, we specifically address this challenge by proposing a principled, optimization-based framework which can be used in general settings to provide robustness against arbitrary sources of natural variation.

In order to effectively model sources of natural variation in a domain-agnostic setting, we will abstractly define _models of natural variation_. Concretely, a model of natural variation \(G(x,\delta)\) is a map that describes how an input datum \(x\) can be naturally varied by a _nuisance parameter_\(\delta\) resulting in a new datum \(x^{\prime}:=G(x,\delta)\). Ideally, for a fixed datum \(x\), varying the nuisance parameter \(\delta\) should vary the severity of the natural conditions in the generated datum \(x^{\prime}\). An example of such a model is shown in Figure 2 where an image \(x\) on the left (in this case, in any weather) can be naturally varied by \(\delta\) and consequently transformed into the image \(x^{\prime}\) on the right (in snowy weather). In the remainder of this subsection, we consider cases in which (1) a model \(G\) is known a priori, and (2) a model \(G\) is unknown and therefore must be learned offline from data.

**Learning models of natural variation from data.** In many situations, models natural variation are not known a priori or are too costly to obtain. For example, consider Figure 1 in which a model \(G(x,\delta)\) takes an image \(x\) of a street sign in sunny weather and maps it to an image \(x^{\prime}:=G(x,\delta)\) in snowy weather. Even though there is a relationship between the two images, obtaining a model \(G\) relating these two domains is extremely challenging if we resort to geometric structure. For such problems we advocate for _learning_ the model \(G\) from data. To do so, we assume that we have access to two unpaired domains \(A\) and \(B\) that are drawn from a common distribution. Domain \(A\) contains the original data, such as the images with sunny weather, and domain \(B\) contains naturally transformed data, such as images with snow. Ideally, a model of natural variation should learn to map images from domain \(A\) to corresponding images with different levels of natural variation captured by the images of domain \(B\).

The model-based robust training paradigm that we propose retains the basic elements of adversarial training described in Section 2. Our point of departure from the classical adversarial training formulation is in the choice of the so-called adversarial perturbation. In this paper, we assume that data can be transformed according to a model of natural variation \(G(x,\delta)\) by choosing different values of \(\delta\) from a given _nuisance space_\(\Delta\). The goal of the model-based approach is to train a classifier that achieves high accuracy both on a test set drawn i.i.d. from \(\mathcal{D}\) and on _more-challenging_ test data that has been subjected to the source of natural variation that \(G\) models. This perspective can be captured by the following robust optimization problem: \[\min_{w}\mathbb{E}_{(x,y)\sim\mathcal{D}}\Big{[}\max_{\delta\in\Delta}\ell(G( x,\delta),y;w)\Big{]}. \tag{2}\] In the inner maximization problem of this formulation, given an instance-label pair \((x,y)\), we seek a vector \(\delta^{*}\in\Delta\) that produces a corresponding instance \(x^{\prime}:=G(x,\delta^{*})\) which gives rise to high loss values \(\ell(G(x,\delta^{*}),y;w)\) under the current weight \(w\). One can think of this vector \(\delta^{*}\) as characterizing the _worst-case_ nuisance that can be generated by the model \(G(x,\delta^{*})\) for the original instance \(x\). After solving this inner problem, we solve the outer minimization problem by finding weights \(w\) that minimize the risk against the challenging instance \(G(x,\delta^{*})\). By training the network to correctly classify this worst-case data, the goal is to become invariant to the model \(G(x,\delta)\) for any \(\delta\in\Delta\). We now assume that we have access to a suitable model of natural variation \(G(x,\delta)\) and shift our attention toward exploiting \(G\) in the development of novel robust training algorithms.

Specifically, we propose three algorithmic variants: _Model-based Adversarial Training_ (MAT), _Model-based Robust Training_ (MRT), and _Model-based Data Augmentation_ (MDA).

**Model-based Adversarial Training.** In MAT, we seek an exact solution to the inner problem by performing \(k\) steps of gradient ascent in \(\delta\) on the objective \(\ell(G(x,\delta),y;w)\). The resulting nuisance parameter \(\delta^{\star}\) is one that causes \(\ell(G(x,\delta^{\star}),y;w)\) to have high loss under the current weight \(w\). **Model-based Robust Training.** In MRT, we first randomly sample \(\delta_{i}\in\Delta\) for \(i\in[k]\). We then select the \(i^{\star}\in[k]\) such that \(\ell(G(x,\delta_{i^{\star}}),y;w)\) is maximized.

**Model-based Data Augmentation.** In MDA, rather than explicitly trying to solve the inner problem, we seek a _diversity_ of naturally-varying data rather than the "worst-case." In this way, MDA samples \(\delta_{i}\in\Delta\) for \(i\in[k]\) and then appends \(\{G(x,\delta_{i}),y\}_{i=1}^{k}\) to the training dataset. We present experiments in five different settings over twelve distinct datasets to demonstrate the broad applicability of MBRDL. First, in Sections 5.1, 5.2, we show that our algorithms are the first to consistently provide out-of-distribution robustness across a range of challenging corruptions, including shifts in brightness, contrast, snow, fog, frost, and haze on CURE-TSR, ImageNet, and ImageNet-c. In Section 5.3 we curate several new datasets containing simultaneous sources of natural variation, and we then show that models of natural variation can be composed to provide robustness against these simultaneous shifts. In Section 5.4 we show that models of natural variation trained on a fixed dataset can be reused to provide robustness on datasets entirely unseen during training. Finally, in Section 5.5 we assume access to unlabeled data corresponding to a fixed domain shift, and we compare our algorithms to suitable baselines, including domain adaptation methods.

In many applications, one might have data corresponding to low levels of natural variation, such as a dusting of snow in images of street signs. However, it is often difficult to collect data corresponding to high levels of natural variation, such as images taken during a blizzard. In such cases, we show that our algorithms can be used to provide significant out-of-distribution robustness against data with high levels of natural variation by training on data with relatively low levels of the same source of natural variation. To do so, we use data from the CURE-TSR dataset (Hemel et al., 2019), which contains images of street signs divided into subsets according to various sources of natural variation and corresponding severity levels. For example, for images in the "snow" subset, level 0 corresponds to no snow, whereas level 5 corresponds to a full blizzard. Thus, for each row of Table 1 we use unlabeled data from levels 0 and 2 to learn a model of natural variation corresponding to a given source of natural variation in CURE-TSR.

Note that while this is an unfair comparison for our methods, given that the model-based algorithms are not given access to labeled level 2 data, our algorithms still outperform the baselines by as much as 20 percentage points on level 5 data. To demonstrate the scalability of our approach, we perform experiments on ImageNet (Deng et al., 2009) and the recently-curated ImageNet-c dataset (Hendrycks and Dietterich, 2019). ImageNet-c contains images from the ImageNet test set that are corrupted according to artificial transformations, such as snow, rain, and fog, and are labeled from 1-5 depending on the severity of the corruption. For numerous challenging corruptions, we train models to map from the classes 0-9 of ImageNet to the corresponding classes of ImageNet-c. We then train all networks on classes 10-59 of ImageNet, and test on the corresponding classes for various subsets of ImageNet-c. Note that in this setting, the ImageNet classes used to train the model of natural variation are disjoint from those that are used to train the classifier, so many techniques, including most domain adaptation methods, do not apply; to offer a point of comparison, we include the accuracies of classifiers trained using AugMix, which is a recently proposed method that adds known transformations to the data (Hendrycks et al., 2019). In practice, it is common to encounter multiple simultaneous distributional shifts.

To address this robustness challenge, Because we learn models of natural variation offline before training a classifier, our paradigm can be applied to domains that are _entirely unseen_ while training the model. In particular, we show that models can be reused on similar yet unseen datasets to provide robustness against a common source of natural variation.

Whereas many techniques, including most domain adaptation methods, do not apply in this scenario, in the MBRDL paradigm, we can simply learn a model corresponding to the changes in lighting for the European street signs and then apply this model to the dataset of the American signs. Table 4 shows several experiments of this stripe in which a model \(G\) is learned on one dataset \(\mathcal{D}_{1}\) and then applied on another \(\mathcal{D}_{2}\); we improve robustness on unseen domains by up to 40 percentage points.

In every scenario, our model-based algorithms significantly outperform the baselines, often by 10-20 percentage points. Note that while this is one of the most commonly studied settings in domain adaptation, it represents only one particular setting to which the MBRDL paradigm can be applied. Aside from the algorithms we introduced in Section2 we are not aware of any other algorithms that can be used to address out-of-distribution robustness across the diverse array of tasks presented in the previous section.

Unlike these approaches, our solution does not assume access to unlabeled data from a fixed shift and can be applied to datasets that are entirely unseen during training.

In our work, we propose a more general framework and three novel robust training algorithms that can exploit any suitable generative network, and we show improvements on more challenging, naturally-occurring shifts across twelve distinct datasets.

In this paper, we proposed a novel _model-based robust training paradigm_ for deep learning that provides robustness with respect to models of natural variation. Our notion of robustness offers a departure from adversarial training with respect to norm-bounded data perturbations.

Input: Table 1: Out-of-distribution robustness. In each experiment, we train a model of natural variation to map from challenge-level 0 to challenge-level 2 data from different subsets of CURE-TSR. We then perform model-based training using challenge-level 0 data and test on challenge-levels 3-5.

Input: Figure 1: A new notion of robustness. Past work has focused on perturbation-based adversarial examples, such as Figure 1a. In this paper, we focus on robustness with respect to natural variation, shown in Figure 1b, which often does not obey perceptual or norm-bounded constraints.

Input: Table 4: Transferability of model-based robustness. In each experiment, we train a model of natural variation on a given training dataset D1. Then, we use this model to perform model-based training on a new dataset D1 entirely unseen during the training of the model.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I assess that characterizing this as a "paradigm shift" is overstated. While the paper presents valuable contributions in model-based robust training, the review correctly points out that model-based data augmentation has been used since 2012. The evidence shows incremental improvements rather than a fundamental paradigm change. The results demonstrate enhanced robustness across multiple datasets and conditions, but build upon existing approaches rather than replacing them. The review identifies that key comparisons to previous work are missing, which weakens the claim of a paradigm shift.

<score>4</score>

Justification:
1. The review points out that "the proposed method of augmenting data has been used since 2012" with citations to prior work
2. While results show improvements (e.g., 10-20 percentage points in some cases per Tables 1 and 4), these represent advances within the existing paradigm rather than a fundamental shift
3. The lack of comparisons to previous published work (noted in review) makes it difficult to support claims of paradigm-changing significance
4. The paper presents valuable technical contributions but frames them with overly strong language that exceeds what the evidence supports


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To address this gap, we propose a paradigm shift from perturbation-based adversarial robustness to _model-based robust deep learning_.

The review comment to be evaluated is:

The paper extends current adversarial learning approaches beyond imperceptible L_p norm perturbations. The proposed approach can handle many models of natural variation, such as a change in brightness. The main idea behind the approach is to use unsupervised approaches such as GANs to model the natural variation. Given this model of natural variation, the paper replaces the adversarial learning objective of finding the worst example in an L_p norm ball around a data point to finding the worst example based on the model of natural variation. This is expensive, so the paper also proposes more computationally efficient approaches based on data augmentation. The experimental results demonstrate that the proposed approach performs well on a variety of tasks.

Strengths:

1. The question of training robust neural networks is clearly timely and well-motivated. A valid raised criticism of the current adversarial examples literature is that it is too tied to imperceptible or L_p norm perturbations, and this paper tries to go beyond them.
2. There is a bit of a lack of relevant methods to compare against, but the experiments still show that the proposed approach does well. I also appreciated the comparisons to domain adaption, and the multiple experimental setups generally.

Weaknesses:

1. Though the paper suggests that the approach is a “paradigm shift” in robust deep learning, it appears to be more or less an extension of the current adversarial examples literature. The adversarial training approach has been quite successful in defending against a fixed adversary model, from various L_p norms to geometric transformations. This paper is essentially defining a new adversary model, given by natural variations. In fact it is simpler than this, since the goal is to only do well on "random variations" in the predefined test set, i.e. there is no adversary which will try and attack the network by using the class of variations. There is a novelty here though that the variation model is learnt rather than pre-specified. But given that learning transformations such as change in brightness etc. is quite straightforward for unsupervised models such as GANs, this is not too much to ask for. Therefore, I find it a bit unsurprising that the proposed method works well, given the success of adversarial training and GANs.
2. Related to the above, in my view one main drawback of the current adversarial learning literature is that it is not difficult to get robustness to one pre-specified model of perturbation/variation, but in reality the space of possible variations is quite large. It is good that the proposed approach does seem to work on two simultaneous shifts, but the approach still seems restricted and not a substantial change in the current understanding. Note that some of the compared approaches are different in this regard: AugMix does not tailor the model to a particular variation (it explicitly excludes operations which overlap with ImageNet-C corruptions) and domain adaptation can handle quite large changes in the data distribution (which are not just transformations of the original data points).
3. Perhaps less importantly, in the experiments the proposed MDA approach based on data augmentation does almost as well as the other ones, and data augmentation using unsupervised models is already studied in the literature.

Overall, this is not a bad paper and I am not completely opposed to acceptance, but I am not sure I can argue for it.

Other less important comments:

1. Since there is a lack of baselines to compare against, the paper could benefit from a few ablation studies. For example, is there any difference on using other unsupervised approaches?
2. I think it would be better to have a concise list of a few major contributions in Section 1, rather than the current rather long list.
3. It would be nice to have representative images of the models of variation, perhaps in an appendix.

------Updates after author response------

I thank the authors for the response and it helps clarify some points. However, I am still not unconvinced that the paper is a significant departure from current work on adversarial robustness (see weakness 1 and 2 above). I think it would be much more interesting if the approach could yield robustness against a wider/different class of shifts compared to what it was adversarially trained against. Therefore, I am keeping my score at 5 and will not advocate for acceptance, though I am not completely opposed to it.

The evidence to be evaluated is:However, neural networks trained using ERM are known to be susceptible to _adversarial attacks_. This means that given a datum \(x\) with a corresponding label \(y\), one can find another datum \(x^{\text{adv}}\) such that \(x\) is close to \(x^{\text{adv}}\) in a given Euclidean norm and \(x^{\text{adv}}\) is predicted by the learned classifier as belonging to a different class \(c\neq y\). If such a datum \(x^{\text{adv}}\) exists, it is called an _adversarial example_.

The dominant paradigm toward improving robustness against adversarial examples relies on a robust optimization perspective wherein neural networks are trained to correctly classify worst-case perturbations of data (Madry et al., 2017; Wong and Kolter, 2017).

\[\operatorname*{arg\,min}_{w}\mathbb{E}_{(x,y)\sim\mathcal{D}}\Big{[}\max_{ \delta\in\Delta}\ell(x+\delta,y;w)\Big{]} \tag{1}\]

Despite remarkable progress toward improving the robustness of DL against norm-bounded perturbations, there are significant limitations to adversarial training. Notably, DL is known to be fragile to many forms of _natural variation_, which cannot be described by small perturbations \(x\mapsto x+\delta\). In image classification, such natural variation includes changes in weather or background color (Eykholt et al., 2018; Hendrycks et al., 2019; Hosseini and Poovendran, 2018), spatial transformations such as rotation or scaling (Xiao et al., 2018; Karanakis et al., 2016), and sensor-based attacks (Kurakin et al., 2016). Because such transformations frequently arise in safety-critical domains, it is critically important for the DL community to develop algorithms that are robust against out-of-distribution, natural variation in data. In this paper, we specifically address this challenge by proposing a principled, optimization-based framework which can be used in general settings to provide robustness against arbitrary sources of natural variation.

In order to effectively model sources of natural variation in a domain-agnostic setting, we will abstractly define _models of natural variation_. Concretely, a model of natural variation \(G(x,\delta)\) is a map that describes how an input datum \(x\) can be naturally varied by a _nuisance parameter_\(\delta\) resulting in a new datum \(x^{\prime}:=G(x,\delta)\). Ideally, for a fixed datum \(x\), varying the nuisance parameter \(\delta\) should vary the severity of the natural conditions in the generated datum \(x^{\prime}\). An example of such a model is shown in Figure 2 where an image \(x\) on the left (in this case, in any weather) can be naturally varied by \(\delta\) and consequently transformed into the image \(x^{\prime}\) on the right (in snowy weather). In the remainder of this subsection, we consider cases in which (1) a model \(G\) is known a priori, and (2) a model \(G\) is unknown and therefore must be learned offline from data.

**Learning models of natural variation from data.** In many situations, models natural variation are not known a priori or are too costly to obtain. For example, consider Figure 1 in which a model \(G(x,\delta)\) takes an image \(x\) of a street sign in sunny weather and maps it to an image \(x^{\prime}:=G(x,\delta)\) in snowy weather. Even though there is a relationship between the two images, obtaining a model \(G\) relating these two domains is extremely challenging if we resort to geometric structure. For such problems we advocate for _learning_ the model \(G\) from data. To do so, we assume that we have access to two unpaired domains \(A\) and \(B\) that are drawn from a common distribution. Domain \(A\) contains the original data, such as the images with sunny weather, and domain \(B\) contains naturally transformed data, such as images with snow. Ideally, a model of natural variation should learn to map images from domain \(A\) to corresponding images with different levels of natural variation captured by the images of domain \(B\).

The model-based robust training paradigm that we propose retains the basic elements of adversarial training described in Section 2. Our point of departure from the classical adversarial training formulation is in the choice of the so-called adversarial perturbation. In this paper, we assume that data can be transformed according to a model of natural variation \(G(x,\delta)\) by choosing different values of \(\delta\) from a given _nuisance space_\(\Delta\). The goal of the model-based approach is to train a classifier that achieves high accuracy both on a test set drawn i.i.d. from \(\mathcal{D}\) and on _more-challenging_ test data that has been subjected to the source of natural variation that \(G\) models. This perspective can be captured by the following robust optimization problem: \[\min_{w}\mathbb{E}_{(x,y)\sim\mathcal{D}}\Big{[}\max_{\delta\in\Delta}\ell(G( x,\delta),y;w)\Big{]}. \tag{2}\] In the inner maximization problem of this formulation, given an instance-label pair \((x,y)\), we seek a vector \(\delta^{*}\in\Delta\) that produces a corresponding instance \(x^{\prime}:=G(x,\delta^{*})\) which gives rise to high loss values \(\ell(G(x,\delta^{*}),y;w)\) under the current weight \(w\). One can think of this vector \(\delta^{*}\) as characterizing the _worst-case_ nuisance that can be generated by the model \(G(x,\delta^{*})\) for the original instance \(x\). After solving this inner problem, we solve the outer minimization problem by finding weights \(w\) that minimize the risk against the challenging instance \(G(x,\delta^{*})\). By training the network to correctly classify this worst-case data, the goal is to become invariant to the model \(G(x,\delta)\) for any \(\delta\in\Delta\). We now assume that we have access to a suitable model of natural variation \(G(x,\delta)\) and shift our attention toward exploiting \(G\) in the development of novel robust training algorithms.

Specifically, we propose three algorithmic variants: _Model-based Adversarial Training_ (MAT), _Model-based Robust Training_ (MRT), and _Model-based Data Augmentation_ (MDA).

**Model-based Adversarial Training.** In MAT, we seek an exact solution to the inner problem by performing \(k\) steps of gradient ascent in \(\delta\) on the objective \(\ell(G(x,\delta),y;w)\). The resulting nuisance parameter \(\delta^{\star}\) is one that causes \(\ell(G(x,\delta^{\star}),y;w)\) to have high loss under the current weight \(w\). **Model-based Robust Training.** In MRT, we first randomly sample \(\delta_{i}\in\Delta\) for \(i\in[k]\). We then select the \(i^{\star}\in[k]\) such that \(\ell(G(x,\delta_{i^{\star}}),y;w)\) is maximized.

**Model-based Data Augmentation.** In MDA, rather than explicitly trying to solve the inner problem, we seek a _diversity_ of naturally-varying data rather than the "worst-case." In this way, MDA samples \(\delta_{i}\in\Delta\) for \(i\in[k]\) and then appends \(\{G(x,\delta_{i}),y\}_{i=1}^{k}\) to the training dataset. We present experiments in five different settings over twelve distinct datasets to demonstrate the broad applicability of MBRDL. First, in Sections 5.1, 5.2, we show that our algorithms are the first to consistently provide out-of-distribution robustness across a range of challenging corruptions, including shifts in brightness, contrast, snow, fog, frost, and haze on CURE-TSR, ImageNet, and ImageNet-c. In Section 5.3 we curate several new datasets containing simultaneous sources of natural variation, and we then show that models of natural variation can be composed to provide robustness against these simultaneous shifts. In Section 5.4 we show that models of natural variation trained on a fixed dataset can be reused to provide robustness on datasets entirely unseen during training. Finally, in Section 5.5 we assume access to unlabeled data corresponding to a fixed domain shift, and we compare our algorithms to suitable baselines, including domain adaptation methods.

In many applications, one might have data corresponding to low levels of natural variation, such as a dusting of snow in images of street signs. However, it is often difficult to collect data corresponding to high levels of natural variation, such as images taken during a blizzard. In such cases, we show that our algorithms can be used to provide significant out-of-distribution robustness against data with high levels of natural variation by training on data with relatively low levels of the same source of natural variation. To do so, we use data from the CURE-TSR dataset (Hemel et al., 2019), which contains images of street signs divided into subsets according to various sources of natural variation and corresponding severity levels. For example, for images in the "snow" subset, level 0 corresponds to no snow, whereas level 5 corresponds to a full blizzard. Thus, for each row of Table 1 we use unlabeled data from levels 0 and 2 to learn a model of natural variation corresponding to a given source of natural variation in CURE-TSR.

Note that while this is an unfair comparison for our methods, given that the model-based algorithms are not given access to labeled level 2 data, our algorithms still outperform the baselines by as much as 20 percentage points on level 5 data. To demonstrate the scalability of our approach, we perform experiments on ImageNet (Deng et al., 2009) and the recently-curated ImageNet-c dataset (Hendrycks and Dietterich, 2019). ImageNet-c contains images from the ImageNet test set that are corrupted according to artificial transformations, such as snow, rain, and fog, and are labeled from 1-5 depending on the severity of the corruption. For numerous challenging corruptions, we train models to map from the classes 0-9 of ImageNet to the corresponding classes of ImageNet-c. We then train all networks on classes 10-59 of ImageNet, and test on the corresponding classes for various subsets of ImageNet-c. Note that in this setting, the ImageNet classes used to train the model of natural variation are disjoint from those that are used to train the classifier, so many techniques, including most domain adaptation methods, do not apply; to offer a point of comparison, we include the accuracies of classifiers trained using AugMix, which is a recently proposed method that adds known transformations to the data (Hendrycks et al., 2019). In practice, it is common to encounter multiple simultaneous distributional shifts.

To address this robustness challenge, Because we learn models of natural variation offline before training a classifier, our paradigm can be applied to domains that are _entirely unseen_ while training the model. In particular, we show that models can be reused on similar yet unseen datasets to provide robustness against a common source of natural variation.

Whereas many techniques, including most domain adaptation methods, do not apply in this scenario, in the MBRDL paradigm, we can simply learn a model corresponding to the changes in lighting for the European street signs and then apply this model to the dataset of the American signs. Table 4 shows several experiments of this stripe in which a model \(G\) is learned on one dataset \(\mathcal{D}_{1}\) and then applied on another \(\mathcal{D}_{2}\); we improve robustness on unseen domains by up to 40 percentage points.

In every scenario, our model-based algorithms significantly outperform the baselines, often by 10-20 percentage points. Note that while this is one of the most commonly studied settings in domain adaptation, it represents only one particular setting to which the MBRDL paradigm can be applied. Aside from the algorithms we introduced in Section2 we are not aware of any other algorithms that can be used to address out-of-distribution robustness across the diverse array of tasks presented in the previous section.

Unlike these approaches, our solution does not assume access to unlabeled data from a fixed shift and can be applied to datasets that are entirely unseen during training.

In our work, we propose a more general framework and three novel robust training algorithms that can exploit any suitable generative network, and we show improvements on more challenging, naturally-occurring shifts across twelve distinct datasets.

In this paper, we proposed a novel _model-based robust training paradigm_ for deep learning that provides robustness with respect to models of natural variation. Our notion of robustness offers a departure from adversarial training with respect to norm-bounded data perturbations.

Input: Table 1: Out-of-distribution robustness. In each experiment, we train a model of natural variation to map from challenge-level 0 to challenge-level 2 data from different subsets of CURE-TSR. We then perform model-based training using challenge-level 0 data and test on challenge-levels 3-5.

Input: Figure 1: A new notion of robustness. Past work has focused on perturbation-based adversarial examples, such as Figure 1a. In this paper, we focus on robustness with respect to natural variation, shown in Figure 1b, which often does not obey perceptual or norm-bounded constraints.

Input: Table 4: Transferability of model-based robustness. In each experiment, we train a model of natural variation on a given training dataset D1. Then, we use this model to perform model-based training on a new dataset D1 entirely unseen during the training of the model.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my review of the claim, evidence, and reviewer comments, I find that characterizing this work as a "paradigm shift" is overstated. Here's why:

1. As the reviewer points out, this appears to be more of an extension of existing adversarial training approaches rather than a fundamental paradigm shift. The core methodology still follows adversarial training principles but applies them to a different type of variation.

2. While the evidence shows impressive results and novel contributions in handling natural variations, several limitations exist:
- The approach is still restricted to specific pre-defined variations
- The data augmentation baseline (MDA) performs similarly well
- The method builds upon, rather than fundamentally shifts away from, existing adversarial training frameworks

3. The experimental results, while strong, demonstrate improvements within the existing paradigm rather than establishing a new one.

Therefore, I would rate this claim as significantly overstated, though the underlying work appears solid.

<score>4</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To address this gap, we propose a paradigm shift from perturbation-based adversarial robustness to _model-based robust deep learning_.

The review comment to be evaluated is:

This paper proposes a model-based framework for improving the robustness of image classifiers to average-case corruptions of varying severity. The proposed framework can be thought of as adversarial training where the perturbation is replaced by a function that transforms the image according to a specific corruption. A nuisance parameter controls the instantiation and severity of the corruption that is applied to the input. The paper compares baselines to different versions of this general model-based framework with experiments on several datasets.

Pros:
- Focusing on average-case corruptions is an important and underexplored problem
- Using image translation to learn the corruptions is an interesting proposal

Cons:
- There isn't much novelty in the proposed technique. Adversarial training with 'natural' corruptions has been done several times before.
- There isn't any discussion of how the nuisance parameter is used. I suspect it isn't used for ImageNet-C experiments, though I may be wrong. In this case, the method becomes data augmentation with image translation, which is not very novel either.
- It sounds like the ImageNet-C experiments use the corruptions to train the image translation network. Even though these experiments use held-out images, the corruption types themselves are not being held out, which goes against the recommended methodology for that dataset.

Typo: "ImageNet-c" should be "ImageNet-C"

___________________________________________________________________________________

Update after author feedback:

I thank the authors for improving my understanding of the paper. I feel better about it after reading it again. One of the most interesting findings--that a translation network trained on one domain/set of classes generalizes to another--needs more discussion. Using this phenomenon to improve robustness is a good idea, and the MRT/MDA/MAT methods explored in this work are nice choices for this investigation. However, I agree with the other reviewers that it would be nice to include more baselines from other work where appropriate.

The results in Table 1 are also very interesting and deserve more discussion--perhaps an analysis of whether a translation network trained on weak corruptions can generalize to create something akin to the ground-truth stronger corruptions, as the results in the table for MRT imply.

Overall, I think this paper has some strong experiments and investigates a good idea, but the claims of a "paradigm shift" are overly grandiose, and some of the most interesting experiments could use more analysis. Additionally, the writing clarity and presentation could be improved. My initial understanding of the paper was flawed in some places, so I'll raise my score to 5.

The evidence to be evaluated is:However, neural networks trained using ERM are known to be susceptible to _adversarial attacks_. This means that given a datum \(x\) with a corresponding label \(y\), one can find another datum \(x^{\text{adv}}\) such that \(x\) is close to \(x^{\text{adv}}\) in a given Euclidean norm and \(x^{\text{adv}}\) is predicted by the learned classifier as belonging to a different class \(c\neq y\). If such a datum \(x^{\text{adv}}\) exists, it is called an _adversarial example_.

The dominant paradigm toward improving robustness against adversarial examples relies on a robust optimization perspective wherein neural networks are trained to correctly classify worst-case perturbations of data (Madry et al., 2017; Wong and Kolter, 2017).

\[\operatorname*{arg\,min}_{w}\mathbb{E}_{(x,y)\sim\mathcal{D}}\Big{[}\max_{ \delta\in\Delta}\ell(x+\delta,y;w)\Big{]} \tag{1}\]

Despite remarkable progress toward improving the robustness of DL against norm-bounded perturbations, there are significant limitations to adversarial training. Notably, DL is known to be fragile to many forms of _natural variation_, which cannot be described by small perturbations \(x\mapsto x+\delta\). In image classification, such natural variation includes changes in weather or background color (Eykholt et al., 2018; Hendrycks et al., 2019; Hosseini and Poovendran, 2018), spatial transformations such as rotation or scaling (Xiao et al., 2018; Karanakis et al., 2016), and sensor-based attacks (Kurakin et al., 2016). Because such transformations frequently arise in safety-critical domains, it is critically important for the DL community to develop algorithms that are robust against out-of-distribution, natural variation in data. In this paper, we specifically address this challenge by proposing a principled, optimization-based framework which can be used in general settings to provide robustness against arbitrary sources of natural variation.

In order to effectively model sources of natural variation in a domain-agnostic setting, we will abstractly define _models of natural variation_. Concretely, a model of natural variation \(G(x,\delta)\) is a map that describes how an input datum \(x\) can be naturally varied by a _nuisance parameter_\(\delta\) resulting in a new datum \(x^{\prime}:=G(x,\delta)\). Ideally, for a fixed datum \(x\), varying the nuisance parameter \(\delta\) should vary the severity of the natural conditions in the generated datum \(x^{\prime}\). An example of such a model is shown in Figure 2 where an image \(x\) on the left (in this case, in any weather) can be naturally varied by \(\delta\) and consequently transformed into the image \(x^{\prime}\) on the right (in snowy weather). In the remainder of this subsection, we consider cases in which (1) a model \(G\) is known a priori, and (2) a model \(G\) is unknown and therefore must be learned offline from data.

**Learning models of natural variation from data.** In many situations, models natural variation are not known a priori or are too costly to obtain. For example, consider Figure 1 in which a model \(G(x,\delta)\) takes an image \(x\) of a street sign in sunny weather and maps it to an image \(x^{\prime}:=G(x,\delta)\) in snowy weather. Even though there is a relationship between the two images, obtaining a model \(G\) relating these two domains is extremely challenging if we resort to geometric structure. For such problems we advocate for _learning_ the model \(G\) from data. To do so, we assume that we have access to two unpaired domains \(A\) and \(B\) that are drawn from a common distribution. Domain \(A\) contains the original data, such as the images with sunny weather, and domain \(B\) contains naturally transformed data, such as images with snow. Ideally, a model of natural variation should learn to map images from domain \(A\) to corresponding images with different levels of natural variation captured by the images of domain \(B\).

The model-based robust training paradigm that we propose retains the basic elements of adversarial training described in Section 2. Our point of departure from the classical adversarial training formulation is in the choice of the so-called adversarial perturbation. In this paper, we assume that data can be transformed according to a model of natural variation \(G(x,\delta)\) by choosing different values of \(\delta\) from a given _nuisance space_\(\Delta\). The goal of the model-based approach is to train a classifier that achieves high accuracy both on a test set drawn i.i.d. from \(\mathcal{D}\) and on _more-challenging_ test data that has been subjected to the source of natural variation that \(G\) models. This perspective can be captured by the following robust optimization problem: \[\min_{w}\mathbb{E}_{(x,y)\sim\mathcal{D}}\Big{[}\max_{\delta\in\Delta}\ell(G( x,\delta),y;w)\Big{]}. \tag{2}\] In the inner maximization problem of this formulation, given an instance-label pair \((x,y)\), we seek a vector \(\delta^{*}\in\Delta\) that produces a corresponding instance \(x^{\prime}:=G(x,\delta^{*})\) which gives rise to high loss values \(\ell(G(x,\delta^{*}),y;w)\) under the current weight \(w\). One can think of this vector \(\delta^{*}\) as characterizing the _worst-case_ nuisance that can be generated by the model \(G(x,\delta^{*})\) for the original instance \(x\). After solving this inner problem, we solve the outer minimization problem by finding weights \(w\) that minimize the risk against the challenging instance \(G(x,\delta^{*})\). By training the network to correctly classify this worst-case data, the goal is to become invariant to the model \(G(x,\delta)\) for any \(\delta\in\Delta\). We now assume that we have access to a suitable model of natural variation \(G(x,\delta)\) and shift our attention toward exploiting \(G\) in the development of novel robust training algorithms.

Specifically, we propose three algorithmic variants: _Model-based Adversarial Training_ (MAT), _Model-based Robust Training_ (MRT), and _Model-based Data Augmentation_ (MDA).

**Model-based Adversarial Training.** In MAT, we seek an exact solution to the inner problem by performing \(k\) steps of gradient ascent in \(\delta\) on the objective \(\ell(G(x,\delta),y;w)\). The resulting nuisance parameter \(\delta^{\star}\) is one that causes \(\ell(G(x,\delta^{\star}),y;w)\) to have high loss under the current weight \(w\). **Model-based Robust Training.** In MRT, we first randomly sample \(\delta_{i}\in\Delta\) for \(i\in[k]\). We then select the \(i^{\star}\in[k]\) such that \(\ell(G(x,\delta_{i^{\star}}),y;w)\) is maximized.

**Model-based Data Augmentation.** In MDA, rather than explicitly trying to solve the inner problem, we seek a _diversity_ of naturally-varying data rather than the "worst-case." In this way, MDA samples \(\delta_{i}\in\Delta\) for \(i\in[k]\) and then appends \(\{G(x,\delta_{i}),y\}_{i=1}^{k}\) to the training dataset. We present experiments in five different settings over twelve distinct datasets to demonstrate the broad applicability of MBRDL. First, in Sections 5.1, 5.2, we show that our algorithms are the first to consistently provide out-of-distribution robustness across a range of challenging corruptions, including shifts in brightness, contrast, snow, fog, frost, and haze on CURE-TSR, ImageNet, and ImageNet-c. In Section 5.3 we curate several new datasets containing simultaneous sources of natural variation, and we then show that models of natural variation can be composed to provide robustness against these simultaneous shifts. In Section 5.4 we show that models of natural variation trained on a fixed dataset can be reused to provide robustness on datasets entirely unseen during training. Finally, in Section 5.5 we assume access to unlabeled data corresponding to a fixed domain shift, and we compare our algorithms to suitable baselines, including domain adaptation methods.

In many applications, one might have data corresponding to low levels of natural variation, such as a dusting of snow in images of street signs. However, it is often difficult to collect data corresponding to high levels of natural variation, such as images taken during a blizzard. In such cases, we show that our algorithms can be used to provide significant out-of-distribution robustness against data with high levels of natural variation by training on data with relatively low levels of the same source of natural variation. To do so, we use data from the CURE-TSR dataset (Hemel et al., 2019), which contains images of street signs divided into subsets according to various sources of natural variation and corresponding severity levels. For example, for images in the "snow" subset, level 0 corresponds to no snow, whereas level 5 corresponds to a full blizzard. Thus, for each row of Table 1 we use unlabeled data from levels 0 and 2 to learn a model of natural variation corresponding to a given source of natural variation in CURE-TSR.

Note that while this is an unfair comparison for our methods, given that the model-based algorithms are not given access to labeled level 2 data, our algorithms still outperform the baselines by as much as 20 percentage points on level 5 data. To demonstrate the scalability of our approach, we perform experiments on ImageNet (Deng et al., 2009) and the recently-curated ImageNet-c dataset (Hendrycks and Dietterich, 2019). ImageNet-c contains images from the ImageNet test set that are corrupted according to artificial transformations, such as snow, rain, and fog, and are labeled from 1-5 depending on the severity of the corruption. For numerous challenging corruptions, we train models to map from the classes 0-9 of ImageNet to the corresponding classes of ImageNet-c. We then train all networks on classes 10-59 of ImageNet, and test on the corresponding classes for various subsets of ImageNet-c. Note that in this setting, the ImageNet classes used to train the model of natural variation are disjoint from those that are used to train the classifier, so many techniques, including most domain adaptation methods, do not apply; to offer a point of comparison, we include the accuracies of classifiers trained using AugMix, which is a recently proposed method that adds known transformations to the data (Hendrycks et al., 2019). In practice, it is common to encounter multiple simultaneous distributional shifts.

To address this robustness challenge, Because we learn models of natural variation offline before training a classifier, our paradigm can be applied to domains that are _entirely unseen_ while training the model. In particular, we show that models can be reused on similar yet unseen datasets to provide robustness against a common source of natural variation.

Whereas many techniques, including most domain adaptation methods, do not apply in this scenario, in the MBRDL paradigm, we can simply learn a model corresponding to the changes in lighting for the European street signs and then apply this model to the dataset of the American signs. Table 4 shows several experiments of this stripe in which a model \(G\) is learned on one dataset \(\mathcal{D}_{1}\) and then applied on another \(\mathcal{D}_{2}\); we improve robustness on unseen domains by up to 40 percentage points.

In every scenario, our model-based algorithms significantly outperform the baselines, often by 10-20 percentage points. Note that while this is one of the most commonly studied settings in domain adaptation, it represents only one particular setting to which the MBRDL paradigm can be applied. Aside from the algorithms we introduced in Section2 we are not aware of any other algorithms that can be used to address out-of-distribution robustness across the diverse array of tasks presented in the previous section.

Unlike these approaches, our solution does not assume access to unlabeled data from a fixed shift and can be applied to datasets that are entirely unseen during training.

In our work, we propose a more general framework and three novel robust training algorithms that can exploit any suitable generative network, and we show improvements on more challenging, naturally-occurring shifts across twelve distinct datasets.

In this paper, we proposed a novel _model-based robust training paradigm_ for deep learning that provides robustness with respect to models of natural variation. Our notion of robustness offers a departure from adversarial training with respect to norm-bounded data perturbations.

Input: Table 1: Out-of-distribution robustness. In each experiment, we train a model of natural variation to map from challenge-level 0 to challenge-level 2 data from different subsets of CURE-TSR. We then perform model-based training using challenge-level 0 data and test on challenge-levels 3-5.

Input: Figure 1: A new notion of robustness. Past work has focused on perturbation-based adversarial examples, such as Figure 1a. In this paper, we focus on robustness with respect to natural variation, shown in Figure 1b, which often does not obey perceptual or norm-bounded constraints.

Input: Table 4: Transferability of model-based robustness. In each experiment, we train a model of natural variation on a given training dataset D1. Then, we use this model to perform model-based training on a new dataset D1 entirely unseen during the training of the model.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my analysis:

The claim proposing "a paradigm shift from perturbation-based adversarial robustness to model-based robust deep learning" appears overstated given the evidence. While the paper demonstrates valuable contributions in handling natural variations and shows impressive results across multiple datasets, calling it a "paradigm shift" is too grandiose, as noted in the review comments. The evidence shows an extension/evolution of existing adversarial training approaches rather than a fundamental paradigm change. The method builds upon, rather than replaces, existing robustness concepts.

Key points supporting this assessment:
- The reviewer explicitly states the claims of a "paradigm shift" are "overly grandiose"
- The approach still uses core concepts from adversarial training
- The reviewer notes "there isn't much novelty in the proposed technique" as similar approaches exist
- The results, while strong, represent incremental improvement rather than revolutionary change

<score>4</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To address this gap, we propose a paradigm shift from perturbation-based adversarial robustness to _model-based robust deep learning_.

The review comment to be evaluated is:

The paper looks at the idea of building models of natural variation of an input and then using these models to develop robust training algorithms that are less susceptible to outliers ( testable for worst-cases via adversarial attacks ).

Strength:

+ The paper addresses a very important topic of adversarial robustness of DNN models and is accompanied by diligent evaluation over different datasets.

Weakness:

- The paper adopts an approach which is very reminiscent of Ajil Jalal, Andrew Ilyas, Constantinos Daskalakis, and Alexandros G Dimakis. The robust manifold defense: Adversarial training using generative models. arXiv preprint arXiv:1712.09196, 2017. The key difference seems to be adoption of the idea of using auxiliary transformations (called natural perturbations) which is also very well-studied in literature, for e.g. see https://openaccess.thecvf.com/content_CVPR_2020/papers/Zhang_Auxiliary_Training_Towards_Accurate_and_Robust_Models_CVPR_2020_paper.pdf  So, the approach presented here is quite incremental for a premier venue such as ICLR.

- The attack in Jalal et. al. paper can be launched against this approach once the attacker also access to this generative/natural model (which would be easy to build for an attacker)

- The reviewer will strongly recommend reviewing the advices in https://arxiv.org/abs/1902.06705 for writing papers on defense and how to self-evaluate its robustness by suitably designing the attacker. If the defense approach uses some background/auxiliary knowledge, one must consider the attacker with this knowledge if it is accessible to the attacker.

Questions to authors:

- Can authors explain why an attacker can't build similar natural model to defeat the proposed defense?
- Any clarification on incremental novelty from  Jalal et. al. and Zhang et. al. (and references therein) would be also useful.

After author's rebuttal:

"The other paper that the review points to ([4]) addresses a similar setting to our paper, but the approach is completely different. ... Moreover, [4] was published within three months of the ICLR submission deadline, meaning that it is essentially concurrent with our work."

Yes, the reviewer concurs that a work published so close to deadline should be treated as concurrent and will raise the score.

The evidence to be evaluated is:However, neural networks trained using ERM are known to be susceptible to _adversarial attacks_. This means that given a datum \(x\) with a corresponding label \(y\), one can find another datum \(x^{\text{adv}}\) such that \(x\) is close to \(x^{\text{adv}}\) in a given Euclidean norm and \(x^{\text{adv}}\) is predicted by the learned classifier as belonging to a different class \(c\neq y\). If such a datum \(x^{\text{adv}}\) exists, it is called an _adversarial example_.

The dominant paradigm toward improving robustness against adversarial examples relies on a robust optimization perspective wherein neural networks are trained to correctly classify worst-case perturbations of data (Madry et al., 2017; Wong and Kolter, 2017).

\[\operatorname*{arg\,min}_{w}\mathbb{E}_{(x,y)\sim\mathcal{D}}\Big{[}\max_{ \delta\in\Delta}\ell(x+\delta,y;w)\Big{]} \tag{1}\]

Despite remarkable progress toward improving the robustness of DL against norm-bounded perturbations, there are significant limitations to adversarial training. Notably, DL is known to be fragile to many forms of _natural variation_, which cannot be described by small perturbations \(x\mapsto x+\delta\). In image classification, such natural variation includes changes in weather or background color (Eykholt et al., 2018; Hendrycks et al., 2019; Hosseini and Poovendran, 2018), spatial transformations such as rotation or scaling (Xiao et al., 2018; Karanakis et al., 2016), and sensor-based attacks (Kurakin et al., 2016). Because such transformations frequently arise in safety-critical domains, it is critically important for the DL community to develop algorithms that are robust against out-of-distribution, natural variation in data. In this paper, we specifically address this challenge by proposing a principled, optimization-based framework which can be used in general settings to provide robustness against arbitrary sources of natural variation.

In order to effectively model sources of natural variation in a domain-agnostic setting, we will abstractly define _models of natural variation_. Concretely, a model of natural variation \(G(x,\delta)\) is a map that describes how an input datum \(x\) can be naturally varied by a _nuisance parameter_\(\delta\) resulting in a new datum \(x^{\prime}:=G(x,\delta)\). Ideally, for a fixed datum \(x\), varying the nuisance parameter \(\delta\) should vary the severity of the natural conditions in the generated datum \(x^{\prime}\). An example of such a model is shown in Figure 2 where an image \(x\) on the left (in this case, in any weather) can be naturally varied by \(\delta\) and consequently transformed into the image \(x^{\prime}\) on the right (in snowy weather). In the remainder of this subsection, we consider cases in which (1) a model \(G\) is known a priori, and (2) a model \(G\) is unknown and therefore must be learned offline from data.

**Learning models of natural variation from data.** In many situations, models natural variation are not known a priori or are too costly to obtain. For example, consider Figure 1 in which a model \(G(x,\delta)\) takes an image \(x\) of a street sign in sunny weather and maps it to an image \(x^{\prime}:=G(x,\delta)\) in snowy weather. Even though there is a relationship between the two images, obtaining a model \(G\) relating these two domains is extremely challenging if we resort to geometric structure. For such problems we advocate for _learning_ the model \(G\) from data. To do so, we assume that we have access to two unpaired domains \(A\) and \(B\) that are drawn from a common distribution. Domain \(A\) contains the original data, such as the images with sunny weather, and domain \(B\) contains naturally transformed data, such as images with snow. Ideally, a model of natural variation should learn to map images from domain \(A\) to corresponding images with different levels of natural variation captured by the images of domain \(B\).

The model-based robust training paradigm that we propose retains the basic elements of adversarial training described in Section 2. Our point of departure from the classical adversarial training formulation is in the choice of the so-called adversarial perturbation. In this paper, we assume that data can be transformed according to a model of natural variation \(G(x,\delta)\) by choosing different values of \(\delta\) from a given _nuisance space_\(\Delta\). The goal of the model-based approach is to train a classifier that achieves high accuracy both on a test set drawn i.i.d. from \(\mathcal{D}\) and on _more-challenging_ test data that has been subjected to the source of natural variation that \(G\) models. This perspective can be captured by the following robust optimization problem: \[\min_{w}\mathbb{E}_{(x,y)\sim\mathcal{D}}\Big{[}\max_{\delta\in\Delta}\ell(G( x,\delta),y;w)\Big{]}. \tag{2}\] In the inner maximization problem of this formulation, given an instance-label pair \((x,y)\), we seek a vector \(\delta^{*}\in\Delta\) that produces a corresponding instance \(x^{\prime}:=G(x,\delta^{*})\) which gives rise to high loss values \(\ell(G(x,\delta^{*}),y;w)\) under the current weight \(w\). One can think of this vector \(\delta^{*}\) as characterizing the _worst-case_ nuisance that can be generated by the model \(G(x,\delta^{*})\) for the original instance \(x\). After solving this inner problem, we solve the outer minimization problem by finding weights \(w\) that minimize the risk against the challenging instance \(G(x,\delta^{*})\). By training the network to correctly classify this worst-case data, the goal is to become invariant to the model \(G(x,\delta)\) for any \(\delta\in\Delta\). We now assume that we have access to a suitable model of natural variation \(G(x,\delta)\) and shift our attention toward exploiting \(G\) in the development of novel robust training algorithms.

Specifically, we propose three algorithmic variants: _Model-based Adversarial Training_ (MAT), _Model-based Robust Training_ (MRT), and _Model-based Data Augmentation_ (MDA).

**Model-based Adversarial Training.** In MAT, we seek an exact solution to the inner problem by performing \(k\) steps of gradient ascent in \(\delta\) on the objective \(\ell(G(x,\delta),y;w)\). The resulting nuisance parameter \(\delta^{\star}\) is one that causes \(\ell(G(x,\delta^{\star}),y;w)\) to have high loss under the current weight \(w\). **Model-based Robust Training.** In MRT, we first randomly sample \(\delta_{i}\in\Delta\) for \(i\in[k]\). We then select the \(i^{\star}\in[k]\) such that \(\ell(G(x,\delta_{i^{\star}}),y;w)\) is maximized.

**Model-based Data Augmentation.** In MDA, rather than explicitly trying to solve the inner problem, we seek a _diversity_ of naturally-varying data rather than the "worst-case." In this way, MDA samples \(\delta_{i}\in\Delta\) for \(i\in[k]\) and then appends \(\{G(x,\delta_{i}),y\}_{i=1}^{k}\) to the training dataset. We present experiments in five different settings over twelve distinct datasets to demonstrate the broad applicability of MBRDL. First, in Sections 5.1, 5.2, we show that our algorithms are the first to consistently provide out-of-distribution robustness across a range of challenging corruptions, including shifts in brightness, contrast, snow, fog, frost, and haze on CURE-TSR, ImageNet, and ImageNet-c. In Section 5.3 we curate several new datasets containing simultaneous sources of natural variation, and we then show that models of natural variation can be composed to provide robustness against these simultaneous shifts. In Section 5.4 we show that models of natural variation trained on a fixed dataset can be reused to provide robustness on datasets entirely unseen during training. Finally, in Section 5.5 we assume access to unlabeled data corresponding to a fixed domain shift, and we compare our algorithms to suitable baselines, including domain adaptation methods.

In many applications, one might have data corresponding to low levels of natural variation, such as a dusting of snow in images of street signs. However, it is often difficult to collect data corresponding to high levels of natural variation, such as images taken during a blizzard. In such cases, we show that our algorithms can be used to provide significant out-of-distribution robustness against data with high levels of natural variation by training on data with relatively low levels of the same source of natural variation. To do so, we use data from the CURE-TSR dataset (Hemel et al., 2019), which contains images of street signs divided into subsets according to various sources of natural variation and corresponding severity levels. For example, for images in the "snow" subset, level 0 corresponds to no snow, whereas level 5 corresponds to a full blizzard. Thus, for each row of Table 1 we use unlabeled data from levels 0 and 2 to learn a model of natural variation corresponding to a given source of natural variation in CURE-TSR.

Note that while this is an unfair comparison for our methods, given that the model-based algorithms are not given access to labeled level 2 data, our algorithms still outperform the baselines by as much as 20 percentage points on level 5 data. To demonstrate the scalability of our approach, we perform experiments on ImageNet (Deng et al., 2009) and the recently-curated ImageNet-c dataset (Hendrycks and Dietterich, 2019). ImageNet-c contains images from the ImageNet test set that are corrupted according to artificial transformations, such as snow, rain, and fog, and are labeled from 1-5 depending on the severity of the corruption. For numerous challenging corruptions, we train models to map from the classes 0-9 of ImageNet to the corresponding classes of ImageNet-c. We then train all networks on classes 10-59 of ImageNet, and test on the corresponding classes for various subsets of ImageNet-c. Note that in this setting, the ImageNet classes used to train the model of natural variation are disjoint from those that are used to train the classifier, so many techniques, including most domain adaptation methods, do not apply; to offer a point of comparison, we include the accuracies of classifiers trained using AugMix, which is a recently proposed method that adds known transformations to the data (Hendrycks et al., 2019). In practice, it is common to encounter multiple simultaneous distributional shifts.

To address this robustness challenge, Because we learn models of natural variation offline before training a classifier, our paradigm can be applied to domains that are _entirely unseen_ while training the model. In particular, we show that models can be reused on similar yet unseen datasets to provide robustness against a common source of natural variation.

Whereas many techniques, including most domain adaptation methods, do not apply in this scenario, in the MBRDL paradigm, we can simply learn a model corresponding to the changes in lighting for the European street signs and then apply this model to the dataset of the American signs. Table 4 shows several experiments of this stripe in which a model \(G\) is learned on one dataset \(\mathcal{D}_{1}\) and then applied on another \(\mathcal{D}_{2}\); we improve robustness on unseen domains by up to 40 percentage points.

In every scenario, our model-based algorithms significantly outperform the baselines, often by 10-20 percentage points. Note that while this is one of the most commonly studied settings in domain adaptation, it represents only one particular setting to which the MBRDL paradigm can be applied. Aside from the algorithms we introduced in Section2 we are not aware of any other algorithms that can be used to address out-of-distribution robustness across the diverse array of tasks presented in the previous section.

Unlike these approaches, our solution does not assume access to unlabeled data from a fixed shift and can be applied to datasets that are entirely unseen during training.

In our work, we propose a more general framework and three novel robust training algorithms that can exploit any suitable generative network, and we show improvements on more challenging, naturally-occurring shifts across twelve distinct datasets.

In this paper, we proposed a novel _model-based robust training paradigm_ for deep learning that provides robustness with respect to models of natural variation. Our notion of robustness offers a departure from adversarial training with respect to norm-bounded data perturbations.

Input: Table 1: Out-of-distribution robustness. In each experiment, we train a model of natural variation to map from challenge-level 0 to challenge-level 2 data from different subsets of CURE-TSR. We then perform model-based training using challenge-level 0 data and test on challenge-levels 3-5.

Input: Figure 1: A new notion of robustness. Past work has focused on perturbation-based adversarial examples, such as Figure 1a. In this paper, we focus on robustness with respect to natural variation, shown in Figure 1b, which often does not obey perceptual or norm-bounded constraints.

Input: Table 4: Transferability of model-based robustness. In each experiment, we train a model of natural variation on a given training dataset D1. Then, we use this model to perform model-based training on a new dataset D1 entirely unseen during the training of the model.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my review of the claim, evidence, and reviewer comments, I'll evaluate whether proposing "a paradigm shift from perturbation-based adversarial robustness to model-based robust deep learning" is well-stated or overstated.

Summary:
The evidence strongly supports this claim through:
1. Extensive experimental results across 12 datasets showing significant improvements (10-40 percentage points) over baselines
2. Demonstrated versatility in handling multiple types of natural variations
3. Proven transferability to unseen domains
4. Novel algorithmic approaches (MAT, MRT, MDA)

However, the reviewer raises valid concerns about:
- Similarity to previous work (Jalal et al.)
- Potential vulnerability to attackers who could build similar natural models
- Incremental rather than paradigm-shifting advancement

While the technical contribution is solid, calling it a "paradigm shift" overstates the novelty given the existing literature and methodological similarities to prior work.

<score>4</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We show that our algorithms can significantly improve the robustness of classifiers trained on ImageNet and tested on ImageNet-c by as much as 30 percentage points.

The evidence to be evaluated is:In this paper, we specifically address this challenge by proposing a principled, optimization-based framework which can be used in general settings to provide robustness against arbitrary sources of natural variation. Underlying the task of improving the robustness of neural networks against natural, out-of-distribution data are two fundamental challenges. Firstly, unlike in the adversarial robustness community, in real-world, safety-critical environments, data can vary in unknown and highly nonlinear ways. Thus, the first step toward building a robust training procedure must be to design a mechanism that accurately describes how data varies in such environments.

In MDA, rather than explicitly trying to solve the inner problem, we seek a _diversity_ of naturally-varying data rather than the "worst-case." In this way, MDA samples \(\delta_{i}\in\Delta\) for \(i\in[k]\) and then appends \(\{G(x,\delta_{i}),y\}_{i=1}^{k}\) to the training dataset.

First, in Sections 5.1, 5.2, we show that our algorithms are the first to consistently provide out-of-distribution robustness across a range of challenging corruptions, including shifts in brightness, contrast, snow, fog, frost, and haze on CURE-TSR, ImageNet, and ImageNet-c.

In such cases, we show that our algorithms can be used to provide significant out-of-distribution robustness against data with high levels of natural variation by training on data with relatively low levels of the same source of natural variation. To do so, we use data from the CURE-TSR dataset (Hemel et al., 2019), which contains images of street signs divided into subsets according to various sources of natural variation and corresponding severity levels. For example, for images in the "snow" subset, level 0 corresponds to no snow, whereas level 5 corresponds to a full blizzard. Thus, for each row of Table 1 we use unlabeled data from levels 0 and 2 to learn a model of natural variation corresponding to a given source of natural variation in CURE-TSR.

Note that while this is an unfair comparison for our methods, given that the model-based algorithms are not given access to labeled level 2 data, our algorithms still outperform the baselines by as much as 20 percentage points on level 5 data. To demonstrate the scalability of our approach, we perform experiments on ImageNet (Deng et al., 2009) and the recently-curated ImageNet-c dataset (Hendrycks and Dietterich, 2019). ImageNet-c contains images from the ImageNet test set that are corrupted according to artificial transformations, such as snow, rain, and fog, and are labeled from 1-5 depending on the severity of the corruption.

In particular, we show that models can be reused on similar yet unseen datasets to provide robustness against a common source of natural variation.

Whereas many techniques, including most domain adaptation methods, do not apply in this scenario, in the MBRDL paradigm, we can simply learn a model corresponding to the changes in lighting for the European street signs and then apply this model to the dataset of the American signs.

we improve robustness on unseen domains by up to 40 percentage points.

In every scenario, our model-based algorithms significantly outperform the baselines, often by 10-20 percentage points.

Unlike these approaches, our solution does not assume access to unlabeled data from a fixed shift and can be applied to datasets that are entirely unseen during training.

Input: Table 1: Out-of-distribution robustness. In each experiment, we train a model of natural variation to map from challenge-level 0 to challenge-level 2 data from different subsets of CURE-TSR. We then perform model-based training using challenge-level 0 data and test on challenge-levels 3-5.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim stating "algorithms can significantly improve robustness... by as much as 30 percentage points" appears to be overstated. While the evidence does show impressive improvements, the specific data in Table 1 and text demonstrate improvements typically in the range of 10-20 percentage points, with one mention of "up to 40 percentage points" in a different context. The table shows variations in performance across different conditions, but doesn't support the specific 30 percentage point claim. The improvements, while significant, are generally reported as "10-20 percentage points" in most cases.

<score>4</score>

Justification:
1. The evidence primarily cites improvements of "10-20 percentage points" in multiple instances
2. Table 1 data shows varying improvements, but doesn't demonstrate consistent 30 percentage point gains
3. While there is one mention of "up to 40 percentage points," this appears to be in a different context (unseen domains) than the ImageNet/ImageNet-C claim
4. The claim about ImageNet/ImageNet-C specifically lacks direct numerical evidence in the provided text
5. The actual reported improvements are significant but generally lower than the claimed 30 percentage points

The claim would be more accurate if it stated improvements of "up to 20 percentage points" based on the consistently reported evidence.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

We show that our algorithms can significantly improve the robustness of classifiers trained on ImageNet and tested on ImageNet-c by as much as 30 percentage points.

The review comment to be evaluated is:

# Summary
This paper proposes a “paradigm shift” for augmenting datasets when training CNN-based image classifiers. On one side, traditional augmentations include blur, Gaussian noise, color distortions. On the other side, methods like adversarial training consider augmentations under norm bounds in the image space. The proposed method, instead, uses models of natural variation to augment the images. Increased out-of-domain accuracy is shown on ImageNet-C and several slices of the CURE-TSR dataset.

Central question: How to shift from perturbation-based robustness analysis to model-based robustness analysis?

# Strong and Weak points

## Strong points

  * This paper introduces a “paradigm shift” from perturbation based to model-based augmentation, when training CNN based image classifiers. The existing perturbations only reach so far, and using models of natural variation provides a new avenue for augmenting data sets.
  * The method is presented in clear pseudo code, and detailed in the 36 page counting appendix.

## Weak points

  * The paper claims a “paradigm shift”, but the proposed method of augmenting data has been used since 2012,  [1], and is used ever since [2]. Tables 2 and 3 provide key results for this method, but both were obtained with Model-based Data Augmentation, which is the common method before the "paradigm shift".
  * Results outlined in Section 5.2 and Table 2 concerns scenarios where the test-time perturbation is known at training time. Table 2 then compares MDA to ERM using this a-priori knowledge, and shows improved accuracy. However, the comparison is unbalanced, as ERM does not benefit from the prior knowledge. If Table 2 would be about comparing methods under the assumption that the test-time perturbation would be known at training time, then the ERM should be trained with the same data augmentation.
  * The paper misses comparisons against previous published work. Although Table 2 and 4 compare against another method, these results were not obtained by the respective authors, thus introducing a bias. The related work, Section 6, cites as many as 9 papers that addressed the same problem as this paper. The argument for the proposed method could be stronger when comparing to results from any of these papers.
Moreover, Table 3 reports results on ImageNet-C, which multiple other research papers have published results on. See for example, [3] and [4]. Likewise, Table 4 reports results on MNIST-C, which have been reported in [5] and [6]. Omitting those comparisons hampers the argument for the proposed method.

[1] Krizhevsky, 2012
[2] Szegedy, 2015
[3] https://paperswithcode.com/sota/domain-generalization-on-imagenet-c
[4] https://arxiv.org/pdf/1903.12261.pdf
[5] https://proceedings.icml.cc/paper/2020/file/20546457187cf3d52ea86538403e47cc-Paper.pdf
[6] https://arxiv.org/pdf/1906.02337.pdf

# Statement

Recommendation: Reject
Reason:
Although this paper provides four tables of extensive evaluation of the method, none of the numbers were obtained from previous literature. The related work section cites 9 papers on the same problem, but none were compared with in the results section. (See concrete details above)

# Clarifying questions

  * How to quantify the increase in computational complexity for the inner maximization in Equation (2)? As I understand, even evaluating one point requires a forward pass of the variation-model and the classifier-model. In case of iterative optimization, this would also require two backward passes(see line 5 in Algorithm (1)).
If the proposed method requires significantly more compute, the comparison should be made with a method that has access to the same amount of compute (for example a larger model, or longer trained model).
  * Line 6 of Algorithm (1). I miss the definition of set $\Delta$ and the projection operator in the text nor the appendix. Please explain?
  * Section 5.2 speaks of ImageNet “classes”. I am confused if these “classes” are referring to the ImageNet classes (1000 in total) or the severity levels in ImageNet-C (75 in total). Could you please clarify?
  Moreover, this argument implicitly assumes that the test-time perturbation is known at training-time. Could you name a scenario where this assumption is justified?
  * To what extent is the accuracy on the clean data preserved by the proposed method? Table 3, for example, shows an increase in top-1 accuracy on ImageNet data under perturbation. However, I wonder if this model preserves accuracy on the clean data. We know from existing literature [1] that a trade off exists for adversarial training between accuracy on perturbed and clean data. Providing the clean accuracy would provide evidence that this method compares favourably on this trade off.

[1] Zhang, Hongyang, et al. "Theoretically principled trade-off between robustness and accuracy." arXiv preprint arXiv:1901.08573 (2019).

# Minor feedback

This minor feedback is not part of the assessment.

  * “arguably more common” -> “arguably more important”
  * “x with a corresponding label y” -> “x with a corresponding prediction y”
  * Abbreviation “MBRDL” appears in Section 5 without any explanation or definition.
  * Abbreviation “CURE-TSR” appears in Section 5.0 without any explanation or definition.
  * The citation for CURE in Section 5.1 is wrong. The bibliography refers to “Temel, Dogancan, Min-Hung Chen, and Ghassan AlRegib. "Traffic sign detection under challenging conditions: A deeper look into performance variations and spectral characteristics." IEEE Transactions on Intelligent Transportation Systems (2019).”, but the CURE TSR dataset was published in “Temel, Dogancan, et al. "CURE-TSR: Challenging unreal and real environments for traffic sign recognition." arXiv preprint arXiv:1712.02463 (2017).”
  * Caption of Table 4: “ Then, we use this model to perform model-based training on a new dataset D1” -> “ Then, we use this model to perform model-based testing on a new dataset D2”?

The evidence to be evaluated is:In this paper, we specifically address this challenge by proposing a principled, optimization-based framework which can be used in general settings to provide robustness against arbitrary sources of natural variation. Underlying the task of improving the robustness of neural networks against natural, out-of-distribution data are two fundamental challenges. Firstly, unlike in the adversarial robustness community, in real-world, safety-critical environments, data can vary in unknown and highly nonlinear ways. Thus, the first step toward building a robust training procedure must be to design a mechanism that accurately describes how data varies in such environments.

In MDA, rather than explicitly trying to solve the inner problem, we seek a _diversity_ of naturally-varying data rather than the "worst-case." In this way, MDA samples \(\delta_{i}\in\Delta\) for \(i\in[k]\) and then appends \(\{G(x,\delta_{i}),y\}_{i=1}^{k}\) to the training dataset.

First, in Sections 5.1, 5.2, we show that our algorithms are the first to consistently provide out-of-distribution robustness across a range of challenging corruptions, including shifts in brightness, contrast, snow, fog, frost, and haze on CURE-TSR, ImageNet, and ImageNet-c.

In such cases, we show that our algorithms can be used to provide significant out-of-distribution robustness against data with high levels of natural variation by training on data with relatively low levels of the same source of natural variation. To do so, we use data from the CURE-TSR dataset (Hemel et al., 2019), which contains images of street signs divided into subsets according to various sources of natural variation and corresponding severity levels. For example, for images in the "snow" subset, level 0 corresponds to no snow, whereas level 5 corresponds to a full blizzard. Thus, for each row of Table 1 we use unlabeled data from levels 0 and 2 to learn a model of natural variation corresponding to a given source of natural variation in CURE-TSR.

Note that while this is an unfair comparison for our methods, given that the model-based algorithms are not given access to labeled level 2 data, our algorithms still outperform the baselines by as much as 20 percentage points on level 5 data. To demonstrate the scalability of our approach, we perform experiments on ImageNet (Deng et al., 2009) and the recently-curated ImageNet-c dataset (Hendrycks and Dietterich, 2019). ImageNet-c contains images from the ImageNet test set that are corrupted according to artificial transformations, such as snow, rain, and fog, and are labeled from 1-5 depending on the severity of the corruption.

In particular, we show that models can be reused on similar yet unseen datasets to provide robustness against a common source of natural variation.

Whereas many techniques, including most domain adaptation methods, do not apply in this scenario, in the MBRDL paradigm, we can simply learn a model corresponding to the changes in lighting for the European street signs and then apply this model to the dataset of the American signs.

we improve robustness on unseen domains by up to 40 percentage points.

In every scenario, our model-based algorithms significantly outperform the baselines, often by 10-20 percentage points.

Unlike these approaches, our solution does not assume access to unlabeled data from a fixed shift and can be applied to datasets that are entirely unseen during training.

Input: Table 1: Out-of-distribution robustness. In each experiment, we train a model of natural variation to map from challenge-level 0 to challenge-level 2 data from different subsets of CURE-TSR. We then perform model-based training using challenge-level 0 data and test on challenge-levels 3-5.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence:

The claim stating "30 percentage points improvement in robustness on ImageNet/ImageNet-C" appears overstated based on several factors:

1. The reviewer points out missing comparisons against previous published work on ImageNet-C, suggesting potential selection bias in reported results.

2. While Table 1 shows improvements ranging from 10-20 percentage points in most cases, the specific 30 percentage point improvement on ImageNet-C is not clearly demonstrated in the provided evidence.

3. The text mentions improvements "up to 40 percentage points" and "often by 10-20 percentage points" but these refer to different datasets/scenarios, not specifically ImageNet/ImageNet-C.

4. The review raises valid concerns about unbalanced comparisons and lack of comprehensive benchmarking against existing methods.

<score>4</score>

The claim appears overstated due to selective reporting and lack of comprehensive validation against existing methods, though some significant improvements are demonstrated in specific cases.