Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To address the mentioned challenges, we propose to use a "non-adversarial" diffusion-based anomaly data generation method, which can estimate the true normal distribution accurately and smoothly over the training time.

The review comment to be evaluated is:

### Strengths

- The paper is, for the most part, well-written, easy-to-follow, and does a good job of
contextualising and motivating the work.
- The proposed method is simple and intuitive, and seems to perform well empirically
compared with the baselines in both the regular ND setting and the near ND setting, showing a
significant reduction in the performance gap of going from the former to the latter.
the NR to near-NR setting.
- The figures are well-chosen and well-put-together: they convey well the essential points of the
  paper, such as the shortcomings of existing methods and the feasibility of using GANs vs diffusion
  models for creating the synthetic data.
- Evaluation performed using a good range and selection of baseline methods and benchmark datasets.
- Results are aggregated over a good number (10) of replicates.

### Weaknesses
- Despite the authors claiming otherwise, the method is quite sensitive to the stopping criterion in
  the near ND setting with an empirical range of [68.2, 90.0]; in practice, different datasets may
  require wildly different FID thresholds and there is no reliable way of validating this given the
  nature of the problem. Also, I wonder if the number of sampling steps and the choice of sampler have a
  significant effect on the results -- there doesn't seem to be any mention of this.
  Taking this a step further, might it be feasible to train the diffusion model fully and then rely
  upon early-stopping the reverse diffusion process according to the FID score?
- The need to train, and indeed sample, from a diffusion model can be computationally burdensome for
  higher-resolution datasets.
- The description of the SDE model is a little overdone; a much briefer description with a pointing of the
  reader to the original paper would be sufficient. This is especially the case given that, as
  I understand it, the proposed method should work with any diffusion based model (Imagen, Parti,
  DALLE-2, etc.) or are there theoretically some constraints on the form said model and the
  associated sampler should take?
- While this applies to several methods of the same class, the method seems to make certain
assumptions about the distribution of the ID and OOD data that might not hold in the wild.
- While results are aggregated over a good number (10) of replicates no measure of dispersion
  (standard deviation/error) seems to be reported alongside the mean in each case.
- Some typographic and syntactical errors (e.g. 'that unlikely come' (line 1 of the introduction)).

The evidence to be evaluated is:In the first step, a fake dataset of anomalies is generated by a SDE-based diffusion model. We quantitatively and qualitatively show that the generated fake outliers are high-quality, diverse, and yet show semantic differences compared to the normal inputs.

In order to generate near anomalous samples, a basic diffusion model, called Stochastic Differential Equation (SDE) (43) is adopted.

A SDE model considers a continuum of distributions that develop over time in accordance with a diffusion process, to corrupt a sample and turn it into pure noise.

The model is trained to learn how to gradually transform the noisy image back to its original form through the score fucntion (backward process). Running the backward process allows us to seamlessly transform a random noise into the data that could be used for sample generation as follows: \[x_{n-1}-x_{n}=[f(x,t)-g^{2}(t)s_{\theta}(x,t)]dt+g(t)d\hat{w},\quad x_{T}\sim N (0,1), \tag{1}\] where \(x_{n}\) is the data sample at time \(n\), \(f=0\) and \(g=\sigma^{t}\) are called the drift and diffusion coefficients, respectively. \(s_{\theta}(x,t)\) is called the score function, as is expected to estimate \(\nabla_{x}\log p_{t}(x)\).

The main benefit of using a score-based generative model is that in the backward process (Eq. 1), the score function denoises the input gradually that results in a relatively smooth decline in the Frechet Inception Distance (FID) (17) across training epochs. This enables us to stably produce near anomalous samples based on the FID score of the generated outputs, which could be achieved by stopping the training process earlier than the stage where the model achieves its maximum performance.

Note that a totally different training trajectory is obtained in GANs, where the FID oscillates and premature training does not necessarily produce high-quality anomalous samples.

In addition, Fig. 3 right, shows that the quality and closeness of the samples that are generated by the diffusion model are visually more appealing that the SOTA GAN-based models.

Having generated a high-quality fake dataset, a lightweight projection head is added to the feature extractor, and the whole network is trained to solve a binary classification task between the given normal and generated abnormal inputs. This way, normal class boundaries are tightened and adjusted according to the abnormal inputs.

Note that due to the feature fine-tuning phase of our approach, the borders of the normal samples become well-structured, thus better representing the normal semantic features.

Here, we provide some rationale on the preference of diffusion models over GANs for the the task of near-ND.

We note that in diffusion models, such as DDPM, the model is trained by optimizing the Evidence Lower Bound (ELBO) [(18)]. That is, the objective function is to minimize an upper bound on the negative evidence: \[\mathbb{E}_{x\sim q}[-\log p_{\theta}(x)], \tag{4}\] where \(q\) and \(p_{\theta}\) are the original and estimated distribution of the data, respectively. Here, \(\theta\) stands for the denoiser parameters in the DDPM. But, we note that maximizing the evidence is equivalent with minimizing the Kullback-Leibler (KL) divergence between \(q\) and \(p_{\theta}\):

Therefore, if the gap in the ELBO bound is small, minimizing the DDPM loss is equivalent with making the estimated generative distribution \(p_{\theta}\) close to the original data distribution \(q\). Hence, we expect the estimated generative distribution \(p_{\theta}\) ends up _sufficiently_ close to \(q\) if the training is prematurely stopped, and therefore generates "near-distribution" patterns.

However, such metrics exhibit a steady and consistent increase till the estimated distribution \(p_{\theta}\) gets sufficiently close to the true distribution \(q\) (see Fig. 3 left).

We proposed to take a non-adaptive generative modeling that allows controllable generation of anomalous data. It turns out that SDEs fit this specification much better compared to other SOTA generative models such as GANs. The generated data levels up pre-trained distance-based novelty detection methods not only based on their AUROCs in the standard ND setting but also in the near-ND setting.

Algan: Anomaly detection by generating pseudo anomalous data via latent variables.

For the data generation phase, we use the SDE-based model explained in (43) with exactly the same setup default hyper-parameters e.g. we used predictor-corrector as a sampler.

The density and coverage metrics can be used to assess the diversity and fidelity of generative models, respectively.

Because in our setup, only one class (the normal class) is available for the training of the generative model. As the sample size becomes limited in such a setting, model convergence and generation of high quality real samples become a challenge. Even under such circumstances, the SDE-based models converge. Furthermore, such models can eventually generate high quality real images.

As opposed to other generative methods, the proposed premature training of the SDE yields authentic, near-distribution, diverse, and artifact-free images. This makes it a better choice for the near-ND setup.

Input: Figure 11: Generated anomaly images on the Caltech-UCSD Birds 200 dataset. The images highlighted in green are normal samples.

Input: Figure 12: Generated anomaly images on the MVTecAD dataset. The images highlighted in green are normal samples.

Input: Figure 11: Generated anomaly images on the Caltech-UCSD Birds 200 dataset. The images highlighted in green are normal samples.

Input: Figure 12: Generated anomaly images on the MVTecAD dataset. The images highlighted in green are normal samples.

Input: Figure 1: Spectrum of AUCs, based on Stopping points for WBC dataset and CIFAR10-vs100 setting, as proxies for low- and high-resolution data, respectively.

Input: Figure 1: Sensitivity of state-of-the-art anomaly detectors when differences between normal and anomalous samples are subtle. For each model, its performance in detecting far (Blue) and near (Orange) outliers has been reported. A single class of CIFAR-10 is considered as normal, the rest of the classes are considered far anomalies, and the semantically nearest class of CIFAR-100 to the normal class is the source of near anomalies. This is the class that corresponds to the lowest test AUROC in anomaly detection. A considerable performance drop happens for the most recent methods, which means the learned normal boundaries are not tight enough and need to be adapted. For instance, while some might be able to distinguish a cat image from a car, they are still unable to detect tiger or lion images as anomalies.

Input: Figure 10: Generated anomaly images on the StanfordCars dataset. The images highlighted in green are normal samples.

Input: Figure 4: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 9: Generated anomaly images on 102 category of the Flowers dataset. The images highlighted in green are normal samples.

Input: Figure 2: The performance of novelty detection methods (AUROC %) in the near-ND and ND settings. In the near-ND setting, results are reported according to both criteria, i.e. CLP and bottom-1.

Input: Figure 2: Overview of our framework for near distribution novelty detection. In the first step, the SDE diffusion model generates semantically close outliers, after its training is early-stopped. As it is shown, they have very subtle yet semantic differences with the normal data distribution (downleft). After that, using a linear layer, a pre-trained feature extractor is fine-tuned to solve a binary classification between the normal and abnormal inputs. This modifies the normal boundaries toward including more distinctive features. Finally, normal embeddings are stored and used to compute the k-NN distance at the test time as the novelty score.

Input: Figure 5: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 10: Generated anomaly images on the StanfordCars dataset. The images highlighted in green are normal samples.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on the evidence provided, I've evaluated the claim: "To address the mentioned challenges, we propose to use a "non-adversarial" diffusion-based anomaly data generation method, which can estimate the true normal distribution accurately and smoothly over the training time."

The evidence strongly supports this claim through:
- Visual examples showing high-quality generated anomalies across multiple datasets
- Demonstrated smooth FID score progression during training
- Mathematical foundation explaining how the SDE model estimates distributions
- Reviewer feedback confirming the method's effectiveness
- Comparative results showing advantages over GAN-based approaches

However, the reviewer points out some limitations regarding sensitivity to stopping criteria and computational burden, suggesting the claim slightly overstates the method's robustness.

<score>2</score>

The claim is mostly well-stated but slightly optimistic in terms of accuracy and smoothness claims, given the noted limitations.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

By providing comprehensive experiments and visualizations, we show that a prematurely trained SDE-based model can generate _diverse_, _high quality_, and _non-noisy_ near-outliers, which considerably beat samples that are generated by GANs or obtained from the available datasets for tuning the novelty detection task.

The evidence to be evaluated is:In the first step, a fake dataset of anomalies is generated by a SDE-based diffusion model. We quantitatively and qualitatively show that the generated fake outliers are high-quality, diverse, and yet show semantic differences compared to the normal inputs.

In order to generate near anomalous samples, a basic diffusion model, called Stochastic Differential Equation (SDE) (43) is adopted.

A SDE model considers a continuum of distributions that develop over time in accordance with a diffusion process, to corrupt a sample and turn it into pure noise.

The model is trained to learn how to gradually transform the noisy image back to its original form through the score fucntion (backward process). Running the backward process allows us to seamlessly transform a random noise into the data that could be used for sample generation as follows: \[x_{n-1}-x_{n}=[f(x,t)-g^{2}(t)s_{\theta}(x,t)]dt+g(t)d\hat{w},\quad x_{T}\sim N (0,1), \tag{1}\] where \(x_{n}\) is the data sample at time \(n\), \(f=0\) and \(g=\sigma^{t}\) are called the drift and diffusion coefficients, respectively. \(s_{\theta}(x,t)\) is called the score function, as is expected to estimate \(\nabla_{x}\log p_{t}(x)\). \(\hat{w}\) represents a Brownian motion.

The main benefit of using a score-based generative model is that in the backward process (Eq. 1), the score function denoises the input gradually that results in a relatively smooth decline in the Frechet Inception Distance (FID) (17) across training epochs. This enables us to stably produce near anomalous samples based on the FID score of the generated outputs, which could be achieved by stopping the training process earlier than the stage where the model achieves its maximum performance.

Note that a totally different training trajectory is obtained in GANs, where the FID oscillates and premature training does not necessarily produce high-quality anomalous samples.

In addition, Fig. 3 right, shows that the quality and closeness of the samples that are generated by the diffusion model are visually more appealing that the SOTA GAN-based models.

Hence, we expect the estimated generative distribution \(p_{\theta}\) ends up _sufficiently_ close to \(q\) if the training is prematurely stopped, and therefore generates "near-distribution" patterns.

One may also wonder why GANs do not exhibit the same property. It is also well-known that if one uses an optimal discriminator for a given generator, the generator loss becomes proportional to the Jensen-Shannon divergence of the original and estimated distributions (13). However, this does not happen in GANs, due to practical considerations, as the discriminator and generator are alternatively updated through SGD. Hence the discriminator is not necessarily optimal in each step. Therefore, early stopping does not necessarily lead to near-distribution generators. Our empirical results also confirm that various quality metrics, such as coverage and density(28), which were previously proposed to evaluate synthetic images' fidelity and diversity, where larger values that are close to 1.0 indicate better fidelity and diversity, respectively.

Due to learning more distinguishing features, we pass the SOTA by 12 percentage points in the AUROC score, showing the applicability of our approach in detecting a wide range of anomalies.

To further validate our method, we adapt OpenGAN, as SOTA OOD detection, to the ND setting and report its near-ND results. Surprisingly, even by employing fake abnormal samples generated by a GAN and being exposed to real outliers, it is not able to achieve a decent performance in the near-ND setting (Table 1 (c)), which again supports the benefits of our proposed approach for generating near distribution abnormal samples.

The performance boost is surprisingly large for some backbones such as ResNet-152, ViT-B_16, and R50+ViT-B_16 with roughly 16%, 25%, and 36% boosts in AUROCs in the near-ND setting. Moreover, ViT-B_16 and R50+ViT-B_16 perform roughly 8% and 16% better after the fine-tuning in the ND setup, showing the _generality_ of our approach regardless of the setting or backbone.

Table 3 shows clear superiority of using a SDE-based model compared to the well-known generative models (20; 42; 5; 14) with 2% to 20% better AUROCs based on the chosen dataset. Evidently, employing our fake samples generated by the SDE is almost always beneficial as opposed the ones generated by the other models.

Having artifacts, mode-collapse, and undiversified generated samples could be the reasons behind this observation, which are shown in Appendix10. This highlights that not all the generative models are beneficial for the ND and near-ND tasks, and they need to be carefully selected based on the constraints defined in such domains.

We proposed to take a non-adaptive generative modeling that allows controllable generation of anomalous data. It turns out that SDEs fit this specification much better compared to other SOTA generative models such as GANs. The generated data levels up pre-trained distance-based novelty detection methods not only based on their AUROCs in the standard ND setting but also in the near-ND setting. The improvements that are made by our method are consistent across wide variety of datasets, and choices of the backbone model.

It is important to note that the synthesized images were not an effective anomaly in the first training procedure because they are extremely noisy.

Even under such circumstances, the SDE-based models converge. Furthermore, such models can eventually generate high quality real images. However, other generative models do not converge properly, and the images that are sampled using the early stopped model are often noisy and contain artifacts. As opposed to other generative methods, the proposed premature training of the SDE yields authentic, near-distribution, diverse, and artifact-free images. This makes it a better choice for the near-ND setup.

As can be seen, the images are clearly different semantically from the normal class. Despite this semantic difference, most novelty detection models have a poor performance in the detection of these images as anomalous. \begin{table}\end{table}
Table 1: The performance of novelty detection methods (AUROC %) in near-ND and ND on various datasets. Our method outperforms SOTA by significant margins on all settings. Note that except CSI, all the other SOTA approaches employ a pre-trained architecture, particularly Transformaly (7), that utilizes a ViT-backbone.

Input: Table 1: The performance of novelty detection methods (AUROC %) in near-ND and ND on various datasets. Our method outperforms SOTA by significant margins on all settings. Note that except CSI, all the other SOTA approaches employ a pre-trained architecture, particularly Transformaly (7), that utilizes a ViT-backbone.

Input: Table 3: Comparison of our model performance (in AUROC %) upon using different generative models for each class of the CIFAR-10, Weather, WBC and MvTecAD dataset. The performance of the base backbone is also included to measure the effectiveness of the training process. Diffusionbased data generation considerably passes the GAN-based SOTAs in the novelty detection task. Coverage&Density metrics are calculated according to the dataset and generative model. These metrics measure the diversity and fidelity of the samples that are generated by generative models.

Input: Figure 12: Generated anomaly images on the MVTecAD dataset. The images highlighted in green are normal samples.

Input: Figure 7: Generated anomalous samples by different generative models on the WBC dataset.

Input: Figure 4: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 9: Generated anomaly images on 102 category of the Flowers dataset. The images highlighted in green are normal samples.

Input: Figure 5: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 6: Generated anomalous samples by different generative models on the FGVC-Aircraft dataset.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After careful review of the claim and evidence, I find that it is moderately overstated (score of 3). While the evidence does demonstrate that the SDE-based model can generate high-quality samples that outperform GANs in novelty detection tasks (shown through AUROC scores in Tables 1 and 3, and visual examples in Figures 4-9), the claim's strong assertions about "diverse" and "non-noisy" samples are not fully substantiated. The coverage/density metrics in Table 3 show good but not exceptional diversity scores (around 0.8), and some generated samples in the figures still show artifacts. The evidence supports superior performance compared to GANs but doesn't fully justify the definitiveness of all claimed attributes.

<score>3</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

By providing comprehensive experiments and visualizations, we show that a prematurely trained SDE-based model can generate _diverse_, _high quality_, and _non-noisy_ near-outliers, which considerably beat samples that are generated by GANs or obtained from the available datasets for tuning the novelty detection task.

The review comment to be evaluated is:

The paper investigates the near anomalies detection and points a significant decrease in performance of state of the art approaches on near anomalies cases. A new solution is proposed: 1) using SDE models trained to generate data from the training distribution. 2) generate a dataset from the trained SDE. 3) train a binary classifier on top of a feature extractor to distinguish between the generated data and real data. 4) use nearest neighbour for detecting anomalies.
Extensive experiments and empirical analysis are carried on. The method shows good results.

The evidence to be evaluated is:In the first step, a fake dataset of anomalies is generated by a SDE-based diffusion model. We quantitatively and qualitatively show that the generated fake outliers are high-quality, diverse, and yet show semantic differences compared to the normal inputs.

In order to generate near anomalous samples, a basic diffusion model, called Stochastic Differential Equation (SDE) (43) is adopted.

A SDE model considers a continuum of distributions that develop over time in accordance with a diffusion process, to corrupt a sample and turn it into pure noise.

The model is trained to learn how to gradually transform the noisy image back to its original form through the score fucntion (backward process). Running the backward process allows us to seamlessly transform a random noise into the data that could be used for sample generation as follows: \[x_{n-1}-x_{n}=[f(x,t)-g^{2}(t)s_{\theta}(x,t)]dt+g(t)d\hat{w},\quad x_{T}\sim N (0,1), \tag{1}\] where \(x_{n}\) is the data sample at time \(n\), \(f=0\) and \(g=\sigma^{t}\) are called the drift and diffusion coefficients, respectively. \(s_{\theta}(x,t)\) is called the score function, as is expected to estimate \(\nabla_{x}\log p_{t}(x)\). \(\hat{w}\) represents a Brownian motion.

The main benefit of using a score-based generative model is that in the backward process (Eq. 1), the score function denoises the input gradually that results in a relatively smooth decline in the Frechet Inception Distance (FID) (17) across training epochs. This enables us to stably produce near anomalous samples based on the FID score of the generated outputs, which could be achieved by stopping the training process earlier than the stage where the model achieves its maximum performance.

Note that a totally different training trajectory is obtained in GANs, where the FID oscillates and premature training does not necessarily produce high-quality anomalous samples.

In addition, Fig. 3 right, shows that the quality and closeness of the samples that are generated by the diffusion model are visually more appealing that the SOTA GAN-based models.

Hence, we expect the estimated generative distribution \(p_{\theta}\) ends up _sufficiently_ close to \(q\) if the training is prematurely stopped, and therefore generates "near-distribution" patterns.

One may also wonder why GANs do not exhibit the same property. It is also well-known that if one uses an optimal discriminator for a given generator, the generator loss becomes proportional to the Jensen-Shannon divergence of the original and estimated distributions (13). However, this does not happen in GANs, due to practical considerations, as the discriminator and generator are alternatively updated through SGD. Hence the discriminator is not necessarily optimal in each step. Therefore, early stopping does not necessarily lead to near-distribution generators. Our empirical results also confirm that various quality metrics, such as coverage and density(28), which were previously proposed to evaluate synthetic images' fidelity and diversity, where larger values that are close to 1.0 indicate better fidelity and diversity, respectively.

Due to learning more distinguishing features, we pass the SOTA by 12 percentage points in the AUROC score, showing the applicability of our approach in detecting a wide range of anomalies.

To further validate our method, we adapt OpenGAN, as SOTA OOD detection, to the ND setting and report its near-ND results. Surprisingly, even by employing fake abnormal samples generated by a GAN and being exposed to real outliers, it is not able to achieve a decent performance in the near-ND setting (Table 1 (c)), which again supports the benefits of our proposed approach for generating near distribution abnormal samples.

The performance boost is surprisingly large for some backbones such as ResNet-152, ViT-B_16, and R50+ViT-B_16 with roughly 16%, 25%, and 36% boosts in AUROCs in the near-ND setting. Moreover, ViT-B_16 and R50+ViT-B_16 perform roughly 8% and 16% better after the fine-tuning in the ND setup, showing the _generality_ of our approach regardless of the setting or backbone.

Table 3 shows clear superiority of using a SDE-based model compared to the well-known generative models (20; 42; 5; 14) with 2% to 20% better AUROCs based on the chosen dataset. Evidently, employing our fake samples generated by the SDE is almost always beneficial as opposed the ones generated by the other models.

Having artifacts, mode-collapse, and undiversified generated samples could be the reasons behind this observation, which are shown in Appendix10. This highlights that not all the generative models are beneficial for the ND and near-ND tasks, and they need to be carefully selected based on the constraints defined in such domains.

We proposed to take a non-adaptive generative modeling that allows controllable generation of anomalous data. It turns out that SDEs fit this specification much better compared to other SOTA generative models such as GANs. The generated data levels up pre-trained distance-based novelty detection methods not only based on their AUROCs in the standard ND setting but also in the near-ND setting. The improvements that are made by our method are consistent across wide variety of datasets, and choices of the backbone model.

It is important to note that the synthesized images were not an effective anomaly in the first training procedure because they are extremely noisy.

Even under such circumstances, the SDE-based models converge. Furthermore, such models can eventually generate high quality real images. However, other generative models do not converge properly, and the images that are sampled using the early stopped model are often noisy and contain artifacts. As opposed to other generative methods, the proposed premature training of the SDE yields authentic, near-distribution, diverse, and artifact-free images. This makes it a better choice for the near-ND setup.

As can be seen, the images are clearly different semantically from the normal class. Despite this semantic difference, most novelty detection models have a poor performance in the detection of these images as anomalous. \begin{table}\end{table}
Table 1: The performance of novelty detection methods (AUROC %) in near-ND and ND on various datasets. Our method outperforms SOTA by significant margins on all settings. Note that except CSI, all the other SOTA approaches employ a pre-trained architecture, particularly Transformaly (7), that utilizes a ViT-backbone.

Input: Table 1: The performance of novelty detection methods (AUROC %) in near-ND and ND on various datasets. Our method outperforms SOTA by significant margins on all settings. Note that except CSI, all the other SOTA approaches employ a pre-trained architecture, particularly Transformaly (7), that utilizes a ViT-backbone.

Input: Table 3: Comparison of our model performance (in AUROC %) upon using different generative models for each class of the CIFAR-10, Weather, WBC and MvTecAD dataset. The performance of the base backbone is also included to measure the effectiveness of the training process. Diffusionbased data generation considerably passes the GAN-based SOTAs in the novelty detection task. Coverage&Density metrics are calculated according to the dataset and generative model. These metrics measure the diversity and fidelity of the samples that are generated by generative models.

Input: Figure 12: Generated anomaly images on the MVTecAD dataset. The images highlighted in green are normal samples.

Input: Figure 7: Generated anomalous samples by different generative models on the WBC dataset.

Input: Figure 4: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 9: Generated anomaly images on 102 category of the Flowers dataset. The images highlighted in green are normal samples.

Input: Figure 5: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 6: Generated anomalous samples by different generative models on the FGVC-Aircraft dataset.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, the claim appears somewhat overstated (score 3). While the evidence demonstrates that SDE-based models can generate high-quality samples and achieve superior performance compared to GANs and other methods (shown in Tables 1 and 3), some aspects of the claim are overstated:

1. The term "comprehensive" experiments is vague - while there are multiple experiments, they focus mainly on specific datasets and metrics.
2. The claim of "considerably beat" GANs needs qualification - performance improvements vary significantly across datasets (2-20% range per Table 3).
3. The evidence supports that SDE generates "high quality" and "non-noisy" samples but the claim of "diverse" is less thoroughly demonstrated beyond basic coverage metrics.

<score>3</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

By providing comprehensive experiments and visualizations, we show that a prematurely trained SDE-based model can generate _diverse_, _high quality_, and _non-noisy_ near-outliers, which considerably beat samples that are generated by GANs or obtained from the available datasets for tuning the novelty detection task.

The review comment to be evaluated is:

* There is a large room for improvement in clarity.
    * The paper should be clearer on how exactly the generative model is trained and used. The most important fact, in my opinion, is that the score-based generative model is trained on in-distribution data and terminated prematurely. However, the fact is only briefly mentioned in Page 2, and then mentioned in Figure 2. Given the importance of this information, this point should be described in detail in Section 2.1. Also, other details and design choices, for example, how early the training of the score-based model should be stopped, needs to be elaborated.
    * The experimental setup described in Section 3.2 is difficult to follow. There are several comparison settings but are not well organized.
* Discussion on the differences among diffusion models, GANs, and VAEs in Section 2.2 are not very persuasive. It does not have concrete examples or quantitative arguments. Plus, the argument of Section 3.2 is mostly about generative modeling itself and not about its application to generating auxiliary data in novelty detection.
* When performing CIFAR-10 vs CIFAR-100 experiment, it is more standard and more challenging to incorporate all the classes, instead of using only one class from each dataset. Overall, I am not convinced that the experiment settings used in the paper are standard.
* Replacing a GAN with a diffusion model in the data generator for novelty detection itself is not a very novel contribution to the community.

The evidence to be evaluated is:In the first step, a fake dataset of anomalies is generated by a SDE-based diffusion model. We quantitatively and qualitatively show that the generated fake outliers are high-quality, diverse, and yet show semantic differences compared to the normal inputs.

In order to generate near anomalous samples, a basic diffusion model, called Stochastic Differential Equation (SDE) (43) is adopted.

A SDE model considers a continuum of distributions that develop over time in accordance with a diffusion process, to corrupt a sample and turn it into pure noise.

The model is trained to learn how to gradually transform the noisy image back to its original form through the score fucntion (backward process). Running the backward process allows us to seamlessly transform a random noise into the data that could be used for sample generation as follows: \[x_{n-1}-x_{n}=[f(x,t)-g^{2}(t)s_{\theta}(x,t)]dt+g(t)d\hat{w},\quad x_{T}\sim N (0,1), \tag{1}\] where \(x_{n}\) is the data sample at time \(n\), \(f=0\) and \(g=\sigma^{t}\) are called the drift and diffusion coefficients, respectively. \(s_{\theta}(x,t)\) is called the score function, as is expected to estimate \(\nabla_{x}\log p_{t}(x)\). \(\hat{w}\) represents a Brownian motion.

The main benefit of using a score-based generative model is that in the backward process (Eq. 1), the score function denoises the input gradually that results in a relatively smooth decline in the Frechet Inception Distance (FID) (17) across training epochs. This enables us to stably produce near anomalous samples based on the FID score of the generated outputs, which could be achieved by stopping the training process earlier than the stage where the model achieves its maximum performance.

Note that a totally different training trajectory is obtained in GANs, where the FID oscillates and premature training does not necessarily produce high-quality anomalous samples.

In addition, Fig. 3 right, shows that the quality and closeness of the samples that are generated by the diffusion model are visually more appealing that the SOTA GAN-based models.

Hence, we expect the estimated generative distribution \(p_{\theta}\) ends up _sufficiently_ close to \(q\) if the training is prematurely stopped, and therefore generates "near-distribution" patterns.

One may also wonder why GANs do not exhibit the same property. It is also well-known that if one uses an optimal discriminator for a given generator, the generator loss becomes proportional to the Jensen-Shannon divergence of the original and estimated distributions (13). However, this does not happen in GANs, due to practical considerations, as the discriminator and generator are alternatively updated through SGD. Hence the discriminator is not necessarily optimal in each step. Therefore, early stopping does not necessarily lead to near-distribution generators. Our empirical results also confirm that various quality metrics, such as coverage and density(28), which were previously proposed to evaluate synthetic images' fidelity and diversity, where larger values that are close to 1.0 indicate better fidelity and diversity, respectively.

Due to learning more distinguishing features, we pass the SOTA by 12 percentage points in the AUROC score, showing the applicability of our approach in detecting a wide range of anomalies.

To further validate our method, we adapt OpenGAN, as SOTA OOD detection, to the ND setting and report its near-ND results. Surprisingly, even by employing fake abnormal samples generated by a GAN and being exposed to real outliers, it is not able to achieve a decent performance in the near-ND setting (Table 1 (c)), which again supports the benefits of our proposed approach for generating near distribution abnormal samples.

The performance boost is surprisingly large for some backbones such as ResNet-152, ViT-B_16, and R50+ViT-B_16 with roughly 16%, 25%, and 36% boosts in AUROCs in the near-ND setting. Moreover, ViT-B_16 and R50+ViT-B_16 perform roughly 8% and 16% better after the fine-tuning in the ND setup, showing the _generality_ of our approach regardless of the setting or backbone.

Table 3 shows clear superiority of using a SDE-based model compared to the well-known generative models (20; 42; 5; 14) with 2% to 20% better AUROCs based on the chosen dataset. Evidently, employing our fake samples generated by the SDE is almost always beneficial as opposed the ones generated by the other models.

Having artifacts, mode-collapse, and undiversified generated samples could be the reasons behind this observation, which are shown in Appendix10. This highlights that not all the generative models are beneficial for the ND and near-ND tasks, and they need to be carefully selected based on the constraints defined in such domains.

We proposed to take a non-adaptive generative modeling that allows controllable generation of anomalous data. It turns out that SDEs fit this specification much better compared to other SOTA generative models such as GANs. The generated data levels up pre-trained distance-based novelty detection methods not only based on their AUROCs in the standard ND setting but also in the near-ND setting. The improvements that are made by our method are consistent across wide variety of datasets, and choices of the backbone model.

It is important to note that the synthesized images were not an effective anomaly in the first training procedure because they are extremely noisy.

Even under such circumstances, the SDE-based models converge. Furthermore, such models can eventually generate high quality real images. However, other generative models do not converge properly, and the images that are sampled using the early stopped model are often noisy and contain artifacts. As opposed to other generative methods, the proposed premature training of the SDE yields authentic, near-distribution, diverse, and artifact-free images. This makes it a better choice for the near-ND setup.

As can be seen, the images are clearly different semantically from the normal class. Despite this semantic difference, most novelty detection models have a poor performance in the detection of these images as anomalous. \begin{table}\end{table}
Table 1: The performance of novelty detection methods (AUROC %) in near-ND and ND on various datasets. Our method outperforms SOTA by significant margins on all settings. Note that except CSI, all the other SOTA approaches employ a pre-trained architecture, particularly Transformaly (7), that utilizes a ViT-backbone.

Input: Table 1: The performance of novelty detection methods (AUROC %) in near-ND and ND on various datasets. Our method outperforms SOTA by significant margins on all settings. Note that except CSI, all the other SOTA approaches employ a pre-trained architecture, particularly Transformaly (7), that utilizes a ViT-backbone.

Input: Table 3: Comparison of our model performance (in AUROC %) upon using different generative models for each class of the CIFAR-10, Weather, WBC and MvTecAD dataset. The performance of the base backbone is also included to measure the effectiveness of the training process. Diffusionbased data generation considerably passes the GAN-based SOTAs in the novelty detection task. Coverage&Density metrics are calculated according to the dataset and generative model. These metrics measure the diversity and fidelity of the samples that are generated by generative models.

Input: Figure 12: Generated anomaly images on the MVTecAD dataset. The images highlighted in green are normal samples.

Input: Figure 7: Generated anomalous samples by different generative models on the WBC dataset.

Input: Figure 4: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 9: Generated anomaly images on 102 category of the Flowers dataset. The images highlighted in green are normal samples.

Input: Figure 5: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 6: Generated anomalous samples by different generative models on the FGVC-Aircraft dataset.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my review of the claim and evidence:

The claim about generating "diverse, high quality, and non-noisy near-outliers" is moderately overstated. While the evidence shows the SDE model outperforms GANs on various metrics (FID scores, AUROC improvements of 2-20%, better Coverage & Density scores), and visual examples demonstrate reasonable quality outputs, several key limitations exist:

1. The review notes lack of clarity around training details and stopping criteria
2. Limited experimental validation across standard settings
3. Visual examples show varying quality - some images have artifacts/distortions
4. Results focus heavily on comparative performance vs. absolute quality measures

The evidence supports that the method works better than GANs, but doesn't fully substantiate the strong claims about consistent "high quality" and "non-noisy" outputs.

<score>3</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

By providing comprehensive experiments and visualizations, we show that a prematurely trained SDE-based model can generate _diverse_, _high quality_, and _non-noisy_ near-outliers, which considerably beat samples that are generated by GANs or obtained from the available datasets for tuning the novelty detection task.

The review comment to be evaluated is:

The idea of this paper is very natural: try to generate near-distribution outliers to augment the original training dataset which only contains normal data (semi-supervised learning), such that we can fine-tune the pre-trained feature extractor by training a binary classifier.

Strengths:
(1) Using generated outliers to improve the accuracy of novelty detection is not new. There have been some similar works which use GANs instead of SDE.  The main contribution of this paper is that they replace the GANs with the diffusion model (which is increasingly popular recently).  The motivation here is that the outliers generated by SDE are “closer“ to the nominal distribution, and then using these "boarder" points  to deal with the near-ND setting.  Even though the pipeline and SDE are all from existing works, the novel combination is meaningful considering the theoretically and experimental results.

(2) The experimental results show that the proposed method considerably improves over existing models (especially for the near-ND setting).  It is not surprising that SDE over-perform GANs for the near-ND setting, but I am surprised that the improvement is so obvious.  It will be great if the code can be provided.

Weakness:
(1) In Section 2.2 (Diffusion models vs. GANs), they try to explain, for near-ND, why choosing diffusion models instead of other generative modes (GAN and VAE), it would be better to include flow-based models in this part. Also,  VAE-related results are not included in figure3.

(2) In Figure3,  density, coverage and FID are used as the metrics to measure the fidelity and diversity.  Could you add a brief introductions/definitions for these metrics? Also, I am confused how to get the "density" from GAN-based models as they are not tractable models.

(3)About Stopping point: In section 4, "the method is robust against the stopping point with at most 2% variation for the reasonable FID scores". However,  from table4,  it is difficult to get this conclusion. what is "reasonable" FID? Actually, the stopping point decide how similar the fake points and the real points. How to decide when to stop ?

Other minor comments:
The "normal distribution" used in this paper sometimes are confusing, which means the distribution of nominal data instead of Gaussian distribution. It would be better to use other notations.

The evidence to be evaluated is:In the first step, a fake dataset of anomalies is generated by a SDE-based diffusion model. We quantitatively and qualitatively show that the generated fake outliers are high-quality, diverse, and yet show semantic differences compared to the normal inputs.

In order to generate near anomalous samples, a basic diffusion model, called Stochastic Differential Equation (SDE) (43) is adopted.

A SDE model considers a continuum of distributions that develop over time in accordance with a diffusion process, to corrupt a sample and turn it into pure noise.

The model is trained to learn how to gradually transform the noisy image back to its original form through the score fucntion (backward process). Running the backward process allows us to seamlessly transform a random noise into the data that could be used for sample generation as follows: \[x_{n-1}-x_{n}=[f(x,t)-g^{2}(t)s_{\theta}(x,t)]dt+g(t)d\hat{w},\quad x_{T}\sim N (0,1), \tag{1}\] where \(x_{n}\) is the data sample at time \(n\), \(f=0\) and \(g=\sigma^{t}\) are called the drift and diffusion coefficients, respectively. \(s_{\theta}(x,t)\) is called the score function, as is expected to estimate \(\nabla_{x}\log p_{t}(x)\). \(\hat{w}\) represents a Brownian motion.

The main benefit of using a score-based generative model is that in the backward process (Eq. 1), the score function denoises the input gradually that results in a relatively smooth decline in the Frechet Inception Distance (FID) (17) across training epochs. This enables us to stably produce near anomalous samples based on the FID score of the generated outputs, which could be achieved by stopping the training process earlier than the stage where the model achieves its maximum performance.

Note that a totally different training trajectory is obtained in GANs, where the FID oscillates and premature training does not necessarily produce high-quality anomalous samples.

In addition, Fig. 3 right, shows that the quality and closeness of the samples that are generated by the diffusion model are visually more appealing that the SOTA GAN-based models.

Hence, we expect the estimated generative distribution \(p_{\theta}\) ends up _sufficiently_ close to \(q\) if the training is prematurely stopped, and therefore generates "near-distribution" patterns.

One may also wonder why GANs do not exhibit the same property. It is also well-known that if one uses an optimal discriminator for a given generator, the generator loss becomes proportional to the Jensen-Shannon divergence of the original and estimated distributions (13). However, this does not happen in GANs, due to practical considerations, as the discriminator and generator are alternatively updated through SGD. Hence the discriminator is not necessarily optimal in each step. Therefore, early stopping does not necessarily lead to near-distribution generators. Our empirical results also confirm that various quality metrics, such as coverage and density(28), which were previously proposed to evaluate synthetic images' fidelity and diversity, where larger values that are close to 1.0 indicate better fidelity and diversity, respectively.

Due to learning more distinguishing features, we pass the SOTA by 12 percentage points in the AUROC score, showing the applicability of our approach in detecting a wide range of anomalies.

To further validate our method, we adapt OpenGAN, as SOTA OOD detection, to the ND setting and report its near-ND results. Surprisingly, even by employing fake abnormal samples generated by a GAN and being exposed to real outliers, it is not able to achieve a decent performance in the near-ND setting (Table 1 (c)), which again supports the benefits of our proposed approach for generating near distribution abnormal samples.

The performance boost is surprisingly large for some backbones such as ResNet-152, ViT-B_16, and R50+ViT-B_16 with roughly 16%, 25%, and 36% boosts in AUROCs in the near-ND setting. Moreover, ViT-B_16 and R50+ViT-B_16 perform roughly 8% and 16% better after the fine-tuning in the ND setup, showing the _generality_ of our approach regardless of the setting or backbone.

Table 3 shows clear superiority of using a SDE-based model compared to the well-known generative models (20; 42; 5; 14) with 2% to 20% better AUROCs based on the chosen dataset. Evidently, employing our fake samples generated by the SDE is almost always beneficial as opposed the ones generated by the other models.

Having artifacts, mode-collapse, and undiversified generated samples could be the reasons behind this observation, which are shown in Appendix10. This highlights that not all the generative models are beneficial for the ND and near-ND tasks, and they need to be carefully selected based on the constraints defined in such domains.

We proposed to take a non-adaptive generative modeling that allows controllable generation of anomalous data. It turns out that SDEs fit this specification much better compared to other SOTA generative models such as GANs. The generated data levels up pre-trained distance-based novelty detection methods not only based on their AUROCs in the standard ND setting but also in the near-ND setting. The improvements that are made by our method are consistent across wide variety of datasets, and choices of the backbone model.

It is important to note that the synthesized images were not an effective anomaly in the first training procedure because they are extremely noisy.

Even under such circumstances, the SDE-based models converge. Furthermore, such models can eventually generate high quality real images. However, other generative models do not converge properly, and the images that are sampled using the early stopped model are often noisy and contain artifacts. As opposed to other generative methods, the proposed premature training of the SDE yields authentic, near-distribution, diverse, and artifact-free images. This makes it a better choice for the near-ND setup.

As can be seen, the images are clearly different semantically from the normal class. Despite this semantic difference, most novelty detection models have a poor performance in the detection of these images as anomalous. \begin{table}\end{table}
Table 1: The performance of novelty detection methods (AUROC %) in near-ND and ND on various datasets. Our method outperforms SOTA by significant margins on all settings. Note that except CSI, all the other SOTA approaches employ a pre-trained architecture, particularly Transformaly (7), that utilizes a ViT-backbone.

Input: Table 1: The performance of novelty detection methods (AUROC %) in near-ND and ND on various datasets. Our method outperforms SOTA by significant margins on all settings. Note that except CSI, all the other SOTA approaches employ a pre-trained architecture, particularly Transformaly (7), that utilizes a ViT-backbone.

Input: Table 3: Comparison of our model performance (in AUROC %) upon using different generative models for each class of the CIFAR-10, Weather, WBC and MvTecAD dataset. The performance of the base backbone is also included to measure the effectiveness of the training process. Diffusionbased data generation considerably passes the GAN-based SOTAs in the novelty detection task. Coverage&Density metrics are calculated according to the dataset and generative model. These metrics measure the diversity and fidelity of the samples that are generated by generative models.

Input: Figure 12: Generated anomaly images on the MVTecAD dataset. The images highlighted in green are normal samples.

Input: Figure 7: Generated anomalous samples by different generative models on the WBC dataset.

Input: Figure 4: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 9: Generated anomaly images on 102 category of the Flowers dataset. The images highlighted in green are normal samples.

Input: Figure 5: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 6: Generated anomalous samples by different generative models on the FGVC-Aircraft dataset.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the evidence, I find that the claim is relatively well-stated but has some minor overstatements. The evidence shows that SDE-based models do generate high-quality and diverse samples, supported by quantitative metrics (density/coverage scores) and qualitative visual examples. The performance improvements are substantial across datasets and architectures. However, the claim of "considerably beat" is somewhat overstated, as while SDE outperforms GANs, the margin varies across datasets (2-20%). The term "non-noisy" is also well-supported by the visual examples showing clean generated images.

<score>2</score>

Justification:
1. The claim about "diverse" is supported by density/coverage metrics in Table 3 showing strong diversity scores for SDE
2. "High quality" is validated by FID scores and visual examples in Figures 4-12 showing clear, artifact-free images
3. "Non-noisy" is demonstrated in multiple figures showing clean generated samples
4. "Considerably beat" is partially supported by performance gains over GANs, but the magnitude varies (2-20% improvement)
5. The review comment supports the technical validity while noting some limitations around stopping point selection
6. Evidence comprehensively validates most aspects of the claim with both quantitative and qualitative results

The claim only slightly overstates the consistently superior performance over GANs but is otherwise well-grounded in the evidence provided.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

By providing comprehensive experiments and visualizations, we show that a prematurely trained SDE-based model can generate _diverse_, _high quality_, and _non-noisy_ near-outliers, which considerably beat samples that are generated by GANs or obtained from the available datasets for tuning the novelty detection task.

The review comment to be evaluated is:

### Strengths

- The paper is, for the most part, well-written, easy-to-follow, and does a good job of
contextualising and motivating the work.
- The proposed method is simple and intuitive, and seems to perform well empirically
compared with the baselines in both the regular ND setting and the near ND setting, showing a
significant reduction in the performance gap of going from the former to the latter.
the NR to near-NR setting.
- The figures are well-chosen and well-put-together: they convey well the essential points of the
  paper, such as the shortcomings of existing methods and the feasibility of using GANs vs diffusion
  models for creating the synthetic data.
- Evaluation performed using a good range and selection of baseline methods and benchmark datasets.
- Results are aggregated over a good number (10) of replicates.

### Weaknesses
- Despite the authors claiming otherwise, the method is quite sensitive to the stopping criterion in
  the near ND setting with an empirical range of [68.2, 90.0]; in practice, different datasets may
  require wildly different FID thresholds and there is no reliable way of validating this given the
  nature of the problem. Also, I wonder if the number of sampling steps and the choice of sampler have a
  significant effect on the results -- there doesn't seem to be any mention of this.
  Taking this a step further, might it be feasible to train the diffusion model fully and then rely
  upon early-stopping the reverse diffusion process according to the FID score?
- The need to train, and indeed sample, from a diffusion model can be computationally burdensome for
  higher-resolution datasets.
- The description of the SDE model is a little overdone; a much briefer description with a pointing of the
  reader to the original paper would be sufficient. This is especially the case given that, as
  I understand it, the proposed method should work with any diffusion based model (Imagen, Parti,
  DALLE-2, etc.) or are there theoretically some constraints on the form said model and the
  associated sampler should take?
- While this applies to several methods of the same class, the method seems to make certain
assumptions about the distribution of the ID and OOD data that might not hold in the wild.
- While results are aggregated over a good number (10) of replicates no measure of dispersion
  (standard deviation/error) seems to be reported alongside the mean in each case.
- Some typographic and syntactical errors (e.g. 'that unlikely come' (line 1 of the introduction)).

The evidence to be evaluated is:In the first step, a fake dataset of anomalies is generated by a SDE-based diffusion model. We quantitatively and qualitatively show that the generated fake outliers are high-quality, diverse, and yet show semantic differences compared to the normal inputs.

In order to generate near anomalous samples, a basic diffusion model, called Stochastic Differential Equation (SDE) (43) is adopted.

A SDE model considers a continuum of distributions that develop over time in accordance with a diffusion process, to corrupt a sample and turn it into pure noise.

The model is trained to learn how to gradually transform the noisy image back to its original form through the score fucntion (backward process). Running the backward process allows us to seamlessly transform a random noise into the data that could be used for sample generation as follows: \[x_{n-1}-x_{n}=[f(x,t)-g^{2}(t)s_{\theta}(x,t)]dt+g(t)d\hat{w},\quad x_{T}\sim N (0,1), \tag{1}\] where \(x_{n}\) is the data sample at time \(n\), \(f=0\) and \(g=\sigma^{t}\) are called the drift and diffusion coefficients, respectively. \(s_{\theta}(x,t)\) is called the score function, as is expected to estimate \(\nabla_{x}\log p_{t}(x)\). \(\hat{w}\) represents a Brownian motion.

The main benefit of using a score-based generative model is that in the backward process (Eq. 1), the score function denoises the input gradually that results in a relatively smooth decline in the Frechet Inception Distance (FID) (17) across training epochs. This enables us to stably produce near anomalous samples based on the FID score of the generated outputs, which could be achieved by stopping the training process earlier than the stage where the model achieves its maximum performance.

Note that a totally different training trajectory is obtained in GANs, where the FID oscillates and premature training does not necessarily produce high-quality anomalous samples.

In addition, Fig. 3 right, shows that the quality and closeness of the samples that are generated by the diffusion model are visually more appealing that the SOTA GAN-based models.

Hence, we expect the estimated generative distribution \(p_{\theta}\) ends up _sufficiently_ close to \(q\) if the training is prematurely stopped, and therefore generates "near-distribution" patterns.

One may also wonder why GANs do not exhibit the same property. It is also well-known that if one uses an optimal discriminator for a given generator, the generator loss becomes proportional to the Jensen-Shannon divergence of the original and estimated distributions (13). However, this does not happen in GANs, due to practical considerations, as the discriminator and generator are alternatively updated through SGD. Hence the discriminator is not necessarily optimal in each step. Therefore, early stopping does not necessarily lead to near-distribution generators. Our empirical results also confirm that various quality metrics, such as coverage and density(28), which were previously proposed to evaluate synthetic images' fidelity and diversity, where larger values that are close to 1.0 indicate better fidelity and diversity, respectively.

Due to learning more distinguishing features, we pass the SOTA by 12 percentage points in the AUROC score, showing the applicability of our approach in detecting a wide range of anomalies.

To further validate our method, we adapt OpenGAN, as SOTA OOD detection, to the ND setting and report its near-ND results. Surprisingly, even by employing fake abnormal samples generated by a GAN and being exposed to real outliers, it is not able to achieve a decent performance in the near-ND setting (Table 1 (c)), which again supports the benefits of our proposed approach for generating near distribution abnormal samples.

The performance boost is surprisingly large for some backbones such as ResNet-152, ViT-B_16, and R50+ViT-B_16 with roughly 16%, 25%, and 36% boosts in AUROCs in the near-ND setting. Moreover, ViT-B_16 and R50+ViT-B_16 perform roughly 8% and 16% better after the fine-tuning in the ND setup, showing the _generality_ of our approach regardless of the setting or backbone.

Table 3 shows clear superiority of using a SDE-based model compared to the well-known generative models (20; 42; 5; 14) with 2% to 20% better AUROCs based on the chosen dataset. Evidently, employing our fake samples generated by the SDE is almost always beneficial as opposed the ones generated by the other models.

Having artifacts, mode-collapse, and undiversified generated samples could be the reasons behind this observation, which are shown in Appendix10. This highlights that not all the generative models are beneficial for the ND and near-ND tasks, and they need to be carefully selected based on the constraints defined in such domains.

We proposed to take a non-adaptive generative modeling that allows controllable generation of anomalous data. It turns out that SDEs fit this specification much better compared to other SOTA generative models such as GANs. The generated data levels up pre-trained distance-based novelty detection methods not only based on their AUROCs in the standard ND setting but also in the near-ND setting. The improvements that are made by our method are consistent across wide variety of datasets, and choices of the backbone model.

It is important to note that the synthesized images were not an effective anomaly in the first training procedure because they are extremely noisy.

Even under such circumstances, the SDE-based models converge. Furthermore, such models can eventually generate high quality real images. However, other generative models do not converge properly, and the images that are sampled using the early stopped model are often noisy and contain artifacts. As opposed to other generative methods, the proposed premature training of the SDE yields authentic, near-distribution, diverse, and artifact-free images. This makes it a better choice for the near-ND setup.

As can be seen, the images are clearly different semantically from the normal class. Despite this semantic difference, most novelty detection models have a poor performance in the detection of these images as anomalous. \begin{table}\end{table}
Table 1: The performance of novelty detection methods (AUROC %) in near-ND and ND on various datasets. Our method outperforms SOTA by significant margins on all settings. Note that except CSI, all the other SOTA approaches employ a pre-trained architecture, particularly Transformaly (7), that utilizes a ViT-backbone.

Input: Table 1: The performance of novelty detection methods (AUROC %) in near-ND and ND on various datasets. Our method outperforms SOTA by significant margins on all settings. Note that except CSI, all the other SOTA approaches employ a pre-trained architecture, particularly Transformaly (7), that utilizes a ViT-backbone.

Input: Table 3: Comparison of our model performance (in AUROC %) upon using different generative models for each class of the CIFAR-10, Weather, WBC and MvTecAD dataset. The performance of the base backbone is also included to measure the effectiveness of the training process. Diffusionbased data generation considerably passes the GAN-based SOTAs in the novelty detection task. Coverage&Density metrics are calculated according to the dataset and generative model. These metrics measure the diversity and fidelity of the samples that are generated by generative models.

Input: Figure 12: Generated anomaly images on the MVTecAD dataset. The images highlighted in green are normal samples.

Input: Figure 7: Generated anomalous samples by different generative models on the WBC dataset.

Input: Figure 4: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 9: Generated anomaly images on 102 category of the Flowers dataset. The images highlighted in green are normal samples.

Input: Figure 5: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 6: Generated anomalous samples by different generative models on the FGVC-Aircraft dataset.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll evaluate the claim about SDE-based model's ability to generate diverse, high-quality, and non-noisy near-outliers:

The evidence largely supports this claim through:
- Quantitative results showing superior AUROC scores compared to GAN-based methods (Tables 1 & 3)
- Visual examples demonstrating high-quality generated samples (Figures 4-9, 12)
- Higher Coverage & Density metrics indicating better fidelity/diversity (Table 3)
- Reviewer comments confirming the method's empirical performance

However, the claim is slightly overstated because:
- The reviewer notes sensitivity to stopping criteria and FID thresholds
- Performance varies across datasets
- Computational burden for higher-resolution datasets isn't acknowledged
- Some limitations around distribution assumptions aren't addressed

While the core claim is supported by evidence, the absolute terms like "considerably beat" and lack of qualification make it moderately overstated.

<score>3</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

The importance of artifact- and noise-free anomalous samples in fine tuning is due to the fact that deep models tend to learn such artifacts as shortcuts, preventing them from generalization to the true essence of the anomalous samples.

The evidence to be evaluated is:In the first step, a fake dataset of anomalies is generated by a SDE-based diffusion model. We quantitatively and qualitatively show that the generated fake outliers are high-quality, diverse, and yet show semantic differences compared to the normal inputs.

The main benefit of using a score-based generative model is that in the backward process (Eq. 1), the score function denoises the input gradually that results in a relatively smooth decline in the Frechet Inception Distance (FID) (17) across training epochs. This enables us to stably produce near anomalous samples based on the FID score of the generated outputs, which could be achieved by stopping the training process earlier than the stage where the model achieves its maximum performance.

Having generated a high-quality fake dataset, a lightweight projection head is added to the feature extractor, and the whole network is trained to solve a binary classification task between the given normal and generated abnormal inputs. This way, normal class boundaries are tightened and adjusted according to the abnormal inputs.

Note that due to the feature fine-tuning phase of our approach, the borders of the normal samples become well-structured, thus better representing the normal semantic features.

The anomalous data that are generated by StyleGAN-ADA contain lots of artifact-full and distorted samples that a practical novelty detector is expected to detect.

Interestingly, other models generated samples are harmful to the performance in Weather and MvTecAD datasets. Having artifacts, mode-collapse, and undiversified generated samples could be the reasons behind this observation, which are shown in Appendix10.

It has recently been shown that contrastive learning can also improve anomaly detection (44), which encourages the model to learn normal features by contrasting positive and negative samples.

Intuitively, abnormal and normal data do not overlap in the feature space because many features model high-level and semantic image properties. So, in the pre-trained feature space, one may classify normal vs. anomaly.

In this paper, we revealed a key weakness of existing SOTA novelty detection methods, where they fail to identify anomalies that are slightly different from the normal data. We proposed to take a non-adaptive generative modeling that allows controllable generation of anomalous data.

It is important to note that the synthesized images were not an effective anomaly in the first training procedure because they are extremely noisy. Furthermore, one has to note that current deep models are highly capable of learning artifacts from their input, instead of paying attention to the general semantics of the input.

However, other generative models do not converge properly, and the images that are sampled using the early stopped model are often noisy and contain artifacts. As opposed to other generative methods, the proposed premature training of the SDE yields authentic, near-distribution, diverse, and artifact-free images.

Input: Figure 7: Generated anomalous samples by different generative models on the WBC dataset.

Input: Figure 4: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 6: Generated anomalous samples by different generative models on the FGVC-Aircraft dataset.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll evaluate the claim based on the evidence provided:

The claim about the importance of artifact- and noise-free anomalous samples is relatively well-stated, but has some minor overstatement. Here's why:

The evidence strongly supports that:
1. Deep models can learn artifacts as shortcuts (directly stated in evidence)
2. Other generative models produce artifacts and noisy samples that harm performance (shown in Figures 4, 6, 7 comparing different methods)
3. The SDE approach produces cleaner, more artifact-free samples that improve performance

However, while the basic premise is supported, the claim makes a somewhat stronger causal statement than fully demonstrated by the evidence. The evidence shows correlation between artifact-free samples and better performance, but doesn't fully prove this is "due to" shortcut learning specifically.

<score>2</score>

The claim receives a 2 - slightly overstated but generally well-supported by the evidence. While the core concept is validated by both textual explanations and visual comparisons, the definitive causal language slightly exceeds what's directly proven by the evidence provided.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

The importance of artifact- and noise-free anomalous samples in fine tuning is due to the fact that deep models tend to learn such artifacts as shortcuts, preventing them from generalization to the true essence of the anomalous samples.

The review comment to be evaluated is:

The paper investigates the near anomalies detection and points a significant decrease in performance of state of the art approaches on near anomalies cases. A new solution is proposed: 1) using SDE models trained to generate data from the training distribution. 2) generate a dataset from the trained SDE. 3) train a binary classifier on top of a feature extractor to distinguish between the generated data and real data. 4) use nearest neighbour for detecting anomalies.
Extensive experiments and empirical analysis are carried on. The method shows good results.

The evidence to be evaluated is:In the first step, a fake dataset of anomalies is generated by a SDE-based diffusion model. We quantitatively and qualitatively show that the generated fake outliers are high-quality, diverse, and yet show semantic differences compared to the normal inputs.

The main benefit of using a score-based generative model is that in the backward process (Eq. 1), the score function denoises the input gradually that results in a relatively smooth decline in the Frechet Inception Distance (FID) (17) across training epochs. This enables us to stably produce near anomalous samples based on the FID score of the generated outputs, which could be achieved by stopping the training process earlier than the stage where the model achieves its maximum performance.

Having generated a high-quality fake dataset, a lightweight projection head is added to the feature extractor, and the whole network is trained to solve a binary classification task between the given normal and generated abnormal inputs. This way, normal class boundaries are tightened and adjusted according to the abnormal inputs.

Note that due to the feature fine-tuning phase of our approach, the borders of the normal samples become well-structured, thus better representing the normal semantic features.

The anomalous data that are generated by StyleGAN-ADA contain lots of artifact-full and distorted samples that a practical novelty detector is expected to detect.

Interestingly, other models generated samples are harmful to the performance in Weather and MvTecAD datasets. Having artifacts, mode-collapse, and undiversified generated samples could be the reasons behind this observation, which are shown in Appendix10.

It has recently been shown that contrastive learning can also improve anomaly detection (44), which encourages the model to learn normal features by contrasting positive and negative samples.

Intuitively, abnormal and normal data do not overlap in the feature space because many features model high-level and semantic image properties. So, in the pre-trained feature space, one may classify normal vs. anomaly.

In this paper, we revealed a key weakness of existing SOTA novelty detection methods, where they fail to identify anomalies that are slightly different from the normal data. We proposed to take a non-adaptive generative modeling that allows controllable generation of anomalous data.

It is important to note that the synthesized images were not an effective anomaly in the first training procedure because they are extremely noisy. Furthermore, one has to note that current deep models are highly capable of learning artifacts from their input, instead of paying attention to the general semantics of the input.

However, other generative models do not converge properly, and the images that are sampled using the early stopped model are often noisy and contain artifacts. As opposed to other generative methods, the proposed premature training of the SDE yields authentic, near-distribution, diverse, and artifact-free images.

Input: Figure 7: Generated anomalous samples by different generative models on the WBC dataset.

Input: Figure 4: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 6: Generated anomalous samples by different generative models on the FGVC-Aircraft dataset.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find that it is generally well-supported but slightly overstated. The evidence does show that artifacts and noise in generated samples can harm model performance, as demonstrated in the Weather and MvTecAD datasets. The figures (particularly 6 and 7) visually demonstrate how different generative models produce artifacts and noise. However, the claim makes a definitive statement about the "fact" of deep models learning shortcuts, when this is presented more as an observation in the evidence ("one has to note that current deep models are highly capable of learning artifacts"). The evidence suggests this is a tendency rather than an absolute fact.

<score>2</score>

Justification:
1. The evidence supports that artifact-full samples harm performance, as shown in multiple datasets and figures
2. Visual examples demonstrate how different generative approaches can produce artifacts (Figures 6 and 7)
3. The paper explicitly states that "having artifacts, mode-collapse, and undiversified generated samples could be the reasons behind this observation"
4. However, the claim overstates by presenting this as a definitive "fact" rather than an observed tendency
5. The evidence presents this more cautiously as a capability ("models are highly capable of learning artifacts") rather than an inevitable outcome

The claim would be better stated if it acknowledged this as a tendency rather than a fact, but otherwise accurately reflects the evidence.


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

The importance of artifact- and noise-free anomalous samples in fine tuning is due to the fact that deep models tend to learn such artifacts as shortcuts, preventing them from generalization to the true essence of the anomalous samples.

The review comment to be evaluated is:

* There is a large room for improvement in clarity.
    * The paper should be clearer on how exactly the generative model is trained and used. The most important fact, in my opinion, is that the score-based generative model is trained on in-distribution data and terminated prematurely. However, the fact is only briefly mentioned in Page 2, and then mentioned in Figure 2. Given the importance of this information, this point should be described in detail in Section 2.1. Also, other details and design choices, for example, how early the training of the score-based model should be stopped, needs to be elaborated.
    * The experimental setup described in Section 3.2 is difficult to follow. There are several comparison settings but are not well organized.
* Discussion on the differences among diffusion models, GANs, and VAEs in Section 2.2 are not very persuasive. It does not have concrete examples or quantitative arguments. Plus, the argument of Section 3.2 is mostly about generative modeling itself and not about its application to generating auxiliary data in novelty detection.
* When performing CIFAR-10 vs CIFAR-100 experiment, it is more standard and more challenging to incorporate all the classes, instead of using only one class from each dataset. Overall, I am not convinced that the experiment settings used in the paper are standard.
* Replacing a GAN with a diffusion model in the data generator for novelty detection itself is not a very novel contribution to the community.

The evidence to be evaluated is:In the first step, a fake dataset of anomalies is generated by a SDE-based diffusion model. We quantitatively and qualitatively show that the generated fake outliers are high-quality, diverse, and yet show semantic differences compared to the normal inputs.

The main benefit of using a score-based generative model is that in the backward process (Eq. 1), the score function denoises the input gradually that results in a relatively smooth decline in the Frechet Inception Distance (FID) (17) across training epochs. This enables us to stably produce near anomalous samples based on the FID score of the generated outputs, which could be achieved by stopping the training process earlier than the stage where the model achieves its maximum performance.

Having generated a high-quality fake dataset, a lightweight projection head is added to the feature extractor, and the whole network is trained to solve a binary classification task between the given normal and generated abnormal inputs. This way, normal class boundaries are tightened and adjusted according to the abnormal inputs.

Note that due to the feature fine-tuning phase of our approach, the borders of the normal samples become well-structured, thus better representing the normal semantic features.

The anomalous data that are generated by StyleGAN-ADA contain lots of artifact-full and distorted samples that a practical novelty detector is expected to detect.

Interestingly, other models generated samples are harmful to the performance in Weather and MvTecAD datasets. Having artifacts, mode-collapse, and undiversified generated samples could be the reasons behind this observation, which are shown in Appendix10.

It has recently been shown that contrastive learning can also improve anomaly detection (44), which encourages the model to learn normal features by contrasting positive and negative samples.

Intuitively, abnormal and normal data do not overlap in the feature space because many features model high-level and semantic image properties. So, in the pre-trained feature space, one may classify normal vs. anomaly.

In this paper, we revealed a key weakness of existing SOTA novelty detection methods, where they fail to identify anomalies that are slightly different from the normal data. We proposed to take a non-adaptive generative modeling that allows controllable generation of anomalous data.

It is important to note that the synthesized images were not an effective anomaly in the first training procedure because they are extremely noisy. Furthermore, one has to note that current deep models are highly capable of learning artifacts from their input, instead of paying attention to the general semantics of the input.

However, other generative models do not converge properly, and the images that are sampled using the early stopped model are often noisy and contain artifacts. As opposed to other generative methods, the proposed premature training of the SDE yields authentic, near-distribution, diverse, and artifact-free images.

Input: Figure 7: Generated anomalous samples by different generative models on the WBC dataset.

Input: Figure 4: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 6: Generated anomalous samples by different generative models on the FGVC-Aircraft dataset.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
After reviewing the claim and evidence, I find this claim to be moderately overstated. While the evidence, particularly Figures 6 and 7, demonstrates that other generative models (GANs, VAEs) produce samples with visible artifacts compared to the SDE approach, the categorical statement about deep models' tendency to learn artifacts as shortcuts lacks comprehensive experimental validation. The evidence primarily focuses on comparing different generative models' outputs rather than directly demonstrating how these artifacts affect model generalization. The review comments also highlight that the discussion lacks concrete examples or quantitative arguments about these differences.

<score>4</score>

Justification:
1. The claim makes a broad statement about deep models' learning behavior without sufficient experimental evidence demonstrating the causal relationship between artifacts and generalization failure.

2. While Figures 6 and 7 show qualitative differences in generated samples, there's limited quantitative analysis of how these artifacts specifically impact model generalization.

3. The review comments note that "Discussion on the differences among diffusion models, GANs, and VAEs... does not have concrete examples or quantitative arguments."

4. The evidence focuses more on showing that SDE produces better quality samples rather than proving that artifacts prevent generalization to "true essence."

5. The paper's own statement that "It is important to note that the synthesized images were not an effective anomaly in the first training procedure because they are extremely noisy" somewhat contradicts the absolute nature of the claim.