Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Next, we propose to exploit a score-based generative model to produce synthetic near-distribution anomalous data.

The review comment to be evaluated is:

The idea of this paper is very natural: try to generate near-distribution outliers to augment the original training dataset which only contains normal data (semi-supervised learning), such that we can fine-tune the pre-trained feature extractor by training a binary classifier.

Strengths:
(1) Using generated outliers to improve the accuracy of novelty detection is not new. There have been some similar works which use GANs instead of SDE.  The main contribution of this paper is that they replace the GANs with the diffusion model (which is increasingly popular recently).  The motivation here is that the outliers generated by SDE are “closer“ to the nominal distribution, and then using these "boarder" points  to deal with the near-ND setting.  Even though the pipeline and SDE are all from existing works, the novel combination is meaningful considering the theoretically and experimental results.

(2) The experimental results show that the proposed method considerably improves over existing models (especially for the near-ND setting).  It is not surprising that SDE over-perform GANs for the near-ND setting, but I am surprised that the improvement is so obvious.  It will be great if the code can be provided.

Weakness:
(1) In Section 2.2 (Diffusion models vs. GANs), they try to explain, for near-ND, why choosing diffusion models instead of other generative modes (GAN and VAE), it would be better to include flow-based models in this part. Also,  VAE-related results are not included in figure3.

(2) In Figure3,  density, coverage and FID are used as the metrics to measure the fidelity and diversity.  Could you add a brief introductions/definitions for these metrics? Also, I am confused how to get the "density" from GAN-based models as they are not tractable models.

(3)About Stopping point: In section 4, "the method is robust against the stopping point with at most 2% variation for the reasonable FID scores". However,  from table4,  it is difficult to get this conclusion. what is "reasonable" FID? Actually, the stopping point decide how similar the fake points and the real points. How to decide when to stop ?

Other minor comments:
The "normal distribution" used in this paper sometimes are confusing, which means the distribution of nominal data instead of Gaussian distribution. It would be better to use other notations.

The evidence to be evaluated is:In the first step, a fake dataset of anomalies is generated by a SDE-based diffusion model. We quantitatively and qualitatively show that the generated fake outliers are high-quality, diverse, and yet show semantic differences compared to the normal inputs.

In order to generate near anomalous samples, a basic diffusion model, called Stochastic Differential Equation (SDE) (43) is adopted. In the diffusion models, the gradient of the logarithmic probability density of the data with respect to the input is called the score. A SDE model considers a continuum of distributions that develop over time in accordance with a diffusion process, to corrupt a sample and turn it into pure noise.

The model is trained to learn how to gradually transform the noisy image back to its original form through the score fucntion (backward process). Running the backward process allows us to seamlessly transform a random noise into the data that could be used for sample generation as follows:

\(s_{\theta}(x,t)\) is called the score function, as is expected to estimate \(\nabla_{x}\log p_{t}(x)\).

The main benefit of using a score-based generative model is that in the backward process (Eq. 1), the score function denoises the input gradually that results in a relatively smooth decline in the Frechet Inception Distance (FID) (17) across training epochs. This enables us to stably produce near anomalous samples based on the FID score of the generated outputs, which could be achieved by stopping the training process earlier than the stage where the model achieves its maximum performance.

Having generated a high-quality fake dataset, a lightweight projection head is added to the feature extractor, and the whole network is trained to solve a binary classification task between the given normal and generated abnormal inputs. This way, normal class boundaries are tightened and adjusted according to the abnormal inputs.

Here, we provide some rationale on the preference of diffusion models over GANs for the the task of near-ND.

We note that in diffusion models, such as DDPM, the model is trained by optimizing the Evidence Lower Bound (ELBO) [(18)]. That is, the objective function is to minimize an upper bound on the negative evidence: \[\mathbb{E}_{x\sim q}[-\log p_{\theta}(x)], \tag{4}\] where \(q\) and \(p_{\theta}\) are the original and estimated distribution of the data, respectively. Here, \(\theta\) stands for the denoiser parameters in the DDPM. But, we note that maximizing the evidence is equivalent with minimizing the Kullback-Leibler (KL) divergence between \(q\) and \(p_{\theta}\):

Therefore, if the gap in the ELBO bound is small, minimizing the DDPM loss is equivalent with making the estimated generative distribution \(p_{\theta}\) close to the original data distribution \(q\). Hence, we expect the estimated generative distribution \(p_{\theta}\) ends up _sufficiently_ close to \(q\) if the training is prematurely stopped, and therefore generates "near-distribution" patterns.

To show the quality of the proposed generated dataset, we evaluate the performance of some of the most outstanding ND methods when the generated dataset is considered as anomalous at the test time. We call the dataset CIFAR-10-FSDE. In this setting, one class of CIFAR-10 is selected as normal and the test set is made using the corresponding test samples of the normal class and a random subset of the fake dataset, as abnormal samples, with the same size as the normal class. The anomalous data that are generated by StyleGAN-ADA contain lots of artifact-full and distorted samples that a practical novelty detector is expected to detect.

Surprisingly, even by employing fake abnormal samples generated by a GAN and being exposed to real outliers, it is not able to achieve a decent performance in the near-ND setting (Table 1 (c)), which again supports the benefits of our proposed approach for generating near distribution abnormal samples.

In this experiment, the effect of using different generative models is explored.

Table 3 shows clear superiority of using a SDE-based model compared to the well-known generative models (20; 42; 5; 14) with 2% to 20% better AUROCs based on the chosen dataset. Evidently, employing our fake samples generated by the SDE is almost always beneficial as opposed the ones generated by the other models. Interestingly, other models generated samples are harmful to the performance in Weather and MvTecAD datasets. Having artifacts, mode-collapse, and undiversified generated samples could be the reasons behind this observation, which are shown in Appendix10. This highlights that not all the generative models are beneficial for the ND and near-ND tasks, and they need to be carefully selected based on the constraints defined in such domains.

Utilizing the fake data for the novelty detection task has previously been considered. The general idea is to employ synthetic images, which may be generated by GANs, to augment the training set (6; 26; 50; 29; 11; 1). In the case of open-set recognition, (21) proposed OpenGAN, which adverserially generates fake open-set images. The discriminator is then utilized at the test time for the novelty detection. The key point in OpenGAN is that a tiny additional dataset, containing both in- and outliers, is used as a validation set for the model selection. This, also known as outlier exposure, is necessary due to the GAN unstable training. Even though these models can work well on simple datasets, they cannot handle more complex datasets and cannot detect hard near-distribution anomalies.

We proposed to take a non-adaptive generative modeling that allows controllable generation of anomalous data. It turns out that SDEs fit this specification much better compared to other SOTA generative models such as GANs. The generated data levels up pre-trained distance-based novelty detection methods not only based on their AUROCs in the standard ND setting but also in the near-ND setting.

ZongYuan Ge, Sergey Demyanov, Zetao Chen, and Rahil Garnavi. Generative openmax for multi-class open set classification.

Algan: Anomaly detection by generating pseudo anomalous data via latent variables.

G2d: generate to detect anomaly.

Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. _arXiv preprint arXiv:2011.13456_, 2020.

Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jinwoo Shin. Csi: Novelty detection via contrastive learning on distributionally shifted instances.

For the data generation phase, we use the SDE-based model explained in (43) with exactly the same setup default hyper-parameters e.g. we used predictor-corrector as a sampler. The inputs are resized to 224\(\times\)224, and as a rule of thumb, the data generation is stopped on FID \(\approx\) 40 and FID \(\approx\) 200 for low and high-resolution datasets.

In this section, according to our main setting, we have replaced the SDE with other generative models, and sampled the images generated by these models before convergence. Because in our setup, only one class (the normal class) is available for the training of the generative model. As the sample size becomes limited in such a setting, model convergence and generation of high quality real samples become a challenge. Even under such circumstances, the SDE-based models converge. Furthermore, such models can eventually generate high quality real images.

As opposed to other generative methods, the proposed premature training of the SDE yields authentic, near-distribution, diverse, and artifact-free images. This makes it a better choice for the near-ND setup.

As can be seen, the images are clearly different semantically from the normal class. Despite this semantic difference, most novelty detection models have a poor performance in the detection of these images as anomalous. \begin{table}\end{table}
Table 1: The performance of novelty detection methods (AUROC %) in near-ND and ND on various datasets. Our method outperforms SOTA by significant margins on all settings. Note that except CSI, all the other SOTA approaches employ a pre-trained architecture, particularly Transformaly (7), that utilizes a ViT-backbone.

Input: Table 1: The performance of novelty detection methods (AUROC %) in near-ND and ND on various datasets. Our method outperforms SOTA by significant margins on all settings. Note that except CSI, all the other SOTA approaches employ a pre-trained architecture, particularly Transformaly (7), that utilizes a ViT-backbone.

Input: Table 3: Comparison of our model performance (in AUROC %) upon using different generative models for each class of the CIFAR-10, Weather, WBC and MvTecAD dataset. The performance of the base backbone is also included to measure the effectiveness of the training process. Diffusionbased data generation considerably passes the GAN-based SOTAs in the novelty detection task. Coverage&Density metrics are calculated according to the dataset and generative model. These metrics measure the diversity and fidelity of the samples that are generated by generative models.

Input: Figure 11: Generated anomaly images on the Caltech-UCSD Birds 200 dataset. The images highlighted in green are normal samples.

Input: Figure 12: Generated anomaly images on the MVTecAD dataset. The images highlighted in green are normal samples.

Input: Figure 2: The performance of novelty detection methods (AUROC %) in the near-ND and ND settings. In the near-ND setting, results are reported according to both criteria, i.e. CLP and bottom-1.

Input: Figure 2: Overview of our framework for near distribution novelty detection. In the first step, the SDE diffusion model generates semantically close outliers, after its training is early-stopped. As it is shown, they have very subtle yet semantic differences with the normal data distribution (downleft). After that, using a linear layer, a pre-trained feature extractor is fine-tuned to solve a binary classification between the normal and abnormal inputs. This modifies the normal boundaries toward including more distinctive features. Finally, normal embeddings are stored and used to compute the k-NN distance at the test time as the novelty score.

Input: Figure 7: Generated anomalous samples by different generative models on the WBC dataset.

Input: Figure 8: An overview images in the synthetic CIFAR10-FSDE dataset.

Input: Figure 11: Generated anomaly images on the Caltech-UCSD Birds 200 dataset. The images highlighted in green are normal samples.

Input: Figure 12: Generated anomaly images on the MVTecAD dataset. The images highlighted in green are normal samples.

Input: Figure 1: Spectrum of AUCs, based on Stopping points for WBC dataset and CIFAR10-vs100 setting, as proxies for low- and high-resolution data, respectively.

Input: Figure 1: Sensitivity of state-of-the-art anomaly detectors when differences between normal and anomalous samples are subtle. For each model, its performance in detecting far (Blue) and near (Orange) outliers has been reported. A single class of CIFAR-10 is considered as normal, the rest of the classes are considered far anomalies, and the semantically nearest class of CIFAR-100 to the normal class is the source of near anomalies. This is the class that corresponds to the lowest test AUROC in anomaly detection. A considerable performance drop happens for the most recent methods, which means the learned normal boundaries are not tight enough and need to be adapted. For instance, while some might be able to distinguish a cat image from a car, they are still unable to detect tiger or lion images as anomalies.

Input: Figure 10: Generated anomaly images on the StanfordCars dataset. The images highlighted in green are normal samples.

Input: Figure 4: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 9: Generated anomaly images on 102 category of the Flowers dataset. The images highlighted in green are normal samples.

Input: Figure 2: The performance of novelty detection methods (AUROC %) in the near-ND and ND settings. In the near-ND setting, results are reported according to both criteria, i.e. CLP and bottom-1.

Input: Figure 2: Overview of our framework for near distribution novelty detection. In the first step, the SDE diffusion model generates semantically close outliers, after its training is early-stopped. As it is shown, they have very subtle yet semantic differences with the normal data distribution (downleft). After that, using a linear layer, a pre-trained feature extractor is fine-tuned to solve a binary classification between the normal and abnormal inputs. This modifies the normal boundaries toward including more distinctive features. Finally, normal embeddings are stored and used to compute the k-NN distance at the test time as the novelty score.

Input: Figure 5: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 10: Generated anomaly images on the StanfordCars dataset. The images highlighted in green are normal samples.

Input: Figure 6: Generated anomalous samples by different generative models on the FGVC-Aircraft dataset.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I've carefully reviewed the claim about exploiting score-based generative models to produce synthetic near-distribution anomalous data. The evidence strongly supports this claim through:

1. Quantitative results showing superior performance of SDE compared to other generative models (Table 3)
2. Visual evidence demonstrating high-quality, diverse generated samples (Figures 11, 12)
3. Theoretical foundation explaining why SDEs are suitable for near-distribution generation
4. Experimental validation across multiple datasets showing consistent improvements
5. Clear methodology for controlling the generation process via early stopping

The claim is well-stated and accurately reflects the demonstrated capabilities without exaggeration. The evidence comprehensively supports the technical feasibility and effectiveness of this approach.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

Next, we propose to exploit a score-based generative model to produce synthetic near-distribution anomalous data.

The review comment to be evaluated is:

### Strengths

- The paper is, for the most part, well-written, easy-to-follow, and does a good job of
contextualising and motivating the work.
- The proposed method is simple and intuitive, and seems to perform well empirically
compared with the baselines in both the regular ND setting and the near ND setting, showing a
significant reduction in the performance gap of going from the former to the latter.
the NR to near-NR setting.
- The figures are well-chosen and well-put-together: they convey well the essential points of the
  paper, such as the shortcomings of existing methods and the feasibility of using GANs vs diffusion
  models for creating the synthetic data.
- Evaluation performed using a good range and selection of baseline methods and benchmark datasets.
- Results are aggregated over a good number (10) of replicates.

### Weaknesses
- Despite the authors claiming otherwise, the method is quite sensitive to the stopping criterion in
  the near ND setting with an empirical range of [68.2, 90.0]; in practice, different datasets may
  require wildly different FID thresholds and there is no reliable way of validating this given the
  nature of the problem. Also, I wonder if the number of sampling steps and the choice of sampler have a
  significant effect on the results -- there doesn't seem to be any mention of this.
  Taking this a step further, might it be feasible to train the diffusion model fully and then rely
  upon early-stopping the reverse diffusion process according to the FID score?
- The need to train, and indeed sample, from a diffusion model can be computationally burdensome for
  higher-resolution datasets.
- The description of the SDE model is a little overdone; a much briefer description with a pointing of the
  reader to the original paper would be sufficient. This is especially the case given that, as
  I understand it, the proposed method should work with any diffusion based model (Imagen, Parti,
  DALLE-2, etc.) or are there theoretically some constraints on the form said model and the
  associated sampler should take?
- While this applies to several methods of the same class, the method seems to make certain
assumptions about the distribution of the ID and OOD data that might not hold in the wild.
- While results are aggregated over a good number (10) of replicates no measure of dispersion
  (standard deviation/error) seems to be reported alongside the mean in each case.
- Some typographic and syntactical errors (e.g. 'that unlikely come' (line 1 of the introduction)).

The evidence to be evaluated is:In the first step, a fake dataset of anomalies is generated by a SDE-based diffusion model. We quantitatively and qualitatively show that the generated fake outliers are high-quality, diverse, and yet show semantic differences compared to the normal inputs.

In order to generate near anomalous samples, a basic diffusion model, called Stochastic Differential Equation (SDE) (43) is adopted. In the diffusion models, the gradient of the logarithmic probability density of the data with respect to the input is called the score. A SDE model considers a continuum of distributions that develop over time in accordance with a diffusion process, to corrupt a sample and turn it into pure noise.

The model is trained to learn how to gradually transform the noisy image back to its original form through the score fucntion (backward process). Running the backward process allows us to seamlessly transform a random noise into the data that could be used for sample generation as follows:

\(s_{\theta}(x,t)\) is called the score function, as is expected to estimate \(\nabla_{x}\log p_{t}(x)\).

The main benefit of using a score-based generative model is that in the backward process (Eq. 1), the score function denoises the input gradually that results in a relatively smooth decline in the Frechet Inception Distance (FID) (17) across training epochs. This enables us to stably produce near anomalous samples based on the FID score of the generated outputs, which could be achieved by stopping the training process earlier than the stage where the model achieves its maximum performance.

Having generated a high-quality fake dataset, a lightweight projection head is added to the feature extractor, and the whole network is trained to solve a binary classification task between the given normal and generated abnormal inputs. This way, normal class boundaries are tightened and adjusted according to the abnormal inputs.

Here, we provide some rationale on the preference of diffusion models over GANs for the the task of near-ND.

We note that in diffusion models, such as DDPM, the model is trained by optimizing the Evidence Lower Bound (ELBO) [(18)]. That is, the objective function is to minimize an upper bound on the negative evidence: \[\mathbb{E}_{x\sim q}[-\log p_{\theta}(x)], \tag{4}\] where \(q\) and \(p_{\theta}\) are the original and estimated distribution of the data, respectively. Here, \(\theta\) stands for the denoiser parameters in the DDPM. But, we note that maximizing the evidence is equivalent with minimizing the Kullback-Leibler (KL) divergence between \(q\) and \(p_{\theta}\):

Therefore, if the gap in the ELBO bound is small, minimizing the DDPM loss is equivalent with making the estimated generative distribution \(p_{\theta}\) close to the original data distribution \(q\). Hence, we expect the estimated generative distribution \(p_{\theta}\) ends up _sufficiently_ close to \(q\) if the training is prematurely stopped, and therefore generates "near-distribution" patterns.

To show the quality of the proposed generated dataset, we evaluate the performance of some of the most outstanding ND methods when the generated dataset is considered as anomalous at the test time. We call the dataset CIFAR-10-FSDE. In this setting, one class of CIFAR-10 is selected as normal and the test set is made using the corresponding test samples of the normal class and a random subset of the fake dataset, as abnormal samples, with the same size as the normal class. The anomalous data that are generated by StyleGAN-ADA contain lots of artifact-full and distorted samples that a practical novelty detector is expected to detect.

Surprisingly, even by employing fake abnormal samples generated by a GAN and being exposed to real outliers, it is not able to achieve a decent performance in the near-ND setting (Table 1 (c)), which again supports the benefits of our proposed approach for generating near distribution abnormal samples.

In this experiment, the effect of using different generative models is explored.

Table 3 shows clear superiority of using a SDE-based model compared to the well-known generative models (20; 42; 5; 14) with 2% to 20% better AUROCs based on the chosen dataset. Evidently, employing our fake samples generated by the SDE is almost always beneficial as opposed the ones generated by the other models. Interestingly, other models generated samples are harmful to the performance in Weather and MvTecAD datasets. Having artifacts, mode-collapse, and undiversified generated samples could be the reasons behind this observation, which are shown in Appendix10. This highlights that not all the generative models are beneficial for the ND and near-ND tasks, and they need to be carefully selected based on the constraints defined in such domains.

Utilizing the fake data for the novelty detection task has previously been considered. The general idea is to employ synthetic images, which may be generated by GANs, to augment the training set (6; 26; 50; 29; 11; 1). In the case of open-set recognition, (21) proposed OpenGAN, which adverserially generates fake open-set images. The discriminator is then utilized at the test time for the novelty detection. The key point in OpenGAN is that a tiny additional dataset, containing both in- and outliers, is used as a validation set for the model selection. This, also known as outlier exposure, is necessary due to the GAN unstable training. Even though these models can work well on simple datasets, they cannot handle more complex datasets and cannot detect hard near-distribution anomalies.

We proposed to take a non-adaptive generative modeling that allows controllable generation of anomalous data. It turns out that SDEs fit this specification much better compared to other SOTA generative models such as GANs. The generated data levels up pre-trained distance-based novelty detection methods not only based on their AUROCs in the standard ND setting but also in the near-ND setting.

ZongYuan Ge, Sergey Demyanov, Zetao Chen, and Rahil Garnavi. Generative openmax for multi-class open set classification.

Algan: Anomaly detection by generating pseudo anomalous data via latent variables.

G2d: generate to detect anomaly.

Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. _arXiv preprint arXiv:2011.13456_, 2020.

Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jinwoo Shin. Csi: Novelty detection via contrastive learning on distributionally shifted instances.

For the data generation phase, we use the SDE-based model explained in (43) with exactly the same setup default hyper-parameters e.g. we used predictor-corrector as a sampler. The inputs are resized to 224\(\times\)224, and as a rule of thumb, the data generation is stopped on FID \(\approx\) 40 and FID \(\approx\) 200 for low and high-resolution datasets.

In this section, according to our main setting, we have replaced the SDE with other generative models, and sampled the images generated by these models before convergence. Because in our setup, only one class (the normal class) is available for the training of the generative model. As the sample size becomes limited in such a setting, model convergence and generation of high quality real samples become a challenge. Even under such circumstances, the SDE-based models converge. Furthermore, such models can eventually generate high quality real images.

As opposed to other generative methods, the proposed premature training of the SDE yields authentic, near-distribution, diverse, and artifact-free images. This makes it a better choice for the near-ND setup.

As can be seen, the images are clearly different semantically from the normal class. Despite this semantic difference, most novelty detection models have a poor performance in the detection of these images as anomalous. \begin{table}\end{table}
Table 1: The performance of novelty detection methods (AUROC %) in near-ND and ND on various datasets. Our method outperforms SOTA by significant margins on all settings. Note that except CSI, all the other SOTA approaches employ a pre-trained architecture, particularly Transformaly (7), that utilizes a ViT-backbone.

Input: Table 1: The performance of novelty detection methods (AUROC %) in near-ND and ND on various datasets. Our method outperforms SOTA by significant margins on all settings. Note that except CSI, all the other SOTA approaches employ a pre-trained architecture, particularly Transformaly (7), that utilizes a ViT-backbone.

Input: Table 3: Comparison of our model performance (in AUROC %) upon using different generative models for each class of the CIFAR-10, Weather, WBC and MvTecAD dataset. The performance of the base backbone is also included to measure the effectiveness of the training process. Diffusionbased data generation considerably passes the GAN-based SOTAs in the novelty detection task. Coverage&Density metrics are calculated according to the dataset and generative model. These metrics measure the diversity and fidelity of the samples that are generated by generative models.

Input: Figure 11: Generated anomaly images on the Caltech-UCSD Birds 200 dataset. The images highlighted in green are normal samples.

Input: Figure 12: Generated anomaly images on the MVTecAD dataset. The images highlighted in green are normal samples.

Input: Figure 2: The performance of novelty detection methods (AUROC %) in the near-ND and ND settings. In the near-ND setting, results are reported according to both criteria, i.e. CLP and bottom-1.

Input: Figure 2: Overview of our framework for near distribution novelty detection. In the first step, the SDE diffusion model generates semantically close outliers, after its training is early-stopped. As it is shown, they have very subtle yet semantic differences with the normal data distribution (downleft). After that, using a linear layer, a pre-trained feature extractor is fine-tuned to solve a binary classification between the normal and abnormal inputs. This modifies the normal boundaries toward including more distinctive features. Finally, normal embeddings are stored and used to compute the k-NN distance at the test time as the novelty score.

Input: Figure 7: Generated anomalous samples by different generative models on the WBC dataset.

Input: Figure 8: An overview images in the synthetic CIFAR10-FSDE dataset.

Input: Figure 11: Generated anomaly images on the Caltech-UCSD Birds 200 dataset. The images highlighted in green are normal samples.

Input: Figure 12: Generated anomaly images on the MVTecAD dataset. The images highlighted in green are normal samples.

Input: Figure 1: Spectrum of AUCs, based on Stopping points for WBC dataset and CIFAR10-vs100 setting, as proxies for low- and high-resolution data, respectively.

Input: Figure 1: Sensitivity of state-of-the-art anomaly detectors when differences between normal and anomalous samples are subtle. For each model, its performance in detecting far (Blue) and near (Orange) outliers has been reported. A single class of CIFAR-10 is considered as normal, the rest of the classes are considered far anomalies, and the semantically nearest class of CIFAR-100 to the normal class is the source of near anomalies. This is the class that corresponds to the lowest test AUROC in anomaly detection. A considerable performance drop happens for the most recent methods, which means the learned normal boundaries are not tight enough and need to be adapted. For instance, while some might be able to distinguish a cat image from a car, they are still unable to detect tiger or lion images as anomalies.

Input: Figure 10: Generated anomaly images on the StanfordCars dataset. The images highlighted in green are normal samples.

Input: Figure 4: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 9: Generated anomaly images on 102 category of the Flowers dataset. The images highlighted in green are normal samples.

Input: Figure 2: The performance of novelty detection methods (AUROC %) in the near-ND and ND settings. In the near-ND setting, results are reported according to both criteria, i.e. CLP and bottom-1.

Input: Figure 2: Overview of our framework for near distribution novelty detection. In the first step, the SDE diffusion model generates semantically close outliers, after its training is early-stopped. As it is shown, they have very subtle yet semantic differences with the normal data distribution (downleft). After that, using a linear layer, a pre-trained feature extractor is fine-tuned to solve a binary classification between the normal and abnormal inputs. This modifies the normal boundaries toward including more distinctive features. Finally, normal embeddings are stored and used to compute the k-NN distance at the test time as the novelty score.

Input: Figure 5: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 10: Generated anomaly images on the StanfordCars dataset. The images highlighted in green are normal samples.

Input: Figure 6: Generated anomalous samples by different generative models on the FGVC-Aircraft dataset.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on the comprehensive evidence provided, including visual examples, experimental results, and theoretical justification, the claim about exploiting score-based generative models for synthetic near-distribution anomalous data appears well-supported. The evidence demonstrates:

- Superior performance of SDE models compared to other generative approaches (Table 3)
- High-quality, diverse sample generation (Figures 8-12)
- Theoretical grounding in ELBO optimization and KL divergence
- Controlled generation through early stopping based on FID scores
- Extensive experimental validation across multiple datasets

While the review points out some limitations regarding stopping criteria sensitivity and computational burden, these don't invalidate the core claim. The proposition is carefully stated without overstating capabilities.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To address the mentioned challenges, we propose to use a "non-adversarial" diffusion-based anomaly data generation method, which can estimate the true normal distribution accurately and smoothly over the training time.

The evidence to be evaluated is:In the first step, a fake dataset of anomalies is generated by a SDE-based diffusion model. We quantitatively and qualitatively show that the generated fake outliers are high-quality, diverse, and yet show semantic differences compared to the normal inputs.

In order to generate near anomalous samples, a basic diffusion model, called Stochastic Differential Equation (SDE) (43) is adopted.

A SDE model considers a continuum of distributions that develop over time in accordance with a diffusion process, to corrupt a sample and turn it into pure noise.

The model is trained to learn how to gradually transform the noisy image back to its original form through the score fucntion (backward process). Running the backward process allows us to seamlessly transform a random noise into the data that could be used for sample generation as follows: \[x_{n-1}-x_{n}=[f(x,t)-g^{2}(t)s_{\theta}(x,t)]dt+g(t)d\hat{w},\quad x_{T}\sim N (0,1), \tag{1}\] where \(x_{n}\) is the data sample at time \(n\), \(f=0\) and \(g=\sigma^{t}\) are called the drift and diffusion coefficients, respectively. \(s_{\theta}(x,t)\) is called the score function, as is expected to estimate \(\nabla_{x}\log p_{t}(x)\).

The main benefit of using a score-based generative model is that in the backward process (Eq. 1), the score function denoises the input gradually that results in a relatively smooth decline in the Frechet Inception Distance (FID) (17) across training epochs. This enables us to stably produce near anomalous samples based on the FID score of the generated outputs, which could be achieved by stopping the training process earlier than the stage where the model achieves its maximum performance.

Note that a totally different training trajectory is obtained in GANs, where the FID oscillates and premature training does not necessarily produce high-quality anomalous samples.

In addition, Fig. 3 right, shows that the quality and closeness of the samples that are generated by the diffusion model are visually more appealing that the SOTA GAN-based models.

Having generated a high-quality fake dataset, a lightweight projection head is added to the feature extractor, and the whole network is trained to solve a binary classification task between the given normal and generated abnormal inputs. This way, normal class boundaries are tightened and adjusted according to the abnormal inputs.

Note that due to the feature fine-tuning phase of our approach, the borders of the normal samples become well-structured, thus better representing the normal semantic features.

Here, we provide some rationale on the preference of diffusion models over GANs for the the task of near-ND.

We note that in diffusion models, such as DDPM, the model is trained by optimizing the Evidence Lower Bound (ELBO) [(18)]. That is, the objective function is to minimize an upper bound on the negative evidence: \[\mathbb{E}_{x\sim q}[-\log p_{\theta}(x)], \tag{4}\] where \(q\) and \(p_{\theta}\) are the original and estimated distribution of the data, respectively. Here, \(\theta\) stands for the denoiser parameters in the DDPM. But, we note that maximizing the evidence is equivalent with minimizing the Kullback-Leibler (KL) divergence between \(q\) and \(p_{\theta}\):

Therefore, if the gap in the ELBO bound is small, minimizing the DDPM loss is equivalent with making the estimated generative distribution \(p_{\theta}\) close to the original data distribution \(q\). Hence, we expect the estimated generative distribution \(p_{\theta}\) ends up _sufficiently_ close to \(q\) if the training is prematurely stopped, and therefore generates "near-distribution" patterns.

However, such metrics exhibit a steady and consistent increase till the estimated distribution \(p_{\theta}\) gets sufficiently close to the true distribution \(q\) (see Fig. 3 left).

We proposed to take a non-adaptive generative modeling that allows controllable generation of anomalous data. It turns out that SDEs fit this specification much better compared to other SOTA generative models such as GANs. The generated data levels up pre-trained distance-based novelty detection methods not only based on their AUROCs in the standard ND setting but also in the near-ND setting.

Algan: Anomaly detection by generating pseudo anomalous data via latent variables.

For the data generation phase, we use the SDE-based model explained in (43) with exactly the same setup default hyper-parameters e.g. we used predictor-corrector as a sampler.

The density and coverage metrics can be used to assess the diversity and fidelity of generative models, respectively.

Because in our setup, only one class (the normal class) is available for the training of the generative model. As the sample size becomes limited in such a setting, model convergence and generation of high quality real samples become a challenge. Even under such circumstances, the SDE-based models converge. Furthermore, such models can eventually generate high quality real images.

As opposed to other generative methods, the proposed premature training of the SDE yields authentic, near-distribution, diverse, and artifact-free images. This makes it a better choice for the near-ND setup.

Input: Figure 11: Generated anomaly images on the Caltech-UCSD Birds 200 dataset. The images highlighted in green are normal samples.

Input: Figure 12: Generated anomaly images on the MVTecAD dataset. The images highlighted in green are normal samples.

Input: Figure 11: Generated anomaly images on the Caltech-UCSD Birds 200 dataset. The images highlighted in green are normal samples.

Input: Figure 12: Generated anomaly images on the MVTecAD dataset. The images highlighted in green are normal samples.

Input: Figure 1: Spectrum of AUCs, based on Stopping points for WBC dataset and CIFAR10-vs100 setting, as proxies for low- and high-resolution data, respectively.

Input: Figure 1: Sensitivity of state-of-the-art anomaly detectors when differences between normal and anomalous samples are subtle. For each model, its performance in detecting far (Blue) and near (Orange) outliers has been reported. A single class of CIFAR-10 is considered as normal, the rest of the classes are considered far anomalies, and the semantically nearest class of CIFAR-100 to the normal class is the source of near anomalies. This is the class that corresponds to the lowest test AUROC in anomaly detection. A considerable performance drop happens for the most recent methods, which means the learned normal boundaries are not tight enough and need to be adapted. For instance, while some might be able to distinguish a cat image from a car, they are still unable to detect tiger or lion images as anomalies.

Input: Figure 10: Generated anomaly images on the StanfordCars dataset. The images highlighted in green are normal samples.

Input: Figure 4: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 9: Generated anomaly images on 102 category of the Flowers dataset. The images highlighted in green are normal samples.

Input: Figure 2: The performance of novelty detection methods (AUROC %) in the near-ND and ND settings. In the near-ND setting, results are reported according to both criteria, i.e. CLP and bottom-1.

Input: Figure 2: Overview of our framework for near distribution novelty detection. In the first step, the SDE diffusion model generates semantically close outliers, after its training is early-stopped. As it is shown, they have very subtle yet semantic differences with the normal data distribution (downleft). After that, using a linear layer, a pre-trained feature extractor is fine-tuned to solve a binary classification between the normal and abnormal inputs. This modifies the normal boundaries toward including more distinctive features. Finally, normal embeddings are stored and used to compute the k-NN distance at the test time as the novelty score.

Input: Figure 5: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 10: Generated anomaly images on the StanfordCars dataset. The images highlighted in green are normal samples.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on the evidence presented, I'll evaluate the claim about the non-adversarial diffusion-based anomaly data generation method.

The evidence strongly supports the claim through:
- Extensive visual examples showing high-quality generated anomalies across multiple datasets (MVTecAD, Birds, Cars, Flowers)
- Quantitative evidence of smooth FID score progression during training
- Mathematical formulation showing the theoretical foundation
- Comparative results against GAN-based approaches showing better stability and quality
- Demonstration of the method's ability to generate semantically meaningful variations

The claim is well-stated as it accurately reflects the capabilities demonstrated by the evidence without overstating them. The approach shows consistent performance across different domains while acknowledging the underlying methodology.

<score>1</score>

Justification:
1. The visual evidence clearly demonstrates the quality and diversity of generated samples
2. The theoretical framework is well-grounded in mathematical principles
3. Performance comparisons are supported by empirical results
4. The claim's language ("accurately and smoothly") is justified by the FID progression data
5. The evidence comprehensively supports all aspects of the claim without exaggeration


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To address the mentioned challenges, we propose to use a "non-adversarial" diffusion-based anomaly data generation method, which can estimate the true normal distribution accurately and smoothly over the training time.

The review comment to be evaluated is:

The paper investigates the near anomalies detection and points a significant decrease in performance of state of the art approaches on near anomalies cases. A new solution is proposed: 1) using SDE models trained to generate data from the training distribution. 2) generate a dataset from the trained SDE. 3) train a binary classifier on top of a feature extractor to distinguish between the generated data and real data. 4) use nearest neighbour for detecting anomalies.
Extensive experiments and empirical analysis are carried on. The method shows good results.

The evidence to be evaluated is:In the first step, a fake dataset of anomalies is generated by a SDE-based diffusion model. We quantitatively and qualitatively show that the generated fake outliers are high-quality, diverse, and yet show semantic differences compared to the normal inputs.

In order to generate near anomalous samples, a basic diffusion model, called Stochastic Differential Equation (SDE) (43) is adopted.

A SDE model considers a continuum of distributions that develop over time in accordance with a diffusion process, to corrupt a sample and turn it into pure noise.

The model is trained to learn how to gradually transform the noisy image back to its original form through the score fucntion (backward process). Running the backward process allows us to seamlessly transform a random noise into the data that could be used for sample generation as follows: \[x_{n-1}-x_{n}=[f(x,t)-g^{2}(t)s_{\theta}(x,t)]dt+g(t)d\hat{w},\quad x_{T}\sim N (0,1), \tag{1}\] where \(x_{n}\) is the data sample at time \(n\), \(f=0\) and \(g=\sigma^{t}\) are called the drift and diffusion coefficients, respectively. \(s_{\theta}(x,t)\) is called the score function, as is expected to estimate \(\nabla_{x}\log p_{t}(x)\).

The main benefit of using a score-based generative model is that in the backward process (Eq. 1), the score function denoises the input gradually that results in a relatively smooth decline in the Frechet Inception Distance (FID) (17) across training epochs. This enables us to stably produce near anomalous samples based on the FID score of the generated outputs, which could be achieved by stopping the training process earlier than the stage where the model achieves its maximum performance.

Note that a totally different training trajectory is obtained in GANs, where the FID oscillates and premature training does not necessarily produce high-quality anomalous samples.

In addition, Fig. 3 right, shows that the quality and closeness of the samples that are generated by the diffusion model are visually more appealing that the SOTA GAN-based models.

Having generated a high-quality fake dataset, a lightweight projection head is added to the feature extractor, and the whole network is trained to solve a binary classification task between the given normal and generated abnormal inputs. This way, normal class boundaries are tightened and adjusted according to the abnormal inputs.

Note that due to the feature fine-tuning phase of our approach, the borders of the normal samples become well-structured, thus better representing the normal semantic features.

Here, we provide some rationale on the preference of diffusion models over GANs for the the task of near-ND.

We note that in diffusion models, such as DDPM, the model is trained by optimizing the Evidence Lower Bound (ELBO) [(18)]. That is, the objective function is to minimize an upper bound on the negative evidence: \[\mathbb{E}_{x\sim q}[-\log p_{\theta}(x)], \tag{4}\] where \(q\) and \(p_{\theta}\) are the original and estimated distribution of the data, respectively. Here, \(\theta\) stands for the denoiser parameters in the DDPM. But, we note that maximizing the evidence is equivalent with minimizing the Kullback-Leibler (KL) divergence between \(q\) and \(p_{\theta}\):

Therefore, if the gap in the ELBO bound is small, minimizing the DDPM loss is equivalent with making the estimated generative distribution \(p_{\theta}\) close to the original data distribution \(q\). Hence, we expect the estimated generative distribution \(p_{\theta}\) ends up _sufficiently_ close to \(q\) if the training is prematurely stopped, and therefore generates "near-distribution" patterns.

However, such metrics exhibit a steady and consistent increase till the estimated distribution \(p_{\theta}\) gets sufficiently close to the true distribution \(q\) (see Fig. 3 left).

We proposed to take a non-adaptive generative modeling that allows controllable generation of anomalous data. It turns out that SDEs fit this specification much better compared to other SOTA generative models such as GANs. The generated data levels up pre-trained distance-based novelty detection methods not only based on their AUROCs in the standard ND setting but also in the near-ND setting.

Algan: Anomaly detection by generating pseudo anomalous data via latent variables.

For the data generation phase, we use the SDE-based model explained in (43) with exactly the same setup default hyper-parameters e.g. we used predictor-corrector as a sampler.

The density and coverage metrics can be used to assess the diversity and fidelity of generative models, respectively.

Because in our setup, only one class (the normal class) is available for the training of the generative model. As the sample size becomes limited in such a setting, model convergence and generation of high quality real samples become a challenge. Even under such circumstances, the SDE-based models converge. Furthermore, such models can eventually generate high quality real images.

As opposed to other generative methods, the proposed premature training of the SDE yields authentic, near-distribution, diverse, and artifact-free images. This makes it a better choice for the near-ND setup.

Input: Figure 11: Generated anomaly images on the Caltech-UCSD Birds 200 dataset. The images highlighted in green are normal samples.

Input: Figure 12: Generated anomaly images on the MVTecAD dataset. The images highlighted in green are normal samples.

Input: Figure 11: Generated anomaly images on the Caltech-UCSD Birds 200 dataset. The images highlighted in green are normal samples.

Input: Figure 12: Generated anomaly images on the MVTecAD dataset. The images highlighted in green are normal samples.

Input: Figure 1: Spectrum of AUCs, based on Stopping points for WBC dataset and CIFAR10-vs100 setting, as proxies for low- and high-resolution data, respectively.

Input: Figure 1: Sensitivity of state-of-the-art anomaly detectors when differences between normal and anomalous samples are subtle. For each model, its performance in detecting far (Blue) and near (Orange) outliers has been reported. A single class of CIFAR-10 is considered as normal, the rest of the classes are considered far anomalies, and the semantically nearest class of CIFAR-100 to the normal class is the source of near anomalies. This is the class that corresponds to the lowest test AUROC in anomaly detection. A considerable performance drop happens for the most recent methods, which means the learned normal boundaries are not tight enough and need to be adapted. For instance, while some might be able to distinguish a cat image from a car, they are still unable to detect tiger or lion images as anomalies.

Input: Figure 10: Generated anomaly images on the StanfordCars dataset. The images highlighted in green are normal samples.

Input: Figure 4: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 9: Generated anomaly images on 102 category of the Flowers dataset. The images highlighted in green are normal samples.

Input: Figure 2: The performance of novelty detection methods (AUROC %) in the near-ND and ND settings. In the near-ND setting, results are reported according to both criteria, i.e. CLP and bottom-1.

Input: Figure 2: Overview of our framework for near distribution novelty detection. In the first step, the SDE diffusion model generates semantically close outliers, after its training is early-stopped. As it is shown, they have very subtle yet semantic differences with the normal data distribution (downleft). After that, using a linear layer, a pre-trained feature extractor is fine-tuned to solve a binary classification between the normal and abnormal inputs. This modifies the normal boundaries toward including more distinctive features. Finally, normal embeddings are stored and used to compute the k-NN distance at the test time as the novelty score.

Input: Figure 5: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 10: Generated anomaly images on the StanfordCars dataset. The images highlighted in green are normal samples.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on my review of the claim and evidence:

The claim about using a "non-adversarial" diffusion-based method for anomaly generation appears well-supported by the extensive evidence provided. The paper demonstrates through quantitative metrics (FID scores) and qualitative results (generated images) that their SDE-based approach can generate high-quality, diverse near-anomalous samples. The evidence shows smooth training trajectories compared to GANs and presents compelling visual results across multiple datasets (MVTecAD, StanfordCars, Birds, Flowers). The theoretical foundation is well-explained through the ELBO objective and KL divergence minimization. The reviewer's comments also validate the method's effectiveness.

<score>1</score>

Justification:
- The claim is carefully worded without exaggeration
- Extensive experimental evidence supports the key points
- Visual results demonstrate the method's capabilities
- Theoretical framework provides solid grounding
- Review comments confirm the method's good performance
- No overstatement of capabilities or limitations


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To address the mentioned challenges, we propose to use a "non-adversarial" diffusion-based anomaly data generation method, which can estimate the true normal distribution accurately and smoothly over the training time.

The review comment to be evaluated is:

* There is a large room for improvement in clarity.
    * The paper should be clearer on how exactly the generative model is trained and used. The most important fact, in my opinion, is that the score-based generative model is trained on in-distribution data and terminated prematurely. However, the fact is only briefly mentioned in Page 2, and then mentioned in Figure 2. Given the importance of this information, this point should be described in detail in Section 2.1. Also, other details and design choices, for example, how early the training of the score-based model should be stopped, needs to be elaborated.
    * The experimental setup described in Section 3.2 is difficult to follow. There are several comparison settings but are not well organized.
* Discussion on the differences among diffusion models, GANs, and VAEs in Section 2.2 are not very persuasive. It does not have concrete examples or quantitative arguments. Plus, the argument of Section 3.2 is mostly about generative modeling itself and not about its application to generating auxiliary data in novelty detection.
* When performing CIFAR-10 vs CIFAR-100 experiment, it is more standard and more challenging to incorporate all the classes, instead of using only one class from each dataset. Overall, I am not convinced that the experiment settings used in the paper are standard.
* Replacing a GAN with a diffusion model in the data generator for novelty detection itself is not a very novel contribution to the community.

The evidence to be evaluated is:In the first step, a fake dataset of anomalies is generated by a SDE-based diffusion model. We quantitatively and qualitatively show that the generated fake outliers are high-quality, diverse, and yet show semantic differences compared to the normal inputs.

In order to generate near anomalous samples, a basic diffusion model, called Stochastic Differential Equation (SDE) (43) is adopted.

A SDE model considers a continuum of distributions that develop over time in accordance with a diffusion process, to corrupt a sample and turn it into pure noise.

The model is trained to learn how to gradually transform the noisy image back to its original form through the score fucntion (backward process). Running the backward process allows us to seamlessly transform a random noise into the data that could be used for sample generation as follows: \[x_{n-1}-x_{n}=[f(x,t)-g^{2}(t)s_{\theta}(x,t)]dt+g(t)d\hat{w},\quad x_{T}\sim N (0,1), \tag{1}\] where \(x_{n}\) is the data sample at time \(n\), \(f=0\) and \(g=\sigma^{t}\) are called the drift and diffusion coefficients, respectively. \(s_{\theta}(x,t)\) is called the score function, as is expected to estimate \(\nabla_{x}\log p_{t}(x)\).

The main benefit of using a score-based generative model is that in the backward process (Eq. 1), the score function denoises the input gradually that results in a relatively smooth decline in the Frechet Inception Distance (FID) (17) across training epochs. This enables us to stably produce near anomalous samples based on the FID score of the generated outputs, which could be achieved by stopping the training process earlier than the stage where the model achieves its maximum performance.

Note that a totally different training trajectory is obtained in GANs, where the FID oscillates and premature training does not necessarily produce high-quality anomalous samples.

In addition, Fig. 3 right, shows that the quality and closeness of the samples that are generated by the diffusion model are visually more appealing that the SOTA GAN-based models.

Having generated a high-quality fake dataset, a lightweight projection head is added to the feature extractor, and the whole network is trained to solve a binary classification task between the given normal and generated abnormal inputs. This way, normal class boundaries are tightened and adjusted according to the abnormal inputs.

Note that due to the feature fine-tuning phase of our approach, the borders of the normal samples become well-structured, thus better representing the normal semantic features.

Here, we provide some rationale on the preference of diffusion models over GANs for the the task of near-ND.

We note that in diffusion models, such as DDPM, the model is trained by optimizing the Evidence Lower Bound (ELBO) [(18)]. That is, the objective function is to minimize an upper bound on the negative evidence: \[\mathbb{E}_{x\sim q}[-\log p_{\theta}(x)], \tag{4}\] where \(q\) and \(p_{\theta}\) are the original and estimated distribution of the data, respectively. Here, \(\theta\) stands for the denoiser parameters in the DDPM. But, we note that maximizing the evidence is equivalent with minimizing the Kullback-Leibler (KL) divergence between \(q\) and \(p_{\theta}\):

Therefore, if the gap in the ELBO bound is small, minimizing the DDPM loss is equivalent with making the estimated generative distribution \(p_{\theta}\) close to the original data distribution \(q\). Hence, we expect the estimated generative distribution \(p_{\theta}\) ends up _sufficiently_ close to \(q\) if the training is prematurely stopped, and therefore generates "near-distribution" patterns.

However, such metrics exhibit a steady and consistent increase till the estimated distribution \(p_{\theta}\) gets sufficiently close to the true distribution \(q\) (see Fig. 3 left).

We proposed to take a non-adaptive generative modeling that allows controllable generation of anomalous data. It turns out that SDEs fit this specification much better compared to other SOTA generative models such as GANs. The generated data levels up pre-trained distance-based novelty detection methods not only based on their AUROCs in the standard ND setting but also in the near-ND setting.

Algan: Anomaly detection by generating pseudo anomalous data via latent variables.

For the data generation phase, we use the SDE-based model explained in (43) with exactly the same setup default hyper-parameters e.g. we used predictor-corrector as a sampler.

The density and coverage metrics can be used to assess the diversity and fidelity of generative models, respectively.

Because in our setup, only one class (the normal class) is available for the training of the generative model. As the sample size becomes limited in such a setting, model convergence and generation of high quality real samples become a challenge. Even under such circumstances, the SDE-based models converge. Furthermore, such models can eventually generate high quality real images.

As opposed to other generative methods, the proposed premature training of the SDE yields authentic, near-distribution, diverse, and artifact-free images. This makes it a better choice for the near-ND setup.

Input: Figure 11: Generated anomaly images on the Caltech-UCSD Birds 200 dataset. The images highlighted in green are normal samples.

Input: Figure 12: Generated anomaly images on the MVTecAD dataset. The images highlighted in green are normal samples.

Input: Figure 11: Generated anomaly images on the Caltech-UCSD Birds 200 dataset. The images highlighted in green are normal samples.

Input: Figure 12: Generated anomaly images on the MVTecAD dataset. The images highlighted in green are normal samples.

Input: Figure 1: Spectrum of AUCs, based on Stopping points for WBC dataset and CIFAR10-vs100 setting, as proxies for low- and high-resolution data, respectively.

Input: Figure 1: Sensitivity of state-of-the-art anomaly detectors when differences between normal and anomalous samples are subtle. For each model, its performance in detecting far (Blue) and near (Orange) outliers has been reported. A single class of CIFAR-10 is considered as normal, the rest of the classes are considered far anomalies, and the semantically nearest class of CIFAR-100 to the normal class is the source of near anomalies. This is the class that corresponds to the lowest test AUROC in anomaly detection. A considerable performance drop happens for the most recent methods, which means the learned normal boundaries are not tight enough and need to be adapted. For instance, while some might be able to distinguish a cat image from a car, they are still unable to detect tiger or lion images as anomalies.

Input: Figure 10: Generated anomaly images on the StanfordCars dataset. The images highlighted in green are normal samples.

Input: Figure 4: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 9: Generated anomaly images on 102 category of the Flowers dataset. The images highlighted in green are normal samples.

Input: Figure 2: The performance of novelty detection methods (AUROC %) in the near-ND and ND settings. In the near-ND setting, results are reported according to both criteria, i.e. CLP and bottom-1.

Input: Figure 2: Overview of our framework for near distribution novelty detection. In the first step, the SDE diffusion model generates semantically close outliers, after its training is early-stopped. As it is shown, they have very subtle yet semantic differences with the normal data distribution (downleft). After that, using a linear layer, a pre-trained feature extractor is fine-tuned to solve a binary classification between the normal and abnormal inputs. This modifies the normal boundaries toward including more distinctive features. Finally, normal embeddings are stored and used to compute the k-NN distance at the test time as the novelty score.

Input: Figure 5: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 10: Generated anomaly images on the StanfordCars dataset. The images highlighted in green are normal samples.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
I'll evaluate the claim: "To address the mentioned challenges, we propose to use a 'non-adversarial' diffusion-based anomaly data generation method, which can estimate the true normal distribution accurately and smoothly over the training time."

Summary: The evidence strongly supports this claim through:
- Demonstrated smooth FID score decline during training (evidenced in text)
- Visual examples showing high-quality generated samples across multiple datasets (Figures 11, 12, 9, 10)
- Mathematical formulation showing how the model estimates distributions through ELBO optimization
- Clear comparison to GAN alternatives showing smoother training trajectory
- Reviewer comments note this as an incremental but valid technical contribution

The claim is well-stated and accurately reflects the capabilities demonstrated in the evidence without overstating them.

<score>1</score>


Input:
Your role is to assess whether a claim is well-stated or overstated based on the available evidence.

The Input Information will include:
1. Original Claim: The claim under evaluation.
2. Evidence: Research findings, including figures, tables, or other relevant data supporting the claim.

Optional. review comment: Reviewer feedback relevant to the claim's validity.

The claim should be assesed with the following:
Score: Evaluate the claim on a scale of 1 to 5, where:
- 1: Well-stated (accurately reflects the evidence without exaggeration)
- 5: Overstated (exaggerates findings' significance, magnitude, or certainty)

Justification: Provide a detailed explanation that includes:
- Instances of exaggerated wording, insufficient experiments, lack of experimental details, gaps in knowledge, weak grounding in evidence, or lack of acknowledgment of limitations.
- Direct citations from relevant evidence that substantiate your reasoning.
- If review comment is included only take into account the relevant comments towards the claim.

The claim to be assessed is:

To address the mentioned challenges, we propose to use a "non-adversarial" diffusion-based anomaly data generation method, which can estimate the true normal distribution accurately and smoothly over the training time.

The review comment to be evaluated is:

The idea of this paper is very natural: try to generate near-distribution outliers to augment the original training dataset which only contains normal data (semi-supervised learning), such that we can fine-tune the pre-trained feature extractor by training a binary classifier.

Strengths:
(1) Using generated outliers to improve the accuracy of novelty detection is not new. There have been some similar works which use GANs instead of SDE.  The main contribution of this paper is that they replace the GANs with the diffusion model (which is increasingly popular recently).  The motivation here is that the outliers generated by SDE are “closer“ to the nominal distribution, and then using these "boarder" points  to deal with the near-ND setting.  Even though the pipeline and SDE are all from existing works, the novel combination is meaningful considering the theoretically and experimental results.

(2) The experimental results show that the proposed method considerably improves over existing models (especially for the near-ND setting).  It is not surprising that SDE over-perform GANs for the near-ND setting, but I am surprised that the improvement is so obvious.  It will be great if the code can be provided.

Weakness:
(1) In Section 2.2 (Diffusion models vs. GANs), they try to explain, for near-ND, why choosing diffusion models instead of other generative modes (GAN and VAE), it would be better to include flow-based models in this part. Also,  VAE-related results are not included in figure3.

(2) In Figure3,  density, coverage and FID are used as the metrics to measure the fidelity and diversity.  Could you add a brief introductions/definitions for these metrics? Also, I am confused how to get the "density" from GAN-based models as they are not tractable models.

(3)About Stopping point: In section 4, "the method is robust against the stopping point with at most 2% variation for the reasonable FID scores". However,  from table4,  it is difficult to get this conclusion. what is "reasonable" FID? Actually, the stopping point decide how similar the fake points and the real points. How to decide when to stop ?

Other minor comments:
The "normal distribution" used in this paper sometimes are confusing, which means the distribution of nominal data instead of Gaussian distribution. It would be better to use other notations.

The evidence to be evaluated is:In the first step, a fake dataset of anomalies is generated by a SDE-based diffusion model. We quantitatively and qualitatively show that the generated fake outliers are high-quality, diverse, and yet show semantic differences compared to the normal inputs.

In order to generate near anomalous samples, a basic diffusion model, called Stochastic Differential Equation (SDE) (43) is adopted.

A SDE model considers a continuum of distributions that develop over time in accordance with a diffusion process, to corrupt a sample and turn it into pure noise.

The model is trained to learn how to gradually transform the noisy image back to its original form through the score fucntion (backward process). Running the backward process allows us to seamlessly transform a random noise into the data that could be used for sample generation as follows: \[x_{n-1}-x_{n}=[f(x,t)-g^{2}(t)s_{\theta}(x,t)]dt+g(t)d\hat{w},\quad x_{T}\sim N (0,1), \tag{1}\] where \(x_{n}\) is the data sample at time \(n\), \(f=0\) and \(g=\sigma^{t}\) are called the drift and diffusion coefficients, respectively. \(s_{\theta}(x,t)\) is called the score function, as is expected to estimate \(\nabla_{x}\log p_{t}(x)\).

The main benefit of using a score-based generative model is that in the backward process (Eq. 1), the score function denoises the input gradually that results in a relatively smooth decline in the Frechet Inception Distance (FID) (17) across training epochs. This enables us to stably produce near anomalous samples based on the FID score of the generated outputs, which could be achieved by stopping the training process earlier than the stage where the model achieves its maximum performance.

Note that a totally different training trajectory is obtained in GANs, where the FID oscillates and premature training does not necessarily produce high-quality anomalous samples.

In addition, Fig. 3 right, shows that the quality and closeness of the samples that are generated by the diffusion model are visually more appealing that the SOTA GAN-based models.

Having generated a high-quality fake dataset, a lightweight projection head is added to the feature extractor, and the whole network is trained to solve a binary classification task between the given normal and generated abnormal inputs. This way, normal class boundaries are tightened and adjusted according to the abnormal inputs.

Note that due to the feature fine-tuning phase of our approach, the borders of the normal samples become well-structured, thus better representing the normal semantic features.

Here, we provide some rationale on the preference of diffusion models over GANs for the the task of near-ND.

We note that in diffusion models, such as DDPM, the model is trained by optimizing the Evidence Lower Bound (ELBO) [(18)]. That is, the objective function is to minimize an upper bound on the negative evidence: \[\mathbb{E}_{x\sim q}[-\log p_{\theta}(x)], \tag{4}\] where \(q\) and \(p_{\theta}\) are the original and estimated distribution of the data, respectively. Here, \(\theta\) stands for the denoiser parameters in the DDPM. But, we note that maximizing the evidence is equivalent with minimizing the Kullback-Leibler (KL) divergence between \(q\) and \(p_{\theta}\):

Therefore, if the gap in the ELBO bound is small, minimizing the DDPM loss is equivalent with making the estimated generative distribution \(p_{\theta}\) close to the original data distribution \(q\). Hence, we expect the estimated generative distribution \(p_{\theta}\) ends up _sufficiently_ close to \(q\) if the training is prematurely stopped, and therefore generates "near-distribution" patterns.

However, such metrics exhibit a steady and consistent increase till the estimated distribution \(p_{\theta}\) gets sufficiently close to the true distribution \(q\) (see Fig. 3 left).

We proposed to take a non-adaptive generative modeling that allows controllable generation of anomalous data. It turns out that SDEs fit this specification much better compared to other SOTA generative models such as GANs. The generated data levels up pre-trained distance-based novelty detection methods not only based on their AUROCs in the standard ND setting but also in the near-ND setting.

Algan: Anomaly detection by generating pseudo anomalous data via latent variables.

For the data generation phase, we use the SDE-based model explained in (43) with exactly the same setup default hyper-parameters e.g. we used predictor-corrector as a sampler.

The density and coverage metrics can be used to assess the diversity and fidelity of generative models, respectively.

Because in our setup, only one class (the normal class) is available for the training of the generative model. As the sample size becomes limited in such a setting, model convergence and generation of high quality real samples become a challenge. Even under such circumstances, the SDE-based models converge. Furthermore, such models can eventually generate high quality real images.

As opposed to other generative methods, the proposed premature training of the SDE yields authentic, near-distribution, diverse, and artifact-free images. This makes it a better choice for the near-ND setup.

Input: Figure 11: Generated anomaly images on the Caltech-UCSD Birds 200 dataset. The images highlighted in green are normal samples.

Input: Figure 12: Generated anomaly images on the MVTecAD dataset. The images highlighted in green are normal samples.

Input: Figure 11: Generated anomaly images on the Caltech-UCSD Birds 200 dataset. The images highlighted in green are normal samples.

Input: Figure 12: Generated anomaly images on the MVTecAD dataset. The images highlighted in green are normal samples.

Input: Figure 1: Spectrum of AUCs, based on Stopping points for WBC dataset and CIFAR10-vs100 setting, as proxies for low- and high-resolution data, respectively.

Input: Figure 1: Sensitivity of state-of-the-art anomaly detectors when differences between normal and anomalous samples are subtle. For each model, its performance in detecting far (Blue) and near (Orange) outliers has been reported. A single class of CIFAR-10 is considered as normal, the rest of the classes are considered far anomalies, and the semantically nearest class of CIFAR-100 to the normal class is the source of near anomalies. This is the class that corresponds to the lowest test AUROC in anomaly detection. A considerable performance drop happens for the most recent methods, which means the learned normal boundaries are not tight enough and need to be adapted. For instance, while some might be able to distinguish a cat image from a car, they are still unable to detect tiger or lion images as anomalies.

Input: Figure 10: Generated anomaly images on the StanfordCars dataset. The images highlighted in green are normal samples.

Input: Figure 4: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 9: Generated anomaly images on 102 category of the Flowers dataset. The images highlighted in green are normal samples.

Input: Figure 2: The performance of novelty detection methods (AUROC %) in the near-ND and ND settings. In the near-ND setting, results are reported according to both criteria, i.e. CLP and bottom-1.

Input: Figure 2: Overview of our framework for near distribution novelty detection. In the first step, the SDE diffusion model generates semantically close outliers, after its training is early-stopped. As it is shown, they have very subtle yet semantic differences with the normal data distribution (downleft). After that, using a linear layer, a pre-trained feature extractor is fine-tuned to solve a binary classification between the normal and abnormal inputs. This modifies the normal boundaries toward including more distinctive features. Finally, normal embeddings are stored and used to compute the k-NN distance at the test time as the novelty score.

Input: Figure 5: For each class of CIFAR-10, nearest class from CIFAR-100 has been provided, according to the closeness score. In each 3-row panel, the first row is assumed as normal, the second one is assumed the nearest abnormal class, and the third row indicates the corresponding generated fake images using the SDE model.

Input: Figure 10: Generated anomaly images on the StanfordCars dataset. The images highlighted in green are normal samples.

Input:
You should:
1. Carefully review the claim and the text and image evidence. Summarize how the evidence influenced your evaluation of the claim's validity and strength and briefly explain whether the claim is wellstated or overstated on a scale of 1 to 5 (up to 100 words).
2. Provide the final score (choose from 1 to 5), in the format: <score>{score}</score>.

Output:
Based on the extensive evidence provided, including theoretical foundations, experimental results, and visual examples, I find the claim to be well-stated and accurately reflects the capabilities of the proposed SDE-based diffusion model approach. The evidence shows:

1. Clear visual demonstrations through multiple datasets (MVTecAD, Birds, Cars, Flowers) showing controlled generation of near-distribution anomalies
2. Theoretical grounding explaining why diffusion models are suitable (ELBO optimization, smooth FID progression)
3. Quantitative validation of performance improvements over GANs
4. Reviewer feedback confirms the novelty and effectiveness of using diffusion models instead of GANs for this purpose

The claim avoids exaggeration by specifically focusing on the distribution estimation capabilities and smoothness of the approach, which is well-supported by both theoretical and empirical evidence.

<score>1</score>