# Divide and Conquer: High-Resolution Industrial Anomaly Detection via Memory
Efficient Tiled Ensemble

Blaž Rolih1 Samet Akçay2 Dick Ameln2 Ashwin Vaidya2 1University of Ljubljana,
Faculty of Computer and Information Science
2Intel
<EMAIL_ADDRESS>{samet.akcay, dick.ameln<EMAIL_ADDRESS>
###### Abstract

Industrial anomaly detection is an important task within computer vision with
a wide range of practical use cases. The small size of anomalous regions in
many real-world datasets necessitates processing the images at a high
resolution. This frequently poses significant challenges concerning memory
consumption during the model training and inference stages, leaving some
existing methods impractical for widespread adoption. To overcome this
challenge, we present the tiled ensemble approach, which reduces memory
consumption by dividing the input images into a grid of tiles and training a
dedicated model for each tile location. The tiled ensemble is compatible with
any existing anomaly detection model without the need for any modification of
the underlying architecture.111Available on GitHub:
https://github.com/openvinotoolkit/anomalib By introducing overlapping tiles,
we utilize the benefits of traditional stacking ensembles, leading to further
improvements in anomaly detection capabilities beyond high resolution alone.
We perform a comprehensive analysis using diverse underlying architectures,
including Padim, PatchCore, FastFlow, and Reverse Distillation, on two
standard anomaly detection datasets: MVTec and VisA. Our method demonstrates a
notable improvement across setups while remaining within GPU memory
constraints, consuming only as much GPU memory as a single model needs to
process a single tile.222Research conducted during Google Summer of Code 2023
at OpenVINO (Intel)

##  1 Introduction

The detection and localization of anomalies in images is a crucial task with a
wide range of industrial applications. The ability to identify hard-to-detect
defects of various sizes within images enables automation of many processes,
maintenance of safety, and prevention of financial loss.

Figure 1: Example of anomaly localization on VisA capsules category. A tiled
ensemble successfully manages to detect an anomaly, while a single model with
a smaller resolution or equivalent resolution fails to do so. The tiled
ensemble achieves this while remaining within the GPU memory constraints.

In recent years, the field has witnessed substantial performance improvements,
primarily driven by advancements in deep learning techniques. In addition, the
high real-world potential of anomaly detection techniques has prompted a shift
of focus towards efficiency, as latency, throughput, and memory consumption
are important metrics to optimize when deploying models on resource-limited
devices [24, 16, 40, 4, 22, 26].

A challenge of real-world datasets is that the size of the anomalous regions
within the images may be very small relative to the full image size. The
common practice of downscaling the input images to a predetermined input size
may in such cases lead to a loss of pixel-level information, which in turn
causes the model to miss small anomalies and incorrectly mark images as
defect-free. Processing the images at their original resolution or reducing
the amount of downscaling may constitute a natural tactic to prevent this type
of false negative, but will at the same time inflate the memory consumption of
the model. As a result, memory constraints may prevent processing the images
in a resolution suitable for detecting the smallest anomalies in the dataset,
especially in low-resource settings.

Tiling mechanisms, which subdivide the input images into a rectangular grid of
tiles as a pre-processing step, have been used to process images at a high
resolution while keeping memory use low [25]. By passing individual tiles to
the model as input instead of full images, tiling reduces the model’s input
dimensions, while maintaining the effective input resolution of the images
content-wise. However, this approach may not always be ideal, particularly for
methods sensitive to object alignment [23], as using a single model for all
patches may compromise spatial information preservation.

Contrary to traditional tiling, our tiled ensemble trains a separate model on
each of the individual tile locations. The full training procedure yields an
ensemble of independently trained models, each specialized on a single
specific tile location. By assigning a separate model to each tile location,
we achieve a direct spatial mapping of feature space to pixel space, making
the method suitable for spatially-aware models. An additional advantage of
using separate models is that it allows us to leverage the benefits of
stacking ensemble methods by introducing overlapping tiles, which further
improves anomaly detection performance. By merging the predictions of the
individual models as a post-processing step, we obtain an end-to-end pipeline,
offering direct application to practical settings while ensuring that the peak
GPU memory usage remains in the range of that required by a model processing a
single tile. Since our approach only changes the pre- and post-processing
stages, it is not limited to a specific model architecture and can be applied
as an extension to any anomaly detection pipeline. Figure 1 illustrates how a
tiled ensemble can detect small anomalies by utilizing increased resolution
without consuming excessive GPU memory.

To showcase the applicability of our tiled ensemble, we benchmark the approach
against non-tiling baselines, as well as traditional tiling methods, using a
diverse set of architectures such as probability density modelling (Padim
[11]), memory bank based (Patchcore [29]), student-teacher (Reverse
Distillation [12]), and normalizing flows (Fastflow [40]). For evaluation, we
use two well-known anomaly detection datasets: MVTec AD [6] and VisA [46],
with an emphasis on detecting smaller anomalies, particularly evident in the
VisA dataset.

Overall, this paper provides the following contributions:

  * •

We propose a practical approach for the detection and localization of
anomalies in high-resolution images while adhering to GPU memory constraints.
This enables the detection of small anomalies in real-world applications,
increasing reliability and performance.

  * •

Our approach offers a model-agnostic framework that can enhance both existing
and upcoming anomaly detection architectures. By adopting our methodology,
these architectures can better handle higher-resolution images, additionally
benefiting in constrained settings without the need for modification of the
underlying architecture.

  * •

Having a dedicated model for each tile location enables the model to highly
specialize in specific part of an image. Additionally, the integration of
overlapping tiles in our approach has the advantages of conventional stacking
ensembles. This results in enhanced performance that surpasses what can be
achieved solely through increased resolution.

##  2 Related work

In recent times, there have been notable advancements in the field of visual
anomaly detection, with numerous techniques being introduced based on various
approaches such as reconstructive methods [43, 8, 5], student-teacher networks
[36, 31, 4, 12, 7], discriminative methods [38, 45, 42, 44, 23, 14],
normalizing flows [40, 30, 16], and embedding-based methods [11, 29, 24].

Apart from the design of novel model architectures, an active area of focus
has been enhancing and extending existing approaches. Recent research has
indicated that the performance of anomaly detection models can benefit from
careful design choices around data augmentation[37], pre-training
characteristics[17], and feature space selection [19]. Other studies have
focused on modifying or extending the architecture of existing models. Ristea
et al. [28] introduced the SSPCAB block, which can be injected into various
state-of-the-art methods to enhance their performance. Similarly, e Silva et
al. [13] demonstrated that significant improvements can be achieved by
extending existing architectures with attention blocks. Finally, Heckler. and
König. [18] developed a feature selection method that optimally selects a
layer from the pre-trained feature extractor depending on the characteristics
of the task. The tiled ensemble method follows a similar strategy, altering
the pre- and post-processing stages of existing anomaly detection pipelines
with the aim of improving the performance on datasets with small anomalies.

A common class of anomaly detection models is patch-based models. A patch
refers to a spatial location in the intermediate feature embedding map of the
model backbone and usually translates directly to a pixel area in the input
images. Patch-based models aim to find the natural distribution of each patch
location from the feature embeddings of normal images during training and
estimate the distance of the feature embeddings to this distribution during
inference. This process yields a set of patch-level anomaly scores which form
the basis of the anomaly localization predictions. To find the distribution of
a given patch location, a model may rely only on the embeddings of that same
patch location [11, 39, 33, 10], or alternatively also consider the
interrelation among patches [35, 29, 26]. Further, the authors of CutPaste
[23] discovered that employing separate models for each patch location yields
superior results. This insight forms the basis of assigning a dedicated model
for each tile location in our tiled ensemble method, which further extends
this into a generic extension for any anomaly detection architecture.

The reduction of memory use in anomaly detection is a common research topic
[4, 22, 16]. This is especially relevant for fields such as pathology [34],
where a high input resolution is needed to distinguish the anomalous
characteristics within the images [25]. Processing images at a higher
resolution prevents missing small anomalies but at the same time leads to
increased GPU memory usage. In light of these challenges, the tiled ensemble
extends the capabilities of various anomaly detection methods to enable
efficient processing of high-resolution images. This approach capitalizes on
findings from Heckler et al. [19], which explores how image resolution impacts
the performance of anomaly detection architectures.

The ensemble approach is often used to increase the performance of a base
model. Several individual models are combined, which results in a better
generalization performance [15], accuracy, stability, and reproducibility [9].
Ensembles are increasingly popular in visual anomaly detection and have shown
promising results [27, 20, 41, 32]. Bergmann et al. [7] used an ensemble of
students to mimic the teacher network. Recent anomaly detection methods employ
ensembles by keeping the architecture consistent while using a different
backbone for feature extraction [3, 29, 21]. In each of these studies ensemble
models consistently outperform single models, achieving state-of-the-art
results in anomaly detection.

##  3 Method

Figure 2: High-level tiled ensemble workflow: Images are first divided into
tiles, and separate models are trained for each tile location. Predictions are
generated individually for each tile, merged back together, and finally post-
processed. Note that tiles can also be overlapping, yet the training is
independent for each model.

The tiled ensemble method is structured as a series of sequential steps. The
method initially divides the high-resolution image into tiles (Section 3.1),
followed by training individual models for each tile location (Section 3.2).
Once predictions are obtained, a merging mechanism is utilized to produce
full-image-level data, followed by standard post-processing steps (Section
3.3).

A high-level overview of the workflow is presented in Figure 2. The approach
encapsulates all the training and inference steps in a pipeline, which enables
immediate application for industrial use cases that involve the analysis of
high-resolution images.

###  3.1 Tiling

To reduce the memory footprint of a high-resolution image, the first step is
to split the image into a set of tiles, which are then separately processed by
an individual model. For an image $X\in\mathbb{R}^{c\times h\times w}$, tile
size $h^{t}\times w^{t}$ and stride $s_{h},s_{w}$, this set of tiles
$\mathcal{T}$ is defined as

$\displaystyle\mathcal{T}=\\{$ $\displaystyle T_{i,j}\in\mathbb{R}^{c\times
h^{t}\times w^{t}}|$ (1) $\displaystyle
i\in[0,...,\left\lfloor\frac{h-h^{t}}{s_{h}}\right\rfloor],h^{t}\leq
h,s_{h}\leq h$ $\displaystyle
j\in[0,...,\left\lfloor\frac{w-w^{t}}{s_{w}}\right\rfloor],w^{t}\leq
w,s_{w}\leq w$ $\displaystyle h^{t},w^{t},s_{h},s_{w}\in\mathbb{N}\\}$

If the stride and tile size don’t precisely match the image, the image is
padded with zeros, which are later removed during untiling. Each tile spans
the following pixels of the original image:

$\displaystyle T_{i,j}=\\{(a,b)|$ $\displaystyle
a\in[s_{h}*i,...,s_{h}*i+h^{t}),$ (2) $\displaystyle
b\in[s_{w}*i,...,s_{w}*i+w^{t})\\}$

The pixels that tiles cover can also overlap in case of stride smaller than
tile size, i.e. $s_{h}<h^{t}$ or $s_{w}<w^{t}$.

For instance, consider an input image with dimensions $512\times 512$ (height
$h=512$ and width $w=512$), a tile size of $256\times 256$ (tile height
$h^{t}=256$ and tile width $w^{t}=256$) and a stride of $s_{h}=128$ and
$s_{w}=128$. This configuration yields 9 overlapping tiles, labeled $T_{0,0}$
through $T_{2,2}$. The overlapping area between tiles $T_{0,0}$ and $T_{0,1}$
consists of pixels ${(a,b)|a\in[0,...,256)\land b\in[128,...,256)}$, which
means it encompasses the right half of $T_{0,0}$ and the left half of
$T_{0,1}$.

Tiling the input image enables predicting images with higher resolution while
allowing models to be trained on smaller inputs. This approach significantly
reduces GPU memory consumption. Another benefit of tiling is that each model
is only responsible for the designated tile location. This localized
processing ensures that anomalies detected within a specific tile do not
influence the detection results in adjacent or distant tiles, which, overall,
prevents the trigger of unrelated spurious predictions across the image.

###  3.2 Training and inference

Training. By splitting the image into smaller tiles, the problem of high GPU
memory consumption is efficiently addressed. However, training a single model
on the combined set of all tile locations could lead to a loss of positional
information, with a potential negative effect on the performance of models
that perform better on aligned objects. The tiled ensemble approach addresses
this by employing a separate model for each tile location, where the
underlying model architecture of the individual models remains unchanged.

Formally, a separate model $M_{i,j}$ is trained for each tile location for the
entire set of tiles $\mathcal{T}$, defined in Equation 1, resulting in a set
of models:

$\displaystyle\mathcal{M}=\\{M_{i,j}|M_{i,j}\text{ trained on }T_{i,j}\\}$ (3)

The tiled ensemble method requires no further modifications to the training
process, which is similar to that of a single model. Since each tile location
is processed independently, it is possible to train the models in parallel
across different devices.

Inference. Once all the models for all locations are trained, the same tiling
procedure is followed and each tile is processed by the corresponding model in
inference. For tile $T^{test}_{i,j}$ in inference time and model $M_{i,j}$,
the pixel-level anomaly map $\mathcal{A}_{i,j}$ and anomaly score
$\mathrm{s}_{i,j}$ is obtained as:

$\displaystyle\mathcal{A}_{i,j},s_{i,j}=M_{i,j}(T^{test}_{i,j})$ (4)

In this case, the score $\mathrm{s}_{i,j}$ is obtained as specified by the
underlying architecture. This can either be achieved as a separate process or
by taking the maximum value from $\mathcal{A}_{i,j}$.

Due to the independence of predictions, the storage of each tile predictions
can also be efficiently managed. By moving the tile predictions to the main
memory, the GPU memory usage remains within the constraints.

###  3.3 Merging

A merging mechanism is utilized to produce a full-resolution anomaly map
$\mathcal{A}$ and the score $s$ from individual tile predictions
$\mathcal{A}_{i,j}$ and $s_{i,j}$ (Figure 3).

Figure 3: Overview of merging procedure and smoothing. Tile-level anomaly maps
are untiled into a full image, with pixel-wise averaging applied to
overlapping regions. Anomaly scores from all tiles are averaged, producing a
single image-level score. After the predictions are merged, the first step of
post-processing involves smoothing the region around the tile seams to enhance
the quality of the anomaly map.

Tile-level anomaly maps $\mathcal{A}_{i,j}$ are simply untiled back into a
full-image representation to get $\mathcal{A}$. As the tiles can also be
overlapping, a pixel-wise averaging strategy [1] is applied on overlapping
regions.

Different strategies can be used to tackle the merging of image-level scores
$s_{i,j}$. An image can be classified as anomalous as soon as one of the
patches is anomalous [29]. Alternatively, the score over all the tiles can be
averaged to obtain a single score, which is in our case the default option:

$\displaystyle s$ $\displaystyle=\frac{1}{N}\sum_{i,j}s_{i,j}$ (5)

The borders of the tiles create seams, leading to undesirable disturbances in
the image. To mitigate this issue and enhance the outcomes, a Gaussian filter
is applied for smoothing. This smoothing process is confined to a narrow
region surrounding the seam, as depicted in Figure 3. The default width of the
smoothing region is $10\%$ of the tile width on each side of the seam. Since
the output at this stage is the same as in a standard anomaly detection model,
it is fully compatible with any of the common post-processing steps, such as
normalization and thresholding.

##  4 Experiments

An in-depth analysis of the tiled ensemble across multiple configurations and
anomaly detection architectures is conducted. The following sections outline
the protocols and setups employed to assess the impact of the tiled ensemble.

###  4.1 Experimental details

Datasets. The method is evaluated on two established industrial datasets:
MVTec AD [6] and VisA [46]. MVTec AD and VisA datasets comprise 15 and 12
categories, respectively. Each category consists of a training set containing
only normal images and a test set, comprising both normal and anomalous
images, with their corresponding pixel-precise ground truth annotations. The
anomalies vary in types, shapes, and scales, with the prevalence of larger
anomalies in MVTec AD and smaller defects in VisA.

For both datasets, the images are of high resolution and are resized according
to the specified dimensions in the experimental setups, as detailed in the
following sections.

Evaluation Metrics Both image and pixel-level performance are evaluated using
standard anomaly detection metrics. For image-level anomaly detection, the
Area Under the Receiver Operator Curve (AUROC) is employed. To evaluate pixel-
wise performance in anomaly localization, the Area Under the Per-Region-
Overlap Curve (AUPRO) is used.

The exact protocol outlined by the authors of EfficientAD [4] is followed to
obtain latency, throughput, and inference GPU memory consumption. The only
exception is the usage of a batch size of 8 instead of 16 for PatchCore, due
to excessive GPU memory usage in the case of a single model with $512\times
512$ resolution. For a tiled ensemble, the benchmark inference step
encapsulates tiling, inference on all tiles, and untiling. Experiments were
conducted on a system with Intel(R) Xeon(R) Gold 5320 CPU and Nvidia Tesla
A100 GPU (training) and Nvidia Tesla V100S (inference).

###  4.2 Evaluation setups

To comprehensively evaluate and compare the performance of the tiled ensemble,
four architectures from diverse paradigms are employed. Padim [11] covers
probability density modelling, Patchcore [29] is a memory bank based approach,
Reverse Distillation [12] represents student-teacher architectures, and
Fastflow [40] normalizing flows. Each architecture is then trained in six
different setups, where two use a single model with varying resolution, two
employ tiled input to a single model and the remaining two utilize the tiled
ensemble.

Single model with 256px image size – SM256. A single model for each
architecture is trained with an input size of $256\times 256$ pixels, aligning
with the tile size of the ensemble models. While the effective final
resolution processed by this setup is smaller, this setup serves as a baseline
as this resolution is the most common in other works.

Single model with 512px image size – SM512. To explore the effect of
resolution without ensembling, a single model is trained with an input image
size of $512\times 512$ pixels. In this case, the model processes the same
effective resolution as our base tiled ensemble. However, it consumes a larger
amount of GPU memory. This setup allows for a comparison of memory usage and
the extent to which the benefits result from ensembling rather than increased
resolution.

Tiled ensemble with 9 overlapping 256px tiles – ENS9. The base tiled ensemble
setup has image resolution of $512\times 512$ pixels, which is then divided
into 9 overlapping $256\times 256$ tiles ($h^{t}=w^{t}=256$,
$s_{h}=s_{w}=128$). The final predicted anomaly map maintains the same
dimensions as the input image, i.e. $512\times 512$. This setup is utilized to
highlight the effects of ensembling properties in addition to the ability to
process high resolution while adhering to memory constraints.

Tiled ensemble with 4 non-overlapping 256px tiles – ENS4. In this setup, the
input image has a resolution of $512\times 512$ and is divided into four non-
overlapping $256\times 256$ tiles ($h^{t}=w^{t}=256$, $s_{h}=s_{w}=256$). The
dimension of the predicted anomaly map remains consistent with the input
image, i.e. $512\times 512$s. This setup is utilized to highlight the
efficient processing of high resolution, without the additional benefits of
multiple (overlapping) predictions as in ENS9.

Single model with 9 overlapping 256px tiles – ST9. This setup involves using a
single model trained on tiled input. The image resolution remains at
$512\times 512$ pixels, divided into nine overlapping $256\times 256$ tiles
($h^{t}=w^{t}=256$, $s_{h}=s_{w}=128$), and stacked batch-wise. A single model
in this case receives a $256\times 256$ tile for input. This setup is used to
compare the effect of having a separate model in a tiled ensemble specializing
solely in a single tile location.

Single model with 512px with 4 non-overlapping 256px tiles – ST4. Matching the
tiled ensemble setup without overlapping tiles, this setup explores how tiling
the input works in the case of utilizing a single model for all tile
locations. Here, a $512\times 512$ input image is split into four non-
overlapping $256\times 256$ tiles ($h^{t}=w^{t}=256$, $s_{h}=s_{w}=256$), and
stacked batch-wise. A single model is then trained on these tiles, with an
input size of $256\times 256$ pixels.

Common properties. Each setup is trained on every category, with every run
repeated 3 times using a different random seed. Center-crop is disabled for
all methods, and a consistent batch size of 32 is used, except for Patchcore
where it’s set to 8 due to memory limitations. The backbone used in all setups
is ResNet18, to keep the comparison fair. Following [4, 19] FastFlow, and
Reverse Distillation are limited to 200 steps for all setups. Other properties
are kept the same as provided by the original authors of the models and as
implemented in Anomalib [2].

##  5 Results

Results on MVTec AD. Table 1 reports anomaly detection and localization
results obtained on the MVTec AD dataset. A tiled ensemble with overlapping
tiles (ENS9) achieves the best results in terms of anomaly detection for Padim
and FastFlow and second best for PatchCore and Reverse Distillation. It also
achieves the best localization performance for PatchCore and FastFlow, and
second best for Padim.

Setup | PatchCore | Padim | FastFlow |  Reverse Distillation
---|---|---|---|---
SM256 | 97.7/92.8 |  89.2 / 91.2 |  93.1 /89.1 |  90.8 / 89.5
SM512 |  98.0 / 94.5 | 83.0/ 91.0 | 90.5/88.5 | 78.5/ 87.2
ENS4 | 96.5/94.1 | 87.3/90.5 | 91.8/ 89.5 | 84.5/82.6
ENS9 |  97.8 / 95.3 |  89.7 / 91.0 |  95.0 / 91.4 |  87.8 /82.6
ST4 |  97.8 /94.0 | 83.8/90.6 | 91.4/85.0 | 80.7/86.2
ST9 |  97.8 /94.3 | 83.3/90.6 | 90.1/82.8 | 77.3/ 89.5
Table 1: Results in anomaly detection and localization (AUROC/AUPRO) on MVTec
AD. Best and second best results are marked. A mean of 3 runs is reported for
each setup.

Results on VisA with all 6 setups are displayed in Table 2. A tiled ensemble
with overlapping tiles (ENS9) achieves the best results in terms of anomaly
detection for Padim, FastFlow, and Reverse Distillation, significantly
outperforming baseline single model (SM256) and single model processing the
same resolution of $512\times 512$ (SM512) in all three cases. It also
achieves the best results for anomaly localization for FastFlow and Reverse
Distillation.

Setup | PatchCore | Padim | FastFlow |  Reverse Distillation
---|---|---|---|---
SM256 | 92.0/87.7 | 83.7/82.1 | 87.4/81.4 | 83.2/86.9
SM512 |  97.2 / 94.0 | 81.9/86.9 |  89.8 / 88.4 | 71.5/89.3
ENS4 | 93.1/93.0 |  83.8 /86.9 | 89.4/86.8 |  88.3 / 89.6
ENS9 | 95.4/ 93.7 |  86.3 /86.9 |  92.5 / 89.2 |  91.4 / 90.0
ST4 | 95.2/93.6 | 81.8/ 87.2 | 87.1/83.4 | 72.5/89.3
ST9 |  96.7 / 93.7 | 82.3/ 87.1 | 85.6/78.6 | 80.6/88.9
Table 2: Results in anomaly detection and localization (AUROC/AUPRO) on VisA.
Best and second best results are marked. A mean of 3 runs is reported for each
setup. Figure 4: Effects of increasing resolution for Padim on VisA PCB3
category. The first column contains full and zoomed-in input images with
corresponding ground truth. The next columns depict full and zoomed-in anomaly
maps with their corresponding binary segmentation. The resolution of results
is written above each block. Notice how localization improves with higher
resolution. The tiled ensemble uses a setup with $h^{t}=w^{t}=256$,
$s_{h}=s_{w}=256$

The anomaly detection results of a tiled ensemble (ENS4 and ENS9) consistently
outperform a single model with tiled input (ST4 and ST9) in almost all setups,
except for PatchCore, and in terms of localization for Padim. This indicates
that having a separate model specialized in each tile location can lead to
better performance in high-resolution images. The tiled ensemble with
overlapping tiles (ENS9) in most setups outperforms a single model processing
the same resolution (SM512) as well as a tiled ensemble with non-overlapping
tiles (ENS4). This demonstrates the potential benefits of the stacking
ensemble mechanism.

In the tiled ensemble setup, the kNN search in PatchCore’s memory bank is
limited to embeddings from within the same tile location, whereas single-model
setups provide access to embeddings from the entire image. This may explain
why PatchCore tends to benefit from a single-model setup. In both MVTec AD and
VisA, Reverse Distillation and Padim struggle if the resolution is increased
without utilizing the tiled ensemble, showing subpar performance when
comparing SM512 to baseline SM256. In the case of MVTec AD, Reverse
Distillation still works best with the baseline model, indicating that for
some architectures and large anomalies, a tiled ensemble is not necessarily
needed.

More detailed results for each category and setup with included standard
deviation on MVTec AD and VisA are included in Appendix B.

GPU memory usage. Inference GPU memory and training GPU memory usage are
presented in Figure 5, respectively.

Figure 5: Inference and training GPU memory usage. The memory consumption of
the tiled ensemble remains within the range of a model processing an image
with the resolution of a single tile (Single model 256). This for some models
results in a notable reduction, particularly evident in models such as
Patchcore.

The tiled ensemble is unaffected by the number of tiles as long as the tiles
have the same resolution. During inference, memory consumption remains
comparable to that of a single model processing the resolution equivalent to a
tile (Single model 256), across all models.

Inference GPU memory consumption holds significant importance for end-
applications, but training memory consumption also poses a challenge with
larger image resolutions. The tiled ensemble maintains a manageable memory
footprint in both training and inference, roughly equating to the memory
consumption of a single model (Figure 5). Note that the relative memory
advantage of the tiled ensemble further increases for higher effective image
resolutions (provided that the tile size remains the same), as the memory
consumption of each individual model is only related to the tile size.

Effect of resolution on performance and GPU memory usage. A case study is
performed on the PCB3 category of the VisA dataset, which contains many small
defects that can benefit from increased resolution. The Padim model is used to
explore memory consumption and verify the effect of resolution on localization
performance. A tile size of 256 with stride 256 is used for ensemble setup
($h^{t}=w^{t}=256$, $s_{h}=s_{w}=256$). The results of localization
performance with respect to resolution are presented in Figure 6 with GPU
memory consumption also reported for each setup.

Figure 6: Localization results in terms of AUPRO on VisA PCB3 category for a
single model and tiled ensemble with different resolutions. The corresponding
memory consumption of each setup is shown on the right. The tiled ensemble
uses a setup with $h^{t}=w^{t}=256$, $s_{h}=s_{w}=256$

Increased resolution offers increased localization performance, most notably
showing an improvement in the initial increase from $256\times 256$ to
$512\times 512$, at which many small anomalies already become better
detectable. While the memory of a single model processing a larger resolution
steeply increases, the tiled ensemble maintains the same memory consumption
for all resolutions. Fig. 4 contains a qualitative example depicting the
localization of a small anomaly from this experiment.

Effect of input resolution on small anomaly detection. Figure 7 illustrates
how small anomaly detection may benefit from the higher input resolutions that
can be achieved by the tiled ensemble approach. Compared to the $256\times
256$ baseline, the tiled ensemble achieves a notable boost in both detection
and localization performance for datasets in which the average size of the
anomalous regions is small. As the average size of the anomalies increases,
the effect diminishes and the performance of both setups converges. By
increasing the effective input resolution, the tiled ensemble approach
facilitates the detection and localization of small anomalous regions that
would otherwise go unnoticed as a result of downscaling the input images.

Figure 7:  Effect of dataset size on anomaly detection and localization
performance of the tiled ensemble (ENS9) and single model with $256\times 256$
resolution (SM256). Each point represents a single dataset category from MVTec
AD or VisA. Trend lines and confidence intervals added for interpretability.
X-axis: average number of anomalous pixels per defective image relative to
image size. Y-axis: average performance of setup across all four model
architectures.

Latency and throughput. The latency and throughput are presented in Figure 8.

Figure 8: Latency and throughput measured for each setup on an Nvidia Tesla
V100S. While the latency considerably increases, throughput sees a relatively
smaller reduction for tiled ensemble configurations.

Primarily, the latency overhead stems from the time needed to transfer
individual models to GPU and back, which does not affect throughput as
significantly since the model’s time on GPU is better utilized. For some
models like Patchcore, the throughput of a 4-tiled ensemble exceeds that of a
single model processing an equivalent resolution. The preliminary studies
showed that the time needed for tiled ensemble inference on GPU still
outperforms the inference of a single model with increased resolution on CPU
in terms of latency by around 4 times, and in terms of throughput by around 80
times.

##  6 Conclusion

This paper introduces a tiled ensemble approach to effectively detect and
localize small anomalies in high-resolution images. This has been a challenge
due to the high GPU memory demands required by existing approaches. The method
works by dividing the image into smaller tiles and training a dedicated model
for each tile location. This strategy ensures that the GPU memory usage
remains comparable to that of a single model that processes an image the size
of one tile. By employing overlapping tiles, this approach also takes
advantage of the performance improvements associated with traditional stacking
ensemble methods, which further improve performance compared to those
achievable by simply increasing image resolution.

The tiled ensemble is designed to be easily integrated into current anomaly
detection architectures without necessitating any architectural changes, which
makes it a flexible and practical solution for small anomaly detection within
high-resolution imagery. In an extensive evaluation using various model
architectures and two established datasets, the method demonstrated notable
performance improvement compared to setups processing images at a lower
resolution or without employing a tiled ensemble, with a particularly
pronounced impact on datasets with small anomalies.

The results presented in this paper demonstrate the feasibility of applying
existing or next-generation anomaly detection models within high-resolution
imagery, which opens up new possibilities across various industries.

Limitations. Despite its promising statistical results, our approach has a
notable inference overhead, which can be partially mitigated through batched
inference. This can be a reasonable sacrifice in cases where the resolution is
very large, to enable detection with resolutions that previously were not
feasible. Heckler et al. [19] suggest and Heckler. and König. [18] verify that
in certain scenarios, strategically choosing a single layer can outperform an
ensemble of multiple layers and backbones. This insight could complement our
approach. It potentially opens avenues for future research where each separate
tile location uses the most suitable layer, further optimizing anomaly
detection in high-resolution images. The approach has additional limitations
due to the tiling of the input, potentially preventing accurate logical
anomaly detection as tiling prevents efficient capturing of the global
context.

## References

  * Adey et al. [2021] Philip A Adey, Samet Akçay, Magnus JR Bordewich, and Toby P Breckon.  Autoencoders Without Reconstruction for Textural Anomaly Detection.  In _2021 International Joint Conference on Neural Networks (IJCNN)_ , pages 1–8. IEEE, 2021.
  * Akcay et al. [2022] Samet Akcay, Dick Ameln, Ashwin Vaidya, Barath Lakshmanan, Nilesh Ahuja, and Utku Genc.  Anomalib: A Deep Learning Library for Anomaly Detection.  In _2022 IEEE International Conference on Image Processing (ICIP)_ , pages 1706–1710. IEEE, 2022.
  * Bae et al. [2023] Jaehyeok Bae, Jae-Han Lee, and Seyun Kim.  PNI: Industrial Anomaly Detection Using Position and Neighborhood Information.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 6373–6383, 2023.
  * Batzner et al. [2024] Kilian Batzner, Lars Heckler, and Rebecca König.  EfficientAD: Accurate Visual Anomaly Detection at Millisecond-Level Latencies.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pages 128–138, 2024.
  * Bergmann et al. [2018] Paul Bergmann, Sindy Löwe, Michael Fauser, David Sattlegger, and Carsten Steger.  Improving Unsupervised Defect Segmentation By Applying Structural Similarity to Autoencoders.  _ArXiv_ , abs/1807.02011, 2018.
  * Bergmann et al. [2019] Paul Bergmann, Michael Fauser, David Sattlegger, and Carsten Steger.  Mvtec AD–A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 9592–9600, 2019.
  * Bergmann et al. [2020] Paul Bergmann, Michael Fauser, David Sattlegger, and Carsten Steger.  Uninformed Students: Student-Teacher Anomaly Detection With Discriminative Latent Embeddings.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 4183–4192, 2020.
  * Bergmann et al. [2022] Paul Bergmann, Kilian Batzner, Michael Fauser, David Sattlegger, and Carsten Steger.  Beyond Dents and Scratches: Logical Constraints in Unsupervised Anomaly Detection and Localization.  _International Journal of Computer Vision_ , 130(4):947–969, 2022.
  * Cao et al. [2020] Yue Cao, Thomas Andrew Geddes, Jean Yee Hwa Yang, and Pengyi Yang.  Ensemble Deep Learning in Bioinformatics.  _Nature Machine Intelligence_ , 2(9):500–508, 2020.
  * Chen et al. [2022] Yuanhong Chen, Yu Tian, Guansong Pang, and Gustavo Carneiro.  Deep One-Class Classification via Interpolated Gaussian Descriptor.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , pages 383–392, 2022.
  * Defard et al. [2021] Thomas Defard, Aleksandr Setkov, Angelique Loesch, and Romaric Audigier.  Padim: A Patch Distribution Modeling Framework for Anomaly Detection and Localization.  In _International Conference on Pattern Recognition_ , pages 475–489. Springer, 2021.
  * Deng and Li [2022] Hanqiu Deng and Xingyu Li.  Anomaly Detection via Reverse Distillation From One-Class Embedding.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 9737–9746, 2022.
  * e Silva et al. [2024] André Luiz Vieira e Silva, Francisco Simões, Danny Kowerko, Tobias Schlosser, Felipe Battisti, and Veronica Teichrieb.  Attention Modules Improve Image-Level Anomaly Detection for Industrial Inspection: A Differnet Case Study.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pages 8246–8255, 2024.
  * Fučka et al. [2023] Matic Fučka, Vitjan Zavrtanik, and Danijel Skočaj.  TransFusion–A Transparency-Based Diffusion Model for Anomaly Detection.  _arXiv preprint arXiv:2311.09999_ , 2023.
  * Ganaie et al. [2022] Mudasir A Ganaie, Minghui Hu, AK Malik, M Tanveer, and PN Suganthan.  Ensemble Deep Learning: A Review.  _Engineering Applications of Artificial Intelligence_ , 115:105151, 2022.
  * Gudovskiy et al. [2022] Denis Gudovskiy, Shun Ishizaka, and Kazuki Kozuka.  Cflow-AD: Real-Time Unsupervised Anomaly Detection With Localization via Conditional Normalizing Flows.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pages 98–107, 2022.
  * He et al. [2024] Haitian He, Sarah Erfani, Mingming Gong, and Qiuhong Ke.  Learning Transferable Representations for Image Anomaly Localization Using Dense Pretraining.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pages 1113–1122, 2024.
  * Heckler. and König. [2024] Lars Heckler. and Rebecca König.  Feature Selection for Unsupervised Anomaly Detection and Localization Using Synthetic Defects.  In _Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP_ , pages 154–165. INSTICC, SciTePress, 2024.
  * Heckler et al. [2023] Lars Heckler, Rebecca König, and Paul Bergmann.  Exploring the Importance of Pretrained Feature Extractors for Unsupervised Anomaly Detection and Localization.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 2916–2925, 2023.
  * Hu et al. [2019] Jingtao Hu, En Zhu, Siqi Wang, Xinwang Liu, Xifeng Guo, and Jianping Yin.  An Efficient and Robust Unsupervised Anomaly Detection Method Using Ensemble Random Projection in Surveillance Videos.  _Sensors_ , 19(19):4145, 2019.
  * Hyun et al. [2024] Jeeho Hyun, Sangyun Kim, Giyoung Jeon, Seung Hwan Kim, Kyunghoon Bae, and Byung Jun Kang.  ReConPatch: Contrastive Patch Representation Learning for Industrial Anomaly Detection.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pages 2052–2061, 2024.
  * Lee et al. [2023] Teng-Yok Lee, Yusuke Nagai, and Akira Minezawa.  Memory-Efficient and Gpu-Oriented Visual Anomaly Detection With Incremental Dimension Reduction.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 2907–2915, 2023.
  * Li et al. [2021] Chun-Liang Li, Kihyuk Sohn, Jinsung Yoon, and Tomas Pfister.  Cutpaste: Self-Supervised Learning for Anomaly Detection and Localization.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 9664–9674, 2021.
  * Liu et al. [2023] Zhikang Liu, Yiming Zhou, Yuansheng Xu, and Zilei Wang.  Simplenet: A Simple Network for Image Anomaly Detection and Localization.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 20402–20411, 2023.
  * Nejat et al. [2024] Peyman Nejat, Areej Alsaafin, Ghazal Alabtah, Nneka I. Comfere, Aaron Mangold, Dennis Murphree, Patricija Zot, Saba Yasir, Joaquin J. Garcia, and Hamid R. Tizhoosh.  Creating An Atlas of Normal Tissue for Pruning Wsi Patching Through Anomaly Detection.  _Scientific Reports_ , 14(3932), 2024.
  * Park et al. [2022] Chaewon Park, MyeongAh Cho, Minhyeok Lee, and Sangyoun Lee.  Fastano: Fast Anomaly Detection via Spatio-Temporal Patch Transformation.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pages 2249–2259, 2022.
  * Rayana and Akoglu [2016] Shebuti Rayana and Leman Akoglu.  Less is More: Building Selective Anomaly Ensembles.  _ACM Trans. Knowl. Discov. Data_ , 10(4), 2016.
  * Ristea et al. [2022] Nicolae-Cătălin Ristea, Neelu Madan, Radu Tudor Ionescu, Kamal Nasrollahi, Fahad Shahbaz Khan, Thomas B Moeslund, and Mubarak Shah.  Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 13576–13586, 2022.
  * Roth et al. [2022] Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Schölkopf, Thomas Brox, and Peter Gehler.  Towards Total Recall in Industrial Anomaly Detection.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 14318–14328, 2022.
  * Rudolph et al. [2022] Marco Rudolph, Tom Wehrbein, Bodo Rosenhahn, and Bastian Wandt.  Fully Convolutional Cross-Scale-Flows for Image-Based Defect Detection.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pages 1088–1097, 2022.
  * Rudolph et al. [2023] Marco Rudolph, Tom Wehrbein, Bodo Rosenhahn, and Bastian Wandt.  Asymmetric Student-Teacher Networks for Industrial Anomaly Detection.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pages 2592–2602, 2023.
  * Singh et al. [2020] Kuldeep Singh, Shantanu Rajora, Dinesh Kumar Vishwakarma, Gaurav Tripathi, Sandeep Kumar, and Gurjit Singh Walia.  Crowd Anomaly Detection Using Aggregation of Ensembles of Fine-Tuned Convnets.  _Neurocomputing_ , 371:188–198, 2020.
  * Sohn et al. [2021] Kihyuk Sohn, Chun-Liang Li, Jinsung Yoon, Minho Jin, and Tomas Pfister.  Learning and Evaluating Representations for Deep One-Class Classification.  In _International Conference on Learning Representations_ , 2021.
  * Srinidhi et al. [2021] Chetan L Srinidhi, Ozan Ciga, and Anne L Martel.  Deep Neural Network Models for Computational Histopathology: A Survey.  _Medical Image Analysis_ , 67:101813, 2021.
  * Tsai et al. [2022] Chin-Chia Tsai, Tsung-Hsuan Wu, and Shang-Hong Lai.  Multi-Scale Patch-Based Representation Learning for Image Anomaly Detection and Segmentation.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pages 3992–4000, 2022.
  * Wang et al. [2021] Guodong Wang, Shumin Han, Errui Ding, and Di Huang.  Student-Teacher Feature Pyramid Matching for Anomaly Detection.  In _The British Machine Vision Conference (BMVC)_ , 2021.
  * Wang et al. [2023] Guodong Wang, Yunhong Wang, Jie Qin, Dongming Zhang, Xiuguo Bao, and Di Huang.  Unilaterally Aggregated Contrastive Learning With Hierarchical Augmentation for Anomaly Detection.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 6888–6897, 2023.
  * Yang et al. [2023] Minghui Yang, Peng Wu, and Hui Feng.  Memseg: A Semi-Supervised Method for Image Surface Defect Detection Using Differences and Commonalities.  _Engineering Applications of Artificial Intelligence_ , 119:105835, 2023.
  * Yi and Yoon [2020] Jihun Yi and Sungroh Yoon.  Patch Svdd: Patch-Level Svdd for Anomaly Detection and Segmentation.  In _Proceedings of the Asian conference on computer vision_ , 2020.
  * Yu et al. [2021] Jiawei Yu, Ye Zheng, Xiang Wang, Wei Li, Yushuang Wu, Rui Zhao, and Liwei Wu.  Fastflow: Unsupervised Anomaly Detection and Localization via 2D Normalizing Flows.  _arXiv preprint arXiv:2111.07677_ , 2021.
  * Zahid et al. [2020] Yumna Zahid, Muhammad Atif Tahir, Nouman M Durrani, and Ahmed Bouridane.  IBaggedFCNet: An Ensemble Framework for Anomaly Detection in Surveillance Videos.  _IEEE Access_ , 8:220620–220630, 2020.
  * Zavrtanik et al. [2021a] Vitjan Zavrtanik, Matej Kristan, and Danijel Skočaj.  Draem-a Discriminatively Trained Reconstruction Embedding for Surface Anomaly Detection.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 8330–8339, 2021a.
  * Zavrtanik et al. [2021b] Vitjan Zavrtanik, Matej Kristan, and Danijel Skočaj.  Reconstruction By Inpainting for Visual Anomaly Detection.  _Pattern Recognition_ , 112:107706, 2021b.
  * Zavrtanik et al. [2022] Vitjan Zavrtanik, Matej Kristan, and Danijel Skočaj.  DSR–A Dual Subspace Re-Projection Network for Surface Anomaly Detection.  In _European conference on computer vision_ , pages 539–554. Springer, 2022.
  * Zhang et al. [2023] Xuan Zhang, Shiyu Li, Xi Li, Ping Huang, Jiulong Shan, and Ting Chen.  Destseg: Segmentation Guided Denoising Student-Teacher for Anomaly Detection.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 3914–3923, 2023.
  * Zou et al. [2022] Yang Zou, Jongheon Jeong, Latha Pemula, Dongqing Zhang, and Onkar Dabeer.  SPot-the-Difference Self-Supervised Pre-Training for Anomaly Detection and Segmentation.  In _European Conference on Computer Vision_ , pages 392–408. Springer, 2022.

Supplementary Material

##  Appendix A Anomaly scales per category

Figure 9 shows the average anomalous pixel ratio (what percentage of defective
image is covered by the defect) present in test set of each category from
MVTecAD [6] and VisA [46]. The ratio is calculated on all defective images
from the category with a resolution of $512\times 512$. VisA categories
contain notably smaller defect, especially in categories such as candle,
macaroni 1, and macaroni 2, where the anomalous pixels on average cover less
than $1\%$ of image.

Figure 9: Average anomalous pixel ratio of defective images per category for
all categories present in VisA and MVTec AD.

##  Appendix B Results of all categories for each setup

This section contains tables with category specific results for all
architectures and their setups.

|  Single model 256 |  Single model 512 |  Tiled ensemble 4 tiles |  Tiled ensemble 9 tiles |  Single model 4 tiles |  Single model 9 tiles
---|---|---|---|---|---|---
Carpet | 97.4/94.3 | 96.5/97.0 | 95.2/96.0 |  97.6/97.0 |  98.1/96.2 | 95.7/96.0
Grid | 96.3/88.8 |  99.9/97.1 | 96.4/96.1 | 96.9/96.7 |  100.0/96.8 |  100.0/96.8
Leather |  100.0/97.3 | 99.4/98.8 | 98.0/98.4 |  98.8/98.9 | 99.7/98.4 | 99.1/98.6
Tile |  99.5/83.9 | 97.8/88.0 | 94.9/86.1 |  96.3/88.0 | 97.1/86.0 | 96.4/86.5
Wood | 99.2/89.1 | 99.1/94.2 | 99.5/93.7 |  99.7/94.3 | 99.1/93.9 | 99.0/94.0
Bottle |  100.0/93.3 |  100.0/96.3 |  100.0/96.0 |  100.0/96.4 |  100.0/96.0 |  100.0/96.1
Cable |  98.0/93.9 | 95.6/93.4 | 95.0/93.5 |  96.9/94.6 | 94.2/92.7 | 96.3/93.0
Capsule | 98.7/93.1 | 99.1/96.9 | 99.5/96.2 |  99.8/96.9 | 99.3/96.4 |  99.7/97.0
Hazelnut | 100.0/95.0 |  100.0/97.0 | 99.7/96.3 | 99.8/97.0 |  100.0/96.2 |  100.0/96.7
Metal nut | 99.8/94.2 |  100.0/96.1 | 97.4/95.1 | 98.7/95.7 | 99.7/95.4 | 99.9/95.5
Pill |  94.0/93.7 |  92.3/96.5 | 91.0/95.7 | 92.8/96.4 | 90.2/95.8 | 93.5/96.5
Screw | 93.7/96.6 |  98.3/97.8 | 84.6/97.8 |  92.1/98.2 | 97.0/97.8 | 98.1/98.1
Toothbrush | 94.4/90.9 |  96.2/96.0 | 98.9/95.8 |  99.7/95.9 | 96.7/95.6 | 95.1/95.7
Transistor |  98.0/92.6 | 97.2/74.6 | 97.2/77.6 |  98.4/86.1 | 97.4/75.1 | 96.8/76.6
Zipper | 95.9/95.1 |  98.6/97.8 |  99.5/96.9 | 99.0/97.6 | 98.7/97.1 | 97.0/97.0
Average |  97.7/92.8 ($\pm 0.06$ / $\pm 0.04$) |  98.0/94.5 ($\pm 0.08$ / $\pm 0.01$) |  96.5/94.1 ($\pm 0.03$ / $\pm 0.04$) |  97.8/95.3 ($\pm 0.04$ / $\pm 0.01$) |  97.8/94.0 ($\pm 0.03$ / $\pm 0.01$) |  97.8/94.3 ($\pm 0.37$ / $\pm 0.05$)
Table 3: MVTec AD results of all 6 setups for Patchcore. The row contains results for a particular category, with columns containing detection and localization results (AUROC/AUPRO) for each setup. The mean of 3 runs is reported with the corresponding standard deviation in parentheses. The best result for each category is in bold. |  Single model 256 |  Single model 512 |  Tiled ensemble 4 tiles |  Tiled ensemble 9 tiles |  Single model 4 tiles |  Single model 9 tiles
---|---|---|---|---|---|---
Carpet |  96.7/95.3 | 92.3/94.0 | 89.2/93.4 | 93.7/94.2 | 92.4/93.5 | 91.6/93.3
Grid | 86.4/79.5 |  93.5/87.2 | 87.6/86.5 |  89.3/87.5 | 92.8/86.6 | 92.7/86.9
Leather |  98.0/93.6 | 96.1/97.9 | 95.0/98.0 |  96.6/98.2 | 96.4/98.0 | 96.0/97.9
Tile |  94.8/82.0 | 84.4/73.2 | 89.8/73.2 | 90.2/73.8 | 84.5/73.2 | 84.4/72.5
Wood |  98.1/92.6 | 94.8/94.1 | 96.4/94.0 |  97.1/94.3 | 94.3/94.0 | 94.8/94.1
Bottle | 99.5/95.2 | 98.8/95.4 | 99.1/95.9 |  99.9/95.7 |  98.7/95.9 | 98.8/95.5
Cable |  82.3/89.2 | 74.1/81.3 | 77.0/82.4 | 81.7/82.8 | 76.6/82.4 | 74.8/82.2
Capsule | 84.3/92.9 | 81.5/93.9 | 88.2/93.8 |  88.8/93.9 | 81.2/93.9 | 82.0/93.8
Hazelnut | 80.0/94.3 | 69.4/95.8 | 87.6/95.8 |  94.9/95.9 | 71.9/95.8 |  70.1/95.9
Metal nut |  96.4/92.2 | 93.4/89.9 | 90.6/90.0 | 93.8/90.5 | 93.8/90.0 | 93.2/90.0
Pill |  86.8/94.4 | 72.9/93.4 | 81.6/93.3 | 81.4/93.6 | 75.1/93.3 | 73.7/94.4
Screw |  74.2/91.7 |  61.3/92.3 | 66.2/92.0 | 69.4/92.2 | 58.7/92.0 | 61.1/92.2
Toothbrush | 86.9/93.3 | 88.1/95.7 | 94.7/95.8 |  99.4/95.8 |  89.8/95.8 | 88.1/95.8
Transistor |  89.8/89.3 | 78.2/87.5 | 83.8/81.5 | 87.6/83.2 | 83.5/81.5 | 81.0/82.4
Zipper |  83.1/93.0 | 66.1/92.7 | 82.9/92.7 | 81.2/93.0 | 67.3/92.7 | 66.9/92.5
Average |  89.2/91.2 ($\pm 1.29$ / $\pm 0.62$) |  83.0/91.0 ($\pm 1.28$ / $\pm 0.36$) |  87.3/90.5 ($\pm 0.76$ / $\pm 0.41$) |  89.7/91.0 ($\pm 0.89$ / $\pm 0.42$) |  83.8/90.6 ($\pm 1.07$ / $\pm 0.41$) |  83.3/90.6 ($\pm 1.12$ / $\pm 0.43$)
Table 4: MVTec AD results of all 6 setups for Padim. The row contains results for a particular category, with columns containing detection and localization results (AUROC/AUPRO) for each setup. The mean of 3 runs is reported with the corresponding standard deviation in parentheses. The best result for each category is in bold. |  Single model 256 |  Single model 512 |  Tiled ensemble 4 tiles |  Tiled ensemble 9 tiles |  Single model 4 tiles |  Single model 9 tiles
---|---|---|---|---|---|---
Carpet |  99.0/93.8 | 94.8/87.7 | 70.9/90.2 | 82.0/91.9 | 95.5/87.3 | 96.3/88.3
Grid | 98.8/93.2 | 99.5/96.9 | 96.8/95.5 |  97.1/96.9 |  99.9/95.8 | 99.2/96.2
Leather |  100.0/108.3 | 99.6/99.1 | 94.5/98.8 | 98.2/99.2 | 99.5/98.8 | 99.6/98.4
Tile |  96.1/81.3 | 90.9/77.1 | 93.6/76.8 | 92.5/80.8 | 90.5/76.8 | 90.8/73.8
Wood |  98.5/92.5 | 97.8/94.0 | 96.9/94.4 |  98.4/95.3 | 97.9/93.8 | 98.0/93.4
Bottle |  100.0/91.1 | 99.8/92.1 | 98.5/92.0 |  99.9/92.6 | 99.9/88.5 | 99.9/90.4
Cable |  93.9/86.5 | 78.3/74.7 | 90.6/84.2 |  93.2/86.6 | 81.2/67.0 | 78.0/57.7
Capsule | 92.6/89.4 | 93.5/95.2 | 93.4/95.3 |  97.6/95.8 | 89.6/94.4 | 87.7/92.1
Hazelnut | 78.2/92.8 | 80.1/93.9 | 92.0/94.1 |  97.7/94.2 | 88.5/92.9 | 83.9/91.2
Metal nut |  96.4/84.7 | 91.2/81.6 | 91.6/87.9 |  95.7/89.2 | 94.6/83.8 | 89.6/77.6
Pill | 93.1/90.1 |  93.1/91.8 | 95.1/91.5 |  97.5/90.7 | 91.9/87.4 | 90.6/83.2
Screw | 74.7/69.0 | 65.7/84.8 | 75.6/82.8 |  81.2/88.6 | 71.4/70.2 | 74.0/74.1
Toothbrush | 88.1/83.5 | 88.1/90.8 | 97.8/90.2 |  99.7/92.1 | 87.5/86.1 | 84.2/83.7
Transistor |  92.1/87.9 | 88.5/73.2 | 92.3/76.5 |  96.7/83.1 | 88.0/62.9 | 85.3/51.0
Zipper | 94.9/92.4 | 96.3/94.0 | 97.3/91.9 |  97.5/94.5 | 94.5/90.0 | 94.4/90.5
Average |  93.1/89.1 ($\pm 0.29$ / $\pm 1.08$) |  90.5/88.5 ($\pm 0.13$ / $\pm 0.35$) |  91.8/89.5 ($\pm 0.31$ / $\pm 0.35$) |  95.0/91.4 ($\pm 0.38$ / $\pm 0.27$) |  91.4/85.0 ($\pm 0.35$ / $\pm 0.27$) |  90.1/82.8 ($\pm 0.36$ / $\pm 0.46$)
Table 5: MVTec AD results of all 6 setups for FastFlow. The row contains results for a particular category, with columns containing detection and localization results (AUROC/AUPRO) for each setup. The mean of 3 runs is reported with the corresponding standard deviation in parentheses. The best result for each category is in bold. |  Single model 256 |  Single model 512 |  Tiled ensemble 4 tiles |  Tiled ensemble 9 tiles |  Single model 4 tiles |  Single model 9 tiles
---|---|---|---|---|---|---
Carpet | 96.2/96.1 |  97.2/96.6 | 84.1/85.6 | 82.5/68.6 |  97.7/96.4 | 86.0/95.1
Grid | 84.6/64.4 | 96.1/96.0 | 82.8/90.5 | 90.9/93.0 |  99.4/98.1 | 99.0/98.1
Leather |  83.6/94.7 | 76.9/92.3 | 49.1/73.5 | 60.3/75.4 | 57.7/67.6 |  44.3/95.7
Tile |  87.1/78.7 | 75.0/64.0 |  89.4/45.9 | 86.8/52.5 | 88.1/67.6 | 47.8/75.5
Wood |  98.6/91.0 | 87.6/90.9 |  97.3/92.9 | 98.1/88.1 | 89.4/88.5 | 53.8/89.9
Bottle |  99.8/95.1 | 68.8/90.9 | 99.2/93.2 | 98.6/92.0 | 88.4/91.4 | 89.5/91.2
Cable |  97.8/91.6 | 59.4/69.8 | 71.1/69.2 | 82.6/77.6 | 67.0/64.3 | 74.1/78.2
Capsule | 81.5/90.6 | 71.3/93.7 | 87.2/93.3 |  89.7/93.8 | 61.0/93.2 | 74.7/92.8
Hazelnut | 86.1/95.2 | 81.6/80.7 | 94.7/73.7 |  98.4/72.6 |  94.3/96.7 | 69.6/96.2
Metal nut |  100.0/94.3 | 90.2/88.2 | 92.2/91.3 | 96.8/92.8 | 89.3/87.9 | 82.4/87.3
Pill | 90.1/94.9 | 59.5/95.2 | 87.7/96.9 |  91.6/97.4 | 66.5/95.9 | 89.6/96.8
Screw | 75.4/92.0 | 64.6/91.6 | 58.8/74.9 | 69.6/76.0 | 72.9/93.1 |  84.3/94.2
Toothbrush | 97.0/92.4 |  97.8/96.6 | 97.0/95.9 |  99.9/96.6 | 89.0/95.5 | 98.1/96.1
Transistor |  95.3/77.1 | 73.1/62.0 | 89.7/66.1 | 91.2/69.0 | 69.3/61.6 | 80.5/60.9
Zipper |  88.6/95.0 | 78.6/94.2 |  88.0/95.7 | 79.5/93.9 | 79.9/95.2 | 85.4/95.5
Average |  90.8/89.5 ($\pm 3.01$ / $\pm 3.38$) |  78.5/87.2 ($\pm 3.25$ / $\pm 1.85$) |  84.5/82.6 ($\pm 2.51$ / $\pm 4.99$) |  87.8/82.6 ($\pm 1.54$ / $\pm 3.03$) |  80.7/86.2 ($\pm 1.76$ / $\pm 0.66$) |  77.3/89.5 ($\pm 0.92$ / $\pm 1.04$)
Table 6: MVTec AD results of all 6 setups for Reverse Distillation. The row contains results for a particular category, with columns containing detection and localization results (AUROC/AUPRO) for each setup. The mean of 3 runs is reported with the corresponding standard deviation in parentheses. The best result for each category is in bold. |  Single model 256 |  Single model 512 |  Tiled ensemble 4 tiles |  Tiled ensemble 9 tiles |  Single model 4 tiles |  Single model 9 tiles
---|---|---|---|---|---|---
Candle | 97.0/95.1 | 98.2/96.2 | 98.7/97.4 |  99.3/97.3 | 98.3/97.5 |  98.5/97.6
Capsules | 71.5/70.0 |  91.9/97.2 | 70.7/94.2 | 78.1/96.0 | 80.8/95.5 | 90.0/96.4
Cashew | 93.7/91.5 | 96.5/91.5 | 97.0/89.6 |  97.4/91.6 | 94.9/90.2 | 96.6/89.7
Chewing gum | 98.9/84.3 | 98.6/85.9 | 99.1/83.8 |  99.8/83.8 |  99.2/88.5 | 98.8/84.9
Fryum | 92.7/83.9 |  98.4/91.7 | 93.0/90.3 |  96.2/91.9 | 94.5/90.5 | 97.0/91.1
Macaroni 1 | 92.3/93.5 |  98.7/96.7 | 95.6/98.2 |  96.0/98.3 | 97.0/97.1 | 98.7/98.1
Macaroni 2 | 72.8/85.9 |  91.4/94.8 | 76.4/94.0 | 85.2/95.3 | 87.0/94.1 |  89.6/96.2
PCB1 | 94.9/92.8 | 97.9/96.5 | 98.2/95.7 |  98.8/96.5 | 98.0/95.2 | 97.9/96.0
PCB2 | 92.4/88.4 | 97.4/93.4 | 96.2/93.6 |  97.5/93.8 |  97.7/93.6 | 97.7/93.0
PCB3 |  99.0/86.6 | 98.4/94.4 | 94.7/93.7 |  97.5/94.6 | 97.5/93.9 | 97.9/94.2
PCB4 | 99.0/86.6 |  99.4/92.6 | 99.5/89.1 |  99.8/89.1 | 98.5/90.8 | 98.5/90.3
Pipe fryum | 99.3/94.1 |  99.6/96.9 | 98.5/96.1 | 99.1/96.8 | 99.6/96.1 | 99.4/96.5
Average |  92.0/87.7 ($\pm 0.29$ / $\pm 0.39$) |  97.2/94.0 ($\pm 0.47$ / $\pm 0.34$) |  93.1/93.0 ($\pm 0.07$ / $\pm 0.05$) |  95.4/93.7 ($\pm 0.81$ / $\pm 0.07$) |  95.2/93.6 ($\pm 0.91$ / $\pm 0.55$) |  96.7/93.7 ($\pm 0.09$ / $\pm 0.31$)
Table 7: VisA results of all 6 setups for Patchcore. The row contains results for a particular category, with columns containing detection and localization results (AUROC/AUPRO) for each setup. The mean of 3 runs is reported with the corresponding standard deviation in parentheses. The best result for each category is in bold. |  Single model 256 |  Single model 512 |  Tiled ensemble 4 tiles |  Tiled ensemble 9 tiles |  Single model 4 tiles |  Single model 9 tiles
---|---|---|---|---|---|---
Candle | 87.4/92.3 |  86.0/96.1 | 90.1/96.1 |  94.0/92.3 | 85.8/96.1 | 86.5/96.1
Capsules | 60.5/58.7 |  62.3/76.4 | 67.9/74.2 |  67.9/75.5 | 61.9/74.2 | 62.3/75.4
Cashew |  88.8/84.7 | 87.3/83.1 | 88.6/83.6 | 88.4/83.6 | 87.6/83.7 | 87.6/83.2
Chewing gum |  98.4/83.7 | 97.5/84.2 | 96.6/83.8 | 97.2/84.3 |  89.2/87.5 | 97.4/83.8
Fryum | 86.4/77.4 | 88.8/86.6 | 87.9/86.9 |  89.6/86.7 |  88.2/87.0 | 88.5/86.6
Macaroni 1 |  80.3/89.2 |  76.7/91.4 | 75.6/91.3 | 79.7/91.3 | 76.6/91.3 | 77.2/91.3
Macaroni 2 | 71.8/76.4 | 66.3/72.3 | 70.3/77.3 |  72.2/78.3 | 66.2/77.3 | 66.2/77.8
PCB1 | 88.6/88.5 | 84.7/91.9 | 87.5/91.9 |  89.9/92.1 | 84.9/91.9 |  80.3/93.2
PCB2 | 81.7/83.9 | 79.3/90.7 | 79.5/90.6 |  84.4/90.9 | 80.4/90.7 | 80.0/90.8
PCB3 | 72.5/80.5 | 73.8/89.6 | 75.5/89.5 |  80.7/89.7 | 74.3/89.4 |  74.1/89.8
PCB4 | 96.4/81.7 |  87.8/88.5 | 96.5/85.3 |  97.1/85.0 | 95.8/85.3 | 95.3/84.7
Pipe fryum | 92.3/88.3 | 92.1/92.4 | 89.8/92.2 |  94.6/92.5 | 91.2/92.2 | 91.7/92.2
Average |  83.7/82.1 ($\pm 0.90$ / $\pm 0.87$) |  81.9/86.9 ($\pm 1.76$ / $\pm 0.81$) |  83.8/86.9 ($\pm 0.96$ / $\pm 0.84$) |  86.3/86.9 ($\pm 0.96$ / $\pm 1.29$) |  81.8/87.2 ($\pm 1.96$ / $\pm 1.12$) |  82.3/87.1 ($\pm 0.58$ / $\pm 0.89$)
Table 8: VisA results of all 6 setups for Padim. The row contains results for a particular category, with columns containing detection and localization results (AUROC/AUPRO) for each setup. The mean of 3 runs is reported with the corresponding standard deviation in parentheses. The best result for each category is in bold. |  Single model 256 |  Single model 512 |  Tiled ensemble 4 tiles |  Tiled ensemble 9 tiles |  Single model 4 tiles |  Single model 9 tiles
---|---|---|---|---|---|---
Candle | 90.7/92.4 | 92.8/96.5 | 92.4/96.3 |  93.8/96.6 | 92.5/96.1 | 90.9/94.8
Capsules | 77.5/81.2 |  84.1/92.9 | 80.4/87.3 |  85.3/92.4 | 78.7/86.0 | 80.8/83.7
Cashew | 88.9/82.6 |  88.5/90.2 | 91.3/88.0 |  92.8/90.0 | 87.3/83.5 | 87.6/79.6
Chewing gum | 98.7/85.6 |  99.6/90.4 |  99.8/88.7 | 99.7/89.2 | 99.0/88.2 | 98.4/85.5
Fryum | 93.8/74.7 |  94.2/81.2 | 93.7/76.6 |  97.2/79.6 | 91.6/75.2 | 89.1/65.0
Macaroni 1 | 87.7/88.1 | 90.6/91.8 | 85.3/91.5 |  91.0/94.2 | 88.6/91.4 | 89.0/87.6
Macaroni 2 |  73.2/82.5 |  72.5/88.0 | 68.3/83.3 | 70.4/83.7 | 69.1/82.2 | 67.3/80.2
PCB1 | 85.5/86.4 | 88.5/90.0 | 91.1/88.5 |  95.0/93.5 | 83.7/83.8 | 81.9/74.3
PCB2 | 83.3/75.4 | 88.5/83.6 | 87.9/84.2 |  94.2/88.3 | 81.9/75.7 | 82.5/70.6
PCB3 | 79.1/65.2 | 86.8/85.2 | 89.2/86.0 |  93.8/88.1 | 83.8/78.2 | 80.0/72.2
PCB4 | 95.3/80.0 | 97.5/82.4 | 99.0/83.9 |  99.4/83.9 | 94.4/78.0 | 86.3/69.7
Pipe fryum | 95.3/83.0 | 94.3/88.0 | 94.5/87.6 |  97.9/90.2 | 94.9/82.3 | 93.3/80.2
Average |  87.4/81.4 ($\pm 0.42$ / $\pm 1.10$) |  89.8/88.4 ($\pm 0.27$ / $\pm 0.22$) |  89.4/86.8 ($\pm 0.31$ / $\pm 0.34$) |  92.5/89.2 ($\pm 0.12$ / $\pm 0.13$) |  87.1/83.4 ($\pm 0.32$ / $\pm 0.70$) |  85.6/78.6 ($\pm 0.18$ / $\pm 2.62$)
Table 9: VisA results of all 6 setups for FastFlow. The row contains results for a particular category, with columns containing detection and localization results (AUROC/AUPRO) for each setup. The mean of 3 runs is reported with the corresponding standard deviation in parentheses. The best result for each category is in bold. |  Single model 256 |  Single model 512 |  Tiled ensemble 4 tiles |  Tiled ensemble 9 tiles |  Single model 4 tiles |  Single model 9 tiles
---|---|---|---|---|---|---
Candle | 88.2/93.7 | 60.4/96.7 |  89.7/97.2 |  93.0/96.9 | 78.4/97.0 | 84.9/96.5
Capsules | 76.9/88.3 | 74.2/87.4 | 80.2/94.8 |  84.0/96.0 | 75.2/93.3 |  87.6/95.5
Cashew | 92.4/80.9 |  88.7/83.6 | 93.7/75.8 |  94.4/75.2 | 72.2/76.8 | 89.0/77.8
Chewing gum |  98.4/85.0 | 69.5/68.1 |  98.9/69.9 | 96.3/64.3 | 78.2/71.2 | 95.8/70.8
Fryum | 81.0/82.7 | 53.7/89.4 | 82.8/88.1 |  92.4/90.0 | 52.1/88.9 | 82.3/86.8
Macaroni 1 | 78.4/87.4 |  88.0/96.3 | 85.8/96.2 | 88.0/96.1 |  88.6/96.3 | 81.5/95.2
Macaroni 2 | 59.5/82.1 |  72.9/93.7 | 72.0/92.9 | 72.7/93.3 | 72.3/92.2 | 69.5/92.1
PCB1 | 64.7/94.0 | 59.3/93.5 | 91.1/94.1 |  93.3/95.4 | 62.8/93.1 | 62.2/92.4
PCB2 | 89.0/87.1 | 45.3/89.3 | 86.6/91.0 |  92.0/92.7 | 60.8/88.2 | 69.4/89.1
PCB3 | 81.3/85.9 | 85.2/90.9 | 85.8/91.2 |  94.6/93.3 | 75.0/89.2 | 82.6/90.4
PCB4 | 95.9/83.7 | 84.6/87.6 | 97.7/88.5 |  99.3/91.7 | 64.6/86.7 | 73.9/85.9
Pipe fryum | 93.1/92.2 |  76.3/95.6 | 95.7/95.2 |  96.7/95.0 | 90.4/94.7 | 88.3/94.7
Average |  83.2/86.9 ($\pm 1.93$ / $\pm 0.66$) |  71.5/89.3 ($\pm 5.96$ / $\pm 1.70$) |  88.3/89.6 ($\pm 1.54$ / $\pm 0.12$) |  91.4/90.0 ($\pm 1.32$ / $\pm 0.47$) |  72.5/89.3 ($\pm 4.29$ / $\pm 0.48$) |  80.6/88.9 ($\pm 2.59$ / $\pm 1.03$)
Table 10: VisA results of all 6 setups for Reverse Distillation. The row
contains results for a particular category, with columns containing detection
and localization results (AUROC/AUPRO) for each setup. The mean of 3 runs is
reported with the corresponding standard deviation in parentheses. The best
result for each category is in bold.

##  Appendix C Additional qualitative examples.

This section contains qualitative examples for every architecture and every
setup on all categories of both MVTec AD and VisA.

Figure 10: Anomaly maps and segmentation masks for each setup using PatchCore
on randomly picked defective image from every category in MVTec AD. Figure 11:
Anomaly maps and segmentation masks for each setup using Padim on randomly
picked defective image from every category in MVTec AD. Figure 12: Anomaly
maps and segmentation masks for each setup using FastFlow on randomly picked
defective image from every category in MVTec AD. Figure 13: Anomaly maps and
segmentation masks for each setup using Reverse Distillation on randomly
picked defective image from every category in MVTec AD. Figure 14: Anomaly
maps and segmentation masks for each setup using PatchCore on randomly picked
defective image from every category in VisA. Figure 15: Anomaly maps and
segmentation masks for each setup using Padim on randomly picked defective
image from every category in VisA. Figure 16: Anomaly maps and segmentation
masks for each setup using FastFlow on randomly picked defective image from
every category in VisA. Figure 17: Anomaly maps and segmentation masks for
each setup using Reverse Distillation on randomly picked defective image from
every category in VisA.