Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,90 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# CAMOUFLaGE: Controllable AnoniMizatiOn throUgh diFfusion-based image coLlection GEneration
|
2 |
+
|
3 |
+
Code ![Here](https://gitlab.com/grains2/camouflage)
|
4 |
+
|
5 |
+
Official implementations of ["Latent Diffusion Models for Attribute-Preserving Image Anonymization"](#latent-diffusion-models-for-attribute-preserving-image-anonymization)
|
6 |
+
and ["Harnessing Foundation Models for Image Anonymization"](#harnessing-foundation-models-for-image-anonymization).
|
7 |
+
|
8 |
+
## Latent Diffusion Models for Attribute-Preserving Image Anonymization
|
9 |
+
|
10 |
+
[[Paper]](https://arxiv.org/abs/2403.14790)
|
11 |
+
|
12 |
+
This paper presents, to the best of our knowledge, the first approach to image anonymization based on
|
13 |
+
Latent Diffusion Models (LDMs). Every element of a scene is maintained to convey the same meaning, yet
|
14 |
+
manipulated in a way that makes re-identification difficult. We propose two LDMs for this purpose:
|
15 |
+
|
16 |
+
- *CAMOUFLaGE-Base*
|
17 |
+
- *CAMOFULaGE-Light*
|
18 |
+
|
19 |
+
The former solution achieves superior performance on most metrics and benchmarks, while the latter cuts
|
20 |
+
the inference time in half at the cost of fine-tuning a lightweight module.
|
21 |
+
|
22 |
+
Compared to state-of-the-art, we anonymize complex scenes by introducing variations in the faces, bodies,
|
23 |
+
and background elements.
|
24 |
+
|
25 |
+
#### CAMOUFLaGE-Base
|
26 |
+
|
27 |
+
CAMOUFLaGE-Base exploits a combination of pre-trained ControlNets and introduces an anonymizazion guidance based on
|
28 |
+
the original image.
|
29 |
+
|
30 |
+
![Architecture_Base](images/camouflage-base.jpg)
|
31 |
+
|
32 |
+
More details on its usage can be found [here](CAMOUFLaGE-Base-v1-0).
|
33 |
+
|
34 |
+
#### CAMOUFLaGE-Light
|
35 |
+
|
36 |
+
CAMOUFLaGE-Light trains a lightweight IP-Adapter to encode key elements of the scene and facial attributes of each
|
37 |
+
person.
|
38 |
+
|
39 |
+
![Architecture_Light](images/camouflage-light.jpg)
|
40 |
+
|
41 |
+
More details on its usage can be found [here](CAMOUFLaGE_light).
|
42 |
+
|
43 |
+
## Harnessing Foundation Models for Image Anonymization
|
44 |
+
|
45 |
+
[[Paper]]()
|
46 |
+
|
47 |
+
We explore how foundation models can be leveraged to solve tasks, specifically focusing on anonymization,
|
48 |
+
without the requirement for training or fine-tuning. By bypassing traditional pipelines, we demonstrate the
|
49 |
+
efficiency and effectiveness of this approach in achieving anonymization objectives directly from the
|
50 |
+
foundation model’s inherent knowledge.
|
51 |
+
|
52 |
+
#### CAMOUFLaGE-Face
|
53 |
+
|
54 |
+
We examine how foundation models can generate anonymized images directly from textual descriptions. Two models
|
55 |
+
were employed for information extraction: FACER, used to identify the 40 CelebA-HQ attributes, and DeepFace,
|
56 |
+
used to determine ethnicity and age. Using this rich information, we craft captions to guide the generation process.
|
57 |
+
Classifier-free guidance was employed to push the image content in the direction of the positive prompt P and far
|
58 |
+
from the negative prompt ¬P.
|
59 |
+
|
60 |
+
![Architecture-Face](images/camouflage-face.jpg)
|
61 |
+
|
62 |
+
More details on its usage can be found [here](GEM2024).
|
63 |
+
|
64 |
+
## Citation
|
65 |
+
|
66 |
+
If you find CAMOUFLaGE-Base and/or CAMOUFLaGE-Light useful, please cite:
|
67 |
+
|
68 |
+
```
|
69 |
+
@misc{camouflage,
|
70 |
+
title={Latent Diffusion Models for Attribute-Preserving Image Anonymization},
|
71 |
+
author={Luca Piano and Pietro Basci and Fabrizio Lamberti and Lia Morra},
|
72 |
+
year={2024},
|
73 |
+
eprint={2403.14790},
|
74 |
+
archivePrefix={arXiv},
|
75 |
+
primaryClass={cs.CV}
|
76 |
+
}
|
77 |
+
```
|
78 |
+
|
79 |
+
If you find CAMOUFLaGE-Face useful, please cite:
|
80 |
+
|
81 |
+
```
|
82 |
+
@inproceedings{pianoGEM24,
|
83 |
+
title={Harnessing Foundation Models for Image Anonymization},
|
84 |
+
author={Piano, Luca and Basci, Pietro and Lamberti, Fabrizio and Morra, Lia},
|
85 |
+
booktitle={2024 IEEE CTSoc Gaming, Entertainment and Media},
|
86 |
+
year={2024},
|
87 |
+
organization={IEEE}
|
88 |
+
}
|
89 |
+
```
|
90 |
+
|