Add pipeline tag and library name

#1
by nielsr HF staff - opened
Files changed (1) hide show
  1. README.md +85 -0
README.md CHANGED
@@ -1,5 +1,7 @@
1
  ---
2
  license: apache-2.0
 
 
3
  ---
4
 
5
  # TRCE: Towards Reliable Malicious Concept Erasure in Text-to-Image Diffusion Models
@@ -17,3 +19,86 @@ license: apache-2.0
17
  </a>
18
 
19
  </div>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ library_name: diffusers
4
+ pipeline_tag: text-to-image
5
  ---
6
 
7
  # TRCE: Towards Reliable Malicious Concept Erasure in Text-to-Image Diffusion Models
 
19
  </a>
20
 
21
  </div>
22
+
23
+
24
+
25
+ ## Setup
26
+
27
+ The environment we conduct experiments are as follows:
28
+ + python: 3.10
29
+ + torch: 2.1.2
30
+ + CUDA Version: 12.4
31
+
32
+ Please run `pip install -r requirement.txt` to install dependency packages.
33
+
34
+ The erased model can be found :hugs:[here](https://huggingface.co/ddgoodgood/trce-erased-model/tree/main). Currently, our implementation is based only on SD1.4. We will release the implementation of TRCE on newer model in the future.
35
+
36
+ ## RUN
37
+
38
+ You can find the pre-cached COCO embeddings :hugs:[here](https://huggingface.co/ddgoodgood/trce-erased-model/tree/main). Please download the `cache` directory and place it in `data/cache`.
39
+
40
+ ### Run stage-1 TRCE
41
+
42
+ In the first stage, TRCE starts with a closed-form edit for the cross-attention layers, simply run:
43
+
44
+ ``` bash
45
+ # for erasing "sexual"
46
+ python run_trce_stage1.py config/stage1/stage1_sexual_default.yaml
47
+
48
+ # for erasing multiple malicious concepts
49
+ python run_trce_stage1.py config/stage1/stage1_unsafe_default.yaml
50
+ ```
51
+
52
+ You can modify the base model path and the output directory for the first-stage fine-tuned model in the configuration files.
53
+
54
+ ### Run stage-2 TRCE
55
+
56
+ Before the second stage, you need to prepare the denosing trajectory samples for the fine-tuning:
57
+
58
+ ```bash
59
+ python stage2_data_preparation.py
60
+ ```
61
+
62
+ This script generates samples for both "sexual" and "multi-concept" fine-tuning, as well as unconditional samples for the regularization loss.
63
+
64
+ Then, you can run the stage-2 using the following scripts:
65
+ ``` bash
66
+ # for erasing "sexual"
67
+ python run_trce_stage2.py config/stage2/stage2_sexual_default.yaml
68
+
69
+ # for erasing multiple malicious concepts
70
+ python run_trce_stage2.py config/stage2/stage2_unsafe_default.yaml
71
+ ```
72
+
73
+ ## Evaluation
74
+ The evaluation relies on the following repositories: [NudeNet](https://github.com/notAI-tech/NudeNet), [Q16 Detector](https://github.com/ml-research/Q16), [Pytorch FID](https://github.com/mseitzer/pytorch-fid), and [CLIP Score](https://github.com/Taited/clip-score). Please install these repositories according to their instructions before proceeding with the evaluation.
75
+
76
+ ### Generate image using erased model
77
+
78
+ Firstly, use the following scripts with the specified UNet path and output path to generate images for different evaluation tasks.
79
+ ```
80
+ # for evaluate "sexual" erasure
81
+ python gen_sexual.py <erased-model-dir> <output_path>
82
+
83
+ # for evaluate "multi concepts" erasure
84
+ python gen_unsafe.py <erased-model-dir> <output_path>
85
+
86
+ # for evaluate knowledge preservation on coco
87
+ python gen_coco.py <erased-model-dir> <output_path>
88
+ ```
89
+
90
+ Then, you can follow the instructions in `eval_nudenet_batch.ipynb`, `eval_unsafe.ipynb` and `eval_coco_batch.ipynb` to evaluate and review the performance of the erasure.
91
+
92
+ If you encounter any issues while using this repository, please feel free to leave messages in issues or contact me at [email protected]. I will respond as soon as possible.
93
+
94
+ ## Citation
95
+ ```
96
+ @article{chen2025reliable,
97
+ title={TRCE: Towards Reliable Malicious Concept Erasure in Text-to-Image Diffusion Models},
98
+ author={Ruidong, Chen and Honglin, Guo and Lanjun, Wang and Chenyu, Zhang and Weizh, Nie and An-An, Liu},
99
+ journal={arXiv preprint arXiv:2503.07389},
100
+ year={2025}
101
+ }
102
+ ```
103
+ ## Acknowledgement
104
+ We built this repository based on the excellent work of previous projects: [RECE](https://github.com/CharlesGong12/RECE/tree/main), [MACE](https://github.com/Shilin-LU/MACE), and [Safree](https://github.com/jaehong31/SAFREE). Thank you to all who contributed.