aka7774/animagine_xl_4.0

Overview
Animagine XL 4.0, also stylized as Anim4gine, is the ultimate anime-themed finetuned SDXL model and the latest installment of Animagine XL series. Despite being a continuation, the model was retrained from Stable Diffusion XL 1.0 with a massive dataset of 8.4M diverse anime-style images from various sources with the knowledge cut-off of January 7th 2025 and finetuned for approximately 2650 GPU hours. Similar to the previous version, this model was trained using tag ordering method for the identity and style training.

Model Details
Developed by: Cagliostro Research Lab

Model type: Diffusion-based text-to-image generative model

License: CreativeML Open RAIL++-M

Model Description: This is a model that can be used to generate and modify specifically anime-themed images based on text prompt

Fine-tuned from: Stable Diffusion XL 1.0

Downstream Use
Use this model in our Hugging Face Spaces

Use it in ComfyUI or Stable Diffusion Webui

Use it with Cagliostro Colab Forge

Use it with 🧨 diffusers

🧨 Diffusers Installation
1. Install Required Libraries
pip install diffusers transformers accelerate safetensors --upgrade
2. Example Code
The example below uses lpw_stable_diffusion_xl pipeline which enables better handling of long, weighted and detailed prompts. The model is already uploaded in FP16 format, so there's no need to specify variant="fp16" in the from_pretrained call.

import torch
from diffusers import StableDiffusionXLPipeline

pipe = StableDiffusionXLPipeline.from_pretrained(
    "cagliostrolab/animagine-xl-4.0",
    torch_dtype=torch.float16,
    use_safetensors=True,
    custom_pipeline="lpw_stable_diffusion_xl",
    add_watermarker=False
)
pipe.to('cuda')

prompt = "1girl, arima kana, oshi no ko, hoshimachi suisei, hoshimachi suisei \(1st costume\), cosplay, looking at viewer, smile, outdoors, night, v, masterpiece, high score, great score, absurdres"
negative_prompt = "lowres, bad anatomy, bad hands, text, error, missing finger, extra digits, fewer digits, cropped, worst quality, low quality, low score, bad score, average score, signature, watermark, username, blurry"

image = pipe(
    prompt,
    negative_prompt=negative_prompt,
    width=832,
    height=1216,
    guidance_scale=6,
    num_inference_steps=25
).images[0]

image.save("./arima_kana.png")
Usage Guidelines
1. Prompt Structure
The model was trained with tag-based captions and the tag-ordering method. Use this structured template:

1girl/1boy/1other, character name, from which series, everything else in any order.
2. Quality Enhancement Tags
Add these tags at the start or end of your prompt:

masterpiece, high score, great score, absurdres
3. Recommended Negative Prompt
lowres, bad anatomy, bad hands, text, error, missing finger, extra digits, fewer digits, cropped, worst quality, low quality, low score, bad score, average score, signature, watermark, username, blurry
4. Optimal Settings
CFG Scale: 5-7 (6 Recommended)

Sampling Steps: 25-28 (25 Recommended)

Preferred Sampler: Euler Ancestral (Euler a)

5. Recommended Resolutions
OrientationDimensionsAspect RatioSquare1024 x 10241:1Landscape1152 x 8969:71216 x 8323:21344 x 7687:41536 x 64012:5Portrait896 x 11527:9832 x 12162:3768 x 13444:7640 x 15365:12

6. Final Prompt Structure Example
masterpiece, high score, great score, absurdres, 1girl, firefly \(honkai: star rail\), honkai \(series\), honkai: star rail, casual, solo, looking at viewer, outdoors, smile, reaching towards viewer, night
Special Tags
The model supports various special tags that can be used to control different aspects of the image generation process. These tags are carefully weighted and tested to provide consistent results across different prompts.

Quality Tags
Quality tags are fundamental controls that directly influence the overall image quality and detail level. Available quality tags:

masterpiece

best quality

low quality

worst quality

Sample image using "masterpiece, best quality" quality tags with negative prompt left empty.Sample image using "low quality, worst quality" quality tags with negative prompt left empty.

Score Tags

Score tags provide a more nuanced control over image quality compared to basic quality tags. They have a stronger impact on steering output quality in this model. Available score tags:

high score

great score

good score

average score

bad score

low score

Sample image using "high score, great score" score tags with negative prompt left empty.Sample image using "bad score, low score" score tags with negative prompt left empty.

Temporal Tags
Temporal tags allow you to influence the artistic style based on specific time periods or years. This can be useful for generating images with era-specific artistic characteristics. Supported year tags:

year 2005

year {n}

year 2025

Sample image of Hatsune Miku with "year 2007" temporal tag. Sample image of Hatsune Miku with "year 2023" temporal tag.

Rating Tags
Rating tags help control the content safety level of generated images. These tags should be used responsibly and in accordance with applicable laws and platform policies. Supported ratings:

safe

sensitive

nsfw

explicit

Training Information
The model was trained using state-of-the-art hardware and optimized hyperparameters to ensure the highest quality output. Below are the detailed technical specifications and parameters used during the training process:

ParameterValueHardware7 x H100 80GB SXM5Num Images8,401,464UNet Learning Rate2.5e-6Text Encoder Learning Rate1.25e-6SchedulerConstant With WarmupWarmup Steps5%Batch Size32Gradient Accumulation Steps2Training Resolution1024x1024OptimizerAdafactorInput Perturbation Noise0.1Debiased Estimation LossEnabledMixed Precisionfp16

Acknowledgement
This long-term project would not have been possible without the groundbreaking work, innovative contributions, and comprehensive documentation provided by Stability AI, Novel AI, and Waifu Diffusion Team. We are especially grateful for the kickstarter grant from Main that enabled us to progress beyond V2. For this iteration, we would like to express our sincere gratitude to everyone in the community for their continuous support, particularly:

Moescape AI: Our invaluable collaboration partner in model distribution and testing

Lesser Rabbit: For providing essential computing and research grants

Kohya SS: For developing the comprehensive open-source training framework

discus0434: For creating the industry-leading open-source Aesthetic Predictor 2.5

Early testers: For their dedication in providing critical feedback and thorough quality assurance

Contributors
We extend our heartfelt appreciation to our dedicated team members who have contributed significantly to this project, including but not limited to:

Model
KayfaHaarukku

Raelina

Linaqruf

Gradio
Damar Jati

Relations, finance, and quality assurance
Scipius

Asahina

Bell

BoboiAzumi

Data
Pomegranata

Kr1SsSzz

Fiqi

William Adams Soeherman

Limitations
Prompt Format: Limited to tag-based text prompts; natural language input may not be effective

Anatomy: May struggle with complex anatomical details, particularly hand poses and finger counting

Text Generation: Text rendering in images is currently not supported and not recommended

New Characters: Recent characters may have lower accuracy due to limited training data availability

Multiple Characters: Scenes with multiple characters may require careful prompt engineering

Resolution: Higher resolutions (e.g., 1536x1536) may show degradation as training used original SDXL resolution

Style Consistency: May require specific style tags as training focused more on identity preservation than style consistency

License
This model adopts the original CreativeML Open RAIL++-M License from Stability AI without any modifications or additional restrictions. The license terms remain exactly as specified in the original SDXL license, which includes:

✅ Permitted: Commercial use, modifications, distributions, private use

❌ Prohibited: Illegal activities, harmful content generation, discrimination, exploitation

⚠️ Requirements: Include license copy, state changes, preserve notices

📝 Warranty: Provided "AS IS" without warranties

Please refer to the original SDXL license for the complete and authoritative terms and conditions.