README.md

---
language:
  - en
tags:
  - text-to-image
  - stable-diffusion
pipeline_tag: text-to-image

---

# ColorfulXL-Lightning

![promo](promo.png)


## Model Details

v1.6. Based on ColorfulXL, with lightning ability. 

 - Fine-tuned on aesthetics
 - Unet trained from 512 to 1280 with 64 steps
 - Good prompt follow, trained text encoder
 - Custom VAE
 - Merged with 2,4,8, steps lightning unets from bytedance (supermario)
 - Merged back with base model (not SGMUniform Euler works)
 - Use with zero/low cfg
 - Custom VAE
 - Ability to generate pure white/black - colorful model with true colors

High range of resolutions supported (576 - 1280), 576*832 example:

![steps](5.png)

Colorful:

![00341-1974381181.png](00341-1974381181.png)

There are problems with hands and faces (But who cares? Just say it's art!):

![problems](6.jpg)

Detailed review (version 1.0), thx to [Stevie2k8](https://civitai.com/user/Stevie2k8)!

![Model_lightning_colorfulxlLightning_v10_result_1_00005_](Model_lightning_colorfulxlLightning_v10_result_1_00005_.png)
![Model_lightning_colorfulxlLightning_v10_result_2_00005_](Model_lightning_colorfulxlLightning_v10_result_2_00005_.png)
![Model_lightning_colorfulxlLightning_v10_result_3_00005_](Model_lightning_colorfulxlLightning_v10_result_3_00005_.png)

## Usage

```python

from diffusers import DiffusionPipeline
from diffusers import EulerDiscreteScheduler
import torch


pipeline = DiffusionPipeline.from_pretrained("recoilme/ColorfulXL-Lightning", torch_dtype=torch.float16,variant="fp16", use_safetensors=True).to("cuda")
pipeline.scheduler = EulerDiscreteScheduler.from_config(pipeline.scheduler.config, timestep_spacing="trailing")

prompt = "girl sitting on a small hill looking at night sky, fflix_dmatter, back view, distant exploding moon, nights darkness, intricate circuits and sensors, photographic realism style, detailed textures, peacefulness, mysterious."
height = 1024
width = 1024
steps = 3
scale = 0
seed = 2139965163
generator = torch.Generator(device="cpu").manual_seed(seed)

image = pipeline(
            prompt = prompt,
            height=height,
            width=width,
            guidance_scale=scale,
            num_inference_steps=steps,
            generator=generator,
        ).images[0]
image.show()
image.save("girl.png")
```

## Local inference

[https://github.com/recoilme/100lineSDXL](https://github.com/recoilme/100lineSDXL)

## Space (upgrade on GPU)

[https://huggingface.co/spaces/recoilme/ColorfulXL-Lightning](https://huggingface.co/spaces/recoilme/ColorfulXL-Lightning)

## Model Details

* **Developed by**: [AiArtLab](https://aiartlab.org/)
* **Model type**: Diffusion-based text-to-image generative model
* **Model Description**: This model is a fine-tuned model based on [colorfulxl](https://civitai.com/models/185258/colorfulxl).
* **License**: This model is not permitted to be used behind API services. Please contact [email protected] for business inquires, commercial licensing, custom models, and consultation.

## Uses

### Direct Use

 
Research: possible research areas/tasks include:

- Generation of artworks and use in design and other artistic processes.
- Applications in educational or creative tools.
- Research on generative models.
- Safe deployment of models which have the potential to generate harmful content.
- Probing and understanding the limitations and biases of generative models.

Excluded uses are described below.

### Out-of-Scope Use

The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.

## Limitations and Bias

### Limitations

- The model does not achieve perfect photorealism
- The model cannot render legible text
- The model struggles with more difficult tasks which involve compositionality, such as rendering an image corresponding to “A red cube on top of a blue sphere”
- Faces and people in general may not be generated properly.
- The autoencoding part of the model is lossy.

### Bias
While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases.


## Contact
* For questions and comments about the model, please join [https://aiartlab.org/](https://aiartlab.org/).
* For future announcements / information about AiArtLab AI models, research, and events, please follow [Discord](https://discord.com/invite/gsvhQEfKQ5).
* For business and partnership inquiries, please contact https://t.me/recoilme