|
--- |
|
language: |
|
- en |
|
tags: |
|
- text-to-image |
|
- stable-diffusion |
|
pipeline_tag: text-to-image |
|
|
|
--- |
|
|
|
# ColorfulXL-Lightning |
|
|
|
 |
|
|
|
|
|
## Model Details |
|
|
|
ColorfulXL merged with lightning loras (2,8 steps) from bytedance, for fast inference (3-6 steps). |
|
|
|
High range of resolutions supported (576 - 1280), 576*832 example: |
|
|
|
 |
|
|
|
It also works at huge resolutions like 1248 * 1824 (with Kohya HRFix Integrated) |
|
 |
|
params: https://civitai.com/images/9415168) |
|
|
|
Due to training LORA's on the base version of SDXL, there are problems with hands and faces: |
|
|
|
 |
|
|
|
|
|
|
|
## Usage |
|
|
|
```python |
|
|
|
from diffusers import DiffusionPipeline |
|
from diffusers import EulerDiscreteScheduler |
|
import torch |
|
|
|
|
|
pipeline = DiffusionPipeline.from_pretrained("recoilme/ColorfulXL-Lightning", torch_dtype=torch.float16,variant="fp16", use_safetensors=True).to("cuda") |
|
pipeline.scheduler = EulerDiscreteScheduler.from_config(pipeline.scheduler.config, timestep_spacing="trailing") |
|
|
|
prompt = "girl sitting on a small hill looking at night sky, fflix_dmatter, back view, distant exploding moon, nights darkness, intricate circuits and sensors, photographic realism style, detailed textures, peacefulness, mysterious." |
|
height = 1024 |
|
width = 1024 |
|
steps = 3 |
|
scale = 0 |
|
seed = 2139965163 |
|
generator = torch.Generator(device="cpu").manual_seed(seed) |
|
|
|
image = pipeline( |
|
prompt = prompt, |
|
height=height, |
|
width=width, |
|
guidance_scale=scale, |
|
num_inference_steps=steps, |
|
generator=generator, |
|
).images[0] |
|
image.show() |
|
image.save("girl.png") |
|
``` |
|
|
|
## Model Details |
|
|
|
* **Developed by**: [AiArtLab](https://aiartlab.org/) |
|
* **Model type**: Diffusion-based text-to-image generative model |
|
* **Model Description**: This model is a fine-tuned model based on [colorfulxl](https://civitai.com/models/185258/colorfulxl). |
|
* **License**: This model is not permitted to be used behind API services. Please contact [email protected] for business inquires, commercial licensing, custom models, and consultation. |
|
|
|
## Uses |
|
|
|
### Direct Use |
|
|
|
|
|
Research: possible research areas/tasks include: |
|
|
|
- Generation of artworks and use in design and other artistic processes. |
|
- Applications in educational or creative tools. |
|
- Research on generative models. |
|
- Safe deployment of models which have the potential to generate harmful content. |
|
- Probing and understanding the limitations and biases of generative models. |
|
|
|
Excluded uses are described below. |
|
|
|
### Out-of-Scope Use |
|
|
|
The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model. |
|
|
|
## Limitations and Bias |
|
|
|
### Limitations |
|
|
|
- The model does not achieve perfect photorealism |
|
- The model cannot render legible text |
|
- The model struggles with more difficult tasks which involve compositionality, such as rendering an image corresponding to “A red cube on top of a blue sphere” |
|
- Faces and people in general may not be generated properly. |
|
- The autoencoding part of the model is lossy. |
|
|
|
### Bias |
|
While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases. |
|
|
|
|
|
## Contact |
|
* For questions and comments about the model, please join [https://aiartlab.org/](https://aiartlab.org/). |
|
* For future announcements / information about AiArtLab AI models, research, and events, please follow [Discord](https://discord.com/invite/gsvhQEfKQ5). |
|
* For business and partnership inquiries, please contact https://t.me/recoilme |
|
|
|
|
