metadata

language:
  - en
tags:
  - text-to-image
  - stable-diffusion
pipeline_tag: text-to-image

ColorfulXL-Lightning

Model Details

ColorfulXL merged with lightning loras (2,8 steps) from bytedance, for fast inference (3-6 steps).

High range of resolutions supported (576 - 1280), 576*832 example:

It also works at huge resolutions like 1248 * 1824 (with Kohya HRFix Integrated) params: https://civitai.com/images/9415168)

Due to training LORA's on the base version of SDXL, there are problems with hands and faces:

Detailed review, thx to Stevie2k8!

Usage


from diffusers import DiffusionPipeline
from diffusers import EulerDiscreteScheduler
import torch


pipeline = DiffusionPipeline.from_pretrained("recoilme/ColorfulXL-Lightning", torch_dtype=torch.float16,variant="fp16", use_safetensors=True).to("cuda")
pipeline.scheduler = EulerDiscreteScheduler.from_config(pipeline.scheduler.config, timestep_spacing="trailing")

prompt = "girl sitting on a small hill looking at night sky, fflix_dmatter, back view, distant exploding moon, nights darkness, intricate circuits and sensors, photographic realism style, detailed textures, peacefulness, mysterious."
height = 1024
width = 1024
steps = 3
scale = 0
seed = 2139965163
generator = torch.Generator(device="cpu").manual_seed(seed)

image = pipeline(
            prompt = prompt,
            height=height,
            width=width,
            guidance_scale=scale,
            num_inference_steps=steps,
            generator=generator,
        ).images[0]
image.show()
image.save("girl.png")

Local inference

https://github.com/recoilme/100lineSDXL

Space (upgrade on GPU)

https://huggingface.co/spaces/recoilme/ColorfulXL-Lightning

Model Details

Developed by: AiArtLab
Model type: Diffusion-based text-to-image generative model
Model Description: This model is a fine-tuned model based on colorfulxl.
License: This model is not permitted to be used behind API services. Please contact [email protected] for business inquires, commercial licensing, custom models, and consultation.

Uses

Direct Use

Research: possible research areas/tasks include:

Generation of artworks and use in design and other artistic processes.
Applications in educational or creative tools.
Research on generative models.
Safe deployment of models which have the potential to generate harmful content.
Probing and understanding the limitations and biases of generative models.

Excluded uses are described below.

Out-of-Scope Use

The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.

Limitations and Bias

Limitations

The model does not achieve perfect photorealism
The model cannot render legible text
The model struggles with more difficult tasks which involve compositionality, such as rendering an image corresponding to “A red cube on top of a blue sphere”
Faces and people in general may not be generated properly.
The autoencoding part of the model is lossy.

Bias

While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases.

Contact

For questions and comments about the model, please join https://aiartlab.org/.
For future announcements / information about AiArtLab AI models, research, and events, please follow Discord.
For business and partnership inquiries, please contact https://t.me/recoilme