leirix-flux-lokr-lion-1e-5-bs4-ga6-v03

This is a LyCORIS adapter derived from black-forest-labs/FLUX.1-dev.

No validation prompt was used during training.

None

Validation settings

CFG: 3.0
CFG Rescale: 0.0
Steps: 20
Sampler: FlowMatchEulerDiscreteScheduler
Seed: 42
Resolution: 1024x1024
Skip-layer guidance:

Note: The validation settings are not necessarily the same as the training settings.

You can find some example images in the following gallery:

Prompt
unconditional (blank prompt)

Negative Prompt
blurry, cropped, ugly

Prompt
leirix, a woman with blonde hair in pigtails with pink and blue highlights, wearing a sleeveless black and red crop top and matching high waisted pants, with long red gloves, holding a large yellow fish over her shoulder, with a playful and happy expression, looking to the side, wearing bold red lipstick, with a small heart tattoo under her eye, background features a red backdrop with white stars

Negative Prompt
blurry, cropped, ugly

Prompt
leirix, a young woman enjoying a sunny day at the beach, with long, flowing blonde hair that catches the light beautifully and bright, expressive blue eyes, wearing a sleek, one-piece athletic swimsuit with shades of blue and white, perfectly complementing her sun-kissed skin, sitting on a light blue beach towel, her posture relaxed as she leans back slightly, with the serene ocean and sandy beach stretching out behind her, sunlight casts a warm glow, creating soft shadows and highlighting the contours of her figure, scene exudes a peaceful, summery vibe, with the clear sky and calm waves in the background

Negative Prompt
blurry, cropped, ugly

Prompt
leirix, a woman with long wavy dark hair, wearing a strapless form-fitting red dress with a sweetheart neckline and matching long gloves, her expression is confident and alluring, she is adorned with dangling earrings, she has one arm raised above her head, the background is a solid pink color

Negative Prompt
blurry, cropped, ugly

Prompt
leirix, a woman with long wavy dark hair, wearing a red bikini, she has a red flower in her hair and is adorned with pearl earrings, her expression is confident and alluring, her body is glistening with water as she stands on a beach, the background features a clear blue sky, palm leaves, and the ocean

Negative Prompt
blurry, cropped, ugly

Prompt
leirix, a woman with red hair, wearing a silver chainmail top and a large fur cloak, she is adorned with gold earrings and a necklace, she is riding a brown horse and holding the reins tightly, the background is a snowy landscape with a clear sky

Negative Prompt
blurry, cropped, ugly

Prompt
leirix, a woman with vibrant red hair styled in an elaborate voluminous fashion, she has green, glowing eyes, wearing a form-fitting green outfit adorned with various plant elements and red berries, her hand is raised as if casting a spell, the background is blurred with soft, muted tones of green and blue

Negative Prompt
blurry, cropped, ugly

The text encoder was not trained. You may reuse the base model text encoder for inference.

Training settings

Training epochs: 57
Training steps: 8840
Learning rate: 1e-05
- Learning rate schedule: polynomial
- Warmup steps: 100
Max grad norm: 0.01
Effective batch size: 12
- Micro-batch size: 2
- Gradient accumulation steps: 3
- Number of GPUs: 2
Gradient checkpointing: True
Prediction type: flow-matching (extra parameters=['flux_schedule_auto_shift', 'shift=0.0', 'flux_guidance_mode=constant', 'flux_guidance_value=1.0', 'flow_matching_loss=compatible'])
Optimizer: optimi-lionweight_decay=1e-3
Trainable parameter precision: Pure BF16
Caption dropout probability: 10.0%

LyCORIS Config:

{
    "algo": "lokr",
    "multiplier": 1.0,
    "linear_dim": 10000,
    "linear_alpha": 1,
    "factor": 12,
    "apply_preset": {
        "target_module": [
            "Attention",
            "FeedForward"
        ],
        "module_algo_map": {
            "Attention": {
                "factor": 12
            },
            "FeedForward": {
                "factor": 6
            }
        }
    }
}

Datasets

LEIRIX-FLUX-LLM-V03-512

Repeats: 1
Total number of images: ~146
Total number of aspect buckets: 2
Resolution: 0.262144 megapixels
Cropped: True
Crop style: random
Crop aspect: closest
Used for regularisation data: No

LEIRIX-FLUX-LLM-V03-768

Repeats: 1
Total number of images: ~148
Total number of aspect buckets: 3
Resolution: 0.589824 megapixels
Cropped: True
Crop style: random
Crop aspect: closest
Used for regularisation data: No

LEIRIX-FLUX-LLM-V03-1024

Repeats: 1
Total number of images: ~148
Total number of aspect buckets: 1
Resolution: 1.048576 megapixels
Cropped: True
Crop style: random
Crop aspect: closest
Used for regularisation data: No

LEIRIX-FLUX-WD-V03-512

Repeats: 1
Total number of images: ~146
Total number of aspect buckets: 1
Resolution: 0.262144 megapixels
Cropped: True
Crop style: random
Crop aspect: closest
Used for regularisation data: No

LEIRIX-FLUX-WD-V03-768

Repeats: 1
Total number of images: ~148
Total number of aspect buckets: 1
Resolution: 0.589824 megapixels
Cropped: True
Crop style: random
Crop aspect: closest
Used for regularisation data: No

LEIRIX-FLUX-WD-V03-1024

Repeats: 1
Total number of images: ~148
Total number of aspect buckets: 4
Resolution: 1.048576 megapixels
Cropped: True
Crop style: random
Crop aspect: closest
Used for regularisation data: No

Inference

import torch
from diffusers import DiffusionPipeline
from lycoris import create_lycoris_from_weights


def download_adapter(repo_id: str):
    import os
    from huggingface_hub import hf_hub_download
    adapter_filename = "pytorch_lora_weights.safetensors"
    cache_dir = os.environ.get('HF_PATH', os.path.expanduser('~/.cache/huggingface/hub/models'))
    cleaned_adapter_path = repo_id.replace("/", "_").replace("\\", "_").replace(":", "_")
    path_to_adapter = os.path.join(cache_dir, cleaned_adapter_path)
    path_to_adapter_file = os.path.join(path_to_adapter, adapter_filename)
    os.makedirs(path_to_adapter, exist_ok=True)
    hf_hub_download(
        repo_id=repo_id, filename=adapter_filename, local_dir=path_to_adapter
    )

    return path_to_adapter_file
    
model_id = 'black-forest-labs/FLUX.1-dev'
adapter_repo_id = 'gattaplayer/leirix-flux-lokr-lion-1e-5-bs4-ga6-v03'
adapter_filename = 'pytorch_lora_weights.safetensors'
adapter_file_path = download_adapter(repo_id=adapter_repo_id)
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) # loading directly in bf16
lora_scale = 1.0
wrapper, _ = create_lycoris_from_weights(lora_scale, adapter_file_path, pipeline.transformer)
wrapper.merge_to()

prompt = "An astronaut is riding a horse through the jungles of Thailand."


## Optional: quantise the model to save on vram.
## Note: The model was quantised during training, and so it is recommended to do the same during inference time.
from optimum.quanto import quantize, freeze, qint8
quantize(pipeline.transformer, weights=qint8)
freeze(pipeline.transformer)
    
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu') # the pipeline is already in its target precision level
image = pipeline(
    prompt=prompt,
    num_inference_steps=20,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(42),
    width=1024,
    height=1024,
    guidance_scale=3.0,
).images[0]
image.save("output.png", format="PNG")

gattaplayer
/

leirix-flux-lokr-lion-1e-5-bs4-ga6-v03