Hugging Face's logo

gattaplayer
/
leirix-flux-lokr-lion-1e-5-bs4-ga6-v03

Text-to-Image
Diffusers
flux
flux-diffusers
simpletuner
Not-For-All-Audiences
lora
template:sd-lora
lycoris
Model card Files Community
leirix-flux-lokr-lion-1e-5-bs4-ga6-v03 / README.md
gattaplayer's picture
gattaplayer
Model card auto-generated by SimpleTuner
cf570f3 verified
preview code
|
raw
history blame contribute delete
8.54 kB
metadata
license: other
base_model: black-forest-labs/FLUX.1-dev
tags:
  - flux
  - flux-diffusers
  - text-to-image
  - diffusers
  - simpletuner
  - not-for-all-audiences
  - lora
  - template:sd-lora
  - lycoris
inference: true
widget:
  - text: unconditional (blank prompt)
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_0_0.png
  - text: >-
      leirix, a woman with blonde hair in pigtails with pink and blue
      highlights, wearing a sleeveless black and red crop top and matching high
      waisted pants, with long red gloves, holding a large yellow fish over her
      shoulder, with a playful and happy expression, looking to the side,
      wearing bold red lipstick, with a small heart tattoo under her eye,
      background features a red backdrop with white stars
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_1_0.png
  - text: >-
      leirix, a young woman enjoying a sunny day at the beach, with long,
      flowing blonde hair that catches the light beautifully and bright,
      expressive blue eyes, wearing a sleek, one-piece athletic swimsuit with
      shades of blue and white, perfectly complementing her sun-kissed skin,
      sitting on a light blue beach towel, her posture relaxed as she leans back
      slightly, with the serene ocean and sandy beach stretching out behind her,
      sunlight casts a warm glow, creating soft shadows and highlighting the
      contours of her figure, scene exudes a peaceful, summery vibe, with the
      clear sky and calm waves in the background
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_2_0.png
  - text: >-
      leirix, a woman with long wavy dark hair, wearing a strapless form-fitting
      red dress with a sweetheart neckline and matching long gloves, her
      expression is confident and alluring, she is adorned with dangling
      earrings, she has one arm raised above her head, the background is a solid
      pink color
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_3_0.png
  - text: >-
      leirix, a woman with long wavy dark hair, wearing a red bikini, she has a
      red flower in her hair and is adorned with pearl earrings, her expression
      is confident and alluring, her body is glistening with water as she stands
      on a beach, the background features a clear blue sky, palm leaves, and the
      ocean
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_4_0.png
  - text: >-
      leirix, a woman with red hair, wearing a silver chainmail top and a large
      fur cloak, she is adorned with gold earrings and a necklace, she is riding
      a brown horse and holding the reins tightly, the background is a snowy
      landscape with a clear sky
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_5_0.png
  - text: >-
      leirix, a woman with vibrant red hair styled in an elaborate voluminous
      fashion, she has green, glowing eyes, wearing a form-fitting green outfit
      adorned with various plant elements and red berries, her hand is raised as
      if casting a spell, the background is blurred with soft, muted tones of
      green and blue
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_6_0.png

leirix-flux-lokr-lion-1e-5-bs4-ga6-v03

This is a LyCORIS adapter derived from black-forest-labs/FLUX.1-dev.

No validation prompt was used during training.

None

Validation settings

  • CFG: 3.0
  • CFG Rescale: 0.0
  • Steps: 20
  • Sampler: FlowMatchEulerDiscreteScheduler
  • Seed: 42
  • Resolution: 1024x1024
  • Skip-layer guidance:

Note: The validation settings are not necessarily the same as the training settings.

You can find some example images in the following gallery:

Prompt
unconditional (blank prompt)
Negative Prompt
blurry, cropped, ugly
Prompt
leirix, a woman with blonde hair in pigtails with pink and blue highlights, wearing a sleeveless black and red crop top and matching high waisted pants, with long red gloves, holding a large yellow fish over her shoulder, with a playful and happy expression, looking to the side, wearing bold red lipstick, with a small heart tattoo under her eye, background features a red backdrop with white stars
Negative Prompt
blurry, cropped, ugly
Prompt
leirix, a young woman enjoying a sunny day at the beach, with long, flowing blonde hair that catches the light beautifully and bright, expressive blue eyes, wearing a sleek, one-piece athletic swimsuit with shades of blue and white, perfectly complementing her sun-kissed skin, sitting on a light blue beach towel, her posture relaxed as she leans back slightly, with the serene ocean and sandy beach stretching out behind her, sunlight casts a warm glow, creating soft shadows and highlighting the contours of her figure, scene exudes a peaceful, summery vibe, with the clear sky and calm waves in the background
Negative Prompt
blurry, cropped, ugly
Prompt
leirix, a woman with long wavy dark hair, wearing a strapless form-fitting red dress with a sweetheart neckline and matching long gloves, her expression is confident and alluring, she is adorned with dangling earrings, she has one arm raised above her head, the background is a solid pink color
Negative Prompt
blurry, cropped, ugly
Prompt
leirix, a woman with long wavy dark hair, wearing a red bikini, she has a red flower in her hair and is adorned with pearl earrings, her expression is confident and alluring, her body is glistening with water as she stands on a beach, the background features a clear blue sky, palm leaves, and the ocean
Negative Prompt
blurry, cropped, ugly
Prompt
leirix, a woman with red hair, wearing a silver chainmail top and a large fur cloak, she is adorned with gold earrings and a necklace, she is riding a brown horse and holding the reins tightly, the background is a snowy landscape with a clear sky
Negative Prompt
blurry, cropped, ugly
Prompt
leirix, a woman with vibrant red hair styled in an elaborate voluminous fashion, she has green, glowing eyes, wearing a form-fitting green outfit adorned with various plant elements and red berries, her hand is raised as if casting a spell, the background is blurred with soft, muted tones of green and blue
Negative Prompt
blurry, cropped, ugly

The text encoder was not trained. You may reuse the base model text encoder for inference.

Training settings

  • Training epochs: 57
  • Training steps: 8840
  • Learning rate: 1e-05
    • Learning rate schedule: polynomial
    • Warmup steps: 100
  • Max grad norm: 0.01
  • Effective batch size: 12
    • Micro-batch size: 2
    • Gradient accumulation steps: 3
    • Number of GPUs: 2
  • Gradient checkpointing: True
  • Prediction type: flow-matching (extra parameters=['flux_schedule_auto_shift', 'shift=0.0', 'flux_guidance_mode=constant', 'flux_guidance_value=1.0', 'flow_matching_loss=compatible'])
  • Optimizer: optimi-lionweight_decay=1e-3
  • Trainable parameter precision: Pure BF16
  • Caption dropout probability: 10.0%

LyCORIS Config: 

{
    "algo": "lokr",
    "multiplier": 1.0,
    "linear_dim": 10000,
    "linear_alpha": 1,
    "factor": 12,
    "apply_preset": {
        "target_module": [
            "Attention",
            "FeedForward"
        ],
        "module_algo_map": {
            "Attention": {
                "factor": 12
            },
            "FeedForward": {
                "factor": 6
            }
        }
    }
}

Datasets

LEIRIX-FLUX-LLM-V03-512

  • Repeats: 1
  • Total number of images: ~146
  • Total number of aspect buckets: 2
  • Resolution: 0.262144 megapixels
  • Cropped: True
  • Crop style: random
  • Crop aspect: closest
  • Used for regularisation data: No

LEIRIX-FLUX-LLM-V03-768

  • Repeats: 1
  • Total number of images: ~148
  • Total number of aspect buckets: 3
  • Resolution: 0.589824 megapixels
  • Cropped: True
  • Crop style: random
  • Crop aspect: closest
  • Used for regularisation data: No

LEIRIX-FLUX-LLM-V03-1024

  • Repeats: 1
  • Total number of images: ~148
  • Total number of aspect buckets: 1
  • Resolution: 1.048576 megapixels
  • Cropped: True
  • Crop style: random
  • Crop aspect: closest
  • Used for regularisation data: No

LEIRIX-FLUX-WD-V03-512

  • Repeats: 1
  • Total number of images: ~146
  • Total number of aspect buckets: 1
  • Resolution: 0.262144 megapixels
  • Cropped: True
  • Crop style: random
  • Crop aspect: closest
  • Used for regularisation data: No

LEIRIX-FLUX-WD-V03-768

  • Repeats: 1
  • Total number of images: ~148
  • Total number of aspect buckets: 1
  • Resolution: 0.589824 megapixels
  • Cropped: True
  • Crop style: random
  • Crop aspect: closest
  • Used for regularisation data: No

LEIRIX-FLUX-WD-V03-1024

  • Repeats: 1
  • Total number of images: ~148
  • Total number of aspect buckets: 4
  • Resolution: 1.048576 megapixels
  • Cropped: True
  • Crop style: random
  • Crop aspect: closest
  • Used for regularisation data: No

Inference 

import torch
from diffusers import DiffusionPipeline
from lycoris import create_lycoris_from_weights


def download_adapter(repo_id: str):
    import os
    from huggingface_hub import hf_hub_download
    adapter_filename = "pytorch_lora_weights.safetensors"
    cache_dir = os.environ.get('HF_PATH', os.path.expanduser('~/.cache/huggingface/hub/models'))
    cleaned_adapter_path = repo_id.replace("/", "_").replace("\\", "_").replace(":", "_")
    path_to_adapter = os.path.join(cache_dir, cleaned_adapter_path)
    path_to_adapter_file = os.path.join(path_to_adapter, adapter_filename)
    os.makedirs(path_to_adapter, exist_ok=True)
    hf_hub_download(
        repo_id=repo_id, filename=adapter_filename, local_dir=path_to_adapter
    )

    return path_to_adapter_file
    
model_id = 'black-forest-labs/FLUX.1-dev'
adapter_repo_id = 'gattaplayer/leirix-flux-lokr-lion-1e-5-bs4-ga6-v03'
adapter_filename = 'pytorch_lora_weights.safetensors'
adapter_file_path = download_adapter(repo_id=adapter_repo_id)
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) # loading directly in bf16
lora_scale = 1.0
wrapper, _ = create_lycoris_from_weights(lora_scale, adapter_file_path, pipeline.transformer)
wrapper.merge_to()

prompt = "An astronaut is riding a horse through the jungles of Thailand."


## Optional: quantise the model to save on vram.
## Note: The model was quantised during training, and so it is recommended to do the same during inference time.
from optimum.quanto import quantize, freeze, qint8
quantize(pipeline.transformer, weights=qint8)
freeze(pipeline.transformer)
    
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu') # the pipeline is already in its target precision level
image = pipeline(
    prompt=prompt,
    num_inference_steps=20,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(42),
    width=1024,
    height=1024,
    guidance_scale=3.0,
).images[0]
image.save("output.png", format="PNG")