metadata

license: other
base_model: black-forest-labs/FLUX.1-schnell
tags:
  - flux
  - flux-diffusers
  - text-to-image
  - diffusers
  - simpletuner
  - safe-for-work
  - lora
  - template:sd-lora
  - standard
inference: true
widget:
  - text: unconditional (blank prompt)
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_0_0.png
  - text: >-
      minimalist icon of a vertically oriented banana silhouette with a slight
      curve, featuring a tapered end and a small stem at the top, outline icon,
      black icon, no gradients, no shadows, clean lines, rounded corners,
      geometric simplicity, modern minimalism
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_1_0.png
  - text: >-
      minimalist icon of a traffic cone featuring a sleek triangular top
      attached to a wider cylindrical base, with clean lines and consistent
      stroke width throughout, outline icon, black icon, no gradients, no
      shadows, clean lines, rounded corners, geometric simplicity, modern
      minimalism
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_2_0.png
  - text: >-
      minimalist icon of an upward-pointing arrow inside a rectangular box,
      paired with a plus symbol on the right side, outline icon, black icon, no
      gradients, no shadows, clean lines, rounded corners, geometric simplicity,
      modern minimalism
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_3_0.png
  - text: >-
      minimalist icon of the letters M, P, and X in a clean and modern
      sans-serif font, evenly spaced in a horizontal row, outline icon, black
      icon, no gradients, no shadows, clean lines, rounded corners, geometric
      simplicity, modern minimalism
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_4_0.png
  - text: >-
      minimalist icon of a fire truck with a rectangular body, rounded cab
      windows, two wheels, extended ladder on top, and simplified geometric
      shapes for the body and details, outline icon, black icon, no gradients,
      no shadows, clean lines, rounded corners, geometric simplicity, modern
      minimalism
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_5_0.png
  - text: >-
      minimalist icon of a heart and a star interlocking, with the bottom point
      of the star nestled within the top curve of the heart, creating a unified
      design where the shapes blend seamlessly at their connection point,
      outline icon, black icon, no gradients, no shadows, clean lines, rounded
      corners, geometric simplicity, modern minimalism
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_6_0.png
  - text: minimalist icon
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_7_0.png

icon-generator-2

This is a standard PEFT LoRA derived from black-forest-labs/FLUX.1-schnell.

The main validation prompt used during training was:

minimalist icon

Validation settings

CFG: 3.0
CFG Rescale: 0.0
Steps: 20
Sampler: FlowMatchEulerDiscreteScheduler
Seed: 42
Resolution: 512x512
Skip-layer guidance:

Note: The validation settings are not necessarily the same as the training settings.

You can find some example images in the following gallery:

Prompt: unconditional (blank prompt)

Negative Prompt: blurry, cropped, ugly

Prompt: minimalist icon of a vertically oriented banana silhouette with a slight curve, featuring a tapered end and a small stem at the top, outline icon, black icon, no gradients, no shadows, clean lines, rounded corners, geometric simplicity, modern minimalism

Negative Prompt: blurry, cropped, ugly

Prompt: minimalist icon of a traffic cone featuring a sleek triangular top attached to a wider cylindrical base, with clean lines and consistent stroke width throughout, outline icon, black icon, no gradients, no shadows, clean lines, rounded corners, geometric simplicity, modern minimalism

Negative Prompt: blurry, cropped, ugly

Prompt: minimalist icon of an upward-pointing arrow inside a rectangular box, paired with a plus symbol on the right side, outline icon, black icon, no gradients, no shadows, clean lines, rounded corners, geometric simplicity, modern minimalism

Negative Prompt: blurry, cropped, ugly

Prompt: minimalist icon of the letters M, P, and X in a clean and modern sans-serif font, evenly spaced in a horizontal row, outline icon, black icon, no gradients, no shadows, clean lines, rounded corners, geometric simplicity, modern minimalism

Negative Prompt: blurry, cropped, ugly

Prompt: minimalist icon of a fire truck with a rectangular body, rounded cab windows, two wheels, extended ladder on top, and simplified geometric shapes for the body and details, outline icon, black icon, no gradients, no shadows, clean lines, rounded corners, geometric simplicity, modern minimalism

Negative Prompt: blurry, cropped, ugly

Prompt: minimalist icon of a heart and a star interlocking, with the bottom point of the star nestled within the top curve of the heart, creating a unified design where the shapes blend seamlessly at their connection point, outline icon, black icon, no gradients, no shadows, clean lines, rounded corners, geometric simplicity, modern minimalism

Negative Prompt: blurry, cropped, ugly

Prompt: minimalist icon

Negative Prompt: blurry, cropped, ugly

The text encoder was not trained. You may reuse the base model text encoder for inference.

Training settings

Training epochs: 2
Training steps: 10000
Learning rate: 8e-05
- Learning rate schedule: polynomial
- Warmup steps: 100
Max grad norm: 2.0
Effective batch size: 1
- Micro-batch size: 1
- Gradient accumulation steps: 1
- Number of GPUs: 1
Gradient checkpointing: True
Prediction type: flow-matching (extra parameters=['flux_fast_schedule', 'shift=3', 'flux_guidance_mode=constant', 'flux_guidance_value=1.0', 'flow_matching_loss=compatible', 'flux_lora_target=all'])
Optimizer: adamw_bf16
Trainable parameter precision: Pure BF16
Caption dropout probability: 10.0%
LoRA Rank: 64
LoRA Alpha: None
LoRA Dropout: 0.1
LoRA initialisation style: default

Datasets

tabler-icons-captioned-512

Repeats: 0
Total number of images: 4918
Total number of aspect buckets: 1
Resolution: 0.262144 megapixels
Cropped: False
Crop style: None
Crop aspect: None
Used for regularisation data: No

Inference

import torch
from diffusers import DiffusionPipeline

model_id = 'black-forest-labs/FLUX.1-schnell'
adapter_id = 'noahyoungs/icon-generator-2'
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) # loading directly in bf16
pipeline.load_lora_weights(adapter_id)

prompt = "minimalist icon"


## Optional: quantise the model to save on vram.
## Note: The model was quantised during training, and so it is recommended to do the same during inference time.
from optimum.quanto import quantize, freeze, qint8
quantize(pipeline.transformer, weights=qint8)
freeze(pipeline.transformer)
    
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu') # the pipeline is already in its target precision level
image = pipeline(
    prompt=prompt,
    num_inference_steps=20,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(42),
    width=512,
    height=512,
    guidance_scale=3.0,
).images[0]
image.save("output.png", format="PNG")