metadata

tags:
  - text-to-image
  - lora
  - diffusers
  - template:diffusion-lora
widget:
  - text: >-
      csetiarcane A medium shot frames a J1nx in neon-washed leather, her
      silhouette sharp against flickering signs. Her long blue hair catches blue
      light as she moves sideways through the frame. Her mesh top gleams beneath
      a studded jacket. Behind her, steam rises from street vents into the rainy
      night.
    output:
      url: images/testd_00149.mp4
  - text: >-
      csetiarcane Nfj1nx stands at the edge of a candlelit rooftop terrace
      overlooking glittering city towers. The camera faces her from the front,
      capturing every detail of her gentle smile as she meets the gaze of her
      partner—dressed in a sleek black blazer, collar slightly undone, and he
      has sporting jet-black hair that frames his face. Soft string lights sway
      overhead, highlighting her long blue hair and the warm reflection in her
      eyes. She takes a small step forward, and he carefully places one hand on
      her cheek before leaning in for a tender kiss, their silhouettes merging
      at the center of the frame.
    output:
      url: images/testd_00154.mp4
  - text: >-
      csetiarcane Nfj1nx moves along a tree-shaded park path, coming directly
      toward the camera. A matte-black pair of over-ear headphones partially
      covers her ears, letting most of her bright blue hair remain visible. She
      wears a simple white athletic hoodie paired with black biker shorts.
      Pastel-toned running shoes quietly crunch on the gravel, reflecting her
      relaxed yet confident rhythm.
    output:
      url: images/testd_00156.mp4
  - text: >-
      csetiarcane Nfj1nx moves along a tree-shaded park path, coming directly
      toward the camera. A matte-black pair of over-ear headphones partially
      covers her ears, letting most of her bright blue hair remain visible. She
      wears a simple white athletic hoodie paired with black biker shorts.
      Pastel-toned running shoes quietly crunch on the gravel, reflecting her
      relaxed yet confident rhythm.
    output:
      url: images/testd_00156-2.mp4
base_model: Wan-AI/Wan2.1-T2V-1.3B
instance_prompt: null

Wan-LoRA-Arcane-Jinx-v1

Prompt: csetiarcane A medium shot frames a J1nx in neon-washed leather, her silhouette sharp against flickering signs. Her long blue hair catches blue light as she moves sideways through the frame. Her mesh top gleams beneath a studded jacket. Behind her, steam rises from street vents into the rainy night.

Prompt: csetiarcane Nfj1nx stands at the edge of a candlelit rooftop terrace overlooking glittering city towers. The camera faces her from the front, capturing every detail of her gentle smile as she meets the gaze of her partner—dressed in a sleek black blazer, collar slightly undone, and he has sporting jet-black hair that frames his face. Soft string lights sway overhead, highlighting her long blue hair and the warm reflection in her eyes. She takes a small step forward, and he carefully places one hand on her cheek before leaning in for a tender kiss, their silhouettes merging at the center of the frame.

Prompt: csetiarcane Nfj1nx moves along a tree-shaded park path, coming directly toward the camera. A matte-black pair of over-ear headphones partially covers her ears, letting most of her bright blue hair remain visible. She wears a simple white athletic hoodie paired with black biker shorts. Pastel-toned running shoes quietly crunch on the gravel, reflecting her relaxed yet confident rhythm.

Prompt: csetiarcane Nfj1nx moves along a tree-shaded park path, coming directly toward the camera. A matte-black pair of over-ear headphones partially covers her ears, letting most of her bright blue hair remain visible. She wears a simple white athletic hoodie paired with black biker shorts. Pastel-toned running shoes quietly crunch on the gravel, reflecting her relaxed yet confident rhythm.

Trainig details:

I tried a method where I first trained the LoRA on images only and then after a certain number of steps continued training only on videos.

LR: 2e-5
Optimizer: adamw_optimi
epochs: 15 (image) + 7 (video)
steps: 2550 (image) + 2520 (video)
dataset: 33 images and 35 videos
repeats: 10
rank: 128
batch size: 2
gradient accumulation steps: 1

For training I used the Diffusion-Pipe repo.

Inference

It seems, the model likes long descriptive prompts. Look at the attached videos for prompt examples. Based on my tests if you use short prompts, the lora effect is weak.

For inference I recommend:

Trigger words: csetiarcane, Nfj1nx, blue hair

(If it isn't enough so you don't get the style/character, I recommend adding "animation style" to you prompt. It can help providing the style but in some cases the result will be too cartoony)

Strength: 0.9-1.0

Acknowledgement

Special thanks to Kijai for the tireless and excelent work done for the community to enable us to use these solutions as soon as possible, as well, as to Comfyanonymous for implementing them so quickly with such a good quality natively into ComfyUI, and also to TDRussel for making LoRA training available to the community so quickly.