Hugging Face's logo

Cseti
/
Basic_camera_motion_LoRAs_sd15-ad2-v1

English
stable diffusion
animatediff
motion lora
Model card Files Community
Basic_camera_motion_LoRAs_sd15-ad2-v1
File size: 4,594 Bytes 
11525f9
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
 
---
license: mit
tags:
- stable diffusion
- animatediff
- motion lora
language:
- en
---
- [Basic camera motion LoRAs for Animatediff v2 motion models - version 1](#basic-camera-motion-loras-for-animatediff-v2-motion-models---version-1)
  - [Model Description](#model-description)
  - [File summary](#file-summary)
  - [How to use](#how-to-use)
  - [General recommendation](#general-recommendation)
  - [Details and examples](#details-and-examples)
    - [Zooming in (16 frame)](#zooming-in-16-frame)
    - [Zooming in (32 frame)](#zooming-in-32-frame)
    - [Zooming out](#zooming-out)
    - [Lateral (horizontal) left](#lateral-horizontal-left)
    - [Lateral (horizontal) right](#lateral-horizontal-right)
    - [Crane (vertical) up](#crane-vertical-up)
    - [Crane (vertical) down](#crane-vertical-down)

# Basic camera motion LoRAs for Animatediff v2 motion models - version 1

## Model Description
This repository contains a collection of basic camera motion LoRAs finetuned on Animatediff v2 models. The training process utilized Kijai's [AnimateDiff Motion Director ComfyUI nodes](https://github.com/kijai/ComfyUI-ADMotionDirector). These LoRAs have been tested exclusively in ComfyUI with Kosinkadink's [AnimateDiff-Evolved nodes](https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved), however they may be compatible with other AnimateDiff implementations.

## File summary

| Name                                                          | Compatible Motion Model | Base AR (w/h) | Frames | input videos |
| ------------------------------------------------------------- | ----------------------- | ------------- | ------ | ------------ |
| 1600_cseti_9192119_camera-zoomout-10vid_mv2.safetensors       | v2                      | (3:2)         | 16     | 10           |
| 1600_cseti_1077723_camera-zoomin-10vid_mv2.safetensors        | v2                      | (3:2)         | 16     | 10           |
| 2800_cseti_1093371_camera-zoomin-32f-10vid_mv2.safetensors    | v2                      | (3:2)         | 32     | 10           |
| 1600_cseti_6664682_camera-lateral_right-10vid_mv2.safetensors | v2                      | (3:2)         | 16     | 10           |
| 1600_cseti_9213319_camera-lateral_left-10vid_mv2.safetensors  | v2                      | (3:2)         | 16     | 10           |
| 1600_cseti_9406077_camera-crane_down-10vid_mv2.safetensors    | v2                      | (3:2)         | 16     | 10           |
| 1600_cseti_1720230_camera-crane_up-10vid_mv2.safetensors      | v2                      | (3:2)         | 16     | 10           |

## How to use
- copy the motion lora files into your "ComfyUI\models\animatediff_motion_lora" folder
- Use the "Load Animatediff LoRA" node

## General recommendation
- These are finetuned on a v2 model. They work also with AnimateLCM but don't work with v3 models
- They work with SD 1.5 checkpoints
- Animatediff v3 adapter LoRA is recommended regardless they are v2 models
- If you want more motion try increasing the scale multival
- I recommend using the above Base AR (Aspect ratio) for inference however they might work with other ARs too
- Try playing with the strength of
  - motion LoRA 
  - scale multival 
  - v3_adapter lora
  
  like increasing the scale multival and lowering the lora strength.
- I recommend using IP-Adapter, however it reduces the effect of the mLORAs, so play with the strength. "Ease in out" weight type is a good starting point.

## Details and examples

### Zooming in (16 frame)
- recommended strength: 0.8-1.0

<video controls src="asset\LoRA-comparison-zoomin16.mp4" title="back view"></video>

### Zooming in (32 frame)
- this is an experimental motion LoRA trained on 32 frames so it can be used with context length 32.
- recommended strength: 0.7-0.85

<video controls src="asset\LoRA-comparison-zoomin32.mp4" title="back view"></video>

### Zooming out
- recommended strength: 0.8-1.0

<video controls src="asset\LoRA-comparison-zoomout16.mp4" title="zoom out"></video>

### Lateral (horizontal) left
- recommended strength: 0.8-1.0

<video controls src="asset\LoRA-comparison-lateral_left.mp4" title="right view"></video>

### Lateral (horizontal) right
- recommended strength: 0.8-1.0

<video controls src="asset\LoRA-comparison-lateral_right.mp4" title="right view"></video>

### Crane (vertical) up
- recommended strength: 0.8-1.0

<video controls src="asset\LoRA-comparison-crane_up.mp4" title="right view"></video>

### Crane (vertical) down
- recommended strength: 0.8-1.0

<video controls src="asset\LoRA-comparison-crane_down.mp4" title="right view"></video>