Hugging Face's logo

Cseti
/
Basic_camera_motion_LoRAs_sd15-ad2-v1

English
stable diffusion
animatediff
motion lora
Model card Files Community
Cseti commited on
Commit
11525f9
·
verified ·
1 Parent(s): bd7102f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +96 -76
README.md CHANGED
@@ -1,76 +1,96 @@
1
- ---
2
- license: mit
3
- tags:
4
- - stable diffusion
5
- - animatediff
6
- - motion lora
7
- language:
8
- - en
9
- ---
10
- - [Basic camera motion LoRAs for Animatediff v2 motion models - version 1](#basic-camera-motion-loras-for-animatediff-v2-motion-models---version-1)
11
- - [Model Description](#model-description)
12
- - [File summary](#file-summary)
13
- - [How to use](#how-to-use)
14
- - [General recommendation](#general-recommendation)
15
- - [Details and examples](#details-and-examples)
16
- - [Zooming in](#zooming-in)
17
- - [Zooming out](#zooming-out)
18
- - [Lateral (horizontal) left](#lateral-horizontal-left)
19
- - [Lateral (horizontal) rigth](#lateral-horizontal-rigth)
20
-
21
- # Basic camera motion LoRAs for Animatediff v2 motion models - version 1
22
-
23
- ## Model Description
24
- This repository contains a collection of basic camera motion LoRAs finetuned on Animatediff v2 models. The training process utilized Kijai's [AnimateDiff Motion Director ComfyUI nodes](https://github.com/kijai/ComfyUI-ADMotionDirector). These LoRAs have been tested exclusively in ComfyUI with Kosinkadink's [AnimateDiff-Evolved nodes](https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved), however they may be compatible with other AnimateDiff implementations.
25
-
26
- ## File summary
27
-
28
- | Name | Compatible Motion Model | Base AR (w/h) | Frames | input videos |
29
- | ------------------------------------------------------------- | ----------------------- | ------------- | ------ | ------------ |
30
- | 1600_cseti_9192119_camera-zoomout-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 |
31
- | 1600_cseti_1077723_camera-zoomin-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 |
32
- | 2800_cseti_1093371_camera-zoomin-32f-10vid_mv2.safetensors | v2 | (3:2) | 32 | 10 |
33
- | 1600_cseti_6664682_camera-lateral_right-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 |
34
- | 1600_cseti_9213319_camera-lateral_left-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 |
35
- | 1600_cseti_9406077_camera-crane_down-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 |
36
- | 1600_cseti_1720230_camera-crane_up-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 |
37
-
38
- ## How to use
39
- - copy the motion lora files into your "ComfyUI\models\animatediff_motion_lora" folder
40
- - Use the "Load Animatediff LoRA" node
41
-
42
- ## General recommendation
43
- - These are finetuned on a v2 model. They work also with AnimateLCM but don't work with v3 models
44
- - They work with SD 1.5 checkpoints
45
- - Animatediff v3 adapter LoRA is recommended regardless they are v2 models
46
- - If you want more motion try increasing the scale multival
47
- - I recommend using the above Base AR (Aspect ratio) for inference however they might work with other ARs too
48
- - Try playing with the strength of
49
- - motion LoRA
50
- - scale multival
51
- - v3_adapter lora
52
-
53
- like increasing the scale multival and lowering the lora strength.
54
- - I recommend using IP-Adapter, however it reduces the effect of the mLORAs, so play with the strength. "Ease in out" weight type is a good starting point.
55
-
56
- ## Details and examples
57
-
58
- ### Zooming in
59
- - recommended strength: 0.8-1.0
60
-
61
- <video controls src="asset\LoRA-comparison-zoomin16.mp4" title="back view"></video>
62
-
63
- ### Zooming out
64
- - recommended strength: 0.8-1.0
65
-
66
- <video controls src="asset\LoRA-comparison-zoomout16.mp4" title="zoom out"></video>
67
-
68
- ### Lateral (horizontal) left
69
- - recommended strength: 0.8-1.0
70
-
71
- <video controls src="asset\LoRA-comparison-lateral_left.mp4" title="right view"></video>
72
-
73
- ### Lateral (horizontal) rigth
74
- - recommended strength: 0.8-1.0
75
-
76
- <video controls src="asset\LoRA-comparison-lateral_right.mp4" title="right view"></video>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ tags:
4
+ - stable diffusion
5
+ - animatediff
6
+ - motion lora
7
+ language:
8
+ - en
9
+ ---
10
+ - [Basic camera motion LoRAs for Animatediff v2 motion models - version 1](#basic-camera-motion-loras-for-animatediff-v2-motion-models---version-1)
11
+ - [Model Description](#model-description)
12
+ - [File summary](#file-summary)
13
+ - [How to use](#how-to-use)
14
+ - [General recommendation](#general-recommendation)
15
+ - [Details and examples](#details-and-examples)
16
+ - [Zooming in (16 frame)](#zooming-in-16-frame)
17
+ - [Zooming in (32 frame)](#zooming-in-32-frame)
18
+ - [Zooming out](#zooming-out)
19
+ - [Lateral (horizontal) left](#lateral-horizontal-left)
20
+ - [Lateral (horizontal) right](#lateral-horizontal-right)
21
+ - [Crane (vertical) up](#crane-vertical-up)
22
+ - [Crane (vertical) down](#crane-vertical-down)
23
+
24
+ # Basic camera motion LoRAs for Animatediff v2 motion models - version 1
25
+
26
+ ## Model Description
27
+ This repository contains a collection of basic camera motion LoRAs finetuned on Animatediff v2 models. The training process utilized Kijai's [AnimateDiff Motion Director ComfyUI nodes](https://github.com/kijai/ComfyUI-ADMotionDirector). These LoRAs have been tested exclusively in ComfyUI with Kosinkadink's [AnimateDiff-Evolved nodes](https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved), however they may be compatible with other AnimateDiff implementations.
28
+
29
+ ## File summary
30
+
31
+ | Name | Compatible Motion Model | Base AR (w/h) | Frames | input videos |
32
+ | ------------------------------------------------------------- | ----------------------- | ------------- | ------ | ------------ |
33
+ | 1600_cseti_9192119_camera-zoomout-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 |
34
+ | 1600_cseti_1077723_camera-zoomin-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 |
35
+ | 2800_cseti_1093371_camera-zoomin-32f-10vid_mv2.safetensors | v2 | (3:2) | 32 | 10 |
36
+ | 1600_cseti_6664682_camera-lateral_right-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 |
37
+ | 1600_cseti_9213319_camera-lateral_left-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 |
38
+ | 1600_cseti_9406077_camera-crane_down-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 |
39
+ | 1600_cseti_1720230_camera-crane_up-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 |
40
+
41
+ ## How to use
42
+ - copy the motion lora files into your "ComfyUI\models\animatediff_motion_lora" folder
43
+ - Use the "Load Animatediff LoRA" node
44
+
45
+ ## General recommendation
46
+ - These are finetuned on a v2 model. They work also with AnimateLCM but don't work with v3 models
47
+ - They work with SD 1.5 checkpoints
48
+ - Animatediff v3 adapter LoRA is recommended regardless they are v2 models
49
+ - If you want more motion try increasing the scale multival
50
+ - I recommend using the above Base AR (Aspect ratio) for inference however they might work with other ARs too
51
+ - Try playing with the strength of
52
+ - motion LoRA
53
+ - scale multival
54
+ - v3_adapter lora
55
+
56
+ like increasing the scale multival and lowering the lora strength.
57
+ - I recommend using IP-Adapter, however it reduces the effect of the mLORAs, so play with the strength. "Ease in out" weight type is a good starting point.
58
+
59
+ ## Details and examples
60
+
61
+ ### Zooming in (16 frame)
62
+ - recommended strength: 0.8-1.0
63
+
64
+ <video controls src="asset\LoRA-comparison-zoomin16.mp4" title="back view"></video>
65
+
66
+ ### Zooming in (32 frame)
67
+ - this is an experimental motion LoRA trained on 32 frames so it can be used with context length 32.
68
+ - recommended strength: 0.7-0.85
69
+
70
+ <video controls src="asset\LoRA-comparison-zoomin32.mp4" title="back view"></video>
71
+
72
+ ### Zooming out
73
+ - recommended strength: 0.8-1.0
74
+
75
+ <video controls src="asset\LoRA-comparison-zoomout16.mp4" title="zoom out"></video>
76
+
77
+ ### Lateral (horizontal) left
78
+ - recommended strength: 0.8-1.0
79
+
80
+ <video controls src="asset\LoRA-comparison-lateral_left.mp4" title="right view"></video>
81
+
82
+ ### Lateral (horizontal) right
83
+ - recommended strength: 0.8-1.0
84
+
85
+ <video controls src="asset\LoRA-comparison-lateral_right.mp4" title="right view"></video>
86
+
87
+ ### Crane (vertical) up
88
+ - recommended strength: 0.8-1.0
89
+
90
+ <video controls src="asset\LoRA-comparison-crane_up.mp4" title="right view"></video>
91
+
92
+ ### Crane (vertical) down
93
+ - recommended strength: 0.8-1.0
94
+
95
+ <video controls src="asset\LoRA-comparison-crane_down.mp4" title="right view"></video>
96
+