Spaces:
Runtime error
Runtime error
upload resources
Browse files- .gitattributes +4 -0
- resources/Training_Instructions.md +61 -0
- resources/applications.jpg +3 -0
- resources/framework.jpg +0 -0
- resources/gif/demo-1.gif +3 -0
- resources/gif/demo-2.gif +3 -0
- resources/teaser.jpg +3 -0
.gitattributes
CHANGED
|
@@ -38,3 +38,7 @@ example/Donut.mp4 filter=lfs diff=lfs merge=lfs -text
|
|
| 38 |
example/durian.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 39 |
example/pillow_huskies.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 40 |
example/wooden_car.mp4 filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
example/durian.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 39 |
example/pillow_huskies.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 40 |
example/wooden_car.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
resources/applications.jpg filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
resources/gif/demo-1.gif filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
resources/gif/demo-2.gif filter=lfs diff=lfs merge=lfs -text
|
| 44 |
+
resources/teaser.jpg filter=lfs diff=lfs merge=lfs -text
|
resources/Training_Instructions.md
ADDED
|
@@ -0,0 +1,61 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Preparations
|
| 2 |
+
|
| 3 |
+
## Cloning the Repository
|
| 4 |
+
The repository contains submodules, thus please check it out with
|
| 5 |
+
```shell
|
| 6 |
+
# SSH
|
| 7 |
+
git clone [email protected]:EnVision-Research/LucidDreamer.git --recursive
|
| 8 |
+
```
|
| 9 |
+
or
|
| 10 |
+
```shell
|
| 11 |
+
# HTTPS
|
| 12 |
+
git clone https://github.com/EnVision-Research/LucidDreamer.git --recursive
|
| 13 |
+
```
|
| 14 |
+
## Setup
|
| 15 |
+
Our default, provided install method is based on Conda package.
|
| 16 |
+
Firstly, you need to create an virtual environment and install the submodoules we provide. (slightly difference from original [3DGS](https://github.com/graphdeco-inria/gaussian-splatting))
|
| 17 |
+
```shell
|
| 18 |
+
conda env create --file environment.yml
|
| 19 |
+
conda activate LucidDreamer
|
| 20 |
+
pip install submodules/diff-gaussian-rasterization/
|
| 21 |
+
pip install submodules/simple-knn/
|
| 22 |
+
```
|
| 23 |
+
Then, you need to install [Point-E](https://github.com/openai/point-e) follow the instruction under this repo:
|
| 24 |
+
```shell
|
| 25 |
+
https://github.com/openai/point-e
|
| 26 |
+
```
|
| 27 |
+
|
| 28 |
+
# Running
|
| 29 |
+
We will provide a detailed guideline of our implementation about the description of each hyperparameter and how to tune them later. Now, we release 9 config files for you to evaluate the effectiveness of our framework (all configs can be trained in a single RTX3090).
|
| 30 |
+
|
| 31 |
+
Firstly, you may need to change ```model_key:``` in the ```configs\<config_file>.yaml``` to link the local Pretrained Diffusion Models ( [Stable Diffusion 2.1-base](https://github.com/Stability-AI/StableDiffusion) in default)
|
| 32 |
+
|
| 33 |
+
|
| 34 |
+
Then, you can simply use:
|
| 35 |
+
|
| 36 |
+
```shell
|
| 37 |
+
python train.py --opt <path to config file>
|
| 38 |
+
```
|
| 39 |
+
or you can see an exmaple and use the script we provide after you identify the visualable GPU:
|
| 40 |
+
```shell
|
| 41 |
+
bash train.sh
|
| 42 |
+
```
|
| 43 |
+
We provide config files in ```configs\``` that serve for different tasks:
|
| 44 |
+
|
| 45 |
+
Text-to-3D generation:
|
| 46 |
+
```shell
|
| 47 |
+
axe.yaml
|
| 48 |
+
bagel.yaml
|
| 49 |
+
cat_armor.yaml
|
| 50 |
+
crown.yaml
|
| 51 |
+
football_helmet.yaml
|
| 52 |
+
hamburger.yaml
|
| 53 |
+
white_hair_ironman.yaml
|
| 54 |
+
zombie_joker.yaml
|
| 55 |
+
```
|
| 56 |
+
|
| 57 |
+
Personalized Text-to-3D:
|
| 58 |
+
```shell
|
| 59 |
+
ts_lora.yaml
|
| 60 |
+
```
|
| 61 |
+
You can also use your own LoRA thourgh modify the: ```LoRA_path:```
|
resources/applications.jpg
ADDED
|
Git LFS Details
|
resources/framework.jpg
ADDED
|
resources/gif/demo-1.gif
ADDED
|
Git LFS Details
|
resources/gif/demo-2.gif
ADDED
|
Git LFS Details
|
resources/teaser.jpg
ADDED
|
Git LFS Details
|