Commit
·
188229c
1
Parent(s):
ecf588e
add meta files
Browse files
README.md
CHANGED
@@ -6,7 +6,7 @@ language:
|
|
6 |
license: other
|
7 |
model_creator: lmsys
|
8 |
model_link: https://huggingface.co/lmsys/vicuna-13b-v1.5-16k
|
9 |
-
model_name:
|
10 |
model_type: vicuna
|
11 |
pipeline_tag: text-generation
|
12 |
quantized_by: shaowenchen
|
@@ -45,3 +45,19 @@ docker run --rm -it -p 8000:8000 -v /path/to/models:/models -e MODEL=/models/ggu
|
|
45 |
```
|
46 |
|
47 |
and you can view http://localhost:8000/docs to see the swagger UI.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
6 |
license: other
|
7 |
model_creator: lmsys
|
8 |
model_link: https://huggingface.co/lmsys/vicuna-13b-v1.5-16k
|
9 |
+
model_name: vicuna-13b-v1.5-16k
|
10 |
model_type: vicuna
|
11 |
pipeline_tag: text-generation
|
12 |
quantized_by: shaowenchen
|
|
|
45 |
```
|
46 |
|
47 |
and you can view http://localhost:8000/docs to see the swagger UI.
|
48 |
+
|
49 |
+
## Provided images
|
50 |
+
|
51 |
+
| Name | Quant method | Compressed Size |
|
52 |
+
| ------------------------------------------- | ------------ | --------------- |
|
53 |
+
| `shaowenchen/vicuna-13b-v1.5-16k-gguf:Q2_K` | Q2_K | 2.88 GB |
|
54 |
+
| `shaowenchen/vicuna-13b-v1.5-16k-gguf:Q3_K` | Q3_K | 3.3 GB |
|
55 |
+
| `shaowenchen/vicuna-13b-v1.5-16k-gguf:Q4_K` | Q4_K | 4 GB |
|
56 |
+
|
57 |
+
Usage:
|
58 |
+
|
59 |
+
```
|
60 |
+
docker run --rm -p 8000:8000 shaowenchen/vicuna-13b-v1.5-16k-gguf:Q2_K
|
61 |
+
```
|
62 |
+
|
63 |
+
and you can view http://localhost:8000/docs to see the swagger UI.
|