Update README.md
Browse files
README.md
CHANGED
|
@@ -20,7 +20,8 @@ Answer user questions in Reasoning mode.
|
|
| 20 |
|
| 21 |
for using
|
| 22 |
1. install RWKV-Infer(see how to install)
|
| 23 |
-
2. loadmodel(choose FP16 or FP6 (dont choose FP8))
|
|
|
|
| 24 |
```
|
| 25 |
curl http://127.0.0.1:9000/loadmodel -X POST -H "Content-Type: application/json" -d '{"model_filename":"models/ARWKV-7B-CJE-30%.pth","model_viewname":"ARWKV-7B-CJE-30%","model_strategy":"fp16"}'
|
| 26 |
|
|
|
|
| 20 |
|
| 21 |
for using
|
| 22 |
1. install RWKV-Infer(see how to install)
|
| 23 |
+
2. loadmodel(choose FP16 or FP6 or FP5 (dont choose FP8))
|
| 24 |
+
2.1. need 19GB VRAM in FP16, 12GB VRAM in FP6
|
| 25 |
```
|
| 26 |
curl http://127.0.0.1:9000/loadmodel -X POST -H "Content-Type: application/json" -d '{"model_filename":"models/ARWKV-7B-CJE-30%.pth","model_viewname":"ARWKV-7B-CJE-30%","model_strategy":"fp16"}'
|
| 27 |
|