Hugging Face's logo

OpenMOSE
/
RWKV-x070-2B9-CJE-Instruct

Model card Files
xet
Community
OpenMOSE commited on
Commit
6e08c30
·
verified ·
1 Parent(s): a2274c6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -15,7 +15,7 @@ license: apache-2.0
15
  ## Fine-tuning Details
16
 
17
  ### Training Configuration
18
- - **Trainer**: RWKV-LM-RLHF
19
  - **PEFT Mode**: Hybrid Training combining frozen embeddings and Bone (Block Affine Transformation) + full parameter training
20
  - **SFT Method**: SmoothingLoss SFT
21
  - **Context Window**: 5120 (trained with 1024 token overlap)
@@ -31,12 +31,12 @@ license: apache-2.0
31
  - Chain-of-Thought reasoning tasks
32
 
33
  ### How to use
34
- - Install latest RWKV-Infer (Linux,WSL)
35
  - make folder 'models'
36
  ```
37
  curl http://127.0.0.1:9000/loadmodel -X POST -H "Content-Type: application/json" -d '{"model_filename":"models/rwkv-x070-2b9-cje-instruct-1.pth","model_viewname":"RWKV x070 2B9 CJE Instruct-1","model_strategy":"fp16","endtoken":"\\n\\n\\x17"}'
38
  ```
39
- - Enjou with openai compatible api http://127.0.0.1:9000/v1 :)
40
 
41
  ### Important Note
42
  - Set the end token as '\n\n\x17'
 
15
  ## Fine-tuning Details
16
 
17
  ### Training Configuration
18
+ - **Trainer**: RWKV-LM-RLHF (https://github.com/OpenMOSE/RWKV-LM-RLHF)
19
  - **PEFT Mode**: Hybrid Training combining frozen embeddings and Bone (Block Affine Transformation) + full parameter training
20
  - **SFT Method**: SmoothingLoss SFT
21
  - **Context Window**: 5120 (trained with 1024 token overlap)
 
31
  - Chain-of-Thought reasoning tasks
32
 
33
  ### How to use
34
+ - Install latest RWKV-Infer (Linux,WSL) (https://github.com/OpenMOSE/RWKV-Infer)
35
  - make folder 'models'
36
  ```
37
  curl http://127.0.0.1:9000/loadmodel -X POST -H "Content-Type: application/json" -d '{"model_filename":"models/rwkv-x070-2b9-cje-instruct-1.pth","model_viewname":"RWKV x070 2B9 CJE Instruct-1","model_strategy":"fp16","endtoken":"\\n\\n\\x17"}'
38
  ```
39
+ - Enjoy with openai compatible api http://127.0.0.1:9000/v1 :)
40
 
41
  ### Important Note
42
  - Set the end token as '\n\n\x17'