SmerkyG commited on
Commit
cd3a3f2
·
verified ·
1 Parent(s): 3c3f5bb

Upload folder using huggingface_hub

Browse files
Files changed (2) hide show
  1. README.md +8 -8
  2. model.safetensors +2 -2
README.md CHANGED
@@ -1,15 +1,15 @@
1
  ---
2
- license: apache-2.0
 
3
  datasets:
4
  - EleutherAI/the_pile_deduplicated
5
  language:
6
  - en
 
7
  metrics:
8
  - accuracy
9
- base_model:
10
- - BlinkDL/rwkv-7-pile
11
  pipeline_tag: text-generation
12
- library_name: transformers
13
  ---
14
 
15
  # rwkv7-168M-pile
@@ -38,16 +38,16 @@ This is RWKV-7 model under flash-linear attention format.
38
  <!-- Provide the basic links for the model. -->
39
 
40
  - **Repository:** https://github.com/fla-org/flash-linear-attention ; https://github.com/BlinkDL/RWKV-LM
41
- - **Paper:** [RWKV-7 "Goose" with Expressive Dynamic State Evolution](https://arxiv.org/abs/2503.14456)
42
  - **Weights:** Converted from https://modelscope.cn/models/RWKV/rwkv-7-pile/file/view/master?fileName=RWKV-x070-Pile-168M-20241120-ctx4096.pth
43
 
44
  ## Uses
45
 
46
  <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
47
- Install `flash-linear-attention` <= 0.1.2 and the latest version of `transformers` before using this model:
48
 
49
  ```bash
50
- pip install --no-use-pep517 flash-linear-attention==0.1.2
51
  pip install 'transformers>=4.48.0'
52
  ```
53
 
@@ -82,4 +82,4 @@ This model is trained on the Pile with a total of 332 billion tokens.
82
  ## FAQ
83
  Q: safetensors metadata is none.
84
 
85
- A: upgrade transformers to >=4.48.0: `pip install 'transformers>=4.48.0'`
 
1
  ---
2
+ base_model:
3
+ - BlinkDL/rwkv-7-pile
4
  datasets:
5
  - EleutherAI/the_pile_deduplicated
6
  language:
7
  - en
8
+ license: apache-2.0
9
  metrics:
10
  - accuracy
 
 
11
  pipeline_tag: text-generation
12
+ library_name: rwkv
13
  ---
14
 
15
  # rwkv7-168M-pile
 
38
  <!-- Provide the basic links for the model. -->
39
 
40
  - **Repository:** https://github.com/fla-org/flash-linear-attention ; https://github.com/BlinkDL/RWKV-LM
41
+ - **Paper:** https://huggingface.co/papers/2503.14456
42
  - **Weights:** Converted from https://modelscope.cn/models/RWKV/rwkv-7-pile/file/view/master?fileName=RWKV-x070-Pile-168M-20241120-ctx4096.pth
43
 
44
  ## Uses
45
 
46
  <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
47
+ Install `flash-linear-attention` and the latest version of `transformers` before using this model:
48
 
49
  ```bash
50
+ pip install git+https://github.com/fla-org/flash-linear-attention
51
  pip install 'transformers>=4.48.0'
52
  ```
53
 
 
82
  ## FAQ
83
  Q: safetensors metadata is none.
84
 
85
+ A: upgrade transformers to >=4.48.0: `pip install 'transformers>=4.48.0'`
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f7ba6afcdc9e797ca225f413cd0d2c2e7fab2012dd15d133ec6e169949829184
3
- size 670588632
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8fbeda2b50f0a09f6c98f4d263a542e2bb81d2beda353ee3c72d8c1576efd65a
3
+ size 335318368