Chickaboo commited on
Commit
e756f21
·
verified ·
1 Parent(s): dc002d6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -9
README.md CHANGED
@@ -8,6 +8,9 @@ tags:
8
  - merge
9
 
10
  ---
 
 
 
11
  # mergedmodel
12
 
13
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
@@ -15,12 +18,12 @@ This is a merge of pre-trained language models created using [mergekit](https://
15
  ## Merge Details
16
  ### Merge Method
17
 
18
- This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [Qwen/Qwen1.5-0.5B-Chat](https://huggingface.co/Qwen/Qwen1.5-0.5B-Chat) as a base.
19
 
20
  ### Models Merged
21
 
22
  The following models were included in the merge:
23
- * [vilm/Quyen-SE-v0.1](https://huggingface.co/vilm/Quyen-SE-v0.1)
24
 
25
  ### Configuration
26
 
@@ -28,15 +31,16 @@ The following YAML configuration was used to produce this model:
28
 
29
  ```yaml
30
  models:
31
- - model: Qwen/Qwen1.5-0.5B-Chat
32
- # no parameters necessary for base model
33
  - model: vilm/Quyen-SE-v0.1
 
 
34
  parameters:
35
- density: 1
36
- weight: 1
37
- merge_method: dare_ties
38
- base_model: Qwen/Qwen1.5-0.5B-Chat
39
  parameters:
40
  normalize: true
41
  dtype: float16
42
- ```
 
 
8
  - merge
9
 
10
  ---
11
+ # Models in the ChickaQ family
12
+ - **ChickaQ (0.6B)**
13
+ - **ChickaQ-Large (1.8B)**
14
  # mergedmodel
15
 
16
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
 
18
  ## Merge Details
19
  ### Merge Method
20
 
21
+ This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [vilm/Quyen-SE-v0.1](https://huggingface.co/vilm/Quyen-SE-v0.1) as a base.
22
 
23
  ### Models Merged
24
 
25
  The following models were included in the merge:
26
+ * [Qwen/Qwen1.5-0.5B-Chat](https://huggingface.co/Qwen/Qwen1.5-0.5B-Chat)
27
 
28
  ### Configuration
29
 
 
31
 
32
  ```yaml
33
  models:
 
 
34
  - model: vilm/Quyen-SE-v0.1
35
+ # no parameters necessary for base model
36
+ - model: Qwen/Qwen1.5-0.5B-Chat
37
  parameters:
38
+ density: 0.5
39
+ weight: 0.5
40
+ merge_method: ties
41
+ base_model: vilm/Quyen-SE-v0.1
42
  parameters:
43
  normalize: true
44
  dtype: float16
45
+
46
+ ```