Update README.md
Browse files
README.md
CHANGED
@@ -24,7 +24,10 @@ Bingo again. I hit 3.45 ppl512 wikieng, 62+ or ARC-C, and 82+ on ARC-E. Absolute
|
|
24 |
|
25 |
No cheating, no contaminating, just the wonderful MergeKit model-stock merge technique leveraged to a new level (compared to what I already saw being done, at least).
|
26 |
|
27 |
-
Next projects will involve that model as the "smarts pillar" of further merges, aimed at any use case.
|
|
|
|
|
|
|
28 |
|
29 |
Edit : the mothodology I use is actually partly rediscovered hot water.
|
30 |
|
@@ -43,8 +46,9 @@ The adventure continues with Doberman_V1, a Hermes flavored Dobby on Smarteaz ab
|
|
43 |
|
44 |
And the saga continues again with
|
45 |
|
46 |
-
Nemesis_V1 (ex Negames), a Hermes flavored Negative Llama on Smarteaz abliterated steroids
|
47 |
-
-
|
|
|
48 |
|
49 |
Evasion_V1 (ex Hermeva), a Hermes flavored Eva_01 on Smarteaz abliterated steroids (the most creative) :
|
50 |
- Nexesenex/Llama_3.x_70b_Evasion_V1 : https://huggingface.co/Nexesenex/Llama_3.x_70b_Evasion_V1 (less than 3.40 ppl 512 wiki-eng, -0.02 compared to Doberman_V1)
|
@@ -52,14 +56,45 @@ Evasion_V1 (ex Hermeva), a Hermes flavored Eva_01 on Smarteaz abliterated steroi
|
|
52 |
Trinity_V1, a merge of Evasion as base, Doberman and NegaTessTease to include a bit of Tess (to be tested) :
|
53 |
- Nexesenex/Llama_3.x_70b_Trinity_V1 : https://huggingface.co/Nexesenex/Llama_3.x_70b_Trinity_V1 (less than 3.40 ppl 512 wiki-eng, -0.03 compared to Doberman_V1)
|
54 |
|
55 |
-
I
|
56 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
57 |
|
58 |
---
|
59 |
# credits
|
60 |
|
61 |
Kudos go to the model authors, and to the Arcee / MergeKit folks, as well as to HF hosting the MergeKit App.
|
62 |
Also a big-up to SteelSkull, observing him cooking Nevoria decided me to try to make some merges by myself.
|
|
|
63 |
|
64 |
---
|
65 |
# historic
|
|
|
24 |
|
25 |
No cheating, no contaminating, just the wonderful MergeKit model-stock merge technique leveraged to a new level (compared to what I already saw being done, at least).
|
26 |
|
27 |
+
Next projects will involve that model as the "smarts pillar/Block" of further merges, aimed at any use case.
|
28 |
+
|
29 |
+
I think that most models can be tweaked the same way, with triple stock merges interlacing intruct finetunes and base finetunes.
|
30 |
+
- This, gaining overall "intelligence" and "quality" at the cost of a bit of its initial instructions, flavor and "personality".
|
31 |
|
32 |
Edit : the mothodology I use is actually partly rediscovered hot water.
|
33 |
|
|
|
46 |
|
47 |
And the saga continues again with
|
48 |
|
49 |
+
Nemesis_V1.1 (ex Negames), a Hermes flavored Negative Llama on Smarteaz abliterated steroids
|
50 |
+
- (More stiff and less creative than Doberman, though. Note : A mistake corrected : Hermes lorablated replace the vanilla version in Nemesis V1.1) :
|
51 |
+
- Nexesenex/Llama_3.x_70b_Nemesis_V1.1 : https://huggingface.co/Nexesenex/Llama_3.x_70b_Nemesis_V1.1 (less than.. 3.35 ppl 512 wiki-eng, -0.05 compared to Doberman_V1)
|
52 |
|
53 |
Evasion_V1 (ex Hermeva), a Hermes flavored Eva_01 on Smarteaz abliterated steroids (the most creative) :
|
54 |
- Nexesenex/Llama_3.x_70b_Evasion_V1 : https://huggingface.co/Nexesenex/Llama_3.x_70b_Evasion_V1 (less than 3.40 ppl 512 wiki-eng, -0.02 compared to Doberman_V1)
|
|
|
56 |
Trinity_V1, a merge of Evasion as base, Doberman and NegaTessTease to include a bit of Tess (to be tested) :
|
57 |
- Nexesenex/Llama_3.x_70b_Trinity_V1 : https://huggingface.co/Nexesenex/Llama_3.x_70b_Trinity_V1 (less than 3.40 ppl 512 wiki-eng, -0.03 compared to Doberman_V1)
|
58 |
|
59 |
+
Alas, I don't have under hand a Tess R1 Limerick lorablated. On the other hand, Mlabonne lorablated Hermes 3 70b Lorablated, and..
|
60 |
+
- I found 2 other models to make a "Hermes Block" and boost the creativity of the next revisions of my models, and not only the smarts.
|
61 |
+
- Here it comes : https://huggingface.co/Nexesenex/Llama_3.x_70b_Harpies_V1
|
62 |
+
|
63 |
+
---
|
64 |
+
# request for help
|
65 |
+
|
66 |
+
I (and many of us mergers, I believe) would need the following models abliterated to improve our merges, if Huihui-ai or someone could help :
|
67 |
+
|
68 |
+
- https://huggingface.co/SicariusSicariiStuff/Negative_LLAMA_70B
|
69 |
+
- https://huggingface.co/SentientAGI/Dobby-Unhinged-Llama-3.3-70B
|
70 |
+
|
71 |
+
I also tried to Lorablatize L3.1 70b Tess R1 Limerick and L3.1 70b Calme 2.3, but I'm not able to do so successfully (if someone could do that, it would be fantastic!)
|
72 |
+
|
73 |
+
- https://huggingface.co/migtissera/Tess-R1-Limerick-Llama-3.1-70B
|
74 |
+
- https://huggingface.co/MaziyarPanahi/calme-2.3-llama3.1-70b
|
75 |
+
- The Lora : https://huggingface.co/mlabonne/Llama-3-70B-Instruct-abliterated-LORA
|
76 |
+
- The yaml I used:
|
77 |
+
|
78 |
+
```yaml
|
79 |
+
base_model: meta-llama/Meta-Llama-3.1-70B-Instruct+Llama-3-70B-Instruct-abliterated-LORA
|
80 |
+
dtype: bfloat16
|
81 |
+
merge_method: task_arithmetic
|
82 |
+
parameters:
|
83 |
+
normalize: false
|
84 |
+
slices:
|
85 |
+
- sources:
|
86 |
+
- layer_range: [0, 80]
|
87 |
+
model: meta-llama/Meta-Llama-3.1-70B-Instruct+Llama-3-70B-Instruct-abliterated-LORA
|
88 |
+
parameters:
|
89 |
+
weight: 1.0
|
90 |
+
```yaml
|
91 |
|
92 |
---
|
93 |
# credits
|
94 |
|
95 |
Kudos go to the model authors, and to the Arcee / MergeKit folks, as well as to HF hosting the MergeKit App.
|
96 |
Also a big-up to SteelSkull, observing him cooking Nevoria decided me to try to make some merges by myself.
|
97 |
+
And to all those inspiring finetuners who give time, sometimes their money, a good time and some inspiration to others by tuning models.
|
98 |
|
99 |
---
|
100 |
# historic
|