Nexesenex commited on
Commit
54e9c56
·
verified ·
1 Parent(s): f07f18f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +40 -5
README.md CHANGED
@@ -24,7 +24,10 @@ Bingo again. I hit 3.45 ppl512 wikieng, 62+ or ARC-C, and 82+ on ARC-E. Absolute
24
 
25
  No cheating, no contaminating, just the wonderful MergeKit model-stock merge technique leveraged to a new level (compared to what I already saw being done, at least).
26
 
27
- Next projects will involve that model as the "smarts pillar" of further merges, aimed at any use case.
 
 
 
28
 
29
  Edit : the mothodology I use is actually partly rediscovered hot water.
30
 
@@ -43,8 +46,9 @@ The adventure continues with Doberman_V1, a Hermes flavored Dobby on Smarteaz ab
43
 
44
  And the saga continues again with
45
 
46
- Nemesis_V1 (ex Negames), a Hermes flavored Negative Llama on Smarteaz abliterated steroids (More stiff and less creative than Doberman, though) :
47
- - Nexesenex/Llama_3.x_70b_Nemesis_V1 : https://huggingface.co/Nexesenex/Llama_3.x_70b_Nemesis_V1 (less than.. 3.35 ppl 512 wiki-eng, -0.05 compared to Doberman_V1)
 
48
 
49
  Evasion_V1 (ex Hermeva), a Hermes flavored Eva_01 on Smarteaz abliterated steroids (the most creative) :
50
  - Nexesenex/Llama_3.x_70b_Evasion_V1 : https://huggingface.co/Nexesenex/Llama_3.x_70b_Evasion_V1 (less than 3.40 ppl 512 wiki-eng, -0.02 compared to Doberman_V1)
@@ -52,14 +56,45 @@ Evasion_V1 (ex Hermeva), a Hermes flavored Eva_01 on Smarteaz abliterated steroi
52
  Trinity_V1, a merge of Evasion as base, Doberman and NegaTessTease to include a bit of Tess (to be tested) :
53
  - Nexesenex/Llama_3.x_70b_Trinity_V1 : https://huggingface.co/Nexesenex/Llama_3.x_70b_Trinity_V1 (less than 3.40 ppl 512 wiki-eng, -0.03 compared to Doberman_V1)
54
 
55
- I think that most models can be tweaked the same way, with triple stock merges interlacing intruct finetunes and base finetunes.
56
- - This, gaining overall "intelligence" and "quality" at the cost of a bit of its initial instructions, flavor and "personality".
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
57
 
58
  ---
59
  # credits
60
 
61
  Kudos go to the model authors, and to the Arcee / MergeKit folks, as well as to HF hosting the MergeKit App.
62
  Also a big-up to SteelSkull, observing him cooking Nevoria decided me to try to make some merges by myself.
 
63
 
64
  ---
65
  # historic
 
24
 
25
  No cheating, no contaminating, just the wonderful MergeKit model-stock merge technique leveraged to a new level (compared to what I already saw being done, at least).
26
 
27
+ Next projects will involve that model as the "smarts pillar/Block" of further merges, aimed at any use case.
28
+
29
+ I think that most models can be tweaked the same way, with triple stock merges interlacing intruct finetunes and base finetunes.
30
+ - This, gaining overall "intelligence" and "quality" at the cost of a bit of its initial instructions, flavor and "personality".
31
 
32
  Edit : the mothodology I use is actually partly rediscovered hot water.
33
 
 
46
 
47
  And the saga continues again with
48
 
49
+ Nemesis_V1.1 (ex Negames), a Hermes flavored Negative Llama on Smarteaz abliterated steroids
50
+ - (More stiff and less creative than Doberman, though. Note : A mistake corrected : Hermes lorablated replace the vanilla version in Nemesis V1.1) :
51
+ - Nexesenex/Llama_3.x_70b_Nemesis_V1.1 : https://huggingface.co/Nexesenex/Llama_3.x_70b_Nemesis_V1.1 (less than.. 3.35 ppl 512 wiki-eng, -0.05 compared to Doberman_V1)
52
 
53
  Evasion_V1 (ex Hermeva), a Hermes flavored Eva_01 on Smarteaz abliterated steroids (the most creative) :
54
  - Nexesenex/Llama_3.x_70b_Evasion_V1 : https://huggingface.co/Nexesenex/Llama_3.x_70b_Evasion_V1 (less than 3.40 ppl 512 wiki-eng, -0.02 compared to Doberman_V1)
 
56
  Trinity_V1, a merge of Evasion as base, Doberman and NegaTessTease to include a bit of Tess (to be tested) :
57
  - Nexesenex/Llama_3.x_70b_Trinity_V1 : https://huggingface.co/Nexesenex/Llama_3.x_70b_Trinity_V1 (less than 3.40 ppl 512 wiki-eng, -0.03 compared to Doberman_V1)
58
 
59
+ Alas, I don't have under hand a Tess R1 Limerick lorablated. On the other hand, Mlabonne lorablated Hermes 3 70b Lorablated, and..
60
+ - I found 2 other models to make a "Hermes Block" and boost the creativity of the next revisions of my models, and not only the smarts.
61
+ - Here it comes : https://huggingface.co/Nexesenex/Llama_3.x_70b_Harpies_V1
62
+
63
+ ---
64
+ # request for help
65
+
66
+ I (and many of us mergers, I believe) would need the following models abliterated to improve our merges, if Huihui-ai or someone could help :
67
+
68
+ - https://huggingface.co/SicariusSicariiStuff/Negative_LLAMA_70B
69
+ - https://huggingface.co/SentientAGI/Dobby-Unhinged-Llama-3.3-70B
70
+
71
+ I also tried to Lorablatize L3.1 70b Tess R1 Limerick and L3.1 70b Calme 2.3, but I'm not able to do so successfully (if someone could do that, it would be fantastic!)
72
+
73
+ - https://huggingface.co/migtissera/Tess-R1-Limerick-Llama-3.1-70B
74
+ - https://huggingface.co/MaziyarPanahi/calme-2.3-llama3.1-70b
75
+ - The Lora : https://huggingface.co/mlabonne/Llama-3-70B-Instruct-abliterated-LORA
76
+ - The yaml I used:
77
+
78
+ ```yaml
79
+ base_model: meta-llama/Meta-Llama-3.1-70B-Instruct+Llama-3-70B-Instruct-abliterated-LORA
80
+ dtype: bfloat16
81
+ merge_method: task_arithmetic
82
+ parameters:
83
+ normalize: false
84
+ slices:
85
+ - sources:
86
+ - layer_range: [0, 80]
87
+ model: meta-llama/Meta-Llama-3.1-70B-Instruct+Llama-3-70B-Instruct-abliterated-LORA
88
+ parameters:
89
+ weight: 1.0
90
+ ```yaml
91
 
92
  ---
93
  # credits
94
 
95
  Kudos go to the model authors, and to the Arcee / MergeKit folks, as well as to HF hosting the MergeKit App.
96
  Also a big-up to SteelSkull, observing him cooking Nevoria decided me to try to make some merges by myself.
97
+ And to all those inspiring finetuners who give time, sometimes their money, a good time and some inspiration to others by tuning models.
98
 
99
  ---
100
  # historic