Update README.md
Browse files
README.md
CHANGED
@@ -5,10 +5,9 @@ datasets:
|
|
5 |
- trollek/Mouse-Diffusion-Instruct
|
6 |
- trollek/CodeMouse
|
7 |
- trollek/Panoia-v01
|
8 |
-
- WhiteRabbitNeo/WRN-Chapter-1
|
9 |
-
- WhiteRabbitNeo/WRN-Chapter-2
|
10 |
- jondurbin/airoboros-3.2
|
11 |
- mlabonne/orpo-dpo-mix-40k
|
|
|
12 |
language:
|
13 |
- en
|
14 |
---
|
@@ -17,8 +16,6 @@ language:
|
|
17 |
|
18 |
A brand spanking new model with a silly name. Brought to you by Anoia, the Goddess of Things That Get Stuck in Drawers, and the psychologial damage of having optic nerves.
|
19 |
|
20 |
-
It seems like the more LoRAs we cake on a base model, the more they hallucinate. (source to be added). So in the last installment of NinjaMouse2 I present to you, a delightful human being, a finetuned danube2 model that have been extended without the Llama Pro method of interleaving new layers/transformer blocks. Just regular ol' [mergekit](https://github.com/arcee-ai/mergekit).
|
21 |
-
|
22 |
```yaml
|
23 |
slices:
|
24 |
- sources:
|
@@ -32,3 +29,18 @@ dtype: bfloat16
|
|
32 |
```
|
33 |
|
34 |
I tried several other layer configerations but this one had the least negative effects on Hellaswag and Winogrande evals. Next up I made different finetunes of danube2 using Badam and merged them with the Model Stock method.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
5 |
- trollek/Mouse-Diffusion-Instruct
|
6 |
- trollek/CodeMouse
|
7 |
- trollek/Panoia-v01
|
|
|
|
|
8 |
- jondurbin/airoboros-3.2
|
9 |
- mlabonne/orpo-dpo-mix-40k
|
10 |
+
- Magpie-Align/Magpie-Air-300K-Filtered
|
11 |
language:
|
12 |
- en
|
13 |
---
|
|
|
16 |
|
17 |
A brand spanking new model with a silly name. Brought to you by Anoia, the Goddess of Things That Get Stuck in Drawers, and the psychologial damage of having optic nerves.
|
18 |
|
|
|
|
|
19 |
```yaml
|
20 |
slices:
|
21 |
- sources:
|
|
|
29 |
```
|
30 |
|
31 |
I tried several other layer configerations but this one had the least negative effects on Hellaswag and Winogrande evals. Next up I made different finetunes of danube2 using Badam and merged them with the Model Stock method.
|
32 |
+
|
33 |
+
It uses the default template of danube2:
|
34 |
+
|
35 |
+
```jinja2
|
36 |
+
<|prompt|>{{instruction}}</s><|answer|>{{response}}</s>
|
37 |
+
```
|
38 |
+
|
39 |
+
And can be used with the Ollama ComfyUI extension:
|
40 |
+
|
41 |
+
[<img src="https://huggingface.co/trollek/NinjaMouse2-2.5B-v0.2/resolve/main/ollama_comfyui.png" width="800px"/>](https://huggingface.co/trollek/NinjaMouse2-2.5B-v0.2/resolve/main/ollama_comfyui.png)
|
42 |
+
|
43 |
+
[<img src="https://huggingface.co/trollek/NinjaMouse2-2.5B-v0.2/resolve/main/tophat_cat.png" width="800px"/>](https://huggingface.co/trollek/NinjaMouse2-2.5B-v0.2/resolve/main/tophat_cat.png)
|
44 |
+
|
45 |
+
|
46 |
+
Trying to fine-tune the chat model even further was a mistake. This is the last mouse ninja.
|