Update README.md
Browse files
README.md
CHANGED
@@ -22,15 +22,18 @@ language:
|
|
22 |
|
23 |

|
24 |
|
25 |
-
This is a dpo aligned merge of
|
26 |
-
It is a merge of the best german 7B models with 7b parameters as a dare_ties merge.
|
27 |
Since the original models based on mistral - three of them on the brilliant german LeoLM/leo-mistral-hessianai-7b - they are reunited in this merged model.
|
28 |
-
Therefore the name, no nationalist ideas involved. To improve result quality they are dpo-trained with a german translation of intel-orca-dpo
|
29 |
-
using our german fork of [LLaMA-Factory](https://github.com/mayflower/LLaMA-Factory-de).
|
30 |
|
31 |
-
|
|
|
32 |
|
33 |
-
|
|
|
|
|
|
|
|
|
34 |
|
35 |
```json
|
36 |
{
|
@@ -57,7 +60,6 @@ Wiedervereinigung-7b itself is a [LazyMergekit](https://colab.research.google.c
|
|
57 |
* [VAGOsolutions/SauerkrautLM-7b-v1-mistral](https://huggingface.co/VAGOsolutions/SauerkrautLM-7b-v1-mistral)
|
58 |
* [malteos/hermeo-7b](https://huggingface.co/malteos/hermeo-7b)
|
59 |
|
60 |
-
All the actual heavylifting has been done by the creators of these models.
|
61 |
|
62 |
## 🧩 Configuration
|
63 |
|
|
|
22 |
|
23 |

|
24 |
|
25 |
+
This is a dpo aligned merge of our favourite german models, scoring 7.11 on the mt-bench-de average.
|
|
|
26 |
Since the original models based on mistral - three of them on the brilliant german LeoLM/leo-mistral-hessianai-7b - they are reunited in this merged model.
|
27 |
+
Therefore the name, no nationalist ideas involved :-). To improve result quality they are dpo-trained with a german translation of intel-orca-dpo we added support for 10 german datasets to [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory).
|
|
|
28 |
|
29 |
+
Kudos to the authors of the original models at [DiscoResearch](https://huggingface.co/DiscoResearch) and [VAGOsolutions](https://huggingface.co/VAGOsolutions), [Malte Ostendorff](https://huggingface.co/malteos)
|
30 |
+
and [Matthias Uhlig](https://huggingface.co/DRXD1000). We are your fan club.
|
31 |
|
32 |
+
This model was brought to you and the nvidia bill was paid by [Mayflower GmbH](https://mayflower.de/).
|
33 |
+
|
34 |
+
## Benchmark results: mt-bench-de
|
35 |
+
|
36 |
+
Is the merged model alone already good? Well, of course. But it is even better with the help of some dpo tuning.
|
37 |
|
38 |
```json
|
39 |
{
|
|
|
60 |
* [VAGOsolutions/SauerkrautLM-7b-v1-mistral](https://huggingface.co/VAGOsolutions/SauerkrautLM-7b-v1-mistral)
|
61 |
* [malteos/hermeo-7b](https://huggingface.co/malteos/hermeo-7b)
|
62 |
|
|
|
63 |
|
64 |
## 🧩 Configuration
|
65 |
|