Commit
·
717c99d
1
Parent(s):
d55f6e3
Update README.md
Browse files
README.md
CHANGED
|
@@ -34,10 +34,6 @@ tags:
|
|
| 34 |
|
| 35 |
*Image drawn by GPT-4 DALL·E 3* TL;DR: Perhaps this 7B model, better than all existing models <= 33B, in most quantitative evaluations...
|
| 36 |
|
| 37 |
-
# Please Stop Using WRONG unofficial quant models unless you know what you're doing
|
| 38 |
-
|
| 39 |
-
GPTQ quants require a good dataset for calibration, and the default C4 dataset is not capable.
|
| 40 |
-
|
| 41 |
**llama.cpp GGUF models**
|
| 42 |
GPT2Tokenizer fixed by [Kerfuffle](https://github.com/KerfuffleV2) on [https://github.com/ggerganov/llama.cpp/pull/3743](https://github.com/ggerganov/llama.cpp/pull/3743), new models are reuploaded.
|
| 43 |
|
|
@@ -72,7 +68,7 @@ other ACC: 70.04
|
|
| 72 |
|
| 73 |
social ACC: 72.41
|
| 74 |
|
| 75 |
-
**AVERAGE ACC:63.82** (Outperforms / Equal to the best Mistral-7B Chat-style fine-tunes, and ALL other models under 33B.)
|
| 76 |
|
| 77 |
## CEval (Val):
|
| 78 |
STEM acc: 61.67
|
|
@@ -85,7 +81,7 @@ Other acc: 68.35
|
|
| 85 |
|
| 86 |
Hard acc:48.03
|
| 87 |
|
| 88 |
-
**AVERAGE acc:70.27** (Outperforms ALL 7B models currently.)
|
| 89 |
|
| 90 |
## GSM8K
|
| 91 |
|
|
@@ -126,7 +122,7 @@ STEM准确率:56.83
|
|
| 126 |
|
| 127 |
社会学准确率:72.41
|
| 128 |
|
| 129 |
-
**平均准确率:63.82** (优于/平于最好的 Mistral-7B
|
| 130 |
|
| 131 |
## CEval(验证集):
|
| 132 |
STEM准确率:61.67
|
|
@@ -139,7 +135,7 @@ STEM准确率:61.67
|
|
| 139 |
|
| 140 |
困难准确率:48.03
|
| 141 |
|
| 142 |
-
**平均准确率:70.27** (优于当前所有7B
|
| 143 |
|
| 144 |
## GSM8K
|
| 145 |
|
|
|
|
| 34 |
|
| 35 |
*Image drawn by GPT-4 DALL·E 3* TL;DR: Perhaps this 7B model, better than all existing models <= 33B, in most quantitative evaluations...
|
| 36 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
**llama.cpp GGUF models**
|
| 38 |
GPT2Tokenizer fixed by [Kerfuffle](https://github.com/KerfuffleV2) on [https://github.com/ggerganov/llama.cpp/pull/3743](https://github.com/ggerganov/llama.cpp/pull/3743), new models are reuploaded.
|
| 39 |
|
|
|
|
| 68 |
|
| 69 |
social ACC: 72.41
|
| 70 |
|
| 71 |
+
**AVERAGE ACC:63.82** (Outperforms / Equal to the best Mistral-7B Chat-style fine-tunes, ChatGLM3-6B and ALL other models under 33B.)
|
| 72 |
|
| 73 |
## CEval (Val):
|
| 74 |
STEM acc: 61.67
|
|
|
|
| 81 |
|
| 82 |
Hard acc:48.03
|
| 83 |
|
| 84 |
+
**AVERAGE acc:70.27** (Outperforms ALL 7B models currently, including ChatGLM3-6B.)
|
| 85 |
|
| 86 |
## GSM8K
|
| 87 |
|
|
|
|
| 122 |
|
| 123 |
社会学准确率:72.41
|
| 124 |
|
| 125 |
+
**平均准确率:63.82** (优于/平于最好的 Mistral-7B 聊天格式的微调,ChatGLM3-6B 和其余的33B及以下模型。)
|
| 126 |
|
| 127 |
## CEval(验证集):
|
| 128 |
STEM准确率:61.67
|
|
|
|
| 135 |
|
| 136 |
困难准确率:48.03
|
| 137 |
|
| 138 |
+
**平均准确率:70.27** (优于当前所有7B模型,包括 ChatGLM3-6B)
|
| 139 |
|
| 140 |
## GSM8K
|
| 141 |
|