jianqing666
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -26,6 +26,17 @@ Models input text only.
|
|
26 |
Models output text only.
|
27 |
## Model Evaluation Results
|
28 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
29 |
<!-- Benchmark evaluation on [Arabic MMLU](https://github.com/FreedomIntelligence/AceGPT) are conducted using accuracy scores as metrics, following the evaluation framework available at https://github.com/FreedomIntelligence/AceGPT/tree/main. -->
|
30 |
| | STEM | Humanities | Social Sciences | Others | Average |
|
31 |
|------------------|------|------|------|------|------|
|
|
|
26 |
Models output text only.
|
27 |
## Model Evaluation Results
|
28 |
|
29 |
+
| Model | Avg. | [ArabicMMLU]((https://github.com/mbzuai-nlp/ArabicMMLU)) | [ArabicMMLU]((https://github.com/mbzuai-nlp/ArabicMMLU)) | ARC | EXAMs | ACVA (clean) | ACVA (all) |
|
30 |
+
|---------------|--------|----------------|-----------------------|-------|-------|--------------|------------|
|
31 |
+
| MgGPT-7B | 45.19 | 34.03 | 37.00 | 17.49 | 37.28 | 72.69 | 72.67 |
|
32 |
+
| MgGPT-8B | 58.94 | 48.41 | 50.17 | 49.91 | 46.15 | 80.14 | 78.84 |
|
33 |
+
| MgGPT-13B | 52.11 | 40.95 | 47.60 | 31.57 | 35.10 | 79.45 | 78.01 |
|
34 |
+
| MgGPT-32B | 68.75 | 58.71 | 65.67 | 71.69 | 52.74 | 82.66 | 81.04 |
|
35 |
+
| MgGPT-70B | 72.62 | 65.19 | 67.71 | 80.93 | 56.19 | 84.79 | 80.93 |
|
36 |
+
| Jais-30B-v3 | 57.02 | 43.42 | 44.47 | 45.56 | 45.70 | 83.39 | 79.51 |
|
37 |
+
| GPT-3.5 | 60.71 | 49.07 | 57.70 | 60.24 | 45.93 | 74.45 | 76.88 |
|
38 |
+
| GPT-4 | 74.08 | 65.06 | 72.50 | 85.67 | 57.76 | 84.06 | 79.43 |
|
39 |
+
|
40 |
<!-- Benchmark evaluation on [Arabic MMLU](https://github.com/FreedomIntelligence/AceGPT) are conducted using accuracy scores as metrics, following the evaluation framework available at https://github.com/FreedomIntelligence/AceGPT/tree/main. -->
|
41 |
| | STEM | Humanities | Social Sciences | Others | Average |
|
42 |
|------------------|------|------|------|------|------|
|