jianqing666 commited on
Commit
1a9c4c0
·
verified ·
1 Parent(s): dbddb7e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -0
README.md CHANGED
@@ -26,6 +26,17 @@ Models input text only.
26
  Models output text only.
27
  ## Model Evaluation Results
28
 
 
 
 
 
 
 
 
 
 
 
 
29
  <!-- Benchmark evaluation on [Arabic MMLU](https://github.com/FreedomIntelligence/AceGPT) are conducted using accuracy scores as metrics, following the evaluation framework available at https://github.com/FreedomIntelligence/AceGPT/tree/main. -->
30
  | | STEM | Humanities | Social Sciences | Others | Average |
31
  |------------------|------|------|------|------|------|
 
26
  Models output text only.
27
  ## Model Evaluation Results
28
 
29
+ | Model | Avg. | [ArabicMMLU]((https://github.com/mbzuai-nlp/ArabicMMLU)) | [ArabicMMLU]((https://github.com/mbzuai-nlp/ArabicMMLU)) | ARC | EXAMs | ACVA (clean) | ACVA (all) |
30
+ |---------------|--------|----------------|-----------------------|-------|-------|--------------|------------|
31
+ | MgGPT-7B | 45.19 | 34.03 | 37.00 | 17.49 | 37.28 | 72.69 | 72.67 |
32
+ | MgGPT-8B | 58.94 | 48.41 | 50.17 | 49.91 | 46.15 | 80.14 | 78.84 |
33
+ | MgGPT-13B | 52.11 | 40.95 | 47.60 | 31.57 | 35.10 | 79.45 | 78.01 |
34
+ | MgGPT-32B | 68.75 | 58.71 | 65.67 | 71.69 | 52.74 | 82.66 | 81.04 |
35
+ | MgGPT-70B | 72.62 | 65.19 | 67.71 | 80.93 | 56.19 | 84.79 | 80.93 |
36
+ | Jais-30B-v3 | 57.02 | 43.42 | 44.47 | 45.56 | 45.70 | 83.39 | 79.51 |
37
+ | GPT-3.5 | 60.71 | 49.07 | 57.70 | 60.24 | 45.93 | 74.45 | 76.88 |
38
+ | GPT-4 | 74.08 | 65.06 | 72.50 | 85.67 | 57.76 | 84.06 | 79.43 |
39
+
40
  <!-- Benchmark evaluation on [Arabic MMLU](https://github.com/FreedomIntelligence/AceGPT) are conducted using accuracy scores as metrics, following the evaluation framework available at https://github.com/FreedomIntelligence/AceGPT/tree/main. -->
41
  | | STEM | Humanities | Social Sciences | Others | Average |
42
  |------------------|------|------|------|------|------|