Intel
/

gpt-j-6b-int4-inc

Model card Files Files and versions Community

yintongl commited on May 21, 2024

Commit

dd59eb5

·

verified ·

1 Parent(s): 11fd213

Update README.md

Files changed (1) hide show

README.md +10 -10

README.md CHANGED Viewed

@@ -51,19 +51,19 @@ lm_eval --model hf --model_args pretrained="Intel/gpt-j-6b-int4-inc",autogptq=Tr
 ```
-| Metric         | FP16   | INT4   |
 | -------------- | ------ | ------ |
-| Avg.           | 0.4998 | 0.4993 |
-| mmlu           | 0.2694 | 0.2750 |
-| lambada_openai | 0.6831 | 0.6845 |
-| hellaswag      | 0.4953 | 0.4897 |
 | winogrande     | 0.6409 | 0.6425 |
-| piqa           | 0.7541 | 0.7448 |
 | truthfulqa_mc1 | 0.2020 | 0.2093 |
-| openbookqa     | 0.2900 | 0.2820 |
-| boolq          | 0.6544 | 0.6511 |
-| arc_easy       | 0.6692 | 0.6709 |
-| arc_challenge  | 0.3396 | 0.3430 |

 ```
+| Metric         | BF16   | INT4   |
 | -------------- | ------ | ------ |
+| Avg.           | 0.4986 | 0.4993 |
+| mmlu           | 0.2594 | 0.2750 |
+| lambada_openai | 0.6827 | 0.6845 |
+| hellaswag      | 0.4954 | 0.4897 |
 | winogrande     | 0.6409 | 0.6425 |
+| piqa           | 0.7535 | 0.7448 |
 | truthfulqa_mc1 | 0.2020 | 0.2093 |
+| openbookqa     | 0.2880 | 0.2820 |
+| boolq          | 0.6550 | 0.6511 |
+| arc_easy       | 0.6700 | 0.6709 |
+| arc_challenge  | 0.3387 | 0.3430 |