Commit
·
ddf5ad2
1
Parent(s):
31ae1b5
Update README.md
Browse files
README.md
CHANGED
|
@@ -19,3 +19,16 @@ base_model: meta-llama/LlamaGuard-7b
|
|
| 19 |
## Model Description
|
| 20 |
|
| 21 |
The model [meta-llama/LlamaGuard-7b](https://huggingface.co/meta-llama/LlamaGuard-7b) was quantized to 4bit, group_size 128, and act-order=True with auto-gptq integration in transformers (https://huggingface.co/blog/gptq-integration).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
## Model Description
|
| 20 |
|
| 21 |
The model [meta-llama/LlamaGuard-7b](https://huggingface.co/meta-llama/LlamaGuard-7b) was quantized to 4bit, group_size 128, and act-order=True with auto-gptq integration in transformers (https://huggingface.co/blog/gptq-integration).
|
| 22 |
+
|
| 23 |
+
## Evaluation
|
| 24 |
+
|
| 25 |
+
To evaluate the qunatized model and compare it with the full precision model, I performed binary classification on the "toxicity" label from the ~5k samples test set of lmsys/toxic-chat.
|
| 26 |
+
|
| 27 |
+
📊 Full Precision Model:
|
| 28 |
+
|
| 29 |
+
Average Precision Score: 0.3625
|
| 30 |
+
|
| 31 |
+
📊 4-bit Quantized Model:
|
| 32 |
+
|
| 33 |
+
Average Precision Score: 0.3450
|
| 34 |
+
|