Update README.md
Browse files
README.md
CHANGED
@@ -66,7 +66,7 @@ license: llama3.1
|
|
66 |
|
67 |
# AtlaAI/Selene-1-Mini-Llama-3.1-8B-GPTQ-W4A16
|
68 |
This model was quantised into a **4-bit** (W4A16) format using GPTQ from [`AtlaAI/Selene-1-Mini-Llama-3.1-8B`](https://huggingface.co/AtlaAI/Selene-1-Mini-Llama-3.1-8B).
|
69 |
-
This was done using vLLM's llm-compressor library (https://docs.vllm.ai/en/
|
70 |
|
71 |
Refer to the [original model card](https://huggingface.co/AtlaAI/Selene-1-Mini-Llama-3.1-8B) for more details on the model.
|
72 |
|
|
|
66 |
|
67 |
# AtlaAI/Selene-1-Mini-Llama-3.1-8B-GPTQ-W4A16
|
68 |
This model was quantised into a **4-bit** (W4A16) format using GPTQ from [`AtlaAI/Selene-1-Mini-Llama-3.1-8B`](https://huggingface.co/AtlaAI/Selene-1-Mini-Llama-3.1-8B).
|
69 |
+
This was done using vLLM's llm-compressor library (https://docs.vllm.ai/en/latest/features/quantization/int4.html)
|
70 |
|
71 |
Refer to the [original model card](https://huggingface.co/AtlaAI/Selene-1-Mini-Llama-3.1-8B) for more details on the model.
|
72 |
|