Hugging Face's logo

kaitchup
/
Llama-3.3-70B-Instruct-AutoRound-GPTQ-4bit

Text Generation
Transformers
Safetensors
English
llama
auto-gptq
AutoRound
conversational
text-generation-inference
Inference Endpoints
4-bit precision
gptq
Model card Files Community
1

Model Details

This is meta-llama/Llama-3.3-70B-Instruct quantized with AutoRound (symmetric quantization) and serialized with the GPTQ format in 4-bit. The model has been created, tested, and evaluated by The Kaitchup.

Details on the quantization process and how to use the model here: How to Quantize and Run Llama 3.3 70B Instruct on Your GPU

Llama 3.3 70B Instruct_ Zero-Shot MMLU (Accuracy).png Llama 3.3 70B Instruct_ Model Size.png

  • Developed by: The Kaitchup
  • Language(s) (NLP): English
  • License: Llama 3.3
Downloads last month
6,102
Safetensors
Model size
11.3B params
Tensor type
FP16
·
I32
·
Inference Providers NEW
Text Generation
This model is not currently available via any of the supported third-party Inference Providers, and the model is not deployed on the HF Inference API.

Collection including kaitchup/Llama-3.3-70B-Instruct-AutoRound-GPTQ-4bit