rombodawg/Llama-3-8B-Base-Coder-v3.5-10k AWQ

** PROCESSING .... ETA 30mins **

About AWQ

AWQ is an efficient, accurate and blazing-fast low-bit weight quantization method, currently supporting 4-bit quantization. Compared to GPTQ, it offers faster Transformers-based inference with equivalent or better quality compared to the most commonly used GPTQ settings.

AWQ models are currently supported on Linux and Windows, with NVidia GPUs only. macOS users: please use GGUF models instead.

It is supported by:

Downloads last month
4
Safetensors
Model size
1.98B params
Tensor type
I32
·
FP16
·
Inference Providers NEW
This model is not currently available via any of the supported third-party Inference Providers, and HF Inference API has been turned off for this model.

Collection including solidrust/Llama-3-8B-Base-Coder-v3.5-10k-AWQ