Currently uploading with slow german internet

upload with 5MB/s running slowly....
- order: IQ4_XS Q4_K_M Q5_K_M Q6_K IQ4_NL IQ2_S IQ2_XS IQ2_XXS IQ3_S IQ3_XS IQ3_XXS Q4_K_S Q5_K_S Q8_0 Q4_0 IQ2_M IQ3_M IQ1_S;
based on https://huggingface.co/anthracite-core/Mistral-Small-3.1-24B-Instruct-2503-HF
quantized with imatrix via bf16
no vision support
BETA & not final! needs official configs!

GGUF

Model size

23.6B params

Architecture

llama

Hardware compatibility

1-bit

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for qwp4w3hyb/Mistral-Small-3.1-24B-Instruct-2503-HF-iMat-GGUF

Base model

Quantized

(14)

this model