gemma-3-12b-it-Q8_0.gguf appears to be broken
#1
by
3Simplex
- opened
Using the latest llama.cpp with your gemma3 merge.
I checked the SHA256 and it matches.
I tested the Q_4KM after trying the Q8_0. The output of the Q_4KM you uploaded is functional. I then downloaded your f16 and quantized to Q8_0 and Q6_k. The Q6_k worked too which means your f16 is okay. Quantizing this model to Q8_0 appears to be broken in llama.cpp at the moment.
I this is just a Llama.cpp problem. I just loaded a known working model and got similar results.
3Simplex
changed discussion status to
closed