Other formats?

#1
by wise-time - opened

Would really like to see a q8_0 version, I find this available on most other webui compatible language models.

rustformers org

@wise-time I decided to exclude q8_0 for now as the difference in performance between q5_1 and q8_0 shouldn't be that great according to llama.cpp.
If the new model format is defined and finalized i will probably uplaod all models in all available quantization formats. But as it's not clear when this will happen i'm waiting for now.

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment