enable gguf

#3
by ehartford - opened

hello @francislabounty
Can you please enable gguf?
I would really like to try sparsetral.
https://github.com/ggerganov/llama.cpp/issues/5365

GPTQ and AWQ too, I imagine changes need to be made to upstream for it to work

It's interesting that exl2 just supported it out of the box

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment