can you quantize this?
#649
by
blankreg
- opened
noob question, can a pt-16bit model be converted into GGUFs ?
the model is 10B, but 16bit is quite big and I'd like something between Q5/Q8...
https://huggingface.co/agentica/JSL-MedFalcon-10B-Merged-v2-Slerp-pt-16bit/tree/main
It's queued. I think it should work as it uses LlamaForCausalLM and it doesn't appear to already be quantized but we will know soon.
You can check for progress at http://hf.tst.eu/status.html or regularly check the model
summary page at https://hf.tst.eu/model#JSL-MedFalcon-10B-Merged-v2-Slerp-pt-16bit-GGUF for quants to appear.
mradermacher
changed discussion status to
closed
thanks!