Please queue KoboldAI/OPT-13B-Erebus for quant generation
When you have an opportunity can you please create quants for https://huggingface.co/KoboldAI/OPT-13B-Erebus
Thank you for your time!
Nice to see an old model from September 2022 with OPTForCausalLM being requested. This model was released before ChatGPT. Unlike MPT and Bloom llama.cpp never implemented OPT support as discussed in https://github.com/ggml-org/llama.cpp/issues/6473 because of which I did not queue this model as I know that it will unfortunately fail.
Oh! Sorry, I didn't realize it was that dated of a model. Thank you for the explanation! Is there another Erebus model or merge that mradermacher has quanted that you'd recommend?
Maybe try https://huggingface.co/mradermacher/Mistral-7B-Erebus-v3-i1-GGUF which are quants of https://huggingface.co/KoboldAI/Mistral-7B-Erebus-v3 which is the latest KoboldAI Erebus model. Keep in mind that mistral 7B is already relatively old so maybe you want to use https://huggingface.co/mradermacher/LLaMA2-13B-Erebus-v3-i1-GGUF based on https://huggingface.co/KoboldAI/LLaMA2-13B-Erebus-v3 instead if you want a slightly more intelligent model.
I queued https://huggingface.co/KoboldAI/GPT-NeoX-20B-Erebus which should feel very simular to the OPT-13B-Erebus model you requested. It has the same old style LLM feel to it as GPT-NeoX-20B got released in Februarry 2023. Generaly I realized the real lack of GPT-NeoX-20B based models we offer. We currently only have the original GPT-NeoX-20B with one derrivate.
If you are interested in GPT-NeoX-20B-Erebus you can check for progress at http://hf.tst.eu/status.html or regularly check the model summary page at https://hf.tst.eu/model#GPT-NeoX-20B-Erebus-GGUF for quants to appear.
Thank you, very valuable info! I'm going to go ahead and grab those suggested models and keep an eye out for the one you just queued. Appreciate you so much you absolute legend!
It's a bit of a shame that those OPT models are lost to llama.cpp.