YAML Metadata
Warning:
empty or missing yaml metadata in repo card
(https://huggingface.co/docs/hub/model-cards#model-card-metadata)
GGUF quants for : https://huggingface.co/alchemonaut/QuartetAnemoi-70B-t0.0001
Available : Q3_K_M, IQ3_XXS. Otw : IQ2_XS
I recommand you folks to try this model, because it's quite an efficient merge of Miqu, WinterGoddess, AuroraNights, and XWin.
The Theta Rope 1,000,000 of Miqu, hence the 32k context, is functional up to 16k accordingly to my tests, and probably above (I need a smaller quant to test, which is otw).
To use it with a quantized KV cache for a higher context, here's KoboldCPP Frankenstein version with several different KV cache quantization lebels to chose from : https://github.com/Nexesenex/kobold.cpp/releases
- Downloads last month
- 723
Hardware compatibility
Log In
to view the estimation
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support
HF Inference deployability: The model has no library tag.