Hugging Face's logo

TeeZee
/
Kyllene-34B-v1.1-GGUF

GGUF
Merge
Inference Endpoints
conversational
Model card Files Community
1

GGUF quants of TeeZee/Kyllene-34B-v1.1, remeber to set your max context length to proper length for your hardware, 4096 is fine. Default context length is 200k so it will eat RAM or VRAM like crazy if left unchecked.

Downloads last month
99
GGUF
Model size
34.4B params
Architecture
llama

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no library tag.

Collection including TeeZee/Kyllene-34B-v1.1-GGUF