We have quantised the model in 2-bit to make it inferenceable in low-end GPU cards at scale. It was achieved thanks to llama.cpp library.

GGUF

Model size

32.8B params

Architecture

qwen2

Hardware compatibility

2-bit

Model tree for sleeping-ai/Intellect-2-TQ2-0

Base model

Quantized

(14)

this model