---
license: apache-2.0
base_model:
- Qwen/QwQ-32B
base_model_relation: quantized
pipeline_tag: text-generation
---
Disclamer: I don't know what I'm doing. I am not an expert at quantizing.

Original Model: https://huggingface.co/Qwen/QwQ-32B
| QwQ 32B EXL2 | Size |
| --- | --- |
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-8.0bpw-exl2">**8.0bpw**</a> | 33.5 GB |
| ~7.0bpw~ | WIP |
| ~6.5bpw~ | WIP |
| ~6.0bpw~ | WIP |
| ~5.5bpw~ | WIP |
| ~5.0bpw~ | WIP |
| ~4.5bpw~ | WIP |
| ~4.0bpw~ | WIP |
| ~3.75bpw~ | WIP |
| ~3.5bpw~ | WIP |