---
license: apache-2.0
base_model:
- Qwen/QwQ-32B
base_model_relation: quantized
pipeline_tag: text-generation
---
Disclamer: I don't know what I'm doing.

Original Model: https://huggingface.co/Qwen/QwQ-32B
| QwQ 32B EXL2 | Size |
| --- | --- |
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-8.0bpw-exl2">**8.0bpw**</a> | 33.5 GB |
| ~7.0bpw~ | WIP |
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-6.5bpw-exl2">**6.5bpw**</a> | 27.5 GB |
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-6.0bpw-exl2">**6.0bpw**</a> | 25.6 GB |
| ~5.5bpw~ | WIP |
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-5.0bpw-exl2">**5.0bpw**</a> | 21.7 GB |
| ~4.5bpw~ | WIP |
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-4.0bpw-exl2">**4.0bpw**</a> | 17.8 GB |
| ~3.75bpw~ | WIP |
| ~3.5bpw~ | WIP |