File size: 1,122 Bytes
c56d3d6 5c16d57 c56d3d6 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
---
license: apache-2.0
base_model:
- Qwen/QwQ-32B
base_model_relation: quantized
pipeline_tag: text-generation
---
Disclamer: I don't know what I'm doing.
Original Model: https://huggingface.co/Qwen/QwQ-32B
| QwQ 32B EXL2 | Size |
| --- | --- |
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-8.0bpw-exl2">**8.0bpw**</a> | 33.5 GB |
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-7.0bpw-exl2">**7.0bpw**</a> | 29.6 GB |
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-6.5bpw-exl2">**6.5bpw**</a> | 27.5 GB |
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-6.0bpw-exl2">**6.0bpw**</a> | 25.6 GB |
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-5.5bpw-exl2">**5.5bpw**</a> | 23.6 GB |
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-5.0bpw-exl2">**5.0bpw**</a> | 21.7 GB |
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-4.5bpw-exl2">**4.5bpw**</a> | 19.7 GB |
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-4.0bpw-exl2">**4.0bpw**</a> | 17.8 GB |
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-3.75bpw-exl2">**3.75bpw**</a> | 16.8 GB |
| ~3.5bpw~ | WIP | |