|
--- |
|
license: apache-2.0 |
|
base_model: |
|
- Qwen/QwQ-32B |
|
base_model_relation: quantized |
|
pipeline_tag: text-generation |
|
--- |
|
Disclaimer: I don't know what I'm doing. |
|
|
|
Original Model: https://huggingface.co/Qwen/QwQ-32B |
|
| QwQ 32B EXL2 | Size | |
|
| --- | --- | |
|
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-8.0bpw-exl2">**8.0bpw**</a> | 33.5 GB | |
|
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-7.0bpw-exl2">**7.0bpw**</a> | 29.6 GB | |
|
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-6.5bpw-exl2">**6.5bpw**</a> | 27.5 GB | |
|
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-6.0bpw-exl2">**6.0bpw**</a> | 25.6 GB | |
|
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-5.5bpw-exl2">**5.5bpw**</a> | 23.6 GB | |
|
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-5.0bpw-exl2">**5.0bpw**</a> | 21.7 GB | |
|
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-4.5bpw-exl2">**4.5bpw**</a> | 19.7 GB | |
|
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-4.0bpw-exl2">**4.0bpw**</a> | 17.8 GB | |
|
| <a href="https://huggingface.co/cshared/Qwen-QwQ-32B-3.75bpw-exl2">**3.75bpw**</a> | 16.8 GB | |
|
| ~3.5bpw~ | WIP | |
