|
--- |
|
base_model: Qwen/QwQ-32B |
|
pipeline_tag: text-generation |
|
inference: true |
|
language: |
|
- en |
|
license: apache-2.0 |
|
model_creator: Qwen |
|
model_name: QwQ-32B |
|
model_type: qwen2 |
|
quantized_by: brittlewis12 |
|
tags: |
|
- reasoning |
|
- qwen2 |
|
--- |
|
|
|
# QwQ 32B GGUF |
|
|
|
<video loop="" src="https://qianwen-res.oss-accelerate-overseas.aliyuncs.com/QwQ.mp4" autoplay=""></video> |
|
|
|
**Original model**: [QwQ 32B](https://huggingface.co/Qwen/QwQ-32B) |
|
|
|
**Model creator**: [Qwen](https://huggingface.co/Qwen) |
|
|
|
> QwQ is the reasoning model of the Qwen series. Compared with conventional instruction-tuned models, QwQ, which is capable of thinking and reasoning, can achieve significantly enhanced performance in downstream tasks, especially hard problems. QwQ-32B is the medium-sized reasoning model, which is capable of achieving competitive performance against state-of-the-art reasoning models, e.g., DeepSeek-R1, o1-mini. |
|
|
|
This repo contains GGUF format model files for Qwen’s QwQ 32B. Learn more on Qwen’s [QwQ 32B blog post](https://qwenlm.github.io/blog/qwq-32b/). |
|
|
|
### What is GGUF? |
|
|
|
GGUF is a file format for representing AI models. It is the third version of the format, |
|
introduced by the llama.cpp team on August 21st 2023. |
|
|
|
Converted with llama.cpp build b4831 (revision [5e43f10](https://github.com/ggml-org/llama.cpp/commits/5e43f104cca1a14874e980326a506b44fde022b8)), |
|
using [autogguf-rs](https://github.com/brittlewis12/autogguf-rs). |
|
|
|
### Prompt template: [ChatML (with `<think>` tokens)](https://huggingface.co/Qwen/QwQ-32B/blob/main/tokenizer_config.json#L230) |
|
|
|
``` |
|
<|im_start|>system |
|
{{system_message}}<|im_end|> |
|
<|im_start|>user |
|
{{prompt}}<|im_end|> |
|
<|im_start|>assistant |
|
<think> |
|
|
|
|
|
``` |
|
|
|
### Note: re: split f16 model files |
|
|
|
To merge the split model files for the f16 precision GGUFs, you can run the `llama-gguf-split` command that comes included when you build llama.cpp & [its examples](https://github.com/ggml-org/llama.cpp/tree/5e43f104cca1a14874e980326a506b44fde022b8/examples/gguf-split). |
|
|
|
It accepts the path to the first of the downloaded splits, assuming the rest are alongside it, and an output path. For example: |
|
|
|
```sh |
|
# from your llama.cpp directory: |
|
$ cmake -B build |
|
$ cmake --build build --config Release |
|
$ ./build/bin/llama-gguf-split --merge \ |
|
~/Downloads/qwq-32b.f16.split-00001-of-00002.gguf \ |
|
~/Downloads/qwq-32b.f16.gguf |
|
``` |
|
|
|
--- |
|
|
|
## Download & run with [cnvrs](https://twitter.com/cnvrsai) on iPhone, iPad, and Mac! |
|
|
|
 |
|
|
|
[cnvrs](https://testflight.apple.com/join/sFWReS7K) is the best app for private, local AI on your device: |
|
- create & save **Characters** with custom system prompts & temperature settings |
|
- download and experiment with any **GGUF model** you can [find on HuggingFace](https://huggingface.co/models?library=gguf)! |
|
* or, use an API key with the chat completions-compatible model provider of your choice -- ChatGPT, Claude, Gemini, DeepSeek, & more! |
|
- make it your own with custom **Theme colors** |
|
- powered by Metal ⚡️ & [Llama.cpp](https://github.com/ggml-org/llama.cpp), with **haptics** during response streaming! |
|
- **try it out** yourself today, on [Testflight](https://testflight.apple.com/join/sFWReS7K)! |
|
* if you **already have the app**, download QwQ 32B now! |
|
* <cnvrsai:///models/search/hf?id=brittlewis12/QwQ-32B-GGUF> |
|
- follow [cnvrs on twitter](https://twitter.com/cnvrsai) to stay up to date |
|
|
|
### QwQ 32B in cnvrs on macOS |
|
|
|
 |
|
|
|
|
|
--- |
|
|
|
## Original Model Evaluation |
|
|
|
 |
|
|
