README.md

metadata

base_model:
  - Qwen/QwQ-32B
library_name: transformers

MISHANM/Qwen-QwQ-32B.gguf

This model is a GGUF version of Qwen/QwQ-32B model, It is specially designed to work smoothly with the llama.cpp framework. It's built to run efficiently on CPU systems and has been tested on the AMD EPYC™ 9755 processor. The model handles various natural language processing tasks really well. It not only processes text quickly but also has strong reasoning and thinking skills, allowing it to manage difficult language-related challenges effectively.

Model Details

1. Language: English
2. Tasks: Text generation
3. Base Model: Qwen/QwQ-32B

Building and Running the Model

To build and run the model using llama.cpp, follow these steps:

Model

Steps to Download the Model:

1. Go to the "Files and Versions" section.
2. Click on the model.
3. Copy the download link.
4. Create a directory (e.g., for Linux: mkdir Qwen32B).
5. Navigate to that directory (cd Qwen32B).
6. Download both model parts: Qwen-QwQ-32B.gguf.part_01 and Qwen-QwQ-32B.gguf.part_02 (e.g., using wget with the copied link).

After downloading the model parts, use the following command to combine them into a complete model:

cat Qwen-QwQ-32B.gguf.part_01 Qwen-QwQ-32B.gguf.part_02 > Qwen-QwQ-32B.gguf

Build llama.cpp Locally

git clone https://github.com/ggerganov/llama.cpp  
cd llama.cpp  
cmake -B build  
cmake --build build --config Release  

Run the Model

Navigate to the build directory and run the model with a prompt:

cd llama.cpp/build/bin   

Inference with llama.cpp

./llama-cli -m /path/to/model/ -p "Your prompt here" -n 128 --ctx-size 8192 --temp 0.6 --seed 3407

Citation Information

@misc{MISHANM/Qwen-QwQ-32B.gguf,
  author = {Mishan Maurya},
  title = {Introducing Qwen QwQ-32B GGUF Model},
  year = {2025},
  publisher = {Hugging Face},
  journal = {Hugging Face repository},
  
}