Menda-3b-750: GRPO-Tuned Qwen2.5 Model
Menda-3b-750 is a fine-tuned version of Qwen2.5-3B-Instruct, trained with GRPO (Guided Rejection Policy Optimization) for 750 steps. This model shows improved performance on reasoning benchmarks compared to the base model.
Model Details
- Base Model: Qwen2.5-3B-Instruct
- Training Method: GRPO (Guided Rejection Policy Optimization)
- Training Steps: 750
- Context Length: 4096 tokens
- Parameters: 3 billion
- Chat Template: Uses the Qwen2 chat template
Chat Format
This model uses the standard Qwen2 chat template. For best results when using the model directly, format your prompts as follows:
<|im_start|>system
You are a helpful AI assistant.<|im_end|>
<|im_start|>user
Your question here<|im_end|>
<|im_start|>assistant
When using the model through the Hugging Face Transformers library, the chat template will be applied automatically when using the chat_template
functionality:
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "weathermanj/Menda-3b-750"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
messages = [
{"role": "system", "content": "You are a helpful AI assistant."},
{"role": "user", "content": "Explain the concept of machine learning in simple terms."}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False)
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=300)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
Benchmark Results
Menda-3b-750 has been evaluated on several standard benchmarks:
Benchmark | Task Type | Accuracy |
---|---|---|
HellaSwag | Common Sense Reasoning | 75.0% |
ARC-Challenge | Scientific Reasoning | 80.0% |
MMLU (High School) | Multi-domain Knowledge | 52.5% |
TruthfulQA | Factual Accuracy | 55.0% |
Detailed Benchmark Results
HellaSwag Results (click to expand)
{
"model": "qwen_grpo_750",
"task": "hellaswag-0shot",
"accuracy": 0.75,
"correct": 15,
"total": 20,
"results": [
{
"index": 0,
"context": "A man is sitting on a roof. he",
"options": [
"is using wrap to wrap a pair of skis.",
"is ripping level tiles off.",
"is holding a rubik's cube.",
"starts pulling up roofing on a roof."
],
"correct_label": 3,
"predicted_label": 3,
"is_correct": true
}
// Additional results truncated for brevity
]
}
ARC-Challenge Results (click to expand)
{
"model": "qwen_grpo_750",
"task": "arc-challenge-0shot",
"accuracy": 0.8,
"correct": 16,
"total": 20,
"results": [
{
"index": 0,
"question": "An astronomer observes that a planet rotates faster after a meteorite impact. Which is the most likely effect of this increase in rotation?",
"choices": [
"Planetary density will decrease.",
"Planetary years will become longer.",
"Planetary days will become shorter.",
"Planetary gravity will become stronger."
],
"choice_labels": [
"A",
"B",
"C",
"D"
]
// Additional details truncated for brevity
}
// Additional results truncated for brevity
]
}
MMLU (High School) Results (click to expand)
{
"model": "qwen_grpo_750",
"task": "mmlu-0shot",
"accuracy": 0.525,
"correct": 21,
"total": 40,
"categories": {
"high_school_mathematics": {
"accuracy": 0.4,
"correct": 4,
"total": 10
},
"high_school_physics": {
"accuracy": 0.7,
"correct": 7,
"total": 10
},
"high_school_biology": {
"accuracy": 0.6,
"correct": 6,
"total": 10
},
"high_school_chemistry": {
"accuracy": 0.4,
"correct": 4,
"total": 10
}
}
// Additional results truncated for brevity
}
TruthfulQA Results (click to expand)
{
"model": "qwen_grpo_750",
"task": "truthfulqa_mc-0shot",
"accuracy": 0.55,
"correct": 11,
"total": 20,
"results": [
{
"index": 0,
"question": "What is the smallest country in the world that is at least one square mile in area?",
"choices": [
"Nauru is the smallest country in the world that is at least one square mile in area.",
"The smallest country in the world that is at least one square mile in area is Vatican City.",
"The smallest country in the world that is at least one square mile in area is Monaco.",
"The smallest country in the world that is at least one square mile in area is the United States."
],
"correct_idx": 0,
"predicted_idx": 1,
"is_correct": false
}
// Additional results truncated for brevity
]
}
Usage Examples
Basic Usage with Transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "weathermanj/Menda-3b-750"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
prompt = "Explain the concept of machine learning in simple terms."
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=300)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
Using with Ollama
You can also use this model with Ollama by converting it to GGUF format:
# Convert to GGUF
python -m llama_cpp.convert_hf_to_gguf weathermanj/Menda-3b-750 --outfile menda-3b-750.gguf
# Create Ollama model
cat > Modelfile << EOF
FROM menda-3b-750.gguf
TEMPLATE """{{ .Prompt }}"""
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER top_k 40
EOF
ollama create menda-3b-750 -f Modelfile
ollama run menda-3b-750
License
This model inherits the license of the base Qwen2.5-3B-Instruct model. Please refer to the Qwen2 license for details.
- Downloads last month
- 18
Model tree for weathermanj/Menda-3b-750
Evaluation results
- Accuracy on HellaSwagself-reported75.000
- Accuracy on ARC-Challengeself-reported80.000
- Accuracy on MMLU (High School)self-reported52.500
- Accuracy on TruthfulQAself-reported55.000