README.md

metadata

language:
  - en
tags:
  - llama
  - reasoning
  - 1b
  - math
  - logic
  - problem-solving
  - unsloth
  - lora
library_name: transformers
license: apache-2.0
datasets:
  - openai/gsm8k
  - lukaemon/bbh
pipeline_tag: text-generation
widget:
  - text: >-
      If a train travels at 60 miles per hour, how far will it travel in 2.5
      hours?
  - text: >-
      If all mammals are animals, and all dogs are mammals, what can we
      conclude?
  - text: >-
      A store sells shoes at $60 per pair and socks at $8 per pair. If I buy 2
      pairs of shoes and 3 pairs of socks, what is my total bill?
  - text: What is the area of a circle with radius 5 cm?
  - text: >-
      If 8 workers can build 4 houses in 10 days, how many days would it take 20
      workers to build 10 houses?
inference:
  parameters:
    temperature: 0.2
    max_new_tokens: 512
    repetition_penalty: 1.1

Vexoo TrailBlazer-1B - Enhanced Reasoning

Vexoo TrailBlazer-1B is a 1B parameter language model fine-tuned specifically for mathematical, logical, and structured reasoning tasks. Built on Llama-3.2-1B, this model incorporates custom reasoning adapters and extensive fine-tuning on problem-solving datasets.

Try the Model

Use the inference widget above to test the model with reasoning problems!

Model Details

• Base Model: Llama-3.2-1B-Instruct
• Parameter Count: 1 billion parameters
• Training Methodology:
  • LoRA fine-tuning using the Unsloth library
  • Custom cascading reasoning adapters in critical transformer layers
  • Trained on structured reasoning datasets like GSM8K
• Capabilities:
  • Step-by-step mathematical problem solving
  • Logical deduction and inference
  • Structured reasoning with clear explanations
  • Self-verification of answers

Recommended System Prompt

You are an advanced reasoning assistant that excels at solving complex problems. Follow these guidelines:
1. Break down problems into clear, logical steps
2. Consider multiple approaches when appropriate 
3. Identify key information and relevant concepts
4. Provide clear explanations for each step in your reasoning
5. Verify your conclusions with examples or counterexamples

Usage with Transformers

The model can be loaded using standard Transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "vexoolabs/Vexoo-TrailBlazer-1B"

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)

# System prompt for reasoning
system_prompt = "You are an advanced reasoning assistant that excels at solving complex problems."
user_question = "If a train travels at 60 miles per hour, how far will it travel in 2.5 hours?"

# Format with system prompt
messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": user_question}
]

# Format prompt
inputs = tokenizer.apply_chat_template(
    messages,
    return_tensors="pt",
    add_generation_prompt=True
).to(model.device)

# Generate
outputs = model.generate(
    inputs,
    max_new_tokens=300,
    temperature=0.2,
    top_p=0.92,
    repetition_penalty=1.05,
    do_sample=True
)

response = tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True)
print(response)

Advanced Usage with Unsloth

For optimal performance, you can also load with Unsloth:

# Import unsloth first
import unsloth
import torch

# Then import specific modules
from unsloth import FastLanguageModel
from unsloth.chat_templates import get_chat_template

# Load model and tokenizer
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="vexoolabs/Vexoo-TrailBlazer-1B",
    max_seq_length=2048,
    dtype=torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float16
)

# Configure tokenizer
tokenizer.pad_token = tokenizer.eos_token
tokenizer = get_chat_template(tokenizer, chat_template="llama-3.1")

# Prepare for inference
FastLanguageModel.for_inference(model)