DMaS-LLaMa-Lite-step-43.5k-instruct

This repository provides access to DMaS-LLaMa-Lite-step-43.5k-instruct, an instruction fine-tuned version of the DMaS-LLaMa-Lite model. Based on the 43,500-step checkpoint, this model has been further fine-tuned on question-answering datasets to enable instruction-following and dialogue capabilities.

It serves as a demonstration of the potential to fine-tune a pretrained LLaMa-based model for downstream tasks, such as question-answering, with improved alignment for human-like responses.

Model Overview

• Base Model: DMaS-LLaMa-Lite-step-43.5k
• Architecture: LLaMa-based
• Parameters: 1.7B (36 layers, 32 attention heads, RMSNorm)
• Tokenizer: GPT-2 tokenizer
• Fine-tuning Data: yahma/alpaca-cleaned
• Chat Template: Follows Vicuna 1.1 format
• Response End: Machine answers are terminated with <|endoftext|>
• Objective: Fine-tune the pretrained model to improve alignment with user instructions and dialogue-based question answering

This model showcases how a LLaMa-based pretrained checkpoint can be effectively adapted for instruction-following tasks, resulting in coherent, relevant, and contextually grounded completions.

Key Features

• Instruction Fine-tuning: Demonstrates alignment to question-answering and dialogue tasks.
• Chat Template Compatibility: Compatible with Vicuna 1.1-style formatting, ensuring easy integration with chat-based applications.
• Early Demonstration: Highlights the feasibility of fine-tuning pretrained models with minimal downstream data.

Chat Template

The model follows the Vicuna 1.1 chat format:

A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.

USER: {user input}
ASSISTANT:{machine response}<|endoftext|>

Usage

You can load and use the model with Hugging Face's Transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "McGill-DMaS/DMaS-LLaMa-Lite-step-43.5k-instruct"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Define the prompt in Vicuna 1.1 format
prompt = "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.\n\nUSER: What are the Pyramids of Giza known for?\nASSISTANT:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=100)

# Decode the machine response
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Citation

If you use this model or its training insights in your work, please cite the following paper:

@article{li2024effectiveness,
  title={Experience of Training a 1.7B-Parameter LLaMa Model From Scratch},
  author={Li, Miles Q and Fung, Benjamin and Huang, Shih-Chia},
  journal={arXiv preprint arXiv:2412.13335},
  year={2024}
}

License

This model and code are released under the Apache License 2.0. Please check the respective repositories for detailed terms.

Training Code

For details on pre-training processes, the scripts are available at:
📄 DMaS-LLaMa-Lite Training Code