|
|
--- |
|
|
base_model: meta-llama/Llama-3.2-3B-Instruct |
|
|
library_name: transformers |
|
|
model_name: Llama_Qlora_Reasoning_ToolCalling_Finetuned_4Bit |
|
|
tags: |
|
|
- generated_from_trainer |
|
|
- sft |
|
|
- trl |
|
|
- peft |
|
|
licence: license |
|
|
--- |
|
|
|
|
|
# Model Card for gemma_function_calling_and_thinking |
|
|
|
|
|
This model is a fine-tuned version of [meta-llama/Llama-3.2-3B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct). |
|
|
It has been trained using [TRL](https://github.com/huggingface/trl). |
|
|
|
|
|
## Quick start |
|
|
|
|
|
```python |
|
|
from peft import PeftModel |
|
|
from transformers import AutoModelForCausalLM |
|
|
from peft import AutoPeftModelForCausalLM |
|
|
from transformers import AutoTokenizer |
|
|
import torch |
|
|
|
|
|
peft_model_id = "Abdulvajid/Llama_Qlora_Reasoning_ToolCalling_Finetuned_4Bit" |
|
|
|
|
|
model = AutoPeftModelForCausalLM.from_pretrained(peft_model_id, |
|
|
torch_dtype=torch.float16, |
|
|
device_map='cuda', |
|
|
load_in_4bit=True) |
|
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained(peft_model_id) |
|
|
|
|
|
tools = [ |
|
|
{ |
|
|
"type": "function", |
|
|
"function": { |
|
|
"name": "search_restaurants", |
|
|
"description": "Search for restaurants in a specific location", |
|
|
"parameters": { |
|
|
"type": "object", |
|
|
"properties": { |
|
|
"location": { |
|
|
"type": "string", |
|
|
"description": "The location to search for restaurants" |
|
|
}, |
|
|
"cuisine": { |
|
|
"type": "string", |
|
|
"description": "The cuisine type to filter the restaurants" |
|
|
}, |
|
|
"price_range": { |
|
|
"type": "integer", |
|
|
"description": "The price range of the restaurants (1 = cheap to 4 = very expensive)" |
|
|
} |
|
|
}, |
|
|
"required": ["location"] |
|
|
} |
|
|
} |
|
|
} |
|
|
] |
|
|
|
|
|
messages=[ |
|
|
{"role": "user", "content": "I'm in Malappuram, can you find a restaurant for me?"} |
|
|
] |
|
|
|
|
|
prompt = tokenizer.apply_chat_template( |
|
|
messages, |
|
|
tools=tools, |
|
|
add_generation_prompt=True, |
|
|
tokenize=True, |
|
|
return_tensors="pt" |
|
|
).to('cuda') |
|
|
|
|
|
output = model.generate(prompt, max_new_tokens=500) |
|
|
|
|
|
print(''.join(tokenizer.batch_decode(output[0][len(prompt[0]):]))) |
|
|
``` |
|
|
|
|
|
## Training procedure |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This model was trained with SFT. |
|
|
|
|
|
### Framework versions |
|
|
|
|
|
- TRL: 0.19.1 |
|
|
- Transformers: 4.53.1 |
|
|
- Pytorch: 2.6.0+cu124 |
|
|
- Datasets: 4.0.0 |
|
|
- Tokenizers: 0.21.2 |
|
|
|
|
|
## Citations |
|
|
|
|
|
|
|
|
|
|
|
Cite TRL as: |
|
|
|
|
|
```bibtex |
|
|
@misc{vonwerra2022trl, |
|
|
title = {{TRL: Transformer Reinforcement Learning}}, |
|
|
author = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec}, |
|
|
year = 2020, |
|
|
journal = {GitHub repository}, |
|
|
publisher = {GitHub}, |
|
|
howpublished = {\url{https://github.com/huggingface/trl}} |
|
|
} |
|
|
``` |
