File size: 3,068 Bytes
540478e ee406fa 540478e ee406fa 540478e ee406fa 540478e ee406fa a87c761 540478e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 |
---
base_model: meta-llama/Llama-3.2-3B-Instruct
library_name: transformers
model_name: Llama_Qlora_Reasoning_ToolCalling_Finetuned_4Bit
tags:
- generated_from_trainer
- sft
- trl
- peft
licence: license
---
# Model Card for gemma_function_calling_and_thinking
This model is a fine-tuned version of [meta-llama/Llama-3.2-3B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct).
It has been trained using [TRL](https://github.com/huggingface/trl).
## Quick start
```python
from peft import PeftModel
from transformers import AutoModelForCausalLM
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer
import torch
peft_model_id = "Abdulvajid/Llama_Qlora_Reasoning_ToolCalling_Finetuned_4Bit"
model = AutoPeftModelForCausalLM.from_pretrained(peft_model_id,
torch_dtype=torch.float16,
device_map='cuda',
load_in_4bit=True)
tokenizer = AutoTokenizer.from_pretrained(peft_model_id)
tools = [
{
"type": "function",
"function": {
"name": "search_restaurants",
"description": "Search for restaurants in a specific location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The location to search for restaurants"
},
"cuisine": {
"type": "string",
"description": "The cuisine type to filter the restaurants"
},
"price_range": {
"type": "integer",
"description": "The price range of the restaurants (1 = cheap to 4 = very expensive)"
}
},
"required": ["location"]
}
}
}
]
messages=[
{"role": "user", "content": "I'm in Malappuram, can you find a restaurant for me?"}
]
prompt = tokenizer.apply_chat_template(
messages,
tools=tools,
add_generation_prompt=True,
tokenize=True,
return_tensors="pt"
).to('cuda')
output = model.generate(prompt, max_new_tokens=500)
print(''.join(tokenizer.batch_decode(output[0][len(prompt[0]):])))
```
## Training procedure
This model was trained with SFT.
### Framework versions
- TRL: 0.19.1
- Transformers: 4.53.1
- Pytorch: 2.6.0+cu124
- Datasets: 4.0.0
- Tokenizers: 0.21.2
## Citations
Cite TRL as:
```bibtex
@misc{vonwerra2022trl,
title = {{TRL: Transformer Reinforcement Learning}},
author = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
year = 2020,
journal = {GitHub repository},
publisher = {GitHub},
howpublished = {\url{https://github.com/huggingface/trl}}
}
``` |