{ "cells": [ { "cell_type": "markdown", "id": "43b502c1-9548-4580-84ad-1cbac158edb8", "metadata": { "id": "43b502c1-9548-4580-84ad-1cbac158edb8" }, "source": [ "# Bonus Unit 1: Fine-Tuning a model for Function-Calling\n", "\n", "In this tutorial, **we're going to Fine-Tune an LLM for Function Calling.**\n", "\n", "This notebook is part of the Hugging Face Agents Course, a free Course from beginner to expert, where you learn to build Agents.\n", "\n", "\"Agent\n" ] }, { "cell_type": "markdown", "id": "gWR4Rvpmjq5T", "metadata": { "id": "gWR4Rvpmjq5T" }, "source": [ "## Prerequisites 🏗️\n", "\n", "Before diving into the notebook, you need to:\n", "\n", "🔲 📚 **Study [What is Function-Calling](https://www.hf.co/learn/agents-course/bonus-unit1/what-is-function-calling) Section**\n", "\n", "🔲 📚 **Study [Fine-Tune your Model and what are LoRAs](https://www.hf.co/learn/agents-course/bonus-unit1/fine-tuning) Section**" ] }, { "cell_type": "markdown", "id": "1rZXU_1wkEPu", "metadata": { "id": "1rZXU_1wkEPu" }, "source": [ "# Step 0: Ask to Access Gemma on Hugging Face\n", "\n", "\"Gemma\"/\n", "\n", "\n", "To access Gemma on Hugging Face:\n", "\n", "1. **Make sure you're signed in** to your Hugging Face Account\n", "\n", "2. Go to https://huggingface.co/google/gemma-2-2b-it\n", "\n", "3. Click on **Acknowledge license** and fill the form.\n", "\n", "Alternatively you can use another model, and modify the code accordingly (it can be a good exercise for you to be sure you know how to fine-tune for Function-Calling).\n", "\n", "You can use for instance:\n", "\n", "- [HuggingFaceTB/SmolLM2-1.7B-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct)\n", "\n", "- [meta-llama/Llama-3.2-3B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct)" ] }, { "cell_type": "markdown", "id": "5hjyx9nJlvKG", "metadata": { "id": "5hjyx9nJlvKG" }, "source": [ "## Step 1: Set the GPU 💪\n", "\n", "If you're on Colab:\n", "\n", "- To **accelerate the fine-tuning training, we'll use a GPU**. To do that, go to `Runtime > Change Runtime type`\n", "\n", "\"GPU\n", "\n", "- `Hardware Accelerator > GPU`\n", "\n", "\"GPU\n", "\n", "\n", "### Important\n", "\n", "For this Unit, **with the free-tier of Colab** it will take around **6h to train**.\n", "\n", "You have three solutions if you want to make it faster:\n", "\n", "1. Train on your computer if you have GPUs. It might take time but you have less risks of timeout.\n", "\n", "2. Use a Google Colab Pro that allows you use to A100 GPU (15-20min training).\n", "\n", "3. Just follow the code to learn how to do it without training." ] }, { "cell_type": "markdown", "id": "5Thjsc9fj6Ej", "metadata": { "id": "5Thjsc9fj6Ej" }, "source": [ "## Step 2: Install dependencies 📚\n", "\n", "We need multiple librairies:\n", "\n", "- `bitsandbytes` for quantization\n", "- `peft`for LoRA adapters\n", "- `Transformers`for loading the model\n", "- `datasets`for loading and using the fine-tuning dataset\n", "- `trl`for the trainer class" ] }, { "cell_type": "code", "execution_count": 1, "id": "e63f4962-c644-491e-aa91-50e453e953a4", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "e63f4962-c644-491e-aa91-50e453e953a4", "outputId": "3c1563d4-74fe-46f0-adcd-3e935261a89d", "tags": [] }, "outputs": [], "source": [ "!pip install -q -U bitsandbytes\n", "!pip install -q -U transformers\n", "!pip install -q -U peft\n", "!pip install -q -U accelerate\n", "!pip install -q -U datasets\n", "!pip install -q trl==0.12.2\n", "!pip install -q -U tensorboardX\n", "!pip install -q wandb" ] }, { "cell_type": "markdown", "id": "UWNoZzi1urSZ", "metadata": { "id": "UWNoZzi1urSZ" }, "source": [ "## Step 3: Create your Hugging Face Token to push your model to the Hub\n", "\n", "To be able to share your model with the community there are some more steps to follow:\n", "\n", "1️⃣ (If it's not already done) create an account to HF ➡ https://huggingface.co/join\n", "\n", "2️⃣ Sign in and then, you need to store your authentication token from the Hugging Face website.\n", "\n", "- Create a new token (https://huggingface.co/settings/tokens) **with write role**\n", "\n", "\"Create\n", "\n", "3️⃣ Store your token as an environment variable under the name \"HF_TOKEN\"\n", "- **Be very carefull not to share it with others** !" ] }, { "cell_type": "markdown", "id": "vBAkwg9zu6A1", "metadata": { "id": "vBAkwg9zu6A1" }, "source": [ "## Step 4: Import the librairies\n", "\n", "Don't forget to put your HF token." ] }, { "cell_type": "code", "execution_count": 2, "id": "7ad2e4c2-593e-463e-9692-8d674c541d76", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 382 }, "id": "7ad2e4c2-593e-463e-9692-8d674c541d76", "outputId": "5004413d-0202-4031-85e3-d1897fb8eba5", "tags": [] }, "outputs": [], "source": [ "from enum import Enum\n", "from functools import partial\n", "import pandas as pd\n", "import torch\n", "import json\n", "\n", "from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, BitsAndBytesConfig, set_seed\n", "from datasets import load_dataset\n", "from trl import SFTTrainer\n", "from peft import get_peft_model, LoraConfig, TaskType\n", "\n", "seed = 42\n", "set_seed(seed)\n", "\n", "import os\n", "\n", "# Put your HF Token here\n", "os.environ['HF_TOKEN']=\"hf_xxxx\"" ] }, { "cell_type": "markdown", "id": "44f30b2c-2cc0-48e0-91ca-4633e6444105", "metadata": { "id": "44f30b2c-2cc0-48e0-91ca-4633e6444105" }, "source": [ "## Step 5: Processing the dataset into inputs\n", "\n", "In order to train the model, we need to **format the inputs into what we want the model to learn**.\n", "\n", "For this tutorial, I enhanced a popular dataset for function calling \"NousResearch/hermes-function-calling-v1\" by adding some new **thinking** step computer from **deepseek-ai/DeepSeek-R1-Distill-Qwen-32B**.\n", "\n", "But in order for the model to learn, we need **to format the conversation correctly**. If you followed Unit 1, you know that going from a list of messages to a prompt is handled by the **chat_template**, or, the default chat_template of gemma-2-2B does not contain tool calls. So we will need to modify it !\n", "\n", "This is the role of our **preprocess** function. To go from a list of messages, to a prompt that the model can understand.\n" ] }, { "cell_type": "code", "execution_count": 3, "id": "29da85c8-33bf-4864-aed7-733cbe703512", "metadata": { "id": "29da85c8-33bf-4864-aed7-733cbe703512", "tags": [] }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "2196bea18c254339847558f804db8853", "version_major": 2, "version_minor": 0 }, "text/plain": [ "tokenizer_config.json: 0%| | 0.00/47.0k [00:00' + message['role'] + '\\n' + message['content'] | trim + '\\n' }}{% endfor %}{% if add_generation_prompt %}{{'model\\n'}}{% endif %}\"\n", "\n", "\n", "def preprocess(samples):\n", " batch = []\n", " for conversations in zip(samples[\"conversations\"]):\n", " conversation = conversations[0]\n", "\n", " # Instead of adding a system message, we merge the content into the first user message\n", " if conversation[0][\"role\"] == \"system\":\n", " system_message_content = conversation[0][\"content\"]\n", " # Merge system content with the first user message\n", " conversation[1][\"content\"] = system_message_content + \"Also, before making a call to a function take the time to plan the function to take. Make that thinking process between {your thoughts}\\n\\n\" + conversation[1][\"content\"]\n", " # Remove the system message from the conversation\n", " conversation.pop(0)\n", "\n", " batch.append(tokenizer.apply_chat_template(conversation, tokenize=False))\n", "\n", " return {\"content\": batch}\n", "\n", "dataset = load_dataset(dataset_name)\n" ] }, { "cell_type": "markdown", "id": "dc8736d5-d64b-4c5c-9738-be08421d3f95", "metadata": { "id": "dc8736d5-d64b-4c5c-9738-be08421d3f95" }, "source": [ "## Step 6: A Dedicated Dataset for This Unit\n", "\n", "For this Bonus Unit, we created a custom dataset based on [NousResearch/hermes-function-calling-v1](https://huggingface.co/datasets/NousResearch/hermes-function-calling-v1), which is considered a **reference** when it comes to function-calling datasets.\n", "\n", "While the original dataset is excellent, it does **not** include a **“thinking”** step.\n", "\n", "In Function-Calling, such a step is optional, but recent work—like the **deepseek** model or the paper [\"Test-Time Compute\"](https://huggingface.co/papers/2408.03314)—suggests that giving an LLM time to “think” before it answers (or in this case, **before** taking an action) can **significantly improve** model performance.\n", "\n", "I, decided to then compute a subset of this dataset and to give it to [deepseek-ai/DeepSeek-R1-Distill-Qwen-32B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B) in order to compute some thinking tokens `` before any function call. Which resulted in the following dataset :\n", "![Input Dataset](https://huggingface.co/datasets/agents-course/course-images/resolve/main/en/bonus-unit1/dataset_function_call.png)\n" ] }, { "cell_type": "code", "execution_count": 4, "id": "b63d4832-d92e-482d-9fe6-6e9dbfee377a", "metadata": { "id": "b63d4832-d92e-482d-9fe6-6e9dbfee377a", "outputId": "bda88f48-ca5d-4f47-b887-48d4ea5a53aa", "tags": [] }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "343ae0d35c79439788601f8cb4e734d1", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Map: 0%| | 0/3570 [00:00` then the user query, here: `\"Can you get me the latest news headlines for the United States?\"`\n", "\n", "2. An *Assistant message* here called \"model\" to fit the criterias from gemma models containing two new phases, a **\"thinking\"** phase contained in `` and an **\"Act\"** phase contained in ``.\n", "\n", "3. If the model contains a ``, we will append the result of this action in a new **\"Tool\"** message containing a `` with the answer from the tool." ] }, { "cell_type": "code", "execution_count": 5, "id": "dc60da04-9411-487a-b629-2c59024a20c0", "metadata": { "id": "dc60da04-9411-487a-b629-2c59024a20c0", "outputId": "6709e478-17b8-4769-865f-2cd025727ad4", "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "human\n", "You are a function calling AI model. You are provided with function signatures within XML tags.You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions.Here are the available tools: [{'type': 'function', 'function': {'name': 'get_news_headlines', 'description': 'Get the latest news headlines', 'parameters': {'type': 'object', 'properties': {'country': {'type': 'string', 'description': 'The country for which headlines are needed'}}, 'required': ['country']}}}, {'type': 'function', 'function': {'name': 'search_recipes', 'description': 'Search for recipes based on ingredients', 'parameters': {'type': 'object', 'properties': {'ingredients': {'type': 'array', 'items': {'type': 'string'}, 'description': 'The list of ingredients'}}, 'required': ['ingredients']}}}] Use the following pydantic model json schema for each tool call you will make: {'title': 'FunctionCall', 'type': 'object', 'properties': {'arguments': {'title': 'Arguments', 'type': 'object'}, 'name': {'title': 'Name', 'type': 'string'}}, 'required': ['arguments', 'name']}For each function call return a json object with function name and arguments within XML tags as follows:\n", "\n", "{tool_call}\n", "Also, before making a call to a function take the time to plan the function to take. Make that thinking process between {your thoughts}\n", "\n", "Can you get me the latest news headlines for the United States?\n", "model\n", "Okay, so the user asked for the latest news headlines in the United States. I need to figure out how to respond appropriately using the tools available.\n", "\n", "Looking at the tools provided, there's a function called get_news_headlines. That seems perfect for this situation. It requires a country code, which the user specified as the United States.\n", "\n", "Since the user is asking specifically about the US, I'll use the country parameter as 'United States' in the function call. This should fetch the relevant news for them.\n", "\n", "I should make sure to format the response correctly as a JSON object with the function name and arguments.\n", "\n", "{'name': 'get_news_headlines', 'arguments': {'country': 'United States'}}\n", "\n", "tool\n", "\n", "{'headlines': ['US President signs new bill into law', 'Major earthquake hits California', 'NASA announces new Mars mission', 'Tech giant releases new smartphone', 'US economy shows signs of recovery']}\n", "\n", "model\n", "Here are the latest news headlines for the United States:\n", "1. US President signs new bill into law\n", "2. Major earthquake hits California\n", "3. NASA announces new Mars mission\n", "4. Tech giant releases new smartphone\n", "5. US economy shows signs of recovery\n", "human\n", "That's interesting. Can you get me the news for France now?\n", "model\n", "Okay, the user initially asked for the latest US news, so I used get_news_headlines with 'United States'. Now the user is asking for France, so I'll call the same function with 'France' as the country. That makes sense because the function is designed for any country.\n", "\n", "{'name': 'get_news_headlines', 'arguments': {'country': 'France'}}\n", "\n", "tool\n", "\n", "{'headlines': ['French President announces new environmental policy', 'Paris Fashion Week highlights', 'France wins World Cup qualifier', 'New culinary trend sweeps across France', 'French tech startup raises millions in funding']}\n", "\n", "model\n", "Here are the latest news headlines for France:\n", "1. French President announces new environmental policy\n", "2. Paris Fashion Week highlights\n", "3. France wins World Cup qualifier\n", "4. New culinary trend sweeps across France\n", "5. French tech startup raises millions in funding\n", "\n" ] } ], "source": [ "# Let's look at how we formatted the dataset\n", "print(dataset[\"train\"][8][\"content\"])" ] }, { "cell_type": "code", "execution_count": 6, "id": "53a48281-2346-4dfb-ad60-cad85129ec9b", "metadata": { "id": "53a48281-2346-4dfb-ad60-cad85129ec9b", "outputId": "da4f7e33-227c-48bc-b3c3-24df31313a69", "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n" ] } ], "source": [ "# Sanity check\n", "print(tokenizer.pad_token)\n", "print(tokenizer.eos_token)" ] }, { "cell_type": "markdown", "id": "d6864b36-6033-445a-b6e2-b6bb02e38e26", "metadata": { "id": "d6864b36-6033-445a-b6e2-b6bb02e38e26" }, "source": [ "## Step 8: Let's Modify the Tokenizer\n", "\n", "Indeed, as we saw in Unit 1, the tokenizer splits text into sub-words by default. This is **not** what we want for our new special tokens!\n", "\n", "While we segmented our example using ``, ``, and ``, the tokenizer does **not** yet treat them as whole tokens—it still tries to break them down into smaller pieces. To ensure the model correctly interprets our new format, we must **add these tokens** to our tokenizer.\n", "\n", "Additionally, since we changed the `chat_template` in our **preprocess** function to format conversations as messages within a prompt, we also need to modify the `chat_template` in the tokenizer to reflect these changes." ] }, { "cell_type": "code", "execution_count": 7, "id": "833ba5d6-4c1e-4689-9fed-22cc03d2a63a", "metadata": { "colab": { "referenced_widgets": [ "fc7eee07d5824955adb7b9bbd025c297" ] }, "id": "833ba5d6-4c1e-4689-9fed-22cc03d2a63a", "outputId": "2ef794e9-3e8e-4dec-d31c-b9242386c2d0", "tags": [] }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "c93adf223ba642ff83433bd0a49b0729", "version_major": 2, "version_minor": 0 }, "text/plain": [ "config.json: 0%| | 0.00/838 [00:00\"\n", " eotools = \"\"\n", " think = \"\"\n", " eothink = \"\"\n", " tool_call=\"\"\n", " eotool_call=\"\"\n", " tool_response=\"\"\n", " eotool_response=\"\"\n", " pad_token = \"\"\n", " eos_token = \"\"\n", " @classmethod\n", " def list(cls):\n", " return [c.value for c in cls]\n", "\n", "tokenizer = AutoTokenizer.from_pretrained(\n", " model_name,\n", " pad_token=ChatmlSpecialTokens.pad_token.value,\n", " additional_special_tokens=ChatmlSpecialTokens.list()\n", " )\n", "tokenizer.chat_template = \"{{ bos_token }}{% if messages[0]['role'] == 'system' %}{{ raise_exception('System role not supported') }}{% endif %}{% for message in messages %}{{ '' + message['role'] + '\\n' + message['content'] | trim + '\\n' }}{% endfor %}{% if add_generation_prompt %}{{'model\\n'}}{% endif %}\"\n", "\n", "model = AutoModelForCausalLM.from_pretrained(model_name,\n", " attn_implementation='eager',\n", " device_map=\"auto\")\n", "model.resize_token_embeddings(len(tokenizer))\n", "model.to(torch.bfloat16)\n" ] }, { "cell_type": "markdown", "id": "X6DBY8AqxFLL", "metadata": { "id": "X6DBY8AqxFLL" }, "source": [ "## Step 9: Let's configure the LoRA\n", "\n", "This is we are going to define the parameter of our adapter. Those a the most important parameters in LoRA as they define the size and importance of the adapters we are training." ] }, { "cell_type": "code", "execution_count": 8, "id": "482d36ab-e326-4fd7-bc59-425abcca55e7", "metadata": { "id": "482d36ab-e326-4fd7-bc59-425abcca55e7", "tags": [] }, "outputs": [], "source": [ "from peft import LoraConfig\n", "\n", "# TODO: Configure LoRA parameters\n", "# r: rank dimension for LoRA update matrices (smaller = more compression)\n", "rank_dimension = 16\n", "# lora_alpha: scaling factor for LoRA layers (higher = stronger adaptation)\n", "lora_alpha = 64\n", "# lora_dropout: dropout probability for LoRA layers (helps prevent overfitting)\n", "lora_dropout = 0.05\n", "\n", "peft_config = LoraConfig(r=rank_dimension,\n", " lora_alpha=lora_alpha,\n", " lora_dropout=lora_dropout,\n", " target_modules=[\"gate_proj\",\"q_proj\",\"lm_head\",\"o_proj\",\"k_proj\",\"embed_tokens\",\"down_proj\",\"up_proj\",\"v_proj\"], # wich layer in the transformers do we target ?\n", " task_type=TaskType.CAUSAL_LM)" ] }, { "cell_type": "markdown", "id": "zdDR9hzgxPu2", "metadata": { "id": "zdDR9hzgxPu2" }, "source": [ "## Step 10: Let's define the Trainer and the Fine-Tuning hyperparameters\n", "\n", "In this step, we define the Trainer, the class that we use to fine-tune our model and the hyperparameters." ] }, { "cell_type": "code", "execution_count": null, "id": "3598b688-5a6f-437f-95ac-4794688cd38f", "metadata": { "id": "3598b688-5a6f-437f-95ac-4794688cd38f", "outputId": "515f019f-87b6-40cb-9344-f4c19645e077", "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/user/miniconda/lib/python3.9/site-packages/transformers/training_args.py:1568: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead\n", " warnings.warn(\n" ] } ], "source": [ "output_dir = \"gemma-2-2B-it-thinking-function_calling\" # The directory where the trained model checkpoints, logs, and other artifacts will be saved. It will also be the default name of the model when pushed to the hub if not redefined later.\n", "per_device_train_batch_size = 1\n", "per_device_eval_batch_size = 1\n", "gradient_accumulation_steps = 4\n", "logging_steps = 5\n", "learning_rate = 1e-4 # The initial learning rate for the optimizer.\n", "\n", "max_grad_norm = 1.0\n", "num_train_epochs=1\n", "warmup_ratio = 0.1\n", "lr_scheduler_type = \"cosine\"\n", "max_seq_length = 2048\n", "\n", "training_arguments = TrainingArguments(\n", " output_dir=output_dir,\n", " per_device_train_batch_size=per_device_train_batch_size,\n", " per_device_eval_batch_size=per_device_eval_batch_size,\n", " gradient_accumulation_steps=gradient_accumulation_steps,\n", " save_strategy=\"no\",\n", " evaluation_strategy=\"epoch\",\n", " logging_steps=logging_steps,\n", " learning_rate=learning_rate,\n", " max_grad_norm=max_grad_norm,\n", " weight_decay=0.1,\n", " warmup_ratio=warmup_ratio,\n", " lr_scheduler_type=lr_scheduler_type,\n", " report_to=\"tensorboard\",\n", " bf16=True,\n", " hub_private_repo=False,\n", " push_to_hub=False,\n", " num_train_epochs=num_train_epochs,\n", " gradient_checkpointing=True,\n", " gradient_checkpointing_kwargs={\"use_reentrant\": False}\n", ")" ] }, { "cell_type": "markdown", "id": "59TTqmW2xmV2", "metadata": { "id": "59TTqmW2xmV2" }, "source": [ "As Trainer, we use the `SFTTrainer` which is a Supervised Fine-Tuning Trainer." ] }, { "cell_type": "code", "execution_count": null, "id": "ba0366b5-c9d0-4f7e-97e0-1f964cfad147", "metadata": { "id": "ba0366b5-c9d0-4f7e-97e0-1f964cfad147", "outputId": "8b2836b3-3a06-4c05-b046-6c7923911e40", "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/user/miniconda/lib/python3.9/site-packages/huggingface_hub/utils/_deprecation.py:100: FutureWarning: Deprecated argument(s) used in '__init__': packing, dataset_text_field, max_seq_length, dataset_kwargs. Will not be supported from version '0.13.0'.\n", "\n", "Deprecated positional argument(s) used in SFTTrainer, please use the SFTConfig to set these arguments instead.\n", " warnings.warn(message, FutureWarning)\n", "/home/user/miniconda/lib/python3.9/site-packages/transformers/training_args.py:1568: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead\n", " warnings.warn(\n", "/home/user/miniconda/lib/python3.9/site-packages/trl/trainer/sft_trainer.py:212: UserWarning: You passed a `packing` argument to the SFTTrainer, the value you passed will override the one in the `SFTConfig`.\n", " warnings.warn(\n", "/home/user/miniconda/lib/python3.9/site-packages/peft/tuners/tuners_utils.py:543: UserWarning: Model with `tie_word_embeddings=True` and the tied_target_modules=['lm_head'] are part of the adapter. This can lead to complications, for example when merging the adapter or converting your model to formats other than safetensors. See for example https://github.com/huggingface/peft/issues/2018.\n", " warnings.warn(\n", "/home/user/miniconda/lib/python3.9/site-packages/trl/trainer/sft_trainer.py:300: UserWarning: You passed a `max_seq_length` argument to the SFTTrainer, the value you passed will override the one in the `SFTConfig`.\n", " warnings.warn(\n", "/home/user/miniconda/lib/python3.9/site-packages/trl/trainer/sft_trainer.py:328: UserWarning: You passed a `dataset_text_field` argument to the SFTTrainer, the value you passed will override the one in the `SFTConfig`.\n", " warnings.warn(\n", "/home/user/miniconda/lib/python3.9/site-packages/trl/trainer/sft_trainer.py:334: UserWarning: You passed a `dataset_kwargs` argument to the SFTTrainer, the value you passed will override the one in the `SFTConfig`.\n", " warnings.warn(\n", "/home/user/miniconda/lib/python3.9/site-packages/trl/trainer/sft_trainer.py:403: UserWarning: You passed a processing_class with `padding_side` not equal to `right` to the SFTTrainer. This might lead to some unexpected behaviour due to overflow issues when training a model in half-precision. You might consider adding `processing_class.padding_side = 'right'` to your code.\n", " warnings.warn(\n" ] } ], "source": [ "trainer = SFTTrainer(\n", " model=model,\n", " args=training_arguments,\n", " train_dataset=dataset[\"train\"],\n", " eval_dataset=dataset[\"test\"],\n", " tokenizer=tokenizer,\n", " packing=True,\n", " dataset_text_field=\"content\",\n", " max_seq_length=max_seq_length,\n", " peft_config=peft_config,\n", " dataset_kwargs={\n", " \"append_concat_token\": False,\n", " \"add_special_tokens\": False,\n", " },\n", ")" ] }, { "cell_type": "markdown", "id": "MtHjukK9xviB", "metadata": { "id": "MtHjukK9xviB" }, "source": [ "Here, we launch the training 🔥. Perfect time for you to pause and grab a coffee ☕." ] }, { "cell_type": "code", "execution_count": null, "id": "9e2df2e9-a82b-4540-aa89-1b40b70a7781", "metadata": { "id": "9e2df2e9-a82b-4540-aa89-1b40b70a7781", "outputId": "8ad7e555-678b-4904-aa6b-000b619c9341", "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.\n" ] }, { "data": { "text/html": [ "\n", "
\n", " \n", " \n", " [389/389 14:14, Epoch 0/1]\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
EpochTraining LossValidation Loss
00.2946000.289091

" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stderr", "output_type": "stream", "text": [ "/home/user/miniconda/lib/python3.9/site-packages/peft/utils/save_and_load.py:230: UserWarning: Setting `save_embedding_layers` to `True` as embedding layers found in `target_modules`.\n", " warnings.warn(\"Setting `save_embedding_layers` to `True` as embedding layers found in `target_modules`.\")\n" ] } ], "source": [ "trainer.train()\n", "trainer.save_model()" ] }, { "cell_type": "markdown", "id": "1d7ea3ab-7c8c-47ad-acd2-99fbe5b68393", "metadata": { "id": "1d7ea3ab-7c8c-47ad-acd2-99fbe5b68393", "tags": [] }, "source": [ "## Step 11: Let's push the Model and the Tokenizer to the Hub\n", "\n", "Let's push our model and out tokenizer to the Hub ! The model will be pushed under your username + the output_dir that we specified earlier." ] }, { "cell_type": "code", "execution_count": null, "id": "370af020-9319-4ff7-bea1-2842a4847caa", "metadata": { "colab": { "referenced_widgets": [ "68b34e3d2eae4f24b83fa65cf5815738", "15e7c632053c4ed88267061a8112d641", "047ebf7fda8643c090129bc2b86a7e3e", "ef5dab829e8b491581f6dae2b7718113", "97ba6384a2f94db0880a17ff433a8ed9", "ba0c0c53b23047ac9336fdbf8597f32f", "8a033d5bd32b4a969fd5af612c550243", "714d4a40a67a4763ba8f7ae029befbd7", "52451f392b434fb39b882c9f094bd995", "08c2ca8a719e4142bd877ae94d242f2e", "02e660bd44a7414fae439751e9ffa1f2" ] }, "id": "370af020-9319-4ff7-bea1-2842a4847caa", "outputId": "f5797c01-306d-48ee-a009-3c115f5b1ca5", "tags": [] }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "68b34e3d2eae4f24b83fa65cf5815738", "version_major": 2, "version_minor": 0 }, "text/plain": [ "adapter_model.safetensors: 0%| | 0.00/2.48G [00:00\"\n", "# push the tokenizer to hub ( replace with your username and your previously specified\n", "tokenizer.push_to_hub(f\"username/{output_dir}\", token=True)" ] }, { "cell_type": "markdown", "id": "76d275ce-a3e6-4d30-8d8c-0ee274de5370", "metadata": { "id": "76d275ce-a3e6-4d30-8d8c-0ee274de5370" }, "source": [ "## Step 12: Let's now test our model !\n", "\n", "To so, we will :\n", "\n", "1. Load the adapter from the hub !\n", "2. Load the base model : **\"google/gemma-2-2b-it\"** from the hub\n", "3." ] }, { "cell_type": "code", "execution_count": null, "id": "56b89825-70ac-42c1-934c-26e2d54f3b7b", "metadata": { "colab": { "referenced_widgets": [ "390c54434b6448b988ce015eeafe34c9", "35b2fe2d357b46488ccef710f2a9bfd7", "9c313149d4324bdaa9c8ddc373964d18" ] }, "id": "56b89825-70ac-42c1-934c-26e2d54f3b7b", "outputId": "a4cd00b8-61fa-4522-d563-c4ef7e18807d", "tags": [] }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "390c54434b6448b988ce015eeafe34c9", "version_major": 2, "version_minor": 0 }, "text/plain": [ "adapter_config.json: 0%| | 0.00/829 [00:00human\n", "You are a function calling AI model. You are provided with function signatures within XML tags.You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions.Here are the available tools: [{'type': 'function', 'function': {'name': 'convert_currency', 'description': 'Convert from one currency to another', 'parameters': {'type': 'object', 'properties': {'amount': {'type': 'number', 'description': 'The amount to convert'}, 'from_currency': {'type': 'string', 'description': 'The currency to convert from'}, 'to_currency': {'type': 'string', 'description': 'The currency to convert to'}}, 'required': ['amount', 'from_currency', 'to_currency']}}}, {'type': 'function', 'function': {'name': 'calculate_distance', 'description': 'Calculate the distance between two locations', 'parameters': {'type': 'object', 'properties': {'start_location': {'type': 'string', 'description': 'The starting location'}, 'end_location': {'type': 'string', 'description': 'The ending location'}}, 'required': ['start_location', 'end_location']}}}] Use the following pydantic model json schema for each tool call you will make: {'title': 'FunctionCall', 'type': 'object', 'properties': {'arguments': {'title': 'Arguments', 'type': 'object'}, 'name': {'title': 'Name', 'type': 'string'}}, 'required': ['arguments', 'name']}For each function call return a json object with function name and arguments within XML tags as follows:\n", "\n", "{tool_call}\n", "Also, before making a call to a function take the time to plan the function to take. Make that thinking process between {your thoughts}\n", "\n", "Hi, I need to convert 500 USD to Euros. Can you help me with that?\n", "model\n", "Alright, so I'm looking at this scenario where the user needs help converting 500 USD to Euros. Let's break it down step by step to understand how to approach this.\n", "\n", "First, the user has asked for conversion from USD to EUR. I know there's a specific tool available for currency conversion called `convert_currency`. This tool requires three parameters: amount, from_currency, and to_currency. In the user's message, they've provided all necessary details—500 USD and the desire to convert to Euros. \n", "\n", "I don't need to make any assumptions here because the user has already provided the exact amount and the target currency. My next step is to prepare the function call with the correct parameters. So, I'll structure the request with the amount as 500, from_currency as USD, and to_currency as EUR. \n", "\n", "This will be sent in JSON format as specified, ensuring that the tool receives all the necessary information to perform the conversion accurately. Once the function is called, it should return the converted amount, which I can then present back to the user.\n", "\n", "{'name': 'convert_currency', 'arguments': {'amount': 500, 'from_currency': 'USD', 'to_currency': 'EUR'}}\n", "\n", "tool\n", "\n", "{'converted_amount': 420.5, 'from_currency': 'USD', 'to_currency': 'EUR'}\n", "\n", "model\n", "Sure, 500 USD is approximately 420.50 Euros.\n", "human\n", "That's great! Now, can you convert 1000 Euros back to USD?\n", "model\n", "Okay, so I'm trying to figure out how to handle the next function call based on the conversation and the previous steps. Let me see...\n", "\n", "In the conversation, the user first asked to convert 500 USD to Euros, and the model used the `convert_currency` function with the parameters amount=500, from_currency=USD, to_currency=EUR. Then, after getting the response, the model provided the converted amount. Now, the user is asking to convert 1000 Euros back to USD. \n", "\n", "So, the user's request is clear: convert 1000 EUR to USD. Since the function `convert_currency` is designed to handle currency conversions, and it requires amount, from_currency, and to_currency as parameters, it makes sense to call this function again with the updated values. \n", "\n", "The amount is 1000, from_currency should be EUR because the user is converting from Euros, and to_currency should be USD because they want the result in US Dollars. I don't need to make any assumptions here because all the required information is provided by the user. \n", "\n", "I'll structure the function call with these parameters. This should give the user the accurate converted amount from Euros to USD. Once the tool processes this, the model can present the result back to the user.\n", "\n", "{'name': 'convert_currency', 'arguments': {'amount': 1000, 'from_currency': 'EUR', 'to_currency': 'USD'}}\n", "\n", "tool\n", "\n", "{'converted_amount': 1190.0, 'from_currency': 'EUR', 'to_currency': 'USD'}\n", "\n", "model\n", "1000 Euros is approximately 1190.00 USD.\n", "human\n", "Thank you for your help!\n", "model\n", "You're welcome! If you have any other questions, feel free to ask.\n", "\n" ] } ], "source": [ "print(dataset[\"test\"][8][\"content\"])" ] }, { "cell_type": "markdown", "id": "b47fd511-ea00-47ce-8618-6e78e25672b2", "metadata": { "id": "b47fd511-ea00-47ce-8618-6e78e25672b2" }, "source": [ "### Testing the model 🚀\n", "\n", "In that case, we will take the start of one of the samples from the test set and hope that it will generate the expected output.\n", "\n", "Since we want to test the function-calling capacities of our newly fine-tuned model, the input will be a user message with the available tools, a\n", "\n", "\n", "### Disclaimer ⚠️\n", "\n", "The dataset we’re using **does not contain sufficient training data** and is purely for **educational purposes**. As a result, **your trained model’s outputs may differ** from the examples shown in this course. **Don’t be discouraged** if your results vary—our primary goal here is to illustrate the core concepts rather than produce a fully optimized or production-ready model.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "37bf938d-08fa-4577-9966-0238339afcdb", "metadata": { "id": "37bf938d-08fa-4577-9966-0238339afcdb", "outputId": "e97e7a1e-5ab2-46a2-dc3a-f436964fe004", "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "human\n", "You are a function calling AI model. You are provided with function signatures within XML tags.You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions.Here are the available tools: [{'type': 'function', 'function': {'name': 'convert_currency', 'description': 'Convert from one currency to another', 'parameters': {'type': 'object', 'properties': {'amount': {'type': 'number', 'description': 'The amount to convert'}, 'from_currency': {'type': 'string', 'description': 'The currency to convert from'}, 'to_currency': {'type': 'string', 'description': 'The currency to convert to'}}, 'required': ['amount', 'from_currency', 'to_currency']}}}, {'type': 'function', 'function': {'name': 'calculate_distance', 'description': 'Calculate the distance between two locations', 'parameters': {'type': 'object', 'properties': {'start_location': {'type': 'string', 'description': 'The starting location'}, 'end_location': {'type': 'string', 'description': 'The ending location'}}, 'required': ['start_location', 'end_location']}}}] Use the following pydantic model json schema for each tool call you will make: {'title': 'FunctionCall', 'type': 'object', 'properties': {'arguments': {'title': 'Arguments', 'type': 'object'}, 'name': {'title': 'Name', 'type': 'string'}}, 'required': ['arguments', 'name']}For each function call return a json object with function name and arguments within XML tags as follows:\n", "\n", "{tool_call}\n", "Also, before making a call to a function take the time to plan the function to take. Make that thinking process between {your thoughts}\n", "\n", "Hi, I need to convert 500 USD to Euros. Can you help me with that?\n", "model\n", "Okay, so the user is asking to convert 500 USD to Euros. I need to figure out how to respond. Looking at the available tools, there's a function called convert_currency which does exactly that. It takes an amount, the source currency, and the target currency. The user provided all the necessary details: 500, USD, and EUR. So, I should call convert_currency with these parameters. That should give the user the converted amount they need.\n", "\n", "{'name': 'convert_currency', 'arguments': {'amount': 500, 'from_currency': 'USD', 'to_currency': 'EUR'}}\n", "\n" ] } ], "source": [ "#this prompt is a sub-sample of one of the test set examples. In this example we start the generation after the model generation starts.\n", "prompt=\"\"\"human\n", "You are a function calling AI model. You are provided with function signatures within XML tags.You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions.Here are the available tools: [{'type': 'function', 'function': {'name': 'convert_currency', 'description': 'Convert from one currency to another', 'parameters': {'type': 'object', 'properties': {'amount': {'type': 'number', 'description': 'The amount to convert'}, 'from_currency': {'type': 'string', 'description': 'The currency to convert from'}, 'to_currency': {'type': 'string', 'description': 'The currency to convert to'}}, 'required': ['amount', 'from_currency', 'to_currency']}}}, {'type': 'function', 'function': {'name': 'calculate_distance', 'description': 'Calculate the distance between two locations', 'parameters': {'type': 'object', 'properties': {'start_location': {'type': 'string', 'description': 'The starting location'}, 'end_location': {'type': 'string', 'description': 'The ending location'}}, 'required': ['start_location', 'end_location']}}}] Use the following pydantic model json schema for each tool call you will make: {'title': 'FunctionCall', 'type': 'object', 'properties': {'arguments': {'title': 'Arguments', 'type': 'object'}, 'name': {'title': 'Name', 'type': 'string'}}, 'required': ['arguments', 'name']}For each function call return a json object with function name and arguments within XML tags as follows:\n", "\n", "{tool_call}\n", "Also, before making a call to a function take the time to plan the function to take. Make that thinking process between {your thoughts}\n", "\n", "Hi, I need to convert 500 USD to Euros. Can you help me with that?\n", "model\n", "\"\"\"\n", "\n", "inputs = tokenizer(prompt, return_tensors=\"pt\", add_special_tokens=False)\n", "inputs = {k: v.to(\"cuda\") for k,v in inputs.items()}\n", "outputs = model.generate(**inputs,\n", " max_new_tokens=300,\n", " do_sample=True,\n", " top_p=0.65,\n", " temperature=0.01,\n", " repetition_penalty=1.0,\n", " eos_token_id=tokenizer.eos_token_id)\n", "print(tokenizer.decode(outputs[0]))" ] }, { "cell_type": "markdown", "id": "xWewPCZOyfJQ", "metadata": { "id": "xWewPCZOyfJQ" }, "source": [ "## Congratulations\n", "Congratulations on finishing this first Bonus Unit 🥳\n", "\n", "You've just **mastered what Function-Calling is and how to fine-tune your model to do Function-Calling**!\n", "\n", "If it's the first time you do this, it's normal that you're feeling puzzled. Take time to check the documentation and understand each part of the code and why we did it this way.\n", "\n", "Also, don't hesitate to try to **fine-tune different models**. The **best way to learn is by trying.**\n", "\n", "### Keep Learning, Stay Awesome 🤗" ] } ], "metadata": { "colab": { "provenance": [] }, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.5" } }, "nbformat": 4, "nbformat_minor": 5 }