GPT4 template
Is it possible to use 'GPT4 template' with functions?
messages = [
{"role": "user", "content": "Hello"},
{"role": "assistant", "content": "Hi"},
{"role": "user", "content": "How are you today?"}
]
tokens = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
assert tokens == [1, 420, 6316, 28781, 3198, 3123, 1247, 28747, 22557, 32000, 420, 6316, 28781, 3198, 3123, 21631, 28747, 15359, 32000, 420, 6316, 28781, 3198, 3123, 1247, 28747, 1602, 460, 368, 3154, 28804, 32000, 420, 6316, 28781, 3198, 3123, 21631, 28747]
Howdy @raestrada , right now the template for apply_chat_template is that of the base model.
So, it's not possible to use that right now.
That said, I think it's a good idea for me to update the template so that it can be used. It would make things quite a bit easier. Let me take a look over the next two days.
In the meantime, I've added more info to the model card now to help with prompt setup.
Howdy @raestrada I've added a chat template. LMK how it goes
Quick other updates:
- I've compared this now with Llama 2 7B and Mistral 7B. OpenChat is stronger and capable of chaining function calls together. For example:
user: What clothes should I wear? I am in Dublin
function_call: {
"name": "get_current_weather",
"arguments": {
"city": "Dublin"
}
}
function_response: {
"temperature": "18 C",
"condition": "Partly Cloudy"
}
function_call: {
"name": "get_clothes",
"arguments": {
"temperature": "18 C",
"condition": "Partly Cloudy"
}
}
function_response: moderate clothing, like a long-sleeve shirt and jeans
assistant: You should wear moderate clothing, like a long-sleeve shirt and jeans
- I'm not sure if you have bought access to the inference repo, but I've been working on making function calling and execution seamless via API. Hoping to make a video within the next week.
@RonanMcGovern
I am working on a Hackathon project of migrating from OpenAI function calling to opensource alternatives for cal.com.
Pull Request: https://github.com/calcom/cal.com/pull/12669/files#diff-a7c21727d9a0d4d1aaff9d24fca9293bc28468861bc8022dfa03116908b6fae7
Could you grant me access to the model?
howdy @sp6370 , this model here is paid, but for an open (Llama 2 community licensed) model you may wanna check out this one: https://huggingface.co/Trelis/Llama-2-7b-chat-hf-function-calling-v3
Kindly create a new/separate issue if you have further Qs, cheers, Ronan
Quick other updates:
- I've compared this now with Llama 2 7B and Mistral 7B. OpenChat is stronger and capable of chaining function calls together. For example:
user: What clothes should I wear? I am in Dublin function_call: { "name": "get_current_weather", "arguments": { "city": "Dublin" } } function_response: { "temperature": "18 C", "condition": "Partly Cloudy" } function_call: { "name": "get_clothes", "arguments": { "temperature": "18 C", "condition": "Partly Cloudy" } } function_response: moderate clothing, like a long-sleeve shirt and jeans assistant: You should wear moderate clothing, like a long-sleeve shirt and jeans
- I'm not sure if you have bought access to the inference repo, but I've been working on making function calling and execution seamless via API. Hoping to make a video within the next week.
Hi @RonanMcGovern thank you for the update in the chat template, it was very helpful and I now have it working and it functions very well. I also just watched the YouTube video, it's very good and I loved the Spanish part. I don't have access to that repo currently, but I am considering buying the DeepSeek model after finishing some tests and implementations with openchat. Thank you for all the help.
Excellent @raestrada !