Xenova HF Staff commited on
Commit
4a97241
·
verified ·
1 Parent(s): 36a0454

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +73 -1
README.md CHANGED
@@ -112,7 +112,8 @@ If you haven't already, you can install the [Transformers.js](https://huggingfac
112
  npm i @huggingface/transformers
113
  ```
114
 
115
- You can then generate text as follows:
 
116
  ```js
117
  import { pipeline, TextStreamer } from "@huggingface/transformers";
118
 
@@ -139,6 +140,77 @@ console.log(output[0].generated_text.at(-1).content);
139
  // The capital of France is Paris.
140
  ```
141
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
142
  ### ONNXRuntime
143
 
144
  ```py
 
112
  npm i @huggingface/transformers
113
  ```
114
 
115
+ **Example**: Basic example
116
+
117
  ```js
118
  import { pipeline, TextStreamer } from "@huggingface/transformers";
119
 
 
140
  // The capital of France is Paris.
141
  ```
142
 
143
+
144
+ **Example**: Tool calling
145
+
146
+ ```js
147
+ import { AutoModelForCausalLM, AutoTokenizer, TextStreamer } from "@huggingface/transformers";
148
+
149
+ // Load tokenizer and model
150
+ const model_id = "onnx-community/LFM2-1.2B-ONNX";
151
+ const tokenizer = await AutoTokenizer.from_pretrained(model_id);
152
+ const model = await AutoModelForCausalLM.from_pretrained(
153
+ model_id, { dtype: "q4f16", device: "webgpu" },
154
+ );
155
+
156
+ const tools = [
157
+ {
158
+ name: "get_weather",
159
+ description: "Get current weather information for a location",
160
+ parameters: {
161
+ type: "object",
162
+ properties: {
163
+ location: {
164
+ type: "string",
165
+ description: "The city and state, e.g. San Francisco, CA",
166
+ },
167
+ unit: {
168
+ type: "string",
169
+ enum: ["celsius", "fahrenheit"],
170
+ description: "The unit of temperature to use",
171
+ },
172
+ },
173
+ required: ["location"],
174
+ },
175
+ },
176
+ ];
177
+ const messages = [
178
+ {
179
+ role: "user",
180
+ content: "What's the weather like in New York?"
181
+ },
182
+ ];
183
+
184
+ // Tokenize input
185
+ const input = tokenizer.apply_chat_template(messages, {
186
+ tools,
187
+ add_generation_prompt: true,
188
+ return_dict: true,
189
+ });
190
+
191
+ // Set up the streamer
192
+ const streamer = new TextStreamer(tokenizer, {
193
+ skip_prompt: true,
194
+ skip_special_tokens: false,
195
+ });
196
+
197
+ // Generate output
198
+ const sequences = await model.generate({
199
+ ...input,
200
+ max_new_tokens: 512,
201
+ do_sample: false,
202
+ streamer,
203
+ });
204
+
205
+ // Decode and print the generated text
206
+ const response = tokenizer.batch_decode(
207
+ sequences.slice(null, [input.input_ids.dims[1], null]),
208
+ { skip_special_tokens: true },
209
+ );
210
+ console.log(response[0]); // [get_weather(location="New York", unit="fahrenheit")]
211
+ ```
212
+
213
+
214
  ### ONNXRuntime
215
 
216
  ```py