minpeter
/

Llama-3.2-1B-chatml-tool-v2

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

minpeter commited on Feb 9

Commit

1635bf7

·

verified ·

1 Parent(s): 48b692b

Update README.md

Files changed (1) hide show

README.md +7 -6

README.md CHANGED Viewed

@@ -26,11 +26,12 @@ need to check whether this phenomenon is repeated in larger models (3B, 8B).
 ## Model Performance Comparison (BFCL)
-| task name        | minpeter/Llama-3.2-1B-chatml-tool-v2 | meta-llama/Llama-3.2-1B-Instruct |
-|-----------------|-----------------------------------|-----------------------------------|
-| parallel_multiple | 0.000                               | 0.025                               |
-| parallel        | 0.000                               | 0.035                               |
-| simple          | 0.72                             | 0.215                               |
-| multiple        | 0.695                                | 0.17                                |
 *Parallel calls are not taken into account. 0 points are expected. We plan to fix this in v3.

 ## Model Performance Comparison (BFCL)
+| task name        | minpeter/Llama-3.2-1B-chatml-tool-v2 | meta-llama/Llama-3.2-1B-Instruct (measure) | meta-llama/Llama-3.2-1B-Instruct (Reported) |
+|-----------------|-----------------------------------|-----------------------------------|---------------------------------------|
+| parallel_multiple | 0.000                               | 0.025                               | **0.15** |
+| parallel        | 0.000                               | 0.035                               | **0.36** |
+| simple          | **0.72**                             | 0.215                               | 0.2925 |
+| multiple        | **0.695**                                | 0.17                                | 0.335 |
 *Parallel calls are not taken into account. 0 points are expected. We plan to fix this in v3.