Hugging Face's logo

NECOUDBFM
/
Jellyfish-8B

Text Generation
Transformers
Safetensors
English
llama
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
yuyangdong commited on
Commit
f43388e
·
verified ·
1 Parent(s): fc8fd87

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +19 -19
README.md CHANGED
@@ -45,25 +45,25 @@ If you find our work useful, please give us credit by citing:
45
 
46
  | Task | Type | Dataset | Non-LLM SoTA<sup>1</sup> | GPT-3.5<sup>2</sup> | GPT-4<sup>2</sup> | Jellyfish-13B| Jellyfish-7B | Jellyfish-8B |
47
  | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |
48
- | Entity Matching | Seen | Fodors-Zagats | 100 | 100 | 100 | 100 | 100 | 97.67 |
49
- | Entity Matching | Seen | Beer | 94.37| 96.30 | 100 | 96.77 | 96.55| 100 |
50
- | Entity Matching | Seen | iTunes-Amazon | 97.06| 96.43 | 100 | 98.11 | 96.30| 96.30 |
51
- | Entity Matching | Seen | DBLP-ACM | 98.99| 96.99 | 97.44 | 98.98 | 98.88| 99.44 |
52
- | Entity Matching | Seen | DBLP-GoogleScholar | 95.60| 76.12 | 91.87 | 98.51 | 95.15| 95.26 |
53
- | Entity Matching | Seen | Amazon-Google | 75.58| 66.53 | 74.21 | 81.34 | 80.83 | 80.23 |
54
- | Entity Matching | Unseen | Walmart-Amazon | 86.76| 86.17 | 90.27 | 89.42 | 85.64 | 90.81 |
55
- | Entity Matching | Unseen | Abt-Buy | 89.33 | -- | 92.77 | 89.58 | 82.38 | 93.60 |
56
- | Data Imputation | Seen | Restaurant | 77.20| 94.19 | 97.67 | 94.19 | 88.37 | 88.37 |
57
- | Data Imputation | Seen | Buy | 96.50| 98.46 | 100 | 100 | 96.62 | 98.46 |
58
- | Data Imputation | Unseen | Filpkart | 68.00 | -- | 89.94 | 81.68 | 79.44| 92.34 |
59
- | Data Imputation | Unseen | Phone | 86.70| -- | 90.79 | 87.21 | 85.00| 86.93 |
60
- | Error Detection | Seen | Hosptial | 94.40| 90.74 | 90.74 | 95.59 | 96.27 | 93.66|
61
- | Error Detection | Seen | Adult | 99.10| 92.01 | 92.01 | 99.33 | 91.96 | 92.98|
62
- | Error Detection | Unseen | Flights | 81.00 | -- | 83.48 | 82.52 | 66.92 | 82.33 |
63
- | Error Detection | Unseen | Rayyan | 79.00| -- | 81.95 | 90.65 | 69.82 | 98.31 |
64
- | Schema Matching | Seen | Sythea | 38.50| 57.14 | 66.67 | 36.36 | 44.44 | 40.00 |
65
- | Schema Matching | Seen | MIMIC | 20.00| -- | 40.00 | 40.00 | 40.00 | 40.82|
66
- | Schema Matching | Unseen | CMS | 50.00| -- | 19.35 | 59.29 | 13.79 | 68.49|
67
 
68
  _For GPT-3.5 and GPT-4, we used the few-shot approach on all datasets. However, for Jellyfish-13B and Jellyfish-Interpreter, the few-shot approach is disabled on seen datasets and enabled on unseen datasets._
69
  _Accuracy as the metric for data imputation and the F1 score for other tasks._
 
45
 
46
  | Task | Type | Dataset | Non-LLM SoTA<sup>1</sup> | GPT-3.5<sup>2</sup> | GPT-4<sup>2</sup> | Jellyfish-13B| Jellyfish-7B | Jellyfish-8B |
47
  | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |
48
+ | Entity Matching | Seen | Fodors-Zagats | 100 | 100 | 100 | 100 | 100 | 92.68 |
49
+ | Entity Matching | Seen | Beer | 94.37| 96.30 | 100 | 96.77 | 96.55| 96.30 |
50
+ | Entity Matching | Seen | iTunes-Amazon | 97.06| 96.43 | 100 | 98.11 | 96.30| 92.00 |
51
+ | Entity Matching | Seen | DBLP-ACM | 98.99| 96.99 | 97.44 | 98.98 | 98.88| 98.76 |
52
+ | Entity Matching | Seen | DBLP-GoogleScholar | 95.60| 76.12 | 91.87 | 98.51 | 95.15| 93.20 |
53
+ | Entity Matching | Seen | Amazon-Google | 75.58| 66.53 | 74.21 | 81.34 | 80.83 | 74.49 |
54
+ | Entity Matching | Unseen | Walmart-Amazon | 86.76| 86.17 | 90.27 | 89.42 | 85.64 | 89.97 |
55
+ | Entity Matching | Unseen | Abt-Buy | 89.33 | -- | 92.77 | 89.58 | 82.38 | 92.54 |
56
+ | Data Imputation | Seen | Restaurant | 77.20| 94.19 | 97.67 | 94.19 | 88.37 | 87.21 |
57
+ | Data Imputation | Seen | Buy | 96.50| 98.46 | 100 | 100 | 96.62 | 92.31 |
58
+ | Data Imputation | Unseen | Filpkart | 68.00 | -- | 89.94 | 81.68 | 79.44| 90.17 |
59
+ | Data Imputation | Unseen | Phone | 86.70| -- | 90.79 | 87.21 | 85.00| 83.92 |
60
+ | Error Detection | Seen | Hosptial | 94.40| 90.74 | 90.74 | 95.59 | 96.27 | 80.72|
61
+ | Error Detection | Seen | Adult | 99.10| 92.01 | 92.01 | 99.33 | 91.96 | 81.72|
62
+ | Error Detection | Unseen | Flights | 81.00 | -- | 83.48 | 82.52 | 66.92 | 75.18 |
63
+ | Error Detection | Unseen | Rayyan | 79.00| -- | 81.95 | 90.65 | 69.82 | 91.54 |
64
+ | Schema Matching | Seen | Sythea | 38.50| 57.14 | 66.67 | 36.36 | 44.44 | 27.27 |
65
+ | Schema Matching | Seen | MIMIC | 20.00| -- | 40.00 | 40.00 | 40.00 | 34.04|
66
+ | Schema Matching | Unseen | CMS | 50.00| -- | 19.35 | 59.29 | 13.79 | 56.72|
67
 
68
  _For GPT-3.5 and GPT-4, we used the few-shot approach on all datasets. However, for Jellyfish-13B and Jellyfish-Interpreter, the few-shot approach is disabled on seen datasets and enabled on unseen datasets._
69
  _Accuracy as the metric for data imputation and the F1 score for other tasks._