Date= qui 23 set 2021 10:25:08 -03
Folder=/mnt/Files/nl2sql/gap-text2sql/rat-sql-gap
SubFolder=iedir/mT5-large-NoGAP-en-pt-es-fr-train
Gold File=/mnt/Files/nl2sql/gap-text2sql/rat-sql-gap/ie_dirs/mT5-large-NoGAP-en-pt-es-fr-train/divided/gold-en_mT5-large-NoGAP-en-pt-es-fr-train_en-pt-es-fr-eval.txt
Predict File=/mnt/Files/nl2sql/gap-text2sql/rat-sql-gap/ie_dirs/mT5-large-NoGAP-en-pt-es-fr-train/divided/predict-en_mT5-large-NoGAP-en-pt-es-fr-train_en-pt-es-fr-eval_1_true_1-step-step.txt
Database=/mnt/Files/nl2sql/gap-text2sql/rat-sql-gap/data/spider-en/database
Type=match
Command example:
python3 evaluation.py --gold=/mnt/Files/nl2sql/gap-text2sql/rat-sql-gap/ie_dirs/mT5-large-NoGAP-en-pt-es-fr-train/divided/gold-en_mT5-large-NoGAP-en-pt-es-fr-train_en-pt-es-fr-eval.txt --pred=/mnt/Files/nl2sql/gap-text2sql/rat-sql-gap/ie_dirs/mT5-large-NoGAP-en-pt-es-fr-train/divided/predict-en_mT5-large-NoGAP-en-pt-es-fr-train_en-pt-es-fr-eval_1_true_1-step51300.txt --db=/mnt/Files/nl2sql/gap-text2sql/rat-sql-gap/data/spider-en/database --table=/mnt/Files/nl2sql/gap-text2sql/rat-sql-gap/data/spider-en/tables.json --etype match --plug_value
====================== EXACT MATCHING ACCURACY =====================
. . . . count 248 446 174 166 1034
Train Infer Checkpoint Type Type Easy Medium Hard Extra All
mT5-large-NoGAP-en-pt-es-fr-train en-eval 42100 exact match 0.875 0.742 0.569 0.476 0.702
mT5-large-NoGAP-en-pt-es-fr-train en-eval 49100 exact match 0.867 0.729 0.592 0.494 0.701
mT5-large-NoGAP-en-pt-es-fr-train en-eval 50100 exact match 0.891 0.735 0.615 0.476 0.711
mT5-large-NoGAP-en-pt-es-fr-train en-eval 51100 exact match 0.887 0.753 0.609 0.464 0.715
mT5-large-NoGAP-en-pt-es-fr-train en-eval 51300 exact match 0.887 0.742 0.609 0.458 0.709