javi8979 commited on
Commit
1129866
·
verified ·
1 Parent(s): 2066e91

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +155 -0
README.md CHANGED
@@ -438,6 +438,161 @@ Below are the evaluation results on the Flores+200 devtest set, compared against
438
  | SalamandraTA-7b-base | 24.71 | 62.33 | 52.96 | **0.85** | **0.84** | 0.73 | 1.06 | 1.37 |
439
 
440
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
441
 
442
  ## Ethical Considerations and Limitations
443
 
 
438
  | SalamandraTA-7b-base | 24.71 | 62.33 | 52.96 | **0.85** | **0.84** | 0.73 | 1.06 | 1.37 |
439
 
440
 
441
+ <details>
442
+ <summary>Click to show full table ES-XX Flores+devtest</summary>
443
+
444
+ | | Source | Target | Bleu↑ | Ter↓ | ChrF↑ | Comet↑ | Comet-kiwi↑ | Bleurt↑ | MetricX↓ | MetricX-QE↓ |
445
+ |:---------------------------------|:---------|:---------|-------:|------:|-------:|--------:|-------------:|---------:|----------:|-------------:|
446
+ | SalamandraTA-7b-base | es | uk | 19.15 | 74.57 | 49.57 | 0.88 | 0.85 | 0.76 | 1.33 | 0.78 |
447
+ | MADLAD400-7B | es | uk | 19.19 | 74.61 | 49.87 | 0.89 | 0.86 | 0.77 | 1.2 | 0.75 |
448
+ | SalamandraTA-7b-instruct | es | uk | 20.42 | 70.67 | 49.85 | 0.89 | 0.85 | 0.77 | 1.18 | 0.85 |
449
+ | | | | | | | | | | | |
450
+ | MADLAD400-7B | es | sv | 26.55 | 63.25 | 57.28 | 0.89 | 0.86 | 0.8 | 0.68 | 0.69 |
451
+ | SalamandraTA-7b-base | es | sv | 25.87 | 64.49 | 56.08 | 0.88 | 0.85 | 0.79 | 0.87 | 1 |
452
+ | SalamandraTA-7b-instruct | es | sv | 27.54 | 60.34 | 56.72 | 0.89 | 0.85 | 0.8 | 0.64 | 0.72 |
453
+ | | | | | | | | | | | |
454
+ | MADLAD400-7B | es | sr | 22.48 | 66.61 | 52.22 | 0.88 | 0.86 | 0.83 | 1.14 | 1.25 |
455
+ | SalamandraTA-7b-instruct | es | sr | 22.06 | 66.38 | 51.32 | 0.88 | 0.86 | 0.82 | 1.16 | 0.95 |
456
+ | SalamandraTA-7b-base | es | sr | 21.02 | 69.04 | 51 | 0.87 | 0.86 | 0.81 | 1.46 | 1 |
457
+ | | | | | | | | | | | |
458
+ | SalamandraTA-7b-instruct | es | sl | 22.24 | 68.56 | 51.13 | 0.89 | 0.84 | 0.84 | 0.76 | 0.96 |
459
+ | MADLAD400-7B | es | sl | 22.84 | 69.52 | 52.85 | 0.89 | 0.85 | 0.85 | 0.64 | 0.67 |
460
+ | SalamandraTA-7b-base | es | sl | 20.86 | 72.13 | 51.11 | 0.88 | 0.85 | 0.83 | 0.99 | 0.83 |
461
+ | | | | | | | | | | | |
462
+ | SalamandraTA-7b-instruct | es | sk | 21.96 | 67.95 | 50.76 | 0.9 | 0.85 | 0.81 | 0.78 | 0.88 |
463
+ | MADLAD400-7B | es | sk | 22.52 | 69.42 | 52.29 | 0.9 | 0.86 | 0.83 | 0.66 | 0.72 |
464
+ | SalamandraTA-7b-base | es | sk | 20.96 | 71.63 | 50.87 | 0.89 | 0.86 | 0.8 | 0.98 | 0.82 |
465
+ | | | | | | | | | | | |
466
+ | SalamandraTA-7b-base | es | ru | 21.07 | 72.57 | 50.84 | 0.88 | 0.84 | 0.75 | 1.3 | 0.84 |
467
+ | MADLAD400-7B | es | ru | 21.41 | 72.23 | 52.16 | 0.89 | 0.85 | 0.76 | 1.07 | 0.74 |
468
+ | SalamandraTA-7b-instruct | es | ru | 21.95 | 68.71 | 51.26 | 0.89 | 0.83 | 0.76 | 1.1 | 0.9 |
469
+ | | | | | | | | | | | |
470
+ | SalamandraTA-7b-base | es | ro | 25.38 | 65.94 | 54.6 | 0.88 | 0.86 | 0.8 | 1 | 0.85 |
471
+ | SalamandraTA-7b-instruct | es | ro | 27.71 | 62.01 | 54.8 | 0.89 | 0.85 | 0.81 | 0.87 | 0.97 |
472
+ | MADLAD400-7B | es | ro | 26.65 | 64.11 | 55.68 | 0.89 | 0.87 | 0.82 | 0.76 | 0.78 |
473
+ | | | | | | | | | | | |
474
+ | SalamandraTA-7b-base | es | pt | 26.23 | 63.59 | 56.16 | 0.88 | 0.85 | 0.74 | 0.9 | 1.11 |
475
+ | MADLAD400-7B | es | pt | 27.11 | 62.5 | 56.95 | 0.88 | 0.85 | 0.74 | 0.84 | 1.01 |
476
+ | SalamandraTA-7b-instruct | es | pt | 32.05 | 56.01 | 58.85 | 0.88 | 0.84 | 0.75 | 0.77 | 0.99 |
477
+ | | | | | | | | | | | |
478
+ | SalamandraTA-7b-base | es | pl | 16.98 | 77.3 | 47.99 | 0.89 | 0.85 | 0.79 | 0.99 | 0.81 |
479
+ | MADLAD400-7B | es | pl | 18.07 | 76.69 | 49.32 | 0.89 | 0.85 | 0.81 | 0.84 | 0.8 |
480
+ | SalamandraTA-7b-instruct | es | pl | 18.6 | 73.56 | 48.14 | 0.9 | 0.84 | 0.81 | 0.71 | 0.68 |
481
+ | | | | | | | | | | | |
482
+ | SalamandraTA-7b-instruct | es | oc | 17.65 | 71.75 | 49.05 | 0.69 | 0.6 | 0.45 | 2.59 | 3.19 |
483
+ | MADLAD400-7B | es | oc | 17.02 | 73.48 | 47.74 | 0.66 | 0.55 | 0.4 | 3.87 | 5.55 |
484
+ | SalamandraTA-7b-base | es | oc | 16.06 | 75.17 | 47.47 | 0.69 | 0.62 | 0.45 | 2.66 | 3.55 |
485
+ | | | | | | | | | | | |
486
+ | SalamandraTA-7b-base | es | nn | 18.87 | 76.97 | 49.2 | 0.86 | 0.83 | 0.69 | 1.12 | 1.3 |
487
+ | MADLAD400-7B | es | nn | 21.69 | 67.33 | 51.71 | 0.87 | 0.84 | 0.7 | 0.84 | 1.06 |
488
+ | SalamandraTA-7b-instruct | es | nn | 20.13 | 68.82 | 49.91 | 0.86 | 0.83 | 0.69 | 1.03 | 1.03 |
489
+ | | | | | | | | | | | |
490
+ | MADLAD400-7B | es | nl | 20.94 | 70.16 | 53.83 | 0.86 | 0.86 | 0.77 | 0.77 | 0.71 |
491
+ | SalamandraTA-7b-instruct | es | nl | 21.63 | 66.87 | 53.23 | 0.86 | 0.84 | 0.76 | 0.78 | 0.84 |
492
+ | SalamandraTA-7b-base | es | nl | 20.12 | 71.76 | 53.1 | 0.85 | 0.85 | 0.75 | 0.95 | 0.82 |
493
+ | | | | | | | | | | | |
494
+ | SalamandraTA-7b-base | es | mt | 17.26 | 73.99 | 55.22 | 0.66 | 0.65 | 0.26 | 2.12 | 1.74 |
495
+ | SalamandraTA-7b-instruct | es | mt | 20.66 | 65.35 | 56.07 | 0.67 | 0.62 | 0.3 | 1.97 | 1.93 |
496
+ | MADLAD400-7B | es | mt | 20.36 | 67.41 | 56.96 | 0.67 | 0.65 | 0.29 | 1.67 | 1.47 |
497
+ | | | | | | | | | | | |
498
+ | SalamandraTA-7b-base | es | lv | 20.51 | 72.13 | 52.25 | 0.88 | 0.81 | 0.81 | 1.49 | 0.93 |
499
+ | MADLAD400-7B | es | lv | 22.24 | 70.18 | 54.08 | 0.89 | 0.82 | 0.84 | 1.12 | 0.89 |
500
+ | SalamandraTA-7b-instruct | es | lv | 21.57 | 69.41 | 52.32 | 0.89 | 0.8 | 0.84 | 1.16 | 0.98 |
501
+ | | | | | | | | | | | |
502
+ | MADLAD400-7B | es | lt | 19.38 | 74.98 | 53.39 | 0.89 | 0.83 | 0.85 | 1.07 | 1.22 |
503
+ | SalamandraTA-7b-base | es | lt | 17.87 | 75.34 | 51.84 | 0.88 | 0.82 | 0.83 | 1.37 | 0.91 |
504
+ | SalamandraTA-7b-instruct | es | lt | 19.26 | 73.59 | 51.18 | 0.89 | 0.81 | 0.84 | 0.96 | 0.8 |
505
+ | | | | | | | | | | | |
506
+ | SalamandraTA-7b-base | es | it | 23.91 | 67.25 | 54.32 | 0.88 | 0.86 | 0.77 | 0.93 | 0.92 |
507
+ | MADLAD400-7B | es | it | 24.97 | 66.18 | 54.93 | 0.88 | 0.86 | 0.78 | 0.87 | 0.81 |
508
+ | SalamandraTA-7b-instruct | es | it | 26.42 | 63.18 | 54.89 | 0.88 | 0.85 | 0.78 | 0.77 | 0.87 |
509
+ | | | | | | | | | | | |
510
+ | MADLAD400-7B | es | hu | 18.84 | 72.31 | 52.62 | 0.88 | 0.85 | 0.9 | 0.72 | 0.79 |
511
+ | SalamandraTA-7b-instruct | es | hu | 18.86 | 70.36 | 51.18 | 0.88 | 0.84 | 0.89 | 0.8 | 0.79 |
512
+ | SalamandraTA-7b-base | es | hu | 18.14 | 73.22 | 51.26 | 0.87 | 0.84 | 0.89 | 0.97 | 0.65 |
513
+ | | | | | | | | | | | |
514
+ | SalamandraTA-7b-base | es | hr | 20.05 | 71.73 | 52.09 | 0.89 | 0.86 | 0.83 | 1.89 | 1.6 |
515
+ | MADLAD400-7B | es | hr | 21.07 | 70.6 | 53.07 | 0.9 | 0.86 | 0.84 | 1.59 | 1.71 |
516
+ | SalamandraTA-7b-instruct | es | hr | 22.14 | 66.85 | 52.72 | 0.9 | 0.85 | 0.83 | 1.59 | 1.58 |
517
+ | | | | | | | | | | | |
518
+ | SalamandraTA-7b-base | es | gl | 22.28 | 66.67 | 52.86 | 0.87 | 0.84 | 0.67 | 1.13 | 1.81 |
519
+ | MADLAD400-7B | es | gl | 21.98 | 66.74 | 53.1 | 0.87 | 0.84 | 0.67 | 0.94 | 1.42 |
520
+ | SalamandraTA-7b-instruct | es | gl | 28.27 | 58.19 | 56.35 | 0.88 | 0.85 | 0.7 | 0.79 | 0.97 |
521
+ | | | | | | | | | | | |
522
+ | SalamandraTA-7b-instruct | es | ga | 20.07 | 70.37 | 49.64 | 0.77 | 0.65 | 0.6 | 1.93 | 1.7 |
523
+ | SalamandraTA-7b-base | es | ga | 20.73 | 71.47 | 50.62 | 0.76 | 0.67 | 0.6 | 1.87 | 1.48 |
524
+ | MADLAD400-7B | es | ga | 21.5 | 70.22 | 51.6 | 0.77 | 0.67 | 0.61 | 1.89 | 1.61 |
525
+ | | | | | | | | | | | |
526
+ | SalamandraTA-7b-instruct | es | fr | 33.83 | 54.94 | 60.21 | 0.86 | 0.85 | 0.74 | 0.87 | 1.21 |
527
+ | SalamandraTA-7b-base | es | fr | 30.8 | 59.03 | 59.38 | 0.86 | 0.86 | 0.73 | 0.92 | 1.06 |
528
+ | MADLAD400-7B | es | fr | 30.68 | 59.12 | 59.76 | 0.86 | 0.86 | 0.73 | 0.88 | 1.04 |
529
+ | | | | | | | | | | | |
530
+ | SalamandraTA-7b-base | es | fi | 15.84 | 76.23 | 52.44 | 0.89 | 0.82 | 0.82 | 1.16 | 0.79 |
531
+ | MADLAD400-7B | es | fi | 18.49 | 74.35 | 55.35 | 0.91 | 0.84 | 0.85 | 0.71 | 0.54 |
532
+ | SalamandraTA-7b-instruct | es | fi | 18.36 | 70.94 | 53.7 | 0.91 | 0.83 | 0.85 | 0.61 | 0.5 |
533
+ | | | | | | | | | | | |
534
+ | SalamandraTA-7b-base | es | eu | 14.72 | 78.76 | 54.44 | 0.86 | 0.85 | 0.79 | 0.95 | 0.76 |
535
+ | SalamandraTA-7b-instruct | es | eu | 16.52 | 75.42 | 54.27 | 0.86 | 0.84 | 0.8 | 0.73 | 0.6 |
536
+ | MADLAD400-7B | es | eu | 13.24 | 84.15 | 54.54 | 0.87 | 0.86 | 0.81 | 0.76 | 0.63 |
537
+ | | | | | | | | | | | |
538
+ | SalamandraTA-7b-base | es | et | 17.98 | 74.39 | 52.84 | 0.89 | 0.83 | 0.81 | 1.01 | 0.83 |
539
+ | SalamandraTA-7b-instruct | es | et | 18.91 | 70.78 | 52.93 | 0.9 | 0.83 | 0.82 | 0.69 | 0.77 |
540
+ | MADLAD400-7B | es | et | 17.21 | 78.52 | 52.85 | 0.89 | 0.83 | 0.82 | 0.92 | 1.54 |
541
+ | | | | | | | | | | | |
542
+ | SalamandraTA-7b-base | es | en | 32.37 | 55.92 | 61.52 | 0.87 | 0.86 | 0.78 | 1.26 | 1.41 |
543
+ | MADLAD400-7B | es | en | 31.45 | 56.44 | 61.37 | 0.88 | 0.86 | 0.77 | 1.23 | 1.05 |
544
+ | SalamandraTA-7b-instruct | es | en | 37.22 | 49.72 | 63.39 | 0.88 | 0.86 | 0.78 | 1.09 | 1.07 |
545
+ | | | | | | | | | | | |
546
+ | SalamandraTA-7b-base | es | el | 20.14 | 70.61 | 48.13 | 0.88 | 0.84 | 0.74 | 1.37 | 0.91 |
547
+ | MADLAD400-7B | es | el | 21.44 | 68.53 | 49.72 | 0.88 | 0.84 | 0.75 | 1.24 | 0.97 |
548
+ | SalamandraTA-7b-instruct | es | el | 20.63 | 67.07 | 48.29 | 0.88 | 0.84 | 0.74 | 1.14 | 0.79 |
549
+ | | | | | | | | | | | |
550
+ | SalamandraTA-7b-instruct | es | de | 25.67 | 63.66 | 55.75 | 0.85 | 0.84 | 0.77 | 0.68 | 0.95 |
551
+ | MADLAD400-7B | es | de | 25.67 | 65.67 | 56.68 | 0.85 | 0.85 | 0.77 | 0.68 | 0.8 |
552
+ | SalamandraTA-7b-base | es | de | 24.69 | 67.29 | 55.79 | 0.85 | 0.84 | 0.76 | 0.75 | 1.02 |
553
+ | | | | | | | | | | | |
554
+ | SalamandraTA-7b-instruct | es | da | 29.24 | 59.71 | 57.14 | 0.89 | 0.84 | 0.77 | 0.72 | 0.76 |
555
+ | MADLAD400-7B | es | da | 28.59 | 62.21 | 57.84 | 0.89 | 0.85 | 0.77 | 0.68 | 0.75 |
556
+ | SalamandraTA-7b-base | es | da | 27.26 | 63.93 | 56.75 | 0.88 | 0.84 | 0.76 | 0.97 | 1.03 |
557
+ | | | | | | | | | | | |
558
+ | SalamandraTA-7b-instruct | es | cy | 25.35 | 63.81 | 54.64 | 0.83 | 0.76 | 0.48 | 1.85 | 1.65 |
559
+ | SalamandraTA-7b-base | es | cy | 24.94 | 65.32 | 55.22 | 0.82 | 0.77 | 0.48 | 1.72 | 1.39 |
560
+ | MADLAD400-7B | es | cy | 23.64 | 67.17 | 53.03 | 0.83 | 0.78 | 0.47 | 1.83 | 1.63 |
561
+ | | | | | | | | | | | |
562
+ | MADLAD400-7B | es | cs | 21.87 | 69.43 | 51.43 | 0.91 | 0.87 | 0.82 | 0.7 | 0.76 |
563
+ | SalamandraTA-7b-instruct | es | cs | 21.76 | 67.89 | 50 | 0.9 | 0.86 | 0.81 | 0.68 | 0.71 |
564
+ | SalamandraTA-7b-base | es | cs | 20.42 | 71.58 | 49.78 | 0.9 | 0.86 | 0.8 | 0.95 | 0.86 |
565
+ | | | | | | | | | | | |
566
+ | SalamandraTA-7b-base | es | ca | 23.31 | 66.87 | 54.09 | 0.86 | 0.85 | 0.72 | 1.13 | 1.33 |
567
+ | SalamandraTA-7b-instruct | es | ca | 31.15 | 57.76 | 57.49 | 0.87 | 0.84 | 0.75 | 0.92 | 1.18 |
568
+ | MADLAD400-7B | es | ca | 23.1 | 66.98 | 54.12 | 0.86 | 0.85 | 0.73 | 1.1 | 1.19 |
569
+ | | | | | | | | | | | |
570
+ | SalamandraTA-7b-instruct | es | bg | 27.47 | 62.02 | 56.35 | 0.9 | 0.86 | 0.87 | 1 | 0.86 |
571
+ | MADLAD400-7B | es | bg | 27.2 | 64.02 | 57.46 | 0.9 | 0.87 | 0.87 | 1.09 | 0.92 |
572
+ | SalamandraTA-7b-base | es | bg | 26.41 | 65.63 | 56.51 | 0.89 | 0.87 | 0.86 | 1.13 | 0.85 |
573
+
574
+ </details>
575
+
576
+
577
+ ### Catalan
578
+
579
+ | | Bleu↑ | Ter↓ | ChrF↑ | Comet↑ | Comet-kiwi↑ | Bleurt↑ | MetricX↓ | MetricX-QE↓ |
580
+ |:---------------------------------|-------:|------:|-------:|--------:|-------------:|---------:|----------:|-------------:|
581
+ | **CA-XX** | | | | | | | | |
582
+ | MADLAD400-7B | **29.37** | 59.01 | **58.47** | **0.87** | **0.81** | **0.77** | **1.08** | 1.31 |
583
+ | SalamandraTA-7b-instruct | 29.23 | **58.32** | 57.76 | **0.87** | **0.81** | **0.77** | **1.08** | **1.22** |
584
+ | SalamandraTA-7b-base | 29.06 | 59.32 | 58 | **0.87** | **0.81** | 0.76 | 1.23 | 1.28 |
585
+ | **XX-CA** | | | | | | | | |
586
+ | SalamandraTA-7b-instruct | **33.64** | **54.49** | 59.03 | **0.86** | 0.8 | **0.75** | **1.07** | **1.6** |
587
+ | MADLAD400-7B | 33.02 | 55.01 | 59.38 | **0.86** | **0.81** | **0.75** | 1.18 | 1.79 |
588
+ | SalamandraTA-7b-base | 32.75 | 55.78 | **59.42** | **0.86** | **0.81** | **0.75** | 1.17 | 1.63 |
589
+
590
+ <details>
591
+ <summary>Click to show full table CA-XX Flores+devtest</summary>
592
+
593
+
594
+ </details>
595
+
596
 
597
  ## Ethical Considerations and Limitations
598