long-t5-local-base-finetuned-justification-v01

This model is a fine-tuned version of google/long-t5-local-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.2782

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss
5.0215	1.0	676	2.2439
1.8427	2.0	1352	1.9667
1.4788	3.0	2028	1.8219
1.4318	4.0	2704	1.7180
1.36	5.0	3380	1.6445
1.2795	6.0	4056	1.5939
1.2306	7.0	4732	1.5489
1.2008	8.0	5408	1.5060
1.1682	9.0	6084	1.4751
1.1206	10.0	6760	1.4442
1.1637	11.0	7436	1.4284
1.06	12.0	8112	1.4042
1.0504	13.0	8788	1.3878
1.0627	14.0	9464	1.3797
0.9759	15.0	10140	1.3627
0.9984	16.0	10816	1.3512
0.9694	17.0	11492	1.3408
0.926	18.0	12168	1.3294
0.9894	19.0	12844	1.3268
0.9264	20.0	13520	1.3209
0.9287	21.0	14196	1.3170
0.9005	22.0	14872	1.3068
0.8925	23.0	15548	1.3105
0.9153	24.0	16224	1.3091
0.867	25.0	16900	1.2988
0.8946	26.0	17576	1.2933
0.8671	27.0	18252	1.2958
0.8247	28.0	18928	1.2906
0.8585	29.0	19604	1.2899
0.8453	30.0	20280	1.2951
0.8339	31.0	20956	1.2893
0.8483	32.0	21632	1.2915
0.8026	33.0	22308	1.2864
0.8126	34.0	22984	1.2813
0.8384	35.0	23660	1.2788
0.7992	36.0	24336	1.2803
0.8104	37.0	25012	1.2777
0.8145	38.0	25688	1.2850
0.7612	39.0	26364	1.2812
0.7684	40.0	27040	1.2773
0.7829	41.0	27716	1.2786
0.775	42.0	28392	1.2751
0.8112	43.0	29068	1.2783
0.7733	44.0	29744	1.2774
0.7525	45.0	30420	1.2795
0.7667	46.0	31096	1.2785
0.7734	47.0	31772	1.2765
0.7712	48.0	32448	1.2775
0.7764	49.0	33124	1.2775
0.783	50.0	33800	1.2782

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2