t5-base-finetuned-qmsum

This model is a fine-tuned version of google-t5/t5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.1567
• Rouge1: 28.3882
• Rouge2: 8.4191
• Rougel: 22.8604

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel
3.5399	1.0	126	3.2929	27.9871	8.2442	23.2939
3.1401	2.0	252	3.2076	27.7588	7.6926	22.8498
2.9706	3.0	378	3.1678	28.9533	8.4516	23.4899
2.8244	4.0	504	3.1509	28.274	8.0721	22.897
2.7238	5.0	630	3.1472	27.9718	8.26	22.7717
2.6687	6.0	756	3.1513	28.3972	8.4436	22.9446
2.5844	7.0	882	3.1554	28.6233	8.5011	23.1638
2.5715	8.0	1008	3.1567	28.3882	8.4191	22.8604

Framework versions

• Transformers 4.42.4
• Pytorch 2.3.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1