flan-t5-base-model2

This model is a fine-tuned version of google/flan-t5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2236
• Rouge1: 73.2746
• Rouge2: 65.1173
• Rougel: 72.149
• Rougelsum: 73.1838
• Gen Len: 16.1625

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 200
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
11.4224	0.71	200	0.5002	49.2994	40.066	49.0355	49.3658	7.6113
0.4129	1.41	400	0.3030	72.7746	63.8666	71.4539	72.5604	16.2138
0.3131	2.12	600	0.2793	73.6239	64.6532	72.2694	73.5208	16.1148
0.2615	2.83	800	0.2674	73.2672	64.6251	72.0459	73.0736	16.2067
0.2347	3.53	1000	0.2631	73.0069	64.3272	71.8482	72.963	16.2049
0.2222	4.24	1200	0.2437	73.3821	64.9656	72.1995	73.2511	16.0795
0.2077	4.95	1400	0.2450	73.1663	64.7168	72.023	73.0977	16.0936
0.1976	5.65	1600	0.2296	73.2977	64.8011	72.2179	73.3089	16.1661
0.1804	6.36	1800	0.2268	73.1599	64.852	72.0518	73.1532	16.1802
0.1842	7.07	2000	0.2284	73.2343	64.944	72.046	73.1038	16.159
0.1776	7.77	2200	0.2255	73.3332	65.119	72.1684	73.2489	16.1449
0.1621	8.48	2400	0.2231	73.2057	64.9477	72.1727	73.1358	16.1219
0.1657	9.19	2600	0.2234	73.2285	65.0575	72.0227	73.2392	16.1608
0.1653	9.89	2800	0.2236	73.2746	65.1173	72.149	73.1838	16.1625

Framework versions

• Transformers 4.32.1
• Pytorch 2.0.1+cu117
• Datasets 2.14.4
• Tokenizers 0.13.3