---
base_model: google/pegasus-x-base
tags:
- generated_from_trainer
datasets:
- eur-lex-sum
model-index:
- name: PegasusX_no_extraction_V1
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# PegasusX_no_extraction_V1

This model is a fine-tuned version of [google/pegasus-x-base](https://huggingface.co/google/pegasus-x-base) on the eur-lex-sum dataset.
It achieves the following results on the evaluation set:
- Loss: 1.6795

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 2
- eval_batch_size: 1
- seed: 42
- distributed_type: multi-GPU
- num_devices: 4
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- total_eval_batch_size: 4
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 40

### Training results

| Training Loss | Epoch   | Step | Validation Loss |
|:-------------:|:-------:|:----:|:---------------:|
| 4.9176        | 0.9927  | 68   | 3.9239          |
| 3.8267        | 2.0     | 137  | 3.2236          |
| 3.2452        | 2.9927  | 205  | 2.6649          |
| 2.7272        | 4.0     | 274  | 2.2625          |
| 2.4546        | 4.9927  | 342  | 2.0656          |
| 2.2504        | 6.0     | 411  | 1.9579          |
| 2.1713        | 6.9927  | 479  | 1.8934          |
| 2.0563        | 8.0     | 548  | 1.8536          |
| 2.023         | 8.9927  | 616  | 1.8237          |
| 1.9452        | 10.0    | 685  | 1.8021          |
| 1.9365        | 10.9927 | 753  | 1.7839          |
| 1.8701        | 12.0    | 822  | 1.7746          |
| 1.8756        | 12.9927 | 890  | 1.7641          |
| 1.8261        | 14.0    | 959  | 1.7505          |
| 1.827         | 14.9927 | 1027 | 1.7454          |
| 1.7861        | 16.0    | 1096 | 1.7353          |
| 1.7943        | 16.9927 | 1164 | 1.7280          |
| 1.7501        | 18.0    | 1233 | 1.7276          |
| 1.7606        | 18.9927 | 1301 | 1.7176          |
| 1.7264        | 20.0    | 1370 | 1.7119          |
| 1.7371        | 20.9927 | 1438 | 1.6997          |
| 1.7008        | 22.0    | 1507 | 1.7067          |
| 1.7101        | 22.9927 | 1575 | 1.7002          |
| 1.6865        | 24.0    | 1644 | 1.6997          |
| 1.6967        | 24.9927 | 1712 | 1.6914          |
| 1.6648        | 26.0    | 1781 | 1.6915          |
| 1.6761        | 26.9927 | 1849 | 1.6893          |
| 1.6432        | 28.0    | 1918 | 1.6918          |
| 1.6688        | 28.9927 | 1986 | 1.6863          |
| 1.6289        | 30.0    | 2055 | 1.6858          |
| 1.6475        | 30.9927 | 2123 | 1.6878          |
| 1.6176        | 32.0    | 2192 | 1.6838          |
| 1.6435        | 32.9927 | 2260 | 1.6835          |
| 1.6139        | 34.0    | 2329 | 1.6802          |
| 1.638         | 34.9927 | 2397 | 1.6806          |
| 1.6099        | 36.0    | 2466 | 1.6830          |
| 1.6359        | 36.9927 | 2534 | 1.6778          |
| 1.6056        | 38.0    | 2603 | 1.6813          |
| 1.6281        | 38.9927 | 2671 | 1.6789          |
| 1.6132        | 39.7080 | 2720 | 1.6795          |


### Framework versions

- Transformers 4.40.1
- Pytorch 2.2.1+cu121
- Datasets 2.17.1
- Tokenizers 0.19.1