---
language:
- en
license: apache-2.0
base_model: google/flan-t5-small
tags:
- generated_from_trainer
datasets:
- sentence-paraphrases
model-index:
- name: flan-t5-small-simplifier
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# flan-t5-small-simplifier

For paraphrasing and simplifying English text.

Fine-tuned version of [google/flan-t5-small](https://huggingface.co/google/flan-t5-small) on the agentlans/sentence-paraphrases dataset.
It achieves the following results on the evaluation set:
- Loss: 1.1518
- Num Input Tokens Seen: 32939232

## Intended uses & limitations

Works best on sentence length texts.

```
import torch
from transformers import pipeline

# Check if GPU is available
device = 0 if torch.cuda.is_available() else -1

# Initialize the pipeline
model_name = "agentlans/flan-t5-small-simplifier"
flan_t5_pipeline = pipeline("text2text-generation", model=model_name, device=device)

# Example input
input_text = "While navigating the labyrinthine corridors of epistemological uncertainty, the precocious philosopher—whose seminal work on phenomenological interpretation had already garnered significant academic acclaim—paused momentarily to contemplate the intricate interplay between subjective perception and objective reality, ultimately recognizing that the boundaries of human understanding are perpetually fluid and dynamically reconstructed through continuous intellectual discourse and empirical investigation."

# Generate output
output = flan_t5_pipeline(input_text, max_length=1024)

# Print the result
print(output[0]["generated_text"])
# The precocious philosopher, who had already been a major academic acclaim for his seminal work on phenomenological interpretation, paused momentarily to contemplate the intricate interplay between subjective perception and objective reality, recognizing that the boundaries of human understanding are perpetually fluid and dynamically reconstructed through continuous intellectual discourse and empirical investigation.
```

Limitations:
- English only
- Doesn't handle mixed language texts well (for example, English with Greek letter words)
- Might not be able to simplify some texts

## Training and evaluation data

agentlans/sentence-paraphrases

This dataset is a curated collection of sentence-length paraphrases derived from two primary sources:

    humarin/chatgpt-paraphrases
    xwjzds/paraphrase_collections.

Dataset Details
Dataset Description

The dataset is structured to provide pairs of sentences from an original text and its paraphrase(s). For each entry:

    The "text" field contains the least readable paraphrase.
    The "paraphrase" field contains the most readable paraphrase.

Readability was assessed using the agentlans/deberta-v3-xsmall-zyda-2-readability model.

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 2.0

### Training results

| Training Loss | Epoch  | Step  | Validation Loss | Input Tokens Seen |
|:-------------:|:------:|:-----:|:---------------:|:-----------------:|
| 1.4423        | 0.2224 | 10000 | 1.2431          | 3655312           |
| 1.3884        | 0.4448 | 20000 | 1.2093          | 7331520           |
| 1.3782        | 0.6673 | 30000 | 1.1859          | 10990432          |
| 1.3595        | 0.8897 | 40000 | 1.1787          | 14653328          |
| 1.3059        | 1.1121 | 50000 | 1.1665          | 18326104          |
| 1.3298        | 1.3345 | 60000 | 1.1589          | 21991016          |
| 1.2994        | 1.5569 | 70000 | 1.1562          | 25656600          |
| 1.2952        | 1.7794 | 80000 | 1.1518          | 29314808          |


### Framework versions

- Transformers 4.43.3
- Pytorch 2.3.0+cu121
- Datasets 3.2.0
- Tokenizers 0.19.1