yeniguno/turkish-abstractive-summary-mt5

Model Card for Model ID

This model is a fine-tuned version of mukayese/mt5-base-turkish-summarization, specifically adapted for generating concise and coherent summaries of Turkish news articles. The fine-tuning was performed using the yeniguno/turkish-news-summary-onesentence dataset, which consists of approximately 60,000 Turkish news articles paired with one-sentence summaries. The objective is to enhance the model's ability to produce shorter, more concise, and compact news summaries.

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import pipeline

pipe = pipeline("summarization", model="yeniguno/turkish-abstractive-summary-mt5")

text = """Brezilya'nın kuzeydoğu kıyısındaki Recife kentinde yangın çıkan bir gökdelen alevlere teslim oldu. Paylaşılan video kaydında, binayı alt katlarından üst katlarına kadar alevlerin sardığı görüldü. İlk belirlemelere göre ölen ya da yaralanan olmadı. Timesnow'ın haberine göre, binadan molozlar düşmesi nedeniyle civardaki binaların elektriği kesildi ve binalar tahliye edildi."""

response = pipe(
            text, 
            max_length=150,  # Adjust as needed
            num_beams=4,     # Beam search for better quality
            length_penalty=3.0,  # Penalize longer sequences
            early_stopping=True  # Stop when at least num_beams sentences are finished)
)

print(response[0]["summary_text"])

# Brezilya'nın Recebi kentinde yangın çıkan gökdelen, alevlere teslim olurken, binadan molozlar düşmesi nedeniyle binaların elektriği kesildi ve tahliye edildi.

Uses

This model is intended for applications requiring the summarization of Turkish news content, such as news aggregation platforms, content curation services, and applications aiming to provide quick overviews of lengthy news articles.​

Bias, Risks, and Limitations

The model's performance is contingent upon the quality and diversity of the training data. It may not perform optimally on news topics or styles not represented in the training dataset. Users should exercise caution and consider the context when interpreting the generated summaries.

Training Details

Training Data

The model was fine-tuned on the yeniguno/turkish-news-summary-onesentence dataset, comprising approximately 60,000 Turkish news articles and their corresponding one-sentence summaries.

Training Procedure

The model was fine-tuned using mukayese/mt5-base-turkish-summarization on the yeniguno/turkish-news-summary-onesentence dataset. The training was conducted using Hugging Face's transformers library with the following hyperparameters:

• Learning Rate: 5e-6
• Batch Size: 8 per device for both training and evaluation
• Weight Decay: 0.01
• Epochs: 10
• Evaluation Strategy: epoch (evaluated at the end of each epoch)
• Loss Function: Cross-entropy loss
• Optimizer: AdamW
• Training Steps: 49,560
• Total FLOPs: 7.78e+17
• Predict with Generate: Enabled

The fine-tuning process was conducted on a single GPU, with dynamic padding applied using the DataCollatorForSeq2Seq.

Evaluation

To assess the model's performance, we used ROUGE scores, a widely used metric for text summarization tasks. The following metrics were calculated on the validation set at the end of each epoch:

Epoch	Training Loss	Validation Loss	ROUGE-1	ROUGE-2	ROUGE-L	Gen Len
1	1.3854	1.2058	35.10	22.95	31.92	8.86
2	1.2895	1.1541	36.27	24.05	33.05	8.87
3	1.2631	1.1258	36.58	24.55	33.41	8.85
4	1.2318	1.1072	36.98	24.95	33.80	8.84
5	1.2130	1.0946	37.17	25.18	34.01	8.83
6	1.1948	1.0861	37.38	25.41	34.22	8.83
7	1.1888	1.0803	37.56	25.60	34.39	8.83
8	1.1810	1.0764	37.58	25.63	34.41	8.84
9	1.1690	1.0738	37.68	25.74	34.52	8.83
10	1.1814	1.0732	37.68	25.73	34.52	8.84

• ROUGE-1: Measures the overlap of unigrams between generated summaries and reference summaries.
• ROUGE-2: Measures the overlap of bigrams.
• ROUGE-L: Measures the longest common subsequence between reference and generated summaries.
• Gen Len: Represents the average length of generated summaries.

After 10 epochs, the model achieved ROUGE-1: 37.68, ROUGE-2: 25.73, ROUGE-L: 34.52 on the validation dataset, indicating improved summarization capabilities. At the end of training, the final training loss was 1.2444, with the last recorded validation loss of 1.0732. The model was optimized to generate more concise and compact Turkish news summaries while maintaining high semantic accuracy and readability.