README.md

---
widget:
  - text: Kuchli yomg‘irlar tufayli bir qator <mask> kuchli sel oqishi kuzatildi.
    example_title: Example 1
  - text: >-
      Shu munosabat bilan O‘zbekiston Prezidenti global inqiroz sharoitida savdo-iqtisodiy hamkorlikni <mask> va hududlararo aloqalarni rivojlantirishning muhim masalalariga to‘xtalib o‘tdi.
    example_title: Example 2
tags:
- generated_from_trainer
datasets:
- sinonimayzer/mixed-data
language:
- uz
library_name: transformers
pipeline_tag: fill-mask
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# UzRoBERTa-v2

This model achieves the following results on the evaluation set:
- Loss: 1.9097

## How to use

```python
>>> from transformers import pipeline
>>> unmasker = pipeline('fill-mask', model='sinonimayzer/UzRoBERTa-v2')
>>> unmasker("Kuchli yomg‘irlar tufayli bir qator <mask> kuchli sel oqishi kuzatildi.")

[{'score': 0.3318027853965759,
  'token': 4877,
  'token_str': ' hududlarda',
  'sequence': 'Kuchli yomg‘irlar tufayli bir qator hududlarda kuchli sel oqishi kuzatildi.'},
 {'score': 0.13175441324710846,
  'token': 14470,
  'token_str': ' viloyatlarda',
  'sequence': 'Kuchli yomg‘irlar tufayli bir qator viloyatlarda kuchli sel oqishi kuzatildi.'},
 {'score': 0.09735308587551117,
  'token': 13555,
  'token_str': ' tumanlarda',
  'sequence': 'Kuchli yomg‘irlar tufayli bir qator tumanlarda kuchli sel oqishi kuzatildi.'},
 {'score': 0.09112472087144852,
  'token': 12261,
  'token_str': ' shaharlarda',
  'sequence': 'Kuchli yomg‘irlar tufayli bir qator shaharlarda kuchli sel oqishi kuzatildi.'},
 {'score': 0.05940879508852959,
  'token': 2767,
  'token_str': ' joylarda',
  'sequence': 'Kuchli yomg‘irlar tufayli bir qator joylarda kuchli sel oqishi kuzatildi.'}]

```

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 92
- eval_batch_size: 92
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 500000

### Training results

| Training Loss | Epoch | Step   | Validation Loss |
|:-------------:|:-----:|:------:|:---------------:|
| 2.3673        | 0.25  | 100000 | 2.4588          |
| 2.0797        | 0.51  | 200000 | 2.1653          |
| 1.9369        | 0.76  | 300000 | 2.0265          |
| 1.8545        | 1.02  | 400000 | 1.9456          |
| 1.8133        | 1.27  | 500000 | 1.9101          |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.1+cu121
- Datasets 2.15.0
- Tokenizers 0.15.0