File size: 3,200 Bytes
37f05c6 acc1550 85cde26 37f05c6 4bd5455 37f05c6 4bd5455 37f05c6 4bd5455 37f05c6 4bd5455 37f05c6 4bd5455 acc1550 37f05c6 4bd5455 37f05c6 acc1550 37f05c6 85cde26 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 |
---
tags:
- generated_from_trainer
model-index:
- name: sentiment-polish-gpt2-large
results:
- task:
type: text-classification
dataset:
type: allegro/klej-polemo2-out
name: klej-polemo2-out
metrics:
- type: accuracy
value: 98.58%
license: mit
datasets:
- clarin-pl/polemo2-official
language:
- pl
metrics:
- accuracy
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# sentiment-polish-gpt2-large
This model is a fine-tuned version of [sdadas/polish-gpt2-large](https://huggingface.co/sdadas/polish-gpt2-large) on the [polemo2-official](https://huggingface.co/datasets/clarin-pl/polemo2-official) dataset.
It achieves the following results on the evaluation set:
- epoch: 10.0
- eval_accuracy: 0.9634
- eval_loss: 0.3139
- eval_runtime: 132.9089
- eval_samples_per_second: 197.428
- eval_steps_per_second: 98.714
- step: 65610
## Model description
Trained from [polish-gpt2-large](https://huggingface.co/sdadas/polish-gpt2-large)
## Intended uses & limitations
Sentiment analysis - neutral/negative/positive/ambiguous
## Training and evaluation data
Merged all rows from [polemo2-official](https://huggingface.co/datasets/clarin-pl/polemo2-official) dataset.
Discarded rows with length > 512.
Train/test split: 80%/20%
Datacollator:
```py
data_collator = DataCollatorWithPadding(
tokenizer=tokenizer,
padding="longest",
max_length=MAX_INPUT_LENGTH,
pad_to_multiple_of=8
)
```
## Training procedure
GPU: 2x RTX 4060Ti 16GB
Training time: 29:16:50
Using accelerate + DeepSpeed
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10
### Evaluation
Evaluated on [allegro/klej-polemo2-out](https://huggingface.co/datasets/allegro/klej-polemo2-out) test dataset.
```py
from datasets import load_dataset
from evaluate import evaluator
data = load_dataset("allegro/klej-polemo2-out", split="test").shuffle(seed=42)
task_evaluator = evaluator("text-classification")
# fix labels
l = {
"__label__meta_zero": 0,
"__label__meta_minus_m": 1,
"__label__meta_plus_m": 2,
"__label__meta_amb": 3
}
def fix_labels(examples):
examples["target"] = l[examples["target"]]
return examples
data = data.map(fix_labels)
eval_resutls = task_evaluator.compute(
model_or_pipeline="nie3e/sentiment-polish-gpt2-large",
data=data,
label_mapping={"NEUTRAL": 0, "NEGATIVE": 1, "POSITIVE": 2, "AMBIGUOUS": 3},
input_column="sentence",
label_column="target"
)
print(eval_resutls)
```
```json
{
"accuracy": 0.9858299595141701,
"total_time_in_seconds": 12.71777104900002,
"samples_per_second": 38.8432845737416,
"latency_in_seconds": 0.02574447580769235
}
```
### Framework versions
- Transformers 4.37.2
- Pytorch 2.2.0+cu121
- Datasets 2.17.0
- Tokenizers 0.15.1 |