yemen2016's picture
End of training
c60d6bf verified
metadata
base_model: MiMe-MeMo/MeMo-BERT-03
tags:
  - generated_from_trainer
model-index:
  - name: Noise_MeMo_BERT-3_02
    results: []

Noise_MeMo_BERT-3_02

This model is a fine-tuned version of MiMe-MeMo/MeMo-BERT-03 on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0729
  • F1-score: 0.6452

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 20

Training results

Training Loss Epoch Step Validation Loss F1-score
0.095 1.0 915 0.0707 0.5373
0.0532 2.0 1830 0.0877 0.4286
0.0317 3.0 2745 0.0951 0.4889
0.0196 4.0 3660 0.1114 0.3590
0.0265 5.0 4575 0.0810 0.6333
0.0118 6.0 5490 0.1097 0.4783
0.0247 7.0 6405 0.1153 0.4583
0.0255 8.0 7320 0.0781 0.5634
0.0242 9.0 8235 0.1156 0.5455
0.0423 10.0 9150 0.1186 0.4
0.0246 11.0 10065 0.1057 0.5000
0.0224 12.0 10980 0.0998 0.56
0.0168 13.0 11895 0.0729 0.6452
0.0106 14.0 12810 0.1171 0.4444
0.0097 15.0 13725 0.0735 0.5818
0.0187 16.0 14640 0.0943 0.5417
0.0128 17.0 15555 0.1011 0.5417
0.0098 18.0 16470 0.1029 0.5714
0.0116 19.0 17385 0.0949 0.6182
0.0084 20.0 18300 0.0956 0.6154

Framework versions

  • Transformers 4.38.2
  • Pytorch 2.2.1+cu121
  • Datasets 2.18.0
  • Tokenizers 0.15.2