bert_tiny_trained

This model is a fine-tuned version of nreimers/BERT-Tiny_L-2_H-128_A-2 on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.5820

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • num_epochs: 50

Training results

Training Loss Epoch Step Validation Loss
No log 1.0 11 1.3285
No log 2.0 22 1.3144
No log 3.0 33 1.2977
No log 4.0 44 1.2683
No log 5.0 55 1.2476
No log 6.0 66 1.2256
No log 7.0 77 1.1961
No log 8.0 88 1.1853
No log 9.0 99 1.1625
No log 10.0 110 1.1284
No log 11.0 121 1.1036
No log 12.0 132 1.0812
No log 13.0 143 1.0573
No log 14.0 154 1.0323
No log 15.0 165 1.0092
No log 16.0 176 0.9913
No log 17.0 187 0.9725
No log 18.0 198 0.9492
No log 19.0 209 0.9269
No log 20.0 220 0.9061
No log 21.0 231 0.8869
No log 22.0 242 0.8719
No log 23.0 253 0.8521
No log 24.0 264 0.8357
No log 25.0 275 0.8169
No log 26.0 286 0.8026
No log 27.0 297 0.7936
No log 28.0 308 0.7783
No log 29.0 319 0.7677
No log 30.0 330 0.7577
No log 31.0 341 0.7516
No log 32.0 352 0.7431
No log 33.0 363 0.7355
No log 34.0 374 0.7287
No log 35.0 385 0.7220
No log 36.0 396 0.7154
No log 37.0 407 0.7119
No log 38.0 418 0.7073
No log 39.0 429 0.7025
No log 40.0 440 0.6976
No log 41.0 451 0.6931
No log 42.0 462 0.6890
No log 43.0 473 0.6859
No log 44.0 484 0.6830
No log 45.0 495 0.6807
0.7544 46.0 506 0.6785
0.7544 47.0 517 0.6774
0.7544 48.0 528 0.6769
0.7544 49.0 539 0.6768
0.7544 50.0 550 0.6767

Framework versions

  • Transformers 4.48.3
  • Pytorch 2.5.1+cu124
  • Datasets 3.3.2
  • Tokenizers 0.21.0
Downloads last month
16
Safetensors
Model size
4.39M params
Tensor type
F32
·
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.

Model tree for geobase/geo-task-classifier-transformer

Finetuned
(1)
this model