File size: 5,167 Bytes
6555f3d |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 |
---
library_name: transformers
base_model: dccuchile/bert-base-spanish-wwm-cased
tags:
- generated_from_trainer
model-index:
- name: Neuria_BERT_Contexto_0108
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# Neuria_BERT_Contexto_0108
This model is a fine-tuned version of [dccuchile/bert-base-spanish-wwm-cased](https://huggingface.co/dccuchile/bert-base-spanish-wwm-cased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0851
- F1 Micro: 0.8428
- F1 Macro: 0.5221
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 50
### Training results
| Training Loss | Epoch | Step | Validation Loss | F1 Micro | F1 Macro |
|:-------------:|:-------:|:----:|:---------------:|:--------:|:--------:|
| 0.4708 | 0.96 | 18 | 0.3215 | 0.0 | 0.0 |
| 0.3044 | 1.9733 | 37 | 0.2748 | 0.0230 | 0.0096 |
| 0.2664 | 2.9867 | 56 | 0.2490 | 0.3301 | 0.0883 |
| 0.2376 | 4.0 | 75 | 0.2225 | 0.3619 | 0.1010 |
| 0.2196 | 4.96 | 93 | 0.1997 | 0.5254 | 0.1891 |
| 0.1815 | 5.9733 | 112 | 0.1802 | 0.6190 | 0.2344 |
| 0.1592 | 6.9867 | 131 | 0.1655 | 0.6032 | 0.2441 |
| 0.1362 | 8.0 | 150 | 0.1492 | 0.7059 | 0.3614 |
| 0.126 | 8.96 | 168 | 0.1383 | 0.7234 | 0.4036 |
| 0.1054 | 9.9733 | 187 | 0.1311 | 0.7639 | 0.4380 |
| 0.095 | 10.9867 | 206 | 0.1291 | 0.7639 | 0.4369 |
| 0.0858 | 12.0 | 225 | 0.1195 | 0.7891 | 0.4683 |
| 0.0816 | 12.96 | 243 | 0.1179 | 0.7974 | 0.4815 |
| 0.0707 | 13.9733 | 262 | 0.1080 | 0.8105 | 0.4927 |
| 0.0655 | 14.9867 | 281 | 0.1074 | 0.8129 | 0.4962 |
| 0.0609 | 16.0 | 300 | 0.1041 | 0.8333 | 0.5166 |
| 0.0599 | 16.96 | 318 | 0.1011 | 0.8258 | 0.5037 |
| 0.0537 | 17.9733 | 337 | 0.0988 | 0.8235 | 0.4994 |
| 0.0512 | 18.9867 | 356 | 0.0976 | 0.8258 | 0.5115 |
| 0.0485 | 20.0 | 375 | 0.0965 | 0.8153 | 0.5075 |
| 0.0491 | 20.96 | 393 | 0.0945 | 0.8333 | 0.5181 |
| 0.0447 | 21.9733 | 412 | 0.0939 | 0.8375 | 0.5102 |
| 0.0426 | 22.9867 | 431 | 0.0949 | 0.8258 | 0.5010 |
| 0.0418 | 24.0 | 450 | 0.0926 | 0.8447 | 0.5247 |
| 0.0423 | 24.96 | 468 | 0.0929 | 0.8375 | 0.5102 |
| 0.0389 | 25.9733 | 487 | 0.0920 | 0.85 | 0.5331 |
| 0.0375 | 26.9867 | 506 | 0.0921 | 0.8462 | 0.5246 |
| 0.0368 | 28.0 | 525 | 0.0900 | 0.8101 | 0.4962 |
| 0.0375 | 28.96 | 543 | 0.0914 | 0.8408 | 0.5125 |
| 0.0349 | 29.9733 | 562 | 0.0894 | 0.8481 | 0.5243 |
| 0.034 | 30.9867 | 581 | 0.0887 | 0.8447 | 0.5235 |
| 0.0334 | 32.0 | 600 | 0.0871 | 0.8428 | 0.5221 |
| 0.0342 | 32.96 | 618 | 0.0863 | 0.8354 | 0.5184 |
| 0.0317 | 33.9733 | 637 | 0.0875 | 0.8280 | 0.5052 |
| 0.0311 | 34.9867 | 656 | 0.0877 | 0.8354 | 0.5089 |
| 0.0307 | 36.0 | 675 | 0.0874 | 0.8354 | 0.5184 |
| 0.0318 | 36.96 | 693 | 0.0863 | 0.8428 | 0.5221 |
| 0.0297 | 37.9733 | 712 | 0.0854 | 0.8280 | 0.5145 |
| 0.0294 | 38.9867 | 731 | 0.0867 | 0.8375 | 0.5200 |
| 0.0292 | 40.0 | 750 | 0.0856 | 0.8428 | 0.5221 |
| 0.0306 | 40.96 | 768 | 0.0857 | 0.8354 | 0.5184 |
| 0.0287 | 41.9733 | 787 | 0.0856 | 0.8428 | 0.5221 |
| 0.0284 | 42.9867 | 806 | 0.0847 | 0.8354 | 0.5184 |
| 0.0284 | 44.0 | 825 | 0.0849 | 0.8428 | 0.5221 |
| 0.0296 | 44.96 | 843 | 0.0854 | 0.8428 | 0.5221 |
| 0.028 | 45.9733 | 862 | 0.0852 | 0.8428 | 0.5221 |
| 0.0278 | 46.9867 | 881 | 0.0850 | 0.8428 | 0.5221 |
| 0.0279 | 48.0 | 900 | 0.0851 | 0.8428 | 0.5221 |
### Framework versions
- Transformers 4.44.1
- Pytorch 2.4.1
- Datasets 2.19.1
- Tokenizers 0.19.1
|