
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# dit-base-small_rvl_cdip-NK1000_hint

This model is a fine-tuned version of [WinKawaks/vit-small-patch16-224](https://huggingface.co/WinKawaks/vit-small-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 2.2813
- Accuracy: 0.841
- Brier Loss: 0.2948
- Nll: 1.9789
- F1 Micro: 0.841
- F1 Macro: 0.8409
- Ece: 0.1435
- Aurc: 0.0496

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| 2.5147        | 1.0   | 1000  | 2.3332          | 0.659    | 0.4557     | 2.1792 | 0.659    | 0.6434   | 0.0525 | 0.1283 |
| 2.0194        | 2.0   | 2000  | 1.9888          | 0.7225   | 0.3793     | 2.1426 | 0.7225   | 0.7275   | 0.0522 | 0.0910 |
| 1.7429        | 3.0   | 3000  | 1.8097          | 0.759    | 0.3332     | 1.9995 | 0.7590   | 0.7599   | 0.0480 | 0.0713 |
| 1.5665        | 4.0   | 4000  | 1.8189          | 0.7652   | 0.3417     | 2.0345 | 0.7652   | 0.7676   | 0.0920 | 0.0706 |
| 1.4286        | 5.0   | 5000  | 1.7489          | 0.7588   | 0.3435     | 2.0233 | 0.7588   | 0.7639   | 0.0654 | 0.0745 |
| 1.3258        | 6.0   | 6000  | 1.7223          | 0.7837   | 0.3212     | 1.9911 | 0.7837   | 0.7818   | 0.0939 | 0.0595 |
| 1.1929        | 7.0   | 7000  | 1.6777          | 0.8015   | 0.3090     | 1.9764 | 0.8015   | 0.8045   | 0.0991 | 0.0554 |
| 1.0707        | 8.0   | 8000  | 1.8240          | 0.79     | 0.3365     | 1.9708 | 0.79     | 0.7907   | 0.1334 | 0.0588 |
| 1.0054        | 9.0   | 9000  | 1.9355          | 0.7853   | 0.3482     | 2.0445 | 0.7853   | 0.7861   | 0.1404 | 0.0637 |
| 0.924         | 10.0  | 10000 | 2.0125          | 0.799    | 0.3399     | 2.0334 | 0.799    | 0.7993   | 0.1471 | 0.0584 |
| 0.9043        | 11.0  | 11000 | 1.9868          | 0.8075   | 0.3252     | 2.0171 | 0.8075   | 0.8078   | 0.1429 | 0.0526 |
| 0.8442        | 12.0  | 12000 | 2.1626          | 0.8087   | 0.3339     | 2.0839 | 0.8087   | 0.8075   | 0.1523 | 0.0534 |
| 0.8158        | 13.0  | 13000 | 2.1123          | 0.7903   | 0.3567     | 2.0193 | 0.7903   | 0.7922   | 0.1592 | 0.0580 |
| 0.8087        | 14.0  | 14000 | 2.1296          | 0.814    | 0.3269     | 2.0417 | 0.8140   | 0.8152   | 0.1482 | 0.0533 |
| 0.7924        | 15.0  | 15000 | 2.0768          | 0.816    | 0.3181     | 2.1244 | 0.816    | 0.8152   | 0.1435 | 0.0500 |
| 0.7574        | 16.0  | 16000 | 2.3326          | 0.797    | 0.3596     | 2.1292 | 0.797    | 0.7960   | 0.1668 | 0.0590 |
| 0.7501        | 17.0  | 17000 | 2.3190          | 0.8067   | 0.3475     | 2.0635 | 0.8067   | 0.8106   | 0.1617 | 0.0588 |
| 0.7097        | 18.0  | 18000 | 2.3543          | 0.8067   | 0.3468     | 2.0987 | 0.8067   | 0.8063   | 0.1636 | 0.0561 |
| 0.6939        | 19.0  | 19000 | 2.3151          | 0.8067   | 0.3405     | 2.0927 | 0.8067   | 0.8047   | 0.1617 | 0.0546 |
| 0.6972        | 20.0  | 20000 | 2.3289          | 0.8087   | 0.3409     | 2.0291 | 0.8087   | 0.8113   | 0.1628 | 0.0550 |
| 0.6869        | 21.0  | 21000 | 2.3878          | 0.797    | 0.3567     | 2.0736 | 0.797    | 0.7995   | 0.1727 | 0.0599 |
| 0.6621        | 22.0  | 22000 | 2.3796          | 0.8137   | 0.3323     | 2.1080 | 0.8137   | 0.8138   | 0.1570 | 0.0553 |
| 0.6496        | 23.0  | 23000 | 2.4264          | 0.8143   | 0.3342     | 2.0650 | 0.8143   | 0.8182   | 0.1598 | 0.0562 |
| 0.6478        | 24.0  | 24000 | 2.3300          | 0.8167   | 0.3313     | 2.0509 | 0.8167   | 0.8170   | 0.1564 | 0.0552 |
| 0.6184        | 25.0  | 25000 | 2.3143          | 0.8283   | 0.3178     | 1.9842 | 0.8283   | 0.8284   | 0.1498 | 0.0509 |
| 0.6203        | 26.0  | 26000 | 2.3665          | 0.825    | 0.3181     | 2.0647 | 0.825    | 0.8246   | 0.1501 | 0.0530 |
| 0.6151        | 27.0  | 27000 | 2.2673          | 0.8335   | 0.3042     | 1.9961 | 0.8335   | 0.8351   | 0.1431 | 0.0501 |
| 0.6054        | 28.0  | 28000 | 2.2927          | 0.8283   | 0.3100     | 2.0229 | 0.8283   | 0.8281   | 0.1490 | 0.0508 |
| 0.5965        | 29.0  | 29000 | 2.4001          | 0.8223   | 0.3235     | 2.0393 | 0.8223   | 0.8266   | 0.1553 | 0.0536 |
| 0.5869        | 30.0  | 30000 | 2.4167          | 0.821    | 0.3269     | 1.9890 | 0.821    | 0.8213   | 0.1576 | 0.0553 |
| 0.5852        | 31.0  | 31000 | 2.4030          | 0.83     | 0.3120     | 2.0384 | 0.83     | 0.8304   | 0.1500 | 0.0539 |
| 0.5741        | 32.0  | 32000 | 2.4800          | 0.8233   | 0.3259     | 2.1183 | 0.8233   | 0.8212   | 0.1561 | 0.0550 |
| 0.5743        | 33.0  | 33000 | 2.4718          | 0.821    | 0.3269     | 2.0593 | 0.821    | 0.8191   | 0.1582 | 0.0578 |
| 0.5694        | 34.0  | 34000 | 2.3638          | 0.8297   | 0.3135     | 2.0746 | 0.8297   | 0.8293   | 0.1484 | 0.0534 |
| 0.5567        | 35.0  | 35000 | 2.3320          | 0.8313   | 0.3054     | 2.0498 | 0.8313   | 0.8311   | 0.1470 | 0.0524 |
| 0.5547        | 36.0  | 36000 | 2.3902          | 0.8293   | 0.3138     | 2.0001 | 0.8293   | 0.8294   | 0.1506 | 0.0513 |
| 0.55          | 37.0  | 37000 | 2.3812          | 0.822    | 0.3239     | 2.0567 | 0.822    | 0.8226   | 0.1560 | 0.0541 |
| 0.5482        | 38.0  | 38000 | 2.3680          | 0.8333   | 0.3063     | 2.0447 | 0.8333   | 0.8332   | 0.1460 | 0.0496 |
| 0.546         | 39.0  | 39000 | 2.3250          | 0.8323   | 0.3051     | 1.9994 | 0.8323   | 0.8315   | 0.1478 | 0.0504 |
| 0.541         | 40.0  | 40000 | 2.3498          | 0.837    | 0.3003     | 2.0228 | 0.8370   | 0.8367   | 0.1456 | 0.0493 |
| 0.5448        | 41.0  | 41000 | 2.3504          | 0.833    | 0.3053     | 1.9875 | 0.833    | 0.8323   | 0.1492 | 0.0532 |
| 0.5365        | 42.0  | 42000 | 2.3421          | 0.8323   | 0.3077     | 2.0985 | 0.8323   | 0.8314   | 0.1477 | 0.0501 |
| 0.5324        | 43.0  | 43000 | 2.2976          | 0.84     | 0.2929     | 1.9862 | 0.8400   | 0.8403   | 0.1418 | 0.0507 |
| 0.5326        | 44.0  | 44000 | 2.3270          | 0.838    | 0.2993     | 2.0043 | 0.838    | 0.8384   | 0.1438 | 0.0521 |
| 0.5303        | 45.0  | 45000 | 2.2919          | 0.839    | 0.2957     | 1.9625 | 0.839    | 0.8395   | 0.1430 | 0.0494 |
| 0.5276        | 46.0  | 46000 | 2.2861          | 0.8397   | 0.2973     | 1.9382 | 0.8397   | 0.8402   | 0.1442 | 0.0512 |
| 0.5279        | 47.0  | 47000 | 2.2930          | 0.8387   | 0.2962     | 1.9738 | 0.8387   | 0.8384   | 0.1456 | 0.0499 |
| 0.5251        | 48.0  | 48000 | 2.2841          | 0.84     | 0.2950     | 1.9888 | 0.8400   | 0.8399   | 0.1441 | 0.0490 |
| 0.5257        | 49.0  | 49000 | 2.2802          | 0.84     | 0.2950     | 1.9827 | 0.8400   | 0.8400   | 0.1443 | 0.0491 |
| 0.5232        | 50.0  | 50000 | 2.2813          | 0.841    | 0.2948     | 1.9789 | 0.841    | 0.8409   | 0.1435 | 0.0496 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo