
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# dit-base-finetuned-rvlcdip-small_rvl_cdip-NK1000_kd_NKD_t1.0_g1.5

This model is a fine-tuned version of [WinKawaks/vit-small-patch16-224](https://huggingface.co/WinKawaks/vit-small-patch16-224) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 5.3085
- Accuracy: 0.828
- Brier Loss: 0.3005
- Nll: 1.3339
- F1 Micro: 0.828
- F1 Macro: 0.8285
- Ece: 0.1391
- Aurc: 0.0474

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 96
- eval_batch_size: 96
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| No log        | 1.0   | 167  | 5.3750          | 0.61     | 0.5591     | 2.2520 | 0.61     | 0.5922   | 0.1827 | 0.1664 |
| No log        | 2.0   | 334  | 5.0343          | 0.7117   | 0.4389     | 1.9483 | 0.7117   | 0.7096   | 0.1691 | 0.0962 |
| 5.4927        | 3.0   | 501  | 4.8554          | 0.7472   | 0.3777     | 1.6689 | 0.7472   | 0.7474   | 0.1221 | 0.0780 |
| 5.4927        | 4.0   | 668  | 4.7917          | 0.76     | 0.3524     | 1.5715 | 0.76     | 0.7644   | 0.0915 | 0.0699 |
| 5.4927        | 5.0   | 835  | 4.7792          | 0.765    | 0.3461     | 1.5348 | 0.765    | 0.7590   | 0.0737 | 0.0704 |
| 4.6216        | 6.0   | 1002 | 4.6378          | 0.7993   | 0.2954     | 1.3769 | 0.7993   | 0.7995   | 0.0546 | 0.0559 |
| 4.6216        | 7.0   | 1169 | 4.8666          | 0.771    | 0.3359     | 1.5727 | 0.771    | 0.7728   | 0.0670 | 0.0666 |
| 4.6216        | 8.0   | 1336 | 4.6834          | 0.7897   | 0.3047     | 1.3537 | 0.7897   | 0.7914   | 0.0531 | 0.0564 |
| 4.2978        | 9.0   | 1503 | 4.6558          | 0.7997   | 0.2912     | 1.3758 | 0.7997   | 0.7988   | 0.0521 | 0.0508 |
| 4.2978        | 10.0  | 1670 | 4.8214          | 0.7923   | 0.3144     | 1.5316 | 0.7923   | 0.7928   | 0.0815 | 0.0561 |
| 4.2978        | 11.0  | 1837 | 4.8908          | 0.7923   | 0.3201     | 1.4158 | 0.7923   | 0.7931   | 0.0988 | 0.0573 |
| 4.1375        | 12.0  | 2004 | 4.7703          | 0.8093   | 0.2971     | 1.3642 | 0.8093   | 0.8097   | 0.0812 | 0.0514 |
| 4.1375        | 13.0  | 2171 | 4.8126          | 0.806    | 0.3039     | 1.3759 | 0.806    | 0.8053   | 0.0916 | 0.0491 |
| 4.1375        | 14.0  | 2338 | 4.7875          | 0.8063   | 0.2990     | 1.3712 | 0.8062   | 0.8065   | 0.0904 | 0.0481 |
| 4.0665        | 15.0  | 2505 | 4.7995          | 0.805    | 0.3016     | 1.4133 | 0.805    | 0.8049   | 0.0909 | 0.0503 |
| 4.0665        | 16.0  | 2672 | 4.7712          | 0.8075   | 0.2957     | 1.4018 | 0.8075   | 0.8082   | 0.0880 | 0.0484 |
| 4.0665        | 17.0  | 2839 | 4.7245          | 0.812    | 0.2886     | 1.2816 | 0.8120   | 0.8139   | 0.0831 | 0.0464 |
| 4.0204        | 18.0  | 3006 | 4.8990          | 0.8055   | 0.3079     | 1.3884 | 0.8055   | 0.8046   | 0.1117 | 0.0504 |
| 4.0204        | 19.0  | 3173 | 4.9286          | 0.802    | 0.3147     | 1.3977 | 0.802    | 0.7995   | 0.1078 | 0.0522 |
| 4.0204        | 20.0  | 3340 | 4.9510          | 0.8055   | 0.3121     | 1.4482 | 0.8055   | 0.8062   | 0.1125 | 0.0521 |
| 3.9854        | 21.0  | 3507 | 4.8837          | 0.8033   | 0.3082     | 1.4528 | 0.8033   | 0.8022   | 0.1052 | 0.0502 |
| 3.9854        | 22.0  | 3674 | 5.0103          | 0.813    | 0.3069     | 1.4217 | 0.813    | 0.8169   | 0.1207 | 0.0500 |
| 3.9854        | 23.0  | 3841 | 4.9602          | 0.8093   | 0.3091     | 1.4672 | 0.8093   | 0.8103   | 0.1187 | 0.0494 |
| 3.9599        | 24.0  | 4008 | 4.8980          | 0.8177   | 0.2953     | 1.3589 | 0.8178   | 0.8203   | 0.1083 | 0.0451 |
| 3.9599        | 25.0  | 4175 | 4.8753          | 0.8145   | 0.2932     | 1.3219 | 0.8145   | 0.8140   | 0.1054 | 0.0460 |
| 3.9599        | 26.0  | 4342 | 4.9644          | 0.8193   | 0.3000     | 1.4336 | 0.8193   | 0.8200   | 0.1173 | 0.0458 |
| 3.9358        | 27.0  | 4509 | 4.9648          | 0.8203   | 0.2985     | 1.4117 | 0.8203   | 0.8197   | 0.1132 | 0.0471 |
| 3.9358        | 28.0  | 4676 | 5.0130          | 0.8195   | 0.3014     | 1.4618 | 0.8195   | 0.8201   | 0.1205 | 0.0456 |
| 3.9358        | 29.0  | 4843 | 4.8974          | 0.8255   | 0.2874     | 1.3041 | 0.8255   | 0.8258   | 0.1097 | 0.0421 |
| 3.9151        | 30.0  | 5010 | 4.9045          | 0.8255   | 0.2878     | 1.2801 | 0.8255   | 0.8250   | 0.1119 | 0.0426 |
| 3.9151        | 31.0  | 5177 | 4.9720          | 0.823    | 0.2945     | 1.3551 | 0.823    | 0.8240   | 0.1212 | 0.0439 |
| 3.9151        | 32.0  | 5344 | 4.9508          | 0.826    | 0.2913     | 1.2669 | 0.826    | 0.8268   | 0.1201 | 0.0422 |
| 3.9003        | 33.0  | 5511 | 5.0336          | 0.8237   | 0.2991     | 1.3443 | 0.8237   | 0.8240   | 0.1243 | 0.0433 |
| 3.9003        | 34.0  | 5678 | 4.9828          | 0.8237   | 0.2901     | 1.2843 | 0.8237   | 0.8239   | 0.1214 | 0.0440 |
| 3.9003        | 35.0  | 5845 | 5.0256          | 0.8287   | 0.2920     | 1.2961 | 0.8287   | 0.8291   | 0.1232 | 0.0422 |
| 3.89          | 36.0  | 6012 | 5.0229          | 0.8283   | 0.2922     | 1.2471 | 0.8283   | 0.8283   | 0.1236 | 0.0432 |
| 3.89          | 37.0  | 6179 | 5.0835          | 0.8285   | 0.2936     | 1.2892 | 0.8285   | 0.8289   | 0.1254 | 0.0442 |
| 3.89          | 38.0  | 6346 | 5.1148          | 0.8287   | 0.2946     | 1.3106 | 0.8287   | 0.8282   | 0.1287 | 0.0427 |
| 3.8846        | 39.0  | 6513 | 5.1238          | 0.827    | 0.2954     | 1.2964 | 0.827    | 0.8275   | 0.1298 | 0.0441 |
| 3.8846        | 40.0  | 6680 | 5.1481          | 0.8307   | 0.2950     | 1.3136 | 0.8308   | 0.8311   | 0.1277 | 0.0453 |
| 3.8846        | 41.0  | 6847 | 5.1491          | 0.8293   | 0.2943     | 1.2841 | 0.8293   | 0.8294   | 0.1298 | 0.0451 |
| 3.881         | 42.0  | 7014 | 5.1982          | 0.829    | 0.2969     | 1.3111 | 0.8290   | 0.8292   | 0.1331 | 0.0459 |
| 3.881         | 43.0  | 7181 | 5.2041          | 0.8283   | 0.2970     | 1.3427 | 0.8283   | 0.8283   | 0.1327 | 0.0465 |
| 3.881         | 44.0  | 7348 | 5.2310          | 0.8297   | 0.2985     | 1.3351 | 0.8297   | 0.8303   | 0.1346 | 0.0471 |
| 3.8796        | 45.0  | 7515 | 5.2394          | 0.83     | 0.2999     | 1.3308 | 0.83     | 0.8305   | 0.1348 | 0.0467 |
| 3.8796        | 46.0  | 7682 | 5.2632          | 0.83     | 0.2990     | 1.3350 | 0.83     | 0.8304   | 0.1355 | 0.0471 |
| 3.8796        | 47.0  | 7849 | 5.2821          | 0.828    | 0.2998     | 1.3354 | 0.828    | 0.8286   | 0.1383 | 0.0470 |
| 3.8753        | 48.0  | 8016 | 5.2949          | 0.829    | 0.2998     | 1.3341 | 0.8290   | 0.8294   | 0.1374 | 0.0472 |
| 3.8753        | 49.0  | 8183 | 5.3026          | 0.8287   | 0.3004     | 1.3281 | 0.8287   | 0.8293   | 0.1382 | 0.0474 |
| 3.8753        | 50.0  | 8350 | 5.3085          | 0.828    | 0.3005     | 1.3339 | 0.828    | 0.8285   | 0.1391 | 0.0474 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo