
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-base_rvl_cdip_crl_softmax_rank1_fixed

This model is a fine-tuned version of [jordyvl/vit-base_rvl-cdip](https://huggingface.co/jordyvl/vit-base_rvl-cdip) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7091
- Accuracy: 0.9032
- Brier Loss: 0.1756
- Nll: 1.0964
- F1 Micro: 0.9032
- F1 Macro: 0.9033
- Ece: 0.0854
- Aurc: 0.0188

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 10

### Training results

| Training Loss | Epoch | Step   | Validation Loss | Accuracy | Brier Loss | Nll    | F1 Micro | F1 Macro | Ece    | Aurc   |
|:-------------:|:-----:|:------:|:---------------:|:--------:|:----------:|:------:|:--------:|:--------:|:------:|:------:|
| 0.2289        | 1.0   | 10000  | 0.4298          | 0.8826   | 0.1794     | 1.2461 | 0.8826   | 0.8841   | 0.0553 | 0.0199 |
| 0.1972        | 2.0   | 20000  | 0.4350          | 0.8859   | 0.1769     | 1.3140 | 0.8859   | 0.8862   | 0.0558 | 0.0197 |
| 0.1414        | 3.0   | 30000  | 0.4423          | 0.8938   | 0.1702     | 1.2433 | 0.8938   | 0.8948   | 0.0639 | 0.0181 |
| 0.0903        | 4.0   | 40000  | 0.5076          | 0.8943   | 0.1753     | 1.2033 | 0.8943   | 0.8941   | 0.0766 | 0.0181 |
| 0.0684        | 5.0   | 50000  | 0.5592          | 0.8963   | 0.1783     | 1.2422 | 0.8963   | 0.8965   | 0.0811 | 0.0194 |
| 0.0313        | 6.0   | 60000  | 0.6384          | 0.8956   | 0.1836     | 1.2359 | 0.8957   | 0.8957   | 0.0861 | 0.0218 |
| 0.0163        | 7.0   | 70000  | 0.6673          | 0.9005   | 0.1788     | 1.1927 | 0.9005   | 0.9006   | 0.0855 | 0.0215 |
| 0.0104        | 8.0   | 80000  | 0.6929          | 0.9001   | 0.1791     | 1.1768 | 0.9001   | 0.9000   | 0.0860 | 0.0204 |
| 0.0036        | 9.0   | 90000  | 0.7131          | 0.9018   | 0.1780     | 1.1295 | 0.9018   | 0.9018   | 0.0866 | 0.0195 |
| 0.0023        | 10.0  | 100000 | 0.7091          | 0.9032   | 0.1756     | 1.0964 | 0.9032   | 0.9033   | 0.0854 | 0.0188 |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2

letter form email handwritten advertisement scientific_report scientific_publication specification file_folder news_article budget invoice presentation questionnaire resume memo