
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# finetuned-FER2013

This model is a fine-tuned version of [microsoft/beit-base-patch16-224-pt22k-ft22k](https://huggingface.co/microsoft/beit-base-patch16-224-pt22k-ft22k) on the imagefolder dataset.
It achieves the following results on the evaluation set:
- Loss: 0.8366
- Accuracy: 0.7081

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-06
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 1.8119        | 1.0   | 202   | 1.7993          | 0.3079   |
| 1.6155        | 2.0   | 404   | 1.5446          | 0.4302   |
| 1.4279        | 3.0   | 606   | 1.3084          | 0.5301   |
| 1.3222        | 4.0   | 808   | 1.1817          | 0.5590   |
| 1.2532        | 5.0   | 1010  | 1.1026          | 0.5789   |
| 1.2019        | 6.0   | 1212  | 1.0432          | 0.5998   |
| 1.2037        | 7.0   | 1414  | 1.0030          | 0.6137   |
| 1.1757        | 8.0   | 1616  | 0.9873          | 0.6235   |
| 1.1359        | 9.0   | 1818  | 0.9377          | 0.6423   |
| 1.1282        | 10.0  | 2020  | 0.9231          | 0.6486   |
| 1.1019        | 11.0  | 2222  | 0.9011          | 0.6562   |
| 1.0494        | 12.0  | 2424  | 0.8968          | 0.6545   |
| 0.9951        | 13.0  | 2626  | 0.8876          | 0.6607   |
| 1.0121        | 14.0  | 2828  | 0.8720          | 0.6695   |
| 1.0571        | 15.0  | 3030  | 0.8776          | 0.6691   |
| 1.0049        | 16.0  | 3232  | 0.8627          | 0.6733   |
| 0.988         | 17.0  | 3434  | 0.8639          | 0.6719   |
| 0.9955        | 18.0  | 3636  | 0.8397          | 0.6806   |
| 0.9381        | 19.0  | 3838  | 0.8430          | 0.6820   |
| 0.9911        | 20.0  | 4040  | 0.8370          | 0.6837   |
| 0.9305        | 21.0  | 4242  | 0.8373          | 0.6837   |
| 0.9653        | 22.0  | 4444  | 0.8283          | 0.6883   |
| 0.9134        | 23.0  | 4646  | 0.8289          | 0.6879   |
| 0.9098        | 24.0  | 4848  | 0.8365          | 0.6837   |
| 0.8761        | 25.0  | 5050  | 0.8190          | 0.6869   |
| 0.9067        | 26.0  | 5252  | 0.8303          | 0.6876   |
| 0.8765        | 27.0  | 5454  | 0.8188          | 0.6942   |
| 0.8486        | 28.0  | 5656  | 0.8142          | 0.6959   |
| 0.9357        | 29.0  | 5858  | 0.8114          | 0.6984   |
| 0.9037        | 30.0  | 6060  | 0.8150          | 0.6917   |
| 0.8758        | 31.0  | 6262  | 0.8165          | 0.6931   |
| 0.8688        | 32.0  | 6464  | 0.8061          | 0.6994   |
| 0.8736        | 33.0  | 6666  | 0.8056          | 0.6994   |
| 0.8785        | 34.0  | 6868  | 0.8045          | 0.6991   |
| 0.8292        | 35.0  | 7070  | 0.8095          | 0.6987   |
| 0.8407        | 36.0  | 7272  | 0.8096          | 0.6956   |
| 0.8609        | 37.0  | 7474  | 0.8137          | 0.6984   |
| 0.9055        | 38.0  | 7676  | 0.8054          | 0.7018   |
| 0.8355        | 39.0  | 7878  | 0.8080          | 0.6980   |
| 0.8391        | 40.0  | 8080  | 0.8087          | 0.6966   |
| 0.7987        | 41.0  | 8282  | 0.8041          | 0.6998   |
| 0.818         | 42.0  | 8484  | 0.8070          | 0.7039   |
| 0.7836        | 43.0  | 8686  | 0.8091          | 0.7025   |
| 0.8348        | 44.0  | 8888  | 0.8047          | 0.7025   |
| 0.8205        | 45.0  | 9090  | 0.8076          | 0.7025   |
| 0.8023        | 46.0  | 9292  | 0.8056          | 0.7053   |
| 0.8241        | 47.0  | 9494  | 0.8022          | 0.7039   |
| 0.763         | 48.0  | 9696  | 0.8079          | 0.6994   |
| 0.7422        | 49.0  | 9898  | 0.8062          | 0.7039   |
| 0.7762        | 50.0  | 10100 | 0.8090          | 0.6998   |
| 0.7786        | 51.0  | 10302 | 0.8122          | 0.6994   |
| 0.8027        | 52.0  | 10504 | 0.8129          | 0.7043   |
| 0.7966        | 53.0  | 10706 | 0.8094          | 0.7039   |
| 0.8103        | 54.0  | 10908 | 0.8107          | 0.7039   |
| 0.7827        | 55.0  | 11110 | 0.8126          | 0.7057   |
| 0.7949        | 56.0  | 11312 | 0.8104          | 0.7119   |
| 0.7511        | 57.0  | 11514 | 0.8122          | 0.7050   |
| 0.7727        | 58.0  | 11716 | 0.8123          | 0.7078   |
| 0.7723        | 59.0  | 11918 | 0.8194          | 0.7015   |
| 0.7796        | 60.0  | 12120 | 0.8193          | 0.7053   |
| 0.7768        | 61.0  | 12322 | 0.8159          | 0.7029   |
| 0.7604        | 62.0  | 12524 | 0.8081          | 0.7085   |
| 0.7784        | 63.0  | 12726 | 0.8169          | 0.7106   |
| 0.7235        | 64.0  | 12928 | 0.8131          | 0.7015   |
| 0.7384        | 65.0  | 13130 | 0.8149          | 0.7085   |
| 0.6638        | 66.0  | 13332 | 0.8192          | 0.7078   |
| 0.6998        | 67.0  | 13534 | 0.8243          | 0.7113   |
| 0.7249        | 68.0  | 13736 | 0.8200          | 0.7015   |
| 0.6809        | 69.0  | 13938 | 0.8140          | 0.7081   |
| 0.701         | 70.0  | 14140 | 0.8177          | 0.7095   |
| 0.7122        | 71.0  | 14342 | 0.8245          | 0.7053   |
| 0.7269        | 72.0  | 14544 | 0.8245          | 0.7050   |
| 0.6973        | 73.0  | 14746 | 0.8207          | 0.7095   |
| 0.7241        | 74.0  | 14948 | 0.8210          | 0.7057   |
| 0.7397        | 75.0  | 15150 | 0.8230          | 0.7060   |
| 0.6832        | 76.0  | 15352 | 0.8308          | 0.7057   |
| 0.7213        | 77.0  | 15554 | 0.8256          | 0.7025   |
| 0.7115        | 78.0  | 15756 | 0.8291          | 0.7057   |
| 0.688         | 79.0  | 15958 | 0.8337          | 0.7088   |
| 0.6997        | 80.0  | 16160 | 0.8312          | 0.7060   |
| 0.6924        | 81.0  | 16362 | 0.8321          | 0.7053   |
| 0.7382        | 82.0  | 16564 | 0.8340          | 0.7050   |
| 0.7513        | 83.0  | 16766 | 0.8320          | 0.7015   |
| 0.656         | 84.0  | 16968 | 0.8389          | 0.7053   |
| 0.6503        | 85.0  | 17170 | 0.8321          | 0.7085   |
| 0.6661        | 86.0  | 17372 | 0.8355          | 0.7092   |
| 0.7026        | 87.0  | 17574 | 0.8339          | 0.7088   |
| 0.76          | 88.0  | 17776 | 0.8361          | 0.7092   |
| 0.696         | 89.0  | 17978 | 0.8343          | 0.7106   |
| 0.6713        | 90.0  | 18180 | 0.8337          | 0.7106   |
| 0.6621        | 91.0  | 18382 | 0.8349          | 0.7057   |
| 0.7042        | 92.0  | 18584 | 0.8360          | 0.7085   |
| 0.7087        | 93.0  | 18786 | 0.8353          | 0.7085   |
| 0.64          | 94.0  | 18988 | 0.8371          | 0.7088   |
| 0.659         | 95.0  | 19190 | 0.8376          | 0.7071   |
| 0.6246        | 96.0  | 19392 | 0.8376          | 0.7088   |
| 0.6797        | 97.0  | 19594 | 0.8368          | 0.7092   |
| 0.6652        | 98.0  | 19796 | 0.8376          | 0.7092   |
| 0.629         | 99.0  | 19998 | 0.8370          | 0.7088   |
| 0.6762        | 100.0 | 20200 | 0.8366          | 0.7081   |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu121
- Datasets 2.16.1
- Tokenizers 0.15.0

angry disgust fear happy neutral sad surprise