---
library_name: transformers
license: mit
base_model: FacebookAI/xlm-roberta-large
tags:
- generated_from_trainer
model-index:
- name: multiclass-classifier-patents
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# multiclass-classifier-patents

This model is a fine-tuned version of [FacebookAI/xlm-roberta-large](https://huggingface.co/FacebookAI/xlm-roberta-large) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0067
- F1 Micro: 0.7001
- Precision Micro: 0.8337
- Recall Micro: 0.6034
- Exact Match F1: 0.5296
- Exact Match Precision: 0.5296
- Exact Match Recall: 0.5296
- Any Match F1: 0.9079
- Any Match Precision: 0.9079
- Any Match Recall: 0.9079

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | F1 Micro | Precision Micro | Recall Micro | Exact Match F1 | Exact Match Precision | Exact Match Recall | Any Match F1 | Any Match Precision | Any Match Recall |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:---------------:|:------------:|:--------------:|:---------------------:|:------------------:|:------------:|:-------------------:|:----------------:|
| 0.01          | 1.0   | 1292  | 0.0083          | 0.5977   | 0.8265          | 0.4681       | 0.4300         | 0.4300                | 0.4300             | 0.7675       | 0.7675              | 0.7675           |
| 0.0077        | 2.0   | 2584  | 0.0074          | 0.6595   | 0.8326          | 0.5460       | 0.4879         | 0.4879                | 0.4879             | 0.8636       | 0.8636              | 0.8636           |
| 0.007         | 3.0   | 3876  | 0.0071          | 0.6829   | 0.8173          | 0.5864       | 0.5035         | 0.5035                | 0.5035             | 0.8958       | 0.8958              | 0.8958           |
| 0.0063        | 4.0   | 5168  | 0.0069          | 0.6883   | 0.8317          | 0.5871       | 0.5140         | 0.5140                | 0.5140             | 0.8956       | 0.8956              | 0.8956           |
| 0.0058        | 5.0   | 6460  | 0.0068          | 0.6957   | 0.8337          | 0.5969       | 0.5182         | 0.5182                | 0.5182             | 0.9058       | 0.9058              | 0.9058           |
| 0.0053        | 6.0   | 7752  | 0.0069          | 0.6999   | 0.8366          | 0.6017       | 0.5271         | 0.5271                | 0.5271             | 0.9082       | 0.9082              | 0.9082           |
| 0.0048        | 7.0   | 9044  | 0.0069          | 0.7046   | 0.8159          | 0.6201       | 0.5225         | 0.5225                | 0.5225             | 0.9185       | 0.9185              | 0.9185           |
| 0.0046        | 8.0   | 10336 | 0.0069          | 0.7069   | 0.8100          | 0.6271       | 0.5241         | 0.5241                | 0.5241             | 0.9196       | 0.9196              | 0.9196           |
| 0.0042        | 9.0   | 11628 | 0.0070          | 0.7064   | 0.8208          | 0.6200       | 0.5282         | 0.5282                | 0.5282             | 0.9174       | 0.9174              | 0.9174           |
| 0.004         | 10.0  | 12920 | 0.0070          | 0.7064   | 0.8184          | 0.6214       | 0.5276         | 0.5276                | 0.5276             | 0.9177       | 0.9177              | 0.9177           |


### Framework versions

- Transformers 4.45.2
- Pytorch 2.0.1+cu117
- Datasets 3.0.1
- Tokenizers 0.20.3