xlm-roberta-base-finetuned-panx-all-langs

This model is a fine-tuned version of xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3089
• F1 Score: 0.8140

Model description

"트랜스 포머를 활용한 자연어 처리" O'Reilly 책을 활용하였습니다.

xlm-roberta-base 를 이용하여 파인 튜닝을 한 객체명 인식 입니다.

각 언어별 f1_score
ko : f1_score = 0.8611821192789028
en : f1_score = 0.7868391074180795
ja : f1_score = 0.6440401846320934
es : f1_score = 0.8533862565120316

Intended uses & limitations

코드는 책을 활용 및 참고 하였습니다.

Training and evaluation data

XTREME 데이터 셋을 이용하였습니다. 특히 PAN-X 데이터 셋을 이용했습니다.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 24
• eval_batch_size: 24
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score
0.5093	1.0	917	0.3530	0.7679
0.309	2.0	1834	0.3101	0.8029
0.2176	3.0	2751	0.3089	0.8140

Framework versions

• Transformers 4.46.2
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.20.3