Model Card for DistilBERT NER Model

Model Details

Model Description

This model is a fine-tuned version of distilbert-base-uncased for Named Entity Recognition (NER). It was fine-tuned on a domain-specific dataset to classify tokens into entities related to finance and compensation, as well as general non-entity tokens.

Developed by: Burak Kilic
Model type: Token Classification (NER)
Language(s): English
Finetuned from model: DistilBERT (uncased)

Model Sources

Repository: distilbert-ner-finetuned
Base Model: distilbert-base-uncased

Uses

Direct Use

This model is intended for Named Entity Recognition tasks and can be directly used to identify entities in financial texts, such as: B-DebtInstrumentInterestRateStatedPercentage B-LineOfCreditFacilityMaximumBorrowingCapacity B-DebtInstrumentBasisSpreadOnVariableRate1 B-AllocatedShareBasedCompensationExpense

Out-of-Scope Use

This model is not suitable for tasks outside Named Entity Recognition or for domains unrelated to finance.

Training Details

Training Procedure

Dataset: Custom annotated dataset with ~20,000 training examples.
Layers Fine-Tuned: Fully connected classification layer added for NER.
Training Regime: Mixed precision (fp16) with AdamW optimizer.

Training Hyperparameters

Parameter	Value
Learning Rate	2e-5
Warmup Ratio	0.1
Batch Size	128
Epochs	8
Weight Decay	0.01
Mixed Precision Training	True
Evaluation Metric	F1

Evaluation

Testing Data

The model was evaluated on a test set of ~1,600 examples, balanced across multiple entity types.

Results

Precision: 96%
Recall: 98%
F1 Score: 97%
Accuracy: 99.77%

How to Get Started with the Model

from transformers import pipeline

# Load the fine-tuned model
ner_pipeline = pipeline("ner", model="sojimanatsu/sojimanatsu/finer-selected-4-labels")

# Example text
text = "The bond yields 4.5% annually."
entities = ner_pipeline(text)
print(entities)

Limitations

Performance may degrade for texts outside the finance domain.
Rare entities may have lower recognition rates.

sojimanatsu
/

finer-selected-4-labels