---
library_name: transformers
language:
- sq
license: apache-2.0
base_model: openai/whisper-base
tags:
- generated_from_trainer
datasets:
- Kushtrim/audioshqip
metrics:
- wer
model-index:
- name: Whisper Base Shqip
  results:
  - task:
      type: automatic-speech-recognition
      name: Automatic Speech Recognition
    dataset:
      name: Audio Shqip 97 orë
      type: Kushtrim/audioshqip
      args: 'config: sq, split: test'
    metrics:
    - type: wer
      value: 40.143396979133186
      name: Wer
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Whisper Base Shqip

This model is a fine-tuned version of [openai/whisper-base](https://huggingface.co/openai/whisper-base) on the Audio Shqip 97 orë dataset.
It achieves the following results on the evaluation set:
- Loss: 0.5274
- Wer: 40.1434

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- training_steps: 10000
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch  | Step  | Validation Loss | Wer     |
|:-------------:|:------:|:-----:|:---------------:|:-------:|
| 1.0357        | 0.3249 | 500   | 1.0437          | 70.9649 |
| 0.7862        | 0.6498 | 1000  | 0.7759          | 57.9971 |
| 0.6561        | 0.9747 | 1500  | 0.6805          | 51.6728 |
| 0.5704        | 1.2995 | 2000  | 0.6337          | 49.0896 |
| 0.5511        | 1.6244 | 2500  | 0.5968          | 47.4252 |
| 0.522         | 1.9493 | 3000  | 0.5740          | 47.2168 |
| 0.4252        | 2.2742 | 3500  | 0.5612          | 43.5865 |
| 0.4411        | 2.5991 | 4000  | 0.5487          | 43.2817 |
| 0.4434        | 2.9240 | 4500  | 0.5373          | 43.3737 |
| 0.3791        | 3.2489 | 5000  | 0.5353          | 42.3143 |
| 0.371         | 3.5737 | 5500  | 0.5297          | 41.3114 |
| 0.4173        | 3.8986 | 6000  | 0.5231          | 41.4012 |
| 0.3009        | 4.2235 | 6500  | 0.5276          | 40.9756 |
| 0.3337        | 4.5484 | 7000  | 0.5249          | 40.4393 |
| 0.3145        | 4.8733 | 7500  | 0.5222          | 40.2154 |
| 0.2897        | 5.1982 | 8000  | 0.5264          | 40.4925 |
| 0.2717        | 5.5231 | 8500  | 0.5256          | 40.6387 |
| 0.2947        | 5.8480 | 9000  | 0.5251          | 40.2753 |
| 0.2933        | 6.1728 | 9500  | 0.5268          | 40.5601 |
| 0.2644        | 6.4977 | 10000 | 0.5274          | 40.1434 |


### Framework versions

- Transformers 4.45.2
- Pytorch 2.5.1+cu121
- Datasets 3.2.0
- Tokenizers 0.20.3