Create README.md

README.md

@@ -0,0 +1,53 @@
+# Circassian (Kabardian) ASR Model
+
+This is a fine-tuned model for Automatic Speech Recognition (ASR) in `kbd`, based on the `facebook/w2v-bert-2.0` model. The model was trained on a combination of the `anzorq/kbd_speech` (filtered on `country=russia`) and `anzorq/sixuxar_yijiri_mak7` datasets.
+
+## Model Details
+
+- **Base Model**: facebook/w2v-bert-2.0
+- **Language**: Kabardian
+- **Task**: Automatic Speech Recognition (ASR)
+- **Datasets**: anzorq/kbd_speech, anzorq/sixuxar_yijiri_mak7
+- **Training Steps**: 5000
+
+## Training
+
+The model was fine-tuned using the following training arguments:
+
+```python
+TrainingArguments(
+   output_dir='output',
+   group_by_length=True,
+   per_device_train_batch_size=8,
+   gradient_accumulation_steps=2,
+   evaluation_strategy="steps",
+   num_train_epochs=10,
+   gradient_checkpointing=True,
+   fp16=True,
+   save_steps=1000,
+   eval_steps=500,
+   logging_steps=300,
+   learning_rate=5e-5,
+   warmup_steps=500,
+   save_total_limit=2,
+   push_to_hub=True,
+   report_to="wandb"
+)
+```
+
+## Performance
+
+The model's performance during training:
+
+| Step | Training Loss | Validation Loss | WER     |
+|------|---------------|-----------------|---------|
+| 500  | 2.859600      | inf             | 0.870362|
+| 1000 | 0.355500      | inf             | 0.703617|
+| 1500 | 0.247100      | inf             | 0.549942|
+| 2000 | 0.196700      | inf             | 0.471762|
+| 2500 | 0.181500      | inf             | 0.361494|
+| 3000 | 0.152200      | inf             | 0.314119|
+| 3500 | 0.135700      | inf             | 0.275146|
+| 4000 | 0.113400      | inf             | 0.252625|
+| 4500 | 0.102900      | inf             | 0.277013|
+| 5000 | 0.078500      | inf             | 0.250175|