# Multilingual GPT Model (Byte-Level)

This model is a multilingual GPT model trained on byte-level encodings of Wikipedia articles in Arabic (ar) and Egyptian Arabic (ary).

**Model Details:**
- Trained using a byte-level vocabulary (size: 32000).
- Architecture: Transformer-based GPT model.
- Languages: Arabic (ar), Egyptian Arabic (ary).
- Training Data: Streamed Wikipedia dataset (limited to 10000 articles per language).
- Training Code: [Link to your training script/GitHub repo if available]

**Usage:**

[Provide instructions on how to load and use the model. E.g., using `torch.load` and the provided `GPTLanguageModel` class.]

**Example (Conceptual - Adapt to your actual loading process):**

```python
import torch
from your_model_definition_script import GPTLanguageModel # Assuming you save model definition

# Initialize model architecture (must be defined in a separate script)
model = GPTLanguageModel()
model.load_state_dict(torch.load('model_weights.pth')) # Load from local if downloaded from HF
model.eval()

# ... (rest of your inference code) ...
```

**Training Hyperparameters:**
- Batch Size: 32
- Block Size: 256
- Embedding Dimension: 384
- Number of Heads: 6
- Number of Layers: 6
- Dropout: 0.2
- Optimizer: AdamW
- Learning Rate: 0.0006
- Max Iterations: 5000

**Loss Curve:**
[You can optionally add a link or embed the training plot image here]

**License:**
[Specify your license, e.g., MIT License]

**Contact:**
[Your name/contact information]