Llama-3.2-400M-Amharic-Poems-Stories-V6

This model is a fine-tuned version of rasyosef/Llama-3.2-400M-Amharic on the generator dataset. It achieves the following results on the evaluation set:

• Loss: 1.0045
• Model Preparation Time: 0.0031

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 8e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• lr_scheduler_warmup_steps: 250
• num_epochs: 4
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time
2.6077	0.3300	500	1.4733	0.0031
1.2277	0.6601	1000	1.0349	0.0031
0.99	0.9901	1500	0.8903	0.0031
0.4502	1.3201	2000	0.9601	0.0031
0.4186	1.6502	2500	0.9117	0.0031
0.424	1.9802	3000	0.8837	0.0031
0.1552	2.3102	3500	0.9935	0.0031
0.1404	2.6403	4000	0.9865	0.0031
0.1351	2.9703	4500	0.9807	0.0031
0.0842	3.3003	5000	1.0042	0.0031
0.0773	3.6304	5500	1.0049	0.0031
0.0795	3.9604	6000	1.0045	0.0031

Framework versions

• Transformers 4.45.0
• Pytorch 2.4.1+cu121
• Datasets 3.3.2
• Tokenizers 0.20.3