htlou/backup_0202_llamafactory_Compcap_l0_0_50-llava-mistral

Compcap_l0_0_50

This model is a fine-tuned version of llava-hf/llava-v1.6-mistral-7b-hf on the Compcap_l0_0_50 dataset. It achieves the following results on the evaluation set:

• Loss: 0.8117

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 8
• gradient_accumulation_steps: 4
• total_train_batch_size: 256
• total_eval_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 10
• num_epochs: 3.0

Training results

Training Loss	Epoch	Step	Validation Loss
0.9428	0.2706	50	0.9270
0.8747	0.5413	100	0.8709
0.8698	0.8119	150	0.8460
0.7652	1.0825	200	0.8337
0.7822	1.3532	250	0.8234
0.7628	1.6238	300	0.8164
0.768	1.8945	350	0.8106
0.7075	2.1651	400	0.8137
0.71	2.4357	450	0.8127
0.7069	2.7064	500	0.8119
0.7088	2.9770	550	0.8117

Framework versions

• Transformers 4.45.2
• Pytorch 2.4.0+cu121
• Datasets 2.21.0
• Tokenizers 0.20.3