lemonhat/Qwen2.5-7B-Instruct-agenttuning_v4_10k_tag5

agenttuning_v4_10k_tag5

This model is a fine-tuned version of Qwen/Qwen2.5-7B-Instruct on the agenttuning_v4_10k_tag5 dataset. It achieves the following results on the evaluation set:

• Loss: 0.3634

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-06
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 4
• total_eval_batch_size: 4
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
0.5887	0.0386	100	0.5013
0.5464	0.0772	200	0.4804
0.5244	0.1158	300	0.4643
0.454	0.1544	400	0.4629
0.455	0.1930	500	0.4487
0.5026	0.2316	600	0.4363
0.48	0.2702	700	0.4406
0.4557	0.3088	800	0.4192
0.5715	0.3474	900	0.4098
0.3408	0.3860	1000	0.4053
0.3671	0.4245	1100	0.3955
0.5876	0.4631	1200	0.4024
0.45	0.5017	1300	0.4049
0.336	0.5403	1400	0.3939
0.5008	0.5789	1500	0.3893
0.3772	0.6175	1600	0.3889
0.2965	0.6561	1700	0.3778
0.4337	0.6947	1800	0.3701
0.3552	0.7333	1900	0.3686
0.3369	0.7719	2000	0.3660
0.2917	0.8105	2100	0.3655
0.3829	0.8491	2200	0.3661
0.4447	0.8877	2300	0.3646
0.4003	0.9263	2400	0.3638
0.3373	0.9649	2500	0.3639

Framework versions

• Transformers 4.46.1
• Pytorch 2.6.0+cu124
• Datasets 3.1.0
• Tokenizers 0.20.3