lemonhat/Qwen2.5-7B-Instruct-agenttuning_v2_15k_tag5

agenttuning_v2_15k_tag5

This model is a fine-tuned version of Qwen/Qwen2.5-7B-Instruct on the agenttuning_v2_15k_tag5 dataset. It achieves the following results on the evaluation set:

• Loss: 0.4994

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-06
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 4
• total_eval_batch_size: 4
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
0.7186	0.0281	100	0.6700
0.7683	0.0562	200	0.6378
0.4844	0.0842	300	0.6146
0.3803	0.1123	400	0.6067
0.4946	0.1404	500	0.6136
0.3734	0.1685	600	0.6001
0.5855	0.1966	700	0.5941
0.4913	0.2247	800	0.5866
0.5574	0.2527	900	0.5815
0.4587	0.2808	1000	0.5788
0.5672	0.3089	1100	0.5623
0.5146	0.3370	1200	0.5607
0.5222	0.3651	1300	0.5461
0.4087	0.3931	1400	0.5524
0.4473	0.4212	1500	0.5461
0.4301	0.4493	1600	0.5413
0.4627	0.4774	1700	0.5397
0.4617	0.5055	1800	0.5291
0.5551	0.5336	1900	0.5290
0.4792	0.5616	2000	0.5192
0.5295	0.5897	2100	0.5211
0.4153	0.6178	2200	0.5207
0.493	0.6459	2300	0.5155
0.7251	0.6740	2400	0.5141
0.3545	0.7020	2500	0.5153
0.4691	0.7301	2600	0.5110
0.3731	0.7582	2700	0.5091
0.4002	0.7863	2800	0.5063
0.4789	0.8144	2900	0.5041
0.435	0.8425	3000	0.5022
0.4143	0.8705	3100	0.5005
0.3868	0.8986	3200	0.5001
0.4968	0.9267	3300	0.4996
0.4871	0.9548	3400	0.4994
0.3787	0.9829	3500	0.4995

Framework versions

• Transformers 4.46.1
• Pytorch 2.6.0+cu124
• Datasets 3.1.0
• Tokenizers 0.20.3