pretrained-hist-l2_tenKQ_finetune-itemseg_v12-tssp-m0

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.3248
• Accuracy: 0.9294
• Macro F1: 0.8204

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 3365
• training_steps: 67312

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Macro F1
18.3081	2.0010	201	92.0443	0.1413	0.0501
6.328	5.0009	402	133.2266	0.5761	0.1625
4.838	8.0009	603	145.9778	0.6359	0.1990
3.6763	11.0008	804	119.4216	0.6674	0.2212
2.7128	14.0008	1005	51.8430	0.7070	0.2573
2.3342	17.0007	1206	27.4778	0.7250	0.2894
1.8815	20.0007	1407	19.0125	0.7462	0.3373
1.65	23.0007	1608	13.0926	0.7608	0.3662
1.4902	26.0006	1809	8.4115	0.7567	0.4072
1.2953	29.0006	2010	7.8777	0.7823	0.4407
1.1867	32.0005	2211	6.7955	0.7812	0.4557
1.0704	35.0005	2412	5.3807	0.7841	0.4912
0.9792	38.0004	2613	4.6642	0.8160	0.5335
0.8649	41.0004	2814	4.1745	0.8178	0.5658
0.7768	44.0003	3015	3.8817	0.8240	0.5736
0.7169	47.0003	3216	4.2183	0.8330	0.5971
0.6459	50.0003	3417	3.4359	0.8295	0.6069
0.5664	53.0002	3618	3.3957	0.8538	0.6294
0.516	56.0002	3819	3.8816	0.8540	0.6424
0.4745	59.0001	4020	3.9219	0.8692	0.6628
0.4204	62.0001	4221	3.8606	0.8709	0.6657
0.3926	65.0000	4422	4.2133	0.8666	0.6703
0.3569	67.0010	4623	4.5565	0.8644	0.6744
0.3342	70.0010	4824	4.9640	0.8809	0.6919
0.303	73.0009	5025	5.1881	0.8813	0.7000
0.2858	76.0009	5226	5.0722	0.8878	0.7098
0.2741	79.0008	5427	6.1618	0.8920	0.7198
0.2592	82.0008	5628	6.5562	0.8897	0.7178
0.2432	85.0007	5829	7.2766	0.8939	0.7291
0.2265	88.0007	6030	8.0923	0.8949	0.7300
0.2201	91.0006	6231	7.3557	0.8984	0.7339
0.2093	94.0006	6432	7.3704	0.8954	0.7321
0.1983	97.0005	6633	8.7920	0.8962	0.7394
0.195	100.0005	6834	8.1665	0.8975	0.7421
0.1828	103.0005	7035	7.8452	0.9033	0.7492
0.18	106.0004	7236	7.6154	0.9012	0.7537
0.1723	109.0004	7437	6.4922	0.9060	0.7585
0.1694	112.0003	7638	7.5438	0.9040	0.7616
0.1672	115.0003	7839	6.9874	0.9053	0.7650
0.1552	118.0002	8040	7.1907	0.9063	0.7634
0.1588	121.0002	8241	7.0263	0.9086	0.7636
0.1509	124.0001	8442	6.8458	0.9070	0.7644
0.1514	127.0001	8643	5.4099	0.9089	0.7701
0.1457	130.0001	8844	5.8632	0.9107	0.7727
0.1425	133.0000	9045	5.6742	0.9111	0.7753
0.1415	135.0010	9246	5.7430	0.9117	0.7761
0.1436	138.0009	9447	5.0272	0.9103	0.7734
0.1434	141.0009	9648	4.4936	0.9096	0.7746
0.1377	144.0008	9849	5.2322	0.9078	0.7769
0.1383	147.0008	10050	4.7013	0.9149	0.7835
0.133	150.0008	10251	4.6257	0.9142	0.7836
0.1349	153.0007	10452	3.9570	0.9153	0.7830
0.1273	156.0007	10653	3.6838	0.9142	0.7806
0.1255	159.0006	10854	3.6121	0.9144	0.7836
0.1247	162.0006	11055	3.2919	0.9160	0.7879
0.129	165.0005	11256	3.1283	0.9132	0.7823
0.1215	168.0005	11457	3.2925	0.9152	0.7854
0.1216	171.0004	11658	3.1251	0.9168	0.7869
0.1205	174.0004	11859	3.1548	0.9182	0.7893
0.1231	177.0004	12060	2.7044	0.9122	0.7835
0.1308	180.0003	12261	2.7894	0.9158	0.7891
0.1254	183.0003	12462	2.6146	0.9158	0.7864
0.1194	186.0002	12663	2.9000	0.9179	0.7899
0.1171	189.0002	12864	2.7811	0.9157	0.7899
0.1159	192.0001	13065	2.6063	0.9180	0.7926
0.1164	195.0001	13266	2.7002	0.9180	0.7915
0.1148	198.0000	13467	2.5541	0.9192	0.7946
0.1172	200.0010	13668	2.3960	0.9173	0.7894
0.1252	203.0010	13869	2.2429	0.9164	0.7887
0.1122	206.0009	14070	2.1204	0.9187	0.7935
0.115	209.0009	14271	2.3396	0.9190	0.7961
0.108	212.0008	14472	2.0145	0.9198	0.7957
0.1081	215.0008	14673	2.1137	0.9202	0.7969
0.1093	218.0007	14874	2.0863	0.9205	0.7972
0.1073	221.0007	15075	1.8909	0.9196	0.7946
0.11	224.0007	15276	1.8784	0.9212	0.7977
0.1087	227.0006	15477	1.8960	0.9224	0.8016
0.1065	230.0006	15678	1.8995	0.9186	0.7946
0.1065	233.0005	15879	1.8899	0.9200	0.7968
0.1108	236.0005	16080	1.8815	0.9205	0.7982
0.1064	239.0004	16281	1.8124	0.9220	0.7989
0.1038	242.0004	16482	1.8478	0.9235	0.8016
0.1047	245.0003	16683	1.8597	0.9218	0.8015
0.1042	248.0003	16884	1.8254	0.9221	0.8011
0.1064	251.0003	17085	1.7683	0.9222	0.8005
0.1007	254.0002	17286	1.7810	0.9216	0.8044
0.1009	257.0002	17487	1.8142	0.9225	0.8025
0.1011	260.0001	17688	1.7981	0.9203	0.7992
0.0963	263.0001	17889	1.6290	0.9222	0.8056
0.1089	266.0000	18090	1.6993	0.9214	0.8007
0.1119	268.0010	18291	1.6256	0.9214	0.8022
0.1008	271.0010	18492	1.7093	0.9232	0.8026
0.0963	274.0009	18693	1.6050	0.9234	0.8048
0.0997	277.0009	18894	1.5403	0.9237	0.8080
0.0978	280.0008	19095	1.6604	0.9229	0.8034
0.0964	283.0008	19296	1.5679	0.9232	0.8071
0.0963	286.0007	19497	1.5459	0.9230	0.8048
0.0973	289.0007	19698	1.4886	0.9196	0.8025
0.0996	292.0006	19899	1.4917	0.9208	0.8024
0.0963	295.0006	20100	1.3997	0.9225	0.8051
0.095	298.0005	20301	1.4833	0.9228	0.8050
0.0955	301.0005	20502	1.5128	0.9217	0.8033
0.0969	304.0005	20703	1.4938	0.9208	0.8025
0.1025	307.0004	20904	1.5228	0.9236	0.8031
0.096	310.0004	21105	1.5700	0.9241	0.8084
0.0981	313.0003	21306	1.4501	0.9202	0.8003
0.0968	316.0003	21507	1.5587	0.9238	0.8046
0.0958	319.0002	21708	1.4369	0.9254	0.8081
0.091	322.0002	21909	1.4935	0.9251	0.8074
0.0945	325.0001	22110	1.4459	0.9269	0.8120
0.0923	328.0001	22311	1.3652	0.9259	0.8109
0.0916	331.0001	22512	1.5116	0.9260	0.8099
0.091	334.0000	22713	1.4167	0.9245	0.8084
0.0924	336.0010	22914	1.4038	0.9258	0.8095
0.0917	339.0009	23115	1.4282	0.9258	0.8103
0.0892	342.0009	23316	1.4295	0.9267	0.8108
0.0898	345.0008	23517	1.4366	0.9257	0.8137
0.0916	348.0008	23718	1.4560	0.9234	0.8075
0.088	351.0008	23919	1.3876	0.9258	0.8118
0.0896	354.0007	24120	1.5172	0.9227	0.8065
0.088	357.0007	24321	1.5016	0.9261	0.8097
0.0874	360.0006	24522	1.4182	0.9254	0.8089
0.0918	363.0006	24723	1.4840	0.9241	0.8045
0.09	366.0005	24924	1.5154	0.9233	0.8081
0.0888	369.0005	25125	1.3604	0.9277	0.8143
0.0866	372.0004	25326	1.4135	0.9240	0.8081
0.089	375.0004	25527	1.4785	0.9260	0.8117
0.0928	378.0004	25728	1.3908	0.9256	0.8094
0.0927	381.0003	25929	1.4106	0.9246	0.8058
0.0877	384.0003	26130	1.3791	0.9263	0.8111
0.0859	387.0002	26331	1.4158	0.9256	0.8112
0.0878	390.0002	26532	1.3359	0.9256	0.8111
0.085	393.0001	26733	1.3835	0.9249	0.8096
0.0863	396.0001	26934	1.3753	0.9255	0.8109
0.0854	399.0000	27135	1.3575	0.9266	0.8108
0.0856	401.0010	27336	1.3802	0.9270	0.8122
0.0856	404.0010	27537	1.3584	0.9278	0.8138
0.0868	407.0009	27738	1.4511	0.9240	0.8079
0.0866	410.0009	27939	1.3532	0.9270	0.8125
0.0843	413.0008	28140	1.3818	0.9264	0.8134
0.086	416.0008	28341	1.3797	0.9262	0.8131
0.0849	419.0007	28542	1.3713	0.9267	0.8149
0.0831	422.0007	28743	1.4248	0.9260	0.8123
0.0843	425.0007	28944	1.3671	0.9272	0.8133
0.0838	428.0006	29145	1.5038	0.9260	0.8124
0.0839	431.0006	29346	1.4157	0.9272	0.8136
0.0827	434.0005	29547	1.3950	0.9269	0.8137
0.0832	437.0005	29748	1.3595	0.9267	0.8128
0.0867	440.0004	29949	1.2980	0.9243	0.8134
0.0846	443.0004	30150	1.4668	0.9279	0.8149
0.0853	446.0003	30351	1.3490	0.9259	0.8133
0.0849	449.0003	30552	1.4092	0.9275	0.8148
0.0856	452.0003	30753	1.3654	0.9246	0.8077
0.0838	455.0002	30954	1.4238	0.9269	0.8115
0.0809	458.0002	31155	1.3319	0.9273	0.8117
0.0826	461.0001	31356	1.4123	0.9276	0.8129
0.0815	464.0001	31557	1.3352	0.9275	0.8136
0.0804	467.0000	31758	1.3803	0.9280	0.8159
0.0797	469.0010	31959	1.3948	0.9272	0.8173
0.0806	472.0010	32160	1.3945	0.9261	0.8104
0.0804	475.0009	32361	1.3707	0.9284	0.8156
0.0808	478.0009	32562	1.4406	0.9268	0.8152
0.0792	481.0008	32763	1.4056	0.9267	0.8144
0.0788	484.0008	32964	1.4086	0.9269	0.8145
0.0805	487.0007	33165	1.3563	0.9276	0.8168
0.0804	490.0007	33366	1.3429	0.9284	0.8149
0.0798	493.0006	33567	1.4408	0.9270	0.8143
0.0791	496.0006	33768	1.4209	0.9274	0.8152
0.0823	499.0005	33969	1.4354	0.9271	0.8144
0.0811	502.0005	34170	1.3956	0.9265	0.8146
0.0978	505.0005	34371	1.4652	0.9256	0.8102
0.0894	508.0004	34572	1.3084	0.9237	0.8135
0.0817	511.0004	34773	1.3392	0.9272	0.8169
0.0793	514.0003	34974	1.3396	0.9294	0.8198
0.0781	517.0003	35175	1.2850	0.9289	0.8204
0.0787	520.0002	35376	1.3228	0.9294	0.8204
0.0778	523.0002	35577	1.3750	0.9289	0.8172
0.0772	526.0001	35778	1.4147	0.9274	0.8143
0.0777	529.0001	35979	1.3052	0.9278	0.8170
0.0783	532.0001	36180	1.2464	0.9286	0.8198
0.077	535.0000	36381	1.3571	0.9283	0.8184
0.0778	537.0010	36582	1.3503	0.9290	0.8184
0.0787	540.0009	36783	1.3391	0.9286	0.8184
0.077	543.0009	36984	1.4027	0.9292	0.8170
0.0779	546.0008	37185	1.3167	0.9281	0.8165
0.0781	549.0008	37386	1.3828	0.9284	0.8147
0.0781	552.0008	37587	1.3284	0.9287	0.8196
0.0765	555.0007	37788	1.4137	0.9286	0.8186
0.0778	558.0007	37989	1.4190	0.9286	0.8174
0.0747	561.0006	38190	1.3886	0.9289	0.8159
0.0768	564.0006	38391	1.3433	0.9291	0.8178
0.0765	567.0005	38592	1.4049	0.9289	0.8169
0.0781	570.0005	38793	1.4670	0.9276	0.8141
0.0776	573.0004	38994	1.5075	0.9280	0.8166
0.0753	576.0004	39195	1.3683	0.9280	0.8164
0.0756	579.0004	39396	1.3317	0.9281	0.8186

Framework versions

• Transformers 4.46.0
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.20.1