bert-suicide-detection-hk-large-new

This model is a fine-tuned version of hon9kon9ize/bert-large-cantonese on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4945
• Accuracy: 0.9085

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.5426	0.0604	20	0.7869	0.7451
0.5984	0.1208	40	0.3943	0.7908
0.4864	0.1813	60	0.9365	0.7843
0.6039	0.2417	80	0.6580	0.7712
0.5741	0.3021	100	0.3454	0.8235
0.4276	0.3625	120	0.5421	0.8170
0.4342	0.4230	140	0.4258	0.8562
0.4915	0.4834	160	0.5961	0.8301
0.4127	0.5438	180	0.2987	0.8693
0.3166	0.6042	200	0.3308	0.8693
0.4018	0.6647	220	0.5286	0.8039
0.3007	0.7251	240	0.5845	0.8627
0.4893	0.7855	260	0.3662	0.8627
0.274	0.8459	280	0.3483	0.8693
0.5741	0.9063	300	0.3280	0.8824
0.3752	0.9668	320	0.5251	0.8889
0.2711	1.0272	340	0.6097	0.8562
0.2369	1.0876	360	0.5457	0.8693
0.3756	1.1480	380	0.6890	0.8758
0.6575	1.2085	400	0.4709	0.8693
0.3268	1.2689	420	0.5219	0.8497
0.3994	1.3293	440	0.4282	0.8693
0.0879	1.3897	460	0.6294	0.8758
0.2566	1.4502	480	0.7143	0.8627
0.2897	1.5106	500	0.6120	0.8693
0.321	1.5710	520	0.4749	0.8758
0.1871	1.6314	540	0.4392	0.9085
0.1654	1.6918	560	0.4663	0.9085
0.3166	1.7523	580	0.5048	0.8889
0.222	1.8127	600	0.4550	0.9085
0.4299	1.8731	620	0.3445	0.9085
0.0942	1.9335	640	0.3735	0.9281
0.3991	1.9940	660	0.3646	0.9085
0.0581	2.0544	680	0.3527	0.9085
0.2712	2.1148	700	0.4270	0.9020
0.0443	2.1752	720	0.5462	0.8954
0.3831	2.2356	740	0.3419	0.9216
0.2267	2.2961	760	0.4925	0.8889
0.1821	2.3565	780	0.3625	0.9216
0.2926	2.4169	800	0.3671	0.9020
0.2507	2.4773	820	0.3853	0.9020
0.2446	2.5378	840	0.4571	0.8954
0.1926	2.5982	860	0.5436	0.8497
0.1725	2.6586	880	0.6576	0.8497
0.2033	2.7190	900	0.4772	0.9020
0.0095	2.7795	920	0.4103	0.9150
0.2896	2.8399	940	0.4333	0.9085
0.2661	2.9003	960	0.5793	0.8889
0.1338	2.9607	980	0.4543	0.8954
0.0751	3.0211	1000	0.5029	0.8954
0.2093	3.0816	1020	0.4631	0.9020
0.2436	3.1420	1040	0.5888	0.8693
0.1375	3.2024	1060	0.6457	0.8889
0.0049	3.2628	1080	0.6601	0.8889
0.0089	3.3233	1100	0.6462	0.8824
0.0616	3.3837	1120	0.6607	0.8889
0.006	3.4441	1140	0.6243	0.9020
0.1769	3.5045	1160	0.5257	0.9020
0.0044	3.5650	1180	0.5508	0.9085
0.2295	3.6254	1200	0.4846	0.9150
0.1175	3.6858	1220	0.4764	0.9020
0.0746	3.7462	1240	0.4761	0.9020
0.0222	3.8066	1260	0.4836	0.9020
0.0012	3.8671	1280	0.4775	0.9216
0.2131	3.9275	1300	0.4607	0.9020
0.0006	3.9879	1320	0.4935	0.9085
0.0758	4.0483	1340	0.4592	0.9020
0.1466	4.1088	1360	0.4464	0.9085
0.0488	4.1692	1380	0.4816	0.9085
0.0014	4.2296	1400	0.4570	0.9150
0.082	4.2900	1420	0.4545	0.9216
0.0009	4.3505	1440	0.4721	0.9150
0.0008	4.4109	1460	0.4874	0.9216
0.0014	4.4713	1480	0.5003	0.9150
0.1612	4.5317	1500	0.5064	0.9150
0.2079	4.5921	1520	0.4994	0.9150
0.1423	4.6526	1540	0.4835	0.9150
0.0009	4.7130	1560	0.4825	0.9085
0.0017	4.7734	1580	0.4918	0.9085
0.0648	4.8338	1600	0.4917	0.9150
0.0531	4.8943	1620	0.4919	0.9085
0.0008	4.9547	1640	0.4945	0.9085

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0