--- library_name: transformers license: cc-by-4.0 base_model: hon9kon9ize/bert-large-cantonese tags: - generated_from_trainer metrics: - accuracy model-index: - name: bert-suicide-detection-hk-large-new results: [] --- # bert-suicide-detection-hk-large-new This model is a fine-tuned version of [hon9kon9ize/bert-large-cantonese](https://huggingface.co/hon9kon9ize/bert-large-cantonese) on the None dataset. It achieves the following results on the evaluation set: - Loss: 0.2903 - Accuracy: 0.9467 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 4 - eval_batch_size: 4 - seed: 42 - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: linear - num_epochs: 5 ### Training results | Training Loss | Epoch | Step | Validation Loss | Accuracy | |:-------------:|:------:|:----:|:---------------:|:--------:| | 0.6203 | 0.0613 | 20 | 0.4148 | 0.8267 | | 0.3246 | 0.1227 | 40 | 0.8805 | 0.8 | | 0.5453 | 0.1840 | 60 | 0.3735 | 0.8667 | | 0.4513 | 0.2454 | 80 | 0.4391 | 0.8867 | | 0.7729 | 0.3067 | 100 | 0.4407 | 0.82 | | 0.5867 | 0.3681 | 120 | 0.4013 | 0.8467 | | 0.4073 | 0.4294 | 140 | 0.5397 | 0.86 | | 0.1883 | 0.4908 | 160 | 0.7620 | 0.8667 | | 0.4166 | 0.5521 | 180 | 0.6517 | 0.8933 | | 0.4672 | 0.6135 | 200 | 0.6163 | 0.88 | | 0.6858 | 0.6748 | 220 | 0.3484 | 0.8667 | | 0.335 | 0.7362 | 240 | 0.6031 | 0.8533 | | 0.4525 | 0.7975 | 260 | 0.6941 | 0.82 | | 0.2385 | 0.8589 | 280 | 0.5618 | 0.88 | | 0.4256 | 0.9202 | 300 | 0.5899 | 0.88 | | 0.4934 | 0.9816 | 320 | 0.3289 | 0.9 | | 0.277 | 1.0429 | 340 | 0.5671 | 0.88 | | 0.5097 | 1.1043 | 360 | 0.5247 | 0.88 | | 0.105 | 1.1656 | 380 | 0.4810 | 0.9 | | 0.3976 | 1.2270 | 400 | 0.4562 | 0.8933 | | 0.3506 | 1.2883 | 420 | 0.3943 | 0.8867 | | 0.2057 | 1.3497 | 440 | 0.4944 | 0.8933 | | 0.2788 | 1.4110 | 460 | 0.4718 | 0.9 | | 0.4049 | 1.4724 | 480 | 0.5067 | 0.88 | | 0.415 | 1.5337 | 500 | 0.4395 | 0.9 | | 0.3565 | 1.5951 | 520 | 0.3682 | 0.9 | | 0.3111 | 1.6564 | 540 | 0.3298 | 0.9 | | 0.4191 | 1.7178 | 560 | 0.4493 | 0.8733 | | 0.2731 | 1.7791 | 580 | 0.3832 | 0.9067 | | 0.1803 | 1.8405 | 600 | 0.4403 | 0.8933 | | 0.4462 | 1.9018 | 620 | 0.3844 | 0.9067 | | 0.0025 | 1.9632 | 640 | 0.4563 | 0.9067 | | 0.1574 | 2.0245 | 660 | 0.5508 | 0.8933 | | 0.0927 | 2.0859 | 680 | 0.5529 | 0.9067 | | 0.184 | 2.1472 | 700 | 0.5161 | 0.9 | | 0.2446 | 2.2086 | 720 | 0.5064 | 0.8933 | | 0.2498 | 2.2699 | 740 | 0.4034 | 0.92 | | 0.2217 | 2.3313 | 760 | 0.5095 | 0.8733 | | 0.2938 | 2.3926 | 780 | 0.3754 | 0.9067 | | 0.109 | 2.4540 | 800 | 0.4771 | 0.8933 | | 0.0282 | 2.5153 | 820 | 0.5535 | 0.8933 | | 0.2455 | 2.5767 | 840 | 0.4206 | 0.9067 | | 0.4728 | 2.6380 | 860 | 0.3018 | 0.9067 | | 0.1145 | 2.6994 | 880 | 0.3053 | 0.9067 | | 0.1045 | 2.7607 | 900 | 0.3431 | 0.9067 | | 0.2207 | 2.8221 | 920 | 0.6482 | 0.86 | | 0.427 | 2.8834 | 940 | 0.4396 | 0.9133 | | 0.1898 | 2.9448 | 960 | 0.3327 | 0.92 | | 0.0019 | 3.0061 | 980 | 0.3993 | 0.92 | | 0.0842 | 3.0675 | 1000 | 0.4166 | 0.9267 | | 0.1619 | 3.1288 | 1020 | 0.4181 | 0.9133 | | 0.1849 | 3.1902 | 1040 | 0.4727 | 0.92 | | 0.1949 | 3.2515 | 1060 | 0.3346 | 0.8933 | | 0.1796 | 3.3129 | 1080 | 0.3471 | 0.9267 | | 0.086 | 3.3742 | 1100 | 0.4089 | 0.8867 | | 0.0187 | 3.4356 | 1120 | 0.3868 | 0.92 | | 0.0768 | 3.4969 | 1140 | 0.4095 | 0.9267 | | 0.0008 | 3.5583 | 1160 | 0.3780 | 0.9067 | | 0.183 | 3.6196 | 1180 | 0.3827 | 0.9 | | 0.204 | 3.6810 | 1200 | 0.5133 | 0.9 | | 0.0758 | 3.7423 | 1220 | 0.4280 | 0.9133 | | 0.0237 | 3.8037 | 1240 | 0.3942 | 0.92 | | 0.2143 | 3.8650 | 1260 | 0.3680 | 0.9067 | | 0.0106 | 3.9264 | 1280 | 0.5633 | 0.8867 | | 0.2221 | 3.9877 | 1300 | 0.3815 | 0.92 | | 0.0212 | 4.0491 | 1320 | 0.4599 | 0.9267 | | 0.1678 | 4.1104 | 1340 | 0.3458 | 0.92 | | 0.1153 | 4.1718 | 1360 | 0.3261 | 0.92 | | 0.0006 | 4.2331 | 1380 | 0.3404 | 0.9133 | | 0.0193 | 4.2945 | 1400 | 0.3602 | 0.92 | | 0.0994 | 4.3558 | 1420 | 0.3303 | 0.94 | | 0.0032 | 4.4172 | 1440 | 0.2885 | 0.94 | | 0.0008 | 4.4785 | 1460 | 0.3112 | 0.92 | | 0.0823 | 4.5399 | 1480 | 0.3145 | 0.9267 | | 0.0086 | 4.6012 | 1500 | 0.2954 | 0.94 | | 0.0009 | 4.6626 | 1520 | 0.3082 | 0.94 | | 0.1619 | 4.7239 | 1540 | 0.2928 | 0.94 | | 0.0004 | 4.7853 | 1560 | 0.2909 | 0.9333 | | 0.0006 | 4.8466 | 1580 | 0.2879 | 0.9467 | | 0.0005 | 4.9080 | 1600 | 0.2894 | 0.9467 | | 0.0559 | 4.9693 | 1620 | 0.2903 | 0.9467 | ### Framework versions - Transformers 4.47.1 - Pytorch 2.5.1+cu121 - Datasets 3.2.0 - Tokenizers 0.21.0