---
library_name: transformers
license: cc-by-4.0
base_model: hon9kon9ize/bert-large-cantonese
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: bert-suicide-detection-hk-large-new
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# bert-suicide-detection-hk-large-new

This model is a fine-tuned version of [hon9kon9ize/bert-large-cantonese](https://huggingface.co/hon9kon9ize/bert-large-cantonese) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4945
- Accuracy: 0.9085

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 5

### Training results

| Training Loss | Epoch  | Step | Validation Loss | Accuracy |
|:-------------:|:------:|:----:|:---------------:|:--------:|
| 0.5426        | 0.0604 | 20   | 0.7869          | 0.7451   |
| 0.5984        | 0.1208 | 40   | 0.3943          | 0.7908   |
| 0.4864        | 0.1813 | 60   | 0.9365          | 0.7843   |
| 0.6039        | 0.2417 | 80   | 0.6580          | 0.7712   |
| 0.5741        | 0.3021 | 100  | 0.3454          | 0.8235   |
| 0.4276        | 0.3625 | 120  | 0.5421          | 0.8170   |
| 0.4342        | 0.4230 | 140  | 0.4258          | 0.8562   |
| 0.4915        | 0.4834 | 160  | 0.5961          | 0.8301   |
| 0.4127        | 0.5438 | 180  | 0.2987          | 0.8693   |
| 0.3166        | 0.6042 | 200  | 0.3308          | 0.8693   |
| 0.4018        | 0.6647 | 220  | 0.5286          | 0.8039   |
| 0.3007        | 0.7251 | 240  | 0.5845          | 0.8627   |
| 0.4893        | 0.7855 | 260  | 0.3662          | 0.8627   |
| 0.274         | 0.8459 | 280  | 0.3483          | 0.8693   |
| 0.5741        | 0.9063 | 300  | 0.3280          | 0.8824   |
| 0.3752        | 0.9668 | 320  | 0.5251          | 0.8889   |
| 0.2711        | 1.0272 | 340  | 0.6097          | 0.8562   |
| 0.2369        | 1.0876 | 360  | 0.5457          | 0.8693   |
| 0.3756        | 1.1480 | 380  | 0.6890          | 0.8758   |
| 0.6575        | 1.2085 | 400  | 0.4709          | 0.8693   |
| 0.3268        | 1.2689 | 420  | 0.5219          | 0.8497   |
| 0.3994        | 1.3293 | 440  | 0.4282          | 0.8693   |
| 0.0879        | 1.3897 | 460  | 0.6294          | 0.8758   |
| 0.2566        | 1.4502 | 480  | 0.7143          | 0.8627   |
| 0.2897        | 1.5106 | 500  | 0.6120          | 0.8693   |
| 0.321         | 1.5710 | 520  | 0.4749          | 0.8758   |
| 0.1871        | 1.6314 | 540  | 0.4392          | 0.9085   |
| 0.1654        | 1.6918 | 560  | 0.4663          | 0.9085   |
| 0.3166        | 1.7523 | 580  | 0.5048          | 0.8889   |
| 0.222         | 1.8127 | 600  | 0.4550          | 0.9085   |
| 0.4299        | 1.8731 | 620  | 0.3445          | 0.9085   |
| 0.0942        | 1.9335 | 640  | 0.3735          | 0.9281   |
| 0.3991        | 1.9940 | 660  | 0.3646          | 0.9085   |
| 0.0581        | 2.0544 | 680  | 0.3527          | 0.9085   |
| 0.2712        | 2.1148 | 700  | 0.4270          | 0.9020   |
| 0.0443        | 2.1752 | 720  | 0.5462          | 0.8954   |
| 0.3831        | 2.2356 | 740  | 0.3419          | 0.9216   |
| 0.2267        | 2.2961 | 760  | 0.4925          | 0.8889   |
| 0.1821        | 2.3565 | 780  | 0.3625          | 0.9216   |
| 0.2926        | 2.4169 | 800  | 0.3671          | 0.9020   |
| 0.2507        | 2.4773 | 820  | 0.3853          | 0.9020   |
| 0.2446        | 2.5378 | 840  | 0.4571          | 0.8954   |
| 0.1926        | 2.5982 | 860  | 0.5436          | 0.8497   |
| 0.1725        | 2.6586 | 880  | 0.6576          | 0.8497   |
| 0.2033        | 2.7190 | 900  | 0.4772          | 0.9020   |
| 0.0095        | 2.7795 | 920  | 0.4103          | 0.9150   |
| 0.2896        | 2.8399 | 940  | 0.4333          | 0.9085   |
| 0.2661        | 2.9003 | 960  | 0.5793          | 0.8889   |
| 0.1338        | 2.9607 | 980  | 0.4543          | 0.8954   |
| 0.0751        | 3.0211 | 1000 | 0.5029          | 0.8954   |
| 0.2093        | 3.0816 | 1020 | 0.4631          | 0.9020   |
| 0.2436        | 3.1420 | 1040 | 0.5888          | 0.8693   |
| 0.1375        | 3.2024 | 1060 | 0.6457          | 0.8889   |
| 0.0049        | 3.2628 | 1080 | 0.6601          | 0.8889   |
| 0.0089        | 3.3233 | 1100 | 0.6462          | 0.8824   |
| 0.0616        | 3.3837 | 1120 | 0.6607          | 0.8889   |
| 0.006         | 3.4441 | 1140 | 0.6243          | 0.9020   |
| 0.1769        | 3.5045 | 1160 | 0.5257          | 0.9020   |
| 0.0044        | 3.5650 | 1180 | 0.5508          | 0.9085   |
| 0.2295        | 3.6254 | 1200 | 0.4846          | 0.9150   |
| 0.1175        | 3.6858 | 1220 | 0.4764          | 0.9020   |
| 0.0746        | 3.7462 | 1240 | 0.4761          | 0.9020   |
| 0.0222        | 3.8066 | 1260 | 0.4836          | 0.9020   |
| 0.0012        | 3.8671 | 1280 | 0.4775          | 0.9216   |
| 0.2131        | 3.9275 | 1300 | 0.4607          | 0.9020   |
| 0.0006        | 3.9879 | 1320 | 0.4935          | 0.9085   |
| 0.0758        | 4.0483 | 1340 | 0.4592          | 0.9020   |
| 0.1466        | 4.1088 | 1360 | 0.4464          | 0.9085   |
| 0.0488        | 4.1692 | 1380 | 0.4816          | 0.9085   |
| 0.0014        | 4.2296 | 1400 | 0.4570          | 0.9150   |
| 0.082         | 4.2900 | 1420 | 0.4545          | 0.9216   |
| 0.0009        | 4.3505 | 1440 | 0.4721          | 0.9150   |
| 0.0008        | 4.4109 | 1460 | 0.4874          | 0.9216   |
| 0.0014        | 4.4713 | 1480 | 0.5003          | 0.9150   |
| 0.1612        | 4.5317 | 1500 | 0.5064          | 0.9150   |
| 0.2079        | 4.5921 | 1520 | 0.4994          | 0.9150   |
| 0.1423        | 4.6526 | 1540 | 0.4835          | 0.9150   |
| 0.0009        | 4.7130 | 1560 | 0.4825          | 0.9085   |
| 0.0017        | 4.7734 | 1580 | 0.4918          | 0.9085   |
| 0.0648        | 4.8338 | 1600 | 0.4917          | 0.9150   |
| 0.0531        | 4.8943 | 1620 | 0.4919          | 0.9085   |
| 0.0008        | 4.9547 | 1640 | 0.4945          | 0.9085   |


### Framework versions

- Transformers 4.47.1
- Pytorch 2.5.1+cu121
- Datasets 3.2.0
- Tokenizers 0.21.0