Hugging Face's logo

aiface
/
CafeBERT_massive_v2

Transformers
Safetensors
Generated from Trainer
Model card Files
xet
Community
aiface commited on
Commit
c5a1952
·
verified ·
1 Parent(s): 3da4534

Model save

Browse files
Files changed (5) hide show
  1. README.md +72 -0
  2. intent_report_test.txt +75 -0
  3. model.safetensors +1 -1
  4. model_predict_test.csv +0 -0
  5. slot_report_test.txt +59 -0
README.md ADDED
@@ -0,0 +1,72 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ license: apache-2.0
4
+ base_model: uitnlp/CafeBERT
5
+ tags:
6
+ - generated_from_trainer
7
+ model-index:
8
+ - name: CafeBERT_massive_v2
9
+ results: []
10
+ ---
11
+
12
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
13
+ should probably proofread and complete it, then remove this comment. -->
14
+
15
+ # CafeBERT_massive_v2
16
+
17
+ This model is a fine-tuned version of [uitnlp/CafeBERT](https://huggingface.co/uitnlp/CafeBERT) on an unknown dataset.
18
+ It achieves the following results on the evaluation set:
19
+ - Loss: 0.9485
20
+ - Slot P: 0.0090
21
+ - Slot R: 0.0199
22
+ - Slot F1: 0.0124
23
+ - Slot Exact Match: 0.0418
24
+ - Intent Acc: 0.8647
25
+
26
+ ## Model description
27
+
28
+ More information needed
29
+
30
+ ## Intended uses & limitations
31
+
32
+ More information needed
33
+
34
+ ## Training and evaluation data
35
+
36
+ More information needed
37
+
38
+ ## Training procedure
39
+
40
+ ### Training hyperparameters
41
+
42
+ The following hyperparameters were used during training:
43
+ - learning_rate: 5e-05
44
+ - train_batch_size: 128
45
+ - eval_batch_size: 128
46
+ - seed: 42
47
+ - gradient_accumulation_steps: 2
48
+ - total_train_batch_size: 256
49
+ - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
50
+ - lr_scheduler_type: cosine
51
+ - lr_scheduler_warmup_ratio: 0.06
52
+ - num_epochs: 30
53
+ - mixed_precision_training: Native AMP
54
+
55
+ ### Training results
56
+
57
+ | Training Loss | Epoch | Step | Validation Loss | Slot P | Slot R | Slot F1 | Slot Exact Match | Intent Acc |
58
+ |:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:-------:|:----------------:|:----------:|
59
+ | No log | 1.0 | 45 | 4.1897 | 0.0 | 0.0 | 0.0 | 0.4088 | 0.3478 |
60
+ | 12.6262 | 2.0 | 90 | 1.3454 | 0.0090 | 0.0129 | 0.0106 | 0.1943 | 0.8406 |
61
+ | 4.1265 | 3.0 | 135 | 1.0228 | 0.0102 | 0.0193 | 0.0134 | 0.1181 | 0.8564 |
62
+ | 1.987 | 4.0 | 180 | 0.9293 | 0.0100 | 0.0188 | 0.0130 | 0.1313 | 0.8692 |
63
+ | 1.382 | 5.0 | 225 | 0.9144 | 0.0096 | 0.0205 | 0.0131 | 0.0644 | 0.8706 |
64
+ | 1.0689 | 6.0 | 270 | 0.9485 | 0.0090 | 0.0199 | 0.0124 | 0.0418 | 0.8647 |
65
+
66
+
67
+ ### Framework versions
68
+
69
+ - Transformers 4.55.0
70
+ - Pytorch 2.7.0+cu126
71
+ - Datasets 3.6.0
72
+ - Tokenizers 0.21.4
intent_report_test.txt ADDED
@@ -0,0 +1,75 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ precision recall f1-score support
2
+
3
+ 0 0.89 0.97 0.92 88
4
+ 1 0.89 0.86 0.87 36
5
+ 2 0.92 0.94 0.93 35
6
+ 3 0.91 0.89 0.90 35
7
+ 4 0.85 0.88 0.87 26
8
+ 5 0.00 0.00 0.00 1
9
+ 6 0.66 0.86 0.75 43
10
+ 7 1.00 0.25 0.40 4
11
+ 8 1.00 0.78 0.88 18
12
+ 9 0.97 0.90 0.94 72
13
+ 10 0.97 1.00 0.99 39
14
+ 11 0.75 1.00 0.86 15
15
+ 12 0.68 0.56 0.61 169
16
+ 13 0.97 0.96 0.96 156
17
+ 14 0.83 0.77 0.80 13
18
+ 15 0.56 0.75 0.64 12
19
+ 16 0.83 0.86 0.84 22
20
+ 17 0.81 0.85 0.83 26
21
+ 18 0.81 0.96 0.88 27
22
+ 19 0.87 0.87 0.87 31
23
+ 20 0.82 0.78 0.80 41
24
+ 21 0.88 0.72 0.79 39
25
+ 22 0.71 0.92 0.80 124
26
+ 23 1.00 0.85 0.92 34
27
+ 24 1.00 0.80 0.89 10
28
+ 25 1.00 1.00 1.00 19
29
+ 26 0.94 0.84 0.89 57
30
+ 27 0.90 0.72 0.80 25
31
+ 28 0.43 0.50 0.46 6
32
+ 29 1.00 0.50 0.67 6
33
+ 30 0.93 0.97 0.95 67
34
+ 31 0.90 0.90 0.90 21
35
+ 32 0.72 0.82 0.77 126
36
+ 33 0.94 0.92 0.93 114
37
+ 34 0.96 0.88 0.92 26
38
+ 35 0.91 0.91 0.91 11
39
+ 36 0.77 0.88 0.82 72
40
+ 37 0.00 0.00 0.00 0
41
+ 38 1.00 0.53 0.70 15
42
+ 39 0.88 0.88 0.88 25
43
+ 40 0.93 0.95 0.94 43
44
+ 41 0.00 0.00 0.00 3
45
+ 42 0.89 0.78 0.83 51
46
+ 43 0.88 0.83 0.86 36
47
+ 44 0.96 0.93 0.94 119
48
+ 45 0.87 0.93 0.90 176
49
+ 46 0.86 0.97 0.91 32
50
+ 47 0.94 0.91 0.93 81
51
+ 48 0.95 0.95 0.95 41
52
+ 49 0.77 0.78 0.78 141
53
+ 50 0.98 0.91 0.94 209
54
+ 51 0.97 0.94 0.96 35
55
+ 52 1.00 1.00 1.00 21
56
+ 53 0.92 0.90 0.91 52
57
+ 54 0.92 0.96 0.94 23
58
+ 55 0.67 0.80 0.73 20
59
+ 56 1.00 0.97 0.99 36
60
+ 57 0.94 0.83 0.88 35
61
+ 58 0.92 0.76 0.83 63
62
+ 59 0.77 0.86 0.81 51
63
+
64
+ accuracy 0.87 2974
65
+ macro avg 0.83 0.80 0.81 2974
66
+ weighted avg 0.87 0.87 0.87 2974
67
+
68
+ Confusion matrix:
69
+ [[85 0 0 ... 0 0 0]
70
+ [ 0 31 0 ... 0 0 0]
71
+ [ 0 0 33 ... 0 0 0]
72
+ ...
73
+ [ 0 0 0 ... 29 0 0]
74
+ [ 0 0 0 ... 0 48 0]
75
+ [ 0 0 0 ... 0 1 44]]
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:15cb6f9f4eb802206644e066fb6a3e27ed04345afc7bf413b3600b71ef978f19
3
  size 2240311788
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f70db3715d9ace92e827e5164c3ad2bbc00859adfc36e3a1acdb5d77c8b4ea86
3
  size 2240311788
model_predict_test.csv ADDED
The diff for this file is too large to render. See raw diff
 
slot_report_test.txt ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ precision recall f1-score support
2
+
3
+ alarm_type 0.00 0.00 0.00 2
4
+ app_name 0.00 0.00 0.00 5
5
+ artist_name 0.00 0.02 0.01 52
6
+ audiobook_author 0.00 0.00 0.00 5
7
+ audiobook_name 0.00 0.00 0.00 22
8
+ business_name 0.00 0.00 0.00 89
9
+ business_type 0.00 0.00 0.00 27
10
+ change_amount 0.00 0.00 0.00 7
11
+ coffee_type 0.00 0.00 0.00 3
12
+ color_type 0.00 0.00 0.00 9
13
+ cooking_type 0.00 0.00 0.00 8
14
+ currency_name 0.01 0.02 0.01 50
15
+ date 0.00 0.00 0.00 365
16
+ definition_word 0.00 0.00 0.00 51
17
+ device_type 0.00 0.00 0.00 28
18
+ drink_type 0.00 0.00 0.00 1
19
+ email_address 0.00 0.00 0.00 9
20
+ email_folder 0.00 0.00 0.00 5
21
+ event_name 0.00 0.00 0.00 237
22
+ food_type 0.01 0.03 0.02 63
23
+ game_name 0.00 0.00 0.00 23
24
+ general_frequency 0.00 0.00 0.00 17
25
+ house_place 0.00 0.00 0.00 14
26
+ ingredient 0.00 0.00 0.00 5
27
+ joke_type 0.00 0.00 0.00 11
28
+ list_name 0.00 0.00 0.00 61
29
+ meal_type 0.00 0.00 0.00 13
30
+ media_type 0.09 0.20 0.13 119
31
+ movie_name 0.00 0.00 0.00 2
32
+ movie_type 0.00 0.00 0.00 3
33
+ music_album 0.00 0.00 0.00 1
34
+ music_descriptor 0.00 0.00 0.00 4
35
+ music_genre 0.01 0.02 0.01 44
36
+ news_topic 0.00 0.00 0.00 43
37
+ order_type 0.00 0.00 0.00 9
38
+ person 0.02 0.04 0.02 212
39
+ personal_info 0.00 0.00 0.00 13
40
+ place_name 0.00 0.00 0.00 257
41
+ player_setting 0.09 0.09 0.09 35
42
+ playlist_name 0.00 0.00 0.00 12
43
+ podcast_descriptor 0.00 0.00 0.00 21
44
+ podcast_name 0.00 0.00 0.00 16
45
+ radio_name 0.00 0.00 0.00 29
46
+ relation 0.00 0.00 0.00 53
47
+ song_name 0.00 0.00 0.00 34
48
+ time 0.00 0.00 0.00 161
49
+ time_zone 0.00 0.00 0.00 13
50
+ timeofday 0.03 0.02 0.03 48
51
+ transport_agency 0.00 0.00 0.00 9
52
+ transport_descriptor 0.00 0.00 0.00 2
53
+ transport_name 0.00 0.00 0.00 4
54
+ transport_type 0.01 0.02 0.01 63
55
+ weather_descriptor 0.00 0.00 0.00 48
56
+
57
+ micro avg 0.01 0.02 0.01 2437
58
+ macro avg 0.01 0.01 0.01 2437
59
+ weighted avg 0.01 0.02 0.01 2437