Hugging Face's logo

MikeMpapa
/
8_bar_lmd_clean_custom_final_token_scheme_epochs10

Text Generation
Transformers
TensorBoard
Safetensors
gpt2
Generated from Trainer
text-generation-inference
Model card Files
xet
Metrics Community
MikeMpapa commited on
Commit
c3edcac
·
verified ·
1 Parent(s): 059e4d1

Model save

Browse files
Files changed (5) hide show
  1. README.md +233 -0
  2. generation_config.json +5 -0
  3. model.safetensors +1 -1
  4. runs/Aug01_09-50-27_algo-1/events.out.tfevents.1722505855.algo-1.52.0 +2 -2
  5. training_args.bin +1 -1
README.md ADDED
@@ -0,0 +1,233 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ base_model: gpt2
4
+ tags:
5
+ - generated_from_trainer
6
+ model-index:
7
+ - name: 8_bar_lmd_clean_custom_final_token_scheme_epochs10
8
+ results: []
9
+ ---
10
+
11
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
12
+ should probably proofread and complete it, then remove this comment. -->
13
+
14
+ # 8_bar_lmd_clean_custom_final_token_scheme_epochs10
15
+
16
+ This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on an unknown dataset.
17
+ It achieves the following results on the evaluation set:
18
+ - Loss: 5.0474
19
+
20
+ ## Model description
21
+
22
+ More information needed
23
+
24
+ ## Intended uses & limitations
25
+
26
+ More information needed
27
+
28
+ ## Training and evaluation data
29
+
30
+ More information needed
31
+
32
+ ## Training procedure
33
+
34
+ ### Training hyperparameters
35
+
36
+ The following hyperparameters were used during training:
37
+ - learning_rate: 0.005
38
+ - train_batch_size: 48
39
+ - eval_batch_size: 32
40
+ - seed: 1
41
+ - gradient_accumulation_steps: 2
42
+ - total_train_batch_size: 96
43
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
44
+ - lr_scheduler_type: cosine
45
+ - lr_scheduler_warmup_ratio: 0.01
46
+ - num_epochs: 5
47
+
48
+ ### Training results
49
+
50
+ | Training Loss | Epoch | Step | Validation Loss |
51
+ |:-------------:|:-----:|:----:|:---------------:|
52
+ | 8.2004 | 0.03 | 10 | 5.8581 |
53
+ | 5.8284 | 0.06 | 20 | 5.7712 |
54
+ | 5.7438 | 0.09 | 30 | 5.7220 |
55
+ | 5.682 | 0.11 | 40 | 5.6185 |
56
+ | 5.6048 | 0.14 | 50 | 5.5303 |
57
+ | 5.5522 | 0.17 | 60 | 5.5000 |
58
+ | 5.5071 | 0.2 | 70 | 5.4540 |
59
+ | 5.4512 | 0.23 | 80 | 5.3570 |
60
+ | 5.2932 | 0.26 | 90 | 5.1704 |
61
+ | 5.1931 | 0.29 | 100 | 5.1101 |
62
+ | 5.109 | 0.32 | 110 | 5.0844 |
63
+ | 5.1019 | 0.34 | 120 | 5.0580 |
64
+ | 5.0702 | 0.37 | 130 | 5.0452 |
65
+ | 5.0727 | 0.4 | 140 | 5.0375 |
66
+ | 5.0654 | 0.43 | 150 | 5.0376 |
67
+ | 5.0625 | 0.46 | 160 | 5.0292 |
68
+ | 5.0426 | 0.49 | 170 | 5.0376 |
69
+ | 5.0428 | 0.52 | 180 | 5.0328 |
70
+ | 5.0153 | 0.54 | 190 | 5.0280 |
71
+ | 5.0307 | 0.57 | 200 | 5.0256 |
72
+ | 5.0321 | 0.6 | 210 | 5.0202 |
73
+ | 5.0638 | 0.63 | 220 | 5.0150 |
74
+ | 5.0363 | 0.66 | 230 | 5.0077 |
75
+ | 5.0151 | 0.69 | 240 | 5.0067 |
76
+ | 5.0359 | 0.72 | 250 | 5.0131 |
77
+ | 4.9987 | 0.74 | 260 | 4.9949 |
78
+ | 5.0128 | 0.77 | 270 | 4.9826 |
79
+ | 5.0164 | 0.8 | 280 | 4.9872 |
80
+ | 4.9941 | 0.83 | 290 | 4.9869 |
81
+ | 5.0258 | 0.86 | 300 | 4.9968 |
82
+ | 5.0132 | 0.89 | 310 | 5.0012 |
83
+ | 5.0568 | 0.92 | 320 | 5.0877 |
84
+ | 5.1161 | 0.95 | 330 | 5.0727 |
85
+ | 5.1321 | 0.97 | 340 | 5.1333 |
86
+ | 5.1533 | 1.0 | 350 | 5.1079 |
87
+ | 5.1469 | 1.03 | 360 | 5.0910 |
88
+ | 5.1172 | 1.06 | 370 | 5.0900 |
89
+ | 5.1259 | 1.09 | 380 | 5.1383 |
90
+ | 5.1742 | 1.12 | 390 | 5.1814 |
91
+ | 5.2056 | 1.15 | 400 | 5.1766 |
92
+ | 5.1964 | 1.17 | 410 | 5.1935 |
93
+ | 5.2422 | 1.2 | 420 | 5.2032 |
94
+ | 5.2544 | 1.23 | 430 | 5.1723 |
95
+ | 5.2853 | 1.26 | 440 | 5.2317 |
96
+ | 5.302 | 1.29 | 450 | 5.3312 |
97
+ | 5.319 | 1.32 | 460 | 5.2069 |
98
+ | 5.2633 | 1.35 | 470 | 5.2114 |
99
+ | 5.2548 | 1.38 | 480 | 5.2350 |
100
+ | 5.294 | 1.4 | 490 | 5.3470 |
101
+ | 5.2876 | 1.43 | 500 | 5.1773 |
102
+ | 5.2857 | 1.46 | 510 | 5.2445 |
103
+ | 5.3095 | 1.49 | 520 | 5.2099 |
104
+ | 5.2322 | 1.52 | 530 | 5.2158 |
105
+ | 5.215 | 1.55 | 540 | 5.1505 |
106
+ | 5.2248 | 1.58 | 550 | 5.1520 |
107
+ | 5.2123 | 1.6 | 560 | 5.1412 |
108
+ | 5.2098 | 1.63 | 570 | 5.1431 |
109
+ | 5.2088 | 1.66 | 580 | 5.1443 |
110
+ | 5.2007 | 1.69 | 590 | 5.1595 |
111
+ | 5.212 | 1.72 | 600 | 5.2016 |
112
+ | 5.2143 | 1.75 | 610 | 5.1499 |
113
+ | 5.2152 | 1.78 | 620 | 5.1333 |
114
+ | 5.2003 | 1.81 | 630 | 5.1810 |
115
+ | 5.2761 | 1.83 | 640 | 5.1993 |
116
+ | 5.2707 | 1.86 | 650 | 5.1884 |
117
+ | 5.2622 | 1.89 | 660 | 5.1815 |
118
+ | 5.242 | 1.92 | 670 | 5.1830 |
119
+ | 5.2705 | 1.95 | 680 | 5.2060 |
120
+ | 5.28 | 1.98 | 690 | 5.1905 |
121
+ | 5.2443 | 2.01 | 700 | 5.1681 |
122
+ | 5.2252 | 2.03 | 710 | 5.1609 |
123
+ | 5.2256 | 2.06 | 720 | 5.1565 |
124
+ | 5.2043 | 2.09 | 730 | 5.1541 |
125
+ | 5.2366 | 2.12 | 740 | 5.1640 |
126
+ | 5.2441 | 2.15 | 750 | 5.1742 |
127
+ | 5.2585 | 2.18 | 760 | 5.1817 |
128
+ | 5.2182 | 2.21 | 770 | 5.1588 |
129
+ | 5.2153 | 2.23 | 780 | 5.1710 |
130
+ | 5.218 | 2.26 | 790 | 5.1414 |
131
+ | 5.2067 | 2.29 | 800 | 5.1374 |
132
+ | 5.1897 | 2.32 | 810 | 5.1294 |
133
+ | 5.195 | 2.35 | 820 | 5.1315 |
134
+ | 5.2114 | 2.38 | 830 | 5.1335 |
135
+ | 5.2061 | 2.41 | 840 | 5.1366 |
136
+ | 5.1933 | 2.44 | 850 | 5.1307 |
137
+ | 5.1856 | 2.46 | 860 | 5.1311 |
138
+ | 5.1972 | 2.49 | 870 | 5.1317 |
139
+ | 5.1958 | 2.52 | 880 | 5.1356 |
140
+ | 5.2131 | 2.55 | 890 | 5.1300 |
141
+ | 5.1906 | 2.58 | 900 | 5.1177 |
142
+ | 5.1984 | 2.61 | 910 | 5.1166 |
143
+ | 5.1799 | 2.64 | 920 | 5.1297 |
144
+ | 5.1874 | 2.66 | 930 | 5.1328 |
145
+ | 5.1896 | 2.69 | 940 | 5.1219 |
146
+ | 5.1864 | 2.72 | 950 | 5.1286 |
147
+ | 5.203 | 2.75 | 960 | 5.1238 |
148
+ | 5.2038 | 2.78 | 970 | 5.1183 |
149
+ | 5.1992 | 2.81 | 980 | 5.1116 |
150
+ | 5.1733 | 2.84 | 990 | 5.1099 |
151
+ | 5.1604 | 2.87 | 1000 | 5.1101 |
152
+ | 5.1657 | 2.89 | 1010 | 5.1067 |
153
+ | 5.1594 | 2.92 | 1020 | 5.0977 |
154
+ | 5.1501 | 2.95 | 1030 | 5.0945 |
155
+ | 5.1485 | 2.98 | 1040 | 5.0897 |
156
+ | 5.164 | 3.01 | 1050 | 5.0835 |
157
+ | 5.1394 | 3.04 | 1060 | 5.0955 |
158
+ | 5.1472 | 3.07 | 1070 | 5.1000 |
159
+ | 5.2015 | 3.09 | 1080 | 5.0947 |
160
+ | 5.1665 | 3.12 | 1090 | 5.0916 |
161
+ | 5.1483 | 3.15 | 1100 | 5.0958 |
162
+ | 5.1753 | 3.18 | 1110 | 5.0902 |
163
+ | 5.1565 | 3.21 | 1120 | 5.0886 |
164
+ | 5.1572 | 3.24 | 1130 | 5.0876 |
165
+ | 5.1193 | 3.27 | 1140 | 5.0900 |
166
+ | 5.144 | 3.3 | 1150 | 5.0807 |
167
+ | 5.1362 | 3.32 | 1160 | 5.0913 |
168
+ | 5.1574 | 3.35 | 1170 | 5.0780 |
169
+ | 5.1381 | 3.38 | 1180 | 5.0738 |
170
+ | 5.1405 | 3.41 | 1190 | 5.0739 |
171
+ | 5.1463 | 3.44 | 1200 | 5.0739 |
172
+ | 5.1324 | 3.47 | 1210 | 5.0729 |
173
+ | 5.1102 | 3.5 | 1220 | 5.0703 |
174
+ | 5.1575 | 3.52 | 1230 | 5.0700 |
175
+ | 5.125 | 3.55 | 1240 | 5.0674 |
176
+ | 5.1391 | 3.58 | 1250 | 5.0673 |
177
+ | 5.1405 | 3.61 | 1260 | 5.0678 |
178
+ | 5.141 | 3.64 | 1270 | 5.0708 |
179
+ | 5.1059 | 3.67 | 1280 | 5.0719 |
180
+ | 5.1423 | 3.7 | 1290 | 5.0719 |
181
+ | 5.1098 | 3.72 | 1300 | 5.0698 |
182
+ | 5.1165 | 3.75 | 1310 | 5.0674 |
183
+ | 5.1249 | 3.78 | 1320 | 5.0660 |
184
+ | 5.1129 | 3.81 | 1330 | 5.0675 |
185
+ | 5.1469 | 3.84 | 1340 | 5.0677 |
186
+ | 5.1215 | 3.87 | 1350 | 5.0688 |
187
+ | 5.1392 | 3.9 | 1360 | 5.0685 |
188
+ | 5.1355 | 3.93 | 1370 | 5.0674 |
189
+ | 5.1372 | 3.95 | 1380 | 5.0666 |
190
+ | 5.1237 | 3.98 | 1390 | 5.0641 |
191
+ | 5.1452 | 4.01 | 1400 | 5.0623 |
192
+ | 5.117 | 4.04 | 1410 | 5.0642 |
193
+ | 5.1467 | 4.07 | 1420 | 5.0604 |
194
+ | 5.1221 | 4.1 | 1430 | 5.0590 |
195
+ | 5.0959 | 4.13 | 1440 | 5.0540 |
196
+ | 5.1088 | 4.15 | 1450 | 5.0538 |
197
+ | 5.116 | 4.18 | 1460 | 5.0542 |
198
+ | 5.1293 | 4.21 | 1470 | 5.0542 |
199
+ | 5.1337 | 4.24 | 1480 | 5.0526 |
200
+ | 5.1154 | 4.27 | 1490 | 5.0522 |
201
+ | 5.1196 | 4.3 | 1500 | 5.0538 |
202
+ | 5.1122 | 4.33 | 1510 | 5.0515 |
203
+ | 5.094 | 4.36 | 1520 | 5.0503 |
204
+ | 5.116 | 4.38 | 1530 | 5.0505 |
205
+ | 5.1142 | 4.41 | 1540 | 5.0520 |
206
+ | 5.1106 | 4.44 | 1550 | 5.0517 |
207
+ | 5.1023 | 4.47 | 1560 | 5.0502 |
208
+ | 5.1153 | 4.5 | 1570 | 5.0497 |
209
+ | 5.1096 | 4.53 | 1580 | 5.0493 |
210
+ | 5.1331 | 4.56 | 1590 | 5.0503 |
211
+ | 5.1178 | 4.58 | 1600 | 5.0506 |
212
+ | 5.0984 | 4.61 | 1610 | 5.0503 |
213
+ | 5.0992 | 4.64 | 1620 | 5.0493 |
214
+ | 5.0888 | 4.67 | 1630 | 5.0487 |
215
+ | 5.1153 | 4.7 | 1640 | 5.0484 |
216
+ | 5.1102 | 4.73 | 1650 | 5.0480 |
217
+ | 5.1143 | 4.76 | 1660 | 5.0478 |
218
+ | 5.1012 | 4.79 | 1670 | 5.0477 |
219
+ | 5.104 | 4.81 | 1680 | 5.0478 |
220
+ | 5.1129 | 4.84 | 1690 | 5.0475 |
221
+ | 5.0908 | 4.87 | 1700 | 5.0475 |
222
+ | 5.091 | 4.9 | 1710 | 5.0474 |
223
+ | 5.1115 | 4.93 | 1720 | 5.0474 |
224
+ | 5.1267 | 4.96 | 1730 | 5.0474 |
225
+ | 5.1245 | 4.99 | 1740 | 5.0474 |
226
+
227
+
228
+ ### Framework versions
229
+
230
+ - Transformers 4.36.0
231
+ - Pytorch 2.1.0
232
+ - Datasets 2.15.0
233
+ - Tokenizers 0.15.1
generation_config.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "pad_token_id": 3,
4
+ "transformers_version": "4.36.0"
5
+ }
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a477b0c94d92ec1b78a43b2f0414e5501b34461600ae32bd8bcc8e9d1f3a7dcd
3
  size 141303176
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:01547fe1a3321945e2658d17fc577d59743704a9d3bcd23fd2d9d71cb195824b
3
  size 141303176
runs/Aug01_09-50-27_algo-1/events.out.tfevents.1722505855.algo-1.52.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2f358ca570fc1e16b5b5b7ebd18cc6e7f11e870dbde0de4579549112be5ac4e4
3
- size 77993
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e1878482323d94aaf5b0268838baca0fa7143706115ad92d49de67c82add1496
3
+ size 79631
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:eaab9c98694dbc3aa23b03d4ddb489d6e2a3e596934b52cf9544a862c460bf40
3
  size 4856
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4c94a70ab557e0290df0231cd5f77305881f00142d888de736fc5a694229529a
3
  size 4856