Hugging Face's logo

lesso14
/
418e6629-dcd3-424e-b3db-50c11d0d7c6b

PEFT
Safetensors
falcon
axolotl
Generated from Trainer
custom_code
Model card Files Community
lesso14 commited on
Commit
4ea8de5
·
verified ·
1 Parent(s): d9716f7

End of training

Browse files
Files changed (1) hide show
  1. README.md +116 -0
README.md ADDED
@@ -0,0 +1,116 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ license: mit
4
+ base_model: fxmarty/really-tiny-falcon-testing
5
+ tags:
6
+ - axolotl
7
+ - generated_from_trainer
8
+ model-index:
9
+ - name: 418e6629-dcd3-424e-b3db-50c11d0d7c6b
10
+ results: []
11
+ ---
12
+
13
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
14
+ should probably proofread and complete it, then remove this comment. -->
15
+
16
+ [<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
17
+ <br>
18
+
19
+ # 418e6629-dcd3-424e-b3db-50c11d0d7c6b
20
+
21
+ This model is a fine-tuned version of [fxmarty/really-tiny-falcon-testing](https://huggingface.co/fxmarty/really-tiny-falcon-testing) on the None dataset.
22
+ It achieves the following results on the evaluation set:
23
+ - Loss: 10.9416
24
+
25
+ ## Model description
26
+
27
+ More information needed
28
+
29
+ ## Intended uses & limitations
30
+
31
+ More information needed
32
+
33
+ ## Training and evaluation data
34
+
35
+ More information needed
36
+
37
+ ## Training procedure
38
+
39
+ ### Training hyperparameters
40
+
41
+ The following hyperparameters were used during training:
42
+ - learning_rate: 0.000214
43
+ - train_batch_size: 4
44
+ - eval_batch_size: 4
45
+ - seed: 140
46
+ - gradient_accumulation_steps: 8
47
+ - total_train_batch_size: 32
48
+ - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
49
+ - lr_scheduler_type: cosine
50
+ - lr_scheduler_warmup_steps: 100
51
+ - training_steps: 25000
52
+
53
+ ### Training results
54
+
55
+ | Training Loss | Epoch | Step | Validation Loss |
56
+ |:-------------:|:------:|:-----:|:---------------:|
57
+ | No log | 0.0001 | 1 | 11.0842 |
58
+ | 87.9206 | 0.0563 | 500 | 10.9867 |
59
+ | 87.8458 | 0.1127 | 1000 | 10.9740 |
60
+ | 87.8012 | 0.1690 | 1500 | 10.9673 |
61
+ | 87.7685 | 0.2254 | 2000 | 10.9628 |
62
+ | 87.7475 | 0.2817 | 2500 | 10.9602 |
63
+ | 87.7401 | 0.3381 | 3000 | 10.9575 |
64
+ | 87.7205 | 0.3944 | 3500 | 10.9554 |
65
+ | 87.6986 | 0.4508 | 4000 | 10.9532 |
66
+ | 87.6961 | 0.5071 | 4500 | 10.9514 |
67
+ | 87.677 | 0.5635 | 5000 | 10.9498 |
68
+ | 87.6867 | 0.6198 | 5500 | 10.9488 |
69
+ | 87.6773 | 0.6762 | 6000 | 10.9481 |
70
+ | 87.6696 | 0.7325 | 6500 | 10.9472 |
71
+ | 87.664 | 0.7889 | 7000 | 10.9466 |
72
+ | 87.6684 | 0.8452 | 7500 | 10.9463 |
73
+ | 87.6553 | 0.9015 | 8000 | 10.9458 |
74
+ | 87.6592 | 0.9579 | 8500 | 10.9454 |
75
+ | 87.6622 | 1.0143 | 9000 | 10.9448 |
76
+ | 87.6536 | 1.0706 | 9500 | 10.9444 |
77
+ | 87.6519 | 1.1270 | 10000 | 10.9442 |
78
+ | 87.6443 | 1.1833 | 10500 | 10.9439 |
79
+ | 87.6468 | 1.2397 | 11000 | 10.9438 |
80
+ | 87.6477 | 1.2960 | 11500 | 10.9434 |
81
+ | 87.6381 | 1.3524 | 12000 | 10.9433 |
82
+ | 87.6359 | 1.4087 | 12500 | 10.9430 |
83
+ | 87.6432 | 1.4650 | 13000 | 10.9428 |
84
+ | 87.6428 | 1.5214 | 13500 | 10.9428 |
85
+ | 87.634 | 1.5777 | 14000 | 10.9426 |
86
+ | 87.6254 | 1.6341 | 14500 | 10.9425 |
87
+ | 87.6294 | 1.6904 | 15000 | 10.9424 |
88
+ | 87.6307 | 1.7468 | 15500 | 10.9422 |
89
+ | 87.6393 | 1.8031 | 16000 | 10.9422 |
90
+ | 87.6266 | 1.8595 | 16500 | 10.9421 |
91
+ | 87.6327 | 1.9158 | 17000 | 10.9419 |
92
+ | 87.6298 | 1.9722 | 17500 | 10.9419 |
93
+ | 87.6353 | 2.0285 | 18000 | 10.9420 |
94
+ | 87.6329 | 2.0849 | 18500 | 10.9418 |
95
+ | 87.6332 | 2.1412 | 19000 | 10.9418 |
96
+ | 87.6301 | 2.1976 | 19500 | 10.9417 |
97
+ | 87.6308 | 2.2539 | 20000 | 10.9417 |
98
+ | 87.6302 | 2.3103 | 20500 | 10.9417 |
99
+ | 87.6378 | 2.3666 | 21000 | 10.9416 |
100
+ | 87.6317 | 2.4230 | 21500 | 10.9416 |
101
+ | 87.6272 | 2.4793 | 22000 | 10.9416 |
102
+ | 87.6308 | 2.5357 | 22500 | 10.9416 |
103
+ | 87.6299 | 2.5920 | 23000 | 10.9416 |
104
+ | 87.6303 | 2.6484 | 23500 | 10.9416 |
105
+ | 87.6262 | 2.7047 | 24000 | 10.9415 |
106
+ | 87.6281 | 2.7610 | 24500 | 10.9415 |
107
+ | 87.6344 | 2.8174 | 25000 | 10.9416 |
108
+
109
+
110
+ ### Framework versions
111
+
112
+ - PEFT 0.13.2
113
+ - Transformers 4.46.0
114
+ - Pytorch 2.5.0+cu124
115
+ - Datasets 3.0.1
116
+ - Tokenizers 0.20.1