Hugging Face's logo

strickvl
/
isafpr-tiny-llama-lora

PEFT
Safetensors
llama
axolotl
Generated from Trainer
4-bit precision
bitsandbytes
Model card Files Community
strickvl commited on
Commit
d0713fa
·
verified ·
1 Parent(s): 14cca0c

End of training

Browse files
Files changed (2) hide show
  1. README.md +16 -18
  2. adapter_model.bin +1 -1
README.md CHANGED
@@ -96,7 +96,7 @@ special_tokens:
96
 
97
  This model is a fine-tuned version of [TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T) on the None dataset.
98
  It achieves the following results on the evaluation set:
99
- - Loss: 0.0212
100
 
101
  ## Model description
102
 
@@ -133,23 +133,21 @@ The following hyperparameters were used during training:
133
 
134
  | Training Loss | Epoch | Step | Validation Loss |
135
  |:-------------:|:------:|:----:|:---------------:|
136
- | 0.8068 | 0.0227 | 1 | 0.8529 |
137
- | 0.4759 | 0.25 | 11 | 0.4152 |
138
- | 0.0851 | 0.5 | 22 | 0.0833 |
139
- | 0.0385 | 0.75 | 33 | 0.0434 |
140
- | 0.0321 | 1.0 | 44 | 0.0365 |
141
- | 0.0326 | 1.1705 | 55 | 0.0315 |
142
- | 0.1114 | 1.4205 | 66 | 0.0283 |
143
- | 0.0275 | 1.6705 | 77 | 0.0261 |
144
- | 0.0282 | 1.9205 | 88 | 0.0246 |
145
- | 0.0206 | 2.0909 | 99 | 0.0237 |
146
- | 0.0675 | 2.3409 | 110 | 0.0228 |
147
- | 0.0201 | 2.5909 | 121 | 0.0222 |
148
- | 0.0176 | 2.8409 | 132 | 0.0218 |
149
- | 0.0941 | 3.0114 | 143 | 0.0214 |
150
- | 0.0262 | 3.2614 | 154 | 0.0213 |
151
- | 0.051 | 3.5114 | 165 | 0.0213 |
152
- | 0.0184 | 3.7614 | 176 | 0.0212 |
153
 
154
 
155
  ### Framework versions
 
96
 
97
  This model is a fine-tuned version of [TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T) on the None dataset.
98
  It achieves the following results on the evaluation set:
99
+ - Loss: 0.0557
100
 
101
  ## Model description
102
 
 
133
 
134
  | Training Loss | Epoch | Step | Validation Loss |
135
  |:-------------:|:------:|:----:|:---------------:|
136
+ | 1.7724 | 0.0303 | 1 | 1.7779 |
137
+ | 1.2158 | 0.2727 | 9 | 1.0692 |
138
+ | 0.2116 | 0.5455 | 18 | 0.1796 |
139
+ | 0.1051 | 0.8182 | 27 | 0.1048 |
140
+ | 0.0762 | 1.0227 | 36 | 0.0859 |
141
+ | 0.0704 | 1.2955 | 45 | 0.0763 |
142
+ | 0.0661 | 1.5682 | 54 | 0.0692 |
143
+ | 0.073 | 1.8409 | 63 | 0.0646 |
144
+ | 0.0625 | 2.0455 | 72 | 0.0621 |
145
+ | 0.0522 | 2.3182 | 81 | 0.0602 |
146
+ | 0.0472 | 2.5909 | 90 | 0.0580 |
147
+ | 0.0545 | 2.8636 | 99 | 0.0571 |
148
+ | 0.0467 | 3.0682 | 108 | 0.0561 |
149
+ | 0.057 | 3.3409 | 117 | 0.0557 |
150
+ | 0.0477 | 3.6136 | 126 | 0.0557 |
 
 
151
 
152
 
153
  ### Framework versions
adapter_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f999cad7e14f1e3ae89430a9cad5ba5d51d1d035a409ac2f6b22a690a6ef6baa
3
  size 101036698
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8807cbd5b63ed7c6a4e11d0030904f68fe580184ce532161badaa79e40a890f9
3
  size 101036698