Hugging Face's logo

Sorour
/
cls_fomc_phi3_v1

PEFT
TensorBoard
Safetensors
trl
sft
Generated from Trainer
Model card Files Metrics Community
Sorour commited on
Commit
ad662de
·
verified ·
1 Parent(s): d189e55

Model save

Browse files
Files changed (1) hide show
  1. README.md +8 -11
README.md CHANGED
@@ -20,7 +20,7 @@ should probably proofread and complete it, then remove this comment. -->
20
 
21
  This model is a fine-tuned version of [microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct) on the generator dataset.
22
  It achieves the following results on the evaluation set:
23
- - Loss: 0.6130
24
 
25
  ## Model description
26
 
@@ -40,11 +40,11 @@ More information needed
40
 
41
  The following hyperparameters were used during training:
42
  - learning_rate: 0.0002
43
- - train_batch_size: 1
44
  - eval_batch_size: 8
45
  - seed: 42
46
  - gradient_accumulation_steps: 4
47
- - total_train_batch_size: 4
48
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
49
  - lr_scheduler_type: constant
50
  - lr_scheduler_warmup_ratio: 0.03
@@ -55,18 +55,15 @@ The following hyperparameters were used during training:
55
 
56
  | Training Loss | Epoch | Step | Validation Loss |
57
  |:-------------:|:------:|:----:|:---------------:|
58
- | 0.6352 | 0.3089 | 20 | 0.6405 |
59
- | 0.6186 | 0.6178 | 40 | 0.6192 |
60
- | 0.6122 | 0.9266 | 60 | 0.6108 |
61
- | 0.5154 | 1.2355 | 80 | 0.6209 |
62
- | 0.5187 | 1.5444 | 100 | 0.6251 |
63
- | 0.515 | 1.8533 | 120 | 0.6130 |
64
 
65
 
66
  ### Framework versions
67
 
68
  - PEFT 0.11.1
69
- - Transformers 4.41.0
70
- - Pytorch 2.2.1+cu121
71
  - Datasets 2.19.1
72
  - Tokenizers 0.19.1
 
20
 
21
  This model is a fine-tuned version of [microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct) on the generator dataset.
22
  It achieves the following results on the evaluation set:
23
+ - Loss: 0.6211
24
 
25
  ## Model description
26
 
 
40
 
41
  The following hyperparameters were used during training:
42
  - learning_rate: 0.0002
43
+ - train_batch_size: 2
44
  - eval_batch_size: 8
45
  - seed: 42
46
  - gradient_accumulation_steps: 4
47
+ - total_train_batch_size: 8
48
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
49
  - lr_scheduler_type: constant
50
  - lr_scheduler_warmup_ratio: 0.03
 
55
 
56
  | Training Loss | Epoch | Step | Validation Loss |
57
  |:-------------:|:------:|:----:|:---------------:|
58
+ | 0.639 | 0.6299 | 20 | 0.6435 |
59
+ | 0.5796 | 1.2598 | 40 | 0.6208 |
60
+ | 0.5579 | 1.8898 | 60 | 0.6211 |
 
 
 
61
 
62
 
63
  ### Framework versions
64
 
65
  - PEFT 0.11.1
66
+ - Transformers 4.41.1
67
+ - Pytorch 2.3.0+cu121
68
  - Datasets 2.19.1
69
  - Tokenizers 0.19.1