Yassinj commited on
Commit
22dae32
verified
1 Parent(s): 4aafc45

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -1
README.md CHANGED
@@ -56,10 +56,20 @@ The model was fine-tuned with the following hyperparameters:
56
  - **Mixed Precision**: FP16
57
  - **Best Model Criteria**: `eval_loss`, with `greater_is_better=False`
58
 
 
 
 
 
 
 
 
 
 
 
59
  Validation was performed using a separate subset of the dataset. The final training and validation loss are as follows:
60
 
61
  <p align="center">
62
- <img src="train-val-curve.png" alt="Training and Validation Loss" width="45%"/>
63
  </p>
64
 
65
  ## Evaluation Results
 
56
  - **Mixed Precision**: FP16
57
  - **Best Model Criteria**: `eval_loss`, with `greater_is_better=False`
58
 
59
+ ### LoRA Hyperparameters
60
+
61
+ The fine-tuning process also included the following LoRA (Low-Rank Adaptation) configuration:
62
+
63
+ - **Rank (r)**: 8
64
+ - **Alpha**: 16
65
+ - **Dropout**: 0.05
66
+ - **Bias**: None
67
+ - **Task Type**: Causal Language Modeling (CAUSAL_LM)
68
+
69
  Validation was performed using a separate subset of the dataset. The final training and validation loss are as follows:
70
 
71
  <p align="center">
72
+ <img src="train-val-curve.png" alt="Training and Validation Loss" width="85%"/>
73
  </p>
74
 
75
  ## Evaluation Results