Update README.md
Browse files
README.md
CHANGED
@@ -56,10 +56,20 @@ The model was fine-tuned with the following hyperparameters:
|
|
56 |
- **Mixed Precision**: FP16
|
57 |
- **Best Model Criteria**: `eval_loss`, with `greater_is_better=False`
|
58 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
59 |
Validation was performed using a separate subset of the dataset. The final training and validation loss are as follows:
|
60 |
|
61 |
<p align="center">
|
62 |
-
<img src="train-val-curve.png" alt="Training and Validation Loss" width="
|
63 |
</p>
|
64 |
|
65 |
## Evaluation Results
|
|
|
56 |
- **Mixed Precision**: FP16
|
57 |
- **Best Model Criteria**: `eval_loss`, with `greater_is_better=False`
|
58 |
|
59 |
+
### LoRA Hyperparameters
|
60 |
+
|
61 |
+
The fine-tuning process also included the following LoRA (Low-Rank Adaptation) configuration:
|
62 |
+
|
63 |
+
- **Rank (r)**: 8
|
64 |
+
- **Alpha**: 16
|
65 |
+
- **Dropout**: 0.05
|
66 |
+
- **Bias**: None
|
67 |
+
- **Task Type**: Causal Language Modeling (CAUSAL_LM)
|
68 |
+
|
69 |
Validation was performed using a separate subset of the dataset. The final training and validation loss are as follows:
|
70 |
|
71 |
<p align="center">
|
72 |
+
<img src="train-val-curve.png" alt="Training and Validation Loss" width="85%"/>
|
73 |
</p>
|
74 |
|
75 |
## Evaluation Results
|