Upload README.md
Browse files
README.md
CHANGED
|
@@ -3,4 +3,29 @@ datasets:
|
|
| 3 |
- GAIR/LIMO
|
| 4 |
base_model:
|
| 5 |
- Qwen/Qwen2.5-Coder-14B-Instruct
|
| 6 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
- GAIR/LIMO
|
| 4 |
base_model:
|
| 5 |
- Qwen/Qwen2.5-Coder-14B-Instruct
|
| 6 |
+
---
|
| 7 |
+
|
| 8 |
+
# Fine-Tuned Model: Qwen/Qwen2.5-Coder-14B-Instruct on GAIR/LIMO
|
| 9 |
+
|
| 10 |
+
## Overview
|
| 11 |
+
|
| 12 |
+
This model is a fine-tuned version of the base model Qwen/Qwen2.5-Coder-14B-Instruct. It was trained on a subset of problems from the GAIR/LIMO dataset, specifically focusing on 611 problems over 2 training epochs.
|
| 13 |
+
|
| 14 |
+
## Training Details
|
| 15 |
+
|
| 16 |
+
- **Base Model**: Qwen/Qwen2.5-Coder-14B-Instruct
|
| 17 |
+
- **Dataset**: GAIR/LIMO (subset of 611 problems)
|
| 18 |
+
- **Epochs**: 2
|
| 19 |
+
- **Training Limitations**: The training was constrained by the computational resources available on my machine, which means I haven't yet conducted a thorough evaluation of the model's performance improvements.
|
| 20 |
+
|
| 21 |
+
## Key Observations
|
| 22 |
+
|
| 23 |
+
During testing, the fine-tuned model demonstrated significant improvements in reasoning ability compared to the base model. It began to provide more coherent and accurate responses, avoiding the mistakes observed in the base model during my initial tests.
|
| 24 |
+
|
| 25 |
+
## Next Steps
|
| 26 |
+
|
| 27 |
+
While preliminary results are promising, further evaluation is needed to assess the overall improvement in model quality. I encourage the community to test the model and share their findings. Your feedback will be invaluable in understanding the extent of the improvements.
|
| 28 |
+
|
| 29 |
+
## Acknowledgments
|
| 30 |
+
|
| 31 |
+
Special thanks to the creators of the Qwen/Qwen2.5-Coder-14B-Instruct and GAIR/LIMO datasets for providing the foundational resources.
|