Update README.md
Browse files
README.md
CHANGED
@@ -22,26 +22,29 @@ base_model:
|
|
22 |
|
23 |
You can use ChatML & Alpaca format.
|
24 |
|
25 |
-
**Overview
|
|
|
26 |
FastLlama is a highly optimized version of the Llama-3.2-1B-Instruct model. Designed for superior performance in constrained environments, it combines speed, compactness, and high accuracy. This version has been fine-tuned using the MetaMathQA-50k section of the HuggingFaceTB/smoltalk dataset to enhance its mathematical reasoning and problem-solving abilities.
|
27 |
|
28 |
-
**Features
|
|
|
29 |
Lightweight and Fast: Optimized to deliver Llama-class capabilities with reduced computational overhead.
|
30 |
Fine-Tuned for Math Reasoning: Utilizes MetaMathQA-50k for better handling of complex mathematical problems and logical reasoning tasks.
|
31 |
Instruction-Tuned: Pre-trained on instruction-following tasks, making it robust in understanding and executing detailed queries.
|
32 |
Versatile Use Cases: Suitable for educational tools, tutoring systems, or any application requiring mathematical reasoning.
|
33 |
|
34 |
-
**Performance Highlights
|
|
|
35 |
Smaller Footprint: The model delivers comparable results to larger counterparts while operating efficiently on smaller hardware.
|
36 |
Enhanced Accuracy: Demonstrates improved performance on mathematical QA benchmarks.
|
37 |
Instruction Adherence: Retains high fidelity in understanding and following user instructions, even for complex queries.
|
38 |
|
39 |
-
**Loading the Model
|
40 |
```py
|
41 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
42 |
|
43 |
# Load the model and tokenizer
|
44 |
-
model_name = "
|
45 |
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
46 |
model = AutoModelForCausalLM.from_pretrained(model_name)
|
47 |
|
@@ -54,7 +57,8 @@ response = tokenizer.decode(outputs[0], skip_special_tokens=True)
|
|
54 |
print(response)
|
55 |
```
|
56 |
|
57 |
-
**Dataset
|
|
|
58 |
Dataset: MetaMathQA-50k
|
59 |
The MetaMathQA-50k subset of HuggingFaceTB/smoltalk was selected for fine-tuning due to its focus on mathematical reasoning, multi-step problem-solving, and logical inference. The dataset includes:
|
60 |
|
@@ -63,7 +67,8 @@ Geometric reasoning tasks
|
|
63 |
Statistical and probabilistic questions
|
64 |
Logical deduction problems
|
65 |
|
66 |
-
**Model Fine-Tuning
|
|
|
67 |
Fine-tuning was conducted using the following configuration:
|
68 |
|
69 |
Learning Rate: 2e-4
|
@@ -71,5 +76,6 @@ Epochs: 1
|
|
71 |
Optimizer: AdamW
|
72 |
Framework: Unsloth
|
73 |
|
74 |
-
**License
|
|
|
75 |
This model is licensed under the Apache 2.0 License. See the LICENSE file for details.
|
|
|
22 |
|
23 |
You can use ChatML & Alpaca format.
|
24 |
|
25 |
+
**Overview:**
|
26 |
+
|
27 |
FastLlama is a highly optimized version of the Llama-3.2-1B-Instruct model. Designed for superior performance in constrained environments, it combines speed, compactness, and high accuracy. This version has been fine-tuned using the MetaMathQA-50k section of the HuggingFaceTB/smoltalk dataset to enhance its mathematical reasoning and problem-solving abilities.
|
28 |
|
29 |
+
**Features:**
|
30 |
+
|
31 |
Lightweight and Fast: Optimized to deliver Llama-class capabilities with reduced computational overhead.
|
32 |
Fine-Tuned for Math Reasoning: Utilizes MetaMathQA-50k for better handling of complex mathematical problems and logical reasoning tasks.
|
33 |
Instruction-Tuned: Pre-trained on instruction-following tasks, making it robust in understanding and executing detailed queries.
|
34 |
Versatile Use Cases: Suitable for educational tools, tutoring systems, or any application requiring mathematical reasoning.
|
35 |
|
36 |
+
**Performance Highlights:**
|
37 |
+
|
38 |
Smaller Footprint: The model delivers comparable results to larger counterparts while operating efficiently on smaller hardware.
|
39 |
Enhanced Accuracy: Demonstrates improved performance on mathematical QA benchmarks.
|
40 |
Instruction Adherence: Retains high fidelity in understanding and following user instructions, even for complex queries.
|
41 |
|
42 |
+
**Loading the Model:**
|
43 |
```py
|
44 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
45 |
|
46 |
# Load the model and tokenizer
|
47 |
+
model_name = "suayptalha/FastLlama-3.2-1B-Instruct"
|
48 |
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
49 |
model = AutoModelForCausalLM.from_pretrained(model_name)
|
50 |
|
|
|
57 |
print(response)
|
58 |
```
|
59 |
|
60 |
+
**Dataset:**
|
61 |
+
|
62 |
Dataset: MetaMathQA-50k
|
63 |
The MetaMathQA-50k subset of HuggingFaceTB/smoltalk was selected for fine-tuning due to its focus on mathematical reasoning, multi-step problem-solving, and logical inference. The dataset includes:
|
64 |
|
|
|
67 |
Statistical and probabilistic questions
|
68 |
Logical deduction problems
|
69 |
|
70 |
+
**Model Fine-Tuning:**
|
71 |
+
|
72 |
Fine-tuning was conducted using the following configuration:
|
73 |
|
74 |
Learning Rate: 2e-4
|
|
|
76 |
Optimizer: AdamW
|
77 |
Framework: Unsloth
|
78 |
|
79 |
+
**License:**
|
80 |
+
|
81 |
This model is licensed under the Apache 2.0 License. See the LICENSE file for details.
|