Add comprehensive model card for Meta-Llama-3-8B-Instruct fine-tuned on xLAM

README.md

@@ -1,3 +1,4 @@
+
 ---
 license: cc-by-nc-4.0
 tags:
@@ -31,7 +32,7 @@ This is a fine-tuned version of the Meta-Llama-3-8B-Instruct model. The model wa
 - **Finetuned from model:** meta-llama/Meta-Llama-3-8B-Instruct
 - **Model size:** Meta-Llama-3-8B-Instruct parameters
 - **Vocab size:** 128,256 tokens
-- **Max sequence length:** 512 tokens
+- **Max sequence length:** 2,048 tokens
 - **Tensor type:** BF16
 - **Pad token:** `<|eot_id|>` (ID: 128009)
 
@@ -47,11 +48,11 @@ The model was fine-tuned using the following configuration:
 
 ### Training Parameters
 - **Learning Rate:** 0.0001
-- **Batch Size:** 8
-- **Gradient Accumulation Steps:** 4
-- **Max Training Steps:** 10
+- **Batch Size:** 16
+- **Gradient Accumulation Steps:** 8
+- **Max Training Steps:** 1,000
 - **Warmup Ratio:** 0.1
-- **Max Sequence Length:** 512
+- **Max Sequence Length:** 2,048
 - **Output Directory:** ./Meta_Llama_3_8B_Instruct_xLAM
 
 ### LoRA Configuration