Upload folder using huggingface_hub

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a44c433d1d2471c754528eab3ff9a4024e4275cb205d77e15461656b50f32278
+size 109069176

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-1298/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-1298/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-1298/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:18b6554ea4ef6bac2efc86df4685882fdd102ee951a06f539e71f1f62df0309a
+size 109069176

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-1298/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d015481056925fc897a30c15ff6112485e76723a4612ab8b52e6f299169afd1e
+size 55532666

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-1298/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fc5c74ab5028964ebe81887d5bab96bc9b52cbefa368c9bbe87daa5d227f840
+size 14244

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-1298/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e7d569e82154ffde0850754b22a8d297db4e4ea0b3a3e5316f520a7417a43812
+size 1064

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-1298/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-1298/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-1298/trainer_state.json

@@ -0,0 +1,952 @@
+{
+  "best_metric": 1.212496042251587,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-649",
+  "epoch": 2.0,
+  "eval_steps": 10,
+  "global_step": 1298,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.015408320493066256,
+      "grad_norm": 0.939333975315094,
+      "learning_rate": 0.0002,
+      "loss": 1.9609,
+      "step": 10
+    },
+    {
+      "epoch": 0.030816640986132512,
+      "grad_norm": 0.9446700215339661,
+      "learning_rate": 0.0002,
+      "loss": 1.5975,
+      "step": 20
+    },
+    {
+      "epoch": 0.046224961479198766,
+      "grad_norm": 0.7268466353416443,
+      "learning_rate": 0.0002,
+      "loss": 1.5196,
+      "step": 30
+    },
+    {
+      "epoch": 0.061633281972265024,
+      "grad_norm": 0.8900066018104553,
+      "learning_rate": 0.0002,
+      "loss": 1.5021,
+      "step": 40
+    },
+    {
+      "epoch": 0.07704160246533127,
+      "grad_norm": 4.246077060699463,
+      "learning_rate": 0.0002,
+      "loss": 1.4155,
+      "step": 50
+    },
+    {
+      "epoch": 0.09244992295839753,
+      "grad_norm": 0.7887561321258545,
+      "learning_rate": 0.0002,
+      "loss": 1.4052,
+      "step": 60
+    },
+    {
+      "epoch": 0.10785824345146379,
+      "grad_norm": 0.6822794675827026,
+      "learning_rate": 0.0002,
+      "loss": 1.215,
+      "step": 70
+    },
+    {
+      "epoch": 0.12326656394453005,
+      "grad_norm": 0.8871720433235168,
+      "learning_rate": 0.0002,
+      "loss": 1.2524,
+      "step": 80
+    },
+    {
+      "epoch": 0.1386748844375963,
+      "grad_norm": 0.6524078249931335,
+      "learning_rate": 0.0002,
+      "loss": 1.191,
+      "step": 90
+    },
+    {
+      "epoch": 0.15408320493066255,
+      "grad_norm": 0.5519863367080688,
+      "learning_rate": 0.0002,
+      "loss": 1.2455,
+      "step": 100
+    },
+    {
+      "epoch": 0.1694915254237288,
+      "grad_norm": 0.6716212630271912,
+      "learning_rate": 0.0002,
+      "loss": 1.2515,
+      "step": 110
+    },
+    {
+      "epoch": 0.18489984591679506,
+      "grad_norm": 1.527921199798584,
+      "learning_rate": 0.0002,
+      "loss": 1.2239,
+      "step": 120
+    },
+    {
+      "epoch": 0.20030816640986132,
+      "grad_norm": 0.6893861293792725,
+      "learning_rate": 0.0002,
+      "loss": 1.2887,
+      "step": 130
+    },
+    {
+      "epoch": 0.21571648690292758,
+      "grad_norm": 0.5514246821403503,
+      "learning_rate": 0.0002,
+      "loss": 1.1722,
+      "step": 140
+    },
+    {
+      "epoch": 0.23112480739599384,
+      "grad_norm": 0.5830065608024597,
+      "learning_rate": 0.0002,
+      "loss": 1.1946,
+      "step": 150
+    },
+    {
+      "epoch": 0.2465331278890601,
+      "grad_norm": 0.7411216497421265,
+      "learning_rate": 0.0002,
+      "loss": 1.3105,
+      "step": 160
+    },
+    {
+      "epoch": 0.26194144838212635,
+      "grad_norm": 0.7470705509185791,
+      "learning_rate": 0.0002,
+      "loss": 1.2008,
+      "step": 170
+    },
+    {
+      "epoch": 0.2773497688751926,
+      "grad_norm": 0.6140725016593933,
+      "learning_rate": 0.0002,
+      "loss": 1.2497,
+      "step": 180
+    },
+    {
+      "epoch": 0.29275808936825887,
+      "grad_norm": 0.7563071250915527,
+      "learning_rate": 0.0002,
+      "loss": 1.1514,
+      "step": 190
+    },
+    {
+      "epoch": 0.3081664098613251,
+      "grad_norm": 0.5491266250610352,
+      "learning_rate": 0.0002,
+      "loss": 1.2703,
+      "step": 200
+    },
+    {
+      "epoch": 0.3235747303543914,
+      "grad_norm": 0.5403220057487488,
+      "learning_rate": 0.0002,
+      "loss": 1.2168,
+      "step": 210
+    },
+    {
+      "epoch": 0.3389830508474576,
+      "grad_norm": 0.9616869688034058,
+      "learning_rate": 0.0002,
+      "loss": 1.2409,
+      "step": 220
+    },
+    {
+      "epoch": 0.3543913713405239,
+      "grad_norm": 0.5448639988899231,
+      "learning_rate": 0.0002,
+      "loss": 1.3112,
+      "step": 230
+    },
+    {
+      "epoch": 0.3697996918335901,
+      "grad_norm": 0.6322043538093567,
+      "learning_rate": 0.0002,
+      "loss": 1.3023,
+      "step": 240
+    },
+    {
+      "epoch": 0.3852080123266564,
+      "grad_norm": 0.5178093314170837,
+      "learning_rate": 0.0002,
+      "loss": 1.2382,
+      "step": 250
+    },
+    {
+      "epoch": 0.40061633281972264,
+      "grad_norm": 0.6274669766426086,
+      "learning_rate": 0.0002,
+      "loss": 1.2755,
+      "step": 260
+    },
+    {
+      "epoch": 0.41602465331278893,
+      "grad_norm": 1.1093109846115112,
+      "learning_rate": 0.0002,
+      "loss": 1.3263,
+      "step": 270
+    },
+    {
+      "epoch": 0.43143297380585516,
+      "grad_norm": 1.8002032041549683,
+      "learning_rate": 0.0002,
+      "loss": 1.2201,
+      "step": 280
+    },
+    {
+      "epoch": 0.44684129429892144,
+      "grad_norm": 0.5977614521980286,
+      "learning_rate": 0.0002,
+      "loss": 1.2335,
+      "step": 290
+    },
+    {
+      "epoch": 0.4622496147919877,
+      "grad_norm": 0.5583769083023071,
+      "learning_rate": 0.0002,
+      "loss": 1.2076,
+      "step": 300
+    },
+    {
+      "epoch": 0.4776579352850539,
+      "grad_norm": 0.5563502311706543,
+      "learning_rate": 0.0002,
+      "loss": 1.197,
+      "step": 310
+    },
+    {
+      "epoch": 0.4930662557781202,
+      "grad_norm": 0.9157887101173401,
+      "learning_rate": 0.0002,
+      "loss": 1.164,
+      "step": 320
+    },
+    {
+      "epoch": 0.5084745762711864,
+      "grad_norm": 0.551705539226532,
+      "learning_rate": 0.0002,
+      "loss": 1.1266,
+      "step": 330
+    },
+    {
+      "epoch": 0.5238828967642527,
+      "grad_norm": 9.011388778686523,
+      "learning_rate": 0.0002,
+      "loss": 1.2801,
+      "step": 340
+    },
+    {
+      "epoch": 0.539291217257319,
+      "grad_norm": 0.6527810096740723,
+      "learning_rate": 0.0002,
+      "loss": 1.2024,
+      "step": 350
+    },
+    {
+      "epoch": 0.5546995377503852,
+      "grad_norm": 1.3121646642684937,
+      "learning_rate": 0.0002,
+      "loss": 1.1424,
+      "step": 360
+    },
+    {
+      "epoch": 0.5701078582434514,
+      "grad_norm": 0.7247968316078186,
+      "learning_rate": 0.0002,
+      "loss": 1.2555,
+      "step": 370
+    },
+    {
+      "epoch": 0.5855161787365177,
+      "grad_norm": 0.5430527925491333,
+      "learning_rate": 0.0002,
+      "loss": 1.196,
+      "step": 380
+    },
+    {
+      "epoch": 0.600924499229584,
+      "grad_norm": 0.6050328612327576,
+      "learning_rate": 0.0002,
+      "loss": 1.1822,
+      "step": 390
+    },
+    {
+      "epoch": 0.6163328197226502,
+      "grad_norm": 0.49382615089416504,
+      "learning_rate": 0.0002,
+      "loss": 1.2754,
+      "step": 400
+    },
+    {
+      "epoch": 0.6317411402157165,
+      "grad_norm": 0.5804041624069214,
+      "learning_rate": 0.0002,
+      "loss": 1.2595,
+      "step": 410
+    },
+    {
+      "epoch": 0.6471494607087828,
+      "grad_norm": 0.524357259273529,
+      "learning_rate": 0.0002,
+      "loss": 1.2926,
+      "step": 420
+    },
+    {
+      "epoch": 0.662557781201849,
+      "grad_norm": 1.8817486763000488,
+      "learning_rate": 0.0002,
+      "loss": 1.1928,
+      "step": 430
+    },
+    {
+      "epoch": 0.6779661016949152,
+      "grad_norm": 0.6104950308799744,
+      "learning_rate": 0.0002,
+      "loss": 1.2103,
+      "step": 440
+    },
+    {
+      "epoch": 0.6933744221879815,
+      "grad_norm": 0.631179928779602,
+      "learning_rate": 0.0002,
+      "loss": 1.2296,
+      "step": 450
+    },
+    {
+      "epoch": 0.7087827426810478,
+      "grad_norm": 0.6562483310699463,
+      "learning_rate": 0.0002,
+      "loss": 1.1843,
+      "step": 460
+    },
+    {
+      "epoch": 0.724191063174114,
+      "grad_norm": 0.578336775302887,
+      "learning_rate": 0.0002,
+      "loss": 1.2424,
+      "step": 470
+    },
+    {
+      "epoch": 0.7395993836671803,
+      "grad_norm": 0.8468874096870422,
+      "learning_rate": 0.0002,
+      "loss": 1.192,
+      "step": 480
+    },
+    {
+      "epoch": 0.7550077041602465,
+      "grad_norm": 0.7162539958953857,
+      "learning_rate": 0.0002,
+      "loss": 1.2913,
+      "step": 490
+    },
+    {
+      "epoch": 0.7704160246533128,
+      "grad_norm": 0.7655543684959412,
+      "learning_rate": 0.0002,
+      "loss": 1.2158,
+      "step": 500
+    },
+    {
+      "epoch": 0.785824345146379,
+      "grad_norm": 0.5711562037467957,
+      "learning_rate": 0.0002,
+      "loss": 1.2637,
+      "step": 510
+    },
+    {
+      "epoch": 0.8012326656394453,
+      "grad_norm": 0.7273485660552979,
+      "learning_rate": 0.0002,
+      "loss": 1.1277,
+      "step": 520
+    },
+    {
+      "epoch": 0.8166409861325116,
+      "grad_norm": 0.47050145268440247,
+      "learning_rate": 0.0002,
+      "loss": 1.188,
+      "step": 530
+    },
+    {
+      "epoch": 0.8320493066255779,
+      "grad_norm": 0.4930959939956665,
+      "learning_rate": 0.0002,
+      "loss": 1.1462,
+      "step": 540
+    },
+    {
+      "epoch": 0.847457627118644,
+      "grad_norm": 0.553955614566803,
+      "learning_rate": 0.0002,
+      "loss": 1.167,
+      "step": 550
+    },
+    {
+      "epoch": 0.8628659476117103,
+      "grad_norm": 0.6495056748390198,
+      "learning_rate": 0.0002,
+      "loss": 1.1831,
+      "step": 560
+    },
+    {
+      "epoch": 0.8782742681047766,
+      "grad_norm": 0.4586578905582428,
+      "learning_rate": 0.0002,
+      "loss": 1.0984,
+      "step": 570
+    },
+    {
+      "epoch": 0.8936825885978429,
+      "grad_norm": 0.4870392680168152,
+      "learning_rate": 0.0002,
+      "loss": 1.2415,
+      "step": 580
+    },
+    {
+      "epoch": 0.9090909090909091,
+      "grad_norm": 0.6337013244628906,
+      "learning_rate": 0.0002,
+      "loss": 1.1735,
+      "step": 590
+    },
+    {
+      "epoch": 0.9244992295839753,
+      "grad_norm": 0.7260186672210693,
+      "learning_rate": 0.0002,
+      "loss": 1.1781,
+      "step": 600
+    },
+    {
+      "epoch": 0.9399075500770416,
+      "grad_norm": 1.4133737087249756,
+      "learning_rate": 0.0002,
+      "loss": 1.2536,
+      "step": 610
+    },
+    {
+      "epoch": 0.9553158705701078,
+      "grad_norm": 0.573525071144104,
+      "learning_rate": 0.0002,
+      "loss": 1.2211,
+      "step": 620
+    },
+    {
+      "epoch": 0.9707241910631741,
+      "grad_norm": 0.8403644561767578,
+      "learning_rate": 0.0002,
+      "loss": 1.11,
+      "step": 630
+    },
+    {
+      "epoch": 0.9861325115562404,
+      "grad_norm": 0.7665438055992126,
+      "learning_rate": 0.0002,
+      "loss": 1.2932,
+      "step": 640
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.212496042251587,
+      "eval_runtime": 120.3264,
+      "eval_samples_per_second": 3.79,
+      "eval_steps_per_second": 0.474,
+      "step": 649
+    },
+    {
+      "epoch": 1.0015408320493067,
+      "grad_norm": 0.6874801516532898,
+      "learning_rate": 0.0002,
+      "loss": 1.176,
+      "step": 650
+    },
+    {
+      "epoch": 1.0169491525423728,
+      "grad_norm": 0.6675221920013428,
+      "learning_rate": 0.0002,
+      "loss": 1.0957,
+      "step": 660
+    },
+    {
+      "epoch": 1.0323574730354392,
+      "grad_norm": 0.6837265491485596,
+      "learning_rate": 0.0002,
+      "loss": 1.1616,
+      "step": 670
+    },
+    {
+      "epoch": 1.0477657935285054,
+      "grad_norm": 0.6796931028366089,
+      "learning_rate": 0.0002,
+      "loss": 1.045,
+      "step": 680
+    },
+    {
+      "epoch": 1.0631741140215716,
+      "grad_norm": 0.6596675515174866,
+      "learning_rate": 0.0002,
+      "loss": 1.0889,
+      "step": 690
+    },
+    {
+      "epoch": 1.078582434514638,
+      "grad_norm": 0.6630653142929077,
+      "learning_rate": 0.0002,
+      "loss": 1.067,
+      "step": 700
+    },
+    {
+      "epoch": 1.0939907550077042,
+      "grad_norm": 0.7065498232841492,
+      "learning_rate": 0.0002,
+      "loss": 1.0222,
+      "step": 710
+    },
+    {
+      "epoch": 1.1093990755007703,
+      "grad_norm": 0.694682240486145,
+      "learning_rate": 0.0002,
+      "loss": 1.081,
+      "step": 720
+    },
+    {
+      "epoch": 1.1248073959938367,
+      "grad_norm": 0.8984217047691345,
+      "learning_rate": 0.0002,
+      "loss": 1.1019,
+      "step": 730
+    },
+    {
+      "epoch": 1.140215716486903,
+      "grad_norm": 1.0035051107406616,
+      "learning_rate": 0.0002,
+      "loss": 1.0506,
+      "step": 740
+    },
+    {
+      "epoch": 1.1556240369799693,
+      "grad_norm": 0.8646948933601379,
+      "learning_rate": 0.0002,
+      "loss": 1.0502,
+      "step": 750
+    },
+    {
+      "epoch": 1.1710323574730355,
+      "grad_norm": 0.6287558674812317,
+      "learning_rate": 0.0002,
+      "loss": 1.0256,
+      "step": 760
+    },
+    {
+      "epoch": 1.1864406779661016,
+      "grad_norm": 0.6975560784339905,
+      "learning_rate": 0.0002,
+      "loss": 1.051,
+      "step": 770
+    },
+    {
+      "epoch": 1.201848998459168,
+      "grad_norm": 0.5972192883491516,
+      "learning_rate": 0.0002,
+      "loss": 1.0409,
+      "step": 780
+    },
+    {
+      "epoch": 1.2172573189522342,
+      "grad_norm": 0.7299932241439819,
+      "learning_rate": 0.0002,
+      "loss": 1.0837,
+      "step": 790
+    },
+    {
+      "epoch": 1.2326656394453004,
+      "grad_norm": 0.8376814126968384,
+      "learning_rate": 0.0002,
+      "loss": 1.1291,
+      "step": 800
+    },
+    {
+      "epoch": 1.2480739599383668,
+      "grad_norm": 1.117690086364746,
+      "learning_rate": 0.0002,
+      "loss": 1.0809,
+      "step": 810
+    },
+    {
+      "epoch": 1.263482280431433,
+      "grad_norm": 0.7850839495658875,
+      "learning_rate": 0.0002,
+      "loss": 1.0762,
+      "step": 820
+    },
+    {
+      "epoch": 1.2788906009244991,
+      "grad_norm": 0.7941734790802002,
+      "learning_rate": 0.0002,
+      "loss": 1.2278,
+      "step": 830
+    },
+    {
+      "epoch": 1.2942989214175655,
+      "grad_norm": 0.5682743787765503,
+      "learning_rate": 0.0002,
+      "loss": 0.9723,
+      "step": 840
+    },
+    {
+      "epoch": 1.3097072419106317,
+      "grad_norm": 0.7182576060295105,
+      "learning_rate": 0.0002,
+      "loss": 1.0262,
+      "step": 850
+    },
+    {
+      "epoch": 1.325115562403698,
+      "grad_norm": 0.7436819672584534,
+      "learning_rate": 0.0002,
+      "loss": 1.0801,
+      "step": 860
+    },
+    {
+      "epoch": 1.3405238828967643,
+      "grad_norm": 0.7665132880210876,
+      "learning_rate": 0.0002,
+      "loss": 1.0705,
+      "step": 870
+    },
+    {
+      "epoch": 1.3559322033898304,
+      "grad_norm": 1.060669183731079,
+      "learning_rate": 0.0002,
+      "loss": 1.1925,
+      "step": 880
+    },
+    {
+      "epoch": 1.3713405238828968,
+      "grad_norm": 0.6262772083282471,
+      "learning_rate": 0.0002,
+      "loss": 1.0581,
+      "step": 890
+    },
+    {
+      "epoch": 1.386748844375963,
+      "grad_norm": 1.010703682899475,
+      "learning_rate": 0.0002,
+      "loss": 1.0934,
+      "step": 900
+    },
+    {
+      "epoch": 1.4021571648690292,
+      "grad_norm": 0.5669811367988586,
+      "learning_rate": 0.0002,
+      "loss": 1.111,
+      "step": 910
+    },
+    {
+      "epoch": 1.4175654853620956,
+      "grad_norm": 0.8644649386405945,
+      "learning_rate": 0.0002,
+      "loss": 1.0638,
+      "step": 920
+    },
+    {
+      "epoch": 1.4329738058551618,
+      "grad_norm": 0.6250067949295044,
+      "learning_rate": 0.0002,
+      "loss": 1.1973,
+      "step": 930
+    },
+    {
+      "epoch": 1.448382126348228,
+      "grad_norm": 0.6507946848869324,
+      "learning_rate": 0.0002,
+      "loss": 1.0592,
+      "step": 940
+    },
+    {
+      "epoch": 1.4637904468412943,
+      "grad_norm": 0.9305022954940796,
+      "learning_rate": 0.0002,
+      "loss": 1.0306,
+      "step": 950
+    },
+    {
+      "epoch": 1.4791987673343605,
+      "grad_norm": 0.86021888256073,
+      "learning_rate": 0.0002,
+      "loss": 0.9792,
+      "step": 960
+    },
+    {
+      "epoch": 1.494607087827427,
+      "grad_norm": 1.1122219562530518,
+      "learning_rate": 0.0002,
+      "loss": 1.0656,
+      "step": 970
+    },
+    {
+      "epoch": 1.510015408320493,
+      "grad_norm": 0.7431066036224365,
+      "learning_rate": 0.0002,
+      "loss": 1.0745,
+      "step": 980
+    },
+    {
+      "epoch": 1.5254237288135593,
+      "grad_norm": 1.022744059562683,
+      "learning_rate": 0.0002,
+      "loss": 1.0822,
+      "step": 990
+    },
+    {
+      "epoch": 1.5408320493066254,
+      "grad_norm": 0.6638363599777222,
+      "learning_rate": 0.0002,
+      "loss": 1.0689,
+      "step": 1000
+    },
+    {
+      "epoch": 1.5562403697996918,
+      "grad_norm": 0.6290505528450012,
+      "learning_rate": 0.0002,
+      "loss": 1.1032,
+      "step": 1010
+    },
+    {
+      "epoch": 1.5716486902927582,
+      "grad_norm": 0.6446037888526917,
+      "learning_rate": 0.0002,
+      "loss": 1.0572,
+      "step": 1020
+    },
+    {
+      "epoch": 1.5870570107858244,
+      "grad_norm": 0.7383618354797363,
+      "learning_rate": 0.0002,
+      "loss": 1.0814,
+      "step": 1030
+    },
+    {
+      "epoch": 1.6024653312788906,
+      "grad_norm": 3.5893094539642334,
+      "learning_rate": 0.0002,
+      "loss": 1.0643,
+      "step": 1040
+    },
+    {
+      "epoch": 1.6178736517719567,
+      "grad_norm": 0.7388538718223572,
+      "learning_rate": 0.0002,
+      "loss": 0.9806,
+      "step": 1050
+    },
+    {
+      "epoch": 1.6332819722650231,
+      "grad_norm": 0.7813242077827454,
+      "learning_rate": 0.0002,
+      "loss": 1.1,
+      "step": 1060
+    },
+    {
+      "epoch": 1.6486902927580893,
+      "grad_norm": 0.6643418669700623,
+      "learning_rate": 0.0002,
+      "loss": 1.0463,
+      "step": 1070
+    },
+    {
+      "epoch": 1.6640986132511557,
+      "grad_norm": 0.7817712426185608,
+      "learning_rate": 0.0002,
+      "loss": 1.0344,
+      "step": 1080
+    },
+    {
+      "epoch": 1.6795069337442219,
+      "grad_norm": 0.8741177916526794,
+      "learning_rate": 0.0002,
+      "loss": 1.0447,
+      "step": 1090
+    },
+    {
+      "epoch": 1.694915254237288,
+      "grad_norm": 0.969813883304596,
+      "learning_rate": 0.0002,
+      "loss": 1.0724,
+      "step": 1100
+    },
+    {
+      "epoch": 1.7103235747303542,
+      "grad_norm": 0.7666333913803101,
+      "learning_rate": 0.0002,
+      "loss": 0.9887,
+      "step": 1110
+    },
+    {
+      "epoch": 1.7257318952234206,
+      "grad_norm": 0.789859414100647,
+      "learning_rate": 0.0002,
+      "loss": 0.9159,
+      "step": 1120
+    },
+    {
+      "epoch": 1.741140215716487,
+      "grad_norm": 0.7018347382545471,
+      "learning_rate": 0.0002,
+      "loss": 1.0679,
+      "step": 1130
+    },
+    {
+      "epoch": 1.7565485362095532,
+      "grad_norm": 0.8899882435798645,
+      "learning_rate": 0.0002,
+      "loss": 1.1429,
+      "step": 1140
+    },
+    {
+      "epoch": 1.7719568567026194,
+      "grad_norm": 0.7738426923751831,
+      "learning_rate": 0.0002,
+      "loss": 0.9214,
+      "step": 1150
+    },
+    {
+      "epoch": 1.7873651771956856,
+      "grad_norm": 1.032209873199463,
+      "learning_rate": 0.0002,
+      "loss": 1.0733,
+      "step": 1160
+    },
+    {
+      "epoch": 1.802773497688752,
+      "grad_norm": 0.7310711741447449,
+      "learning_rate": 0.0002,
+      "loss": 1.0522,
+      "step": 1170
+    },
+    {
+      "epoch": 1.8181818181818183,
+      "grad_norm": 0.8506733775138855,
+      "learning_rate": 0.0002,
+      "loss": 1.0943,
+      "step": 1180
+    },
+    {
+      "epoch": 1.8335901386748845,
+      "grad_norm": 116.00221252441406,
+      "learning_rate": 0.0002,
+      "loss": 1.1349,
+      "step": 1190
+    },
+    {
+      "epoch": 1.8489984591679507,
+      "grad_norm": 0.8819684982299805,
+      "learning_rate": 0.0002,
+      "loss": 1.0595,
+      "step": 1200
+    },
+    {
+      "epoch": 1.8644067796610169,
+      "grad_norm": 0.8846774697303772,
+      "learning_rate": 0.0002,
+      "loss": 0.995,
+      "step": 1210
+    },
+    {
+      "epoch": 1.879815100154083,
+      "grad_norm": 1.6417866945266724,
+      "learning_rate": 0.0002,
+      "loss": 1.116,
+      "step": 1220
+    },
+    {
+      "epoch": 1.8952234206471494,
+      "grad_norm": 1.1875122785568237,
+      "learning_rate": 0.0002,
+      "loss": 1.044,
+      "step": 1230
+    },
+    {
+      "epoch": 1.9106317411402158,
+      "grad_norm": 0.6520240306854248,
+      "learning_rate": 0.0002,
+      "loss": 1.0322,
+      "step": 1240
+    },
+    {
+      "epoch": 1.926040061633282,
+      "grad_norm": 0.9059738516807556,
+      "learning_rate": 0.0002,
+      "loss": 1.0175,
+      "step": 1250
+    },
+    {
+      "epoch": 1.9414483821263482,
+      "grad_norm": 0.7950983643531799,
+      "learning_rate": 0.0002,
+      "loss": 1.1097,
+      "step": 1260
+    },
+    {
+      "epoch": 1.9568567026194144,
+      "grad_norm": 0.6911653280258179,
+      "learning_rate": 0.0002,
+      "loss": 1.0313,
+      "step": 1270
+    },
+    {
+      "epoch": 1.9722650231124808,
+      "grad_norm": 0.9487978219985962,
+      "learning_rate": 0.0002,
+      "loss": 1.0762,
+      "step": 1280
+    },
+    {
+      "epoch": 1.9876733436055471,
+      "grad_norm": 0.716439425945282,
+      "learning_rate": 0.0002,
+      "loss": 1.1608,
+      "step": 1290
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.215390682220459,
+      "eval_runtime": 120.3893,
+      "eval_samples_per_second": 3.788,
+      "eval_steps_per_second": 0.473,
+      "step": 1298
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 5192,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.69493952659456e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-1298/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7da6f114429a9a993fcd6ed877a996e6a07ac3f427895c4f50ec6990a9e85e69
+size 5560

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-1947/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-1947/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-1947/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ede62a76c14ef2734653f04a750f957a6502bcbd37a3e9a3def7b7feabd17df8
+size 109069176

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-1947/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:057d41dd8028c5a7d0a1e2154619d82b686b9ba9a88e1b7dec00d79eeb369fe8
+size 55532666

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-1947/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c004990006f909f350874d51b5b3526213b505cbff24cc2a360d1ec7328a44d7
+size 14244

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-1947/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84c8aac6917b5c47a13dcb9e0693d6d51edc67202804dc7ba98b606799a8bfab
+size 1064

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-1947/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-1947/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-1947/trainer_state.json

@@ -0,0 +1,1415 @@
+{
+  "best_metric": 1.212496042251587,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-649",
+  "epoch": 3.0,
+  "eval_steps": 10,
+  "global_step": 1947,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.015408320493066256,
+      "grad_norm": 0.939333975315094,
+      "learning_rate": 0.0002,
+      "loss": 1.9609,
+      "step": 10
+    },
+    {
+      "epoch": 0.030816640986132512,
+      "grad_norm": 0.9446700215339661,
+      "learning_rate": 0.0002,
+      "loss": 1.5975,
+      "step": 20
+    },
+    {
+      "epoch": 0.046224961479198766,
+      "grad_norm": 0.7268466353416443,
+      "learning_rate": 0.0002,
+      "loss": 1.5196,
+      "step": 30
+    },
+    {
+      "epoch": 0.061633281972265024,
+      "grad_norm": 0.8900066018104553,
+      "learning_rate": 0.0002,
+      "loss": 1.5021,
+      "step": 40
+    },
+    {
+      "epoch": 0.07704160246533127,
+      "grad_norm": 4.246077060699463,
+      "learning_rate": 0.0002,
+      "loss": 1.4155,
+      "step": 50
+    },
+    {
+      "epoch": 0.09244992295839753,
+      "grad_norm": 0.7887561321258545,
+      "learning_rate": 0.0002,
+      "loss": 1.4052,
+      "step": 60
+    },
+    {
+      "epoch": 0.10785824345146379,
+      "grad_norm": 0.6822794675827026,
+      "learning_rate": 0.0002,
+      "loss": 1.215,
+      "step": 70
+    },
+    {
+      "epoch": 0.12326656394453005,
+      "grad_norm": 0.8871720433235168,
+      "learning_rate": 0.0002,
+      "loss": 1.2524,
+      "step": 80
+    },
+    {
+      "epoch": 0.1386748844375963,
+      "grad_norm": 0.6524078249931335,
+      "learning_rate": 0.0002,
+      "loss": 1.191,
+      "step": 90
+    },
+    {
+      "epoch": 0.15408320493066255,
+      "grad_norm": 0.5519863367080688,
+      "learning_rate": 0.0002,
+      "loss": 1.2455,
+      "step": 100
+    },
+    {
+      "epoch": 0.1694915254237288,
+      "grad_norm": 0.6716212630271912,
+      "learning_rate": 0.0002,
+      "loss": 1.2515,
+      "step": 110
+    },
+    {
+      "epoch": 0.18489984591679506,
+      "grad_norm": 1.527921199798584,
+      "learning_rate": 0.0002,
+      "loss": 1.2239,
+      "step": 120
+    },
+    {
+      "epoch": 0.20030816640986132,
+      "grad_norm": 0.6893861293792725,
+      "learning_rate": 0.0002,
+      "loss": 1.2887,
+      "step": 130
+    },
+    {
+      "epoch": 0.21571648690292758,
+      "grad_norm": 0.5514246821403503,
+      "learning_rate": 0.0002,
+      "loss": 1.1722,
+      "step": 140
+    },
+    {
+      "epoch": 0.23112480739599384,
+      "grad_norm": 0.5830065608024597,
+      "learning_rate": 0.0002,
+      "loss": 1.1946,
+      "step": 150
+    },
+    {
+      "epoch": 0.2465331278890601,
+      "grad_norm": 0.7411216497421265,
+      "learning_rate": 0.0002,
+      "loss": 1.3105,
+      "step": 160
+    },
+    {
+      "epoch": 0.26194144838212635,
+      "grad_norm": 0.7470705509185791,
+      "learning_rate": 0.0002,
+      "loss": 1.2008,
+      "step": 170
+    },
+    {
+      "epoch": 0.2773497688751926,
+      "grad_norm": 0.6140725016593933,
+      "learning_rate": 0.0002,
+      "loss": 1.2497,
+      "step": 180
+    },
+    {
+      "epoch": 0.29275808936825887,
+      "grad_norm": 0.7563071250915527,
+      "learning_rate": 0.0002,
+      "loss": 1.1514,
+      "step": 190
+    },
+    {
+      "epoch": 0.3081664098613251,
+      "grad_norm": 0.5491266250610352,
+      "learning_rate": 0.0002,
+      "loss": 1.2703,
+      "step": 200
+    },
+    {
+      "epoch": 0.3235747303543914,
+      "grad_norm": 0.5403220057487488,
+      "learning_rate": 0.0002,
+      "loss": 1.2168,
+      "step": 210
+    },
+    {
+      "epoch": 0.3389830508474576,
+      "grad_norm": 0.9616869688034058,
+      "learning_rate": 0.0002,
+      "loss": 1.2409,
+      "step": 220
+    },
+    {
+      "epoch": 0.3543913713405239,
+      "grad_norm": 0.5448639988899231,
+      "learning_rate": 0.0002,
+      "loss": 1.3112,
+      "step": 230
+    },
+    {
+      "epoch": 0.3697996918335901,
+      "grad_norm": 0.6322043538093567,
+      "learning_rate": 0.0002,
+      "loss": 1.3023,
+      "step": 240
+    },
+    {
+      "epoch": 0.3852080123266564,
+      "grad_norm": 0.5178093314170837,
+      "learning_rate": 0.0002,
+      "loss": 1.2382,
+      "step": 250
+    },
+    {
+      "epoch": 0.40061633281972264,
+      "grad_norm": 0.6274669766426086,
+      "learning_rate": 0.0002,
+      "loss": 1.2755,
+      "step": 260
+    },
+    {
+      "epoch": 0.41602465331278893,
+      "grad_norm": 1.1093109846115112,
+      "learning_rate": 0.0002,
+      "loss": 1.3263,
+      "step": 270
+    },
+    {
+      "epoch": 0.43143297380585516,
+      "grad_norm": 1.8002032041549683,
+      "learning_rate": 0.0002,
+      "loss": 1.2201,
+      "step": 280
+    },
+    {
+      "epoch": 0.44684129429892144,
+      "grad_norm": 0.5977614521980286,
+      "learning_rate": 0.0002,
+      "loss": 1.2335,
+      "step": 290
+    },
+    {
+      "epoch": 0.4622496147919877,
+      "grad_norm": 0.5583769083023071,
+      "learning_rate": 0.0002,
+      "loss": 1.2076,
+      "step": 300
+    },
+    {
+      "epoch": 0.4776579352850539,
+      "grad_norm": 0.5563502311706543,
+      "learning_rate": 0.0002,
+      "loss": 1.197,
+      "step": 310
+    },
+    {
+      "epoch": 0.4930662557781202,
+      "grad_norm": 0.9157887101173401,
+      "learning_rate": 0.0002,
+      "loss": 1.164,
+      "step": 320
+    },
+    {
+      "epoch": 0.5084745762711864,
+      "grad_norm": 0.551705539226532,
+      "learning_rate": 0.0002,
+      "loss": 1.1266,
+      "step": 330
+    },
+    {
+      "epoch": 0.5238828967642527,
+      "grad_norm": 9.011388778686523,
+      "learning_rate": 0.0002,
+      "loss": 1.2801,
+      "step": 340
+    },
+    {
+      "epoch": 0.539291217257319,
+      "grad_norm": 0.6527810096740723,
+      "learning_rate": 0.0002,
+      "loss": 1.2024,
+      "step": 350
+    },
+    {
+      "epoch": 0.5546995377503852,
+      "grad_norm": 1.3121646642684937,
+      "learning_rate": 0.0002,
+      "loss": 1.1424,
+      "step": 360
+    },
+    {
+      "epoch": 0.5701078582434514,
+      "grad_norm": 0.7247968316078186,
+      "learning_rate": 0.0002,
+      "loss": 1.2555,
+      "step": 370
+    },
+    {
+      "epoch": 0.5855161787365177,
+      "grad_norm": 0.5430527925491333,
+      "learning_rate": 0.0002,
+      "loss": 1.196,
+      "step": 380
+    },
+    {
+      "epoch": 0.600924499229584,
+      "grad_norm": 0.6050328612327576,
+      "learning_rate": 0.0002,
+      "loss": 1.1822,
+      "step": 390
+    },
+    {
+      "epoch": 0.6163328197226502,
+      "grad_norm": 0.49382615089416504,
+      "learning_rate": 0.0002,
+      "loss": 1.2754,
+      "step": 400
+    },
+    {
+      "epoch": 0.6317411402157165,
+      "grad_norm": 0.5804041624069214,
+      "learning_rate": 0.0002,
+      "loss": 1.2595,
+      "step": 410
+    },
+    {
+      "epoch": 0.6471494607087828,
+      "grad_norm": 0.524357259273529,
+      "learning_rate": 0.0002,
+      "loss": 1.2926,
+      "step": 420
+    },
+    {
+      "epoch": 0.662557781201849,
+      "grad_norm": 1.8817486763000488,
+      "learning_rate": 0.0002,
+      "loss": 1.1928,
+      "step": 430
+    },
+    {
+      "epoch": 0.6779661016949152,
+      "grad_norm": 0.6104950308799744,
+      "learning_rate": 0.0002,
+      "loss": 1.2103,
+      "step": 440
+    },
+    {
+      "epoch": 0.6933744221879815,
+      "grad_norm": 0.631179928779602,
+      "learning_rate": 0.0002,
+      "loss": 1.2296,
+      "step": 450
+    },
+    {
+      "epoch": 0.7087827426810478,
+      "grad_norm": 0.6562483310699463,
+      "learning_rate": 0.0002,
+      "loss": 1.1843,
+      "step": 460
+    },
+    {
+      "epoch": 0.724191063174114,
+      "grad_norm": 0.578336775302887,
+      "learning_rate": 0.0002,
+      "loss": 1.2424,
+      "step": 470
+    },
+    {
+      "epoch": 0.7395993836671803,
+      "grad_norm": 0.8468874096870422,
+      "learning_rate": 0.0002,
+      "loss": 1.192,
+      "step": 480
+    },
+    {
+      "epoch": 0.7550077041602465,
+      "grad_norm": 0.7162539958953857,
+      "learning_rate": 0.0002,
+      "loss": 1.2913,
+      "step": 490
+    },
+    {
+      "epoch": 0.7704160246533128,
+      "grad_norm": 0.7655543684959412,
+      "learning_rate": 0.0002,
+      "loss": 1.2158,
+      "step": 500
+    },
+    {
+      "epoch": 0.785824345146379,
+      "grad_norm": 0.5711562037467957,
+      "learning_rate": 0.0002,
+      "loss": 1.2637,
+      "step": 510
+    },
+    {
+      "epoch": 0.8012326656394453,
+      "grad_norm": 0.7273485660552979,
+      "learning_rate": 0.0002,
+      "loss": 1.1277,
+      "step": 520
+    },
+    {
+      "epoch": 0.8166409861325116,
+      "grad_norm": 0.47050145268440247,
+      "learning_rate": 0.0002,
+      "loss": 1.188,
+      "step": 530
+    },
+    {
+      "epoch": 0.8320493066255779,
+      "grad_norm": 0.4930959939956665,
+      "learning_rate": 0.0002,
+      "loss": 1.1462,
+      "step": 540
+    },
+    {
+      "epoch": 0.847457627118644,
+      "grad_norm": 0.553955614566803,
+      "learning_rate": 0.0002,
+      "loss": 1.167,
+      "step": 550
+    },
+    {
+      "epoch": 0.8628659476117103,
+      "grad_norm": 0.6495056748390198,
+      "learning_rate": 0.0002,
+      "loss": 1.1831,
+      "step": 560
+    },
+    {
+      "epoch": 0.8782742681047766,
+      "grad_norm": 0.4586578905582428,
+      "learning_rate": 0.0002,
+      "loss": 1.0984,
+      "step": 570
+    },
+    {
+      "epoch": 0.8936825885978429,
+      "grad_norm": 0.4870392680168152,
+      "learning_rate": 0.0002,
+      "loss": 1.2415,
+      "step": 580
+    },
+    {
+      "epoch": 0.9090909090909091,
+      "grad_norm": 0.6337013244628906,
+      "learning_rate": 0.0002,
+      "loss": 1.1735,
+      "step": 590
+    },
+    {
+      "epoch": 0.9244992295839753,
+      "grad_norm": 0.7260186672210693,
+      "learning_rate": 0.0002,
+      "loss": 1.1781,
+      "step": 600
+    },
+    {
+      "epoch": 0.9399075500770416,
+      "grad_norm": 1.4133737087249756,
+      "learning_rate": 0.0002,
+      "loss": 1.2536,
+      "step": 610
+    },
+    {
+      "epoch": 0.9553158705701078,
+      "grad_norm": 0.573525071144104,
+      "learning_rate": 0.0002,
+      "loss": 1.2211,
+      "step": 620
+    },
+    {
+      "epoch": 0.9707241910631741,
+      "grad_norm": 0.8403644561767578,
+      "learning_rate": 0.0002,
+      "loss": 1.11,
+      "step": 630
+    },
+    {
+      "epoch": 0.9861325115562404,
+      "grad_norm": 0.7665438055992126,
+      "learning_rate": 0.0002,
+      "loss": 1.2932,
+      "step": 640
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.212496042251587,
+      "eval_runtime": 120.3264,
+      "eval_samples_per_second": 3.79,
+      "eval_steps_per_second": 0.474,
+      "step": 649
+    },
+    {
+      "epoch": 1.0015408320493067,
+      "grad_norm": 0.6874801516532898,
+      "learning_rate": 0.0002,
+      "loss": 1.176,
+      "step": 650
+    },
+    {
+      "epoch": 1.0169491525423728,
+      "grad_norm": 0.6675221920013428,
+      "learning_rate": 0.0002,
+      "loss": 1.0957,
+      "step": 660
+    },
+    {
+      "epoch": 1.0323574730354392,
+      "grad_norm": 0.6837265491485596,
+      "learning_rate": 0.0002,
+      "loss": 1.1616,
+      "step": 670
+    },
+    {
+      "epoch": 1.0477657935285054,
+      "grad_norm": 0.6796931028366089,
+      "learning_rate": 0.0002,
+      "loss": 1.045,
+      "step": 680
+    },
+    {
+      "epoch": 1.0631741140215716,
+      "grad_norm": 0.6596675515174866,
+      "learning_rate": 0.0002,
+      "loss": 1.0889,
+      "step": 690
+    },
+    {
+      "epoch": 1.078582434514638,
+      "grad_norm": 0.6630653142929077,
+      "learning_rate": 0.0002,
+      "loss": 1.067,
+      "step": 700
+    },
+    {
+      "epoch": 1.0939907550077042,
+      "grad_norm": 0.7065498232841492,
+      "learning_rate": 0.0002,
+      "loss": 1.0222,
+      "step": 710
+    },
+    {
+      "epoch": 1.1093990755007703,
+      "grad_norm": 0.694682240486145,
+      "learning_rate": 0.0002,
+      "loss": 1.081,
+      "step": 720
+    },
+    {
+      "epoch": 1.1248073959938367,
+      "grad_norm": 0.8984217047691345,
+      "learning_rate": 0.0002,
+      "loss": 1.1019,
+      "step": 730
+    },
+    {
+      "epoch": 1.140215716486903,
+      "grad_norm": 1.0035051107406616,
+      "learning_rate": 0.0002,
+      "loss": 1.0506,
+      "step": 740
+    },
+    {
+      "epoch": 1.1556240369799693,
+      "grad_norm": 0.8646948933601379,
+      "learning_rate": 0.0002,
+      "loss": 1.0502,
+      "step": 750
+    },
+    {
+      "epoch": 1.1710323574730355,
+      "grad_norm": 0.6287558674812317,
+      "learning_rate": 0.0002,
+      "loss": 1.0256,
+      "step": 760
+    },
+    {
+      "epoch": 1.1864406779661016,
+      "grad_norm": 0.6975560784339905,
+      "learning_rate": 0.0002,
+      "loss": 1.051,
+      "step": 770
+    },
+    {
+      "epoch": 1.201848998459168,
+      "grad_norm": 0.5972192883491516,
+      "learning_rate": 0.0002,
+      "loss": 1.0409,
+      "step": 780
+    },
+    {
+      "epoch": 1.2172573189522342,
+      "grad_norm": 0.7299932241439819,
+      "learning_rate": 0.0002,
+      "loss": 1.0837,
+      "step": 790
+    },
+    {
+      "epoch": 1.2326656394453004,
+      "grad_norm": 0.8376814126968384,
+      "learning_rate": 0.0002,
+      "loss": 1.1291,
+      "step": 800
+    },
+    {
+      "epoch": 1.2480739599383668,
+      "grad_norm": 1.117690086364746,
+      "learning_rate": 0.0002,
+      "loss": 1.0809,
+      "step": 810
+    },
+    {
+      "epoch": 1.263482280431433,
+      "grad_norm": 0.7850839495658875,
+      "learning_rate": 0.0002,
+      "loss": 1.0762,
+      "step": 820
+    },
+    {
+      "epoch": 1.2788906009244991,
+      "grad_norm": 0.7941734790802002,
+      "learning_rate": 0.0002,
+      "loss": 1.2278,
+      "step": 830
+    },
+    {
+      "epoch": 1.2942989214175655,
+      "grad_norm": 0.5682743787765503,
+      "learning_rate": 0.0002,
+      "loss": 0.9723,
+      "step": 840
+    },
+    {
+      "epoch": 1.3097072419106317,
+      "grad_norm": 0.7182576060295105,
+      "learning_rate": 0.0002,
+      "loss": 1.0262,
+      "step": 850
+    },
+    {
+      "epoch": 1.325115562403698,
+      "grad_norm": 0.7436819672584534,
+      "learning_rate": 0.0002,
+      "loss": 1.0801,
+      "step": 860
+    },
+    {
+      "epoch": 1.3405238828967643,
+      "grad_norm": 0.7665132880210876,
+      "learning_rate": 0.0002,
+      "loss": 1.0705,
+      "step": 870
+    },
+    {
+      "epoch": 1.3559322033898304,
+      "grad_norm": 1.060669183731079,
+      "learning_rate": 0.0002,
+      "loss": 1.1925,
+      "step": 880
+    },
+    {
+      "epoch": 1.3713405238828968,
+      "grad_norm": 0.6262772083282471,
+      "learning_rate": 0.0002,
+      "loss": 1.0581,
+      "step": 890
+    },
+    {
+      "epoch": 1.386748844375963,
+      "grad_norm": 1.010703682899475,
+      "learning_rate": 0.0002,
+      "loss": 1.0934,
+      "step": 900
+    },
+    {
+      "epoch": 1.4021571648690292,
+      "grad_norm": 0.5669811367988586,
+      "learning_rate": 0.0002,
+      "loss": 1.111,
+      "step": 910
+    },
+    {
+      "epoch": 1.4175654853620956,
+      "grad_norm": 0.8644649386405945,
+      "learning_rate": 0.0002,
+      "loss": 1.0638,
+      "step": 920
+    },
+    {
+      "epoch": 1.4329738058551618,
+      "grad_norm": 0.6250067949295044,
+      "learning_rate": 0.0002,
+      "loss": 1.1973,
+      "step": 930
+    },
+    {
+      "epoch": 1.448382126348228,
+      "grad_norm": 0.6507946848869324,
+      "learning_rate": 0.0002,
+      "loss": 1.0592,
+      "step": 940
+    },
+    {
+      "epoch": 1.4637904468412943,
+      "grad_norm": 0.9305022954940796,
+      "learning_rate": 0.0002,
+      "loss": 1.0306,
+      "step": 950
+    },
+    {
+      "epoch": 1.4791987673343605,
+      "grad_norm": 0.86021888256073,
+      "learning_rate": 0.0002,
+      "loss": 0.9792,
+      "step": 960
+    },
+    {
+      "epoch": 1.494607087827427,
+      "grad_norm": 1.1122219562530518,
+      "learning_rate": 0.0002,
+      "loss": 1.0656,
+      "step": 970
+    },
+    {
+      "epoch": 1.510015408320493,
+      "grad_norm": 0.7431066036224365,
+      "learning_rate": 0.0002,
+      "loss": 1.0745,
+      "step": 980
+    },
+    {
+      "epoch": 1.5254237288135593,
+      "grad_norm": 1.022744059562683,
+      "learning_rate": 0.0002,
+      "loss": 1.0822,
+      "step": 990
+    },
+    {
+      "epoch": 1.5408320493066254,
+      "grad_norm": 0.6638363599777222,
+      "learning_rate": 0.0002,
+      "loss": 1.0689,
+      "step": 1000
+    },
+    {
+      "epoch": 1.5562403697996918,
+      "grad_norm": 0.6290505528450012,
+      "learning_rate": 0.0002,
+      "loss": 1.1032,
+      "step": 1010
+    },
+    {
+      "epoch": 1.5716486902927582,
+      "grad_norm": 0.6446037888526917,
+      "learning_rate": 0.0002,
+      "loss": 1.0572,
+      "step": 1020
+    },
+    {
+      "epoch": 1.5870570107858244,
+      "grad_norm": 0.7383618354797363,
+      "learning_rate": 0.0002,
+      "loss": 1.0814,
+      "step": 1030
+    },
+    {
+      "epoch": 1.6024653312788906,
+      "grad_norm": 3.5893094539642334,
+      "learning_rate": 0.0002,
+      "loss": 1.0643,
+      "step": 1040
+    },
+    {
+      "epoch": 1.6178736517719567,
+      "grad_norm": 0.7388538718223572,
+      "learning_rate": 0.0002,
+      "loss": 0.9806,
+      "step": 1050
+    },
+    {
+      "epoch": 1.6332819722650231,
+      "grad_norm": 0.7813242077827454,
+      "learning_rate": 0.0002,
+      "loss": 1.1,
+      "step": 1060
+    },
+    {
+      "epoch": 1.6486902927580893,
+      "grad_norm": 0.6643418669700623,
+      "learning_rate": 0.0002,
+      "loss": 1.0463,
+      "step": 1070
+    },
+    {
+      "epoch": 1.6640986132511557,
+      "grad_norm": 0.7817712426185608,
+      "learning_rate": 0.0002,
+      "loss": 1.0344,
+      "step": 1080
+    },
+    {
+      "epoch": 1.6795069337442219,
+      "grad_norm": 0.8741177916526794,
+      "learning_rate": 0.0002,
+      "loss": 1.0447,
+      "step": 1090
+    },
+    {
+      "epoch": 1.694915254237288,
+      "grad_norm": 0.969813883304596,
+      "learning_rate": 0.0002,
+      "loss": 1.0724,
+      "step": 1100
+    },
+    {
+      "epoch": 1.7103235747303542,
+      "grad_norm": 0.7666333913803101,
+      "learning_rate": 0.0002,
+      "loss": 0.9887,
+      "step": 1110
+    },
+    {
+      "epoch": 1.7257318952234206,
+      "grad_norm": 0.789859414100647,
+      "learning_rate": 0.0002,
+      "loss": 0.9159,
+      "step": 1120
+    },
+    {
+      "epoch": 1.741140215716487,
+      "grad_norm": 0.7018347382545471,
+      "learning_rate": 0.0002,
+      "loss": 1.0679,
+      "step": 1130
+    },
+    {
+      "epoch": 1.7565485362095532,
+      "grad_norm": 0.8899882435798645,
+      "learning_rate": 0.0002,
+      "loss": 1.1429,
+      "step": 1140
+    },
+    {
+      "epoch": 1.7719568567026194,
+      "grad_norm": 0.7738426923751831,
+      "learning_rate": 0.0002,
+      "loss": 0.9214,
+      "step": 1150
+    },
+    {
+      "epoch": 1.7873651771956856,
+      "grad_norm": 1.032209873199463,
+      "learning_rate": 0.0002,
+      "loss": 1.0733,
+      "step": 1160
+    },
+    {
+      "epoch": 1.802773497688752,
+      "grad_norm": 0.7310711741447449,
+      "learning_rate": 0.0002,
+      "loss": 1.0522,
+      "step": 1170
+    },
+    {
+      "epoch": 1.8181818181818183,
+      "grad_norm": 0.8506733775138855,
+      "learning_rate": 0.0002,
+      "loss": 1.0943,
+      "step": 1180
+    },
+    {
+      "epoch": 1.8335901386748845,
+      "grad_norm": 116.00221252441406,
+      "learning_rate": 0.0002,
+      "loss": 1.1349,
+      "step": 1190
+    },
+    {
+      "epoch": 1.8489984591679507,
+      "grad_norm": 0.8819684982299805,
+      "learning_rate": 0.0002,
+      "loss": 1.0595,
+      "step": 1200
+    },
+    {
+      "epoch": 1.8644067796610169,
+      "grad_norm": 0.8846774697303772,
+      "learning_rate": 0.0002,
+      "loss": 0.995,
+      "step": 1210
+    },
+    {
+      "epoch": 1.879815100154083,
+      "grad_norm": 1.6417866945266724,
+      "learning_rate": 0.0002,
+      "loss": 1.116,
+      "step": 1220
+    },
+    {
+      "epoch": 1.8952234206471494,
+      "grad_norm": 1.1875122785568237,
+      "learning_rate": 0.0002,
+      "loss": 1.044,
+      "step": 1230
+    },
+    {
+      "epoch": 1.9106317411402158,
+      "grad_norm": 0.6520240306854248,
+      "learning_rate": 0.0002,
+      "loss": 1.0322,
+      "step": 1240
+    },
+    {
+      "epoch": 1.926040061633282,
+      "grad_norm": 0.9059738516807556,
+      "learning_rate": 0.0002,
+      "loss": 1.0175,
+      "step": 1250
+    },
+    {
+      "epoch": 1.9414483821263482,
+      "grad_norm": 0.7950983643531799,
+      "learning_rate": 0.0002,
+      "loss": 1.1097,
+      "step": 1260
+    },
+    {
+      "epoch": 1.9568567026194144,
+      "grad_norm": 0.6911653280258179,
+      "learning_rate": 0.0002,
+      "loss": 1.0313,
+      "step": 1270
+    },
+    {
+      "epoch": 1.9722650231124808,
+      "grad_norm": 0.9487978219985962,
+      "learning_rate": 0.0002,
+      "loss": 1.0762,
+      "step": 1280
+    },
+    {
+      "epoch": 1.9876733436055471,
+      "grad_norm": 0.716439425945282,
+      "learning_rate": 0.0002,
+      "loss": 1.1608,
+      "step": 1290
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.215390682220459,
+      "eval_runtime": 120.3893,
+      "eval_samples_per_second": 3.788,
+      "eval_steps_per_second": 0.473,
+      "step": 1298
+    },
+    {
+      "epoch": 2.0030816640986133,
+      "grad_norm": 0.7292942404747009,
+      "learning_rate": 0.0002,
+      "loss": 1.0205,
+      "step": 1300
+    },
+    {
+      "epoch": 2.0184899845916795,
+      "grad_norm": 0.8386905789375305,
+      "learning_rate": 0.0002,
+      "loss": 0.8264,
+      "step": 1310
+    },
+    {
+      "epoch": 2.0338983050847457,
+      "grad_norm": 1.114105224609375,
+      "learning_rate": 0.0002,
+      "loss": 0.7925,
+      "step": 1320
+    },
+    {
+      "epoch": 2.049306625577812,
+      "grad_norm": 1.068211317062378,
+      "learning_rate": 0.0002,
+      "loss": 0.8673,
+      "step": 1330
+    },
+    {
+      "epoch": 2.0647149460708785,
+      "grad_norm": 0.8620569109916687,
+      "learning_rate": 0.0002,
+      "loss": 0.8883,
+      "step": 1340
+    },
+    {
+      "epoch": 2.0801232665639446,
+      "grad_norm": 0.8984315991401672,
+      "learning_rate": 0.0002,
+      "loss": 0.8949,
+      "step": 1350
+    },
+    {
+      "epoch": 2.095531587057011,
+      "grad_norm": 1.0469365119934082,
+      "learning_rate": 0.0002,
+      "loss": 0.8992,
+      "step": 1360
+    },
+    {
+      "epoch": 2.110939907550077,
+      "grad_norm": 0.9392943978309631,
+      "learning_rate": 0.0002,
+      "loss": 0.8053,
+      "step": 1370
+    },
+    {
+      "epoch": 2.126348228043143,
+      "grad_norm": 1.2067331075668335,
+      "learning_rate": 0.0002,
+      "loss": 0.8791,
+      "step": 1380
+    },
+    {
+      "epoch": 2.1417565485362093,
+      "grad_norm": 0.8019351363182068,
+      "learning_rate": 0.0002,
+      "loss": 0.7925,
+      "step": 1390
+    },
+    {
+      "epoch": 2.157164869029276,
+      "grad_norm": 0.9445130228996277,
+      "learning_rate": 0.0002,
+      "loss": 0.8669,
+      "step": 1400
+    },
+    {
+      "epoch": 2.172573189522342,
+      "grad_norm": 0.9806232452392578,
+      "learning_rate": 0.0002,
+      "loss": 0.8293,
+      "step": 1410
+    },
+    {
+      "epoch": 2.1879815100154083,
+      "grad_norm": 1.2135679721832275,
+      "learning_rate": 0.0002,
+      "loss": 0.8499,
+      "step": 1420
+    },
+    {
+      "epoch": 2.2033898305084745,
+      "grad_norm": 1.1036803722381592,
+      "learning_rate": 0.0002,
+      "loss": 0.8908,
+      "step": 1430
+    },
+    {
+      "epoch": 2.2187981510015407,
+      "grad_norm": 0.9235773086547852,
+      "learning_rate": 0.0002,
+      "loss": 0.9725,
+      "step": 1440
+    },
+    {
+      "epoch": 2.2342064714946073,
+      "grad_norm": 1.0520254373550415,
+      "learning_rate": 0.0002,
+      "loss": 0.8176,
+      "step": 1450
+    },
+    {
+      "epoch": 2.2496147919876734,
+      "grad_norm": 1.0121252536773682,
+      "learning_rate": 0.0002,
+      "loss": 0.9138,
+      "step": 1460
+    },
+    {
+      "epoch": 2.2650231124807396,
+      "grad_norm": 1.0718834400177002,
+      "learning_rate": 0.0002,
+      "loss": 0.8722,
+      "step": 1470
+    },
+    {
+      "epoch": 2.280431432973806,
+      "grad_norm": 1.1842551231384277,
+      "learning_rate": 0.0002,
+      "loss": 0.8295,
+      "step": 1480
+    },
+    {
+      "epoch": 2.295839753466872,
+      "grad_norm": 1.2985937595367432,
+      "learning_rate": 0.0002,
+      "loss": 0.8706,
+      "step": 1490
+    },
+    {
+      "epoch": 2.3112480739599386,
+      "grad_norm": 2.0254523754119873,
+      "learning_rate": 0.0002,
+      "loss": 0.873,
+      "step": 1500
+    },
+    {
+      "epoch": 2.3266563944530048,
+      "grad_norm": 1.1501885652542114,
+      "learning_rate": 0.0002,
+      "loss": 0.8569,
+      "step": 1510
+    },
+    {
+      "epoch": 2.342064714946071,
+      "grad_norm": 1.1277996301651,
+      "learning_rate": 0.0002,
+      "loss": 0.9394,
+      "step": 1520
+    },
+    {
+      "epoch": 2.357473035439137,
+      "grad_norm": 0.9330579042434692,
+      "learning_rate": 0.0002,
+      "loss": 0.8856,
+      "step": 1530
+    },
+    {
+      "epoch": 2.3728813559322033,
+      "grad_norm": 1.3631645441055298,
+      "learning_rate": 0.0002,
+      "loss": 0.8679,
+      "step": 1540
+    },
+    {
+      "epoch": 2.3882896764252695,
+      "grad_norm": 1.070383071899414,
+      "learning_rate": 0.0002,
+      "loss": 0.8644,
+      "step": 1550
+    },
+    {
+      "epoch": 2.403697996918336,
+      "grad_norm": 1.5154823064804077,
+      "learning_rate": 0.0002,
+      "loss": 0.9235,
+      "step": 1560
+    },
+    {
+      "epoch": 2.4191063174114023,
+      "grad_norm": 0.8797892332077026,
+      "learning_rate": 0.0002,
+      "loss": 0.9778,
+      "step": 1570
+    },
+    {
+      "epoch": 2.4345146379044684,
+      "grad_norm": 1.0940500497817993,
+      "learning_rate": 0.0002,
+      "loss": 0.8952,
+      "step": 1580
+    },
+    {
+      "epoch": 2.4499229583975346,
+      "grad_norm": 1.6111960411071777,
+      "learning_rate": 0.0002,
+      "loss": 0.9352,
+      "step": 1590
+    },
+    {
+      "epoch": 2.4653312788906008,
+      "grad_norm": 0.9618533849716187,
+      "learning_rate": 0.0002,
+      "loss": 0.8462,
+      "step": 1600
+    },
+    {
+      "epoch": 2.480739599383667,
+      "grad_norm": 1.3644909858703613,
+      "learning_rate": 0.0002,
+      "loss": 0.7929,
+      "step": 1610
+    },
+    {
+      "epoch": 2.4961479198767336,
+      "grad_norm": 0.9000744223594666,
+      "learning_rate": 0.0002,
+      "loss": 0.8482,
+      "step": 1620
+    },
+    {
+      "epoch": 2.5115562403697997,
+      "grad_norm": 1.2393828630447388,
+      "learning_rate": 0.0002,
+      "loss": 0.9031,
+      "step": 1630
+    },
+    {
+      "epoch": 2.526964560862866,
+      "grad_norm": 1.231459617614746,
+      "learning_rate": 0.0002,
+      "loss": 0.8986,
+      "step": 1640
+    },
+    {
+      "epoch": 2.542372881355932,
+      "grad_norm": 1.605279564857483,
+      "learning_rate": 0.0002,
+      "loss": 0.8904,
+      "step": 1650
+    },
+    {
+      "epoch": 2.5577812018489983,
+      "grad_norm": 0.9993997812271118,
+      "learning_rate": 0.0002,
+      "loss": 0.9044,
+      "step": 1660
+    },
+    {
+      "epoch": 2.573189522342065,
+      "grad_norm": 0.8446758985519409,
+      "learning_rate": 0.0002,
+      "loss": 0.8079,
+      "step": 1670
+    },
+    {
+      "epoch": 2.588597842835131,
+      "grad_norm": 0.9734597206115723,
+      "learning_rate": 0.0002,
+      "loss": 0.8295,
+      "step": 1680
+    },
+    {
+      "epoch": 2.6040061633281972,
+      "grad_norm": 0.9620639681816101,
+      "learning_rate": 0.0002,
+      "loss": 0.8905,
+      "step": 1690
+    },
+    {
+      "epoch": 2.6194144838212634,
+      "grad_norm": 1.2597887516021729,
+      "learning_rate": 0.0002,
+      "loss": 0.893,
+      "step": 1700
+    },
+    {
+      "epoch": 2.6348228043143296,
+      "grad_norm": 0.9959160685539246,
+      "learning_rate": 0.0002,
+      "loss": 0.8567,
+      "step": 1710
+    },
+    {
+      "epoch": 2.650231124807396,
+      "grad_norm": 1.729129672050476,
+      "learning_rate": 0.0002,
+      "loss": 0.7528,
+      "step": 1720
+    },
+    {
+      "epoch": 2.6656394453004624,
+      "grad_norm": 0.8466572761535645,
+      "learning_rate": 0.0002,
+      "loss": 0.8859,
+      "step": 1730
+    },
+    {
+      "epoch": 2.6810477657935285,
+      "grad_norm": 0.9277077913284302,
+      "learning_rate": 0.0002,
+      "loss": 0.9075,
+      "step": 1740
+    },
+    {
+      "epoch": 2.6964560862865947,
+      "grad_norm": 1.1184828281402588,
+      "learning_rate": 0.0002,
+      "loss": 0.8168,
+      "step": 1750
+    },
+    {
+      "epoch": 2.711864406779661,
+      "grad_norm": 1.0716012716293335,
+      "learning_rate": 0.0002,
+      "loss": 0.9063,
+      "step": 1760
+    },
+    {
+      "epoch": 2.7272727272727275,
+      "grad_norm": 1.8334401845932007,
+      "learning_rate": 0.0002,
+      "loss": 0.9066,
+      "step": 1770
+    },
+    {
+      "epoch": 2.7426810477657937,
+      "grad_norm": 1.032091736793518,
+      "learning_rate": 0.0002,
+      "loss": 0.8721,
+      "step": 1780
+    },
+    {
+      "epoch": 2.75808936825886,
+      "grad_norm": 1.0596864223480225,
+      "learning_rate": 0.0002,
+      "loss": 0.9065,
+      "step": 1790
+    },
+    {
+      "epoch": 2.773497688751926,
+      "grad_norm": 0.9625331163406372,
+      "learning_rate": 0.0002,
+      "loss": 0.8729,
+      "step": 1800
+    },
+    {
+      "epoch": 2.788906009244992,
+      "grad_norm": 1.1381267309188843,
+      "learning_rate": 0.0002,
+      "loss": 0.8942,
+      "step": 1810
+    },
+    {
+      "epoch": 2.8043143297380584,
+      "grad_norm": 1.0851205587387085,
+      "learning_rate": 0.0002,
+      "loss": 0.8753,
+      "step": 1820
+    },
+    {
+      "epoch": 2.8197226502311246,
+      "grad_norm": 0.9770793914794922,
+      "learning_rate": 0.0002,
+      "loss": 0.8532,
+      "step": 1830
+    },
+    {
+      "epoch": 2.835130970724191,
+      "grad_norm": 1.067268967628479,
+      "learning_rate": 0.0002,
+      "loss": 0.8686,
+      "step": 1840
+    },
+    {
+      "epoch": 2.8505392912172574,
+      "grad_norm": 1.0252294540405273,
+      "learning_rate": 0.0002,
+      "loss": 0.8931,
+      "step": 1850
+    },
+    {
+      "epoch": 2.8659476117103235,
+      "grad_norm": 1.442360281944275,
+      "learning_rate": 0.0002,
+      "loss": 0.9265,
+      "step": 1860
+    },
+    {
+      "epoch": 2.8813559322033897,
+      "grad_norm": 0.8977653384208679,
+      "learning_rate": 0.0002,
+      "loss": 0.8893,
+      "step": 1870
+    },
+    {
+      "epoch": 2.896764252696456,
+      "grad_norm": 1.0946455001831055,
+      "learning_rate": 0.0002,
+      "loss": 0.8779,
+      "step": 1880
+    },
+    {
+      "epoch": 2.9121725731895225,
+      "grad_norm": 0.9993954300880432,
+      "learning_rate": 0.0002,
+      "loss": 0.8775,
+      "step": 1890
+    },
+    {
+      "epoch": 2.9275808936825887,
+      "grad_norm": 1.1941032409667969,
+      "learning_rate": 0.0002,
+      "loss": 0.8004,
+      "step": 1900
+    },
+    {
+      "epoch": 2.942989214175655,
+      "grad_norm": 1.5158019065856934,
+      "learning_rate": 0.0002,
+      "loss": 0.8862,
+      "step": 1910
+    },
+    {
+      "epoch": 2.958397534668721,
+      "grad_norm": 1.4000548124313354,
+      "learning_rate": 0.0002,
+      "loss": 0.9363,
+      "step": 1920
+    },
+    {
+      "epoch": 2.973805855161787,
+      "grad_norm": 0.9906305074691772,
+      "learning_rate": 0.0002,
+      "loss": 0.8666,
+      "step": 1930
+    },
+    {
+      "epoch": 2.989214175654854,
+      "grad_norm": 1.7382938861846924,
+      "learning_rate": 0.0002,
+      "loss": 0.8434,
+      "step": 1940
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 1.2514833211898804,
+      "eval_runtime": 119.8709,
+      "eval_samples_per_second": 3.804,
+      "eval_steps_per_second": 0.476,
+      "step": 1947
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 5192,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 8.54240928989184e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-1947/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7da6f114429a9a993fcd6ed877a996e6a07ac3f427895c4f50ec6990a9e85e69
+size 5560

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-2596/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-2596/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-2596/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fdfd727c0fdda5e35ef04166620a4ae2e1e0ee99c352b3802122990177ab3f6f
+size 109069176

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-2596/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d4fa6fe25b3b146361946d8a416b27b7c69623b52190c1afbb42f0c3f195cb67
+size 55532666

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-2596/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d3cd4d7b01ac2d9db58f0da5efa0ca40fad30eed1aa4628a83d2f6f773589ef7
+size 14244

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-2596/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:30efe1d0afc5640279320794cade4d996759e229c72d2184c501c3f8130e7fa7
+size 1064

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-2596/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-2596/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-2596/trainer_state.json

@@ -0,0 +1,1878 @@
+{
+  "best_metric": 1.212496042251587,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-649",
+  "epoch": 4.0,
+  "eval_steps": 10,
+  "global_step": 2596,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.015408320493066256,
+      "grad_norm": 0.939333975315094,
+      "learning_rate": 0.0002,
+      "loss": 1.9609,
+      "step": 10
+    },
+    {
+      "epoch": 0.030816640986132512,
+      "grad_norm": 0.9446700215339661,
+      "learning_rate": 0.0002,
+      "loss": 1.5975,
+      "step": 20
+    },
+    {
+      "epoch": 0.046224961479198766,
+      "grad_norm": 0.7268466353416443,
+      "learning_rate": 0.0002,
+      "loss": 1.5196,
+      "step": 30
+    },
+    {
+      "epoch": 0.061633281972265024,
+      "grad_norm": 0.8900066018104553,
+      "learning_rate": 0.0002,
+      "loss": 1.5021,
+      "step": 40
+    },
+    {
+      "epoch": 0.07704160246533127,
+      "grad_norm": 4.246077060699463,
+      "learning_rate": 0.0002,
+      "loss": 1.4155,
+      "step": 50
+    },
+    {
+      "epoch": 0.09244992295839753,
+      "grad_norm": 0.7887561321258545,
+      "learning_rate": 0.0002,
+      "loss": 1.4052,
+      "step": 60
+    },
+    {
+      "epoch": 0.10785824345146379,
+      "grad_norm": 0.6822794675827026,
+      "learning_rate": 0.0002,
+      "loss": 1.215,
+      "step": 70
+    },
+    {
+      "epoch": 0.12326656394453005,
+      "grad_norm": 0.8871720433235168,
+      "learning_rate": 0.0002,
+      "loss": 1.2524,
+      "step": 80
+    },
+    {
+      "epoch": 0.1386748844375963,
+      "grad_norm": 0.6524078249931335,
+      "learning_rate": 0.0002,
+      "loss": 1.191,
+      "step": 90
+    },
+    {
+      "epoch": 0.15408320493066255,
+      "grad_norm": 0.5519863367080688,
+      "learning_rate": 0.0002,
+      "loss": 1.2455,
+      "step": 100
+    },
+    {
+      "epoch": 0.1694915254237288,
+      "grad_norm": 0.6716212630271912,
+      "learning_rate": 0.0002,
+      "loss": 1.2515,
+      "step": 110
+    },
+    {
+      "epoch": 0.18489984591679506,
+      "grad_norm": 1.527921199798584,
+      "learning_rate": 0.0002,
+      "loss": 1.2239,
+      "step": 120
+    },
+    {
+      "epoch": 0.20030816640986132,
+      "grad_norm": 0.6893861293792725,
+      "learning_rate": 0.0002,
+      "loss": 1.2887,
+      "step": 130
+    },
+    {
+      "epoch": 0.21571648690292758,
+      "grad_norm": 0.5514246821403503,
+      "learning_rate": 0.0002,
+      "loss": 1.1722,
+      "step": 140
+    },
+    {
+      "epoch": 0.23112480739599384,
+      "grad_norm": 0.5830065608024597,
+      "learning_rate": 0.0002,
+      "loss": 1.1946,
+      "step": 150
+    },
+    {
+      "epoch": 0.2465331278890601,
+      "grad_norm": 0.7411216497421265,
+      "learning_rate": 0.0002,
+      "loss": 1.3105,
+      "step": 160
+    },
+    {
+      "epoch": 0.26194144838212635,
+      "grad_norm": 0.7470705509185791,
+      "learning_rate": 0.0002,
+      "loss": 1.2008,
+      "step": 170
+    },
+    {
+      "epoch": 0.2773497688751926,
+      "grad_norm": 0.6140725016593933,
+      "learning_rate": 0.0002,
+      "loss": 1.2497,
+      "step": 180
+    },
+    {
+      "epoch": 0.29275808936825887,
+      "grad_norm": 0.7563071250915527,
+      "learning_rate": 0.0002,
+      "loss": 1.1514,
+      "step": 190
+    },
+    {
+      "epoch": 0.3081664098613251,
+      "grad_norm": 0.5491266250610352,
+      "learning_rate": 0.0002,
+      "loss": 1.2703,
+      "step": 200
+    },
+    {
+      "epoch": 0.3235747303543914,
+      "grad_norm": 0.5403220057487488,
+      "learning_rate": 0.0002,
+      "loss": 1.2168,
+      "step": 210
+    },
+    {
+      "epoch": 0.3389830508474576,
+      "grad_norm": 0.9616869688034058,
+      "learning_rate": 0.0002,
+      "loss": 1.2409,
+      "step": 220
+    },
+    {
+      "epoch": 0.3543913713405239,
+      "grad_norm": 0.5448639988899231,
+      "learning_rate": 0.0002,
+      "loss": 1.3112,
+      "step": 230
+    },
+    {
+      "epoch": 0.3697996918335901,
+      "grad_norm": 0.6322043538093567,
+      "learning_rate": 0.0002,
+      "loss": 1.3023,
+      "step": 240
+    },
+    {
+      "epoch": 0.3852080123266564,
+      "grad_norm": 0.5178093314170837,
+      "learning_rate": 0.0002,
+      "loss": 1.2382,
+      "step": 250
+    },
+    {
+      "epoch": 0.40061633281972264,
+      "grad_norm": 0.6274669766426086,
+      "learning_rate": 0.0002,
+      "loss": 1.2755,
+      "step": 260
+    },
+    {
+      "epoch": 0.41602465331278893,
+      "grad_norm": 1.1093109846115112,
+      "learning_rate": 0.0002,
+      "loss": 1.3263,
+      "step": 270
+    },
+    {
+      "epoch": 0.43143297380585516,
+      "grad_norm": 1.8002032041549683,
+      "learning_rate": 0.0002,
+      "loss": 1.2201,
+      "step": 280
+    },
+    {
+      "epoch": 0.44684129429892144,
+      "grad_norm": 0.5977614521980286,
+      "learning_rate": 0.0002,
+      "loss": 1.2335,
+      "step": 290
+    },
+    {
+      "epoch": 0.4622496147919877,
+      "grad_norm": 0.5583769083023071,
+      "learning_rate": 0.0002,
+      "loss": 1.2076,
+      "step": 300
+    },
+    {
+      "epoch": 0.4776579352850539,
+      "grad_norm": 0.5563502311706543,
+      "learning_rate": 0.0002,
+      "loss": 1.197,
+      "step": 310
+    },
+    {
+      "epoch": 0.4930662557781202,
+      "grad_norm": 0.9157887101173401,
+      "learning_rate": 0.0002,
+      "loss": 1.164,
+      "step": 320
+    },
+    {
+      "epoch": 0.5084745762711864,
+      "grad_norm": 0.551705539226532,
+      "learning_rate": 0.0002,
+      "loss": 1.1266,
+      "step": 330
+    },
+    {
+      "epoch": 0.5238828967642527,
+      "grad_norm": 9.011388778686523,
+      "learning_rate": 0.0002,
+      "loss": 1.2801,
+      "step": 340
+    },
+    {
+      "epoch": 0.539291217257319,
+      "grad_norm": 0.6527810096740723,
+      "learning_rate": 0.0002,
+      "loss": 1.2024,
+      "step": 350
+    },
+    {
+      "epoch": 0.5546995377503852,
+      "grad_norm": 1.3121646642684937,
+      "learning_rate": 0.0002,
+      "loss": 1.1424,
+      "step": 360
+    },
+    {
+      "epoch": 0.5701078582434514,
+      "grad_norm": 0.7247968316078186,
+      "learning_rate": 0.0002,
+      "loss": 1.2555,
+      "step": 370
+    },
+    {
+      "epoch": 0.5855161787365177,
+      "grad_norm": 0.5430527925491333,
+      "learning_rate": 0.0002,
+      "loss": 1.196,
+      "step": 380
+    },
+    {
+      "epoch": 0.600924499229584,
+      "grad_norm": 0.6050328612327576,
+      "learning_rate": 0.0002,
+      "loss": 1.1822,
+      "step": 390
+    },
+    {
+      "epoch": 0.6163328197226502,
+      "grad_norm": 0.49382615089416504,
+      "learning_rate": 0.0002,
+      "loss": 1.2754,
+      "step": 400
+    },
+    {
+      "epoch": 0.6317411402157165,
+      "grad_norm": 0.5804041624069214,
+      "learning_rate": 0.0002,
+      "loss": 1.2595,
+      "step": 410
+    },
+    {
+      "epoch": 0.6471494607087828,
+      "grad_norm": 0.524357259273529,
+      "learning_rate": 0.0002,
+      "loss": 1.2926,
+      "step": 420
+    },
+    {
+      "epoch": 0.662557781201849,
+      "grad_norm": 1.8817486763000488,
+      "learning_rate": 0.0002,
+      "loss": 1.1928,
+      "step": 430
+    },
+    {
+      "epoch": 0.6779661016949152,
+      "grad_norm": 0.6104950308799744,
+      "learning_rate": 0.0002,
+      "loss": 1.2103,
+      "step": 440
+    },
+    {
+      "epoch": 0.6933744221879815,
+      "grad_norm": 0.631179928779602,
+      "learning_rate": 0.0002,
+      "loss": 1.2296,
+      "step": 450
+    },
+    {
+      "epoch": 0.7087827426810478,
+      "grad_norm": 0.6562483310699463,
+      "learning_rate": 0.0002,
+      "loss": 1.1843,
+      "step": 460
+    },
+    {
+      "epoch": 0.724191063174114,
+      "grad_norm": 0.578336775302887,
+      "learning_rate": 0.0002,
+      "loss": 1.2424,
+      "step": 470
+    },
+    {
+      "epoch": 0.7395993836671803,
+      "grad_norm": 0.8468874096870422,
+      "learning_rate": 0.0002,
+      "loss": 1.192,
+      "step": 480
+    },
+    {
+      "epoch": 0.7550077041602465,
+      "grad_norm": 0.7162539958953857,
+      "learning_rate": 0.0002,
+      "loss": 1.2913,
+      "step": 490
+    },
+    {
+      "epoch": 0.7704160246533128,
+      "grad_norm": 0.7655543684959412,
+      "learning_rate": 0.0002,
+      "loss": 1.2158,
+      "step": 500
+    },
+    {
+      "epoch": 0.785824345146379,
+      "grad_norm": 0.5711562037467957,
+      "learning_rate": 0.0002,
+      "loss": 1.2637,
+      "step": 510
+    },
+    {
+      "epoch": 0.8012326656394453,
+      "grad_norm": 0.7273485660552979,
+      "learning_rate": 0.0002,
+      "loss": 1.1277,
+      "step": 520
+    },
+    {
+      "epoch": 0.8166409861325116,
+      "grad_norm": 0.47050145268440247,
+      "learning_rate": 0.0002,
+      "loss": 1.188,
+      "step": 530
+    },
+    {
+      "epoch": 0.8320493066255779,
+      "grad_norm": 0.4930959939956665,
+      "learning_rate": 0.0002,
+      "loss": 1.1462,
+      "step": 540
+    },
+    {
+      "epoch": 0.847457627118644,
+      "grad_norm": 0.553955614566803,
+      "learning_rate": 0.0002,
+      "loss": 1.167,
+      "step": 550
+    },
+    {
+      "epoch": 0.8628659476117103,
+      "grad_norm": 0.6495056748390198,
+      "learning_rate": 0.0002,
+      "loss": 1.1831,
+      "step": 560
+    },
+    {
+      "epoch": 0.8782742681047766,
+      "grad_norm": 0.4586578905582428,
+      "learning_rate": 0.0002,
+      "loss": 1.0984,
+      "step": 570
+    },
+    {
+      "epoch": 0.8936825885978429,
+      "grad_norm": 0.4870392680168152,
+      "learning_rate": 0.0002,
+      "loss": 1.2415,
+      "step": 580
+    },
+    {
+      "epoch": 0.9090909090909091,
+      "grad_norm": 0.6337013244628906,
+      "learning_rate": 0.0002,
+      "loss": 1.1735,
+      "step": 590
+    },
+    {
+      "epoch": 0.9244992295839753,
+      "grad_norm": 0.7260186672210693,
+      "learning_rate": 0.0002,
+      "loss": 1.1781,
+      "step": 600
+    },
+    {
+      "epoch": 0.9399075500770416,
+      "grad_norm": 1.4133737087249756,
+      "learning_rate": 0.0002,
+      "loss": 1.2536,
+      "step": 610
+    },
+    {
+      "epoch": 0.9553158705701078,
+      "grad_norm": 0.573525071144104,
+      "learning_rate": 0.0002,
+      "loss": 1.2211,
+      "step": 620
+    },
+    {
+      "epoch": 0.9707241910631741,
+      "grad_norm": 0.8403644561767578,
+      "learning_rate": 0.0002,
+      "loss": 1.11,
+      "step": 630
+    },
+    {
+      "epoch": 0.9861325115562404,
+      "grad_norm": 0.7665438055992126,
+      "learning_rate": 0.0002,
+      "loss": 1.2932,
+      "step": 640
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.212496042251587,
+      "eval_runtime": 120.3264,
+      "eval_samples_per_second": 3.79,
+      "eval_steps_per_second": 0.474,
+      "step": 649
+    },
+    {
+      "epoch": 1.0015408320493067,
+      "grad_norm": 0.6874801516532898,
+      "learning_rate": 0.0002,
+      "loss": 1.176,
+      "step": 650
+    },
+    {
+      "epoch": 1.0169491525423728,
+      "grad_norm": 0.6675221920013428,
+      "learning_rate": 0.0002,
+      "loss": 1.0957,
+      "step": 660
+    },
+    {
+      "epoch": 1.0323574730354392,
+      "grad_norm": 0.6837265491485596,
+      "learning_rate": 0.0002,
+      "loss": 1.1616,
+      "step": 670
+    },
+    {
+      "epoch": 1.0477657935285054,
+      "grad_norm": 0.6796931028366089,
+      "learning_rate": 0.0002,
+      "loss": 1.045,
+      "step": 680
+    },
+    {
+      "epoch": 1.0631741140215716,
+      "grad_norm": 0.6596675515174866,
+      "learning_rate": 0.0002,
+      "loss": 1.0889,
+      "step": 690
+    },
+    {
+      "epoch": 1.078582434514638,
+      "grad_norm": 0.6630653142929077,
+      "learning_rate": 0.0002,
+      "loss": 1.067,
+      "step": 700
+    },
+    {
+      "epoch": 1.0939907550077042,
+      "grad_norm": 0.7065498232841492,
+      "learning_rate": 0.0002,
+      "loss": 1.0222,
+      "step": 710
+    },
+    {
+      "epoch": 1.1093990755007703,
+      "grad_norm": 0.694682240486145,
+      "learning_rate": 0.0002,
+      "loss": 1.081,
+      "step": 720
+    },
+    {
+      "epoch": 1.1248073959938367,
+      "grad_norm": 0.8984217047691345,
+      "learning_rate": 0.0002,
+      "loss": 1.1019,
+      "step": 730
+    },
+    {
+      "epoch": 1.140215716486903,
+      "grad_norm": 1.0035051107406616,
+      "learning_rate": 0.0002,
+      "loss": 1.0506,
+      "step": 740
+    },
+    {
+      "epoch": 1.1556240369799693,
+      "grad_norm": 0.8646948933601379,
+      "learning_rate": 0.0002,
+      "loss": 1.0502,
+      "step": 750
+    },
+    {
+      "epoch": 1.1710323574730355,
+      "grad_norm": 0.6287558674812317,
+      "learning_rate": 0.0002,
+      "loss": 1.0256,
+      "step": 760
+    },
+    {
+      "epoch": 1.1864406779661016,
+      "grad_norm": 0.6975560784339905,
+      "learning_rate": 0.0002,
+      "loss": 1.051,
+      "step": 770
+    },
+    {
+      "epoch": 1.201848998459168,
+      "grad_norm": 0.5972192883491516,
+      "learning_rate": 0.0002,
+      "loss": 1.0409,
+      "step": 780
+    },
+    {
+      "epoch": 1.2172573189522342,
+      "grad_norm": 0.7299932241439819,
+      "learning_rate": 0.0002,
+      "loss": 1.0837,
+      "step": 790
+    },
+    {
+      "epoch": 1.2326656394453004,
+      "grad_norm": 0.8376814126968384,
+      "learning_rate": 0.0002,
+      "loss": 1.1291,
+      "step": 800
+    },
+    {
+      "epoch": 1.2480739599383668,
+      "grad_norm": 1.117690086364746,
+      "learning_rate": 0.0002,
+      "loss": 1.0809,
+      "step": 810
+    },
+    {
+      "epoch": 1.263482280431433,
+      "grad_norm": 0.7850839495658875,
+      "learning_rate": 0.0002,
+      "loss": 1.0762,
+      "step": 820
+    },
+    {
+      "epoch": 1.2788906009244991,
+      "grad_norm": 0.7941734790802002,
+      "learning_rate": 0.0002,
+      "loss": 1.2278,
+      "step": 830
+    },
+    {
+      "epoch": 1.2942989214175655,
+      "grad_norm": 0.5682743787765503,
+      "learning_rate": 0.0002,
+      "loss": 0.9723,
+      "step": 840
+    },
+    {
+      "epoch": 1.3097072419106317,
+      "grad_norm": 0.7182576060295105,
+      "learning_rate": 0.0002,
+      "loss": 1.0262,
+      "step": 850
+    },
+    {
+      "epoch": 1.325115562403698,
+      "grad_norm": 0.7436819672584534,
+      "learning_rate": 0.0002,
+      "loss": 1.0801,
+      "step": 860
+    },
+    {
+      "epoch": 1.3405238828967643,
+      "grad_norm": 0.7665132880210876,
+      "learning_rate": 0.0002,
+      "loss": 1.0705,
+      "step": 870
+    },
+    {
+      "epoch": 1.3559322033898304,
+      "grad_norm": 1.060669183731079,
+      "learning_rate": 0.0002,
+      "loss": 1.1925,
+      "step": 880
+    },
+    {
+      "epoch": 1.3713405238828968,
+      "grad_norm": 0.6262772083282471,
+      "learning_rate": 0.0002,
+      "loss": 1.0581,
+      "step": 890
+    },
+    {
+      "epoch": 1.386748844375963,
+      "grad_norm": 1.010703682899475,
+      "learning_rate": 0.0002,
+      "loss": 1.0934,
+      "step": 900
+    },
+    {
+      "epoch": 1.4021571648690292,
+      "grad_norm": 0.5669811367988586,
+      "learning_rate": 0.0002,
+      "loss": 1.111,
+      "step": 910
+    },
+    {
+      "epoch": 1.4175654853620956,
+      "grad_norm": 0.8644649386405945,
+      "learning_rate": 0.0002,
+      "loss": 1.0638,
+      "step": 920
+    },
+    {
+      "epoch": 1.4329738058551618,
+      "grad_norm": 0.6250067949295044,
+      "learning_rate": 0.0002,
+      "loss": 1.1973,
+      "step": 930
+    },
+    {
+      "epoch": 1.448382126348228,
+      "grad_norm": 0.6507946848869324,
+      "learning_rate": 0.0002,
+      "loss": 1.0592,
+      "step": 940
+    },
+    {
+      "epoch": 1.4637904468412943,
+      "grad_norm": 0.9305022954940796,
+      "learning_rate": 0.0002,
+      "loss": 1.0306,
+      "step": 950
+    },
+    {
+      "epoch": 1.4791987673343605,
+      "grad_norm": 0.86021888256073,
+      "learning_rate": 0.0002,
+      "loss": 0.9792,
+      "step": 960
+    },
+    {
+      "epoch": 1.494607087827427,
+      "grad_norm": 1.1122219562530518,
+      "learning_rate": 0.0002,
+      "loss": 1.0656,
+      "step": 970
+    },
+    {
+      "epoch": 1.510015408320493,
+      "grad_norm": 0.7431066036224365,
+      "learning_rate": 0.0002,
+      "loss": 1.0745,
+      "step": 980
+    },
+    {
+      "epoch": 1.5254237288135593,
+      "grad_norm": 1.022744059562683,
+      "learning_rate": 0.0002,
+      "loss": 1.0822,
+      "step": 990
+    },
+    {
+      "epoch": 1.5408320493066254,
+      "grad_norm": 0.6638363599777222,
+      "learning_rate": 0.0002,
+      "loss": 1.0689,
+      "step": 1000
+    },
+    {
+      "epoch": 1.5562403697996918,
+      "grad_norm": 0.6290505528450012,
+      "learning_rate": 0.0002,
+      "loss": 1.1032,
+      "step": 1010
+    },
+    {
+      "epoch": 1.5716486902927582,
+      "grad_norm": 0.6446037888526917,
+      "learning_rate": 0.0002,
+      "loss": 1.0572,
+      "step": 1020
+    },
+    {
+      "epoch": 1.5870570107858244,
+      "grad_norm": 0.7383618354797363,
+      "learning_rate": 0.0002,
+      "loss": 1.0814,
+      "step": 1030
+    },
+    {
+      "epoch": 1.6024653312788906,
+      "grad_norm": 3.5893094539642334,
+      "learning_rate": 0.0002,
+      "loss": 1.0643,
+      "step": 1040
+    },
+    {
+      "epoch": 1.6178736517719567,
+      "grad_norm": 0.7388538718223572,
+      "learning_rate": 0.0002,
+      "loss": 0.9806,
+      "step": 1050
+    },
+    {
+      "epoch": 1.6332819722650231,
+      "grad_norm": 0.7813242077827454,
+      "learning_rate": 0.0002,
+      "loss": 1.1,
+      "step": 1060
+    },
+    {
+      "epoch": 1.6486902927580893,
+      "grad_norm": 0.6643418669700623,
+      "learning_rate": 0.0002,
+      "loss": 1.0463,
+      "step": 1070
+    },
+    {
+      "epoch": 1.6640986132511557,
+      "grad_norm": 0.7817712426185608,
+      "learning_rate": 0.0002,
+      "loss": 1.0344,
+      "step": 1080
+    },
+    {
+      "epoch": 1.6795069337442219,
+      "grad_norm": 0.8741177916526794,
+      "learning_rate": 0.0002,
+      "loss": 1.0447,
+      "step": 1090
+    },
+    {
+      "epoch": 1.694915254237288,
+      "grad_norm": 0.969813883304596,
+      "learning_rate": 0.0002,
+      "loss": 1.0724,
+      "step": 1100
+    },
+    {
+      "epoch": 1.7103235747303542,
+      "grad_norm": 0.7666333913803101,
+      "learning_rate": 0.0002,
+      "loss": 0.9887,
+      "step": 1110
+    },
+    {
+      "epoch": 1.7257318952234206,
+      "grad_norm": 0.789859414100647,
+      "learning_rate": 0.0002,
+      "loss": 0.9159,
+      "step": 1120
+    },
+    {
+      "epoch": 1.741140215716487,
+      "grad_norm": 0.7018347382545471,
+      "learning_rate": 0.0002,
+      "loss": 1.0679,
+      "step": 1130
+    },
+    {
+      "epoch": 1.7565485362095532,
+      "grad_norm": 0.8899882435798645,
+      "learning_rate": 0.0002,
+      "loss": 1.1429,
+      "step": 1140
+    },
+    {
+      "epoch": 1.7719568567026194,
+      "grad_norm": 0.7738426923751831,
+      "learning_rate": 0.0002,
+      "loss": 0.9214,
+      "step": 1150
+    },
+    {
+      "epoch": 1.7873651771956856,
+      "grad_norm": 1.032209873199463,
+      "learning_rate": 0.0002,
+      "loss": 1.0733,
+      "step": 1160
+    },
+    {
+      "epoch": 1.802773497688752,
+      "grad_norm": 0.7310711741447449,
+      "learning_rate": 0.0002,
+      "loss": 1.0522,
+      "step": 1170
+    },
+    {
+      "epoch": 1.8181818181818183,
+      "grad_norm": 0.8506733775138855,
+      "learning_rate": 0.0002,
+      "loss": 1.0943,
+      "step": 1180
+    },
+    {
+      "epoch": 1.8335901386748845,
+      "grad_norm": 116.00221252441406,
+      "learning_rate": 0.0002,
+      "loss": 1.1349,
+      "step": 1190
+    },
+    {
+      "epoch": 1.8489984591679507,
+      "grad_norm": 0.8819684982299805,
+      "learning_rate": 0.0002,
+      "loss": 1.0595,
+      "step": 1200
+    },
+    {
+      "epoch": 1.8644067796610169,
+      "grad_norm": 0.8846774697303772,
+      "learning_rate": 0.0002,
+      "loss": 0.995,
+      "step": 1210
+    },
+    {
+      "epoch": 1.879815100154083,
+      "grad_norm": 1.6417866945266724,
+      "learning_rate": 0.0002,
+      "loss": 1.116,
+      "step": 1220
+    },
+    {
+      "epoch": 1.8952234206471494,
+      "grad_norm": 1.1875122785568237,
+      "learning_rate": 0.0002,
+      "loss": 1.044,
+      "step": 1230
+    },
+    {
+      "epoch": 1.9106317411402158,
+      "grad_norm": 0.6520240306854248,
+      "learning_rate": 0.0002,
+      "loss": 1.0322,
+      "step": 1240
+    },
+    {
+      "epoch": 1.926040061633282,
+      "grad_norm": 0.9059738516807556,
+      "learning_rate": 0.0002,
+      "loss": 1.0175,
+      "step": 1250
+    },
+    {
+      "epoch": 1.9414483821263482,
+      "grad_norm": 0.7950983643531799,
+      "learning_rate": 0.0002,
+      "loss": 1.1097,
+      "step": 1260
+    },
+    {
+      "epoch": 1.9568567026194144,
+      "grad_norm": 0.6911653280258179,
+      "learning_rate": 0.0002,
+      "loss": 1.0313,
+      "step": 1270
+    },
+    {
+      "epoch": 1.9722650231124808,
+      "grad_norm": 0.9487978219985962,
+      "learning_rate": 0.0002,
+      "loss": 1.0762,
+      "step": 1280
+    },
+    {
+      "epoch": 1.9876733436055471,
+      "grad_norm": 0.716439425945282,
+      "learning_rate": 0.0002,
+      "loss": 1.1608,
+      "step": 1290
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.215390682220459,
+      "eval_runtime": 120.3893,
+      "eval_samples_per_second": 3.788,
+      "eval_steps_per_second": 0.473,
+      "step": 1298
+    },
+    {
+      "epoch": 2.0030816640986133,
+      "grad_norm": 0.7292942404747009,
+      "learning_rate": 0.0002,
+      "loss": 1.0205,
+      "step": 1300
+    },
+    {
+      "epoch": 2.0184899845916795,
+      "grad_norm": 0.8386905789375305,
+      "learning_rate": 0.0002,
+      "loss": 0.8264,
+      "step": 1310
+    },
+    {
+      "epoch": 2.0338983050847457,
+      "grad_norm": 1.114105224609375,
+      "learning_rate": 0.0002,
+      "loss": 0.7925,
+      "step": 1320
+    },
+    {
+      "epoch": 2.049306625577812,
+      "grad_norm": 1.068211317062378,
+      "learning_rate": 0.0002,
+      "loss": 0.8673,
+      "step": 1330
+    },
+    {
+      "epoch": 2.0647149460708785,
+      "grad_norm": 0.8620569109916687,
+      "learning_rate": 0.0002,
+      "loss": 0.8883,
+      "step": 1340
+    },
+    {
+      "epoch": 2.0801232665639446,
+      "grad_norm": 0.8984315991401672,
+      "learning_rate": 0.0002,
+      "loss": 0.8949,
+      "step": 1350
+    },
+    {
+      "epoch": 2.095531587057011,
+      "grad_norm": 1.0469365119934082,
+      "learning_rate": 0.0002,
+      "loss": 0.8992,
+      "step": 1360
+    },
+    {
+      "epoch": 2.110939907550077,
+      "grad_norm": 0.9392943978309631,
+      "learning_rate": 0.0002,
+      "loss": 0.8053,
+      "step": 1370
+    },
+    {
+      "epoch": 2.126348228043143,
+      "grad_norm": 1.2067331075668335,
+      "learning_rate": 0.0002,
+      "loss": 0.8791,
+      "step": 1380
+    },
+    {
+      "epoch": 2.1417565485362093,
+      "grad_norm": 0.8019351363182068,
+      "learning_rate": 0.0002,
+      "loss": 0.7925,
+      "step": 1390
+    },
+    {
+      "epoch": 2.157164869029276,
+      "grad_norm": 0.9445130228996277,
+      "learning_rate": 0.0002,
+      "loss": 0.8669,
+      "step": 1400
+    },
+    {
+      "epoch": 2.172573189522342,
+      "grad_norm": 0.9806232452392578,
+      "learning_rate": 0.0002,
+      "loss": 0.8293,
+      "step": 1410
+    },
+    {
+      "epoch": 2.1879815100154083,
+      "grad_norm": 1.2135679721832275,
+      "learning_rate": 0.0002,
+      "loss": 0.8499,
+      "step": 1420
+    },
+    {
+      "epoch": 2.2033898305084745,
+      "grad_norm": 1.1036803722381592,
+      "learning_rate": 0.0002,
+      "loss": 0.8908,
+      "step": 1430
+    },
+    {
+      "epoch": 2.2187981510015407,
+      "grad_norm": 0.9235773086547852,
+      "learning_rate": 0.0002,
+      "loss": 0.9725,
+      "step": 1440
+    },
+    {
+      "epoch": 2.2342064714946073,
+      "grad_norm": 1.0520254373550415,
+      "learning_rate": 0.0002,
+      "loss": 0.8176,
+      "step": 1450
+    },
+    {
+      "epoch": 2.2496147919876734,
+      "grad_norm": 1.0121252536773682,
+      "learning_rate": 0.0002,
+      "loss": 0.9138,
+      "step": 1460
+    },
+    {
+      "epoch": 2.2650231124807396,
+      "grad_norm": 1.0718834400177002,
+      "learning_rate": 0.0002,
+      "loss": 0.8722,
+      "step": 1470
+    },
+    {
+      "epoch": 2.280431432973806,
+      "grad_norm": 1.1842551231384277,
+      "learning_rate": 0.0002,
+      "loss": 0.8295,
+      "step": 1480
+    },
+    {
+      "epoch": 2.295839753466872,
+      "grad_norm": 1.2985937595367432,
+      "learning_rate": 0.0002,
+      "loss": 0.8706,
+      "step": 1490
+    },
+    {
+      "epoch": 2.3112480739599386,
+      "grad_norm": 2.0254523754119873,
+      "learning_rate": 0.0002,
+      "loss": 0.873,
+      "step": 1500
+    },
+    {
+      "epoch": 2.3266563944530048,
+      "grad_norm": 1.1501885652542114,
+      "learning_rate": 0.0002,
+      "loss": 0.8569,
+      "step": 1510
+    },
+    {
+      "epoch": 2.342064714946071,
+      "grad_norm": 1.1277996301651,
+      "learning_rate": 0.0002,
+      "loss": 0.9394,
+      "step": 1520
+    },
+    {
+      "epoch": 2.357473035439137,
+      "grad_norm": 0.9330579042434692,
+      "learning_rate": 0.0002,
+      "loss": 0.8856,
+      "step": 1530
+    },
+    {
+      "epoch": 2.3728813559322033,
+      "grad_norm": 1.3631645441055298,
+      "learning_rate": 0.0002,
+      "loss": 0.8679,
+      "step": 1540
+    },
+    {
+      "epoch": 2.3882896764252695,
+      "grad_norm": 1.070383071899414,
+      "learning_rate": 0.0002,
+      "loss": 0.8644,
+      "step": 1550
+    },
+    {
+      "epoch": 2.403697996918336,
+      "grad_norm": 1.5154823064804077,
+      "learning_rate": 0.0002,
+      "loss": 0.9235,
+      "step": 1560
+    },
+    {
+      "epoch": 2.4191063174114023,
+      "grad_norm": 0.8797892332077026,
+      "learning_rate": 0.0002,
+      "loss": 0.9778,
+      "step": 1570
+    },
+    {
+      "epoch": 2.4345146379044684,
+      "grad_norm": 1.0940500497817993,
+      "learning_rate": 0.0002,
+      "loss": 0.8952,
+      "step": 1580
+    },
+    {
+      "epoch": 2.4499229583975346,
+      "grad_norm": 1.6111960411071777,
+      "learning_rate": 0.0002,
+      "loss": 0.9352,
+      "step": 1590
+    },
+    {
+      "epoch": 2.4653312788906008,
+      "grad_norm": 0.9618533849716187,
+      "learning_rate": 0.0002,
+      "loss": 0.8462,
+      "step": 1600
+    },
+    {
+      "epoch": 2.480739599383667,
+      "grad_norm": 1.3644909858703613,
+      "learning_rate": 0.0002,
+      "loss": 0.7929,
+      "step": 1610
+    },
+    {
+      "epoch": 2.4961479198767336,
+      "grad_norm": 0.9000744223594666,
+      "learning_rate": 0.0002,
+      "loss": 0.8482,
+      "step": 1620
+    },
+    {
+      "epoch": 2.5115562403697997,
+      "grad_norm": 1.2393828630447388,
+      "learning_rate": 0.0002,
+      "loss": 0.9031,
+      "step": 1630
+    },
+    {
+      "epoch": 2.526964560862866,
+      "grad_norm": 1.231459617614746,
+      "learning_rate": 0.0002,
+      "loss": 0.8986,
+      "step": 1640
+    },
+    {
+      "epoch": 2.542372881355932,
+      "grad_norm": 1.605279564857483,
+      "learning_rate": 0.0002,
+      "loss": 0.8904,
+      "step": 1650
+    },
+    {
+      "epoch": 2.5577812018489983,
+      "grad_norm": 0.9993997812271118,
+      "learning_rate": 0.0002,
+      "loss": 0.9044,
+      "step": 1660
+    },
+    {
+      "epoch": 2.573189522342065,
+      "grad_norm": 0.8446758985519409,
+      "learning_rate": 0.0002,
+      "loss": 0.8079,
+      "step": 1670
+    },
+    {
+      "epoch": 2.588597842835131,
+      "grad_norm": 0.9734597206115723,
+      "learning_rate": 0.0002,
+      "loss": 0.8295,
+      "step": 1680
+    },
+    {
+      "epoch": 2.6040061633281972,
+      "grad_norm": 0.9620639681816101,
+      "learning_rate": 0.0002,
+      "loss": 0.8905,
+      "step": 1690
+    },
+    {
+      "epoch": 2.6194144838212634,
+      "grad_norm": 1.2597887516021729,
+      "learning_rate": 0.0002,
+      "loss": 0.893,
+      "step": 1700
+    },
+    {
+      "epoch": 2.6348228043143296,
+      "grad_norm": 0.9959160685539246,
+      "learning_rate": 0.0002,
+      "loss": 0.8567,
+      "step": 1710
+    },
+    {
+      "epoch": 2.650231124807396,
+      "grad_norm": 1.729129672050476,
+      "learning_rate": 0.0002,
+      "loss": 0.7528,
+      "step": 1720
+    },
+    {
+      "epoch": 2.6656394453004624,
+      "grad_norm": 0.8466572761535645,
+      "learning_rate": 0.0002,
+      "loss": 0.8859,
+      "step": 1730
+    },
+    {
+      "epoch": 2.6810477657935285,
+      "grad_norm": 0.9277077913284302,
+      "learning_rate": 0.0002,
+      "loss": 0.9075,
+      "step": 1740
+    },
+    {
+      "epoch": 2.6964560862865947,
+      "grad_norm": 1.1184828281402588,
+      "learning_rate": 0.0002,
+      "loss": 0.8168,
+      "step": 1750
+    },
+    {
+      "epoch": 2.711864406779661,
+      "grad_norm": 1.0716012716293335,
+      "learning_rate": 0.0002,
+      "loss": 0.9063,
+      "step": 1760
+    },
+    {
+      "epoch": 2.7272727272727275,
+      "grad_norm": 1.8334401845932007,
+      "learning_rate": 0.0002,
+      "loss": 0.9066,
+      "step": 1770
+    },
+    {
+      "epoch": 2.7426810477657937,
+      "grad_norm": 1.032091736793518,
+      "learning_rate": 0.0002,
+      "loss": 0.8721,
+      "step": 1780
+    },
+    {
+      "epoch": 2.75808936825886,
+      "grad_norm": 1.0596864223480225,
+      "learning_rate": 0.0002,
+      "loss": 0.9065,
+      "step": 1790
+    },
+    {
+      "epoch": 2.773497688751926,
+      "grad_norm": 0.9625331163406372,
+      "learning_rate": 0.0002,
+      "loss": 0.8729,
+      "step": 1800
+    },
+    {
+      "epoch": 2.788906009244992,
+      "grad_norm": 1.1381267309188843,
+      "learning_rate": 0.0002,
+      "loss": 0.8942,
+      "step": 1810
+    },
+    {
+      "epoch": 2.8043143297380584,
+      "grad_norm": 1.0851205587387085,
+      "learning_rate": 0.0002,
+      "loss": 0.8753,
+      "step": 1820
+    },
+    {
+      "epoch": 2.8197226502311246,
+      "grad_norm": 0.9770793914794922,
+      "learning_rate": 0.0002,
+      "loss": 0.8532,
+      "step": 1830
+    },
+    {
+      "epoch": 2.835130970724191,
+      "grad_norm": 1.067268967628479,
+      "learning_rate": 0.0002,
+      "loss": 0.8686,
+      "step": 1840
+    },
+    {
+      "epoch": 2.8505392912172574,
+      "grad_norm": 1.0252294540405273,
+      "learning_rate": 0.0002,
+      "loss": 0.8931,
+      "step": 1850
+    },
+    {
+      "epoch": 2.8659476117103235,
+      "grad_norm": 1.442360281944275,
+      "learning_rate": 0.0002,
+      "loss": 0.9265,
+      "step": 1860
+    },
+    {
+      "epoch": 2.8813559322033897,
+      "grad_norm": 0.8977653384208679,
+      "learning_rate": 0.0002,
+      "loss": 0.8893,
+      "step": 1870
+    },
+    {
+      "epoch": 2.896764252696456,
+      "grad_norm": 1.0946455001831055,
+      "learning_rate": 0.0002,
+      "loss": 0.8779,
+      "step": 1880
+    },
+    {
+      "epoch": 2.9121725731895225,
+      "grad_norm": 0.9993954300880432,
+      "learning_rate": 0.0002,
+      "loss": 0.8775,
+      "step": 1890
+    },
+    {
+      "epoch": 2.9275808936825887,
+      "grad_norm": 1.1941032409667969,
+      "learning_rate": 0.0002,
+      "loss": 0.8004,
+      "step": 1900
+    },
+    {
+      "epoch": 2.942989214175655,
+      "grad_norm": 1.5158019065856934,
+      "learning_rate": 0.0002,
+      "loss": 0.8862,
+      "step": 1910
+    },
+    {
+      "epoch": 2.958397534668721,
+      "grad_norm": 1.4000548124313354,
+      "learning_rate": 0.0002,
+      "loss": 0.9363,
+      "step": 1920
+    },
+    {
+      "epoch": 2.973805855161787,
+      "grad_norm": 0.9906305074691772,
+      "learning_rate": 0.0002,
+      "loss": 0.8666,
+      "step": 1930
+    },
+    {
+      "epoch": 2.989214175654854,
+      "grad_norm": 1.7382938861846924,
+      "learning_rate": 0.0002,
+      "loss": 0.8434,
+      "step": 1940
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 1.2514833211898804,
+      "eval_runtime": 119.8709,
+      "eval_samples_per_second": 3.804,
+      "eval_steps_per_second": 0.476,
+      "step": 1947
+    },
+    {
+      "epoch": 3.00462249614792,
+      "grad_norm": 0.9265438318252563,
+      "learning_rate": 0.0002,
+      "loss": 0.8099,
+      "step": 1950
+    },
+    {
+      "epoch": 3.020030816640986,
+      "grad_norm": 1.2885810136795044,
+      "learning_rate": 0.0002,
+      "loss": 0.6058,
+      "step": 1960
+    },
+    {
+      "epoch": 3.0354391371340523,
+      "grad_norm": 1.150202751159668,
+      "learning_rate": 0.0002,
+      "loss": 0.6065,
+      "step": 1970
+    },
+    {
+      "epoch": 3.0508474576271185,
+      "grad_norm": 1.4145002365112305,
+      "learning_rate": 0.0002,
+      "loss": 0.7225,
+      "step": 1980
+    },
+    {
+      "epoch": 3.066255778120185,
+      "grad_norm": 1.3523074388504028,
+      "learning_rate": 0.0002,
+      "loss": 0.6403,
+      "step": 1990
+    },
+    {
+      "epoch": 3.0816640986132513,
+      "grad_norm": 1.4055354595184326,
+      "learning_rate": 0.0002,
+      "loss": 0.6784,
+      "step": 2000
+    },
+    {
+      "epoch": 3.0970724191063175,
+      "grad_norm": 1.3167393207550049,
+      "learning_rate": 0.0002,
+      "loss": 0.617,
+      "step": 2010
+    },
+    {
+      "epoch": 3.1124807395993837,
+      "grad_norm": 0.8515956997871399,
+      "learning_rate": 0.0002,
+      "loss": 0.6228,
+      "step": 2020
+    },
+    {
+      "epoch": 3.12788906009245,
+      "grad_norm": 1.596145510673523,
+      "learning_rate": 0.0002,
+      "loss": 0.722,
+      "step": 2030
+    },
+    {
+      "epoch": 3.143297380585516,
+      "grad_norm": 1.3406099081039429,
+      "learning_rate": 0.0002,
+      "loss": 0.6582,
+      "step": 2040
+    },
+    {
+      "epoch": 3.1587057010785826,
+      "grad_norm": 1.5566312074661255,
+      "learning_rate": 0.0002,
+      "loss": 0.661,
+      "step": 2050
+    },
+    {
+      "epoch": 3.174114021571649,
+      "grad_norm": 1.060834288597107,
+      "learning_rate": 0.0002,
+      "loss": 0.648,
+      "step": 2060
+    },
+    {
+      "epoch": 3.189522342064715,
+      "grad_norm": 1.534487247467041,
+      "learning_rate": 0.0002,
+      "loss": 0.7131,
+      "step": 2070
+    },
+    {
+      "epoch": 3.204930662557781,
+      "grad_norm": 1.4387457370758057,
+      "learning_rate": 0.0002,
+      "loss": 0.6465,
+      "step": 2080
+    },
+    {
+      "epoch": 3.2203389830508473,
+      "grad_norm": 1.6553950309753418,
+      "learning_rate": 0.0002,
+      "loss": 0.651,
+      "step": 2090
+    },
+    {
+      "epoch": 3.2357473035439135,
+      "grad_norm": 1.4905112981796265,
+      "learning_rate": 0.0002,
+      "loss": 0.6329,
+      "step": 2100
+    },
+    {
+      "epoch": 3.25115562403698,
+      "grad_norm": 1.5865840911865234,
+      "learning_rate": 0.0002,
+      "loss": 0.6835,
+      "step": 2110
+    },
+    {
+      "epoch": 3.2665639445300463,
+      "grad_norm": 1.7313302755355835,
+      "learning_rate": 0.0002,
+      "loss": 0.65,
+      "step": 2120
+    },
+    {
+      "epoch": 3.2819722650231125,
+      "grad_norm": 1.3718984127044678,
+      "learning_rate": 0.0002,
+      "loss": 0.6401,
+      "step": 2130
+    },
+    {
+      "epoch": 3.2973805855161786,
+      "grad_norm": 1.4081032276153564,
+      "learning_rate": 0.0002,
+      "loss": 0.6794,
+      "step": 2140
+    },
+    {
+      "epoch": 3.312788906009245,
+      "grad_norm": 1.2884169816970825,
+      "learning_rate": 0.0002,
+      "loss": 0.7119,
+      "step": 2150
+    },
+    {
+      "epoch": 3.3281972265023114,
+      "grad_norm": 1.0338295698165894,
+      "learning_rate": 0.0002,
+      "loss": 0.6476,
+      "step": 2160
+    },
+    {
+      "epoch": 3.3436055469953776,
+      "grad_norm": 1.5023012161254883,
+      "learning_rate": 0.0002,
+      "loss": 0.6544,
+      "step": 2170
+    },
+    {
+      "epoch": 3.3590138674884438,
+      "grad_norm": 1.1718626022338867,
+      "learning_rate": 0.0002,
+      "loss": 0.6606,
+      "step": 2180
+    },
+    {
+      "epoch": 3.37442218798151,
+      "grad_norm": 1.1776877641677856,
+      "learning_rate": 0.0002,
+      "loss": 0.6734,
+      "step": 2190
+    },
+    {
+      "epoch": 3.389830508474576,
+      "grad_norm": 1.1330585479736328,
+      "learning_rate": 0.0002,
+      "loss": 0.6301,
+      "step": 2200
+    },
+    {
+      "epoch": 3.4052388289676427,
+      "grad_norm": 2.024857997894287,
+      "learning_rate": 0.0002,
+      "loss": 0.7051,
+      "step": 2210
+    },
+    {
+      "epoch": 3.420647149460709,
+      "grad_norm": 1.3598058223724365,
+      "learning_rate": 0.0002,
+      "loss": 0.7233,
+      "step": 2220
+    },
+    {
+      "epoch": 3.436055469953775,
+      "grad_norm": 1.3725299835205078,
+      "learning_rate": 0.0002,
+      "loss": 0.6943,
+      "step": 2230
+    },
+    {
+      "epoch": 3.4514637904468413,
+      "grad_norm": 1.1065977811813354,
+      "learning_rate": 0.0002,
+      "loss": 0.6871,
+      "step": 2240
+    },
+    {
+      "epoch": 3.4668721109399074,
+      "grad_norm": 1.5074089765548706,
+      "learning_rate": 0.0002,
+      "loss": 0.6628,
+      "step": 2250
+    },
+    {
+      "epoch": 3.482280431432974,
+      "grad_norm": 1.114187479019165,
+      "learning_rate": 0.0002,
+      "loss": 0.6847,
+      "step": 2260
+    },
+    {
+      "epoch": 3.4976887519260402,
+      "grad_norm": 1.5719729661941528,
+      "learning_rate": 0.0002,
+      "loss": 0.6934,
+      "step": 2270
+    },
+    {
+      "epoch": 3.5130970724191064,
+      "grad_norm": 1.2377736568450928,
+      "learning_rate": 0.0002,
+      "loss": 0.7264,
+      "step": 2280
+    },
+    {
+      "epoch": 3.5285053929121726,
+      "grad_norm": 1.4295454025268555,
+      "learning_rate": 0.0002,
+      "loss": 0.6777,
+      "step": 2290
+    },
+    {
+      "epoch": 3.5439137134052388,
+      "grad_norm": 0.9240845441818237,
+      "learning_rate": 0.0002,
+      "loss": 0.6465,
+      "step": 2300
+    },
+    {
+      "epoch": 3.559322033898305,
+      "grad_norm": 1.5270299911499023,
+      "learning_rate": 0.0002,
+      "loss": 0.6627,
+      "step": 2310
+    },
+    {
+      "epoch": 3.574730354391371,
+      "grad_norm": 1.2820508480072021,
+      "learning_rate": 0.0002,
+      "loss": 0.7018,
+      "step": 2320
+    },
+    {
+      "epoch": 3.5901386748844377,
+      "grad_norm": 1.2932546138763428,
+      "learning_rate": 0.0002,
+      "loss": 0.7225,
+      "step": 2330
+    },
+    {
+      "epoch": 3.605546995377504,
+      "grad_norm": 1.3201853036880493,
+      "learning_rate": 0.0002,
+      "loss": 0.6815,
+      "step": 2340
+    },
+    {
+      "epoch": 3.62095531587057,
+      "grad_norm": 1.7489187717437744,
+      "learning_rate": 0.0002,
+      "loss": 0.6842,
+      "step": 2350
+    },
+    {
+      "epoch": 3.6363636363636362,
+      "grad_norm": 1.2384065389633179,
+      "learning_rate": 0.0002,
+      "loss": 0.7468,
+      "step": 2360
+    },
+    {
+      "epoch": 3.6517719568567024,
+      "grad_norm": 1.0870963335037231,
+      "learning_rate": 0.0002,
+      "loss": 0.6978,
+      "step": 2370
+    },
+    {
+      "epoch": 3.667180277349769,
+      "grad_norm": 1.4325937032699585,
+      "learning_rate": 0.0002,
+      "loss": 0.6214,
+      "step": 2380
+    },
+    {
+      "epoch": 3.682588597842835,
+      "grad_norm": 1.3232696056365967,
+      "learning_rate": 0.0002,
+      "loss": 0.6737,
+      "step": 2390
+    },
+    {
+      "epoch": 3.6979969183359014,
+      "grad_norm": 1.409141182899475,
+      "learning_rate": 0.0002,
+      "loss": 0.6643,
+      "step": 2400
+    },
+    {
+      "epoch": 3.7134052388289676,
+      "grad_norm": 1.4707226753234863,
+      "learning_rate": 0.0002,
+      "loss": 0.7069,
+      "step": 2410
+    },
+    {
+      "epoch": 3.7288135593220337,
+      "grad_norm": 1.3888318538665771,
+      "learning_rate": 0.0002,
+      "loss": 0.678,
+      "step": 2420
+    },
+    {
+      "epoch": 3.7442218798151004,
+      "grad_norm": 1.4682821035385132,
+      "learning_rate": 0.0002,
+      "loss": 0.6718,
+      "step": 2430
+    },
+    {
+      "epoch": 3.7596302003081665,
+      "grad_norm": 1.2217520475387573,
+      "learning_rate": 0.0002,
+      "loss": 0.7593,
+      "step": 2440
+    },
+    {
+      "epoch": 3.7750385208012327,
+      "grad_norm": 1.3346893787384033,
+      "learning_rate": 0.0002,
+      "loss": 0.6912,
+      "step": 2450
+    },
+    {
+      "epoch": 3.790446841294299,
+      "grad_norm": 1.4342081546783447,
+      "learning_rate": 0.0002,
+      "loss": 0.609,
+      "step": 2460
+    },
+    {
+      "epoch": 3.805855161787365,
+      "grad_norm": 1.5921152830123901,
+      "learning_rate": 0.0002,
+      "loss": 0.6985,
+      "step": 2470
+    },
+    {
+      "epoch": 3.8212634822804317,
+      "grad_norm": 1.7322826385498047,
+      "learning_rate": 0.0002,
+      "loss": 0.6751,
+      "step": 2480
+    },
+    {
+      "epoch": 3.836671802773498,
+      "grad_norm": 1.478434681892395,
+      "learning_rate": 0.0002,
+      "loss": 0.6862,
+      "step": 2490
+    },
+    {
+      "epoch": 3.852080123266564,
+      "grad_norm": 1.3945757150650024,
+      "learning_rate": 0.0002,
+      "loss": 0.6983,
+      "step": 2500
+    },
+    {
+      "epoch": 3.86748844375963,
+      "grad_norm": 1.584877848625183,
+      "learning_rate": 0.0002,
+      "loss": 0.7439,
+      "step": 2510
+    },
+    {
+      "epoch": 3.8828967642526964,
+      "grad_norm": 1.6307404041290283,
+      "learning_rate": 0.0002,
+      "loss": 0.7049,
+      "step": 2520
+    },
+    {
+      "epoch": 3.898305084745763,
+      "grad_norm": 1.4129183292388916,
+      "learning_rate": 0.0002,
+      "loss": 0.6823,
+      "step": 2530
+    },
+    {
+      "epoch": 3.9137134052388287,
+      "grad_norm": 1.7380712032318115,
+      "learning_rate": 0.0002,
+      "loss": 0.7092,
+      "step": 2540
+    },
+    {
+      "epoch": 3.9291217257318953,
+      "grad_norm": 1.4169772863388062,
+      "learning_rate": 0.0002,
+      "loss": 0.7277,
+      "step": 2550
+    },
+    {
+      "epoch": 3.9445300462249615,
+      "grad_norm": 2.520665168762207,
+      "learning_rate": 0.0002,
+      "loss": 0.7369,
+      "step": 2560
+    },
+    {
+      "epoch": 3.9599383667180277,
+      "grad_norm": 1.2402763366699219,
+      "learning_rate": 0.0002,
+      "loss": 0.7815,
+      "step": 2570
+    },
+    {
+      "epoch": 3.975346687211094,
+      "grad_norm": 1.5902904272079468,
+      "learning_rate": 0.0002,
+      "loss": 0.8077,
+      "step": 2580
+    },
+    {
+      "epoch": 3.99075500770416,
+      "grad_norm": 2.2361271381378174,
+      "learning_rate": 0.0002,
+      "loss": 0.7272,
+      "step": 2590
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.324176549911499,
+      "eval_runtime": 119.304,
+      "eval_samples_per_second": 3.822,
+      "eval_steps_per_second": 0.478,
+      "step": 2596
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 5192,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.138987905318912e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-2596/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7da6f114429a9a993fcd6ed877a996e6a07ac3f427895c4f50ec6990a9e85e69
+size 5560

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-3245/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-3245/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-3245/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8376e6e852b7a35775bba9f9cd253c6c6f96a59156be0bf4c000538317cc0f1
+size 109069176

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-3245/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2715435ab6c945a12d58e85524c47e8057658981a8747ff049d1d279f6e69bb4
+size 55532666

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-3245/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8807b75eef001623ea445effcb3a4d71a34cccbc985a8af64b260a098158c92f
+size 14244

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-3245/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:43c37addf4b9e6833afbd9789c9c585b9832ff3d163740236c8ce47c5bdb61ee
+size 1064

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-3245/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-3245/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-3245/trainer_state.json

@@ -0,0 +1,2341 @@
+{
+  "best_metric": 1.212496042251587,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.4-num-3923-sd-1/checkpoint-649",
+  "epoch": 5.0,
+  "eval_steps": 10,
+  "global_step": 3245,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.015408320493066256,
+      "grad_norm": 0.939333975315094,
+      "learning_rate": 0.0002,
+      "loss": 1.9609,
+      "step": 10
+    },
+    {
+      "epoch": 0.030816640986132512,
+      "grad_norm": 0.9446700215339661,
+      "learning_rate": 0.0002,
+      "loss": 1.5975,
+      "step": 20
+    },
+    {
+      "epoch": 0.046224961479198766,
+      "grad_norm": 0.7268466353416443,
+      "learning_rate": 0.0002,
+      "loss": 1.5196,
+      "step": 30
+    },
+    {
+      "epoch": 0.061633281972265024,
+      "grad_norm": 0.8900066018104553,
+      "learning_rate": 0.0002,
+      "loss": 1.5021,
+      "step": 40
+    },
+    {
+      "epoch": 0.07704160246533127,
+      "grad_norm": 4.246077060699463,
+      "learning_rate": 0.0002,
+      "loss": 1.4155,
+      "step": 50
+    },
+    {
+      "epoch": 0.09244992295839753,
+      "grad_norm": 0.7887561321258545,
+      "learning_rate": 0.0002,
+      "loss": 1.4052,
+      "step": 60
+    },
+    {
+      "epoch": 0.10785824345146379,
+      "grad_norm": 0.6822794675827026,
+      "learning_rate": 0.0002,
+      "loss": 1.215,
+      "step": 70
+    },
+    {
+      "epoch": 0.12326656394453005,
+      "grad_norm": 0.8871720433235168,
+      "learning_rate": 0.0002,
+      "loss": 1.2524,
+      "step": 80
+    },
+    {
+      "epoch": 0.1386748844375963,
+      "grad_norm": 0.6524078249931335,
+      "learning_rate": 0.0002,
+      "loss": 1.191,
+      "step": 90
+    },
+    {
+      "epoch": 0.15408320493066255,
+      "grad_norm": 0.5519863367080688,
+      "learning_rate": 0.0002,
+      "loss": 1.2455,
+      "step": 100
+    },
+    {
+      "epoch": 0.1694915254237288,
+      "grad_norm": 0.6716212630271912,
+      "learning_rate": 0.0002,
+      "loss": 1.2515,
+      "step": 110
+    },
+    {
+      "epoch": 0.18489984591679506,
+      "grad_norm": 1.527921199798584,
+      "learning_rate": 0.0002,
+      "loss": 1.2239,
+      "step": 120
+    },
+    {
+      "epoch": 0.20030816640986132,
+      "grad_norm": 0.6893861293792725,
+      "learning_rate": 0.0002,
+      "loss": 1.2887,
+      "step": 130
+    },
+    {
+      "epoch": 0.21571648690292758,
+      "grad_norm": 0.5514246821403503,
+      "learning_rate": 0.0002,
+      "loss": 1.1722,
+      "step": 140
+    },
+    {
+      "epoch": 0.23112480739599384,
+      "grad_norm": 0.5830065608024597,
+      "learning_rate": 0.0002,
+      "loss": 1.1946,
+      "step": 150
+    },
+    {
+      "epoch": 0.2465331278890601,
+      "grad_norm": 0.7411216497421265,
+      "learning_rate": 0.0002,
+      "loss": 1.3105,
+      "step": 160
+    },
+    {
+      "epoch": 0.26194144838212635,
+      "grad_norm": 0.7470705509185791,
+      "learning_rate": 0.0002,
+      "loss": 1.2008,
+      "step": 170
+    },
+    {
+      "epoch": 0.2773497688751926,
+      "grad_norm": 0.6140725016593933,
+      "learning_rate": 0.0002,
+      "loss": 1.2497,
+      "step": 180
+    },
+    {
+      "epoch": 0.29275808936825887,
+      "grad_norm": 0.7563071250915527,
+      "learning_rate": 0.0002,
+      "loss": 1.1514,
+      "step": 190
+    },
+    {
+      "epoch": 0.3081664098613251,
+      "grad_norm": 0.5491266250610352,
+      "learning_rate": 0.0002,
+      "loss": 1.2703,
+      "step": 200
+    },
+    {
+      "epoch": 0.3235747303543914,
+      "grad_norm": 0.5403220057487488,
+      "learning_rate": 0.0002,
+      "loss": 1.2168,
+      "step": 210
+    },
+    {
+      "epoch": 0.3389830508474576,
+      "grad_norm": 0.9616869688034058,
+      "learning_rate": 0.0002,
+      "loss": 1.2409,
+      "step": 220
+    },
+    {
+      "epoch": 0.3543913713405239,
+      "grad_norm": 0.5448639988899231,
+      "learning_rate": 0.0002,
+      "loss": 1.3112,
+      "step": 230
+    },
+    {
+      "epoch": 0.3697996918335901,
+      "grad_norm": 0.6322043538093567,
+      "learning_rate": 0.0002,
+      "loss": 1.3023,
+      "step": 240
+    },
+    {
+      "epoch": 0.3852080123266564,
+      "grad_norm": 0.5178093314170837,
+      "learning_rate": 0.0002,
+      "loss": 1.2382,
+      "step": 250
+    },
+    {
+      "epoch": 0.40061633281972264,
+      "grad_norm": 0.6274669766426086,
+      "learning_rate": 0.0002,
+      "loss": 1.2755,
+      "step": 260
+    },
+    {
+      "epoch": 0.41602465331278893,
+      "grad_norm": 1.1093109846115112,
+      "learning_rate": 0.0002,
+      "loss": 1.3263,
+      "step": 270
+    },
+    {
+      "epoch": 0.43143297380585516,
+      "grad_norm": 1.8002032041549683,
+      "learning_rate": 0.0002,
+      "loss": 1.2201,
+      "step": 280
+    },
+    {
+      "epoch": 0.44684129429892144,
+      "grad_norm": 0.5977614521980286,
+      "learning_rate": 0.0002,
+      "loss": 1.2335,
+      "step": 290
+    },
+    {
+      "epoch": 0.4622496147919877,
+      "grad_norm": 0.5583769083023071,
+      "learning_rate": 0.0002,
+      "loss": 1.2076,
+      "step": 300
+    },
+    {
+      "epoch": 0.4776579352850539,
+      "grad_norm": 0.5563502311706543,
+      "learning_rate": 0.0002,
+      "loss": 1.197,
+      "step": 310
+    },
+    {
+      "epoch": 0.4930662557781202,
+      "grad_norm": 0.9157887101173401,
+      "learning_rate": 0.0002,
+      "loss": 1.164,
+      "step": 320
+    },
+    {
+      "epoch": 0.5084745762711864,
+      "grad_norm": 0.551705539226532,
+      "learning_rate": 0.0002,
+      "loss": 1.1266,
+      "step": 330
+    },
+    {
+      "epoch": 0.5238828967642527,
+      "grad_norm": 9.011388778686523,
+      "learning_rate": 0.0002,
+      "loss": 1.2801,
+      "step": 340
+    },
+    {
+      "epoch": 0.539291217257319,
+      "grad_norm": 0.6527810096740723,
+      "learning_rate": 0.0002,
+      "loss": 1.2024,
+      "step": 350
+    },
+    {
+      "epoch": 0.5546995377503852,
+      "grad_norm": 1.3121646642684937,
+      "learning_rate": 0.0002,
+      "loss": 1.1424,
+      "step": 360
+    },
+    {
+      "epoch": 0.5701078582434514,
+      "grad_norm": 0.7247968316078186,
+      "learning_rate": 0.0002,
+      "loss": 1.2555,
+      "step": 370
+    },
+    {
+      "epoch": 0.5855161787365177,
+      "grad_norm": 0.5430527925491333,
+      "learning_rate": 0.0002,
+      "loss": 1.196,
+      "step": 380
+    },
+    {
+      "epoch": 0.600924499229584,
+      "grad_norm": 0.6050328612327576,
+      "learning_rate": 0.0002,
+      "loss": 1.1822,
+      "step": 390
+    },
+    {
+      "epoch": 0.6163328197226502,
+      "grad_norm": 0.49382615089416504,
+      "learning_rate": 0.0002,
+      "loss": 1.2754,
+      "step": 400
+    },
+    {
+      "epoch": 0.6317411402157165,
+      "grad_norm": 0.5804041624069214,
+      "learning_rate": 0.0002,
+      "loss": 1.2595,
+      "step": 410
+    },
+    {
+      "epoch": 0.6471494607087828,
+      "grad_norm": 0.524357259273529,
+      "learning_rate": 0.0002,
+      "loss": 1.2926,
+      "step": 420
+    },
+    {
+      "epoch": 0.662557781201849,
+      "grad_norm": 1.8817486763000488,
+      "learning_rate": 0.0002,
+      "loss": 1.1928,
+      "step": 430
+    },
+    {
+      "epoch": 0.6779661016949152,
+      "grad_norm": 0.6104950308799744,
+      "learning_rate": 0.0002,
+      "loss": 1.2103,
+      "step": 440
+    },
+    {
+      "epoch": 0.6933744221879815,
+      "grad_norm": 0.631179928779602,
+      "learning_rate": 0.0002,
+      "loss": 1.2296,
+      "step": 450
+    },
+    {
+      "epoch": 0.7087827426810478,
+      "grad_norm": 0.6562483310699463,
+      "learning_rate": 0.0002,
+      "loss": 1.1843,
+      "step": 460
+    },
+    {
+      "epoch": 0.724191063174114,
+      "grad_norm": 0.578336775302887,
+      "learning_rate": 0.0002,
+      "loss": 1.2424,
+      "step": 470
+    },
+    {
+      "epoch": 0.7395993836671803,
+      "grad_norm": 0.8468874096870422,
+      "learning_rate": 0.0002,
+      "loss": 1.192,
+      "step": 480
+    },
+    {
+      "epoch": 0.7550077041602465,
+      "grad_norm": 0.7162539958953857,
+      "learning_rate": 0.0002,
+      "loss": 1.2913,
+      "step": 490
+    },
+    {
+      "epoch": 0.7704160246533128,
+      "grad_norm": 0.7655543684959412,
+      "learning_rate": 0.0002,
+      "loss": 1.2158,
+      "step": 500
+    },
+    {
+      "epoch": 0.785824345146379,
+      "grad_norm": 0.5711562037467957,
+      "learning_rate": 0.0002,
+      "loss": 1.2637,
+      "step": 510
+    },
+    {
+      "epoch": 0.8012326656394453,
+      "grad_norm": 0.7273485660552979,
+      "learning_rate": 0.0002,
+      "loss": 1.1277,
+      "step": 520
+    },
+    {
+      "epoch": 0.8166409861325116,
+      "grad_norm": 0.47050145268440247,
+      "learning_rate": 0.0002,
+      "loss": 1.188,
+      "step": 530
+    },
+    {
+      "epoch": 0.8320493066255779,
+      "grad_norm": 0.4930959939956665,
+      "learning_rate": 0.0002,
+      "loss": 1.1462,
+      "step": 540
+    },
+    {
+      "epoch": 0.847457627118644,
+      "grad_norm": 0.553955614566803,
+      "learning_rate": 0.0002,
+      "loss": 1.167,
+      "step": 550
+    },
+    {
+      "epoch": 0.8628659476117103,
+      "grad_norm": 0.6495056748390198,
+      "learning_rate": 0.0002,
+      "loss": 1.1831,
+      "step": 560
+    },
+    {
+      "epoch": 0.8782742681047766,
+      "grad_norm": 0.4586578905582428,
+      "learning_rate": 0.0002,
+      "loss": 1.0984,
+      "step": 570
+    },
+    {
+      "epoch": 0.8936825885978429,
+      "grad_norm": 0.4870392680168152,
+      "learning_rate": 0.0002,
+      "loss": 1.2415,
+      "step": 580
+    },
+    {
+      "epoch": 0.9090909090909091,
+      "grad_norm": 0.6337013244628906,
+      "learning_rate": 0.0002,
+      "loss": 1.1735,
+      "step": 590
+    },
+    {
+      "epoch": 0.9244992295839753,
+      "grad_norm": 0.7260186672210693,
+      "learning_rate": 0.0002,
+      "loss": 1.1781,
+      "step": 600
+    },
+    {
+      "epoch": 0.9399075500770416,
+      "grad_norm": 1.4133737087249756,
+      "learning_rate": 0.0002,
+      "loss": 1.2536,
+      "step": 610
+    },
+    {
+      "epoch": 0.9553158705701078,
+      "grad_norm": 0.573525071144104,
+      "learning_rate": 0.0002,
+      "loss": 1.2211,
+      "step": 620
+    },
+    {
+      "epoch": 0.9707241910631741,
+      "grad_norm": 0.8403644561767578,
+      "learning_rate": 0.0002,
+      "loss": 1.11,
+      "step": 630
+    },
+    {
+      "epoch": 0.9861325115562404,
+      "grad_norm": 0.7665438055992126,
+      "learning_rate": 0.0002,
+      "loss": 1.2932,
+      "step": 640
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.212496042251587,
+      "eval_runtime": 120.3264,
+      "eval_samples_per_second": 3.79,
+      "eval_steps_per_second": 0.474,
+      "step": 649
+    },
+    {
+      "epoch": 1.0015408320493067,
+      "grad_norm": 0.6874801516532898,
+      "learning_rate": 0.0002,
+      "loss": 1.176,
+      "step": 650
+    },
+    {
+      "epoch": 1.0169491525423728,
+      "grad_norm": 0.6675221920013428,
+      "learning_rate": 0.0002,
+      "loss": 1.0957,
+      "step": 660
+    },
+    {
+      "epoch": 1.0323574730354392,
+      "grad_norm": 0.6837265491485596,
+      "learning_rate": 0.0002,
+      "loss": 1.1616,
+      "step": 670
+    },
+    {
+      "epoch": 1.0477657935285054,
+      "grad_norm": 0.6796931028366089,
+      "learning_rate": 0.0002,
+      "loss": 1.045,
+      "step": 680
+    },
+    {
+      "epoch": 1.0631741140215716,
+      "grad_norm": 0.6596675515174866,
+      "learning_rate": 0.0002,
+      "loss": 1.0889,
+      "step": 690
+    },
+    {
+      "epoch": 1.078582434514638,
+      "grad_norm": 0.6630653142929077,
+      "learning_rate": 0.0002,
+      "loss": 1.067,
+      "step": 700
+    },
+    {
+      "epoch": 1.0939907550077042,
+      "grad_norm": 0.7065498232841492,
+      "learning_rate": 0.0002,
+      "loss": 1.0222,
+      "step": 710
+    },
+    {
+      "epoch": 1.1093990755007703,
+      "grad_norm": 0.694682240486145,
+      "learning_rate": 0.0002,
+      "loss": 1.081,
+      "step": 720
+    },
+    {
+      "epoch": 1.1248073959938367,
+      "grad_norm": 0.8984217047691345,
+      "learning_rate": 0.0002,
+      "loss": 1.1019,
+      "step": 730
+    },
+    {
+      "epoch": 1.140215716486903,
+      "grad_norm": 1.0035051107406616,
+      "learning_rate": 0.0002,
+      "loss": 1.0506,
+      "step": 740
+    },
+    {
+      "epoch": 1.1556240369799693,
+      "grad_norm": 0.8646948933601379,
+      "learning_rate": 0.0002,
+      "loss": 1.0502,
+      "step": 750
+    },
+    {
+      "epoch": 1.1710323574730355,
+      "grad_norm": 0.6287558674812317,
+      "learning_rate": 0.0002,
+      "loss": 1.0256,
+      "step": 760
+    },
+    {
+      "epoch": 1.1864406779661016,
+      "grad_norm": 0.6975560784339905,
+      "learning_rate": 0.0002,
+      "loss": 1.051,
+      "step": 770
+    },
+    {
+      "epoch": 1.201848998459168,
+      "grad_norm": 0.5972192883491516,
+      "learning_rate": 0.0002,
+      "loss": 1.0409,
+      "step": 780
+    },
+    {
+      "epoch": 1.2172573189522342,
+      "grad_norm": 0.7299932241439819,
+      "learning_rate": 0.0002,
+      "loss": 1.0837,
+      "step": 790
+    },
+    {
+      "epoch": 1.2326656394453004,
+      "grad_norm": 0.8376814126968384,
+      "learning_rate": 0.0002,
+      "loss": 1.1291,
+      "step": 800
+    },
+    {
+      "epoch": 1.2480739599383668,
+      "grad_norm": 1.117690086364746,
+      "learning_rate": 0.0002,
+      "loss": 1.0809,
+      "step": 810
+    },
+    {
+      "epoch": 1.263482280431433,
+      "grad_norm": 0.7850839495658875,
+      "learning_rate": 0.0002,
+      "loss": 1.0762,
+      "step": 820
+    },
+    {
+      "epoch": 1.2788906009244991,
+      "grad_norm": 0.7941734790802002,
+      "learning_rate": 0.0002,
+      "loss": 1.2278,
+      "step": 830
+    },
+    {
+      "epoch": 1.2942989214175655,
+      "grad_norm": 0.5682743787765503,
+      "learning_rate": 0.0002,
+      "loss": 0.9723,
+      "step": 840
+    },
+    {
+      "epoch": 1.3097072419106317,
+      "grad_norm": 0.7182576060295105,
+      "learning_rate": 0.0002,
+      "loss": 1.0262,
+      "step": 850
+    },
+    {
+      "epoch": 1.325115562403698,
+      "grad_norm": 0.7436819672584534,
+      "learning_rate": 0.0002,
+      "loss": 1.0801,
+      "step": 860
+    },
+    {
+      "epoch": 1.3405238828967643,
+      "grad_norm": 0.7665132880210876,
+      "learning_rate": 0.0002,
+      "loss": 1.0705,
+      "step": 870
+    },
+    {
+      "epoch": 1.3559322033898304,
+      "grad_norm": 1.060669183731079,
+      "learning_rate": 0.0002,
+      "loss": 1.1925,
+      "step": 880
+    },
+    {
+      "epoch": 1.3713405238828968,
+      "grad_norm": 0.6262772083282471,
+      "learning_rate": 0.0002,
+      "loss": 1.0581,
+      "step": 890
+    },
+    {
+      "epoch": 1.386748844375963,
+      "grad_norm": 1.010703682899475,
+      "learning_rate": 0.0002,
+      "loss": 1.0934,
+      "step": 900
+    },
+    {
+      "epoch": 1.4021571648690292,
+      "grad_norm": 0.5669811367988586,
+      "learning_rate": 0.0002,
+      "loss": 1.111,
+      "step": 910
+    },
+    {
+      "epoch": 1.4175654853620956,
+      "grad_norm": 0.8644649386405945,
+      "learning_rate": 0.0002,
+      "loss": 1.0638,
+      "step": 920
+    },
+    {
+      "epoch": 1.4329738058551618,
+      "grad_norm": 0.6250067949295044,
+      "learning_rate": 0.0002,
+      "loss": 1.1973,
+      "step": 930
+    },
+    {
+      "epoch": 1.448382126348228,
+      "grad_norm": 0.6507946848869324,
+      "learning_rate": 0.0002,
+      "loss": 1.0592,
+      "step": 940
+    },
+    {
+      "epoch": 1.4637904468412943,
+      "grad_norm": 0.9305022954940796,
+      "learning_rate": 0.0002,
+      "loss": 1.0306,
+      "step": 950
+    },
+    {
+      "epoch": 1.4791987673343605,
+      "grad_norm": 0.86021888256073,
+      "learning_rate": 0.0002,
+      "loss": 0.9792,
+      "step": 960
+    },
+    {
+      "epoch": 1.494607087827427,
+      "grad_norm": 1.1122219562530518,
+      "learning_rate": 0.0002,
+      "loss": 1.0656,
+      "step": 970
+    },
+    {
+      "epoch": 1.510015408320493,
+      "grad_norm": 0.7431066036224365,
+      "learning_rate": 0.0002,
+      "loss": 1.0745,
+      "step": 980
+    },
+    {
+      "epoch": 1.5254237288135593,
+      "grad_norm": 1.022744059562683,
+      "learning_rate": 0.0002,
+      "loss": 1.0822,
+      "step": 990
+    },
+    {
+      "epoch": 1.5408320493066254,
+      "grad_norm": 0.6638363599777222,
+      "learning_rate": 0.0002,
+      "loss": 1.0689,
+      "step": 1000
+    },
+    {
+      "epoch": 1.5562403697996918,
+      "grad_norm": 0.6290505528450012,
+      "learning_rate": 0.0002,
+      "loss": 1.1032,
+      "step": 1010
+    },
+    {
+      "epoch": 1.5716486902927582,
+      "grad_norm": 0.6446037888526917,
+      "learning_rate": 0.0002,
+      "loss": 1.0572,
+      "step": 1020
+    },
+    {
+      "epoch": 1.5870570107858244,
+      "grad_norm": 0.7383618354797363,
+      "learning_rate": 0.0002,
+      "loss": 1.0814,
+      "step": 1030
+    },
+    {
+      "epoch": 1.6024653312788906,
+      "grad_norm": 3.5893094539642334,
+      "learning_rate": 0.0002,
+      "loss": 1.0643,
+      "step": 1040
+    },
+    {
+      "epoch": 1.6178736517719567,
+      "grad_norm": 0.7388538718223572,
+      "learning_rate": 0.0002,
+      "loss": 0.9806,
+      "step": 1050
+    },
+    {
+      "epoch": 1.6332819722650231,
+      "grad_norm": 0.7813242077827454,
+      "learning_rate": 0.0002,
+      "loss": 1.1,
+      "step": 1060
+    },
+    {
+      "epoch": 1.6486902927580893,
+      "grad_norm": 0.6643418669700623,
+      "learning_rate": 0.0002,
+      "loss": 1.0463,
+      "step": 1070
+    },
+    {
+      "epoch": 1.6640986132511557,
+      "grad_norm": 0.7817712426185608,
+      "learning_rate": 0.0002,
+      "loss": 1.0344,
+      "step": 1080
+    },
+    {
+      "epoch": 1.6795069337442219,
+      "grad_norm": 0.8741177916526794,
+      "learning_rate": 0.0002,
+      "loss": 1.0447,
+      "step": 1090
+    },
+    {
+      "epoch": 1.694915254237288,
+      "grad_norm": 0.969813883304596,
+      "learning_rate": 0.0002,
+      "loss": 1.0724,
+      "step": 1100
+    },
+    {
+      "epoch": 1.7103235747303542,
+      "grad_norm": 0.7666333913803101,
+      "learning_rate": 0.0002,
+      "loss": 0.9887,
+      "step": 1110
+    },
+    {
+      "epoch": 1.7257318952234206,
+      "grad_norm": 0.789859414100647,
+      "learning_rate": 0.0002,
+      "loss": 0.9159,
+      "step": 1120
+    },
+    {
+      "epoch": 1.741140215716487,
+      "grad_norm": 0.7018347382545471,
+      "learning_rate": 0.0002,
+      "loss": 1.0679,
+      "step": 1130
+    },
+    {
+      "epoch": 1.7565485362095532,
+      "grad_norm": 0.8899882435798645,
+      "learning_rate": 0.0002,
+      "loss": 1.1429,
+      "step": 1140
+    },
+    {
+      "epoch": 1.7719568567026194,
+      "grad_norm": 0.7738426923751831,
+      "learning_rate": 0.0002,
+      "loss": 0.9214,
+      "step": 1150
+    },
+    {
+      "epoch": 1.7873651771956856,
+      "grad_norm": 1.032209873199463,
+      "learning_rate": 0.0002,
+      "loss": 1.0733,
+      "step": 1160
+    },
+    {
+      "epoch": 1.802773497688752,
+      "grad_norm": 0.7310711741447449,
+      "learning_rate": 0.0002,
+      "loss": 1.0522,
+      "step": 1170
+    },
+    {
+      "epoch": 1.8181818181818183,
+      "grad_norm": 0.8506733775138855,
+      "learning_rate": 0.0002,
+      "loss": 1.0943,
+      "step": 1180
+    },
+    {
+      "epoch": 1.8335901386748845,
+      "grad_norm": 116.00221252441406,
+      "learning_rate": 0.0002,
+      "loss": 1.1349,
+      "step": 1190
+    },
+    {
+      "epoch": 1.8489984591679507,
+      "grad_norm": 0.8819684982299805,
+      "learning_rate": 0.0002,
+      "loss": 1.0595,
+      "step": 1200
+    },
+    {
+      "epoch": 1.8644067796610169,
+      "grad_norm": 0.8846774697303772,
+      "learning_rate": 0.0002,
+      "loss": 0.995,
+      "step": 1210
+    },
+    {
+      "epoch": 1.879815100154083,
+      "grad_norm": 1.6417866945266724,
+      "learning_rate": 0.0002,
+      "loss": 1.116,
+      "step": 1220
+    },
+    {
+      "epoch": 1.8952234206471494,
+      "grad_norm": 1.1875122785568237,
+      "learning_rate": 0.0002,
+      "loss": 1.044,
+      "step": 1230
+    },
+    {
+      "epoch": 1.9106317411402158,
+      "grad_norm": 0.6520240306854248,
+      "learning_rate": 0.0002,
+      "loss": 1.0322,
+      "step": 1240
+    },
+    {
+      "epoch": 1.926040061633282,
+      "grad_norm": 0.9059738516807556,
+      "learning_rate": 0.0002,
+      "loss": 1.0175,
+      "step": 1250
+    },
+    {
+      "epoch": 1.9414483821263482,
+      "grad_norm": 0.7950983643531799,
+      "learning_rate": 0.0002,
+      "loss": 1.1097,
+      "step": 1260
+    },
+    {
+      "epoch": 1.9568567026194144,
+      "grad_norm": 0.6911653280258179,
+      "learning_rate": 0.0002,
+      "loss": 1.0313,
+      "step": 1270
+    },
+    {
+      "epoch": 1.9722650231124808,
+      "grad_norm": 0.9487978219985962,
+      "learning_rate": 0.0002,
+      "loss": 1.0762,
+      "step": 1280
+    },
+    {
+      "epoch": 1.9876733436055471,
+      "grad_norm": 0.716439425945282,
+      "learning_rate": 0.0002,
+      "loss": 1.1608,
+      "step": 1290
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.215390682220459,
+      "eval_runtime": 120.3893,
+      "eval_samples_per_second": 3.788,
+      "eval_steps_per_second": 0.473,
+      "step": 1298
+    },
+    {
+      "epoch": 2.0030816640986133,
+      "grad_norm": 0.7292942404747009,
+      "learning_rate": 0.0002,
+      "loss": 1.0205,
+      "step": 1300
+    },
+    {
+      "epoch": 2.0184899845916795,
+      "grad_norm": 0.8386905789375305,
+      "learning_rate": 0.0002,
+      "loss": 0.8264,
+      "step": 1310
+    },
+    {
+      "epoch": 2.0338983050847457,
+      "grad_norm": 1.114105224609375,
+      "learning_rate": 0.0002,
+      "loss": 0.7925,
+      "step": 1320
+    },
+    {
+      "epoch": 2.049306625577812,
+      "grad_norm": 1.068211317062378,
+      "learning_rate": 0.0002,
+      "loss": 0.8673,
+      "step": 1330
+    },
+    {
+      "epoch": 2.0647149460708785,
+      "grad_norm": 0.8620569109916687,
+      "learning_rate": 0.0002,
+      "loss": 0.8883,
+      "step": 1340
+    },
+    {
+      "epoch": 2.0801232665639446,
+      "grad_norm": 0.8984315991401672,
+      "learning_rate": 0.0002,
+      "loss": 0.8949,
+      "step": 1350
+    },
+    {
+      "epoch": 2.095531587057011,
+      "grad_norm": 1.0469365119934082,
+      "learning_rate": 0.0002,
+      "loss": 0.8992,
+      "step": 1360
+    },
+    {
+      "epoch": 2.110939907550077,
+      "grad_norm": 0.9392943978309631,
+      "learning_rate": 0.0002,
+      "loss": 0.8053,
+      "step": 1370
+    },
+    {
+      "epoch": 2.126348228043143,
+      "grad_norm": 1.2067331075668335,
+      "learning_rate": 0.0002,
+      "loss": 0.8791,
+      "step": 1380
+    },
+    {
+      "epoch": 2.1417565485362093,
+      "grad_norm": 0.8019351363182068,
+      "learning_rate": 0.0002,
+      "loss": 0.7925,
+      "step": 1390
+    },
+    {
+      "epoch": 2.157164869029276,
+      "grad_norm": 0.9445130228996277,
+      "learning_rate": 0.0002,
+      "loss": 0.8669,
+      "step": 1400
+    },
+    {
+      "epoch": 2.172573189522342,
+      "grad_norm": 0.9806232452392578,
+      "learning_rate": 0.0002,
+      "loss": 0.8293,
+      "step": 1410
+    },
+    {
+      "epoch": 2.1879815100154083,
+      "grad_norm": 1.2135679721832275,
+      "learning_rate": 0.0002,
+      "loss": 0.8499,
+      "step": 1420
+    },
+    {
+      "epoch": 2.2033898305084745,
+      "grad_norm": 1.1036803722381592,
+      "learning_rate": 0.0002,
+      "loss": 0.8908,
+      "step": 1430
+    },
+    {
+      "epoch": 2.2187981510015407,
+      "grad_norm": 0.9235773086547852,
+      "learning_rate": 0.0002,
+      "loss": 0.9725,
+      "step": 1440
+    },
+    {
+      "epoch": 2.2342064714946073,
+      "grad_norm": 1.0520254373550415,
+      "learning_rate": 0.0002,
+      "loss": 0.8176,
+      "step": 1450
+    },
+    {
+      "epoch": 2.2496147919876734,
+      "grad_norm": 1.0121252536773682,
+      "learning_rate": 0.0002,
+      "loss": 0.9138,
+      "step": 1460
+    },
+    {
+      "epoch": 2.2650231124807396,
+      "grad_norm": 1.0718834400177002,
+      "learning_rate": 0.0002,
+      "loss": 0.8722,
+      "step": 1470
+    },
+    {
+      "epoch": 2.280431432973806,
+      "grad_norm": 1.1842551231384277,
+      "learning_rate": 0.0002,
+      "loss": 0.8295,
+      "step": 1480
+    },
+    {
+      "epoch": 2.295839753466872,
+      "grad_norm": 1.2985937595367432,
+      "learning_rate": 0.0002,
+      "loss": 0.8706,
+      "step": 1490
+    },
+    {
+      "epoch": 2.3112480739599386,
+      "grad_norm": 2.0254523754119873,
+      "learning_rate": 0.0002,
+      "loss": 0.873,
+      "step": 1500
+    },
+    {
+      "epoch": 2.3266563944530048,
+      "grad_norm": 1.1501885652542114,
+      "learning_rate": 0.0002,
+      "loss": 0.8569,
+      "step": 1510
+    },
+    {
+      "epoch": 2.342064714946071,
+      "grad_norm": 1.1277996301651,
+      "learning_rate": 0.0002,
+      "loss": 0.9394,
+      "step": 1520
+    },
+    {
+      "epoch": 2.357473035439137,
+      "grad_norm": 0.9330579042434692,
+      "learning_rate": 0.0002,
+      "loss": 0.8856,
+      "step": 1530
+    },
+    {
+      "epoch": 2.3728813559322033,
+      "grad_norm": 1.3631645441055298,
+      "learning_rate": 0.0002,
+      "loss": 0.8679,
+      "step": 1540
+    },
+    {
+      "epoch": 2.3882896764252695,
+      "grad_norm": 1.070383071899414,
+      "learning_rate": 0.0002,
+      "loss": 0.8644,
+      "step": 1550
+    },
+    {
+      "epoch": 2.403697996918336,
+      "grad_norm": 1.5154823064804077,
+      "learning_rate": 0.0002,
+      "loss": 0.9235,
+      "step": 1560
+    },
+    {
+      "epoch": 2.4191063174114023,
+      "grad_norm": 0.8797892332077026,
+      "learning_rate": 0.0002,
+      "loss": 0.9778,
+      "step": 1570
+    },
+    {
+      "epoch": 2.4345146379044684,
+      "grad_norm": 1.0940500497817993,
+      "learning_rate": 0.0002,
+      "loss": 0.8952,
+      "step": 1580
+    },
+    {
+      "epoch": 2.4499229583975346,
+      "grad_norm": 1.6111960411071777,
+      "learning_rate": 0.0002,
+      "loss": 0.9352,
+      "step": 1590
+    },
+    {
+      "epoch": 2.4653312788906008,
+      "grad_norm": 0.9618533849716187,
+      "learning_rate": 0.0002,
+      "loss": 0.8462,
+      "step": 1600
+    },
+    {
+      "epoch": 2.480739599383667,
+      "grad_norm": 1.3644909858703613,
+      "learning_rate": 0.0002,
+      "loss": 0.7929,
+      "step": 1610
+    },
+    {
+      "epoch": 2.4961479198767336,
+      "grad_norm": 0.9000744223594666,
+      "learning_rate": 0.0002,
+      "loss": 0.8482,
+      "step": 1620
+    },
+    {
+      "epoch": 2.5115562403697997,
+      "grad_norm": 1.2393828630447388,
+      "learning_rate": 0.0002,
+      "loss": 0.9031,
+      "step": 1630
+    },
+    {
+      "epoch": 2.526964560862866,
+      "grad_norm": 1.231459617614746,
+      "learning_rate": 0.0002,
+      "loss": 0.8986,
+      "step": 1640
+    },
+    {
+      "epoch": 2.542372881355932,
+      "grad_norm": 1.605279564857483,
+      "learning_rate": 0.0002,
+      "loss": 0.8904,
+      "step": 1650
+    },
+    {
+      "epoch": 2.5577812018489983,
+      "grad_norm": 0.9993997812271118,
+      "learning_rate": 0.0002,
+      "loss": 0.9044,
+      "step": 1660
+    },
+    {
+      "epoch": 2.573189522342065,
+      "grad_norm": 0.8446758985519409,
+      "learning_rate": 0.0002,
+      "loss": 0.8079,
+      "step": 1670
+    },
+    {
+      "epoch": 2.588597842835131,
+      "grad_norm": 0.9734597206115723,
+      "learning_rate": 0.0002,
+      "loss": 0.8295,
+      "step": 1680
+    },
+    {
+      "epoch": 2.6040061633281972,
+      "grad_norm": 0.9620639681816101,
+      "learning_rate": 0.0002,
+      "loss": 0.8905,
+      "step": 1690
+    },
+    {
+      "epoch": 2.6194144838212634,
+      "grad_norm": 1.2597887516021729,
+      "learning_rate": 0.0002,
+      "loss": 0.893,
+      "step": 1700
+    },
+    {
+      "epoch": 2.6348228043143296,
+      "grad_norm": 0.9959160685539246,
+      "learning_rate": 0.0002,
+      "loss": 0.8567,
+      "step": 1710
+    },
+    {
+      "epoch": 2.650231124807396,
+      "grad_norm": 1.729129672050476,
+      "learning_rate": 0.0002,
+      "loss": 0.7528,
+      "step": 1720
+    },
+    {
+      "epoch": 2.6656394453004624,
+      "grad_norm": 0.8466572761535645,
+      "learning_rate": 0.0002,
+      "loss": 0.8859,
+      "step": 1730
+    },
+    {
+      "epoch": 2.6810477657935285,
+      "grad_norm": 0.9277077913284302,
+      "learning_rate": 0.0002,
+      "loss": 0.9075,
+      "step": 1740
+    },
+    {
+      "epoch": 2.6964560862865947,
+      "grad_norm": 1.1184828281402588,
+      "learning_rate": 0.0002,
+      "loss": 0.8168,
+      "step": 1750
+    },
+    {
+      "epoch": 2.711864406779661,
+      "grad_norm": 1.0716012716293335,
+      "learning_rate": 0.0002,
+      "loss": 0.9063,
+      "step": 1760
+    },
+    {
+      "epoch": 2.7272727272727275,
+      "grad_norm": 1.8334401845932007,
+      "learning_rate": 0.0002,
+      "loss": 0.9066,
+      "step": 1770
+    },
+    {
+      "epoch": 2.7426810477657937,
+      "grad_norm": 1.032091736793518,
+      "learning_rate": 0.0002,
+      "loss": 0.8721,
+      "step": 1780
+    },
+    {
+      "epoch": 2.75808936825886,
+      "grad_norm": 1.0596864223480225,
+      "learning_rate": 0.0002,
+      "loss": 0.9065,
+      "step": 1790
+    },
+    {
+      "epoch": 2.773497688751926,
+      "grad_norm": 0.9625331163406372,
+      "learning_rate": 0.0002,
+      "loss": 0.8729,
+      "step": 1800
+    },
+    {
+      "epoch": 2.788906009244992,
+      "grad_norm": 1.1381267309188843,
+      "learning_rate": 0.0002,
+      "loss": 0.8942,
+      "step": 1810
+    },
+    {
+      "epoch": 2.8043143297380584,
+      "grad_norm": 1.0851205587387085,
+      "learning_rate": 0.0002,
+      "loss": 0.8753,
+      "step": 1820
+    },
+    {
+      "epoch": 2.8197226502311246,
+      "grad_norm": 0.9770793914794922,
+      "learning_rate": 0.0002,
+      "loss": 0.8532,
+      "step": 1830
+    },
+    {
+      "epoch": 2.835130970724191,
+      "grad_norm": 1.067268967628479,
+      "learning_rate": 0.0002,
+      "loss": 0.8686,
+      "step": 1840
+    },
+    {
+      "epoch": 2.8505392912172574,
+      "grad_norm": 1.0252294540405273,
+      "learning_rate": 0.0002,
+      "loss": 0.8931,
+      "step": 1850
+    },
+    {
+      "epoch": 2.8659476117103235,
+      "grad_norm": 1.442360281944275,
+      "learning_rate": 0.0002,
+      "loss": 0.9265,
+      "step": 1860
+    },
+    {
+      "epoch": 2.8813559322033897,
+      "grad_norm": 0.8977653384208679,
+      "learning_rate": 0.0002,
+      "loss": 0.8893,
+      "step": 1870
+    },
+    {
+      "epoch": 2.896764252696456,
+      "grad_norm": 1.0946455001831055,
+      "learning_rate": 0.0002,
+      "loss": 0.8779,
+      "step": 1880
+    },
+    {
+      "epoch": 2.9121725731895225,
+      "grad_norm": 0.9993954300880432,
+      "learning_rate": 0.0002,
+      "loss": 0.8775,
+      "step": 1890
+    },
+    {
+      "epoch": 2.9275808936825887,
+      "grad_norm": 1.1941032409667969,
+      "learning_rate": 0.0002,
+      "loss": 0.8004,
+      "step": 1900
+    },
+    {
+      "epoch": 2.942989214175655,
+      "grad_norm": 1.5158019065856934,
+      "learning_rate": 0.0002,
+      "loss": 0.8862,
+      "step": 1910
+    },
+    {
+      "epoch": 2.958397534668721,
+      "grad_norm": 1.4000548124313354,
+      "learning_rate": 0.0002,
+      "loss": 0.9363,
+      "step": 1920
+    },
+    {
+      "epoch": 2.973805855161787,
+      "grad_norm": 0.9906305074691772,
+      "learning_rate": 0.0002,
+      "loss": 0.8666,
+      "step": 1930
+    },
+    {
+      "epoch": 2.989214175654854,
+      "grad_norm": 1.7382938861846924,
+      "learning_rate": 0.0002,
+      "loss": 0.8434,
+      "step": 1940
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 1.2514833211898804,
+      "eval_runtime": 119.8709,
+      "eval_samples_per_second": 3.804,
+      "eval_steps_per_second": 0.476,
+      "step": 1947
+    },
+    {
+      "epoch": 3.00462249614792,
+      "grad_norm": 0.9265438318252563,
+      "learning_rate": 0.0002,
+      "loss": 0.8099,
+      "step": 1950
+    },
+    {
+      "epoch": 3.020030816640986,
+      "grad_norm": 1.2885810136795044,
+      "learning_rate": 0.0002,
+      "loss": 0.6058,
+      "step": 1960
+    },
+    {
+      "epoch": 3.0354391371340523,
+      "grad_norm": 1.150202751159668,
+      "learning_rate": 0.0002,
+      "loss": 0.6065,
+      "step": 1970
+    },
+    {
+      "epoch": 3.0508474576271185,
+      "grad_norm": 1.4145002365112305,
+      "learning_rate": 0.0002,
+      "loss": 0.7225,
+      "step": 1980
+    },
+    {
+      "epoch": 3.066255778120185,
+      "grad_norm": 1.3523074388504028,
+      "learning_rate": 0.0002,
+      "loss": 0.6403,
+      "step": 1990
+    },
+    {
+      "epoch": 3.0816640986132513,
+      "grad_norm": 1.4055354595184326,
+      "learning_rate": 0.0002,
+      "loss": 0.6784,
+      "step": 2000
+    },
+    {
+      "epoch": 3.0970724191063175,
+      "grad_norm": 1.3167393207550049,
+      "learning_rate": 0.0002,
+      "loss": 0.617,
+      "step": 2010
+    },
+    {
+      "epoch": 3.1124807395993837,
+      "grad_norm": 0.8515956997871399,
+      "learning_rate": 0.0002,
+      "loss": 0.6228,
+      "step": 2020
+    },
+    {
+      "epoch": 3.12788906009245,
+      "grad_norm": 1.596145510673523,
+      "learning_rate": 0.0002,
+      "loss": 0.722,
+      "step": 2030
+    },
+    {
+      "epoch": 3.143297380585516,
+      "grad_norm": 1.3406099081039429,
+      "learning_rate": 0.0002,
+      "loss": 0.6582,
+      "step": 2040
+    },
+    {
+      "epoch": 3.1587057010785826,
+      "grad_norm": 1.5566312074661255,
+      "learning_rate": 0.0002,
+      "loss": 0.661,
+      "step": 2050
+    },
+    {
+      "epoch": 3.174114021571649,
+      "grad_norm": 1.060834288597107,
+      "learning_rate": 0.0002,
+      "loss": 0.648,
+      "step": 2060
+    },
+    {
+      "epoch": 3.189522342064715,
+      "grad_norm": 1.534487247467041,
+      "learning_rate": 0.0002,
+      "loss": 0.7131,
+      "step": 2070
+    },
+    {
+      "epoch": 3.204930662557781,
+      "grad_norm": 1.4387457370758057,
+      "learning_rate": 0.0002,
+      "loss": 0.6465,
+      "step": 2080
+    },
+    {
+      "epoch": 3.2203389830508473,
+      "grad_norm": 1.6553950309753418,
+      "learning_rate": 0.0002,
+      "loss": 0.651,
+      "step": 2090
+    },
+    {
+      "epoch": 3.2357473035439135,
+      "grad_norm": 1.4905112981796265,
+      "learning_rate": 0.0002,
+      "loss": 0.6329,
+      "step": 2100
+    },
+    {
+      "epoch": 3.25115562403698,
+      "grad_norm": 1.5865840911865234,
+      "learning_rate": 0.0002,
+      "loss": 0.6835,
+      "step": 2110
+    },
+    {
+      "epoch": 3.2665639445300463,
+      "grad_norm": 1.7313302755355835,
+      "learning_rate": 0.0002,
+      "loss": 0.65,
+      "step": 2120
+    },
+    {
+      "epoch": 3.2819722650231125,
+      "grad_norm": 1.3718984127044678,
+      "learning_rate": 0.0002,
+      "loss": 0.6401,
+      "step": 2130
+    },
+    {
+      "epoch": 3.2973805855161786,
+      "grad_norm": 1.4081032276153564,
+      "learning_rate": 0.0002,
+      "loss": 0.6794,
+      "step": 2140
+    },
+    {
+      "epoch": 3.312788906009245,
+      "grad_norm": 1.2884169816970825,
+      "learning_rate": 0.0002,
+      "loss": 0.7119,
+      "step": 2150
+    },
+    {
+      "epoch": 3.3281972265023114,
+      "grad_norm": 1.0338295698165894,
+      "learning_rate": 0.0002,
+      "loss": 0.6476,
+      "step": 2160
+    },
+    {
+      "epoch": 3.3436055469953776,
+      "grad_norm": 1.5023012161254883,
+      "learning_rate": 0.0002,
+      "loss": 0.6544,
+      "step": 2170
+    },
+    {
+      "epoch": 3.3590138674884438,
+      "grad_norm": 1.1718626022338867,
+      "learning_rate": 0.0002,
+      "loss": 0.6606,
+      "step": 2180
+    },
+    {
+      "epoch": 3.37442218798151,
+      "grad_norm": 1.1776877641677856,
+      "learning_rate": 0.0002,
+      "loss": 0.6734,
+      "step": 2190
+    },
+    {
+      "epoch": 3.389830508474576,
+      "grad_norm": 1.1330585479736328,
+      "learning_rate": 0.0002,
+      "loss": 0.6301,
+      "step": 2200
+    },
+    {
+      "epoch": 3.4052388289676427,
+      "grad_norm": 2.024857997894287,
+      "learning_rate": 0.0002,
+      "loss": 0.7051,
+      "step": 2210
+    },
+    {
+      "epoch": 3.420647149460709,
+      "grad_norm": 1.3598058223724365,
+      "learning_rate": 0.0002,
+      "loss": 0.7233,
+      "step": 2220
+    },
+    {
+      "epoch": 3.436055469953775,
+      "grad_norm": 1.3725299835205078,
+      "learning_rate": 0.0002,
+      "loss": 0.6943,
+      "step": 2230
+    },
+    {
+      "epoch": 3.4514637904468413,
+      "grad_norm": 1.1065977811813354,
+      "learning_rate": 0.0002,
+      "loss": 0.6871,
+      "step": 2240
+    },
+    {
+      "epoch": 3.4668721109399074,
+      "grad_norm": 1.5074089765548706,
+      "learning_rate": 0.0002,
+      "loss": 0.6628,
+      "step": 2250
+    },
+    {
+      "epoch": 3.482280431432974,
+      "grad_norm": 1.114187479019165,
+      "learning_rate": 0.0002,
+      "loss": 0.6847,
+      "step": 2260
+    },
+    {
+      "epoch": 3.4976887519260402,
+      "grad_norm": 1.5719729661941528,
+      "learning_rate": 0.0002,
+      "loss": 0.6934,
+      "step": 2270
+    },
+    {
+      "epoch": 3.5130970724191064,
+      "grad_norm": 1.2377736568450928,
+      "learning_rate": 0.0002,
+      "loss": 0.7264,
+      "step": 2280
+    },
+    {
+      "epoch": 3.5285053929121726,
+      "grad_norm": 1.4295454025268555,
+      "learning_rate": 0.0002,
+      "loss": 0.6777,
+      "step": 2290
+    },
+    {
+      "epoch": 3.5439137134052388,
+      "grad_norm": 0.9240845441818237,
+      "learning_rate": 0.0002,
+      "loss": 0.6465,
+      "step": 2300
+    },
+    {
+      "epoch": 3.559322033898305,
+      "grad_norm": 1.5270299911499023,
+      "learning_rate": 0.0002,
+      "loss": 0.6627,
+      "step": 2310
+    },
+    {
+      "epoch": 3.574730354391371,
+      "grad_norm": 1.2820508480072021,
+      "learning_rate": 0.0002,
+      "loss": 0.7018,
+      "step": 2320
+    },
+    {
+      "epoch": 3.5901386748844377,
+      "grad_norm": 1.2932546138763428,
+      "learning_rate": 0.0002,
+      "loss": 0.7225,
+      "step": 2330
+    },
+    {
+      "epoch": 3.605546995377504,
+      "grad_norm": 1.3201853036880493,
+      "learning_rate": 0.0002,
+      "loss": 0.6815,
+      "step": 2340
+    },
+    {
+      "epoch": 3.62095531587057,
+      "grad_norm": 1.7489187717437744,
+      "learning_rate": 0.0002,
+      "loss": 0.6842,
+      "step": 2350
+    },
+    {
+      "epoch": 3.6363636363636362,
+      "grad_norm": 1.2384065389633179,
+      "learning_rate": 0.0002,
+      "loss": 0.7468,
+      "step": 2360
+    },
+    {
+      "epoch": 3.6517719568567024,
+      "grad_norm": 1.0870963335037231,
+      "learning_rate": 0.0002,
+      "loss": 0.6978,
+      "step": 2370
+    },
+    {
+      "epoch": 3.667180277349769,
+      "grad_norm": 1.4325937032699585,
+      "learning_rate": 0.0002,
+      "loss": 0.6214,
+      "step": 2380
+    },
+    {
+      "epoch": 3.682588597842835,
+      "grad_norm": 1.3232696056365967,
+      "learning_rate": 0.0002,
+      "loss": 0.6737,
+      "step": 2390
+    },
+    {
+      "epoch": 3.6979969183359014,
+      "grad_norm": 1.409141182899475,
+      "learning_rate": 0.0002,
+      "loss": 0.6643,
+      "step": 2400
+    },
+    {
+      "epoch": 3.7134052388289676,
+      "grad_norm": 1.4707226753234863,
+      "learning_rate": 0.0002,
+      "loss": 0.7069,
+      "step": 2410
+    },
+    {
+      "epoch": 3.7288135593220337,
+      "grad_norm": 1.3888318538665771,
+      "learning_rate": 0.0002,
+      "loss": 0.678,
+      "step": 2420
+    },
+    {
+      "epoch": 3.7442218798151004,
+      "grad_norm": 1.4682821035385132,
+      "learning_rate": 0.0002,
+      "loss": 0.6718,
+      "step": 2430
+    },
+    {
+      "epoch": 3.7596302003081665,
+      "grad_norm": 1.2217520475387573,
+      "learning_rate": 0.0002,
+      "loss": 0.7593,
+      "step": 2440
+    },
+    {
+      "epoch": 3.7750385208012327,
+      "grad_norm": 1.3346893787384033,
+      "learning_rate": 0.0002,
+      "loss": 0.6912,
+      "step": 2450
+    },
+    {
+      "epoch": 3.790446841294299,
+      "grad_norm": 1.4342081546783447,
+      "learning_rate": 0.0002,
+      "loss": 0.609,
+      "step": 2460
+    },
+    {
+      "epoch": 3.805855161787365,
+      "grad_norm": 1.5921152830123901,
+      "learning_rate": 0.0002,
+      "loss": 0.6985,
+      "step": 2470
+    },
+    {
+      "epoch": 3.8212634822804317,
+      "grad_norm": 1.7322826385498047,
+      "learning_rate": 0.0002,
+      "loss": 0.6751,
+      "step": 2480
+    },
+    {
+      "epoch": 3.836671802773498,
+      "grad_norm": 1.478434681892395,
+      "learning_rate": 0.0002,
+      "loss": 0.6862,
+      "step": 2490
+    },
+    {
+      "epoch": 3.852080123266564,
+      "grad_norm": 1.3945757150650024,
+      "learning_rate": 0.0002,
+      "loss": 0.6983,
+      "step": 2500
+    },
+    {
+      "epoch": 3.86748844375963,
+      "grad_norm": 1.584877848625183,
+      "learning_rate": 0.0002,
+      "loss": 0.7439,
+      "step": 2510
+    },
+    {
+      "epoch": 3.8828967642526964,
+      "grad_norm": 1.6307404041290283,
+      "learning_rate": 0.0002,
+      "loss": 0.7049,
+      "step": 2520
+    },
+    {
+      "epoch": 3.898305084745763,
+      "grad_norm": 1.4129183292388916,
+      "learning_rate": 0.0002,
+      "loss": 0.6823,
+      "step": 2530
+    },
+    {
+      "epoch": 3.9137134052388287,
+      "grad_norm": 1.7380712032318115,
+      "learning_rate": 0.0002,
+      "loss": 0.7092,
+      "step": 2540
+    },
+    {
+      "epoch": 3.9291217257318953,
+      "grad_norm": 1.4169772863388062,
+      "learning_rate": 0.0002,
+      "loss": 0.7277,
+      "step": 2550
+    },
+    {
+      "epoch": 3.9445300462249615,
+      "grad_norm": 2.520665168762207,
+      "learning_rate": 0.0002,
+      "loss": 0.7369,
+      "step": 2560
+    },
+    {
+      "epoch": 3.9599383667180277,
+      "grad_norm": 1.2402763366699219,
+      "learning_rate": 0.0002,
+      "loss": 0.7815,
+      "step": 2570
+    },
+    {
+      "epoch": 3.975346687211094,
+      "grad_norm": 1.5902904272079468,
+      "learning_rate": 0.0002,
+      "loss": 0.8077,
+      "step": 2580
+    },
+    {
+      "epoch": 3.99075500770416,
+      "grad_norm": 2.2361271381378174,
+      "learning_rate": 0.0002,
+      "loss": 0.7272,
+      "step": 2590
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.324176549911499,
+      "eval_runtime": 119.304,
+      "eval_samples_per_second": 3.822,
+      "eval_steps_per_second": 0.478,
+      "step": 2596
+    },
+    {
+      "epoch": 4.006163328197227,
+      "grad_norm": 0.8134178519248962,
+      "learning_rate": 0.0002,
+      "loss": 0.6403,
+      "step": 2600
+    },
+    {
+      "epoch": 4.021571648690292,
+      "grad_norm": 20.112083435058594,
+      "learning_rate": 0.0002,
+      "loss": 0.4955,
+      "step": 2610
+    },
+    {
+      "epoch": 4.036979969183359,
+      "grad_norm": 18.01171875,
+      "learning_rate": 0.0002,
+      "loss": 0.5607,
+      "step": 2620
+    },
+    {
+      "epoch": 4.052388289676426,
+      "grad_norm": 1.770273208618164,
+      "learning_rate": 0.0002,
+      "loss": 0.4889,
+      "step": 2630
+    },
+    {
+      "epoch": 4.067796610169491,
+      "grad_norm": 1.943787932395935,
+      "learning_rate": 0.0002,
+      "loss": 0.5359,
+      "step": 2640
+    },
+    {
+      "epoch": 4.083204930662558,
+      "grad_norm": 1.9997365474700928,
+      "learning_rate": 0.0002,
+      "loss": 0.5,
+      "step": 2650
+    },
+    {
+      "epoch": 4.098613251155624,
+      "grad_norm": 1.3823848962783813,
+      "learning_rate": 0.0002,
+      "loss": 0.5365,
+      "step": 2660
+    },
+    {
+      "epoch": 4.11402157164869,
+      "grad_norm": 1.4237934350967407,
+      "learning_rate": 0.0002,
+      "loss": 0.5,
+      "step": 2670
+    },
+    {
+      "epoch": 4.129429892141757,
+      "grad_norm": 1.6521705389022827,
+      "learning_rate": 0.0002,
+      "loss": 0.5579,
+      "step": 2680
+    },
+    {
+      "epoch": 4.144838212634823,
+      "grad_norm": 1.2283433675765991,
+      "learning_rate": 0.0002,
+      "loss": 0.5014,
+      "step": 2690
+    },
+    {
+      "epoch": 4.160246533127889,
+      "grad_norm": 1.6849380731582642,
+      "learning_rate": 0.0002,
+      "loss": 0.4952,
+      "step": 2700
+    },
+    {
+      "epoch": 4.175654853620955,
+      "grad_norm": 2.493161678314209,
+      "learning_rate": 0.0002,
+      "loss": 0.4974,
+      "step": 2710
+    },
+    {
+      "epoch": 4.191063174114022,
+      "grad_norm": 1.4243499040603638,
+      "learning_rate": 0.0002,
+      "loss": 0.502,
+      "step": 2720
+    },
+    {
+      "epoch": 4.206471494607088,
+      "grad_norm": 2.471060037612915,
+      "learning_rate": 0.0002,
+      "loss": 0.5069,
+      "step": 2730
+    },
+    {
+      "epoch": 4.221879815100154,
+      "grad_norm": 1.5935828685760498,
+      "learning_rate": 0.0002,
+      "loss": 0.4757,
+      "step": 2740
+    },
+    {
+      "epoch": 4.237288135593221,
+      "grad_norm": 2.989623785018921,
+      "learning_rate": 0.0002,
+      "loss": 0.5223,
+      "step": 2750
+    },
+    {
+      "epoch": 4.252696456086286,
+      "grad_norm": 1.7323458194732666,
+      "learning_rate": 0.0002,
+      "loss": 0.5262,
+      "step": 2760
+    },
+    {
+      "epoch": 4.268104776579353,
+      "grad_norm": 1.6325395107269287,
+      "learning_rate": 0.0002,
+      "loss": 0.4389,
+      "step": 2770
+    },
+    {
+      "epoch": 4.283513097072419,
+      "grad_norm": 2.0011212825775146,
+      "learning_rate": 0.0002,
+      "loss": 0.5205,
+      "step": 2780
+    },
+    {
+      "epoch": 4.298921417565485,
+      "grad_norm": 2.3058269023895264,
+      "learning_rate": 0.0002,
+      "loss": 0.5638,
+      "step": 2790
+    },
+    {
+      "epoch": 4.314329738058552,
+      "grad_norm": 1.360182762145996,
+      "learning_rate": 0.0002,
+      "loss": 0.5479,
+      "step": 2800
+    },
+    {
+      "epoch": 4.329738058551618,
+      "grad_norm": 1.7124749422073364,
+      "learning_rate": 0.0002,
+      "loss": 0.4765,
+      "step": 2810
+    },
+    {
+      "epoch": 4.345146379044684,
+      "grad_norm": 1.439314603805542,
+      "learning_rate": 0.0002,
+      "loss": 0.5113,
+      "step": 2820
+    },
+    {
+      "epoch": 4.36055469953775,
+      "grad_norm": 1.5941680669784546,
+      "learning_rate": 0.0002,
+      "loss": 0.502,
+      "step": 2830
+    },
+    {
+      "epoch": 4.375963020030817,
+      "grad_norm": 1.8157657384872437,
+      "learning_rate": 0.0002,
+      "loss": 0.5283,
+      "step": 2840
+    },
+    {
+      "epoch": 4.391371340523883,
+      "grad_norm": 1.754032015800476,
+      "learning_rate": 0.0002,
+      "loss": 0.5379,
+      "step": 2850
+    },
+    {
+      "epoch": 4.406779661016949,
+      "grad_norm": 2.0685315132141113,
+      "learning_rate": 0.0002,
+      "loss": 0.539,
+      "step": 2860
+    },
+    {
+      "epoch": 4.422187981510016,
+      "grad_norm": 1.605820655822754,
+      "learning_rate": 0.0002,
+      "loss": 0.525,
+      "step": 2870
+    },
+    {
+      "epoch": 4.437596302003081,
+      "grad_norm": 1.7439035177230835,
+      "learning_rate": 0.0002,
+      "loss": 0.5254,
+      "step": 2880
+    },
+    {
+      "epoch": 4.453004622496148,
+      "grad_norm": 2.0108375549316406,
+      "learning_rate": 0.0002,
+      "loss": 0.5594,
+      "step": 2890
+    },
+    {
+      "epoch": 4.4684129429892145,
+      "grad_norm": 1.7569063901901245,
+      "learning_rate": 0.0002,
+      "loss": 0.5695,
+      "step": 2900
+    },
+    {
+      "epoch": 4.48382126348228,
+      "grad_norm": 1.544561743736267,
+      "learning_rate": 0.0002,
+      "loss": 0.566,
+      "step": 2910
+    },
+    {
+      "epoch": 4.499229583975347,
+      "grad_norm": 1.3701506853103638,
+      "learning_rate": 0.0002,
+      "loss": 0.4984,
+      "step": 2920
+    },
+    {
+      "epoch": 4.514637904468413,
+      "grad_norm": 1.2095595598220825,
+      "learning_rate": 0.0002,
+      "loss": 0.5695,
+      "step": 2930
+    },
+    {
+      "epoch": 4.530046224961479,
+      "grad_norm": 1.7037502527236938,
+      "learning_rate": 0.0002,
+      "loss": 0.5415,
+      "step": 2940
+    },
+    {
+      "epoch": 4.545454545454545,
+      "grad_norm": 2.8776841163635254,
+      "learning_rate": 0.0002,
+      "loss": 0.5179,
+      "step": 2950
+    },
+    {
+      "epoch": 4.560862865947612,
+      "grad_norm": 1.360141634941101,
+      "learning_rate": 0.0002,
+      "loss": 0.5832,
+      "step": 2960
+    },
+    {
+      "epoch": 4.576271186440678,
+      "grad_norm": 1.6299976110458374,
+      "learning_rate": 0.0002,
+      "loss": 0.5515,
+      "step": 2970
+    },
+    {
+      "epoch": 4.591679506933744,
+      "grad_norm": 2.0980679988861084,
+      "learning_rate": 0.0002,
+      "loss": 0.5466,
+      "step": 2980
+    },
+    {
+      "epoch": 4.6070878274268106,
+      "grad_norm": 1.5481915473937988,
+      "learning_rate": 0.0002,
+      "loss": 0.5325,
+      "step": 2990
+    },
+    {
+      "epoch": 4.622496147919877,
+      "grad_norm": 1.1016762256622314,
+      "learning_rate": 0.0002,
+      "loss": 0.5168,
+      "step": 3000
+    },
+    {
+      "epoch": 4.637904468412943,
+      "grad_norm": 1.8467375040054321,
+      "learning_rate": 0.0002,
+      "loss": 0.5374,
+      "step": 3010
+    },
+    {
+      "epoch": 4.6533127889060095,
+      "grad_norm": 1.7077717781066895,
+      "learning_rate": 0.0002,
+      "loss": 0.5274,
+      "step": 3020
+    },
+    {
+      "epoch": 4.668721109399075,
+      "grad_norm": 1.6364675760269165,
+      "learning_rate": 0.0002,
+      "loss": 0.5666,
+      "step": 3030
+    },
+    {
+      "epoch": 4.684129429892142,
+      "grad_norm": 1.3262457847595215,
+      "learning_rate": 0.0002,
+      "loss": 0.5072,
+      "step": 3040
+    },
+    {
+      "epoch": 4.699537750385208,
+      "grad_norm": 1.6521623134613037,
+      "learning_rate": 0.0002,
+      "loss": 0.5281,
+      "step": 3050
+    },
+    {
+      "epoch": 4.714946070878274,
+      "grad_norm": 1.1765156984329224,
+      "learning_rate": 0.0002,
+      "loss": 0.5036,
+      "step": 3060
+    },
+    {
+      "epoch": 4.730354391371341,
+      "grad_norm": 1.4242093563079834,
+      "learning_rate": 0.0002,
+      "loss": 0.5251,
+      "step": 3070
+    },
+    {
+      "epoch": 4.745762711864407,
+      "grad_norm": 1.9651737213134766,
+      "learning_rate": 0.0002,
+      "loss": 0.5471,
+      "step": 3080
+    },
+    {
+      "epoch": 4.761171032357473,
+      "grad_norm": 1.9256596565246582,
+      "learning_rate": 0.0002,
+      "loss": 0.5504,
+      "step": 3090
+    },
+    {
+      "epoch": 4.776579352850539,
+      "grad_norm": 1.4493910074234009,
+      "learning_rate": 0.0002,
+      "loss": 0.5452,
+      "step": 3100
+    },
+    {
+      "epoch": 4.7919876733436055,
+      "grad_norm": 1.9169608354568481,
+      "learning_rate": 0.0002,
+      "loss": 0.559,
+      "step": 3110
+    },
+    {
+      "epoch": 4.807395993836672,
+      "grad_norm": 1.585694670677185,
+      "learning_rate": 0.0002,
+      "loss": 0.5357,
+      "step": 3120
+    },
+    {
+      "epoch": 4.822804314329738,
+      "grad_norm": 1.8497015237808228,
+      "learning_rate": 0.0002,
+      "loss": 0.5627,
+      "step": 3130
+    },
+    {
+      "epoch": 4.8382126348228045,
+      "grad_norm": 1.4345619678497314,
+      "learning_rate": 0.0002,
+      "loss": 0.5604,
+      "step": 3140
+    },
+    {
+      "epoch": 4.85362095531587,
+      "grad_norm": 1.3929195404052734,
+      "learning_rate": 0.0002,
+      "loss": 0.5771,
+      "step": 3150
+    },
+    {
+      "epoch": 4.869029275808937,
+      "grad_norm": 1.9688060283660889,
+      "learning_rate": 0.0002,
+      "loss": 0.6083,
+      "step": 3160
+    },
+    {
+      "epoch": 4.8844375963020035,
+      "grad_norm": 1.7507604360580444,
+      "learning_rate": 0.0002,
+      "loss": 0.5391,
+      "step": 3170
+    },
+    {
+      "epoch": 4.899845916795069,
+      "grad_norm": 2.4209811687469482,
+      "learning_rate": 0.0002,
+      "loss": 0.5527,
+      "step": 3180
+    },
+    {
+      "epoch": 4.915254237288136,
+      "grad_norm": 2.582061529159546,
+      "learning_rate": 0.0002,
+      "loss": 0.5557,
+      "step": 3190
+    },
+    {
+      "epoch": 4.9306625577812015,
+      "grad_norm": 1.6832361221313477,
+      "learning_rate": 0.0002,
+      "loss": 0.559,
+      "step": 3200
+    },
+    {
+      "epoch": 4.946070878274268,
+      "grad_norm": 1.2785899639129639,
+      "learning_rate": 0.0002,
+      "loss": 0.5078,
+      "step": 3210
+    },
+    {
+      "epoch": 4.961479198767334,
+      "grad_norm": 1.4990512132644653,
+      "learning_rate": 0.0002,
+      "loss": 0.5644,
+      "step": 3220
+    },
+    {
+      "epoch": 4.9768875192604005,
+      "grad_norm": 1.751932978630066,
+      "learning_rate": 0.0002,
+      "loss": 0.5827,
+      "step": 3230
+    },
+    {
+      "epoch": 4.992295839753467,
+      "grad_norm": 1.843894124031067,
+      "learning_rate": 0.0002,
+      "loss": 0.6049,
+      "step": 3240
+    },
+    {
+      "epoch": 5.0,
+      "eval_loss": 1.4627584218978882,
+      "eval_runtime": 119.266,
+      "eval_samples_per_second": 3.823,
+      "eval_steps_per_second": 0.478,
+      "step": 3245
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 5192,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.42373488164864e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}