Upload folder using huggingface_hub

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9750116b7986e46425f9dc2eb64b3f4647ac087542206a59d49095dfe893fcf4
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-10232/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-10232/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-10232/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8a429a4ae9a145e381353cafffcd66ae7f1e628368d1242da011587da534d62f
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-10232/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:578fb4b1138f3ac38093483593a376e53cd6b5e8808a541b8b76a8ff52da5a96
+size 55532666

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-10232/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:040be4c7f2b38b215c39fbb7d27055b97153e94d38c2ef0fc64d911d3d3c3850
+size 14244

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-10232/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d26dad811ff813f8aac5cb50f8ecf7eaf8d48acd5c8caed89475387705c580f4
+size 1064

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-10232/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-10232/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-10232/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:67c127eb44653a0272979e2cc773b017f5ff5cfc6ed92250a567d1213376a07e
+size 5560

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-1279/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-1279/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-1279/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2970289e187c96d0997889a347f9a51e86f8a53f5625049e38079cbdf9a37740
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-1279/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:274cd9ae246ddfa66ec752ab3eef08251047ae90f890894eb9e5dbb17fd066d9
+size 55532666

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-1279/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:886590d77dcc6a5b77286920048e6289608ea7df56d4192800352b954910e2ff
+size 14244

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-1279/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:17d3c3052408e1b0a9c679c8c5cdb53e0805ecd8446ebaac118d642a80ee82f9
+size 1064

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-1279/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-1279/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-1279/trainer_state.json

@@ -0,0 +1,930 @@
+{
+  "best_metric": 0.39914223551750183,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-1279",
+  "epoch": 1.0,
+  "eval_steps": 10,
+  "global_step": 1279,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.007818608287724784,
+      "grad_norm": 1.1001313924789429,
+      "learning_rate": 0.0002,
+      "loss": 1.7403,
+      "step": 10
+    },
+    {
+      "epoch": 0.01563721657544957,
+      "grad_norm": 1.3678615093231201,
+      "learning_rate": 0.0002,
+      "loss": 1.1795,
+      "step": 20
+    },
+    {
+      "epoch": 0.023455824863174355,
+      "grad_norm": 1.2362287044525146,
+      "learning_rate": 0.0002,
+      "loss": 1.0159,
+      "step": 30
+    },
+    {
+      "epoch": 0.03127443315089914,
+      "grad_norm": 0.9754743576049805,
+      "learning_rate": 0.0002,
+      "loss": 0.9315,
+      "step": 40
+    },
+    {
+      "epoch": 0.039093041438623924,
+      "grad_norm": 1.0572363138198853,
+      "learning_rate": 0.0002,
+      "loss": 0.8015,
+      "step": 50
+    },
+    {
+      "epoch": 0.04691164972634871,
+      "grad_norm": 0.8838446140289307,
+      "learning_rate": 0.0002,
+      "loss": 0.7264,
+      "step": 60
+    },
+    {
+      "epoch": 0.054730258014073496,
+      "grad_norm": 0.831045389175415,
+      "learning_rate": 0.0002,
+      "loss": 0.6807,
+      "step": 70
+    },
+    {
+      "epoch": 0.06254886630179828,
+      "grad_norm": 0.721530556678772,
+      "learning_rate": 0.0002,
+      "loss": 0.6731,
+      "step": 80
+    },
+    {
+      "epoch": 0.07036747458952307,
+      "grad_norm": 0.6918481588363647,
+      "learning_rate": 0.0002,
+      "loss": 0.6427,
+      "step": 90
+    },
+    {
+      "epoch": 0.07818608287724785,
+      "grad_norm": 0.8207236528396606,
+      "learning_rate": 0.0002,
+      "loss": 0.6719,
+      "step": 100
+    },
+    {
+      "epoch": 0.08600469116497264,
+      "grad_norm": 0.9405701756477356,
+      "learning_rate": 0.0002,
+      "loss": 0.6533,
+      "step": 110
+    },
+    {
+      "epoch": 0.09382329945269742,
+      "grad_norm": 0.7389968037605286,
+      "learning_rate": 0.0002,
+      "loss": 0.6102,
+      "step": 120
+    },
+    {
+      "epoch": 0.1016419077404222,
+      "grad_norm": 0.7102212905883789,
+      "learning_rate": 0.0002,
+      "loss": 0.6561,
+      "step": 130
+    },
+    {
+      "epoch": 0.10946051602814699,
+      "grad_norm": 0.6546808481216431,
+      "learning_rate": 0.0002,
+      "loss": 0.6222,
+      "step": 140
+    },
+    {
+      "epoch": 0.11727912431587177,
+      "grad_norm": 1.0097354650497437,
+      "learning_rate": 0.0002,
+      "loss": 0.6369,
+      "step": 150
+    },
+    {
+      "epoch": 0.12509773260359655,
+      "grad_norm": 0.5699719786643982,
+      "learning_rate": 0.0002,
+      "loss": 0.6495,
+      "step": 160
+    },
+    {
+      "epoch": 0.13291634089132134,
+      "grad_norm": 0.6371490359306335,
+      "learning_rate": 0.0002,
+      "loss": 0.6421,
+      "step": 170
+    },
+    {
+      "epoch": 0.14073494917904614,
+      "grad_norm": 0.8385978937149048,
+      "learning_rate": 0.0002,
+      "loss": 0.6101,
+      "step": 180
+    },
+    {
+      "epoch": 0.1485535574667709,
+      "grad_norm": 0.6549069285392761,
+      "learning_rate": 0.0002,
+      "loss": 0.5957,
+      "step": 190
+    },
+    {
+      "epoch": 0.1563721657544957,
+      "grad_norm": 0.5297655463218689,
+      "learning_rate": 0.0002,
+      "loss": 0.5834,
+      "step": 200
+    },
+    {
+      "epoch": 0.1641907740422205,
+      "grad_norm": 0.6385621428489685,
+      "learning_rate": 0.0002,
+      "loss": 0.6054,
+      "step": 210
+    },
+    {
+      "epoch": 0.17200938232994528,
+      "grad_norm": 0.6723865866661072,
+      "learning_rate": 0.0002,
+      "loss": 0.6174,
+      "step": 220
+    },
+    {
+      "epoch": 0.17982799061767005,
+      "grad_norm": 0.6121484041213989,
+      "learning_rate": 0.0002,
+      "loss": 0.5951,
+      "step": 230
+    },
+    {
+      "epoch": 0.18764659890539484,
+      "grad_norm": 0.619121789932251,
+      "learning_rate": 0.0002,
+      "loss": 0.6046,
+      "step": 240
+    },
+    {
+      "epoch": 0.19546520719311963,
+      "grad_norm": 0.49208253622055054,
+      "learning_rate": 0.0002,
+      "loss": 0.5764,
+      "step": 250
+    },
+    {
+      "epoch": 0.2032838154808444,
+      "grad_norm": 0.4991285800933838,
+      "learning_rate": 0.0002,
+      "loss": 0.5679,
+      "step": 260
+    },
+    {
+      "epoch": 0.2111024237685692,
+      "grad_norm": 0.7622858285903931,
+      "learning_rate": 0.0002,
+      "loss": 0.5777,
+      "step": 270
+    },
+    {
+      "epoch": 0.21892103205629398,
+      "grad_norm": 0.5988286733627319,
+      "learning_rate": 0.0002,
+      "loss": 0.5504,
+      "step": 280
+    },
+    {
+      "epoch": 0.22673964034401878,
+      "grad_norm": 0.510055661201477,
+      "learning_rate": 0.0002,
+      "loss": 0.5801,
+      "step": 290
+    },
+    {
+      "epoch": 0.23455824863174354,
+      "grad_norm": 0.47940748929977417,
+      "learning_rate": 0.0002,
+      "loss": 0.5788,
+      "step": 300
+    },
+    {
+      "epoch": 0.24237685691946834,
+      "grad_norm": 0.5604141354560852,
+      "learning_rate": 0.0002,
+      "loss": 0.6093,
+      "step": 310
+    },
+    {
+      "epoch": 0.2501954652071931,
+      "grad_norm": 0.479342520236969,
+      "learning_rate": 0.0002,
+      "loss": 0.5641,
+      "step": 320
+    },
+    {
+      "epoch": 0.2580140734949179,
+      "grad_norm": 0.5401737093925476,
+      "learning_rate": 0.0002,
+      "loss": 0.5304,
+      "step": 330
+    },
+    {
+      "epoch": 0.2658326817826427,
+      "grad_norm": 0.5436083674430847,
+      "learning_rate": 0.0002,
+      "loss": 0.5547,
+      "step": 340
+    },
+    {
+      "epoch": 0.2736512900703675,
+      "grad_norm": 0.6402848362922668,
+      "learning_rate": 0.0002,
+      "loss": 0.5724,
+      "step": 350
+    },
+    {
+      "epoch": 0.2814698983580923,
+      "grad_norm": 0.4756305515766144,
+      "learning_rate": 0.0002,
+      "loss": 0.5452,
+      "step": 360
+    },
+    {
+      "epoch": 0.28928850664581707,
+      "grad_norm": 0.5536904335021973,
+      "learning_rate": 0.0002,
+      "loss": 0.5936,
+      "step": 370
+    },
+    {
+      "epoch": 0.2971071149335418,
+      "grad_norm": 0.6187605857849121,
+      "learning_rate": 0.0002,
+      "loss": 0.5827,
+      "step": 380
+    },
+    {
+      "epoch": 0.3049257232212666,
+      "grad_norm": 0.5297170877456665,
+      "learning_rate": 0.0002,
+      "loss": 0.5435,
+      "step": 390
+    },
+    {
+      "epoch": 0.3127443315089914,
+      "grad_norm": 0.5808210372924805,
+      "learning_rate": 0.0002,
+      "loss": 0.583,
+      "step": 400
+    },
+    {
+      "epoch": 0.3205629397967162,
+      "grad_norm": 0.7509300708770752,
+      "learning_rate": 0.0002,
+      "loss": 0.5587,
+      "step": 410
+    },
+    {
+      "epoch": 0.328381548084441,
+      "grad_norm": 0.5674371719360352,
+      "learning_rate": 0.0002,
+      "loss": 0.5439,
+      "step": 420
+    },
+    {
+      "epoch": 0.33620015637216577,
+      "grad_norm": 0.5833905339241028,
+      "learning_rate": 0.0002,
+      "loss": 0.5431,
+      "step": 430
+    },
+    {
+      "epoch": 0.34401876465989056,
+      "grad_norm": 0.537860095500946,
+      "learning_rate": 0.0002,
+      "loss": 0.5527,
+      "step": 440
+    },
+    {
+      "epoch": 0.3518373729476153,
+      "grad_norm": 0.5747054219245911,
+      "learning_rate": 0.0002,
+      "loss": 0.528,
+      "step": 450
+    },
+    {
+      "epoch": 0.3596559812353401,
+      "grad_norm": 0.6268995404243469,
+      "learning_rate": 0.0002,
+      "loss": 0.5281,
+      "step": 460
+    },
+    {
+      "epoch": 0.3674745895230649,
+      "grad_norm": 0.640737771987915,
+      "learning_rate": 0.0002,
+      "loss": 0.5394,
+      "step": 470
+    },
+    {
+      "epoch": 0.3752931978107897,
+      "grad_norm": 0.6593332290649414,
+      "learning_rate": 0.0002,
+      "loss": 0.5488,
+      "step": 480
+    },
+    {
+      "epoch": 0.3831118060985145,
+      "grad_norm": 0.5977872014045715,
+      "learning_rate": 0.0002,
+      "loss": 0.5384,
+      "step": 490
+    },
+    {
+      "epoch": 0.39093041438623927,
+      "grad_norm": 0.6118639707565308,
+      "learning_rate": 0.0002,
+      "loss": 0.5154,
+      "step": 500
+    },
+    {
+      "epoch": 0.39874902267396406,
+      "grad_norm": 1.2320106029510498,
+      "learning_rate": 0.0002,
+      "loss": 0.5481,
+      "step": 510
+    },
+    {
+      "epoch": 0.4065676309616888,
+      "grad_norm": 0.6538275480270386,
+      "learning_rate": 0.0002,
+      "loss": 0.5225,
+      "step": 520
+    },
+    {
+      "epoch": 0.4143862392494136,
+      "grad_norm": 0.5839771032333374,
+      "learning_rate": 0.0002,
+      "loss": 0.5322,
+      "step": 530
+    },
+    {
+      "epoch": 0.4222048475371384,
+      "grad_norm": 0.6228310465812683,
+      "learning_rate": 0.0002,
+      "loss": 0.5075,
+      "step": 540
+    },
+    {
+      "epoch": 0.4300234558248632,
+      "grad_norm": 0.653239905834198,
+      "learning_rate": 0.0002,
+      "loss": 0.5177,
+      "step": 550
+    },
+    {
+      "epoch": 0.43784206411258797,
+      "grad_norm": 0.6995204091072083,
+      "learning_rate": 0.0002,
+      "loss": 0.5209,
+      "step": 560
+    },
+    {
+      "epoch": 0.44566067240031276,
+      "grad_norm": 0.5197685956954956,
+      "learning_rate": 0.0002,
+      "loss": 0.4914,
+      "step": 570
+    },
+    {
+      "epoch": 0.45347928068803756,
+      "grad_norm": 0.6786061525344849,
+      "learning_rate": 0.0002,
+      "loss": 0.4951,
+      "step": 580
+    },
+    {
+      "epoch": 0.4612978889757623,
+      "grad_norm": 0.6599542498588562,
+      "learning_rate": 0.0002,
+      "loss": 0.5107,
+      "step": 590
+    },
+    {
+      "epoch": 0.4691164972634871,
+      "grad_norm": 0.5535895228385925,
+      "learning_rate": 0.0002,
+      "loss": 0.514,
+      "step": 600
+    },
+    {
+      "epoch": 0.4769351055512119,
+      "grad_norm": 0.667336642742157,
+      "learning_rate": 0.0002,
+      "loss": 0.5206,
+      "step": 610
+    },
+    {
+      "epoch": 0.4847537138389367,
+      "grad_norm": 0.5404567718505859,
+      "learning_rate": 0.0002,
+      "loss": 0.4918,
+      "step": 620
+    },
+    {
+      "epoch": 0.49257232212666147,
+      "grad_norm": 0.5316283702850342,
+      "learning_rate": 0.0002,
+      "loss": 0.501,
+      "step": 630
+    },
+    {
+      "epoch": 0.5003909304143862,
+      "grad_norm": 0.6398797035217285,
+      "learning_rate": 0.0002,
+      "loss": 0.5026,
+      "step": 640
+    },
+    {
+      "epoch": 0.508209538702111,
+      "grad_norm": 0.6197776794433594,
+      "learning_rate": 0.0002,
+      "loss": 0.5017,
+      "step": 650
+    },
+    {
+      "epoch": 0.5160281469898358,
+      "grad_norm": 0.8760672807693481,
+      "learning_rate": 0.0002,
+      "loss": 0.4905,
+      "step": 660
+    },
+    {
+      "epoch": 0.5238467552775606,
+      "grad_norm": 0.7163699269294739,
+      "learning_rate": 0.0002,
+      "loss": 0.4798,
+      "step": 670
+    },
+    {
+      "epoch": 0.5316653635652854,
+      "grad_norm": 0.7599782347679138,
+      "learning_rate": 0.0002,
+      "loss": 0.4974,
+      "step": 680
+    },
+    {
+      "epoch": 0.5394839718530101,
+      "grad_norm": 0.5682359337806702,
+      "learning_rate": 0.0002,
+      "loss": 0.4942,
+      "step": 690
+    },
+    {
+      "epoch": 0.547302580140735,
+      "grad_norm": 0.6667918562889099,
+      "learning_rate": 0.0002,
+      "loss": 0.4826,
+      "step": 700
+    },
+    {
+      "epoch": 0.5551211884284597,
+      "grad_norm": 0.9613338112831116,
+      "learning_rate": 0.0002,
+      "loss": 0.475,
+      "step": 710
+    },
+    {
+      "epoch": 0.5629397967161845,
+      "grad_norm": 0.5658668279647827,
+      "learning_rate": 0.0002,
+      "loss": 0.5005,
+      "step": 720
+    },
+    {
+      "epoch": 0.5707584050039093,
+      "grad_norm": 0.7233469486236572,
+      "learning_rate": 0.0002,
+      "loss": 0.4697,
+      "step": 730
+    },
+    {
+      "epoch": 0.5785770132916341,
+      "grad_norm": 0.5586039423942566,
+      "learning_rate": 0.0002,
+      "loss": 0.4659,
+      "step": 740
+    },
+    {
+      "epoch": 0.5863956215793589,
+      "grad_norm": 0.8161298632621765,
+      "learning_rate": 0.0002,
+      "loss": 0.4893,
+      "step": 750
+    },
+    {
+      "epoch": 0.5942142298670836,
+      "grad_norm": 0.7192191481590271,
+      "learning_rate": 0.0002,
+      "loss": 0.4678,
+      "step": 760
+    },
+    {
+      "epoch": 0.6020328381548085,
+      "grad_norm": 0.5711938738822937,
+      "learning_rate": 0.0002,
+      "loss": 0.4765,
+      "step": 770
+    },
+    {
+      "epoch": 0.6098514464425332,
+      "grad_norm": 0.5471241474151611,
+      "learning_rate": 0.0002,
+      "loss": 0.4676,
+      "step": 780
+    },
+    {
+      "epoch": 0.617670054730258,
+      "grad_norm": 0.5709220767021179,
+      "learning_rate": 0.0002,
+      "loss": 0.4628,
+      "step": 790
+    },
+    {
+      "epoch": 0.6254886630179828,
+      "grad_norm": 0.882448673248291,
+      "learning_rate": 0.0002,
+      "loss": 0.4695,
+      "step": 800
+    },
+    {
+      "epoch": 0.6333072713057076,
+      "grad_norm": 0.5136802196502686,
+      "learning_rate": 0.0002,
+      "loss": 0.4611,
+      "step": 810
+    },
+    {
+      "epoch": 0.6411258795934324,
+      "grad_norm": 0.6611698865890503,
+      "learning_rate": 0.0002,
+      "loss": 0.4663,
+      "step": 820
+    },
+    {
+      "epoch": 0.6489444878811571,
+      "grad_norm": 0.7050015926361084,
+      "learning_rate": 0.0002,
+      "loss": 0.4962,
+      "step": 830
+    },
+    {
+      "epoch": 0.656763096168882,
+      "grad_norm": 0.5757645964622498,
+      "learning_rate": 0.0002,
+      "loss": 0.4624,
+      "step": 840
+    },
+    {
+      "epoch": 0.6645817044566067,
+      "grad_norm": 0.6651985049247742,
+      "learning_rate": 0.0002,
+      "loss": 0.4695,
+      "step": 850
+    },
+    {
+      "epoch": 0.6724003127443315,
+      "grad_norm": 0.6121841669082642,
+      "learning_rate": 0.0002,
+      "loss": 0.4616,
+      "step": 860
+    },
+    {
+      "epoch": 0.6802189210320563,
+      "grad_norm": 0.9026947617530823,
+      "learning_rate": 0.0002,
+      "loss": 0.4763,
+      "step": 870
+    },
+    {
+      "epoch": 0.6880375293197811,
+      "grad_norm": 0.7725462913513184,
+      "learning_rate": 0.0002,
+      "loss": 0.4488,
+      "step": 880
+    },
+    {
+      "epoch": 0.6958561376075059,
+      "grad_norm": 0.896050214767456,
+      "learning_rate": 0.0002,
+      "loss": 0.4334,
+      "step": 890
+    },
+    {
+      "epoch": 0.7036747458952306,
+      "grad_norm": 0.757851243019104,
+      "learning_rate": 0.0002,
+      "loss": 0.4499,
+      "step": 900
+    },
+    {
+      "epoch": 0.7114933541829555,
+      "grad_norm": 0.7172074317932129,
+      "learning_rate": 0.0002,
+      "loss": 0.4466,
+      "step": 910
+    },
+    {
+      "epoch": 0.7193119624706802,
+      "grad_norm": 0.7364748120307922,
+      "learning_rate": 0.0002,
+      "loss": 0.4261,
+      "step": 920
+    },
+    {
+      "epoch": 0.727130570758405,
+      "grad_norm": 0.6359867453575134,
+      "learning_rate": 0.0002,
+      "loss": 0.4531,
+      "step": 930
+    },
+    {
+      "epoch": 0.7349491790461298,
+      "grad_norm": 0.5289077758789062,
+      "learning_rate": 0.0002,
+      "loss": 0.4464,
+      "step": 940
+    },
+    {
+      "epoch": 0.7427677873338546,
+      "grad_norm": 0.6053950786590576,
+      "learning_rate": 0.0002,
+      "loss": 0.4525,
+      "step": 950
+    },
+    {
+      "epoch": 0.7505863956215794,
+      "grad_norm": 0.8122503161430359,
+      "learning_rate": 0.0002,
+      "loss": 0.4568,
+      "step": 960
+    },
+    {
+      "epoch": 0.7584050039093041,
+      "grad_norm": 0.8779653906822205,
+      "learning_rate": 0.0002,
+      "loss": 0.4304,
+      "step": 970
+    },
+    {
+      "epoch": 0.766223612197029,
+      "grad_norm": 0.6312686204910278,
+      "learning_rate": 0.0002,
+      "loss": 0.4337,
+      "step": 980
+    },
+    {
+      "epoch": 0.7740422204847537,
+      "grad_norm": 0.7815352082252502,
+      "learning_rate": 0.0002,
+      "loss": 0.431,
+      "step": 990
+    },
+    {
+      "epoch": 0.7818608287724785,
+      "grad_norm": 0.8249784111976624,
+      "learning_rate": 0.0002,
+      "loss": 0.45,
+      "step": 1000
+    },
+    {
+      "epoch": 0.7896794370602033,
+      "grad_norm": 0.8731566667556763,
+      "learning_rate": 0.0002,
+      "loss": 0.4253,
+      "step": 1010
+    },
+    {
+      "epoch": 0.7974980453479281,
+      "grad_norm": 0.7336146831512451,
+      "learning_rate": 0.0002,
+      "loss": 0.4393,
+      "step": 1020
+    },
+    {
+      "epoch": 0.8053166536356529,
+      "grad_norm": 0.7756309509277344,
+      "learning_rate": 0.0002,
+      "loss": 0.4175,
+      "step": 1030
+    },
+    {
+      "epoch": 0.8131352619233776,
+      "grad_norm": 0.7608133554458618,
+      "learning_rate": 0.0002,
+      "loss": 0.4532,
+      "step": 1040
+    },
+    {
+      "epoch": 0.8209538702111024,
+      "grad_norm": 0.7842742800712585,
+      "learning_rate": 0.0002,
+      "loss": 0.4447,
+      "step": 1050
+    },
+    {
+      "epoch": 0.8287724784988272,
+      "grad_norm": 0.9361023902893066,
+      "learning_rate": 0.0002,
+      "loss": 0.4203,
+      "step": 1060
+    },
+    {
+      "epoch": 0.836591086786552,
+      "grad_norm": 0.8978990912437439,
+      "learning_rate": 0.0002,
+      "loss": 0.4165,
+      "step": 1070
+    },
+    {
+      "epoch": 0.8444096950742768,
+      "grad_norm": 0.7448273301124573,
+      "learning_rate": 0.0002,
+      "loss": 0.4313,
+      "step": 1080
+    },
+    {
+      "epoch": 0.8522283033620016,
+      "grad_norm": 0.9199049472808838,
+      "learning_rate": 0.0002,
+      "loss": 0.4256,
+      "step": 1090
+    },
+    {
+      "epoch": 0.8600469116497264,
+      "grad_norm": 0.7521695494651794,
+      "learning_rate": 0.0002,
+      "loss": 0.4343,
+      "step": 1100
+    },
+    {
+      "epoch": 0.8678655199374511,
+      "grad_norm": 0.7470024228096008,
+      "learning_rate": 0.0002,
+      "loss": 0.4136,
+      "step": 1110
+    },
+    {
+      "epoch": 0.8756841282251759,
+      "grad_norm": 0.5728107690811157,
+      "learning_rate": 0.0002,
+      "loss": 0.4227,
+      "step": 1120
+    },
+    {
+      "epoch": 0.8835027365129007,
+      "grad_norm": 0.8137171268463135,
+      "learning_rate": 0.0002,
+      "loss": 0.4084,
+      "step": 1130
+    },
+    {
+      "epoch": 0.8913213448006255,
+      "grad_norm": 0.7411524057388306,
+      "learning_rate": 0.0002,
+      "loss": 0.4171,
+      "step": 1140
+    },
+    {
+      "epoch": 0.8991399530883503,
+      "grad_norm": 0.705020546913147,
+      "learning_rate": 0.0002,
+      "loss": 0.4061,
+      "step": 1150
+    },
+    {
+      "epoch": 0.9069585613760751,
+      "grad_norm": 0.6366162300109863,
+      "learning_rate": 0.0002,
+      "loss": 0.4196,
+      "step": 1160
+    },
+    {
+      "epoch": 0.9147771696637998,
+      "grad_norm": 0.7566165924072266,
+      "learning_rate": 0.0002,
+      "loss": 0.45,
+      "step": 1170
+    },
+    {
+      "epoch": 0.9225957779515246,
+      "grad_norm": 0.9905046224594116,
+      "learning_rate": 0.0002,
+      "loss": 0.4195,
+      "step": 1180
+    },
+    {
+      "epoch": 0.9304143862392494,
+      "grad_norm": 0.6872445940971375,
+      "learning_rate": 0.0002,
+      "loss": 0.4212,
+      "step": 1190
+    },
+    {
+      "epoch": 0.9382329945269742,
+      "grad_norm": 0.6640546917915344,
+      "learning_rate": 0.0002,
+      "loss": 0.4151,
+      "step": 1200
+    },
+    {
+      "epoch": 0.946051602814699,
+      "grad_norm": 1.0592104196548462,
+      "learning_rate": 0.0002,
+      "loss": 0.4018,
+      "step": 1210
+    },
+    {
+      "epoch": 0.9538702111024238,
+      "grad_norm": 0.9068714380264282,
+      "learning_rate": 0.0002,
+      "loss": 0.3868,
+      "step": 1220
+    },
+    {
+      "epoch": 0.9616888193901486,
+      "grad_norm": 0.7440975308418274,
+      "learning_rate": 0.0002,
+      "loss": 0.3777,
+      "step": 1230
+    },
+    {
+      "epoch": 0.9695074276778733,
+      "grad_norm": 0.9631947875022888,
+      "learning_rate": 0.0002,
+      "loss": 0.4086,
+      "step": 1240
+    },
+    {
+      "epoch": 0.9773260359655981,
+      "grad_norm": 0.708501935005188,
+      "learning_rate": 0.0002,
+      "loss": 0.4042,
+      "step": 1250
+    },
+    {
+      "epoch": 0.9851446442533229,
+      "grad_norm": 0.664806604385376,
+      "learning_rate": 0.0002,
+      "loss": 0.3958,
+      "step": 1260
+    },
+    {
+      "epoch": 0.9929632525410477,
+      "grad_norm": 0.6895506978034973,
+      "learning_rate": 0.0002,
+      "loss": 0.3944,
+      "step": 1270
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 0.39914223551750183,
+      "eval_runtime": 92.7363,
+      "eval_samples_per_second": 3.936,
+      "eval_steps_per_second": 0.496,
+      "step": 1279
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 10232,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.61157754585088e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-1279/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:67c127eb44653a0272979e2cc773b017f5ff5cfc6ed92250a567d1213376a07e
+size 5560

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-2558/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-2558/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-2558/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e05927acea8c4fd33c10364683fdbe90867e3de2b653fd130bb0ddb11ef6f181
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-2558/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:010da7e613147aba1642716ac2a3a315c26e35c8c186915624ab37255cbea6c7
+size 55532666

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-2558/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:70f524b7f798a165ec99fbbbd5f785ae5bc43c1ec2b1fc5c868b75f0401e3df3
+size 14244

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-2558/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:535a1c733634b54951dcb0cda84c1e0fdf82fd1926e3a82e67a802d229840d0b
+size 1064

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-2558/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-2558/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-2558/trainer_state.json

@@ -0,0 +1,1834 @@
+{
+  "best_metric": 0.31702762842178345,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-2558",
+  "epoch": 2.0,
+  "eval_steps": 10,
+  "global_step": 2558,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.007818608287724784,
+      "grad_norm": 1.1001313924789429,
+      "learning_rate": 0.0002,
+      "loss": 1.7403,
+      "step": 10
+    },
+    {
+      "epoch": 0.01563721657544957,
+      "grad_norm": 1.3678615093231201,
+      "learning_rate": 0.0002,
+      "loss": 1.1795,
+      "step": 20
+    },
+    {
+      "epoch": 0.023455824863174355,
+      "grad_norm": 1.2362287044525146,
+      "learning_rate": 0.0002,
+      "loss": 1.0159,
+      "step": 30
+    },
+    {
+      "epoch": 0.03127443315089914,
+      "grad_norm": 0.9754743576049805,
+      "learning_rate": 0.0002,
+      "loss": 0.9315,
+      "step": 40
+    },
+    {
+      "epoch": 0.039093041438623924,
+      "grad_norm": 1.0572363138198853,
+      "learning_rate": 0.0002,
+      "loss": 0.8015,
+      "step": 50
+    },
+    {
+      "epoch": 0.04691164972634871,
+      "grad_norm": 0.8838446140289307,
+      "learning_rate": 0.0002,
+      "loss": 0.7264,
+      "step": 60
+    },
+    {
+      "epoch": 0.054730258014073496,
+      "grad_norm": 0.831045389175415,
+      "learning_rate": 0.0002,
+      "loss": 0.6807,
+      "step": 70
+    },
+    {
+      "epoch": 0.06254886630179828,
+      "grad_norm": 0.721530556678772,
+      "learning_rate": 0.0002,
+      "loss": 0.6731,
+      "step": 80
+    },
+    {
+      "epoch": 0.07036747458952307,
+      "grad_norm": 0.6918481588363647,
+      "learning_rate": 0.0002,
+      "loss": 0.6427,
+      "step": 90
+    },
+    {
+      "epoch": 0.07818608287724785,
+      "grad_norm": 0.8207236528396606,
+      "learning_rate": 0.0002,
+      "loss": 0.6719,
+      "step": 100
+    },
+    {
+      "epoch": 0.08600469116497264,
+      "grad_norm": 0.9405701756477356,
+      "learning_rate": 0.0002,
+      "loss": 0.6533,
+      "step": 110
+    },
+    {
+      "epoch": 0.09382329945269742,
+      "grad_norm": 0.7389968037605286,
+      "learning_rate": 0.0002,
+      "loss": 0.6102,
+      "step": 120
+    },
+    {
+      "epoch": 0.1016419077404222,
+      "grad_norm": 0.7102212905883789,
+      "learning_rate": 0.0002,
+      "loss": 0.6561,
+      "step": 130
+    },
+    {
+      "epoch": 0.10946051602814699,
+      "grad_norm": 0.6546808481216431,
+      "learning_rate": 0.0002,
+      "loss": 0.6222,
+      "step": 140
+    },
+    {
+      "epoch": 0.11727912431587177,
+      "grad_norm": 1.0097354650497437,
+      "learning_rate": 0.0002,
+      "loss": 0.6369,
+      "step": 150
+    },
+    {
+      "epoch": 0.12509773260359655,
+      "grad_norm": 0.5699719786643982,
+      "learning_rate": 0.0002,
+      "loss": 0.6495,
+      "step": 160
+    },
+    {
+      "epoch": 0.13291634089132134,
+      "grad_norm": 0.6371490359306335,
+      "learning_rate": 0.0002,
+      "loss": 0.6421,
+      "step": 170
+    },
+    {
+      "epoch": 0.14073494917904614,
+      "grad_norm": 0.8385978937149048,
+      "learning_rate": 0.0002,
+      "loss": 0.6101,
+      "step": 180
+    },
+    {
+      "epoch": 0.1485535574667709,
+      "grad_norm": 0.6549069285392761,
+      "learning_rate": 0.0002,
+      "loss": 0.5957,
+      "step": 190
+    },
+    {
+      "epoch": 0.1563721657544957,
+      "grad_norm": 0.5297655463218689,
+      "learning_rate": 0.0002,
+      "loss": 0.5834,
+      "step": 200
+    },
+    {
+      "epoch": 0.1641907740422205,
+      "grad_norm": 0.6385621428489685,
+      "learning_rate": 0.0002,
+      "loss": 0.6054,
+      "step": 210
+    },
+    {
+      "epoch": 0.17200938232994528,
+      "grad_norm": 0.6723865866661072,
+      "learning_rate": 0.0002,
+      "loss": 0.6174,
+      "step": 220
+    },
+    {
+      "epoch": 0.17982799061767005,
+      "grad_norm": 0.6121484041213989,
+      "learning_rate": 0.0002,
+      "loss": 0.5951,
+      "step": 230
+    },
+    {
+      "epoch": 0.18764659890539484,
+      "grad_norm": 0.619121789932251,
+      "learning_rate": 0.0002,
+      "loss": 0.6046,
+      "step": 240
+    },
+    {
+      "epoch": 0.19546520719311963,
+      "grad_norm": 0.49208253622055054,
+      "learning_rate": 0.0002,
+      "loss": 0.5764,
+      "step": 250
+    },
+    {
+      "epoch": 0.2032838154808444,
+      "grad_norm": 0.4991285800933838,
+      "learning_rate": 0.0002,
+      "loss": 0.5679,
+      "step": 260
+    },
+    {
+      "epoch": 0.2111024237685692,
+      "grad_norm": 0.7622858285903931,
+      "learning_rate": 0.0002,
+      "loss": 0.5777,
+      "step": 270
+    },
+    {
+      "epoch": 0.21892103205629398,
+      "grad_norm": 0.5988286733627319,
+      "learning_rate": 0.0002,
+      "loss": 0.5504,
+      "step": 280
+    },
+    {
+      "epoch": 0.22673964034401878,
+      "grad_norm": 0.510055661201477,
+      "learning_rate": 0.0002,
+      "loss": 0.5801,
+      "step": 290
+    },
+    {
+      "epoch": 0.23455824863174354,
+      "grad_norm": 0.47940748929977417,
+      "learning_rate": 0.0002,
+      "loss": 0.5788,
+      "step": 300
+    },
+    {
+      "epoch": 0.24237685691946834,
+      "grad_norm": 0.5604141354560852,
+      "learning_rate": 0.0002,
+      "loss": 0.6093,
+      "step": 310
+    },
+    {
+      "epoch": 0.2501954652071931,
+      "grad_norm": 0.479342520236969,
+      "learning_rate": 0.0002,
+      "loss": 0.5641,
+      "step": 320
+    },
+    {
+      "epoch": 0.2580140734949179,
+      "grad_norm": 0.5401737093925476,
+      "learning_rate": 0.0002,
+      "loss": 0.5304,
+      "step": 330
+    },
+    {
+      "epoch": 0.2658326817826427,
+      "grad_norm": 0.5436083674430847,
+      "learning_rate": 0.0002,
+      "loss": 0.5547,
+      "step": 340
+    },
+    {
+      "epoch": 0.2736512900703675,
+      "grad_norm": 0.6402848362922668,
+      "learning_rate": 0.0002,
+      "loss": 0.5724,
+      "step": 350
+    },
+    {
+      "epoch": 0.2814698983580923,
+      "grad_norm": 0.4756305515766144,
+      "learning_rate": 0.0002,
+      "loss": 0.5452,
+      "step": 360
+    },
+    {
+      "epoch": 0.28928850664581707,
+      "grad_norm": 0.5536904335021973,
+      "learning_rate": 0.0002,
+      "loss": 0.5936,
+      "step": 370
+    },
+    {
+      "epoch": 0.2971071149335418,
+      "grad_norm": 0.6187605857849121,
+      "learning_rate": 0.0002,
+      "loss": 0.5827,
+      "step": 380
+    },
+    {
+      "epoch": 0.3049257232212666,
+      "grad_norm": 0.5297170877456665,
+      "learning_rate": 0.0002,
+      "loss": 0.5435,
+      "step": 390
+    },
+    {
+      "epoch": 0.3127443315089914,
+      "grad_norm": 0.5808210372924805,
+      "learning_rate": 0.0002,
+      "loss": 0.583,
+      "step": 400
+    },
+    {
+      "epoch": 0.3205629397967162,
+      "grad_norm": 0.7509300708770752,
+      "learning_rate": 0.0002,
+      "loss": 0.5587,
+      "step": 410
+    },
+    {
+      "epoch": 0.328381548084441,
+      "grad_norm": 0.5674371719360352,
+      "learning_rate": 0.0002,
+      "loss": 0.5439,
+      "step": 420
+    },
+    {
+      "epoch": 0.33620015637216577,
+      "grad_norm": 0.5833905339241028,
+      "learning_rate": 0.0002,
+      "loss": 0.5431,
+      "step": 430
+    },
+    {
+      "epoch": 0.34401876465989056,
+      "grad_norm": 0.537860095500946,
+      "learning_rate": 0.0002,
+      "loss": 0.5527,
+      "step": 440
+    },
+    {
+      "epoch": 0.3518373729476153,
+      "grad_norm": 0.5747054219245911,
+      "learning_rate": 0.0002,
+      "loss": 0.528,
+      "step": 450
+    },
+    {
+      "epoch": 0.3596559812353401,
+      "grad_norm": 0.6268995404243469,
+      "learning_rate": 0.0002,
+      "loss": 0.5281,
+      "step": 460
+    },
+    {
+      "epoch": 0.3674745895230649,
+      "grad_norm": 0.640737771987915,
+      "learning_rate": 0.0002,
+      "loss": 0.5394,
+      "step": 470
+    },
+    {
+      "epoch": 0.3752931978107897,
+      "grad_norm": 0.6593332290649414,
+      "learning_rate": 0.0002,
+      "loss": 0.5488,
+      "step": 480
+    },
+    {
+      "epoch": 0.3831118060985145,
+      "grad_norm": 0.5977872014045715,
+      "learning_rate": 0.0002,
+      "loss": 0.5384,
+      "step": 490
+    },
+    {
+      "epoch": 0.39093041438623927,
+      "grad_norm": 0.6118639707565308,
+      "learning_rate": 0.0002,
+      "loss": 0.5154,
+      "step": 500
+    },
+    {
+      "epoch": 0.39874902267396406,
+      "grad_norm": 1.2320106029510498,
+      "learning_rate": 0.0002,
+      "loss": 0.5481,
+      "step": 510
+    },
+    {
+      "epoch": 0.4065676309616888,
+      "grad_norm": 0.6538275480270386,
+      "learning_rate": 0.0002,
+      "loss": 0.5225,
+      "step": 520
+    },
+    {
+      "epoch": 0.4143862392494136,
+      "grad_norm": 0.5839771032333374,
+      "learning_rate": 0.0002,
+      "loss": 0.5322,
+      "step": 530
+    },
+    {
+      "epoch": 0.4222048475371384,
+      "grad_norm": 0.6228310465812683,
+      "learning_rate": 0.0002,
+      "loss": 0.5075,
+      "step": 540
+    },
+    {
+      "epoch": 0.4300234558248632,
+      "grad_norm": 0.653239905834198,
+      "learning_rate": 0.0002,
+      "loss": 0.5177,
+      "step": 550
+    },
+    {
+      "epoch": 0.43784206411258797,
+      "grad_norm": 0.6995204091072083,
+      "learning_rate": 0.0002,
+      "loss": 0.5209,
+      "step": 560
+    },
+    {
+      "epoch": 0.44566067240031276,
+      "grad_norm": 0.5197685956954956,
+      "learning_rate": 0.0002,
+      "loss": 0.4914,
+      "step": 570
+    },
+    {
+      "epoch": 0.45347928068803756,
+      "grad_norm": 0.6786061525344849,
+      "learning_rate": 0.0002,
+      "loss": 0.4951,
+      "step": 580
+    },
+    {
+      "epoch": 0.4612978889757623,
+      "grad_norm": 0.6599542498588562,
+      "learning_rate": 0.0002,
+      "loss": 0.5107,
+      "step": 590
+    },
+    {
+      "epoch": 0.4691164972634871,
+      "grad_norm": 0.5535895228385925,
+      "learning_rate": 0.0002,
+      "loss": 0.514,
+      "step": 600
+    },
+    {
+      "epoch": 0.4769351055512119,
+      "grad_norm": 0.667336642742157,
+      "learning_rate": 0.0002,
+      "loss": 0.5206,
+      "step": 610
+    },
+    {
+      "epoch": 0.4847537138389367,
+      "grad_norm": 0.5404567718505859,
+      "learning_rate": 0.0002,
+      "loss": 0.4918,
+      "step": 620
+    },
+    {
+      "epoch": 0.49257232212666147,
+      "grad_norm": 0.5316283702850342,
+      "learning_rate": 0.0002,
+      "loss": 0.501,
+      "step": 630
+    },
+    {
+      "epoch": 0.5003909304143862,
+      "grad_norm": 0.6398797035217285,
+      "learning_rate": 0.0002,
+      "loss": 0.5026,
+      "step": 640
+    },
+    {
+      "epoch": 0.508209538702111,
+      "grad_norm": 0.6197776794433594,
+      "learning_rate": 0.0002,
+      "loss": 0.5017,
+      "step": 650
+    },
+    {
+      "epoch": 0.5160281469898358,
+      "grad_norm": 0.8760672807693481,
+      "learning_rate": 0.0002,
+      "loss": 0.4905,
+      "step": 660
+    },
+    {
+      "epoch": 0.5238467552775606,
+      "grad_norm": 0.7163699269294739,
+      "learning_rate": 0.0002,
+      "loss": 0.4798,
+      "step": 670
+    },
+    {
+      "epoch": 0.5316653635652854,
+      "grad_norm": 0.7599782347679138,
+      "learning_rate": 0.0002,
+      "loss": 0.4974,
+      "step": 680
+    },
+    {
+      "epoch": 0.5394839718530101,
+      "grad_norm": 0.5682359337806702,
+      "learning_rate": 0.0002,
+      "loss": 0.4942,
+      "step": 690
+    },
+    {
+      "epoch": 0.547302580140735,
+      "grad_norm": 0.6667918562889099,
+      "learning_rate": 0.0002,
+      "loss": 0.4826,
+      "step": 700
+    },
+    {
+      "epoch": 0.5551211884284597,
+      "grad_norm": 0.9613338112831116,
+      "learning_rate": 0.0002,
+      "loss": 0.475,
+      "step": 710
+    },
+    {
+      "epoch": 0.5629397967161845,
+      "grad_norm": 0.5658668279647827,
+      "learning_rate": 0.0002,
+      "loss": 0.5005,
+      "step": 720
+    },
+    {
+      "epoch": 0.5707584050039093,
+      "grad_norm": 0.7233469486236572,
+      "learning_rate": 0.0002,
+      "loss": 0.4697,
+      "step": 730
+    },
+    {
+      "epoch": 0.5785770132916341,
+      "grad_norm": 0.5586039423942566,
+      "learning_rate": 0.0002,
+      "loss": 0.4659,
+      "step": 740
+    },
+    {
+      "epoch": 0.5863956215793589,
+      "grad_norm": 0.8161298632621765,
+      "learning_rate": 0.0002,
+      "loss": 0.4893,
+      "step": 750
+    },
+    {
+      "epoch": 0.5942142298670836,
+      "grad_norm": 0.7192191481590271,
+      "learning_rate": 0.0002,
+      "loss": 0.4678,
+      "step": 760
+    },
+    {
+      "epoch": 0.6020328381548085,
+      "grad_norm": 0.5711938738822937,
+      "learning_rate": 0.0002,
+      "loss": 0.4765,
+      "step": 770
+    },
+    {
+      "epoch": 0.6098514464425332,
+      "grad_norm": 0.5471241474151611,
+      "learning_rate": 0.0002,
+      "loss": 0.4676,
+      "step": 780
+    },
+    {
+      "epoch": 0.617670054730258,
+      "grad_norm": 0.5709220767021179,
+      "learning_rate": 0.0002,
+      "loss": 0.4628,
+      "step": 790
+    },
+    {
+      "epoch": 0.6254886630179828,
+      "grad_norm": 0.882448673248291,
+      "learning_rate": 0.0002,
+      "loss": 0.4695,
+      "step": 800
+    },
+    {
+      "epoch": 0.6333072713057076,
+      "grad_norm": 0.5136802196502686,
+      "learning_rate": 0.0002,
+      "loss": 0.4611,
+      "step": 810
+    },
+    {
+      "epoch": 0.6411258795934324,
+      "grad_norm": 0.6611698865890503,
+      "learning_rate": 0.0002,
+      "loss": 0.4663,
+      "step": 820
+    },
+    {
+      "epoch": 0.6489444878811571,
+      "grad_norm": 0.7050015926361084,
+      "learning_rate": 0.0002,
+      "loss": 0.4962,
+      "step": 830
+    },
+    {
+      "epoch": 0.656763096168882,
+      "grad_norm": 0.5757645964622498,
+      "learning_rate": 0.0002,
+      "loss": 0.4624,
+      "step": 840
+    },
+    {
+      "epoch": 0.6645817044566067,
+      "grad_norm": 0.6651985049247742,
+      "learning_rate": 0.0002,
+      "loss": 0.4695,
+      "step": 850
+    },
+    {
+      "epoch": 0.6724003127443315,
+      "grad_norm": 0.6121841669082642,
+      "learning_rate": 0.0002,
+      "loss": 0.4616,
+      "step": 860
+    },
+    {
+      "epoch": 0.6802189210320563,
+      "grad_norm": 0.9026947617530823,
+      "learning_rate": 0.0002,
+      "loss": 0.4763,
+      "step": 870
+    },
+    {
+      "epoch": 0.6880375293197811,
+      "grad_norm": 0.7725462913513184,
+      "learning_rate": 0.0002,
+      "loss": 0.4488,
+      "step": 880
+    },
+    {
+      "epoch": 0.6958561376075059,
+      "grad_norm": 0.896050214767456,
+      "learning_rate": 0.0002,
+      "loss": 0.4334,
+      "step": 890
+    },
+    {
+      "epoch": 0.7036747458952306,
+      "grad_norm": 0.757851243019104,
+      "learning_rate": 0.0002,
+      "loss": 0.4499,
+      "step": 900
+    },
+    {
+      "epoch": 0.7114933541829555,
+      "grad_norm": 0.7172074317932129,
+      "learning_rate": 0.0002,
+      "loss": 0.4466,
+      "step": 910
+    },
+    {
+      "epoch": 0.7193119624706802,
+      "grad_norm": 0.7364748120307922,
+      "learning_rate": 0.0002,
+      "loss": 0.4261,
+      "step": 920
+    },
+    {
+      "epoch": 0.727130570758405,
+      "grad_norm": 0.6359867453575134,
+      "learning_rate": 0.0002,
+      "loss": 0.4531,
+      "step": 930
+    },
+    {
+      "epoch": 0.7349491790461298,
+      "grad_norm": 0.5289077758789062,
+      "learning_rate": 0.0002,
+      "loss": 0.4464,
+      "step": 940
+    },
+    {
+      "epoch": 0.7427677873338546,
+      "grad_norm": 0.6053950786590576,
+      "learning_rate": 0.0002,
+      "loss": 0.4525,
+      "step": 950
+    },
+    {
+      "epoch": 0.7505863956215794,
+      "grad_norm": 0.8122503161430359,
+      "learning_rate": 0.0002,
+      "loss": 0.4568,
+      "step": 960
+    },
+    {
+      "epoch": 0.7584050039093041,
+      "grad_norm": 0.8779653906822205,
+      "learning_rate": 0.0002,
+      "loss": 0.4304,
+      "step": 970
+    },
+    {
+      "epoch": 0.766223612197029,
+      "grad_norm": 0.6312686204910278,
+      "learning_rate": 0.0002,
+      "loss": 0.4337,
+      "step": 980
+    },
+    {
+      "epoch": 0.7740422204847537,
+      "grad_norm": 0.7815352082252502,
+      "learning_rate": 0.0002,
+      "loss": 0.431,
+      "step": 990
+    },
+    {
+      "epoch": 0.7818608287724785,
+      "grad_norm": 0.8249784111976624,
+      "learning_rate": 0.0002,
+      "loss": 0.45,
+      "step": 1000
+    },
+    {
+      "epoch": 0.7896794370602033,
+      "grad_norm": 0.8731566667556763,
+      "learning_rate": 0.0002,
+      "loss": 0.4253,
+      "step": 1010
+    },
+    {
+      "epoch": 0.7974980453479281,
+      "grad_norm": 0.7336146831512451,
+      "learning_rate": 0.0002,
+      "loss": 0.4393,
+      "step": 1020
+    },
+    {
+      "epoch": 0.8053166536356529,
+      "grad_norm": 0.7756309509277344,
+      "learning_rate": 0.0002,
+      "loss": 0.4175,
+      "step": 1030
+    },
+    {
+      "epoch": 0.8131352619233776,
+      "grad_norm": 0.7608133554458618,
+      "learning_rate": 0.0002,
+      "loss": 0.4532,
+      "step": 1040
+    },
+    {
+      "epoch": 0.8209538702111024,
+      "grad_norm": 0.7842742800712585,
+      "learning_rate": 0.0002,
+      "loss": 0.4447,
+      "step": 1050
+    },
+    {
+      "epoch": 0.8287724784988272,
+      "grad_norm": 0.9361023902893066,
+      "learning_rate": 0.0002,
+      "loss": 0.4203,
+      "step": 1060
+    },
+    {
+      "epoch": 0.836591086786552,
+      "grad_norm": 0.8978990912437439,
+      "learning_rate": 0.0002,
+      "loss": 0.4165,
+      "step": 1070
+    },
+    {
+      "epoch": 0.8444096950742768,
+      "grad_norm": 0.7448273301124573,
+      "learning_rate": 0.0002,
+      "loss": 0.4313,
+      "step": 1080
+    },
+    {
+      "epoch": 0.8522283033620016,
+      "grad_norm": 0.9199049472808838,
+      "learning_rate": 0.0002,
+      "loss": 0.4256,
+      "step": 1090
+    },
+    {
+      "epoch": 0.8600469116497264,
+      "grad_norm": 0.7521695494651794,
+      "learning_rate": 0.0002,
+      "loss": 0.4343,
+      "step": 1100
+    },
+    {
+      "epoch": 0.8678655199374511,
+      "grad_norm": 0.7470024228096008,
+      "learning_rate": 0.0002,
+      "loss": 0.4136,
+      "step": 1110
+    },
+    {
+      "epoch": 0.8756841282251759,
+      "grad_norm": 0.5728107690811157,
+      "learning_rate": 0.0002,
+      "loss": 0.4227,
+      "step": 1120
+    },
+    {
+      "epoch": 0.8835027365129007,
+      "grad_norm": 0.8137171268463135,
+      "learning_rate": 0.0002,
+      "loss": 0.4084,
+      "step": 1130
+    },
+    {
+      "epoch": 0.8913213448006255,
+      "grad_norm": 0.7411524057388306,
+      "learning_rate": 0.0002,
+      "loss": 0.4171,
+      "step": 1140
+    },
+    {
+      "epoch": 0.8991399530883503,
+      "grad_norm": 0.705020546913147,
+      "learning_rate": 0.0002,
+      "loss": 0.4061,
+      "step": 1150
+    },
+    {
+      "epoch": 0.9069585613760751,
+      "grad_norm": 0.6366162300109863,
+      "learning_rate": 0.0002,
+      "loss": 0.4196,
+      "step": 1160
+    },
+    {
+      "epoch": 0.9147771696637998,
+      "grad_norm": 0.7566165924072266,
+      "learning_rate": 0.0002,
+      "loss": 0.45,
+      "step": 1170
+    },
+    {
+      "epoch": 0.9225957779515246,
+      "grad_norm": 0.9905046224594116,
+      "learning_rate": 0.0002,
+      "loss": 0.4195,
+      "step": 1180
+    },
+    {
+      "epoch": 0.9304143862392494,
+      "grad_norm": 0.6872445940971375,
+      "learning_rate": 0.0002,
+      "loss": 0.4212,
+      "step": 1190
+    },
+    {
+      "epoch": 0.9382329945269742,
+      "grad_norm": 0.6640546917915344,
+      "learning_rate": 0.0002,
+      "loss": 0.4151,
+      "step": 1200
+    },
+    {
+      "epoch": 0.946051602814699,
+      "grad_norm": 1.0592104196548462,
+      "learning_rate": 0.0002,
+      "loss": 0.4018,
+      "step": 1210
+    },
+    {
+      "epoch": 0.9538702111024238,
+      "grad_norm": 0.9068714380264282,
+      "learning_rate": 0.0002,
+      "loss": 0.3868,
+      "step": 1220
+    },
+    {
+      "epoch": 0.9616888193901486,
+      "grad_norm": 0.7440975308418274,
+      "learning_rate": 0.0002,
+      "loss": 0.3777,
+      "step": 1230
+    },
+    {
+      "epoch": 0.9695074276778733,
+      "grad_norm": 0.9631947875022888,
+      "learning_rate": 0.0002,
+      "loss": 0.4086,
+      "step": 1240
+    },
+    {
+      "epoch": 0.9773260359655981,
+      "grad_norm": 0.708501935005188,
+      "learning_rate": 0.0002,
+      "loss": 0.4042,
+      "step": 1250
+    },
+    {
+      "epoch": 0.9851446442533229,
+      "grad_norm": 0.664806604385376,
+      "learning_rate": 0.0002,
+      "loss": 0.3958,
+      "step": 1260
+    },
+    {
+      "epoch": 0.9929632525410477,
+      "grad_norm": 0.6895506978034973,
+      "learning_rate": 0.0002,
+      "loss": 0.3944,
+      "step": 1270
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 0.39914223551750183,
+      "eval_runtime": 92.7363,
+      "eval_samples_per_second": 3.936,
+      "eval_steps_per_second": 0.496,
+      "step": 1279
+    },
+    {
+      "epoch": 1.0007818608287724,
+      "grad_norm": 0.6349056363105774,
+      "learning_rate": 0.0002,
+      "loss": 0.3917,
+      "step": 1280
+    },
+    {
+      "epoch": 1.0086004691164974,
+      "grad_norm": 0.9110808968544006,
+      "learning_rate": 0.0002,
+      "loss": 0.3851,
+      "step": 1290
+    },
+    {
+      "epoch": 1.016419077404222,
+      "grad_norm": 0.8718474507331848,
+      "learning_rate": 0.0002,
+      "loss": 0.3743,
+      "step": 1300
+    },
+    {
+      "epoch": 1.0242376856919468,
+      "grad_norm": 0.9496098756790161,
+      "learning_rate": 0.0002,
+      "loss": 0.3735,
+      "step": 1310
+    },
+    {
+      "epoch": 1.0320562939796716,
+      "grad_norm": 0.5553750991821289,
+      "learning_rate": 0.0002,
+      "loss": 0.3688,
+      "step": 1320
+    },
+    {
+      "epoch": 1.0398749022673963,
+      "grad_norm": 0.8498914241790771,
+      "learning_rate": 0.0002,
+      "loss": 0.3701,
+      "step": 1330
+    },
+    {
+      "epoch": 1.0476935105551213,
+      "grad_norm": 0.6435985565185547,
+      "learning_rate": 0.0002,
+      "loss": 0.3739,
+      "step": 1340
+    },
+    {
+      "epoch": 1.055512118842846,
+      "grad_norm": 0.8342816233634949,
+      "learning_rate": 0.0002,
+      "loss": 0.3778,
+      "step": 1350
+    },
+    {
+      "epoch": 1.0633307271305708,
+      "grad_norm": 0.6142820715904236,
+      "learning_rate": 0.0002,
+      "loss": 0.3873,
+      "step": 1360
+    },
+    {
+      "epoch": 1.0711493354182955,
+      "grad_norm": 0.9387786984443665,
+      "learning_rate": 0.0002,
+      "loss": 0.37,
+      "step": 1370
+    },
+    {
+      "epoch": 1.0789679437060202,
+      "grad_norm": 0.8187823295593262,
+      "learning_rate": 0.0002,
+      "loss": 0.3568,
+      "step": 1380
+    },
+    {
+      "epoch": 1.0867865519937452,
+      "grad_norm": 0.7127028107643127,
+      "learning_rate": 0.0002,
+      "loss": 0.3852,
+      "step": 1390
+    },
+    {
+      "epoch": 1.09460516028147,
+      "grad_norm": 0.7990315556526184,
+      "learning_rate": 0.0002,
+      "loss": 0.3732,
+      "step": 1400
+    },
+    {
+      "epoch": 1.1024237685691947,
+      "grad_norm": 1.0349947214126587,
+      "learning_rate": 0.0002,
+      "loss": 0.3764,
+      "step": 1410
+    },
+    {
+      "epoch": 1.1102423768569194,
+      "grad_norm": 0.5400282740592957,
+      "learning_rate": 0.0002,
+      "loss": 0.3518,
+      "step": 1420
+    },
+    {
+      "epoch": 1.1180609851446444,
+      "grad_norm": 0.9225337505340576,
+      "learning_rate": 0.0002,
+      "loss": 0.3661,
+      "step": 1430
+    },
+    {
+      "epoch": 1.125879593432369,
+      "grad_norm": 0.7267957925796509,
+      "learning_rate": 0.0002,
+      "loss": 0.3661,
+      "step": 1440
+    },
+    {
+      "epoch": 1.1336982017200938,
+      "grad_norm": 0.6454635858535767,
+      "learning_rate": 0.0002,
+      "loss": 0.3745,
+      "step": 1450
+    },
+    {
+      "epoch": 1.1415168100078186,
+      "grad_norm": 1.0288119316101074,
+      "learning_rate": 0.0002,
+      "loss": 0.3675,
+      "step": 1460
+    },
+    {
+      "epoch": 1.1493354182955433,
+      "grad_norm": 0.6535518169403076,
+      "learning_rate": 0.0002,
+      "loss": 0.3807,
+      "step": 1470
+    },
+    {
+      "epoch": 1.1571540265832683,
+      "grad_norm": 0.6860265731811523,
+      "learning_rate": 0.0002,
+      "loss": 0.3664,
+      "step": 1480
+    },
+    {
+      "epoch": 1.164972634870993,
+      "grad_norm": 0.9102330803871155,
+      "learning_rate": 0.0002,
+      "loss": 0.3679,
+      "step": 1490
+    },
+    {
+      "epoch": 1.1727912431587177,
+      "grad_norm": 0.6989532709121704,
+      "learning_rate": 0.0002,
+      "loss": 0.364,
+      "step": 1500
+    },
+    {
+      "epoch": 1.1806098514464425,
+      "grad_norm": 1.1313148736953735,
+      "learning_rate": 0.0002,
+      "loss": 0.3592,
+      "step": 1510
+    },
+    {
+      "epoch": 1.1884284597341672,
+      "grad_norm": 0.6841519474983215,
+      "learning_rate": 0.0002,
+      "loss": 0.363,
+      "step": 1520
+    },
+    {
+      "epoch": 1.1962470680218922,
+      "grad_norm": 0.7030880451202393,
+      "learning_rate": 0.0002,
+      "loss": 0.3736,
+      "step": 1530
+    },
+    {
+      "epoch": 1.204065676309617,
+      "grad_norm": 0.6326259970664978,
+      "learning_rate": 0.0002,
+      "loss": 0.3665,
+      "step": 1540
+    },
+    {
+      "epoch": 1.2118842845973417,
+      "grad_norm": 0.8820798993110657,
+      "learning_rate": 0.0002,
+      "loss": 0.3734,
+      "step": 1550
+    },
+    {
+      "epoch": 1.2197028928850664,
+      "grad_norm": 0.8624477386474609,
+      "learning_rate": 0.0002,
+      "loss": 0.3581,
+      "step": 1560
+    },
+    {
+      "epoch": 1.2275215011727911,
+      "grad_norm": 0.6675921678543091,
+      "learning_rate": 0.0002,
+      "loss": 0.3722,
+      "step": 1570
+    },
+    {
+      "epoch": 1.235340109460516,
+      "grad_norm": 1.0099470615386963,
+      "learning_rate": 0.0002,
+      "loss": 0.3661,
+      "step": 1580
+    },
+    {
+      "epoch": 1.2431587177482408,
+      "grad_norm": 0.8204535841941833,
+      "learning_rate": 0.0002,
+      "loss": 0.3674,
+      "step": 1590
+    },
+    {
+      "epoch": 1.2509773260359656,
+      "grad_norm": 0.7338495850563049,
+      "learning_rate": 0.0002,
+      "loss": 0.3655,
+      "step": 1600
+    },
+    {
+      "epoch": 1.2587959343236903,
+      "grad_norm": 0.7446017861366272,
+      "learning_rate": 0.0002,
+      "loss": 0.3706,
+      "step": 1610
+    },
+    {
+      "epoch": 1.266614542611415,
+      "grad_norm": 0.7122478485107422,
+      "learning_rate": 0.0002,
+      "loss": 0.3487,
+      "step": 1620
+    },
+    {
+      "epoch": 1.27443315089914,
+      "grad_norm": 0.8905506730079651,
+      "learning_rate": 0.0002,
+      "loss": 0.3749,
+      "step": 1630
+    },
+    {
+      "epoch": 1.2822517591868647,
+      "grad_norm": 0.8287106156349182,
+      "learning_rate": 0.0002,
+      "loss": 0.3465,
+      "step": 1640
+    },
+    {
+      "epoch": 1.2900703674745895,
+      "grad_norm": 0.6574750542640686,
+      "learning_rate": 0.0002,
+      "loss": 0.341,
+      "step": 1650
+    },
+    {
+      "epoch": 1.2978889757623144,
+      "grad_norm": 0.6535889506340027,
+      "learning_rate": 0.0002,
+      "loss": 0.3467,
+      "step": 1660
+    },
+    {
+      "epoch": 1.3057075840500392,
+      "grad_norm": 0.7493264675140381,
+      "learning_rate": 0.0002,
+      "loss": 0.3632,
+      "step": 1670
+    },
+    {
+      "epoch": 1.313526192337764,
+      "grad_norm": 0.8663034439086914,
+      "learning_rate": 0.0002,
+      "loss": 0.3607,
+      "step": 1680
+    },
+    {
+      "epoch": 1.3213448006254886,
+      "grad_norm": 0.7360671758651733,
+      "learning_rate": 0.0002,
+      "loss": 0.3605,
+      "step": 1690
+    },
+    {
+      "epoch": 1.3291634089132134,
+      "grad_norm": 0.7367114424705505,
+      "learning_rate": 0.0002,
+      "loss": 0.3674,
+      "step": 1700
+    },
+    {
+      "epoch": 1.3369820172009383,
+      "grad_norm": 0.8030956983566284,
+      "learning_rate": 0.0002,
+      "loss": 0.3593,
+      "step": 1710
+    },
+    {
+      "epoch": 1.344800625488663,
+      "grad_norm": 0.9848132133483887,
+      "learning_rate": 0.0002,
+      "loss": 0.3536,
+      "step": 1720
+    },
+    {
+      "epoch": 1.3526192337763878,
+      "grad_norm": 0.8279334306716919,
+      "learning_rate": 0.0002,
+      "loss": 0.3639,
+      "step": 1730
+    },
+    {
+      "epoch": 1.3604378420641126,
+      "grad_norm": 0.5222904682159424,
+      "learning_rate": 0.0002,
+      "loss": 0.3508,
+      "step": 1740
+    },
+    {
+      "epoch": 1.3682564503518373,
+      "grad_norm": 0.794312596321106,
+      "learning_rate": 0.0002,
+      "loss": 0.3534,
+      "step": 1750
+    },
+    {
+      "epoch": 1.3760750586395623,
+      "grad_norm": 0.737553060054779,
+      "learning_rate": 0.0002,
+      "loss": 0.3468,
+      "step": 1760
+    },
+    {
+      "epoch": 1.383893666927287,
+      "grad_norm": 0.6765537858009338,
+      "learning_rate": 0.0002,
+      "loss": 0.3446,
+      "step": 1770
+    },
+    {
+      "epoch": 1.3917122752150117,
+      "grad_norm": 0.8873873353004456,
+      "learning_rate": 0.0002,
+      "loss": 0.3328,
+      "step": 1780
+    },
+    {
+      "epoch": 1.3995308835027365,
+      "grad_norm": 0.8087615966796875,
+      "learning_rate": 0.0002,
+      "loss": 0.334,
+      "step": 1790
+    },
+    {
+      "epoch": 1.4073494917904612,
+      "grad_norm": 0.7812146544456482,
+      "learning_rate": 0.0002,
+      "loss": 0.3482,
+      "step": 1800
+    },
+    {
+      "epoch": 1.4151681000781862,
+      "grad_norm": 0.9902305006980896,
+      "learning_rate": 0.0002,
+      "loss": 0.3414,
+      "step": 1810
+    },
+    {
+      "epoch": 1.422986708365911,
+      "grad_norm": 0.8695173263549805,
+      "learning_rate": 0.0002,
+      "loss": 0.3497,
+      "step": 1820
+    },
+    {
+      "epoch": 1.4308053166536356,
+      "grad_norm": 0.8341027498245239,
+      "learning_rate": 0.0002,
+      "loss": 0.3501,
+      "step": 1830
+    },
+    {
+      "epoch": 1.4386239249413604,
+      "grad_norm": 0.6223942041397095,
+      "learning_rate": 0.0002,
+      "loss": 0.3488,
+      "step": 1840
+    },
+    {
+      "epoch": 1.4464425332290851,
+      "grad_norm": 0.8860258460044861,
+      "learning_rate": 0.0002,
+      "loss": 0.3474,
+      "step": 1850
+    },
+    {
+      "epoch": 1.45426114151681,
+      "grad_norm": 0.802268922328949,
+      "learning_rate": 0.0002,
+      "loss": 0.3408,
+      "step": 1860
+    },
+    {
+      "epoch": 1.4620797498045348,
+      "grad_norm": 0.6166049242019653,
+      "learning_rate": 0.0002,
+      "loss": 0.3453,
+      "step": 1870
+    },
+    {
+      "epoch": 1.4698983580922595,
+      "grad_norm": 0.6559504270553589,
+      "learning_rate": 0.0002,
+      "loss": 0.3351,
+      "step": 1880
+    },
+    {
+      "epoch": 1.4777169663799843,
+      "grad_norm": 0.6340335607528687,
+      "learning_rate": 0.0002,
+      "loss": 0.3423,
+      "step": 1890
+    },
+    {
+      "epoch": 1.485535574667709,
+      "grad_norm": 0.8462929129600525,
+      "learning_rate": 0.0002,
+      "loss": 0.3474,
+      "step": 1900
+    },
+    {
+      "epoch": 1.493354182955434,
+      "grad_norm": 0.8598943948745728,
+      "learning_rate": 0.0002,
+      "loss": 0.3477,
+      "step": 1910
+    },
+    {
+      "epoch": 1.5011727912431587,
+      "grad_norm": 0.8200817108154297,
+      "learning_rate": 0.0002,
+      "loss": 0.3346,
+      "step": 1920
+    },
+    {
+      "epoch": 1.5089913995308835,
+      "grad_norm": 0.6792778968811035,
+      "learning_rate": 0.0002,
+      "loss": 0.3432,
+      "step": 1930
+    },
+    {
+      "epoch": 1.5168100078186084,
+      "grad_norm": 1.1566815376281738,
+      "learning_rate": 0.0002,
+      "loss": 0.3442,
+      "step": 1940
+    },
+    {
+      "epoch": 1.524628616106333,
+      "grad_norm": 0.6438336372375488,
+      "learning_rate": 0.0002,
+      "loss": 0.3395,
+      "step": 1950
+    },
+    {
+      "epoch": 1.532447224394058,
+      "grad_norm": 0.7384976148605347,
+      "learning_rate": 0.0002,
+      "loss": 0.3343,
+      "step": 1960
+    },
+    {
+      "epoch": 1.5402658326817826,
+      "grad_norm": 0.7964138388633728,
+      "learning_rate": 0.0002,
+      "loss": 0.3361,
+      "step": 1970
+    },
+    {
+      "epoch": 1.5480844409695074,
+      "grad_norm": 0.6302239894866943,
+      "learning_rate": 0.0002,
+      "loss": 0.3455,
+      "step": 1980
+    },
+    {
+      "epoch": 1.5559030492572323,
+      "grad_norm": 1.2721625566482544,
+      "learning_rate": 0.0002,
+      "loss": 0.3546,
+      "step": 1990
+    },
+    {
+      "epoch": 1.5637216575449568,
+      "grad_norm": 0.7145891189575195,
+      "learning_rate": 0.0002,
+      "loss": 0.3357,
+      "step": 2000
+    },
+    {
+      "epoch": 1.5715402658326818,
+      "grad_norm": 1.206936240196228,
+      "learning_rate": 0.0002,
+      "loss": 0.3373,
+      "step": 2010
+    },
+    {
+      "epoch": 1.5793588741204065,
+      "grad_norm": 0.6214511394500732,
+      "learning_rate": 0.0002,
+      "loss": 0.3384,
+      "step": 2020
+    },
+    {
+      "epoch": 1.5871774824081313,
+      "grad_norm": 0.8027235269546509,
+      "learning_rate": 0.0002,
+      "loss": 0.3289,
+      "step": 2030
+    },
+    {
+      "epoch": 1.5949960906958562,
+      "grad_norm": 1.201087236404419,
+      "learning_rate": 0.0002,
+      "loss": 0.3332,
+      "step": 2040
+    },
+    {
+      "epoch": 1.602814698983581,
+      "grad_norm": 0.7836553454399109,
+      "learning_rate": 0.0002,
+      "loss": 0.3391,
+      "step": 2050
+    },
+    {
+      "epoch": 1.6106333072713057,
+      "grad_norm": 0.7517825961112976,
+      "learning_rate": 0.0002,
+      "loss": 0.3299,
+      "step": 2060
+    },
+    {
+      "epoch": 1.6184519155590305,
+      "grad_norm": 0.7465781569480896,
+      "learning_rate": 0.0002,
+      "loss": 0.3363,
+      "step": 2070
+    },
+    {
+      "epoch": 1.6262705238467552,
+      "grad_norm": 0.5759570002555847,
+      "learning_rate": 0.0002,
+      "loss": 0.3463,
+      "step": 2080
+    },
+    {
+      "epoch": 1.6340891321344801,
+      "grad_norm": 1.1590553522109985,
+      "learning_rate": 0.0002,
+      "loss": 0.3182,
+      "step": 2090
+    },
+    {
+      "epoch": 1.6419077404222049,
+      "grad_norm": 0.5870680212974548,
+      "learning_rate": 0.0002,
+      "loss": 0.3329,
+      "step": 2100
+    },
+    {
+      "epoch": 1.6497263487099296,
+      "grad_norm": 0.7370626330375671,
+      "learning_rate": 0.0002,
+      "loss": 0.3276,
+      "step": 2110
+    },
+    {
+      "epoch": 1.6575449569976546,
+      "grad_norm": 0.8450182676315308,
+      "learning_rate": 0.0002,
+      "loss": 0.3335,
+      "step": 2120
+    },
+    {
+      "epoch": 1.665363565285379,
+      "grad_norm": 0.7234358191490173,
+      "learning_rate": 0.0002,
+      "loss": 0.3282,
+      "step": 2130
+    },
+    {
+      "epoch": 1.673182173573104,
+      "grad_norm": 0.6153436303138733,
+      "learning_rate": 0.0002,
+      "loss": 0.329,
+      "step": 2140
+    },
+    {
+      "epoch": 1.6810007818608288,
+      "grad_norm": 0.5760449171066284,
+      "learning_rate": 0.0002,
+      "loss": 0.346,
+      "step": 2150
+    },
+    {
+      "epoch": 1.6888193901485535,
+      "grad_norm": 0.6206227540969849,
+      "learning_rate": 0.0002,
+      "loss": 0.3367,
+      "step": 2160
+    },
+    {
+      "epoch": 1.6966379984362785,
+      "grad_norm": 0.9404999613761902,
+      "learning_rate": 0.0002,
+      "loss": 0.3281,
+      "step": 2170
+    },
+    {
+      "epoch": 1.704456606724003,
+      "grad_norm": 0.8661916851997375,
+      "learning_rate": 0.0002,
+      "loss": 0.3217,
+      "step": 2180
+    },
+    {
+      "epoch": 1.712275215011728,
+      "grad_norm": 0.7642818093299866,
+      "learning_rate": 0.0002,
+      "loss": 0.3271,
+      "step": 2190
+    },
+    {
+      "epoch": 1.7200938232994527,
+      "grad_norm": 0.6853117942810059,
+      "learning_rate": 0.0002,
+      "loss": 0.3258,
+      "step": 2200
+    },
+    {
+      "epoch": 1.7279124315871774,
+      "grad_norm": 0.7656819820404053,
+      "learning_rate": 0.0002,
+      "loss": 0.3282,
+      "step": 2210
+    },
+    {
+      "epoch": 1.7357310398749024,
+      "grad_norm": 0.7168070077896118,
+      "learning_rate": 0.0002,
+      "loss": 0.3201,
+      "step": 2220
+    },
+    {
+      "epoch": 1.743549648162627,
+      "grad_norm": 1.0413419008255005,
+      "learning_rate": 0.0002,
+      "loss": 0.3315,
+      "step": 2230
+    },
+    {
+      "epoch": 1.7513682564503519,
+      "grad_norm": 0.5912154912948608,
+      "learning_rate": 0.0002,
+      "loss": 0.3232,
+      "step": 2240
+    },
+    {
+      "epoch": 1.7591868647380766,
+      "grad_norm": 0.7030780911445618,
+      "learning_rate": 0.0002,
+      "loss": 0.3233,
+      "step": 2250
+    },
+    {
+      "epoch": 1.7670054730258014,
+      "grad_norm": 1.0184153318405151,
+      "learning_rate": 0.0002,
+      "loss": 0.3256,
+      "step": 2260
+    },
+    {
+      "epoch": 1.7748240813135263,
+      "grad_norm": 0.7198194265365601,
+      "learning_rate": 0.0002,
+      "loss": 0.3241,
+      "step": 2270
+    },
+    {
+      "epoch": 1.7826426896012508,
+      "grad_norm": 1.065075159072876,
+      "learning_rate": 0.0002,
+      "loss": 0.3234,
+      "step": 2280
+    },
+    {
+      "epoch": 1.7904612978889758,
+      "grad_norm": 0.8209517598152161,
+      "learning_rate": 0.0002,
+      "loss": 0.3193,
+      "step": 2290
+    },
+    {
+      "epoch": 1.7982799061767005,
+      "grad_norm": 0.7711963653564453,
+      "learning_rate": 0.0002,
+      "loss": 0.3324,
+      "step": 2300
+    },
+    {
+      "epoch": 1.8060985144644253,
+      "grad_norm": 0.7440765500068665,
+      "learning_rate": 0.0002,
+      "loss": 0.319,
+      "step": 2310
+    },
+    {
+      "epoch": 1.8139171227521502,
+      "grad_norm": 0.7835466265678406,
+      "learning_rate": 0.0002,
+      "loss": 0.3163,
+      "step": 2320
+    },
+    {
+      "epoch": 1.8217357310398747,
+      "grad_norm": 0.6313241720199585,
+      "learning_rate": 0.0002,
+      "loss": 0.3239,
+      "step": 2330
+    },
+    {
+      "epoch": 1.8295543393275997,
+      "grad_norm": 0.778071403503418,
+      "learning_rate": 0.0002,
+      "loss": 0.3313,
+      "step": 2340
+    },
+    {
+      "epoch": 1.8373729476153244,
+      "grad_norm": 0.7656646966934204,
+      "learning_rate": 0.0002,
+      "loss": 0.3312,
+      "step": 2350
+    },
+    {
+      "epoch": 1.8451915559030492,
+      "grad_norm": 0.7445554733276367,
+      "learning_rate": 0.0002,
+      "loss": 0.312,
+      "step": 2360
+    },
+    {
+      "epoch": 1.8530101641907741,
+      "grad_norm": 0.6325365304946899,
+      "learning_rate": 0.0002,
+      "loss": 0.3237,
+      "step": 2370
+    },
+    {
+      "epoch": 1.8608287724784989,
+      "grad_norm": 0.8103552460670471,
+      "learning_rate": 0.0002,
+      "loss": 0.3229,
+      "step": 2380
+    },
+    {
+      "epoch": 1.8686473807662236,
+      "grad_norm": 0.9312598705291748,
+      "learning_rate": 0.0002,
+      "loss": 0.3215,
+      "step": 2390
+    },
+    {
+      "epoch": 1.8764659890539483,
+      "grad_norm": 0.848996639251709,
+      "learning_rate": 0.0002,
+      "loss": 0.3124,
+      "step": 2400
+    },
+    {
+      "epoch": 1.884284597341673,
+      "grad_norm": 0.8568035364151001,
+      "learning_rate": 0.0002,
+      "loss": 0.3268,
+      "step": 2410
+    },
+    {
+      "epoch": 1.892103205629398,
+      "grad_norm": 1.1042375564575195,
+      "learning_rate": 0.0002,
+      "loss": 0.3317,
+      "step": 2420
+    },
+    {
+      "epoch": 1.8999218139171228,
+      "grad_norm": 0.7110097408294678,
+      "learning_rate": 0.0002,
+      "loss": 0.3152,
+      "step": 2430
+    },
+    {
+      "epoch": 1.9077404222048475,
+      "grad_norm": 0.7363375425338745,
+      "learning_rate": 0.0002,
+      "loss": 0.317,
+      "step": 2440
+    },
+    {
+      "epoch": 1.9155590304925725,
+      "grad_norm": 0.7423311471939087,
+      "learning_rate": 0.0002,
+      "loss": 0.331,
+      "step": 2450
+    },
+    {
+      "epoch": 1.923377638780297,
+      "grad_norm": 0.8554385900497437,
+      "learning_rate": 0.0002,
+      "loss": 0.3138,
+      "step": 2460
+    },
+    {
+      "epoch": 1.931196247068022,
+      "grad_norm": 0.7054983377456665,
+      "learning_rate": 0.0002,
+      "loss": 0.3147,
+      "step": 2470
+    },
+    {
+      "epoch": 1.9390148553557467,
+      "grad_norm": 0.7259753942489624,
+      "learning_rate": 0.0002,
+      "loss": 0.3126,
+      "step": 2480
+    },
+    {
+      "epoch": 1.9468334636434714,
+      "grad_norm": 0.649142861366272,
+      "learning_rate": 0.0002,
+      "loss": 0.311,
+      "step": 2490
+    },
+    {
+      "epoch": 1.9546520719311964,
+      "grad_norm": 0.6006044745445251,
+      "learning_rate": 0.0002,
+      "loss": 0.3223,
+      "step": 2500
+    },
+    {
+      "epoch": 1.962470680218921,
+      "grad_norm": 0.7815561294555664,
+      "learning_rate": 0.0002,
+      "loss": 0.3105,
+      "step": 2510
+    },
+    {
+      "epoch": 1.9702892885066459,
+      "grad_norm": 0.689166247844696,
+      "learning_rate": 0.0002,
+      "loss": 0.3242,
+      "step": 2520
+    },
+    {
+      "epoch": 1.9781078967943706,
+      "grad_norm": 0.6812887787818909,
+      "learning_rate": 0.0002,
+      "loss": 0.307,
+      "step": 2530
+    },
+    {
+      "epoch": 1.9859265050820953,
+      "grad_norm": 0.6528962254524231,
+      "learning_rate": 0.0002,
+      "loss": 0.3041,
+      "step": 2540
+    },
+    {
+      "epoch": 1.9937451133698203,
+      "grad_norm": 0.6528279185295105,
+      "learning_rate": 0.0002,
+      "loss": 0.3057,
+      "step": 2550
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 0.31702762842178345,
+      "eval_runtime": 63.9344,
+      "eval_samples_per_second": 5.709,
+      "eval_steps_per_second": 0.719,
+      "step": 2558
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 10232,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.122315509170176e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-2558/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:67c127eb44653a0272979e2cc773b017f5ff5cfc6ed92250a567d1213376a07e
+size 5560

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-3837/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-3837/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-3837/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da5110b6986fc5929fc14c3771dd6175702da1783011d311c253245ff559e7f8
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-3837/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7081acbcf0ae0de163f62065eef1b99e394cb3a4912b9fe61dde70f9bf9dd611
+size 55532666

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-3837/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c2bac602a8f1359dbb4b1f8fb442e1a0dc7f199c9b15ab1747fd7878aea5cb82
+size 14244

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-3837/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2ead7494958be91625f3069f4e4ab40c4e3dc78c8f017201fa4799ac4e999e4f
+size 1064

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-3837/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-3837/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-3837/trainer_state.json

@@ -0,0 +1,2738 @@
+{
+  "best_metric": 0.2826317846775055,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-21570-sd-1/checkpoint-3837",
+  "epoch": 3.0,
+  "eval_steps": 10,
+  "global_step": 3837,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.007818608287724784,
+      "grad_norm": 1.1001313924789429,
+      "learning_rate": 0.0002,
+      "loss": 1.7403,
+      "step": 10
+    },
+    {
+      "epoch": 0.01563721657544957,
+      "grad_norm": 1.3678615093231201,
+      "learning_rate": 0.0002,
+      "loss": 1.1795,
+      "step": 20
+    },
+    {
+      "epoch": 0.023455824863174355,
+      "grad_norm": 1.2362287044525146,
+      "learning_rate": 0.0002,
+      "loss": 1.0159,
+      "step": 30
+    },
+    {
+      "epoch": 0.03127443315089914,
+      "grad_norm": 0.9754743576049805,
+      "learning_rate": 0.0002,
+      "loss": 0.9315,
+      "step": 40
+    },
+    {
+      "epoch": 0.039093041438623924,
+      "grad_norm": 1.0572363138198853,
+      "learning_rate": 0.0002,
+      "loss": 0.8015,
+      "step": 50
+    },
+    {
+      "epoch": 0.04691164972634871,
+      "grad_norm": 0.8838446140289307,
+      "learning_rate": 0.0002,
+      "loss": 0.7264,
+      "step": 60
+    },
+    {
+      "epoch": 0.054730258014073496,
+      "grad_norm": 0.831045389175415,
+      "learning_rate": 0.0002,
+      "loss": 0.6807,
+      "step": 70
+    },
+    {
+      "epoch": 0.06254886630179828,
+      "grad_norm": 0.721530556678772,
+      "learning_rate": 0.0002,
+      "loss": 0.6731,
+      "step": 80
+    },
+    {
+      "epoch": 0.07036747458952307,
+      "grad_norm": 0.6918481588363647,
+      "learning_rate": 0.0002,
+      "loss": 0.6427,
+      "step": 90
+    },
+    {
+      "epoch": 0.07818608287724785,
+      "grad_norm": 0.8207236528396606,
+      "learning_rate": 0.0002,
+      "loss": 0.6719,
+      "step": 100
+    },
+    {
+      "epoch": 0.08600469116497264,
+      "grad_norm": 0.9405701756477356,
+      "learning_rate": 0.0002,
+      "loss": 0.6533,
+      "step": 110
+    },
+    {
+      "epoch": 0.09382329945269742,
+      "grad_norm": 0.7389968037605286,
+      "learning_rate": 0.0002,
+      "loss": 0.6102,
+      "step": 120
+    },
+    {
+      "epoch": 0.1016419077404222,
+      "grad_norm": 0.7102212905883789,
+      "learning_rate": 0.0002,
+      "loss": 0.6561,
+      "step": 130
+    },
+    {
+      "epoch": 0.10946051602814699,
+      "grad_norm": 0.6546808481216431,
+      "learning_rate": 0.0002,
+      "loss": 0.6222,
+      "step": 140
+    },
+    {
+      "epoch": 0.11727912431587177,
+      "grad_norm": 1.0097354650497437,
+      "learning_rate": 0.0002,
+      "loss": 0.6369,
+      "step": 150
+    },
+    {
+      "epoch": 0.12509773260359655,
+      "grad_norm": 0.5699719786643982,
+      "learning_rate": 0.0002,
+      "loss": 0.6495,
+      "step": 160
+    },
+    {
+      "epoch": 0.13291634089132134,
+      "grad_norm": 0.6371490359306335,
+      "learning_rate": 0.0002,
+      "loss": 0.6421,
+      "step": 170
+    },
+    {
+      "epoch": 0.14073494917904614,
+      "grad_norm": 0.8385978937149048,
+      "learning_rate": 0.0002,
+      "loss": 0.6101,
+      "step": 180
+    },
+    {
+      "epoch": 0.1485535574667709,
+      "grad_norm": 0.6549069285392761,
+      "learning_rate": 0.0002,
+      "loss": 0.5957,
+      "step": 190
+    },
+    {
+      "epoch": 0.1563721657544957,
+      "grad_norm": 0.5297655463218689,
+      "learning_rate": 0.0002,
+      "loss": 0.5834,
+      "step": 200
+    },
+    {
+      "epoch": 0.1641907740422205,
+      "grad_norm": 0.6385621428489685,
+      "learning_rate": 0.0002,
+      "loss": 0.6054,
+      "step": 210
+    },
+    {
+      "epoch": 0.17200938232994528,
+      "grad_norm": 0.6723865866661072,
+      "learning_rate": 0.0002,
+      "loss": 0.6174,
+      "step": 220
+    },
+    {
+      "epoch": 0.17982799061767005,
+      "grad_norm": 0.6121484041213989,
+      "learning_rate": 0.0002,
+      "loss": 0.5951,
+      "step": 230
+    },
+    {
+      "epoch": 0.18764659890539484,
+      "grad_norm": 0.619121789932251,
+      "learning_rate": 0.0002,
+      "loss": 0.6046,
+      "step": 240
+    },
+    {
+      "epoch": 0.19546520719311963,
+      "grad_norm": 0.49208253622055054,
+      "learning_rate": 0.0002,
+      "loss": 0.5764,
+      "step": 250
+    },
+    {
+      "epoch": 0.2032838154808444,
+      "grad_norm": 0.4991285800933838,
+      "learning_rate": 0.0002,
+      "loss": 0.5679,
+      "step": 260
+    },
+    {
+      "epoch": 0.2111024237685692,
+      "grad_norm": 0.7622858285903931,
+      "learning_rate": 0.0002,
+      "loss": 0.5777,
+      "step": 270
+    },
+    {
+      "epoch": 0.21892103205629398,
+      "grad_norm": 0.5988286733627319,
+      "learning_rate": 0.0002,
+      "loss": 0.5504,
+      "step": 280
+    },
+    {
+      "epoch": 0.22673964034401878,
+      "grad_norm": 0.510055661201477,
+      "learning_rate": 0.0002,
+      "loss": 0.5801,
+      "step": 290
+    },
+    {
+      "epoch": 0.23455824863174354,
+      "grad_norm": 0.47940748929977417,
+      "learning_rate": 0.0002,
+      "loss": 0.5788,
+      "step": 300
+    },
+    {
+      "epoch": 0.24237685691946834,
+      "grad_norm": 0.5604141354560852,
+      "learning_rate": 0.0002,
+      "loss": 0.6093,
+      "step": 310
+    },
+    {
+      "epoch": 0.2501954652071931,
+      "grad_norm": 0.479342520236969,
+      "learning_rate": 0.0002,
+      "loss": 0.5641,
+      "step": 320
+    },
+    {
+      "epoch": 0.2580140734949179,
+      "grad_norm": 0.5401737093925476,
+      "learning_rate": 0.0002,
+      "loss": 0.5304,
+      "step": 330
+    },
+    {
+      "epoch": 0.2658326817826427,
+      "grad_norm": 0.5436083674430847,
+      "learning_rate": 0.0002,
+      "loss": 0.5547,
+      "step": 340
+    },
+    {
+      "epoch": 0.2736512900703675,
+      "grad_norm": 0.6402848362922668,
+      "learning_rate": 0.0002,
+      "loss": 0.5724,
+      "step": 350
+    },
+    {
+      "epoch": 0.2814698983580923,
+      "grad_norm": 0.4756305515766144,
+      "learning_rate": 0.0002,
+      "loss": 0.5452,
+      "step": 360
+    },
+    {
+      "epoch": 0.28928850664581707,
+      "grad_norm": 0.5536904335021973,
+      "learning_rate": 0.0002,
+      "loss": 0.5936,
+      "step": 370
+    },
+    {
+      "epoch": 0.2971071149335418,
+      "grad_norm": 0.6187605857849121,
+      "learning_rate": 0.0002,
+      "loss": 0.5827,
+      "step": 380
+    },
+    {
+      "epoch": 0.3049257232212666,
+      "grad_norm": 0.5297170877456665,
+      "learning_rate": 0.0002,
+      "loss": 0.5435,
+      "step": 390
+    },
+    {
+      "epoch": 0.3127443315089914,
+      "grad_norm": 0.5808210372924805,
+      "learning_rate": 0.0002,
+      "loss": 0.583,
+      "step": 400
+    },
+    {
+      "epoch": 0.3205629397967162,
+      "grad_norm": 0.7509300708770752,
+      "learning_rate": 0.0002,
+      "loss": 0.5587,
+      "step": 410
+    },
+    {
+      "epoch": 0.328381548084441,
+      "grad_norm": 0.5674371719360352,
+      "learning_rate": 0.0002,
+      "loss": 0.5439,
+      "step": 420
+    },
+    {
+      "epoch": 0.33620015637216577,
+      "grad_norm": 0.5833905339241028,
+      "learning_rate": 0.0002,
+      "loss": 0.5431,
+      "step": 430
+    },
+    {
+      "epoch": 0.34401876465989056,
+      "grad_norm": 0.537860095500946,
+      "learning_rate": 0.0002,
+      "loss": 0.5527,
+      "step": 440
+    },
+    {
+      "epoch": 0.3518373729476153,
+      "grad_norm": 0.5747054219245911,
+      "learning_rate": 0.0002,
+      "loss": 0.528,
+      "step": 450
+    },
+    {
+      "epoch": 0.3596559812353401,
+      "grad_norm": 0.6268995404243469,
+      "learning_rate": 0.0002,
+      "loss": 0.5281,
+      "step": 460
+    },
+    {
+      "epoch": 0.3674745895230649,
+      "grad_norm": 0.640737771987915,
+      "learning_rate": 0.0002,
+      "loss": 0.5394,
+      "step": 470
+    },
+    {
+      "epoch": 0.3752931978107897,
+      "grad_norm": 0.6593332290649414,
+      "learning_rate": 0.0002,
+      "loss": 0.5488,
+      "step": 480
+    },
+    {
+      "epoch": 0.3831118060985145,
+      "grad_norm": 0.5977872014045715,
+      "learning_rate": 0.0002,
+      "loss": 0.5384,
+      "step": 490
+    },
+    {
+      "epoch": 0.39093041438623927,
+      "grad_norm": 0.6118639707565308,
+      "learning_rate": 0.0002,
+      "loss": 0.5154,
+      "step": 500
+    },
+    {
+      "epoch": 0.39874902267396406,
+      "grad_norm": 1.2320106029510498,
+      "learning_rate": 0.0002,
+      "loss": 0.5481,
+      "step": 510
+    },
+    {
+      "epoch": 0.4065676309616888,
+      "grad_norm": 0.6538275480270386,
+      "learning_rate": 0.0002,
+      "loss": 0.5225,
+      "step": 520
+    },
+    {
+      "epoch": 0.4143862392494136,
+      "grad_norm": 0.5839771032333374,
+      "learning_rate": 0.0002,
+      "loss": 0.5322,
+      "step": 530
+    },
+    {
+      "epoch": 0.4222048475371384,
+      "grad_norm": 0.6228310465812683,
+      "learning_rate": 0.0002,
+      "loss": 0.5075,
+      "step": 540
+    },
+    {
+      "epoch": 0.4300234558248632,
+      "grad_norm": 0.653239905834198,
+      "learning_rate": 0.0002,
+      "loss": 0.5177,
+      "step": 550
+    },
+    {
+      "epoch": 0.43784206411258797,
+      "grad_norm": 0.6995204091072083,
+      "learning_rate": 0.0002,
+      "loss": 0.5209,
+      "step": 560
+    },
+    {
+      "epoch": 0.44566067240031276,
+      "grad_norm": 0.5197685956954956,
+      "learning_rate": 0.0002,
+      "loss": 0.4914,
+      "step": 570
+    },
+    {
+      "epoch": 0.45347928068803756,
+      "grad_norm": 0.6786061525344849,
+      "learning_rate": 0.0002,
+      "loss": 0.4951,
+      "step": 580
+    },
+    {
+      "epoch": 0.4612978889757623,
+      "grad_norm": 0.6599542498588562,
+      "learning_rate": 0.0002,
+      "loss": 0.5107,
+      "step": 590
+    },
+    {
+      "epoch": 0.4691164972634871,
+      "grad_norm": 0.5535895228385925,
+      "learning_rate": 0.0002,
+      "loss": 0.514,
+      "step": 600
+    },
+    {
+      "epoch": 0.4769351055512119,
+      "grad_norm": 0.667336642742157,
+      "learning_rate": 0.0002,
+      "loss": 0.5206,
+      "step": 610
+    },
+    {
+      "epoch": 0.4847537138389367,
+      "grad_norm": 0.5404567718505859,
+      "learning_rate": 0.0002,
+      "loss": 0.4918,
+      "step": 620
+    },
+    {
+      "epoch": 0.49257232212666147,
+      "grad_norm": 0.5316283702850342,
+      "learning_rate": 0.0002,
+      "loss": 0.501,
+      "step": 630
+    },
+    {
+      "epoch": 0.5003909304143862,
+      "grad_norm": 0.6398797035217285,
+      "learning_rate": 0.0002,
+      "loss": 0.5026,
+      "step": 640
+    },
+    {
+      "epoch": 0.508209538702111,
+      "grad_norm": 0.6197776794433594,
+      "learning_rate": 0.0002,
+      "loss": 0.5017,
+      "step": 650
+    },
+    {
+      "epoch": 0.5160281469898358,
+      "grad_norm": 0.8760672807693481,
+      "learning_rate": 0.0002,
+      "loss": 0.4905,
+      "step": 660
+    },
+    {
+      "epoch": 0.5238467552775606,
+      "grad_norm": 0.7163699269294739,
+      "learning_rate": 0.0002,
+      "loss": 0.4798,
+      "step": 670
+    },
+    {
+      "epoch": 0.5316653635652854,
+      "grad_norm": 0.7599782347679138,
+      "learning_rate": 0.0002,
+      "loss": 0.4974,
+      "step": 680
+    },
+    {
+      "epoch": 0.5394839718530101,
+      "grad_norm": 0.5682359337806702,
+      "learning_rate": 0.0002,
+      "loss": 0.4942,
+      "step": 690
+    },
+    {
+      "epoch": 0.547302580140735,
+      "grad_norm": 0.6667918562889099,
+      "learning_rate": 0.0002,
+      "loss": 0.4826,
+      "step": 700
+    },
+    {
+      "epoch": 0.5551211884284597,
+      "grad_norm": 0.9613338112831116,
+      "learning_rate": 0.0002,
+      "loss": 0.475,
+      "step": 710
+    },
+    {
+      "epoch": 0.5629397967161845,
+      "grad_norm": 0.5658668279647827,
+      "learning_rate": 0.0002,
+      "loss": 0.5005,
+      "step": 720
+    },
+    {
+      "epoch": 0.5707584050039093,
+      "grad_norm": 0.7233469486236572,
+      "learning_rate": 0.0002,
+      "loss": 0.4697,
+      "step": 730
+    },
+    {
+      "epoch": 0.5785770132916341,
+      "grad_norm": 0.5586039423942566,
+      "learning_rate": 0.0002,
+      "loss": 0.4659,
+      "step": 740
+    },
+    {
+      "epoch": 0.5863956215793589,
+      "grad_norm": 0.8161298632621765,
+      "learning_rate": 0.0002,
+      "loss": 0.4893,
+      "step": 750
+    },
+    {
+      "epoch": 0.5942142298670836,
+      "grad_norm": 0.7192191481590271,
+      "learning_rate": 0.0002,
+      "loss": 0.4678,
+      "step": 760
+    },
+    {
+      "epoch": 0.6020328381548085,
+      "grad_norm": 0.5711938738822937,
+      "learning_rate": 0.0002,
+      "loss": 0.4765,
+      "step": 770
+    },
+    {
+      "epoch": 0.6098514464425332,
+      "grad_norm": 0.5471241474151611,
+      "learning_rate": 0.0002,
+      "loss": 0.4676,
+      "step": 780
+    },
+    {
+      "epoch": 0.617670054730258,
+      "grad_norm": 0.5709220767021179,
+      "learning_rate": 0.0002,
+      "loss": 0.4628,
+      "step": 790
+    },
+    {
+      "epoch": 0.6254886630179828,
+      "grad_norm": 0.882448673248291,
+      "learning_rate": 0.0002,
+      "loss": 0.4695,
+      "step": 800
+    },
+    {
+      "epoch": 0.6333072713057076,
+      "grad_norm": 0.5136802196502686,
+      "learning_rate": 0.0002,
+      "loss": 0.4611,
+      "step": 810
+    },
+    {
+      "epoch": 0.6411258795934324,
+      "grad_norm": 0.6611698865890503,
+      "learning_rate": 0.0002,
+      "loss": 0.4663,
+      "step": 820
+    },
+    {
+      "epoch": 0.6489444878811571,
+      "grad_norm": 0.7050015926361084,
+      "learning_rate": 0.0002,
+      "loss": 0.4962,
+      "step": 830
+    },
+    {
+      "epoch": 0.656763096168882,
+      "grad_norm": 0.5757645964622498,
+      "learning_rate": 0.0002,
+      "loss": 0.4624,
+      "step": 840
+    },
+    {
+      "epoch": 0.6645817044566067,
+      "grad_norm": 0.6651985049247742,
+      "learning_rate": 0.0002,
+      "loss": 0.4695,
+      "step": 850
+    },
+    {
+      "epoch": 0.6724003127443315,
+      "grad_norm": 0.6121841669082642,
+      "learning_rate": 0.0002,
+      "loss": 0.4616,
+      "step": 860
+    },
+    {
+      "epoch": 0.6802189210320563,
+      "grad_norm": 0.9026947617530823,
+      "learning_rate": 0.0002,
+      "loss": 0.4763,
+      "step": 870
+    },
+    {
+      "epoch": 0.6880375293197811,
+      "grad_norm": 0.7725462913513184,
+      "learning_rate": 0.0002,
+      "loss": 0.4488,
+      "step": 880
+    },
+    {
+      "epoch": 0.6958561376075059,
+      "grad_norm": 0.896050214767456,
+      "learning_rate": 0.0002,
+      "loss": 0.4334,
+      "step": 890
+    },
+    {
+      "epoch": 0.7036747458952306,
+      "grad_norm": 0.757851243019104,
+      "learning_rate": 0.0002,
+      "loss": 0.4499,
+      "step": 900
+    },
+    {
+      "epoch": 0.7114933541829555,
+      "grad_norm": 0.7172074317932129,
+      "learning_rate": 0.0002,
+      "loss": 0.4466,
+      "step": 910
+    },
+    {
+      "epoch": 0.7193119624706802,
+      "grad_norm": 0.7364748120307922,
+      "learning_rate": 0.0002,
+      "loss": 0.4261,
+      "step": 920
+    },
+    {
+      "epoch": 0.727130570758405,
+      "grad_norm": 0.6359867453575134,
+      "learning_rate": 0.0002,
+      "loss": 0.4531,
+      "step": 930
+    },
+    {
+      "epoch": 0.7349491790461298,
+      "grad_norm": 0.5289077758789062,
+      "learning_rate": 0.0002,
+      "loss": 0.4464,
+      "step": 940
+    },
+    {
+      "epoch": 0.7427677873338546,
+      "grad_norm": 0.6053950786590576,
+      "learning_rate": 0.0002,
+      "loss": 0.4525,
+      "step": 950
+    },
+    {
+      "epoch": 0.7505863956215794,
+      "grad_norm": 0.8122503161430359,
+      "learning_rate": 0.0002,
+      "loss": 0.4568,
+      "step": 960
+    },
+    {
+      "epoch": 0.7584050039093041,
+      "grad_norm": 0.8779653906822205,
+      "learning_rate": 0.0002,
+      "loss": 0.4304,
+      "step": 970
+    },
+    {
+      "epoch": 0.766223612197029,
+      "grad_norm": 0.6312686204910278,
+      "learning_rate": 0.0002,
+      "loss": 0.4337,
+      "step": 980
+    },
+    {
+      "epoch": 0.7740422204847537,
+      "grad_norm": 0.7815352082252502,
+      "learning_rate": 0.0002,
+      "loss": 0.431,
+      "step": 990
+    },
+    {
+      "epoch": 0.7818608287724785,
+      "grad_norm": 0.8249784111976624,
+      "learning_rate": 0.0002,
+      "loss": 0.45,
+      "step": 1000
+    },
+    {
+      "epoch": 0.7896794370602033,
+      "grad_norm": 0.8731566667556763,
+      "learning_rate": 0.0002,
+      "loss": 0.4253,
+      "step": 1010
+    },
+    {
+      "epoch": 0.7974980453479281,
+      "grad_norm": 0.7336146831512451,
+      "learning_rate": 0.0002,
+      "loss": 0.4393,
+      "step": 1020
+    },
+    {
+      "epoch": 0.8053166536356529,
+      "grad_norm": 0.7756309509277344,
+      "learning_rate": 0.0002,
+      "loss": 0.4175,
+      "step": 1030
+    },
+    {
+      "epoch": 0.8131352619233776,
+      "grad_norm": 0.7608133554458618,
+      "learning_rate": 0.0002,
+      "loss": 0.4532,
+      "step": 1040
+    },
+    {
+      "epoch": 0.8209538702111024,
+      "grad_norm": 0.7842742800712585,
+      "learning_rate": 0.0002,
+      "loss": 0.4447,
+      "step": 1050
+    },
+    {
+      "epoch": 0.8287724784988272,
+      "grad_norm": 0.9361023902893066,
+      "learning_rate": 0.0002,
+      "loss": 0.4203,
+      "step": 1060
+    },
+    {
+      "epoch": 0.836591086786552,
+      "grad_norm": 0.8978990912437439,
+      "learning_rate": 0.0002,
+      "loss": 0.4165,
+      "step": 1070
+    },
+    {
+      "epoch": 0.8444096950742768,
+      "grad_norm": 0.7448273301124573,
+      "learning_rate": 0.0002,
+      "loss": 0.4313,
+      "step": 1080
+    },
+    {
+      "epoch": 0.8522283033620016,
+      "grad_norm": 0.9199049472808838,
+      "learning_rate": 0.0002,
+      "loss": 0.4256,
+      "step": 1090
+    },
+    {
+      "epoch": 0.8600469116497264,
+      "grad_norm": 0.7521695494651794,
+      "learning_rate": 0.0002,
+      "loss": 0.4343,
+      "step": 1100
+    },
+    {
+      "epoch": 0.8678655199374511,
+      "grad_norm": 0.7470024228096008,
+      "learning_rate": 0.0002,
+      "loss": 0.4136,
+      "step": 1110
+    },
+    {
+      "epoch": 0.8756841282251759,
+      "grad_norm": 0.5728107690811157,
+      "learning_rate": 0.0002,
+      "loss": 0.4227,
+      "step": 1120
+    },
+    {
+      "epoch": 0.8835027365129007,
+      "grad_norm": 0.8137171268463135,
+      "learning_rate": 0.0002,
+      "loss": 0.4084,
+      "step": 1130
+    },
+    {
+      "epoch": 0.8913213448006255,
+      "grad_norm": 0.7411524057388306,
+      "learning_rate": 0.0002,
+      "loss": 0.4171,
+      "step": 1140
+    },
+    {
+      "epoch": 0.8991399530883503,
+      "grad_norm": 0.705020546913147,
+      "learning_rate": 0.0002,
+      "loss": 0.4061,
+      "step": 1150
+    },
+    {
+      "epoch": 0.9069585613760751,
+      "grad_norm": 0.6366162300109863,
+      "learning_rate": 0.0002,
+      "loss": 0.4196,
+      "step": 1160
+    },
+    {
+      "epoch": 0.9147771696637998,
+      "grad_norm": 0.7566165924072266,
+      "learning_rate": 0.0002,
+      "loss": 0.45,
+      "step": 1170
+    },
+    {
+      "epoch": 0.9225957779515246,
+      "grad_norm": 0.9905046224594116,
+      "learning_rate": 0.0002,
+      "loss": 0.4195,
+      "step": 1180
+    },
+    {
+      "epoch": 0.9304143862392494,
+      "grad_norm": 0.6872445940971375,
+      "learning_rate": 0.0002,
+      "loss": 0.4212,
+      "step": 1190
+    },
+    {
+      "epoch": 0.9382329945269742,
+      "grad_norm": 0.6640546917915344,
+      "learning_rate": 0.0002,
+      "loss": 0.4151,
+      "step": 1200
+    },
+    {
+      "epoch": 0.946051602814699,
+      "grad_norm": 1.0592104196548462,
+      "learning_rate": 0.0002,
+      "loss": 0.4018,
+      "step": 1210
+    },
+    {
+      "epoch": 0.9538702111024238,
+      "grad_norm": 0.9068714380264282,
+      "learning_rate": 0.0002,
+      "loss": 0.3868,
+      "step": 1220
+    },
+    {
+      "epoch": 0.9616888193901486,
+      "grad_norm": 0.7440975308418274,
+      "learning_rate": 0.0002,
+      "loss": 0.3777,
+      "step": 1230
+    },
+    {
+      "epoch": 0.9695074276778733,
+      "grad_norm": 0.9631947875022888,
+      "learning_rate": 0.0002,
+      "loss": 0.4086,
+      "step": 1240
+    },
+    {
+      "epoch": 0.9773260359655981,
+      "grad_norm": 0.708501935005188,
+      "learning_rate": 0.0002,
+      "loss": 0.4042,
+      "step": 1250
+    },
+    {
+      "epoch": 0.9851446442533229,
+      "grad_norm": 0.664806604385376,
+      "learning_rate": 0.0002,
+      "loss": 0.3958,
+      "step": 1260
+    },
+    {
+      "epoch": 0.9929632525410477,
+      "grad_norm": 0.6895506978034973,
+      "learning_rate": 0.0002,
+      "loss": 0.3944,
+      "step": 1270
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 0.39914223551750183,
+      "eval_runtime": 92.7363,
+      "eval_samples_per_second": 3.936,
+      "eval_steps_per_second": 0.496,
+      "step": 1279
+    },
+    {
+      "epoch": 1.0007818608287724,
+      "grad_norm": 0.6349056363105774,
+      "learning_rate": 0.0002,
+      "loss": 0.3917,
+      "step": 1280
+    },
+    {
+      "epoch": 1.0086004691164974,
+      "grad_norm": 0.9110808968544006,
+      "learning_rate": 0.0002,
+      "loss": 0.3851,
+      "step": 1290
+    },
+    {
+      "epoch": 1.016419077404222,
+      "grad_norm": 0.8718474507331848,
+      "learning_rate": 0.0002,
+      "loss": 0.3743,
+      "step": 1300
+    },
+    {
+      "epoch": 1.0242376856919468,
+      "grad_norm": 0.9496098756790161,
+      "learning_rate": 0.0002,
+      "loss": 0.3735,
+      "step": 1310
+    },
+    {
+      "epoch": 1.0320562939796716,
+      "grad_norm": 0.5553750991821289,
+      "learning_rate": 0.0002,
+      "loss": 0.3688,
+      "step": 1320
+    },
+    {
+      "epoch": 1.0398749022673963,
+      "grad_norm": 0.8498914241790771,
+      "learning_rate": 0.0002,
+      "loss": 0.3701,
+      "step": 1330
+    },
+    {
+      "epoch": 1.0476935105551213,
+      "grad_norm": 0.6435985565185547,
+      "learning_rate": 0.0002,
+      "loss": 0.3739,
+      "step": 1340
+    },
+    {
+      "epoch": 1.055512118842846,
+      "grad_norm": 0.8342816233634949,
+      "learning_rate": 0.0002,
+      "loss": 0.3778,
+      "step": 1350
+    },
+    {
+      "epoch": 1.0633307271305708,
+      "grad_norm": 0.6142820715904236,
+      "learning_rate": 0.0002,
+      "loss": 0.3873,
+      "step": 1360
+    },
+    {
+      "epoch": 1.0711493354182955,
+      "grad_norm": 0.9387786984443665,
+      "learning_rate": 0.0002,
+      "loss": 0.37,
+      "step": 1370
+    },
+    {
+      "epoch": 1.0789679437060202,
+      "grad_norm": 0.8187823295593262,
+      "learning_rate": 0.0002,
+      "loss": 0.3568,
+      "step": 1380
+    },
+    {
+      "epoch": 1.0867865519937452,
+      "grad_norm": 0.7127028107643127,
+      "learning_rate": 0.0002,
+      "loss": 0.3852,
+      "step": 1390
+    },
+    {
+      "epoch": 1.09460516028147,
+      "grad_norm": 0.7990315556526184,
+      "learning_rate": 0.0002,
+      "loss": 0.3732,
+      "step": 1400
+    },
+    {
+      "epoch": 1.1024237685691947,
+      "grad_norm": 1.0349947214126587,
+      "learning_rate": 0.0002,
+      "loss": 0.3764,
+      "step": 1410
+    },
+    {
+      "epoch": 1.1102423768569194,
+      "grad_norm": 0.5400282740592957,
+      "learning_rate": 0.0002,
+      "loss": 0.3518,
+      "step": 1420
+    },
+    {
+      "epoch": 1.1180609851446444,
+      "grad_norm": 0.9225337505340576,
+      "learning_rate": 0.0002,
+      "loss": 0.3661,
+      "step": 1430
+    },
+    {
+      "epoch": 1.125879593432369,
+      "grad_norm": 0.7267957925796509,
+      "learning_rate": 0.0002,
+      "loss": 0.3661,
+      "step": 1440
+    },
+    {
+      "epoch": 1.1336982017200938,
+      "grad_norm": 0.6454635858535767,
+      "learning_rate": 0.0002,
+      "loss": 0.3745,
+      "step": 1450
+    },
+    {
+      "epoch": 1.1415168100078186,
+      "grad_norm": 1.0288119316101074,
+      "learning_rate": 0.0002,
+      "loss": 0.3675,
+      "step": 1460
+    },
+    {
+      "epoch": 1.1493354182955433,
+      "grad_norm": 0.6535518169403076,
+      "learning_rate": 0.0002,
+      "loss": 0.3807,
+      "step": 1470
+    },
+    {
+      "epoch": 1.1571540265832683,
+      "grad_norm": 0.6860265731811523,
+      "learning_rate": 0.0002,
+      "loss": 0.3664,
+      "step": 1480
+    },
+    {
+      "epoch": 1.164972634870993,
+      "grad_norm": 0.9102330803871155,
+      "learning_rate": 0.0002,
+      "loss": 0.3679,
+      "step": 1490
+    },
+    {
+      "epoch": 1.1727912431587177,
+      "grad_norm": 0.6989532709121704,
+      "learning_rate": 0.0002,
+      "loss": 0.364,
+      "step": 1500
+    },
+    {
+      "epoch": 1.1806098514464425,
+      "grad_norm": 1.1313148736953735,
+      "learning_rate": 0.0002,
+      "loss": 0.3592,
+      "step": 1510
+    },
+    {
+      "epoch": 1.1884284597341672,
+      "grad_norm": 0.6841519474983215,
+      "learning_rate": 0.0002,
+      "loss": 0.363,
+      "step": 1520
+    },
+    {
+      "epoch": 1.1962470680218922,
+      "grad_norm": 0.7030880451202393,
+      "learning_rate": 0.0002,
+      "loss": 0.3736,
+      "step": 1530
+    },
+    {
+      "epoch": 1.204065676309617,
+      "grad_norm": 0.6326259970664978,
+      "learning_rate": 0.0002,
+      "loss": 0.3665,
+      "step": 1540
+    },
+    {
+      "epoch": 1.2118842845973417,
+      "grad_norm": 0.8820798993110657,
+      "learning_rate": 0.0002,
+      "loss": 0.3734,
+      "step": 1550
+    },
+    {
+      "epoch": 1.2197028928850664,
+      "grad_norm": 0.8624477386474609,
+      "learning_rate": 0.0002,
+      "loss": 0.3581,
+      "step": 1560
+    },
+    {
+      "epoch": 1.2275215011727911,
+      "grad_norm": 0.6675921678543091,
+      "learning_rate": 0.0002,
+      "loss": 0.3722,
+      "step": 1570
+    },
+    {
+      "epoch": 1.235340109460516,
+      "grad_norm": 1.0099470615386963,
+      "learning_rate": 0.0002,
+      "loss": 0.3661,
+      "step": 1580
+    },
+    {
+      "epoch": 1.2431587177482408,
+      "grad_norm": 0.8204535841941833,
+      "learning_rate": 0.0002,
+      "loss": 0.3674,
+      "step": 1590
+    },
+    {
+      "epoch": 1.2509773260359656,
+      "grad_norm": 0.7338495850563049,
+      "learning_rate": 0.0002,
+      "loss": 0.3655,
+      "step": 1600
+    },
+    {
+      "epoch": 1.2587959343236903,
+      "grad_norm": 0.7446017861366272,
+      "learning_rate": 0.0002,
+      "loss": 0.3706,
+      "step": 1610
+    },
+    {
+      "epoch": 1.266614542611415,
+      "grad_norm": 0.7122478485107422,
+      "learning_rate": 0.0002,
+      "loss": 0.3487,
+      "step": 1620
+    },
+    {
+      "epoch": 1.27443315089914,
+      "grad_norm": 0.8905506730079651,
+      "learning_rate": 0.0002,
+      "loss": 0.3749,
+      "step": 1630
+    },
+    {
+      "epoch": 1.2822517591868647,
+      "grad_norm": 0.8287106156349182,
+      "learning_rate": 0.0002,
+      "loss": 0.3465,
+      "step": 1640
+    },
+    {
+      "epoch": 1.2900703674745895,
+      "grad_norm": 0.6574750542640686,
+      "learning_rate": 0.0002,
+      "loss": 0.341,
+      "step": 1650
+    },
+    {
+      "epoch": 1.2978889757623144,
+      "grad_norm": 0.6535889506340027,
+      "learning_rate": 0.0002,
+      "loss": 0.3467,
+      "step": 1660
+    },
+    {
+      "epoch": 1.3057075840500392,
+      "grad_norm": 0.7493264675140381,
+      "learning_rate": 0.0002,
+      "loss": 0.3632,
+      "step": 1670
+    },
+    {
+      "epoch": 1.313526192337764,
+      "grad_norm": 0.8663034439086914,
+      "learning_rate": 0.0002,
+      "loss": 0.3607,
+      "step": 1680
+    },
+    {
+      "epoch": 1.3213448006254886,
+      "grad_norm": 0.7360671758651733,
+      "learning_rate": 0.0002,
+      "loss": 0.3605,
+      "step": 1690
+    },
+    {
+      "epoch": 1.3291634089132134,
+      "grad_norm": 0.7367114424705505,
+      "learning_rate": 0.0002,
+      "loss": 0.3674,
+      "step": 1700
+    },
+    {
+      "epoch": 1.3369820172009383,
+      "grad_norm": 0.8030956983566284,
+      "learning_rate": 0.0002,
+      "loss": 0.3593,
+      "step": 1710
+    },
+    {
+      "epoch": 1.344800625488663,
+      "grad_norm": 0.9848132133483887,
+      "learning_rate": 0.0002,
+      "loss": 0.3536,
+      "step": 1720
+    },
+    {
+      "epoch": 1.3526192337763878,
+      "grad_norm": 0.8279334306716919,
+      "learning_rate": 0.0002,
+      "loss": 0.3639,
+      "step": 1730
+    },
+    {
+      "epoch": 1.3604378420641126,
+      "grad_norm": 0.5222904682159424,
+      "learning_rate": 0.0002,
+      "loss": 0.3508,
+      "step": 1740
+    },
+    {
+      "epoch": 1.3682564503518373,
+      "grad_norm": 0.794312596321106,
+      "learning_rate": 0.0002,
+      "loss": 0.3534,
+      "step": 1750
+    },
+    {
+      "epoch": 1.3760750586395623,
+      "grad_norm": 0.737553060054779,
+      "learning_rate": 0.0002,
+      "loss": 0.3468,
+      "step": 1760
+    },
+    {
+      "epoch": 1.383893666927287,
+      "grad_norm": 0.6765537858009338,
+      "learning_rate": 0.0002,
+      "loss": 0.3446,
+      "step": 1770
+    },
+    {
+      "epoch": 1.3917122752150117,
+      "grad_norm": 0.8873873353004456,
+      "learning_rate": 0.0002,
+      "loss": 0.3328,
+      "step": 1780
+    },
+    {
+      "epoch": 1.3995308835027365,
+      "grad_norm": 0.8087615966796875,
+      "learning_rate": 0.0002,
+      "loss": 0.334,
+      "step": 1790
+    },
+    {
+      "epoch": 1.4073494917904612,
+      "grad_norm": 0.7812146544456482,
+      "learning_rate": 0.0002,
+      "loss": 0.3482,
+      "step": 1800
+    },
+    {
+      "epoch": 1.4151681000781862,
+      "grad_norm": 0.9902305006980896,
+      "learning_rate": 0.0002,
+      "loss": 0.3414,
+      "step": 1810
+    },
+    {
+      "epoch": 1.422986708365911,
+      "grad_norm": 0.8695173263549805,
+      "learning_rate": 0.0002,
+      "loss": 0.3497,
+      "step": 1820
+    },
+    {
+      "epoch": 1.4308053166536356,
+      "grad_norm": 0.8341027498245239,
+      "learning_rate": 0.0002,
+      "loss": 0.3501,
+      "step": 1830
+    },
+    {
+      "epoch": 1.4386239249413604,
+      "grad_norm": 0.6223942041397095,
+      "learning_rate": 0.0002,
+      "loss": 0.3488,
+      "step": 1840
+    },
+    {
+      "epoch": 1.4464425332290851,
+      "grad_norm": 0.8860258460044861,
+      "learning_rate": 0.0002,
+      "loss": 0.3474,
+      "step": 1850
+    },
+    {
+      "epoch": 1.45426114151681,
+      "grad_norm": 0.802268922328949,
+      "learning_rate": 0.0002,
+      "loss": 0.3408,
+      "step": 1860
+    },
+    {
+      "epoch": 1.4620797498045348,
+      "grad_norm": 0.6166049242019653,
+      "learning_rate": 0.0002,
+      "loss": 0.3453,
+      "step": 1870
+    },
+    {
+      "epoch": 1.4698983580922595,
+      "grad_norm": 0.6559504270553589,
+      "learning_rate": 0.0002,
+      "loss": 0.3351,
+      "step": 1880
+    },
+    {
+      "epoch": 1.4777169663799843,
+      "grad_norm": 0.6340335607528687,
+      "learning_rate": 0.0002,
+      "loss": 0.3423,
+      "step": 1890
+    },
+    {
+      "epoch": 1.485535574667709,
+      "grad_norm": 0.8462929129600525,
+      "learning_rate": 0.0002,
+      "loss": 0.3474,
+      "step": 1900
+    },
+    {
+      "epoch": 1.493354182955434,
+      "grad_norm": 0.8598943948745728,
+      "learning_rate": 0.0002,
+      "loss": 0.3477,
+      "step": 1910
+    },
+    {
+      "epoch": 1.5011727912431587,
+      "grad_norm": 0.8200817108154297,
+      "learning_rate": 0.0002,
+      "loss": 0.3346,
+      "step": 1920
+    },
+    {
+      "epoch": 1.5089913995308835,
+      "grad_norm": 0.6792778968811035,
+      "learning_rate": 0.0002,
+      "loss": 0.3432,
+      "step": 1930
+    },
+    {
+      "epoch": 1.5168100078186084,
+      "grad_norm": 1.1566815376281738,
+      "learning_rate": 0.0002,
+      "loss": 0.3442,
+      "step": 1940
+    },
+    {
+      "epoch": 1.524628616106333,
+      "grad_norm": 0.6438336372375488,
+      "learning_rate": 0.0002,
+      "loss": 0.3395,
+      "step": 1950
+    },
+    {
+      "epoch": 1.532447224394058,
+      "grad_norm": 0.7384976148605347,
+      "learning_rate": 0.0002,
+      "loss": 0.3343,
+      "step": 1960
+    },
+    {
+      "epoch": 1.5402658326817826,
+      "grad_norm": 0.7964138388633728,
+      "learning_rate": 0.0002,
+      "loss": 0.3361,
+      "step": 1970
+    },
+    {
+      "epoch": 1.5480844409695074,
+      "grad_norm": 0.6302239894866943,
+      "learning_rate": 0.0002,
+      "loss": 0.3455,
+      "step": 1980
+    },
+    {
+      "epoch": 1.5559030492572323,
+      "grad_norm": 1.2721625566482544,
+      "learning_rate": 0.0002,
+      "loss": 0.3546,
+      "step": 1990
+    },
+    {
+      "epoch": 1.5637216575449568,
+      "grad_norm": 0.7145891189575195,
+      "learning_rate": 0.0002,
+      "loss": 0.3357,
+      "step": 2000
+    },
+    {
+      "epoch": 1.5715402658326818,
+      "grad_norm": 1.206936240196228,
+      "learning_rate": 0.0002,
+      "loss": 0.3373,
+      "step": 2010
+    },
+    {
+      "epoch": 1.5793588741204065,
+      "grad_norm": 0.6214511394500732,
+      "learning_rate": 0.0002,
+      "loss": 0.3384,
+      "step": 2020
+    },
+    {
+      "epoch": 1.5871774824081313,
+      "grad_norm": 0.8027235269546509,
+      "learning_rate": 0.0002,
+      "loss": 0.3289,
+      "step": 2030
+    },
+    {
+      "epoch": 1.5949960906958562,
+      "grad_norm": 1.201087236404419,
+      "learning_rate": 0.0002,
+      "loss": 0.3332,
+      "step": 2040
+    },
+    {
+      "epoch": 1.602814698983581,
+      "grad_norm": 0.7836553454399109,
+      "learning_rate": 0.0002,
+      "loss": 0.3391,
+      "step": 2050
+    },
+    {
+      "epoch": 1.6106333072713057,
+      "grad_norm": 0.7517825961112976,
+      "learning_rate": 0.0002,
+      "loss": 0.3299,
+      "step": 2060
+    },
+    {
+      "epoch": 1.6184519155590305,
+      "grad_norm": 0.7465781569480896,
+      "learning_rate": 0.0002,
+      "loss": 0.3363,
+      "step": 2070
+    },
+    {
+      "epoch": 1.6262705238467552,
+      "grad_norm": 0.5759570002555847,
+      "learning_rate": 0.0002,
+      "loss": 0.3463,
+      "step": 2080
+    },
+    {
+      "epoch": 1.6340891321344801,
+      "grad_norm": 1.1590553522109985,
+      "learning_rate": 0.0002,
+      "loss": 0.3182,
+      "step": 2090
+    },
+    {
+      "epoch": 1.6419077404222049,
+      "grad_norm": 0.5870680212974548,
+      "learning_rate": 0.0002,
+      "loss": 0.3329,
+      "step": 2100
+    },
+    {
+      "epoch": 1.6497263487099296,
+      "grad_norm": 0.7370626330375671,
+      "learning_rate": 0.0002,
+      "loss": 0.3276,
+      "step": 2110
+    },
+    {
+      "epoch": 1.6575449569976546,
+      "grad_norm": 0.8450182676315308,
+      "learning_rate": 0.0002,
+      "loss": 0.3335,
+      "step": 2120
+    },
+    {
+      "epoch": 1.665363565285379,
+      "grad_norm": 0.7234358191490173,
+      "learning_rate": 0.0002,
+      "loss": 0.3282,
+      "step": 2130
+    },
+    {
+      "epoch": 1.673182173573104,
+      "grad_norm": 0.6153436303138733,
+      "learning_rate": 0.0002,
+      "loss": 0.329,
+      "step": 2140
+    },
+    {
+      "epoch": 1.6810007818608288,
+      "grad_norm": 0.5760449171066284,
+      "learning_rate": 0.0002,
+      "loss": 0.346,
+      "step": 2150
+    },
+    {
+      "epoch": 1.6888193901485535,
+      "grad_norm": 0.6206227540969849,
+      "learning_rate": 0.0002,
+      "loss": 0.3367,
+      "step": 2160
+    },
+    {
+      "epoch": 1.6966379984362785,
+      "grad_norm": 0.9404999613761902,
+      "learning_rate": 0.0002,
+      "loss": 0.3281,
+      "step": 2170
+    },
+    {
+      "epoch": 1.704456606724003,
+      "grad_norm": 0.8661916851997375,
+      "learning_rate": 0.0002,
+      "loss": 0.3217,
+      "step": 2180
+    },
+    {
+      "epoch": 1.712275215011728,
+      "grad_norm": 0.7642818093299866,
+      "learning_rate": 0.0002,
+      "loss": 0.3271,
+      "step": 2190
+    },
+    {
+      "epoch": 1.7200938232994527,
+      "grad_norm": 0.6853117942810059,
+      "learning_rate": 0.0002,
+      "loss": 0.3258,
+      "step": 2200
+    },
+    {
+      "epoch": 1.7279124315871774,
+      "grad_norm": 0.7656819820404053,
+      "learning_rate": 0.0002,
+      "loss": 0.3282,
+      "step": 2210
+    },
+    {
+      "epoch": 1.7357310398749024,
+      "grad_norm": 0.7168070077896118,
+      "learning_rate": 0.0002,
+      "loss": 0.3201,
+      "step": 2220
+    },
+    {
+      "epoch": 1.743549648162627,
+      "grad_norm": 1.0413419008255005,
+      "learning_rate": 0.0002,
+      "loss": 0.3315,
+      "step": 2230
+    },
+    {
+      "epoch": 1.7513682564503519,
+      "grad_norm": 0.5912154912948608,
+      "learning_rate": 0.0002,
+      "loss": 0.3232,
+      "step": 2240
+    },
+    {
+      "epoch": 1.7591868647380766,
+      "grad_norm": 0.7030780911445618,
+      "learning_rate": 0.0002,
+      "loss": 0.3233,
+      "step": 2250
+    },
+    {
+      "epoch": 1.7670054730258014,
+      "grad_norm": 1.0184153318405151,
+      "learning_rate": 0.0002,
+      "loss": 0.3256,
+      "step": 2260
+    },
+    {
+      "epoch": 1.7748240813135263,
+      "grad_norm": 0.7198194265365601,
+      "learning_rate": 0.0002,
+      "loss": 0.3241,
+      "step": 2270
+    },
+    {
+      "epoch": 1.7826426896012508,
+      "grad_norm": 1.065075159072876,
+      "learning_rate": 0.0002,
+      "loss": 0.3234,
+      "step": 2280
+    },
+    {
+      "epoch": 1.7904612978889758,
+      "grad_norm": 0.8209517598152161,
+      "learning_rate": 0.0002,
+      "loss": 0.3193,
+      "step": 2290
+    },
+    {
+      "epoch": 1.7982799061767005,
+      "grad_norm": 0.7711963653564453,
+      "learning_rate": 0.0002,
+      "loss": 0.3324,
+      "step": 2300
+    },
+    {
+      "epoch": 1.8060985144644253,
+      "grad_norm": 0.7440765500068665,
+      "learning_rate": 0.0002,
+      "loss": 0.319,
+      "step": 2310
+    },
+    {
+      "epoch": 1.8139171227521502,
+      "grad_norm": 0.7835466265678406,
+      "learning_rate": 0.0002,
+      "loss": 0.3163,
+      "step": 2320
+    },
+    {
+      "epoch": 1.8217357310398747,
+      "grad_norm": 0.6313241720199585,
+      "learning_rate": 0.0002,
+      "loss": 0.3239,
+      "step": 2330
+    },
+    {
+      "epoch": 1.8295543393275997,
+      "grad_norm": 0.778071403503418,
+      "learning_rate": 0.0002,
+      "loss": 0.3313,
+      "step": 2340
+    },
+    {
+      "epoch": 1.8373729476153244,
+      "grad_norm": 0.7656646966934204,
+      "learning_rate": 0.0002,
+      "loss": 0.3312,
+      "step": 2350
+    },
+    {
+      "epoch": 1.8451915559030492,
+      "grad_norm": 0.7445554733276367,
+      "learning_rate": 0.0002,
+      "loss": 0.312,
+      "step": 2360
+    },
+    {
+      "epoch": 1.8530101641907741,
+      "grad_norm": 0.6325365304946899,
+      "learning_rate": 0.0002,
+      "loss": 0.3237,
+      "step": 2370
+    },
+    {
+      "epoch": 1.8608287724784989,
+      "grad_norm": 0.8103552460670471,
+      "learning_rate": 0.0002,
+      "loss": 0.3229,
+      "step": 2380
+    },
+    {
+      "epoch": 1.8686473807662236,
+      "grad_norm": 0.9312598705291748,
+      "learning_rate": 0.0002,
+      "loss": 0.3215,
+      "step": 2390
+    },
+    {
+      "epoch": 1.8764659890539483,
+      "grad_norm": 0.848996639251709,
+      "learning_rate": 0.0002,
+      "loss": 0.3124,
+      "step": 2400
+    },
+    {
+      "epoch": 1.884284597341673,
+      "grad_norm": 0.8568035364151001,
+      "learning_rate": 0.0002,
+      "loss": 0.3268,
+      "step": 2410
+    },
+    {
+      "epoch": 1.892103205629398,
+      "grad_norm": 1.1042375564575195,
+      "learning_rate": 0.0002,
+      "loss": 0.3317,
+      "step": 2420
+    },
+    {
+      "epoch": 1.8999218139171228,
+      "grad_norm": 0.7110097408294678,
+      "learning_rate": 0.0002,
+      "loss": 0.3152,
+      "step": 2430
+    },
+    {
+      "epoch": 1.9077404222048475,
+      "grad_norm": 0.7363375425338745,
+      "learning_rate": 0.0002,
+      "loss": 0.317,
+      "step": 2440
+    },
+    {
+      "epoch": 1.9155590304925725,
+      "grad_norm": 0.7423311471939087,
+      "learning_rate": 0.0002,
+      "loss": 0.331,
+      "step": 2450
+    },
+    {
+      "epoch": 1.923377638780297,
+      "grad_norm": 0.8554385900497437,
+      "learning_rate": 0.0002,
+      "loss": 0.3138,
+      "step": 2460
+    },
+    {
+      "epoch": 1.931196247068022,
+      "grad_norm": 0.7054983377456665,
+      "learning_rate": 0.0002,
+      "loss": 0.3147,
+      "step": 2470
+    },
+    {
+      "epoch": 1.9390148553557467,
+      "grad_norm": 0.7259753942489624,
+      "learning_rate": 0.0002,
+      "loss": 0.3126,
+      "step": 2480
+    },
+    {
+      "epoch": 1.9468334636434714,
+      "grad_norm": 0.649142861366272,
+      "learning_rate": 0.0002,
+      "loss": 0.311,
+      "step": 2490
+    },
+    {
+      "epoch": 1.9546520719311964,
+      "grad_norm": 0.6006044745445251,
+      "learning_rate": 0.0002,
+      "loss": 0.3223,
+      "step": 2500
+    },
+    {
+      "epoch": 1.962470680218921,
+      "grad_norm": 0.7815561294555664,
+      "learning_rate": 0.0002,
+      "loss": 0.3105,
+      "step": 2510
+    },
+    {
+      "epoch": 1.9702892885066459,
+      "grad_norm": 0.689166247844696,
+      "learning_rate": 0.0002,
+      "loss": 0.3242,
+      "step": 2520
+    },
+    {
+      "epoch": 1.9781078967943706,
+      "grad_norm": 0.6812887787818909,
+      "learning_rate": 0.0002,
+      "loss": 0.307,
+      "step": 2530
+    },
+    {
+      "epoch": 1.9859265050820953,
+      "grad_norm": 0.6528962254524231,
+      "learning_rate": 0.0002,
+      "loss": 0.3041,
+      "step": 2540
+    },
+    {
+      "epoch": 1.9937451133698203,
+      "grad_norm": 0.6528279185295105,
+      "learning_rate": 0.0002,
+      "loss": 0.3057,
+      "step": 2550
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 0.31702762842178345,
+      "eval_runtime": 63.9344,
+      "eval_samples_per_second": 5.709,
+      "eval_steps_per_second": 0.719,
+      "step": 2558
+    },
+    {
+      "epoch": 2.001563721657545,
+      "grad_norm": 0.6745762228965759,
+      "learning_rate": 0.0002,
+      "loss": 0.3096,
+      "step": 2560
+    },
+    {
+      "epoch": 2.0093823299452698,
+      "grad_norm": 0.8624046444892883,
+      "learning_rate": 0.0002,
+      "loss": 0.2931,
+      "step": 2570
+    },
+    {
+      "epoch": 2.0172009382329947,
+      "grad_norm": 0.9315873980522156,
+      "learning_rate": 0.0002,
+      "loss": 0.2959,
+      "step": 2580
+    },
+    {
+      "epoch": 2.0250195465207192,
+      "grad_norm": 0.7221729159355164,
+      "learning_rate": 0.0002,
+      "loss": 0.304,
+      "step": 2590
+    },
+    {
+      "epoch": 2.032838154808444,
+      "grad_norm": 0.821427583694458,
+      "learning_rate": 0.0002,
+      "loss": 0.2958,
+      "step": 2600
+    },
+    {
+      "epoch": 2.0406567630961687,
+      "grad_norm": 0.6174030900001526,
+      "learning_rate": 0.0002,
+      "loss": 0.3086,
+      "step": 2610
+    },
+    {
+      "epoch": 2.0484753713838937,
+      "grad_norm": 0.670123815536499,
+      "learning_rate": 0.0002,
+      "loss": 0.3029,
+      "step": 2620
+    },
+    {
+      "epoch": 2.0562939796716186,
+      "grad_norm": 1.0470527410507202,
+      "learning_rate": 0.0002,
+      "loss": 0.303,
+      "step": 2630
+    },
+    {
+      "epoch": 2.064112587959343,
+      "grad_norm": 0.9624953866004944,
+      "learning_rate": 0.0002,
+      "loss": 0.305,
+      "step": 2640
+    },
+    {
+      "epoch": 2.071931196247068,
+      "grad_norm": 0.9372404217720032,
+      "learning_rate": 0.0002,
+      "loss": 0.2886,
+      "step": 2650
+    },
+    {
+      "epoch": 2.0797498045347926,
+      "grad_norm": 0.5635219812393188,
+      "learning_rate": 0.0002,
+      "loss": 0.2883,
+      "step": 2660
+    },
+    {
+      "epoch": 2.0875684128225176,
+      "grad_norm": 0.7124640941619873,
+      "learning_rate": 0.0002,
+      "loss": 0.2889,
+      "step": 2670
+    },
+    {
+      "epoch": 2.0953870211102426,
+      "grad_norm": 0.9465657472610474,
+      "learning_rate": 0.0002,
+      "loss": 0.3011,
+      "step": 2680
+    },
+    {
+      "epoch": 2.103205629397967,
+      "grad_norm": 0.7515174150466919,
+      "learning_rate": 0.0002,
+      "loss": 0.3061,
+      "step": 2690
+    },
+    {
+      "epoch": 2.111024237685692,
+      "grad_norm": 0.9722101092338562,
+      "learning_rate": 0.0002,
+      "loss": 0.2879,
+      "step": 2700
+    },
+    {
+      "epoch": 2.1188428459734165,
+      "grad_norm": 0.6212092041969299,
+      "learning_rate": 0.0002,
+      "loss": 0.3042,
+      "step": 2710
+    },
+    {
+      "epoch": 2.1266614542611415,
+      "grad_norm": 0.8257124423980713,
+      "learning_rate": 0.0002,
+      "loss": 0.2913,
+      "step": 2720
+    },
+    {
+      "epoch": 2.1344800625488665,
+      "grad_norm": 0.555681586265564,
+      "learning_rate": 0.0002,
+      "loss": 0.303,
+      "step": 2730
+    },
+    {
+      "epoch": 2.142298670836591,
+      "grad_norm": 0.5878972411155701,
+      "learning_rate": 0.0002,
+      "loss": 0.2933,
+      "step": 2740
+    },
+    {
+      "epoch": 2.150117279124316,
+      "grad_norm": 0.9192131161689758,
+      "learning_rate": 0.0002,
+      "loss": 0.3048,
+      "step": 2750
+    },
+    {
+      "epoch": 2.1579358874120405,
+      "grad_norm": 0.4973098039627075,
+      "learning_rate": 0.0002,
+      "loss": 0.3003,
+      "step": 2760
+    },
+    {
+      "epoch": 2.1657544956997654,
+      "grad_norm": 1.0267789363861084,
+      "learning_rate": 0.0002,
+      "loss": 0.307,
+      "step": 2770
+    },
+    {
+      "epoch": 2.1735731039874904,
+      "grad_norm": 0.6689954996109009,
+      "learning_rate": 0.0002,
+      "loss": 0.288,
+      "step": 2780
+    },
+    {
+      "epoch": 2.181391712275215,
+      "grad_norm": 1.0326844453811646,
+      "learning_rate": 0.0002,
+      "loss": 0.3012,
+      "step": 2790
+    },
+    {
+      "epoch": 2.18921032056294,
+      "grad_norm": 0.7656895518302917,
+      "learning_rate": 0.0002,
+      "loss": 0.3111,
+      "step": 2800
+    },
+    {
+      "epoch": 2.1970289288506644,
+      "grad_norm": 1.0540403127670288,
+      "learning_rate": 0.0002,
+      "loss": 0.3017,
+      "step": 2810
+    },
+    {
+      "epoch": 2.2048475371383893,
+      "grad_norm": 1.007364273071289,
+      "learning_rate": 0.0002,
+      "loss": 0.2959,
+      "step": 2820
+    },
+    {
+      "epoch": 2.2126661454261143,
+      "grad_norm": 0.7497808337211609,
+      "learning_rate": 0.0002,
+      "loss": 0.3025,
+      "step": 2830
+    },
+    {
+      "epoch": 2.220484753713839,
+      "grad_norm": 0.8607106804847717,
+      "learning_rate": 0.0002,
+      "loss": 0.3057,
+      "step": 2840
+    },
+    {
+      "epoch": 2.2283033620015638,
+      "grad_norm": 0.6298855543136597,
+      "learning_rate": 0.0002,
+      "loss": 0.2927,
+      "step": 2850
+    },
+    {
+      "epoch": 2.2361219702892887,
+      "grad_norm": 0.9291629791259766,
+      "learning_rate": 0.0002,
+      "loss": 0.3003,
+      "step": 2860
+    },
+    {
+      "epoch": 2.2439405785770132,
+      "grad_norm": 0.6486570239067078,
+      "learning_rate": 0.0002,
+      "loss": 0.298,
+      "step": 2870
+    },
+    {
+      "epoch": 2.251759186864738,
+      "grad_norm": 0.9577697515487671,
+      "learning_rate": 0.0002,
+      "loss": 0.3013,
+      "step": 2880
+    },
+    {
+      "epoch": 2.2595777951524627,
+      "grad_norm": 0.8712478876113892,
+      "learning_rate": 0.0002,
+      "loss": 0.3037,
+      "step": 2890
+    },
+    {
+      "epoch": 2.2673964034401877,
+      "grad_norm": 0.783490002155304,
+      "learning_rate": 0.0002,
+      "loss": 0.3063,
+      "step": 2900
+    },
+    {
+      "epoch": 2.275215011727912,
+      "grad_norm": 0.9283138513565063,
+      "learning_rate": 0.0002,
+      "loss": 0.2926,
+      "step": 2910
+    },
+    {
+      "epoch": 2.283033620015637,
+      "grad_norm": 1.7705273628234863,
+      "learning_rate": 0.0002,
+      "loss": 0.2941,
+      "step": 2920
+    },
+    {
+      "epoch": 2.290852228303362,
+      "grad_norm": 0.6388339996337891,
+      "learning_rate": 0.0002,
+      "loss": 0.2879,
+      "step": 2930
+    },
+    {
+      "epoch": 2.2986708365910866,
+      "grad_norm": 0.9375593066215515,
+      "learning_rate": 0.0002,
+      "loss": 0.2916,
+      "step": 2940
+    },
+    {
+      "epoch": 2.3064894448788116,
+      "grad_norm": 0.7213515639305115,
+      "learning_rate": 0.0002,
+      "loss": 0.3004,
+      "step": 2950
+    },
+    {
+      "epoch": 2.3143080531665365,
+      "grad_norm": 0.9552587866783142,
+      "learning_rate": 0.0002,
+      "loss": 0.31,
+      "step": 2960
+    },
+    {
+      "epoch": 2.322126661454261,
+      "grad_norm": 0.8279618620872498,
+      "learning_rate": 0.0002,
+      "loss": 0.2943,
+      "step": 2970
+    },
+    {
+      "epoch": 2.329945269741986,
+      "grad_norm": 0.6073647141456604,
+      "learning_rate": 0.0002,
+      "loss": 0.2973,
+      "step": 2980
+    },
+    {
+      "epoch": 2.3377638780297105,
+      "grad_norm": 0.798538863658905,
+      "learning_rate": 0.0002,
+      "loss": 0.2957,
+      "step": 2990
+    },
+    {
+      "epoch": 2.3455824863174355,
+      "grad_norm": 1.0310461521148682,
+      "learning_rate": 0.0002,
+      "loss": 0.3077,
+      "step": 3000
+    },
+    {
+      "epoch": 2.3534010946051604,
+      "grad_norm": 0.7587346434593201,
+      "learning_rate": 0.0002,
+      "loss": 0.2975,
+      "step": 3010
+    },
+    {
+      "epoch": 2.361219702892885,
+      "grad_norm": 0.8931202292442322,
+      "learning_rate": 0.0002,
+      "loss": 0.2847,
+      "step": 3020
+    },
+    {
+      "epoch": 2.36903831118061,
+      "grad_norm": 0.6081550717353821,
+      "learning_rate": 0.0002,
+      "loss": 0.2998,
+      "step": 3030
+    },
+    {
+      "epoch": 2.3768569194683344,
+      "grad_norm": 0.7393735647201538,
+      "learning_rate": 0.0002,
+      "loss": 0.2792,
+      "step": 3040
+    },
+    {
+      "epoch": 2.3846755277560594,
+      "grad_norm": 0.5691865682601929,
+      "learning_rate": 0.0002,
+      "loss": 0.2911,
+      "step": 3050
+    },
+    {
+      "epoch": 2.3924941360437844,
+      "grad_norm": 0.5669929385185242,
+      "learning_rate": 0.0002,
+      "loss": 0.2893,
+      "step": 3060
+    },
+    {
+      "epoch": 2.400312744331509,
+      "grad_norm": 0.9331319332122803,
+      "learning_rate": 0.0002,
+      "loss": 0.2949,
+      "step": 3070
+    },
+    {
+      "epoch": 2.408131352619234,
+      "grad_norm": 0.684083878993988,
+      "learning_rate": 0.0002,
+      "loss": 0.2962,
+      "step": 3080
+    },
+    {
+      "epoch": 2.415949960906959,
+      "grad_norm": 0.6313386559486389,
+      "learning_rate": 0.0002,
+      "loss": 0.2925,
+      "step": 3090
+    },
+    {
+      "epoch": 2.4237685691946833,
+      "grad_norm": 1.1875200271606445,
+      "learning_rate": 0.0002,
+      "loss": 0.2916,
+      "step": 3100
+    },
+    {
+      "epoch": 2.4315871774824083,
+      "grad_norm": 0.6400235295295715,
+      "learning_rate": 0.0002,
+      "loss": 0.2921,
+      "step": 3110
+    },
+    {
+      "epoch": 2.439405785770133,
+      "grad_norm": 0.8620758652687073,
+      "learning_rate": 0.0002,
+      "loss": 0.2989,
+      "step": 3120
+    },
+    {
+      "epoch": 2.4472243940578577,
+      "grad_norm": 0.5646781325340271,
+      "learning_rate": 0.0002,
+      "loss": 0.2952,
+      "step": 3130
+    },
+    {
+      "epoch": 2.4550430023455823,
+      "grad_norm": 0.9492020606994629,
+      "learning_rate": 0.0002,
+      "loss": 0.2956,
+      "step": 3140
+    },
+    {
+      "epoch": 2.462861610633307,
+      "grad_norm": 0.9361923336982727,
+      "learning_rate": 0.0002,
+      "loss": 0.2887,
+      "step": 3150
+    },
+    {
+      "epoch": 2.470680218921032,
+      "grad_norm": 0.8445627689361572,
+      "learning_rate": 0.0002,
+      "loss": 0.292,
+      "step": 3160
+    },
+    {
+      "epoch": 2.4784988272087567,
+      "grad_norm": 0.9534024000167847,
+      "learning_rate": 0.0002,
+      "loss": 0.2887,
+      "step": 3170
+    },
+    {
+      "epoch": 2.4863174354964817,
+      "grad_norm": 0.7826582789421082,
+      "learning_rate": 0.0002,
+      "loss": 0.2866,
+      "step": 3180
+    },
+    {
+      "epoch": 2.4941360437842066,
+      "grad_norm": 0.9183843731880188,
+      "learning_rate": 0.0002,
+      "loss": 0.2932,
+      "step": 3190
+    },
+    {
+      "epoch": 2.501954652071931,
+      "grad_norm": 0.7662791609764099,
+      "learning_rate": 0.0002,
+      "loss": 0.2877,
+      "step": 3200
+    },
+    {
+      "epoch": 2.509773260359656,
+      "grad_norm": 0.5516344308853149,
+      "learning_rate": 0.0002,
+      "loss": 0.3027,
+      "step": 3210
+    },
+    {
+      "epoch": 2.5175918686473806,
+      "grad_norm": 0.8428415656089783,
+      "learning_rate": 0.0002,
+      "loss": 0.2922,
+      "step": 3220
+    },
+    {
+      "epoch": 2.5254104769351056,
+      "grad_norm": 0.7443435192108154,
+      "learning_rate": 0.0002,
+      "loss": 0.2796,
+      "step": 3230
+    },
+    {
+      "epoch": 2.53322908522283,
+      "grad_norm": 0.7847051024436951,
+      "learning_rate": 0.0002,
+      "loss": 0.2882,
+      "step": 3240
+    },
+    {
+      "epoch": 2.541047693510555,
+      "grad_norm": 0.6948403120040894,
+      "learning_rate": 0.0002,
+      "loss": 0.2863,
+      "step": 3250
+    },
+    {
+      "epoch": 2.54886630179828,
+      "grad_norm": 0.6670306921005249,
+      "learning_rate": 0.0002,
+      "loss": 0.2967,
+      "step": 3260
+    },
+    {
+      "epoch": 2.5566849100860045,
+      "grad_norm": 1.191672682762146,
+      "learning_rate": 0.0002,
+      "loss": 0.2942,
+      "step": 3270
+    },
+    {
+      "epoch": 2.5645035183737295,
+      "grad_norm": 0.7504554390907288,
+      "learning_rate": 0.0002,
+      "loss": 0.2902,
+      "step": 3280
+    },
+    {
+      "epoch": 2.5723221266614544,
+      "grad_norm": 0.9115633964538574,
+      "learning_rate": 0.0002,
+      "loss": 0.2815,
+      "step": 3290
+    },
+    {
+      "epoch": 2.580140734949179,
+      "grad_norm": 0.8133999109268188,
+      "learning_rate": 0.0002,
+      "loss": 0.2888,
+      "step": 3300
+    },
+    {
+      "epoch": 2.587959343236904,
+      "grad_norm": 1.501922845840454,
+      "learning_rate": 0.0002,
+      "loss": 0.2736,
+      "step": 3310
+    },
+    {
+      "epoch": 2.595777951524629,
+      "grad_norm": 1.25674569606781,
+      "learning_rate": 0.0002,
+      "loss": 0.2898,
+      "step": 3320
+    },
+    {
+      "epoch": 2.6035965598123534,
+      "grad_norm": 0.6403043270111084,
+      "learning_rate": 0.0002,
+      "loss": 0.2977,
+      "step": 3330
+    },
+    {
+      "epoch": 2.6114151681000783,
+      "grad_norm": 0.802317202091217,
+      "learning_rate": 0.0002,
+      "loss": 0.2891,
+      "step": 3340
+    },
+    {
+      "epoch": 2.619233776387803,
+      "grad_norm": 0.8865256905555725,
+      "learning_rate": 0.0002,
+      "loss": 0.2894,
+      "step": 3350
+    },
+    {
+      "epoch": 2.627052384675528,
+      "grad_norm": 1.0685266256332397,
+      "learning_rate": 0.0002,
+      "loss": 0.2852,
+      "step": 3360
+    },
+    {
+      "epoch": 2.6348709929632523,
+      "grad_norm": 0.6208046078681946,
+      "learning_rate": 0.0002,
+      "loss": 0.2853,
+      "step": 3370
+    },
+    {
+      "epoch": 2.6426896012509773,
+      "grad_norm": 0.7943758368492126,
+      "learning_rate": 0.0002,
+      "loss": 0.2869,
+      "step": 3380
+    },
+    {
+      "epoch": 2.6505082095387023,
+      "grad_norm": 0.46959924697875977,
+      "learning_rate": 0.0002,
+      "loss": 0.294,
+      "step": 3390
+    },
+    {
+      "epoch": 2.6583268178264268,
+      "grad_norm": 0.915804386138916,
+      "learning_rate": 0.0002,
+      "loss": 0.2872,
+      "step": 3400
+    },
+    {
+      "epoch": 2.6661454261141517,
+      "grad_norm": 0.744133710861206,
+      "learning_rate": 0.0002,
+      "loss": 0.2794,
+      "step": 3410
+    },
+    {
+      "epoch": 2.6739640344018767,
+      "grad_norm": 0.6811481714248657,
+      "learning_rate": 0.0002,
+      "loss": 0.2868,
+      "step": 3420
+    },
+    {
+      "epoch": 2.681782642689601,
+      "grad_norm": 0.7864041328430176,
+      "learning_rate": 0.0002,
+      "loss": 0.2947,
+      "step": 3430
+    },
+    {
+      "epoch": 2.689601250977326,
+      "grad_norm": 0.6726815700531006,
+      "learning_rate": 0.0002,
+      "loss": 0.2824,
+      "step": 3440
+    },
+    {
+      "epoch": 2.6974198592650507,
+      "grad_norm": 0.8560361266136169,
+      "learning_rate": 0.0002,
+      "loss": 0.2911,
+      "step": 3450
+    },
+    {
+      "epoch": 2.7052384675527756,
+      "grad_norm": 0.6773067116737366,
+      "learning_rate": 0.0002,
+      "loss": 0.2867,
+      "step": 3460
+    },
+    {
+      "epoch": 2.7130570758405,
+      "grad_norm": 0.8093675971031189,
+      "learning_rate": 0.0002,
+      "loss": 0.2833,
+      "step": 3470
+    },
+    {
+      "epoch": 2.720875684128225,
+      "grad_norm": 0.7565127611160278,
+      "learning_rate": 0.0002,
+      "loss": 0.2897,
+      "step": 3480
+    },
+    {
+      "epoch": 2.72869429241595,
+      "grad_norm": 0.5608393549919128,
+      "learning_rate": 0.0002,
+      "loss": 0.2826,
+      "step": 3490
+    },
+    {
+      "epoch": 2.7365129007036746,
+      "grad_norm": 0.8530815243721008,
+      "learning_rate": 0.0002,
+      "loss": 0.2757,
+      "step": 3500
+    },
+    {
+      "epoch": 2.7443315089913995,
+      "grad_norm": 0.6655313968658447,
+      "learning_rate": 0.0002,
+      "loss": 0.2854,
+      "step": 3510
+    },
+    {
+      "epoch": 2.7521501172791245,
+      "grad_norm": 0.8840405941009521,
+      "learning_rate": 0.0002,
+      "loss": 0.284,
+      "step": 3520
+    },
+    {
+      "epoch": 2.759968725566849,
+      "grad_norm": 0.5805156826972961,
+      "learning_rate": 0.0002,
+      "loss": 0.2898,
+      "step": 3530
+    },
+    {
+      "epoch": 2.767787333854574,
+      "grad_norm": 0.8929122090339661,
+      "learning_rate": 0.0002,
+      "loss": 0.284,
+      "step": 3540
+    },
+    {
+      "epoch": 2.775605942142299,
+      "grad_norm": 0.8708682656288147,
+      "learning_rate": 0.0002,
+      "loss": 0.2889,
+      "step": 3550
+    },
+    {
+      "epoch": 2.7834245504300235,
+      "grad_norm": 0.7769750952720642,
+      "learning_rate": 0.0002,
+      "loss": 0.2758,
+      "step": 3560
+    },
+    {
+      "epoch": 2.791243158717748,
+      "grad_norm": 0.9139852523803711,
+      "learning_rate": 0.0002,
+      "loss": 0.2884,
+      "step": 3570
+    },
+    {
+      "epoch": 2.799061767005473,
+      "grad_norm": 0.8278502821922302,
+      "learning_rate": 0.0002,
+      "loss": 0.2799,
+      "step": 3580
+    },
+    {
+      "epoch": 2.806880375293198,
+      "grad_norm": 1.2494527101516724,
+      "learning_rate": 0.0002,
+      "loss": 0.2813,
+      "step": 3590
+    },
+    {
+      "epoch": 2.8146989835809224,
+      "grad_norm": 0.7714791893959045,
+      "learning_rate": 0.0002,
+      "loss": 0.2865,
+      "step": 3600
+    },
+    {
+      "epoch": 2.8225175918686474,
+      "grad_norm": 0.7571589946746826,
+      "learning_rate": 0.0002,
+      "loss": 0.2814,
+      "step": 3610
+    },
+    {
+      "epoch": 2.8303362001563723,
+      "grad_norm": 1.0769212245941162,
+      "learning_rate": 0.0002,
+      "loss": 0.2808,
+      "step": 3620
+    },
+    {
+      "epoch": 2.838154808444097,
+      "grad_norm": 0.7629138827323914,
+      "learning_rate": 0.0002,
+      "loss": 0.271,
+      "step": 3630
+    },
+    {
+      "epoch": 2.845973416731822,
+      "grad_norm": 0.625410795211792,
+      "learning_rate": 0.0002,
+      "loss": 0.2932,
+      "step": 3640
+    },
+    {
+      "epoch": 2.8537920250195468,
+      "grad_norm": 0.8575489521026611,
+      "learning_rate": 0.0002,
+      "loss": 0.2797,
+      "step": 3650
+    },
+    {
+      "epoch": 2.8616106333072713,
+      "grad_norm": 0.5776465535163879,
+      "learning_rate": 0.0002,
+      "loss": 0.2837,
+      "step": 3660
+    },
+    {
+      "epoch": 2.8694292415949962,
+      "grad_norm": 0.57872474193573,
+      "learning_rate": 0.0002,
+      "loss": 0.2758,
+      "step": 3670
+    },
+    {
+      "epoch": 2.8772478498827208,
+      "grad_norm": 0.5034054517745972,
+      "learning_rate": 0.0002,
+      "loss": 0.2728,
+      "step": 3680
+    },
+    {
+      "epoch": 2.8850664581704457,
+      "grad_norm": 0.8414032459259033,
+      "learning_rate": 0.0002,
+      "loss": 0.2779,
+      "step": 3690
+    },
+    {
+      "epoch": 2.8928850664581702,
+      "grad_norm": 1.0529718399047852,
+      "learning_rate": 0.0002,
+      "loss": 0.2922,
+      "step": 3700
+    },
+    {
+      "epoch": 2.900703674745895,
+      "grad_norm": 0.642629086971283,
+      "learning_rate": 0.0002,
+      "loss": 0.28,
+      "step": 3710
+    },
+    {
+      "epoch": 2.90852228303362,
+      "grad_norm": 0.8804178237915039,
+      "learning_rate": 0.0002,
+      "loss": 0.2754,
+      "step": 3720
+    },
+    {
+      "epoch": 2.9163408913213447,
+      "grad_norm": 0.8823941349983215,
+      "learning_rate": 0.0002,
+      "loss": 0.272,
+      "step": 3730
+    },
+    {
+      "epoch": 2.9241594996090696,
+      "grad_norm": 0.7909939885139465,
+      "learning_rate": 0.0002,
+      "loss": 0.2764,
+      "step": 3740
+    },
+    {
+      "epoch": 2.9319781078967946,
+      "grad_norm": 0.9231449961662292,
+      "learning_rate": 0.0002,
+      "loss": 0.2815,
+      "step": 3750
+    },
+    {
+      "epoch": 2.939796716184519,
+      "grad_norm": 0.6667030453681946,
+      "learning_rate": 0.0002,
+      "loss": 0.2946,
+      "step": 3760
+    },
+    {
+      "epoch": 2.947615324472244,
+      "grad_norm": 0.7937665581703186,
+      "learning_rate": 0.0002,
+      "loss": 0.2778,
+      "step": 3770
+    },
+    {
+      "epoch": 2.9554339327599686,
+      "grad_norm": 0.6896514892578125,
+      "learning_rate": 0.0002,
+      "loss": 0.2867,
+      "step": 3780
+    },
+    {
+      "epoch": 2.9632525410476935,
+      "grad_norm": 0.8817704319953918,
+      "learning_rate": 0.0002,
+      "loss": 0.2823,
+      "step": 3790
+    },
+    {
+      "epoch": 2.971071149335418,
+      "grad_norm": 0.6947396397590637,
+      "learning_rate": 0.0002,
+      "loss": 0.2798,
+      "step": 3800
+    },
+    {
+      "epoch": 2.978889757623143,
+      "grad_norm": 0.690621554851532,
+      "learning_rate": 0.0002,
+      "loss": 0.2829,
+      "step": 3810
+    },
+    {
+      "epoch": 2.986708365910868,
+      "grad_norm": 0.5144171714782715,
+      "learning_rate": 0.0002,
+      "loss": 0.2712,
+      "step": 3820
+    },
+    {
+      "epoch": 2.9945269741985925,
+      "grad_norm": 0.7612534761428833,
+      "learning_rate": 0.0002,
+      "loss": 0.2816,
+      "step": 3830
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 0.2826317846775055,
+      "eval_runtime": 68.3043,
+      "eval_samples_per_second": 5.344,
+      "eval_steps_per_second": 0.673,
+      "step": 3837
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 10232,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.683473263755264e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}