Upload folder using huggingface_hub

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5b7d36cc3fa8251baf43d682f58cb1e30535f2bcec0330202bb0806e5147aff7
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1026/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1026/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1026/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa7ea09b9331de78f87cffe1c7fdfb3888b5a005db45eca311e78434210e8daa
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1026/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3823df15f9c684dfcdf97b27d420f5fe57c141f644bd86bb6446cc277a95bafe
+size 55532666

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1026/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c17c40a313565f37a91317d6ca1a47621cb8f3350ac9637bd368ea9521c3c7e2
+size 14244

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1026/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:660ab54d0c298f22afda4a2655936aecd813daf93f4fc040b7d285b3684be4c0
+size 1064

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1026/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1026/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1026/trainer_state.json

@@ -0,0 +1,779 @@
+{
+  "best_metric": 0.4771515429019928,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1026",
+  "epoch": 4.0,
+  "eval_steps": 10,
+  "global_step": 1026,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.03898635477582846,
+      "grad_norm": 1.1630783081054688,
+      "learning_rate": 0.0002,
+      "loss": 1.7486,
+      "step": 10
+    },
+    {
+      "epoch": 0.07797270955165692,
+      "grad_norm": 1.3587305545806885,
+      "learning_rate": 0.0002,
+      "loss": 1.1818,
+      "step": 20
+    },
+    {
+      "epoch": 0.11695906432748537,
+      "grad_norm": 0.9663119912147522,
+      "learning_rate": 0.0002,
+      "loss": 1.0017,
+      "step": 30
+    },
+    {
+      "epoch": 0.15594541910331383,
+      "grad_norm": 1.0442273616790771,
+      "learning_rate": 0.0002,
+      "loss": 0.945,
+      "step": 40
+    },
+    {
+      "epoch": 0.1949317738791423,
+      "grad_norm": 0.9459889531135559,
+      "learning_rate": 0.0002,
+      "loss": 0.7863,
+      "step": 50
+    },
+    {
+      "epoch": 0.23391812865497075,
+      "grad_norm": 0.9623780846595764,
+      "learning_rate": 0.0002,
+      "loss": 0.7287,
+      "step": 60
+    },
+    {
+      "epoch": 0.2729044834307992,
+      "grad_norm": 0.8872420787811279,
+      "learning_rate": 0.0002,
+      "loss": 0.6969,
+      "step": 70
+    },
+    {
+      "epoch": 0.31189083820662766,
+      "grad_norm": 0.7173621654510498,
+      "learning_rate": 0.0002,
+      "loss": 0.6828,
+      "step": 80
+    },
+    {
+      "epoch": 0.3508771929824561,
+      "grad_norm": 0.7221336364746094,
+      "learning_rate": 0.0002,
+      "loss": 0.6651,
+      "step": 90
+    },
+    {
+      "epoch": 0.3898635477582846,
+      "grad_norm": 0.7142595648765564,
+      "learning_rate": 0.0002,
+      "loss": 0.621,
+      "step": 100
+    },
+    {
+      "epoch": 0.42884990253411304,
+      "grad_norm": 0.6138886213302612,
+      "learning_rate": 0.0002,
+      "loss": 0.6601,
+      "step": 110
+    },
+    {
+      "epoch": 0.4678362573099415,
+      "grad_norm": 0.8129922151565552,
+      "learning_rate": 0.0002,
+      "loss": 0.6182,
+      "step": 120
+    },
+    {
+      "epoch": 0.50682261208577,
+      "grad_norm": 0.8280786275863647,
+      "learning_rate": 0.0002,
+      "loss": 0.6426,
+      "step": 130
+    },
+    {
+      "epoch": 0.5458089668615984,
+      "grad_norm": 0.5899930596351624,
+      "learning_rate": 0.0002,
+      "loss": 0.6125,
+      "step": 140
+    },
+    {
+      "epoch": 0.5847953216374269,
+      "grad_norm": 0.7101946473121643,
+      "learning_rate": 0.0002,
+      "loss": 0.6444,
+      "step": 150
+    },
+    {
+      "epoch": 0.6237816764132553,
+      "grad_norm": 0.6261710524559021,
+      "learning_rate": 0.0002,
+      "loss": 0.6374,
+      "step": 160
+    },
+    {
+      "epoch": 0.6627680311890838,
+      "grad_norm": 0.7684038877487183,
+      "learning_rate": 0.0002,
+      "loss": 0.594,
+      "step": 170
+    },
+    {
+      "epoch": 0.7017543859649122,
+      "grad_norm": 0.581214189529419,
+      "learning_rate": 0.0002,
+      "loss": 0.5992,
+      "step": 180
+    },
+    {
+      "epoch": 0.7407407407407407,
+      "grad_norm": 0.5658173561096191,
+      "learning_rate": 0.0002,
+      "loss": 0.6179,
+      "step": 190
+    },
+    {
+      "epoch": 0.7797270955165692,
+      "grad_norm": 0.7603134512901306,
+      "learning_rate": 0.0002,
+      "loss": 0.6417,
+      "step": 200
+    },
+    {
+      "epoch": 0.8187134502923976,
+      "grad_norm": 0.5359350442886353,
+      "learning_rate": 0.0002,
+      "loss": 0.5906,
+      "step": 210
+    },
+    {
+      "epoch": 0.8576998050682261,
+      "grad_norm": 0.7978444695472717,
+      "learning_rate": 0.0002,
+      "loss": 0.6125,
+      "step": 220
+    },
+    {
+      "epoch": 0.8966861598440545,
+      "grad_norm": 0.5791468620300293,
+      "learning_rate": 0.0002,
+      "loss": 0.5984,
+      "step": 230
+    },
+    {
+      "epoch": 0.935672514619883,
+      "grad_norm": 0.635198712348938,
+      "learning_rate": 0.0002,
+      "loss": 0.6007,
+      "step": 240
+    },
+    {
+      "epoch": 0.9746588693957114,
+      "grad_norm": 0.5510166883468628,
+      "learning_rate": 0.0002,
+      "loss": 0.5912,
+      "step": 250
+    },
+    {
+      "epoch": 0.9980506822612085,
+      "eval_loss": 0.5850377678871155,
+      "eval_runtime": 72.8118,
+      "eval_samples_per_second": 5.013,
+      "eval_steps_per_second": 0.632,
+      "step": 256
+    },
+    {
+      "epoch": 1.01364522417154,
+      "grad_norm": 0.5623628497123718,
+      "learning_rate": 0.0002,
+      "loss": 0.575,
+      "step": 260
+    },
+    {
+      "epoch": 1.0526315789473684,
+      "grad_norm": 0.6872021555900574,
+      "learning_rate": 0.0002,
+      "loss": 0.5498,
+      "step": 270
+    },
+    {
+      "epoch": 1.0916179337231968,
+      "grad_norm": 0.5543047785758972,
+      "learning_rate": 0.0002,
+      "loss": 0.5373,
+      "step": 280
+    },
+    {
+      "epoch": 1.1306042884990253,
+      "grad_norm": 0.498542845249176,
+      "learning_rate": 0.0002,
+      "loss": 0.5272,
+      "step": 290
+    },
+    {
+      "epoch": 1.1695906432748537,
+      "grad_norm": 0.5940739512443542,
+      "learning_rate": 0.0002,
+      "loss": 0.5528,
+      "step": 300
+    },
+    {
+      "epoch": 1.2085769980506822,
+      "grad_norm": 0.5854314565658569,
+      "learning_rate": 0.0002,
+      "loss": 0.5308,
+      "step": 310
+    },
+    {
+      "epoch": 1.2475633528265107,
+      "grad_norm": 0.5293670892715454,
+      "learning_rate": 0.0002,
+      "loss": 0.5452,
+      "step": 320
+    },
+    {
+      "epoch": 1.286549707602339,
+      "grad_norm": 0.5736058950424194,
+      "learning_rate": 0.0002,
+      "loss": 0.5451,
+      "step": 330
+    },
+    {
+      "epoch": 1.3255360623781676,
+      "grad_norm": 0.7912607789039612,
+      "learning_rate": 0.0002,
+      "loss": 0.5344,
+      "step": 340
+    },
+    {
+      "epoch": 1.364522417153996,
+      "grad_norm": 0.781843364238739,
+      "learning_rate": 0.0002,
+      "loss": 0.531,
+      "step": 350
+    },
+    {
+      "epoch": 1.4035087719298245,
+      "grad_norm": 0.7365643978118896,
+      "learning_rate": 0.0002,
+      "loss": 0.5396,
+      "step": 360
+    },
+    {
+      "epoch": 1.442495126705653,
+      "grad_norm": 0.5565363764762878,
+      "learning_rate": 0.0002,
+      "loss": 0.5363,
+      "step": 370
+    },
+    {
+      "epoch": 1.4814814814814814,
+      "grad_norm": 0.5427217483520508,
+      "learning_rate": 0.0002,
+      "loss": 0.5235,
+      "step": 380
+    },
+    {
+      "epoch": 1.52046783625731,
+      "grad_norm": 0.7150877118110657,
+      "learning_rate": 0.0002,
+      "loss": 0.5173,
+      "step": 390
+    },
+    {
+      "epoch": 1.5594541910331383,
+      "grad_norm": 0.5850045084953308,
+      "learning_rate": 0.0002,
+      "loss": 0.5394,
+      "step": 400
+    },
+    {
+      "epoch": 1.598440545808967,
+      "grad_norm": 0.5376710295677185,
+      "learning_rate": 0.0002,
+      "loss": 0.5045,
+      "step": 410
+    },
+    {
+      "epoch": 1.6374269005847952,
+      "grad_norm": 0.6234501004219055,
+      "learning_rate": 0.0002,
+      "loss": 0.5444,
+      "step": 420
+    },
+    {
+      "epoch": 1.676413255360624,
+      "grad_norm": 0.7774652242660522,
+      "learning_rate": 0.0002,
+      "loss": 0.5186,
+      "step": 430
+    },
+    {
+      "epoch": 1.7153996101364521,
+      "grad_norm": 0.6821911334991455,
+      "learning_rate": 0.0002,
+      "loss": 0.5168,
+      "step": 440
+    },
+    {
+      "epoch": 1.7543859649122808,
+      "grad_norm": 0.7210469841957092,
+      "learning_rate": 0.0002,
+      "loss": 0.5118,
+      "step": 450
+    },
+    {
+      "epoch": 1.793372319688109,
+      "grad_norm": 0.6935250759124756,
+      "learning_rate": 0.0002,
+      "loss": 0.5499,
+      "step": 460
+    },
+    {
+      "epoch": 1.8323586744639377,
+      "grad_norm": 0.6940227150917053,
+      "learning_rate": 0.0002,
+      "loss": 0.516,
+      "step": 470
+    },
+    {
+      "epoch": 1.871345029239766,
+      "grad_norm": 0.6918772459030151,
+      "learning_rate": 0.0002,
+      "loss": 0.4839,
+      "step": 480
+    },
+    {
+      "epoch": 1.9103313840155947,
+      "grad_norm": 0.5278885364532471,
+      "learning_rate": 0.0002,
+      "loss": 0.4906,
+      "step": 490
+    },
+    {
+      "epoch": 1.949317738791423,
+      "grad_norm": 0.655097246170044,
+      "learning_rate": 0.0002,
+      "loss": 0.4954,
+      "step": 500
+    },
+    {
+      "epoch": 1.9883040935672516,
+      "grad_norm": 0.728690505027771,
+      "learning_rate": 0.0002,
+      "loss": 0.4981,
+      "step": 510
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 0.5312528610229492,
+      "eval_runtime": 67.5284,
+      "eval_samples_per_second": 5.405,
+      "eval_steps_per_second": 0.681,
+      "step": 513
+    },
+    {
+      "epoch": 2.02729044834308,
+      "grad_norm": 0.7993459105491638,
+      "learning_rate": 0.0002,
+      "loss": 0.473,
+      "step": 520
+    },
+    {
+      "epoch": 2.0662768031189085,
+      "grad_norm": 0.6997111439704895,
+      "learning_rate": 0.0002,
+      "loss": 0.4514,
+      "step": 530
+    },
+    {
+      "epoch": 2.1052631578947367,
+      "grad_norm": 0.9680729508399963,
+      "learning_rate": 0.0002,
+      "loss": 0.4537,
+      "step": 540
+    },
+    {
+      "epoch": 2.1442495126705654,
+      "grad_norm": 0.67018723487854,
+      "learning_rate": 0.0002,
+      "loss": 0.4278,
+      "step": 550
+    },
+    {
+      "epoch": 2.1832358674463936,
+      "grad_norm": 0.7760737538337708,
+      "learning_rate": 0.0002,
+      "loss": 0.4424,
+      "step": 560
+    },
+    {
+      "epoch": 2.2222222222222223,
+      "grad_norm": 1.9475411176681519,
+      "learning_rate": 0.0002,
+      "loss": 0.4554,
+      "step": 570
+    },
+    {
+      "epoch": 2.2612085769980506,
+      "grad_norm": 0.810126006603241,
+      "learning_rate": 0.0002,
+      "loss": 0.4321,
+      "step": 580
+    },
+    {
+      "epoch": 2.3001949317738792,
+      "grad_norm": 0.8697490692138672,
+      "learning_rate": 0.0002,
+      "loss": 0.4413,
+      "step": 590
+    },
+    {
+      "epoch": 2.3391812865497075,
+      "grad_norm": 0.5910632014274597,
+      "learning_rate": 0.0002,
+      "loss": 0.445,
+      "step": 600
+    },
+    {
+      "epoch": 2.378167641325536,
+      "grad_norm": 1.0113483667373657,
+      "learning_rate": 0.0002,
+      "loss": 0.43,
+      "step": 610
+    },
+    {
+      "epoch": 2.4171539961013644,
+      "grad_norm": 0.5548872351646423,
+      "learning_rate": 0.0002,
+      "loss": 0.4491,
+      "step": 620
+    },
+    {
+      "epoch": 2.456140350877193,
+      "grad_norm": 0.7542755603790283,
+      "learning_rate": 0.0002,
+      "loss": 0.4654,
+      "step": 630
+    },
+    {
+      "epoch": 2.4951267056530213,
+      "grad_norm": 0.6450021862983704,
+      "learning_rate": 0.0002,
+      "loss": 0.4269,
+      "step": 640
+    },
+    {
+      "epoch": 2.53411306042885,
+      "grad_norm": 0.8181225657463074,
+      "learning_rate": 0.0002,
+      "loss": 0.4321,
+      "step": 650
+    },
+    {
+      "epoch": 2.573099415204678,
+      "grad_norm": 0.8882361650466919,
+      "learning_rate": 0.0002,
+      "loss": 0.4476,
+      "step": 660
+    },
+    {
+      "epoch": 2.612085769980507,
+      "grad_norm": 0.9545704126358032,
+      "learning_rate": 0.0002,
+      "loss": 0.4398,
+      "step": 670
+    },
+    {
+      "epoch": 2.651072124756335,
+      "grad_norm": 0.633364200592041,
+      "learning_rate": 0.0002,
+      "loss": 0.4302,
+      "step": 680
+    },
+    {
+      "epoch": 2.690058479532164,
+      "grad_norm": 0.6909534931182861,
+      "learning_rate": 0.0002,
+      "loss": 0.4265,
+      "step": 690
+    },
+    {
+      "epoch": 2.729044834307992,
+      "grad_norm": 0.7342118620872498,
+      "learning_rate": 0.0002,
+      "loss": 0.4508,
+      "step": 700
+    },
+    {
+      "epoch": 2.7680311890838207,
+      "grad_norm": 0.6567948460578918,
+      "learning_rate": 0.0002,
+      "loss": 0.4495,
+      "step": 710
+    },
+    {
+      "epoch": 2.807017543859649,
+      "grad_norm": 0.81608647108078,
+      "learning_rate": 0.0002,
+      "loss": 0.4521,
+      "step": 720
+    },
+    {
+      "epoch": 2.8460038986354776,
+      "grad_norm": 0.7075870633125305,
+      "learning_rate": 0.0002,
+      "loss": 0.4584,
+      "step": 730
+    },
+    {
+      "epoch": 2.884990253411306,
+      "grad_norm": 0.5666054487228394,
+      "learning_rate": 0.0002,
+      "loss": 0.4332,
+      "step": 740
+    },
+    {
+      "epoch": 2.9239766081871346,
+      "grad_norm": 0.5457495450973511,
+      "learning_rate": 0.0002,
+      "loss": 0.423,
+      "step": 750
+    },
+    {
+      "epoch": 2.962962962962963,
+      "grad_norm": 0.8959799408912659,
+      "learning_rate": 0.0002,
+      "loss": 0.4136,
+      "step": 760
+    },
+    {
+      "epoch": 2.9980506822612085,
+      "eval_loss": 0.49315401911735535,
+      "eval_runtime": 67.9039,
+      "eval_samples_per_second": 5.375,
+      "eval_steps_per_second": 0.677,
+      "step": 769
+    },
+    {
+      "epoch": 3.0019493177387915,
+      "grad_norm": 0.8162972927093506,
+      "learning_rate": 0.0002,
+      "loss": 0.4332,
+      "step": 770
+    },
+    {
+      "epoch": 3.0409356725146197,
+      "grad_norm": 0.8814733028411865,
+      "learning_rate": 0.0002,
+      "loss": 0.3778,
+      "step": 780
+    },
+    {
+      "epoch": 3.0799220272904484,
+      "grad_norm": 0.6659106612205505,
+      "learning_rate": 0.0002,
+      "loss": 0.3775,
+      "step": 790
+    },
+    {
+      "epoch": 3.1189083820662766,
+      "grad_norm": 0.7013096213340759,
+      "learning_rate": 0.0002,
+      "loss": 0.3718,
+      "step": 800
+    },
+    {
+      "epoch": 3.1578947368421053,
+      "grad_norm": 0.5946340560913086,
+      "learning_rate": 0.0002,
+      "loss": 0.3799,
+      "step": 810
+    },
+    {
+      "epoch": 3.1968810916179335,
+      "grad_norm": 1.1263338327407837,
+      "learning_rate": 0.0002,
+      "loss": 0.3752,
+      "step": 820
+    },
+    {
+      "epoch": 3.2358674463937622,
+      "grad_norm": 1.028369426727295,
+      "learning_rate": 0.0002,
+      "loss": 0.3819,
+      "step": 830
+    },
+    {
+      "epoch": 3.2748538011695905,
+      "grad_norm": 1.1113781929016113,
+      "learning_rate": 0.0002,
+      "loss": 0.3816,
+      "step": 840
+    },
+    {
+      "epoch": 3.313840155945419,
+      "grad_norm": 0.7735888361930847,
+      "learning_rate": 0.0002,
+      "loss": 0.383,
+      "step": 850
+    },
+    {
+      "epoch": 3.3528265107212474,
+      "grad_norm": 0.8967649936676025,
+      "learning_rate": 0.0002,
+      "loss": 0.3703,
+      "step": 860
+    },
+    {
+      "epoch": 3.391812865497076,
+      "grad_norm": 0.7354698777198792,
+      "learning_rate": 0.0002,
+      "loss": 0.3884,
+      "step": 870
+    },
+    {
+      "epoch": 3.4307992202729043,
+      "grad_norm": 0.8721298575401306,
+      "learning_rate": 0.0002,
+      "loss": 0.3762,
+      "step": 880
+    },
+    {
+      "epoch": 3.469785575048733,
+      "grad_norm": 0.7276861667633057,
+      "learning_rate": 0.0002,
+      "loss": 0.3758,
+      "step": 890
+    },
+    {
+      "epoch": 3.5087719298245617,
+      "grad_norm": 0.8296075463294983,
+      "learning_rate": 0.0002,
+      "loss": 0.3866,
+      "step": 900
+    },
+    {
+      "epoch": 3.54775828460039,
+      "grad_norm": 0.8955655097961426,
+      "learning_rate": 0.0002,
+      "loss": 0.3909,
+      "step": 910
+    },
+    {
+      "epoch": 3.586744639376218,
+      "grad_norm": 0.8008049130439758,
+      "learning_rate": 0.0002,
+      "loss": 0.3882,
+      "step": 920
+    },
+    {
+      "epoch": 3.625730994152047,
+      "grad_norm": 0.7791480422019958,
+      "learning_rate": 0.0002,
+      "loss": 0.3837,
+      "step": 930
+    },
+    {
+      "epoch": 3.6647173489278755,
+      "grad_norm": 0.8780500292778015,
+      "learning_rate": 0.0002,
+      "loss": 0.3877,
+      "step": 940
+    },
+    {
+      "epoch": 3.7037037037037037,
+      "grad_norm": 0.8411193490028381,
+      "learning_rate": 0.0002,
+      "loss": 0.3852,
+      "step": 950
+    },
+    {
+      "epoch": 3.742690058479532,
+      "grad_norm": 0.7564510703086853,
+      "learning_rate": 0.0002,
+      "loss": 0.3789,
+      "step": 960
+    },
+    {
+      "epoch": 3.7816764132553606,
+      "grad_norm": 0.9967873692512512,
+      "learning_rate": 0.0002,
+      "loss": 0.3854,
+      "step": 970
+    },
+    {
+      "epoch": 3.8206627680311893,
+      "grad_norm": 1.0118954181671143,
+      "learning_rate": 0.0002,
+      "loss": 0.3664,
+      "step": 980
+    },
+    {
+      "epoch": 3.8596491228070176,
+      "grad_norm": 0.7355602383613586,
+      "learning_rate": 0.0002,
+      "loss": 0.367,
+      "step": 990
+    },
+    {
+      "epoch": 3.898635477582846,
+      "grad_norm": 0.7059919834136963,
+      "learning_rate": 0.0002,
+      "loss": 0.3802,
+      "step": 1000
+    },
+    {
+      "epoch": 3.9376218323586745,
+      "grad_norm": 0.6720611453056335,
+      "learning_rate": 0.0002,
+      "loss": 0.3907,
+      "step": 1010
+    },
+    {
+      "epoch": 3.976608187134503,
+      "grad_norm": 0.664450466632843,
+      "learning_rate": 0.0002,
+      "loss": 0.392,
+      "step": 1020
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 0.4771515429019928,
+      "eval_runtime": 68.2605,
+      "eval_samples_per_second": 5.347,
+      "eval_steps_per_second": 0.674,
+      "step": 1026
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 2048,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4.50154696015872e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1026/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5036267d1db24ac0a5932b39180f94ba0ea9b0dc9da633253ea9585ce5cdb211
+size 5560

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1282/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1282/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1282/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:adf9b909e7d370b95857d5528e958b8a7bca0148f0302e8dc68562d7a152dd19
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1282/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c23f75f6f27994e7a94f61e92dc79de28612b0ead0bf4f0f4154d30a9eca8d49
+size 55532666

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1282/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a269f00eb5cc597716535d40157c9a4f95803e3a5c27adef552d58f8b3adaf5b
+size 14244

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1282/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8dcc80f22b8e67356e79d8bed6a61c05a8f305ea505dcdf6343406c795c4a4cd
+size 1064

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1282/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1282/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1282/trainer_state.json

@@ -0,0 +1,969 @@
+{
+  "best_metric": 0.4660908877849579,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1282",
+  "epoch": 4.9980506822612085,
+  "eval_steps": 10,
+  "global_step": 1282,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.03898635477582846,
+      "grad_norm": 1.1630783081054688,
+      "learning_rate": 0.0002,
+      "loss": 1.7486,
+      "step": 10
+    },
+    {
+      "epoch": 0.07797270955165692,
+      "grad_norm": 1.3587305545806885,
+      "learning_rate": 0.0002,
+      "loss": 1.1818,
+      "step": 20
+    },
+    {
+      "epoch": 0.11695906432748537,
+      "grad_norm": 0.9663119912147522,
+      "learning_rate": 0.0002,
+      "loss": 1.0017,
+      "step": 30
+    },
+    {
+      "epoch": 0.15594541910331383,
+      "grad_norm": 1.0442273616790771,
+      "learning_rate": 0.0002,
+      "loss": 0.945,
+      "step": 40
+    },
+    {
+      "epoch": 0.1949317738791423,
+      "grad_norm": 0.9459889531135559,
+      "learning_rate": 0.0002,
+      "loss": 0.7863,
+      "step": 50
+    },
+    {
+      "epoch": 0.23391812865497075,
+      "grad_norm": 0.9623780846595764,
+      "learning_rate": 0.0002,
+      "loss": 0.7287,
+      "step": 60
+    },
+    {
+      "epoch": 0.2729044834307992,
+      "grad_norm": 0.8872420787811279,
+      "learning_rate": 0.0002,
+      "loss": 0.6969,
+      "step": 70
+    },
+    {
+      "epoch": 0.31189083820662766,
+      "grad_norm": 0.7173621654510498,
+      "learning_rate": 0.0002,
+      "loss": 0.6828,
+      "step": 80
+    },
+    {
+      "epoch": 0.3508771929824561,
+      "grad_norm": 0.7221336364746094,
+      "learning_rate": 0.0002,
+      "loss": 0.6651,
+      "step": 90
+    },
+    {
+      "epoch": 0.3898635477582846,
+      "grad_norm": 0.7142595648765564,
+      "learning_rate": 0.0002,
+      "loss": 0.621,
+      "step": 100
+    },
+    {
+      "epoch": 0.42884990253411304,
+      "grad_norm": 0.6138886213302612,
+      "learning_rate": 0.0002,
+      "loss": 0.6601,
+      "step": 110
+    },
+    {
+      "epoch": 0.4678362573099415,
+      "grad_norm": 0.8129922151565552,
+      "learning_rate": 0.0002,
+      "loss": 0.6182,
+      "step": 120
+    },
+    {
+      "epoch": 0.50682261208577,
+      "grad_norm": 0.8280786275863647,
+      "learning_rate": 0.0002,
+      "loss": 0.6426,
+      "step": 130
+    },
+    {
+      "epoch": 0.5458089668615984,
+      "grad_norm": 0.5899930596351624,
+      "learning_rate": 0.0002,
+      "loss": 0.6125,
+      "step": 140
+    },
+    {
+      "epoch": 0.5847953216374269,
+      "grad_norm": 0.7101946473121643,
+      "learning_rate": 0.0002,
+      "loss": 0.6444,
+      "step": 150
+    },
+    {
+      "epoch": 0.6237816764132553,
+      "grad_norm": 0.6261710524559021,
+      "learning_rate": 0.0002,
+      "loss": 0.6374,
+      "step": 160
+    },
+    {
+      "epoch": 0.6627680311890838,
+      "grad_norm": 0.7684038877487183,
+      "learning_rate": 0.0002,
+      "loss": 0.594,
+      "step": 170
+    },
+    {
+      "epoch": 0.7017543859649122,
+      "grad_norm": 0.581214189529419,
+      "learning_rate": 0.0002,
+      "loss": 0.5992,
+      "step": 180
+    },
+    {
+      "epoch": 0.7407407407407407,
+      "grad_norm": 0.5658173561096191,
+      "learning_rate": 0.0002,
+      "loss": 0.6179,
+      "step": 190
+    },
+    {
+      "epoch": 0.7797270955165692,
+      "grad_norm": 0.7603134512901306,
+      "learning_rate": 0.0002,
+      "loss": 0.6417,
+      "step": 200
+    },
+    {
+      "epoch": 0.8187134502923976,
+      "grad_norm": 0.5359350442886353,
+      "learning_rate": 0.0002,
+      "loss": 0.5906,
+      "step": 210
+    },
+    {
+      "epoch": 0.8576998050682261,
+      "grad_norm": 0.7978444695472717,
+      "learning_rate": 0.0002,
+      "loss": 0.6125,
+      "step": 220
+    },
+    {
+      "epoch": 0.8966861598440545,
+      "grad_norm": 0.5791468620300293,
+      "learning_rate": 0.0002,
+      "loss": 0.5984,
+      "step": 230
+    },
+    {
+      "epoch": 0.935672514619883,
+      "grad_norm": 0.635198712348938,
+      "learning_rate": 0.0002,
+      "loss": 0.6007,
+      "step": 240
+    },
+    {
+      "epoch": 0.9746588693957114,
+      "grad_norm": 0.5510166883468628,
+      "learning_rate": 0.0002,
+      "loss": 0.5912,
+      "step": 250
+    },
+    {
+      "epoch": 0.9980506822612085,
+      "eval_loss": 0.5850377678871155,
+      "eval_runtime": 72.8118,
+      "eval_samples_per_second": 5.013,
+      "eval_steps_per_second": 0.632,
+      "step": 256
+    },
+    {
+      "epoch": 1.01364522417154,
+      "grad_norm": 0.5623628497123718,
+      "learning_rate": 0.0002,
+      "loss": 0.575,
+      "step": 260
+    },
+    {
+      "epoch": 1.0526315789473684,
+      "grad_norm": 0.6872021555900574,
+      "learning_rate": 0.0002,
+      "loss": 0.5498,
+      "step": 270
+    },
+    {
+      "epoch": 1.0916179337231968,
+      "grad_norm": 0.5543047785758972,
+      "learning_rate": 0.0002,
+      "loss": 0.5373,
+      "step": 280
+    },
+    {
+      "epoch": 1.1306042884990253,
+      "grad_norm": 0.498542845249176,
+      "learning_rate": 0.0002,
+      "loss": 0.5272,
+      "step": 290
+    },
+    {
+      "epoch": 1.1695906432748537,
+      "grad_norm": 0.5940739512443542,
+      "learning_rate": 0.0002,
+      "loss": 0.5528,
+      "step": 300
+    },
+    {
+      "epoch": 1.2085769980506822,
+      "grad_norm": 0.5854314565658569,
+      "learning_rate": 0.0002,
+      "loss": 0.5308,
+      "step": 310
+    },
+    {
+      "epoch": 1.2475633528265107,
+      "grad_norm": 0.5293670892715454,
+      "learning_rate": 0.0002,
+      "loss": 0.5452,
+      "step": 320
+    },
+    {
+      "epoch": 1.286549707602339,
+      "grad_norm": 0.5736058950424194,
+      "learning_rate": 0.0002,
+      "loss": 0.5451,
+      "step": 330
+    },
+    {
+      "epoch": 1.3255360623781676,
+      "grad_norm": 0.7912607789039612,
+      "learning_rate": 0.0002,
+      "loss": 0.5344,
+      "step": 340
+    },
+    {
+      "epoch": 1.364522417153996,
+      "grad_norm": 0.781843364238739,
+      "learning_rate": 0.0002,
+      "loss": 0.531,
+      "step": 350
+    },
+    {
+      "epoch": 1.4035087719298245,
+      "grad_norm": 0.7365643978118896,
+      "learning_rate": 0.0002,
+      "loss": 0.5396,
+      "step": 360
+    },
+    {
+      "epoch": 1.442495126705653,
+      "grad_norm": 0.5565363764762878,
+      "learning_rate": 0.0002,
+      "loss": 0.5363,
+      "step": 370
+    },
+    {
+      "epoch": 1.4814814814814814,
+      "grad_norm": 0.5427217483520508,
+      "learning_rate": 0.0002,
+      "loss": 0.5235,
+      "step": 380
+    },
+    {
+      "epoch": 1.52046783625731,
+      "grad_norm": 0.7150877118110657,
+      "learning_rate": 0.0002,
+      "loss": 0.5173,
+      "step": 390
+    },
+    {
+      "epoch": 1.5594541910331383,
+      "grad_norm": 0.5850045084953308,
+      "learning_rate": 0.0002,
+      "loss": 0.5394,
+      "step": 400
+    },
+    {
+      "epoch": 1.598440545808967,
+      "grad_norm": 0.5376710295677185,
+      "learning_rate": 0.0002,
+      "loss": 0.5045,
+      "step": 410
+    },
+    {
+      "epoch": 1.6374269005847952,
+      "grad_norm": 0.6234501004219055,
+      "learning_rate": 0.0002,
+      "loss": 0.5444,
+      "step": 420
+    },
+    {
+      "epoch": 1.676413255360624,
+      "grad_norm": 0.7774652242660522,
+      "learning_rate": 0.0002,
+      "loss": 0.5186,
+      "step": 430
+    },
+    {
+      "epoch": 1.7153996101364521,
+      "grad_norm": 0.6821911334991455,
+      "learning_rate": 0.0002,
+      "loss": 0.5168,
+      "step": 440
+    },
+    {
+      "epoch": 1.7543859649122808,
+      "grad_norm": 0.7210469841957092,
+      "learning_rate": 0.0002,
+      "loss": 0.5118,
+      "step": 450
+    },
+    {
+      "epoch": 1.793372319688109,
+      "grad_norm": 0.6935250759124756,
+      "learning_rate": 0.0002,
+      "loss": 0.5499,
+      "step": 460
+    },
+    {
+      "epoch": 1.8323586744639377,
+      "grad_norm": 0.6940227150917053,
+      "learning_rate": 0.0002,
+      "loss": 0.516,
+      "step": 470
+    },
+    {
+      "epoch": 1.871345029239766,
+      "grad_norm": 0.6918772459030151,
+      "learning_rate": 0.0002,
+      "loss": 0.4839,
+      "step": 480
+    },
+    {
+      "epoch": 1.9103313840155947,
+      "grad_norm": 0.5278885364532471,
+      "learning_rate": 0.0002,
+      "loss": 0.4906,
+      "step": 490
+    },
+    {
+      "epoch": 1.949317738791423,
+      "grad_norm": 0.655097246170044,
+      "learning_rate": 0.0002,
+      "loss": 0.4954,
+      "step": 500
+    },
+    {
+      "epoch": 1.9883040935672516,
+      "grad_norm": 0.728690505027771,
+      "learning_rate": 0.0002,
+      "loss": 0.4981,
+      "step": 510
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 0.5312528610229492,
+      "eval_runtime": 67.5284,
+      "eval_samples_per_second": 5.405,
+      "eval_steps_per_second": 0.681,
+      "step": 513
+    },
+    {
+      "epoch": 2.02729044834308,
+      "grad_norm": 0.7993459105491638,
+      "learning_rate": 0.0002,
+      "loss": 0.473,
+      "step": 520
+    },
+    {
+      "epoch": 2.0662768031189085,
+      "grad_norm": 0.6997111439704895,
+      "learning_rate": 0.0002,
+      "loss": 0.4514,
+      "step": 530
+    },
+    {
+      "epoch": 2.1052631578947367,
+      "grad_norm": 0.9680729508399963,
+      "learning_rate": 0.0002,
+      "loss": 0.4537,
+      "step": 540
+    },
+    {
+      "epoch": 2.1442495126705654,
+      "grad_norm": 0.67018723487854,
+      "learning_rate": 0.0002,
+      "loss": 0.4278,
+      "step": 550
+    },
+    {
+      "epoch": 2.1832358674463936,
+      "grad_norm": 0.7760737538337708,
+      "learning_rate": 0.0002,
+      "loss": 0.4424,
+      "step": 560
+    },
+    {
+      "epoch": 2.2222222222222223,
+      "grad_norm": 1.9475411176681519,
+      "learning_rate": 0.0002,
+      "loss": 0.4554,
+      "step": 570
+    },
+    {
+      "epoch": 2.2612085769980506,
+      "grad_norm": 0.810126006603241,
+      "learning_rate": 0.0002,
+      "loss": 0.4321,
+      "step": 580
+    },
+    {
+      "epoch": 2.3001949317738792,
+      "grad_norm": 0.8697490692138672,
+      "learning_rate": 0.0002,
+      "loss": 0.4413,
+      "step": 590
+    },
+    {
+      "epoch": 2.3391812865497075,
+      "grad_norm": 0.5910632014274597,
+      "learning_rate": 0.0002,
+      "loss": 0.445,
+      "step": 600
+    },
+    {
+      "epoch": 2.378167641325536,
+      "grad_norm": 1.0113483667373657,
+      "learning_rate": 0.0002,
+      "loss": 0.43,
+      "step": 610
+    },
+    {
+      "epoch": 2.4171539961013644,
+      "grad_norm": 0.5548872351646423,
+      "learning_rate": 0.0002,
+      "loss": 0.4491,
+      "step": 620
+    },
+    {
+      "epoch": 2.456140350877193,
+      "grad_norm": 0.7542755603790283,
+      "learning_rate": 0.0002,
+      "loss": 0.4654,
+      "step": 630
+    },
+    {
+      "epoch": 2.4951267056530213,
+      "grad_norm": 0.6450021862983704,
+      "learning_rate": 0.0002,
+      "loss": 0.4269,
+      "step": 640
+    },
+    {
+      "epoch": 2.53411306042885,
+      "grad_norm": 0.8181225657463074,
+      "learning_rate": 0.0002,
+      "loss": 0.4321,
+      "step": 650
+    },
+    {
+      "epoch": 2.573099415204678,
+      "grad_norm": 0.8882361650466919,
+      "learning_rate": 0.0002,
+      "loss": 0.4476,
+      "step": 660
+    },
+    {
+      "epoch": 2.612085769980507,
+      "grad_norm": 0.9545704126358032,
+      "learning_rate": 0.0002,
+      "loss": 0.4398,
+      "step": 670
+    },
+    {
+      "epoch": 2.651072124756335,
+      "grad_norm": 0.633364200592041,
+      "learning_rate": 0.0002,
+      "loss": 0.4302,
+      "step": 680
+    },
+    {
+      "epoch": 2.690058479532164,
+      "grad_norm": 0.6909534931182861,
+      "learning_rate": 0.0002,
+      "loss": 0.4265,
+      "step": 690
+    },
+    {
+      "epoch": 2.729044834307992,
+      "grad_norm": 0.7342118620872498,
+      "learning_rate": 0.0002,
+      "loss": 0.4508,
+      "step": 700
+    },
+    {
+      "epoch": 2.7680311890838207,
+      "grad_norm": 0.6567948460578918,
+      "learning_rate": 0.0002,
+      "loss": 0.4495,
+      "step": 710
+    },
+    {
+      "epoch": 2.807017543859649,
+      "grad_norm": 0.81608647108078,
+      "learning_rate": 0.0002,
+      "loss": 0.4521,
+      "step": 720
+    },
+    {
+      "epoch": 2.8460038986354776,
+      "grad_norm": 0.7075870633125305,
+      "learning_rate": 0.0002,
+      "loss": 0.4584,
+      "step": 730
+    },
+    {
+      "epoch": 2.884990253411306,
+      "grad_norm": 0.5666054487228394,
+      "learning_rate": 0.0002,
+      "loss": 0.4332,
+      "step": 740
+    },
+    {
+      "epoch": 2.9239766081871346,
+      "grad_norm": 0.5457495450973511,
+      "learning_rate": 0.0002,
+      "loss": 0.423,
+      "step": 750
+    },
+    {
+      "epoch": 2.962962962962963,
+      "grad_norm": 0.8959799408912659,
+      "learning_rate": 0.0002,
+      "loss": 0.4136,
+      "step": 760
+    },
+    {
+      "epoch": 2.9980506822612085,
+      "eval_loss": 0.49315401911735535,
+      "eval_runtime": 67.9039,
+      "eval_samples_per_second": 5.375,
+      "eval_steps_per_second": 0.677,
+      "step": 769
+    },
+    {
+      "epoch": 3.0019493177387915,
+      "grad_norm": 0.8162972927093506,
+      "learning_rate": 0.0002,
+      "loss": 0.4332,
+      "step": 770
+    },
+    {
+      "epoch": 3.0409356725146197,
+      "grad_norm": 0.8814733028411865,
+      "learning_rate": 0.0002,
+      "loss": 0.3778,
+      "step": 780
+    },
+    {
+      "epoch": 3.0799220272904484,
+      "grad_norm": 0.6659106612205505,
+      "learning_rate": 0.0002,
+      "loss": 0.3775,
+      "step": 790
+    },
+    {
+      "epoch": 3.1189083820662766,
+      "grad_norm": 0.7013096213340759,
+      "learning_rate": 0.0002,
+      "loss": 0.3718,
+      "step": 800
+    },
+    {
+      "epoch": 3.1578947368421053,
+      "grad_norm": 0.5946340560913086,
+      "learning_rate": 0.0002,
+      "loss": 0.3799,
+      "step": 810
+    },
+    {
+      "epoch": 3.1968810916179335,
+      "grad_norm": 1.1263338327407837,
+      "learning_rate": 0.0002,
+      "loss": 0.3752,
+      "step": 820
+    },
+    {
+      "epoch": 3.2358674463937622,
+      "grad_norm": 1.028369426727295,
+      "learning_rate": 0.0002,
+      "loss": 0.3819,
+      "step": 830
+    },
+    {
+      "epoch": 3.2748538011695905,
+      "grad_norm": 1.1113781929016113,
+      "learning_rate": 0.0002,
+      "loss": 0.3816,
+      "step": 840
+    },
+    {
+      "epoch": 3.313840155945419,
+      "grad_norm": 0.7735888361930847,
+      "learning_rate": 0.0002,
+      "loss": 0.383,
+      "step": 850
+    },
+    {
+      "epoch": 3.3528265107212474,
+      "grad_norm": 0.8967649936676025,
+      "learning_rate": 0.0002,
+      "loss": 0.3703,
+      "step": 860
+    },
+    {
+      "epoch": 3.391812865497076,
+      "grad_norm": 0.7354698777198792,
+      "learning_rate": 0.0002,
+      "loss": 0.3884,
+      "step": 870
+    },
+    {
+      "epoch": 3.4307992202729043,
+      "grad_norm": 0.8721298575401306,
+      "learning_rate": 0.0002,
+      "loss": 0.3762,
+      "step": 880
+    },
+    {
+      "epoch": 3.469785575048733,
+      "grad_norm": 0.7276861667633057,
+      "learning_rate": 0.0002,
+      "loss": 0.3758,
+      "step": 890
+    },
+    {
+      "epoch": 3.5087719298245617,
+      "grad_norm": 0.8296075463294983,
+      "learning_rate": 0.0002,
+      "loss": 0.3866,
+      "step": 900
+    },
+    {
+      "epoch": 3.54775828460039,
+      "grad_norm": 0.8955655097961426,
+      "learning_rate": 0.0002,
+      "loss": 0.3909,
+      "step": 910
+    },
+    {
+      "epoch": 3.586744639376218,
+      "grad_norm": 0.8008049130439758,
+      "learning_rate": 0.0002,
+      "loss": 0.3882,
+      "step": 920
+    },
+    {
+      "epoch": 3.625730994152047,
+      "grad_norm": 0.7791480422019958,
+      "learning_rate": 0.0002,
+      "loss": 0.3837,
+      "step": 930
+    },
+    {
+      "epoch": 3.6647173489278755,
+      "grad_norm": 0.8780500292778015,
+      "learning_rate": 0.0002,
+      "loss": 0.3877,
+      "step": 940
+    },
+    {
+      "epoch": 3.7037037037037037,
+      "grad_norm": 0.8411193490028381,
+      "learning_rate": 0.0002,
+      "loss": 0.3852,
+      "step": 950
+    },
+    {
+      "epoch": 3.742690058479532,
+      "grad_norm": 0.7564510703086853,
+      "learning_rate": 0.0002,
+      "loss": 0.3789,
+      "step": 960
+    },
+    {
+      "epoch": 3.7816764132553606,
+      "grad_norm": 0.9967873692512512,
+      "learning_rate": 0.0002,
+      "loss": 0.3854,
+      "step": 970
+    },
+    {
+      "epoch": 3.8206627680311893,
+      "grad_norm": 1.0118954181671143,
+      "learning_rate": 0.0002,
+      "loss": 0.3664,
+      "step": 980
+    },
+    {
+      "epoch": 3.8596491228070176,
+      "grad_norm": 0.7355602383613586,
+      "learning_rate": 0.0002,
+      "loss": 0.367,
+      "step": 990
+    },
+    {
+      "epoch": 3.898635477582846,
+      "grad_norm": 0.7059919834136963,
+      "learning_rate": 0.0002,
+      "loss": 0.3802,
+      "step": 1000
+    },
+    {
+      "epoch": 3.9376218323586745,
+      "grad_norm": 0.6720611453056335,
+      "learning_rate": 0.0002,
+      "loss": 0.3907,
+      "step": 1010
+    },
+    {
+      "epoch": 3.976608187134503,
+      "grad_norm": 0.664450466632843,
+      "learning_rate": 0.0002,
+      "loss": 0.392,
+      "step": 1020
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 0.4771515429019928,
+      "eval_runtime": 68.2605,
+      "eval_samples_per_second": 5.347,
+      "eval_steps_per_second": 0.674,
+      "step": 1026
+    },
+    {
+      "epoch": 4.015594541910332,
+      "grad_norm": 0.8478823304176331,
+      "learning_rate": 0.0002,
+      "loss": 0.3554,
+      "step": 1030
+    },
+    {
+      "epoch": 4.05458089668616,
+      "grad_norm": 0.7147605419158936,
+      "learning_rate": 0.0002,
+      "loss": 0.3353,
+      "step": 1040
+    },
+    {
+      "epoch": 4.093567251461988,
+      "grad_norm": 0.8668870329856873,
+      "learning_rate": 0.0002,
+      "loss": 0.3318,
+      "step": 1050
+    },
+    {
+      "epoch": 4.132553606237817,
+      "grad_norm": 0.9445207715034485,
+      "learning_rate": 0.0002,
+      "loss": 0.3284,
+      "step": 1060
+    },
+    {
+      "epoch": 4.171539961013645,
+      "grad_norm": 0.7062109112739563,
+      "learning_rate": 0.0002,
+      "loss": 0.3453,
+      "step": 1070
+    },
+    {
+      "epoch": 4.2105263157894735,
+      "grad_norm": 0.8336073160171509,
+      "learning_rate": 0.0002,
+      "loss": 0.3345,
+      "step": 1080
+    },
+    {
+      "epoch": 4.249512670565302,
+      "grad_norm": 1.3078413009643555,
+      "learning_rate": 0.0002,
+      "loss": 0.3435,
+      "step": 1090
+    },
+    {
+      "epoch": 4.288499025341131,
+      "grad_norm": 0.8631044030189514,
+      "learning_rate": 0.0002,
+      "loss": 0.3443,
+      "step": 1100
+    },
+    {
+      "epoch": 4.3274853801169595,
+      "grad_norm": 0.7545914053916931,
+      "learning_rate": 0.0002,
+      "loss": 0.3269,
+      "step": 1110
+    },
+    {
+      "epoch": 4.366471734892787,
+      "grad_norm": 0.7546519637107849,
+      "learning_rate": 0.0002,
+      "loss": 0.3504,
+      "step": 1120
+    },
+    {
+      "epoch": 4.405458089668616,
+      "grad_norm": 0.8977745175361633,
+      "learning_rate": 0.0002,
+      "loss": 0.3419,
+      "step": 1130
+    },
+    {
+      "epoch": 4.444444444444445,
+      "grad_norm": 1.1983102560043335,
+      "learning_rate": 0.0002,
+      "loss": 0.3472,
+      "step": 1140
+    },
+    {
+      "epoch": 4.483430799220272,
+      "grad_norm": 0.9402135610580444,
+      "learning_rate": 0.0002,
+      "loss": 0.3409,
+      "step": 1150
+    },
+    {
+      "epoch": 4.522417153996101,
+      "grad_norm": 0.7405855655670166,
+      "learning_rate": 0.0002,
+      "loss": 0.3372,
+      "step": 1160
+    },
+    {
+      "epoch": 4.56140350877193,
+      "grad_norm": 0.6910150647163391,
+      "learning_rate": 0.0002,
+      "loss": 0.344,
+      "step": 1170
+    },
+    {
+      "epoch": 4.6003898635477585,
+      "grad_norm": 0.8426865339279175,
+      "learning_rate": 0.0002,
+      "loss": 0.3402,
+      "step": 1180
+    },
+    {
+      "epoch": 4.639376218323587,
+      "grad_norm": 0.6767265200614929,
+      "learning_rate": 0.0002,
+      "loss": 0.3361,
+      "step": 1190
+    },
+    {
+      "epoch": 4.678362573099415,
+      "grad_norm": 0.7790605425834656,
+      "learning_rate": 0.0002,
+      "loss": 0.3417,
+      "step": 1200
+    },
+    {
+      "epoch": 4.717348927875244,
+      "grad_norm": 0.7993232011795044,
+      "learning_rate": 0.0002,
+      "loss": 0.3513,
+      "step": 1210
+    },
+    {
+      "epoch": 4.756335282651072,
+      "grad_norm": 0.6861053705215454,
+      "learning_rate": 0.0002,
+      "loss": 0.346,
+      "step": 1220
+    },
+    {
+      "epoch": 4.7953216374269,
+      "grad_norm": 0.8599495887756348,
+      "learning_rate": 0.0002,
+      "loss": 0.3426,
+      "step": 1230
+    },
+    {
+      "epoch": 4.834307992202729,
+      "grad_norm": 0.8054423332214355,
+      "learning_rate": 0.0002,
+      "loss": 0.3622,
+      "step": 1240
+    },
+    {
+      "epoch": 4.8732943469785575,
+      "grad_norm": 0.949587881565094,
+      "learning_rate": 0.0002,
+      "loss": 0.343,
+      "step": 1250
+    },
+    {
+      "epoch": 4.912280701754386,
+      "grad_norm": 0.6002761125564575,
+      "learning_rate": 0.0002,
+      "loss": 0.3364,
+      "step": 1260
+    },
+    {
+      "epoch": 4.951267056530215,
+      "grad_norm": 0.7506934404373169,
+      "learning_rate": 0.0002,
+      "loss": 0.3417,
+      "step": 1270
+    },
+    {
+      "epoch": 4.990253411306043,
+      "grad_norm": 0.7621583938598633,
+      "learning_rate": 0.0002,
+      "loss": 0.3379,
+      "step": 1280
+    },
+    {
+      "epoch": 4.9980506822612085,
+      "eval_loss": 0.4660908877849579,
+      "eval_runtime": 67.4734,
+      "eval_samples_per_second": 5.41,
+      "eval_steps_per_second": 0.682,
+      "step": 1282
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 2048,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.6269337001984e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1282/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5036267d1db24ac0a5932b39180f94ba0ea9b0dc9da633253ea9585ce5cdb211
+size 5560

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1539/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1539/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1539/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:154bcedca29503345eb5d2d93d525a97373c13b0b7aba28564244d640f0441e6
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1539/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fee9b9f0105f85f3033570a6a522c609ec38eeb84e347c47a41e1bf232257b73
+size 55532666

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1539/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:48aef2dd101d3a7b8ca0e42ead92d01f9f6c9277ada1cc76bd04af8293dc16fe
+size 14244

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1539/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:59f6669c0f0c03f385f5dead19a64e8e7f53ec629d66d8f0fa44f1da6682801b
+size 1064

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1539/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1539/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1539/trainer_state.json

@@ -0,0 +1,1152 @@
+{
+  "best_metric": 0.4633646011352539,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1539",
+  "epoch": 6.0,
+  "eval_steps": 10,
+  "global_step": 1539,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.03898635477582846,
+      "grad_norm": 1.1630783081054688,
+      "learning_rate": 0.0002,
+      "loss": 1.7486,
+      "step": 10
+    },
+    {
+      "epoch": 0.07797270955165692,
+      "grad_norm": 1.3587305545806885,
+      "learning_rate": 0.0002,
+      "loss": 1.1818,
+      "step": 20
+    },
+    {
+      "epoch": 0.11695906432748537,
+      "grad_norm": 0.9663119912147522,
+      "learning_rate": 0.0002,
+      "loss": 1.0017,
+      "step": 30
+    },
+    {
+      "epoch": 0.15594541910331383,
+      "grad_norm": 1.0442273616790771,
+      "learning_rate": 0.0002,
+      "loss": 0.945,
+      "step": 40
+    },
+    {
+      "epoch": 0.1949317738791423,
+      "grad_norm": 0.9459889531135559,
+      "learning_rate": 0.0002,
+      "loss": 0.7863,
+      "step": 50
+    },
+    {
+      "epoch": 0.23391812865497075,
+      "grad_norm": 0.9623780846595764,
+      "learning_rate": 0.0002,
+      "loss": 0.7287,
+      "step": 60
+    },
+    {
+      "epoch": 0.2729044834307992,
+      "grad_norm": 0.8872420787811279,
+      "learning_rate": 0.0002,
+      "loss": 0.6969,
+      "step": 70
+    },
+    {
+      "epoch": 0.31189083820662766,
+      "grad_norm": 0.7173621654510498,
+      "learning_rate": 0.0002,
+      "loss": 0.6828,
+      "step": 80
+    },
+    {
+      "epoch": 0.3508771929824561,
+      "grad_norm": 0.7221336364746094,
+      "learning_rate": 0.0002,
+      "loss": 0.6651,
+      "step": 90
+    },
+    {
+      "epoch": 0.3898635477582846,
+      "grad_norm": 0.7142595648765564,
+      "learning_rate": 0.0002,
+      "loss": 0.621,
+      "step": 100
+    },
+    {
+      "epoch": 0.42884990253411304,
+      "grad_norm": 0.6138886213302612,
+      "learning_rate": 0.0002,
+      "loss": 0.6601,
+      "step": 110
+    },
+    {
+      "epoch": 0.4678362573099415,
+      "grad_norm": 0.8129922151565552,
+      "learning_rate": 0.0002,
+      "loss": 0.6182,
+      "step": 120
+    },
+    {
+      "epoch": 0.50682261208577,
+      "grad_norm": 0.8280786275863647,
+      "learning_rate": 0.0002,
+      "loss": 0.6426,
+      "step": 130
+    },
+    {
+      "epoch": 0.5458089668615984,
+      "grad_norm": 0.5899930596351624,
+      "learning_rate": 0.0002,
+      "loss": 0.6125,
+      "step": 140
+    },
+    {
+      "epoch": 0.5847953216374269,
+      "grad_norm": 0.7101946473121643,
+      "learning_rate": 0.0002,
+      "loss": 0.6444,
+      "step": 150
+    },
+    {
+      "epoch": 0.6237816764132553,
+      "grad_norm": 0.6261710524559021,
+      "learning_rate": 0.0002,
+      "loss": 0.6374,
+      "step": 160
+    },
+    {
+      "epoch": 0.6627680311890838,
+      "grad_norm": 0.7684038877487183,
+      "learning_rate": 0.0002,
+      "loss": 0.594,
+      "step": 170
+    },
+    {
+      "epoch": 0.7017543859649122,
+      "grad_norm": 0.581214189529419,
+      "learning_rate": 0.0002,
+      "loss": 0.5992,
+      "step": 180
+    },
+    {
+      "epoch": 0.7407407407407407,
+      "grad_norm": 0.5658173561096191,
+      "learning_rate": 0.0002,
+      "loss": 0.6179,
+      "step": 190
+    },
+    {
+      "epoch": 0.7797270955165692,
+      "grad_norm": 0.7603134512901306,
+      "learning_rate": 0.0002,
+      "loss": 0.6417,
+      "step": 200
+    },
+    {
+      "epoch": 0.8187134502923976,
+      "grad_norm": 0.5359350442886353,
+      "learning_rate": 0.0002,
+      "loss": 0.5906,
+      "step": 210
+    },
+    {
+      "epoch": 0.8576998050682261,
+      "grad_norm": 0.7978444695472717,
+      "learning_rate": 0.0002,
+      "loss": 0.6125,
+      "step": 220
+    },
+    {
+      "epoch": 0.8966861598440545,
+      "grad_norm": 0.5791468620300293,
+      "learning_rate": 0.0002,
+      "loss": 0.5984,
+      "step": 230
+    },
+    {
+      "epoch": 0.935672514619883,
+      "grad_norm": 0.635198712348938,
+      "learning_rate": 0.0002,
+      "loss": 0.6007,
+      "step": 240
+    },
+    {
+      "epoch": 0.9746588693957114,
+      "grad_norm": 0.5510166883468628,
+      "learning_rate": 0.0002,
+      "loss": 0.5912,
+      "step": 250
+    },
+    {
+      "epoch": 0.9980506822612085,
+      "eval_loss": 0.5850377678871155,
+      "eval_runtime": 72.8118,
+      "eval_samples_per_second": 5.013,
+      "eval_steps_per_second": 0.632,
+      "step": 256
+    },
+    {
+      "epoch": 1.01364522417154,
+      "grad_norm": 0.5623628497123718,
+      "learning_rate": 0.0002,
+      "loss": 0.575,
+      "step": 260
+    },
+    {
+      "epoch": 1.0526315789473684,
+      "grad_norm": 0.6872021555900574,
+      "learning_rate": 0.0002,
+      "loss": 0.5498,
+      "step": 270
+    },
+    {
+      "epoch": 1.0916179337231968,
+      "grad_norm": 0.5543047785758972,
+      "learning_rate": 0.0002,
+      "loss": 0.5373,
+      "step": 280
+    },
+    {
+      "epoch": 1.1306042884990253,
+      "grad_norm": 0.498542845249176,
+      "learning_rate": 0.0002,
+      "loss": 0.5272,
+      "step": 290
+    },
+    {
+      "epoch": 1.1695906432748537,
+      "grad_norm": 0.5940739512443542,
+      "learning_rate": 0.0002,
+      "loss": 0.5528,
+      "step": 300
+    },
+    {
+      "epoch": 1.2085769980506822,
+      "grad_norm": 0.5854314565658569,
+      "learning_rate": 0.0002,
+      "loss": 0.5308,
+      "step": 310
+    },
+    {
+      "epoch": 1.2475633528265107,
+      "grad_norm": 0.5293670892715454,
+      "learning_rate": 0.0002,
+      "loss": 0.5452,
+      "step": 320
+    },
+    {
+      "epoch": 1.286549707602339,
+      "grad_norm": 0.5736058950424194,
+      "learning_rate": 0.0002,
+      "loss": 0.5451,
+      "step": 330
+    },
+    {
+      "epoch": 1.3255360623781676,
+      "grad_norm": 0.7912607789039612,
+      "learning_rate": 0.0002,
+      "loss": 0.5344,
+      "step": 340
+    },
+    {
+      "epoch": 1.364522417153996,
+      "grad_norm": 0.781843364238739,
+      "learning_rate": 0.0002,
+      "loss": 0.531,
+      "step": 350
+    },
+    {
+      "epoch": 1.4035087719298245,
+      "grad_norm": 0.7365643978118896,
+      "learning_rate": 0.0002,
+      "loss": 0.5396,
+      "step": 360
+    },
+    {
+      "epoch": 1.442495126705653,
+      "grad_norm": 0.5565363764762878,
+      "learning_rate": 0.0002,
+      "loss": 0.5363,
+      "step": 370
+    },
+    {
+      "epoch": 1.4814814814814814,
+      "grad_norm": 0.5427217483520508,
+      "learning_rate": 0.0002,
+      "loss": 0.5235,
+      "step": 380
+    },
+    {
+      "epoch": 1.52046783625731,
+      "grad_norm": 0.7150877118110657,
+      "learning_rate": 0.0002,
+      "loss": 0.5173,
+      "step": 390
+    },
+    {
+      "epoch": 1.5594541910331383,
+      "grad_norm": 0.5850045084953308,
+      "learning_rate": 0.0002,
+      "loss": 0.5394,
+      "step": 400
+    },
+    {
+      "epoch": 1.598440545808967,
+      "grad_norm": 0.5376710295677185,
+      "learning_rate": 0.0002,
+      "loss": 0.5045,
+      "step": 410
+    },
+    {
+      "epoch": 1.6374269005847952,
+      "grad_norm": 0.6234501004219055,
+      "learning_rate": 0.0002,
+      "loss": 0.5444,
+      "step": 420
+    },
+    {
+      "epoch": 1.676413255360624,
+      "grad_norm": 0.7774652242660522,
+      "learning_rate": 0.0002,
+      "loss": 0.5186,
+      "step": 430
+    },
+    {
+      "epoch": 1.7153996101364521,
+      "grad_norm": 0.6821911334991455,
+      "learning_rate": 0.0002,
+      "loss": 0.5168,
+      "step": 440
+    },
+    {
+      "epoch": 1.7543859649122808,
+      "grad_norm": 0.7210469841957092,
+      "learning_rate": 0.0002,
+      "loss": 0.5118,
+      "step": 450
+    },
+    {
+      "epoch": 1.793372319688109,
+      "grad_norm": 0.6935250759124756,
+      "learning_rate": 0.0002,
+      "loss": 0.5499,
+      "step": 460
+    },
+    {
+      "epoch": 1.8323586744639377,
+      "grad_norm": 0.6940227150917053,
+      "learning_rate": 0.0002,
+      "loss": 0.516,
+      "step": 470
+    },
+    {
+      "epoch": 1.871345029239766,
+      "grad_norm": 0.6918772459030151,
+      "learning_rate": 0.0002,
+      "loss": 0.4839,
+      "step": 480
+    },
+    {
+      "epoch": 1.9103313840155947,
+      "grad_norm": 0.5278885364532471,
+      "learning_rate": 0.0002,
+      "loss": 0.4906,
+      "step": 490
+    },
+    {
+      "epoch": 1.949317738791423,
+      "grad_norm": 0.655097246170044,
+      "learning_rate": 0.0002,
+      "loss": 0.4954,
+      "step": 500
+    },
+    {
+      "epoch": 1.9883040935672516,
+      "grad_norm": 0.728690505027771,
+      "learning_rate": 0.0002,
+      "loss": 0.4981,
+      "step": 510
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 0.5312528610229492,
+      "eval_runtime": 67.5284,
+      "eval_samples_per_second": 5.405,
+      "eval_steps_per_second": 0.681,
+      "step": 513
+    },
+    {
+      "epoch": 2.02729044834308,
+      "grad_norm": 0.7993459105491638,
+      "learning_rate": 0.0002,
+      "loss": 0.473,
+      "step": 520
+    },
+    {
+      "epoch": 2.0662768031189085,
+      "grad_norm": 0.6997111439704895,
+      "learning_rate": 0.0002,
+      "loss": 0.4514,
+      "step": 530
+    },
+    {
+      "epoch": 2.1052631578947367,
+      "grad_norm": 0.9680729508399963,
+      "learning_rate": 0.0002,
+      "loss": 0.4537,
+      "step": 540
+    },
+    {
+      "epoch": 2.1442495126705654,
+      "grad_norm": 0.67018723487854,
+      "learning_rate": 0.0002,
+      "loss": 0.4278,
+      "step": 550
+    },
+    {
+      "epoch": 2.1832358674463936,
+      "grad_norm": 0.7760737538337708,
+      "learning_rate": 0.0002,
+      "loss": 0.4424,
+      "step": 560
+    },
+    {
+      "epoch": 2.2222222222222223,
+      "grad_norm": 1.9475411176681519,
+      "learning_rate": 0.0002,
+      "loss": 0.4554,
+      "step": 570
+    },
+    {
+      "epoch": 2.2612085769980506,
+      "grad_norm": 0.810126006603241,
+      "learning_rate": 0.0002,
+      "loss": 0.4321,
+      "step": 580
+    },
+    {
+      "epoch": 2.3001949317738792,
+      "grad_norm": 0.8697490692138672,
+      "learning_rate": 0.0002,
+      "loss": 0.4413,
+      "step": 590
+    },
+    {
+      "epoch": 2.3391812865497075,
+      "grad_norm": 0.5910632014274597,
+      "learning_rate": 0.0002,
+      "loss": 0.445,
+      "step": 600
+    },
+    {
+      "epoch": 2.378167641325536,
+      "grad_norm": 1.0113483667373657,
+      "learning_rate": 0.0002,
+      "loss": 0.43,
+      "step": 610
+    },
+    {
+      "epoch": 2.4171539961013644,
+      "grad_norm": 0.5548872351646423,
+      "learning_rate": 0.0002,
+      "loss": 0.4491,
+      "step": 620
+    },
+    {
+      "epoch": 2.456140350877193,
+      "grad_norm": 0.7542755603790283,
+      "learning_rate": 0.0002,
+      "loss": 0.4654,
+      "step": 630
+    },
+    {
+      "epoch": 2.4951267056530213,
+      "grad_norm": 0.6450021862983704,
+      "learning_rate": 0.0002,
+      "loss": 0.4269,
+      "step": 640
+    },
+    {
+      "epoch": 2.53411306042885,
+      "grad_norm": 0.8181225657463074,
+      "learning_rate": 0.0002,
+      "loss": 0.4321,
+      "step": 650
+    },
+    {
+      "epoch": 2.573099415204678,
+      "grad_norm": 0.8882361650466919,
+      "learning_rate": 0.0002,
+      "loss": 0.4476,
+      "step": 660
+    },
+    {
+      "epoch": 2.612085769980507,
+      "grad_norm": 0.9545704126358032,
+      "learning_rate": 0.0002,
+      "loss": 0.4398,
+      "step": 670
+    },
+    {
+      "epoch": 2.651072124756335,
+      "grad_norm": 0.633364200592041,
+      "learning_rate": 0.0002,
+      "loss": 0.4302,
+      "step": 680
+    },
+    {
+      "epoch": 2.690058479532164,
+      "grad_norm": 0.6909534931182861,
+      "learning_rate": 0.0002,
+      "loss": 0.4265,
+      "step": 690
+    },
+    {
+      "epoch": 2.729044834307992,
+      "grad_norm": 0.7342118620872498,
+      "learning_rate": 0.0002,
+      "loss": 0.4508,
+      "step": 700
+    },
+    {
+      "epoch": 2.7680311890838207,
+      "grad_norm": 0.6567948460578918,
+      "learning_rate": 0.0002,
+      "loss": 0.4495,
+      "step": 710
+    },
+    {
+      "epoch": 2.807017543859649,
+      "grad_norm": 0.81608647108078,
+      "learning_rate": 0.0002,
+      "loss": 0.4521,
+      "step": 720
+    },
+    {
+      "epoch": 2.8460038986354776,
+      "grad_norm": 0.7075870633125305,
+      "learning_rate": 0.0002,
+      "loss": 0.4584,
+      "step": 730
+    },
+    {
+      "epoch": 2.884990253411306,
+      "grad_norm": 0.5666054487228394,
+      "learning_rate": 0.0002,
+      "loss": 0.4332,
+      "step": 740
+    },
+    {
+      "epoch": 2.9239766081871346,
+      "grad_norm": 0.5457495450973511,
+      "learning_rate": 0.0002,
+      "loss": 0.423,
+      "step": 750
+    },
+    {
+      "epoch": 2.962962962962963,
+      "grad_norm": 0.8959799408912659,
+      "learning_rate": 0.0002,
+      "loss": 0.4136,
+      "step": 760
+    },
+    {
+      "epoch": 2.9980506822612085,
+      "eval_loss": 0.49315401911735535,
+      "eval_runtime": 67.9039,
+      "eval_samples_per_second": 5.375,
+      "eval_steps_per_second": 0.677,
+      "step": 769
+    },
+    {
+      "epoch": 3.0019493177387915,
+      "grad_norm": 0.8162972927093506,
+      "learning_rate": 0.0002,
+      "loss": 0.4332,
+      "step": 770
+    },
+    {
+      "epoch": 3.0409356725146197,
+      "grad_norm": 0.8814733028411865,
+      "learning_rate": 0.0002,
+      "loss": 0.3778,
+      "step": 780
+    },
+    {
+      "epoch": 3.0799220272904484,
+      "grad_norm": 0.6659106612205505,
+      "learning_rate": 0.0002,
+      "loss": 0.3775,
+      "step": 790
+    },
+    {
+      "epoch": 3.1189083820662766,
+      "grad_norm": 0.7013096213340759,
+      "learning_rate": 0.0002,
+      "loss": 0.3718,
+      "step": 800
+    },
+    {
+      "epoch": 3.1578947368421053,
+      "grad_norm": 0.5946340560913086,
+      "learning_rate": 0.0002,
+      "loss": 0.3799,
+      "step": 810
+    },
+    {
+      "epoch": 3.1968810916179335,
+      "grad_norm": 1.1263338327407837,
+      "learning_rate": 0.0002,
+      "loss": 0.3752,
+      "step": 820
+    },
+    {
+      "epoch": 3.2358674463937622,
+      "grad_norm": 1.028369426727295,
+      "learning_rate": 0.0002,
+      "loss": 0.3819,
+      "step": 830
+    },
+    {
+      "epoch": 3.2748538011695905,
+      "grad_norm": 1.1113781929016113,
+      "learning_rate": 0.0002,
+      "loss": 0.3816,
+      "step": 840
+    },
+    {
+      "epoch": 3.313840155945419,
+      "grad_norm": 0.7735888361930847,
+      "learning_rate": 0.0002,
+      "loss": 0.383,
+      "step": 850
+    },
+    {
+      "epoch": 3.3528265107212474,
+      "grad_norm": 0.8967649936676025,
+      "learning_rate": 0.0002,
+      "loss": 0.3703,
+      "step": 860
+    },
+    {
+      "epoch": 3.391812865497076,
+      "grad_norm": 0.7354698777198792,
+      "learning_rate": 0.0002,
+      "loss": 0.3884,
+      "step": 870
+    },
+    {
+      "epoch": 3.4307992202729043,
+      "grad_norm": 0.8721298575401306,
+      "learning_rate": 0.0002,
+      "loss": 0.3762,
+      "step": 880
+    },
+    {
+      "epoch": 3.469785575048733,
+      "grad_norm": 0.7276861667633057,
+      "learning_rate": 0.0002,
+      "loss": 0.3758,
+      "step": 890
+    },
+    {
+      "epoch": 3.5087719298245617,
+      "grad_norm": 0.8296075463294983,
+      "learning_rate": 0.0002,
+      "loss": 0.3866,
+      "step": 900
+    },
+    {
+      "epoch": 3.54775828460039,
+      "grad_norm": 0.8955655097961426,
+      "learning_rate": 0.0002,
+      "loss": 0.3909,
+      "step": 910
+    },
+    {
+      "epoch": 3.586744639376218,
+      "grad_norm": 0.8008049130439758,
+      "learning_rate": 0.0002,
+      "loss": 0.3882,
+      "step": 920
+    },
+    {
+      "epoch": 3.625730994152047,
+      "grad_norm": 0.7791480422019958,
+      "learning_rate": 0.0002,
+      "loss": 0.3837,
+      "step": 930
+    },
+    {
+      "epoch": 3.6647173489278755,
+      "grad_norm": 0.8780500292778015,
+      "learning_rate": 0.0002,
+      "loss": 0.3877,
+      "step": 940
+    },
+    {
+      "epoch": 3.7037037037037037,
+      "grad_norm": 0.8411193490028381,
+      "learning_rate": 0.0002,
+      "loss": 0.3852,
+      "step": 950
+    },
+    {
+      "epoch": 3.742690058479532,
+      "grad_norm": 0.7564510703086853,
+      "learning_rate": 0.0002,
+      "loss": 0.3789,
+      "step": 960
+    },
+    {
+      "epoch": 3.7816764132553606,
+      "grad_norm": 0.9967873692512512,
+      "learning_rate": 0.0002,
+      "loss": 0.3854,
+      "step": 970
+    },
+    {
+      "epoch": 3.8206627680311893,
+      "grad_norm": 1.0118954181671143,
+      "learning_rate": 0.0002,
+      "loss": 0.3664,
+      "step": 980
+    },
+    {
+      "epoch": 3.8596491228070176,
+      "grad_norm": 0.7355602383613586,
+      "learning_rate": 0.0002,
+      "loss": 0.367,
+      "step": 990
+    },
+    {
+      "epoch": 3.898635477582846,
+      "grad_norm": 0.7059919834136963,
+      "learning_rate": 0.0002,
+      "loss": 0.3802,
+      "step": 1000
+    },
+    {
+      "epoch": 3.9376218323586745,
+      "grad_norm": 0.6720611453056335,
+      "learning_rate": 0.0002,
+      "loss": 0.3907,
+      "step": 1010
+    },
+    {
+      "epoch": 3.976608187134503,
+      "grad_norm": 0.664450466632843,
+      "learning_rate": 0.0002,
+      "loss": 0.392,
+      "step": 1020
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 0.4771515429019928,
+      "eval_runtime": 68.2605,
+      "eval_samples_per_second": 5.347,
+      "eval_steps_per_second": 0.674,
+      "step": 1026
+    },
+    {
+      "epoch": 4.015594541910332,
+      "grad_norm": 0.8478823304176331,
+      "learning_rate": 0.0002,
+      "loss": 0.3554,
+      "step": 1030
+    },
+    {
+      "epoch": 4.05458089668616,
+      "grad_norm": 0.7147605419158936,
+      "learning_rate": 0.0002,
+      "loss": 0.3353,
+      "step": 1040
+    },
+    {
+      "epoch": 4.093567251461988,
+      "grad_norm": 0.8668870329856873,
+      "learning_rate": 0.0002,
+      "loss": 0.3318,
+      "step": 1050
+    },
+    {
+      "epoch": 4.132553606237817,
+      "grad_norm": 0.9445207715034485,
+      "learning_rate": 0.0002,
+      "loss": 0.3284,
+      "step": 1060
+    },
+    {
+      "epoch": 4.171539961013645,
+      "grad_norm": 0.7062109112739563,
+      "learning_rate": 0.0002,
+      "loss": 0.3453,
+      "step": 1070
+    },
+    {
+      "epoch": 4.2105263157894735,
+      "grad_norm": 0.8336073160171509,
+      "learning_rate": 0.0002,
+      "loss": 0.3345,
+      "step": 1080
+    },
+    {
+      "epoch": 4.249512670565302,
+      "grad_norm": 1.3078413009643555,
+      "learning_rate": 0.0002,
+      "loss": 0.3435,
+      "step": 1090
+    },
+    {
+      "epoch": 4.288499025341131,
+      "grad_norm": 0.8631044030189514,
+      "learning_rate": 0.0002,
+      "loss": 0.3443,
+      "step": 1100
+    },
+    {
+      "epoch": 4.3274853801169595,
+      "grad_norm": 0.7545914053916931,
+      "learning_rate": 0.0002,
+      "loss": 0.3269,
+      "step": 1110
+    },
+    {
+      "epoch": 4.366471734892787,
+      "grad_norm": 0.7546519637107849,
+      "learning_rate": 0.0002,
+      "loss": 0.3504,
+      "step": 1120
+    },
+    {
+      "epoch": 4.405458089668616,
+      "grad_norm": 0.8977745175361633,
+      "learning_rate": 0.0002,
+      "loss": 0.3419,
+      "step": 1130
+    },
+    {
+      "epoch": 4.444444444444445,
+      "grad_norm": 1.1983102560043335,
+      "learning_rate": 0.0002,
+      "loss": 0.3472,
+      "step": 1140
+    },
+    {
+      "epoch": 4.483430799220272,
+      "grad_norm": 0.9402135610580444,
+      "learning_rate": 0.0002,
+      "loss": 0.3409,
+      "step": 1150
+    },
+    {
+      "epoch": 4.522417153996101,
+      "grad_norm": 0.7405855655670166,
+      "learning_rate": 0.0002,
+      "loss": 0.3372,
+      "step": 1160
+    },
+    {
+      "epoch": 4.56140350877193,
+      "grad_norm": 0.6910150647163391,
+      "learning_rate": 0.0002,
+      "loss": 0.344,
+      "step": 1170
+    },
+    {
+      "epoch": 4.6003898635477585,
+      "grad_norm": 0.8426865339279175,
+      "learning_rate": 0.0002,
+      "loss": 0.3402,
+      "step": 1180
+    },
+    {
+      "epoch": 4.639376218323587,
+      "grad_norm": 0.6767265200614929,
+      "learning_rate": 0.0002,
+      "loss": 0.3361,
+      "step": 1190
+    },
+    {
+      "epoch": 4.678362573099415,
+      "grad_norm": 0.7790605425834656,
+      "learning_rate": 0.0002,
+      "loss": 0.3417,
+      "step": 1200
+    },
+    {
+      "epoch": 4.717348927875244,
+      "grad_norm": 0.7993232011795044,
+      "learning_rate": 0.0002,
+      "loss": 0.3513,
+      "step": 1210
+    },
+    {
+      "epoch": 4.756335282651072,
+      "grad_norm": 0.6861053705215454,
+      "learning_rate": 0.0002,
+      "loss": 0.346,
+      "step": 1220
+    },
+    {
+      "epoch": 4.7953216374269,
+      "grad_norm": 0.8599495887756348,
+      "learning_rate": 0.0002,
+      "loss": 0.3426,
+      "step": 1230
+    },
+    {
+      "epoch": 4.834307992202729,
+      "grad_norm": 0.8054423332214355,
+      "learning_rate": 0.0002,
+      "loss": 0.3622,
+      "step": 1240
+    },
+    {
+      "epoch": 4.8732943469785575,
+      "grad_norm": 0.949587881565094,
+      "learning_rate": 0.0002,
+      "loss": 0.343,
+      "step": 1250
+    },
+    {
+      "epoch": 4.912280701754386,
+      "grad_norm": 0.6002761125564575,
+      "learning_rate": 0.0002,
+      "loss": 0.3364,
+      "step": 1260
+    },
+    {
+      "epoch": 4.951267056530215,
+      "grad_norm": 0.7506934404373169,
+      "learning_rate": 0.0002,
+      "loss": 0.3417,
+      "step": 1270
+    },
+    {
+      "epoch": 4.990253411306043,
+      "grad_norm": 0.7621583938598633,
+      "learning_rate": 0.0002,
+      "loss": 0.3379,
+      "step": 1280
+    },
+    {
+      "epoch": 4.9980506822612085,
+      "eval_loss": 0.4660908877849579,
+      "eval_runtime": 67.4734,
+      "eval_samples_per_second": 5.41,
+      "eval_steps_per_second": 0.682,
+      "step": 1282
+    },
+    {
+      "epoch": 5.029239766081871,
+      "grad_norm": 0.6844434142112732,
+      "learning_rate": 0.0002,
+      "loss": 0.3272,
+      "step": 1290
+    },
+    {
+      "epoch": 5.0682261208577,
+      "grad_norm": 0.8217318058013916,
+      "learning_rate": 0.0002,
+      "loss": 0.3092,
+      "step": 1300
+    },
+    {
+      "epoch": 5.107212475633529,
+      "grad_norm": 1.0060460567474365,
+      "learning_rate": 0.0002,
+      "loss": 0.3131,
+      "step": 1310
+    },
+    {
+      "epoch": 5.146198830409356,
+      "grad_norm": 0.7044107913970947,
+      "learning_rate": 0.0002,
+      "loss": 0.31,
+      "step": 1320
+    },
+    {
+      "epoch": 5.185185185185185,
+      "grad_norm": 0.9584488868713379,
+      "learning_rate": 0.0002,
+      "loss": 0.3179,
+      "step": 1330
+    },
+    {
+      "epoch": 5.224171539961014,
+      "grad_norm": 0.9667648077011108,
+      "learning_rate": 0.0002,
+      "loss": 0.3138,
+      "step": 1340
+    },
+    {
+      "epoch": 5.2631578947368425,
+      "grad_norm": 0.938228189945221,
+      "learning_rate": 0.0002,
+      "loss": 0.3076,
+      "step": 1350
+    },
+    {
+      "epoch": 5.30214424951267,
+      "grad_norm": 0.7234814167022705,
+      "learning_rate": 0.0002,
+      "loss": 0.2999,
+      "step": 1360
+    },
+    {
+      "epoch": 5.341130604288499,
+      "grad_norm": 0.9198869466781616,
+      "learning_rate": 0.0002,
+      "loss": 0.3112,
+      "step": 1370
+    },
+    {
+      "epoch": 5.380116959064328,
+      "grad_norm": 0.6611506342887878,
+      "learning_rate": 0.0002,
+      "loss": 0.3131,
+      "step": 1380
+    },
+    {
+      "epoch": 5.419103313840156,
+      "grad_norm": 0.7446638345718384,
+      "learning_rate": 0.0002,
+      "loss": 0.3133,
+      "step": 1390
+    },
+    {
+      "epoch": 5.458089668615984,
+      "grad_norm": 1.050093412399292,
+      "learning_rate": 0.0002,
+      "loss": 0.3213,
+      "step": 1400
+    },
+    {
+      "epoch": 5.497076023391813,
+      "grad_norm": 0.7504354119300842,
+      "learning_rate": 0.0002,
+      "loss": 0.3156,
+      "step": 1410
+    },
+    {
+      "epoch": 5.5360623781676415,
+      "grad_norm": 0.6047531962394714,
+      "learning_rate": 0.0002,
+      "loss": 0.3172,
+      "step": 1420
+    },
+    {
+      "epoch": 5.57504873294347,
+      "grad_norm": 0.835421085357666,
+      "learning_rate": 0.0002,
+      "loss": 0.3118,
+      "step": 1430
+    },
+    {
+      "epoch": 5.614035087719298,
+      "grad_norm": 0.8842822313308716,
+      "learning_rate": 0.0002,
+      "loss": 0.3183,
+      "step": 1440
+    },
+    {
+      "epoch": 5.653021442495127,
+      "grad_norm": 0.782640814781189,
+      "learning_rate": 0.0002,
+      "loss": 0.3107,
+      "step": 1450
+    },
+    {
+      "epoch": 5.692007797270955,
+      "grad_norm": 0.7722012996673584,
+      "learning_rate": 0.0002,
+      "loss": 0.3157,
+      "step": 1460
+    },
+    {
+      "epoch": 5.730994152046784,
+      "grad_norm": 0.7458067536354065,
+      "learning_rate": 0.0002,
+      "loss": 0.3168,
+      "step": 1470
+    },
+    {
+      "epoch": 5.769980506822612,
+      "grad_norm": 0.7182063460350037,
+      "learning_rate": 0.0002,
+      "loss": 0.3219,
+      "step": 1480
+    },
+    {
+      "epoch": 5.8089668615984404,
+      "grad_norm": 1.0041741132736206,
+      "learning_rate": 0.0002,
+      "loss": 0.3152,
+      "step": 1490
+    },
+    {
+      "epoch": 5.847953216374269,
+      "grad_norm": 0.894087016582489,
+      "learning_rate": 0.0002,
+      "loss": 0.3148,
+      "step": 1500
+    },
+    {
+      "epoch": 5.886939571150098,
+      "grad_norm": 1.177829384803772,
+      "learning_rate": 0.0002,
+      "loss": 0.3073,
+      "step": 1510
+    },
+    {
+      "epoch": 5.925925925925926,
+      "grad_norm": 1.2548596858978271,
+      "learning_rate": 0.0002,
+      "loss": 0.3088,
+      "step": 1520
+    },
+    {
+      "epoch": 5.964912280701754,
+      "grad_norm": 0.9875953197479248,
+      "learning_rate": 0.0002,
+      "loss": 0.3107,
+      "step": 1530
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 0.4633646011352539,
+      "eval_runtime": 67.64,
+      "eval_samples_per_second": 5.396,
+      "eval_steps_per_second": 0.68,
+      "step": 1539
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 2048,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 6.75232044023808e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1539/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5036267d1db24ac0a5932b39180f94ba0ea9b0dc9da633253ea9585ce5cdb211
+size 5560

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1795/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1795/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1795/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5b7d36cc3fa8251baf43d682f58cb1e30535f2bcec0330202bb0806e5147aff7
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1795/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc6a068cddea287a2c67a7749e2e90466fea0f8c63aff24d1361334216edf97b
+size 55532666

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1795/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e08fa5259b71ec7681cbaddccc5d44eb83bf84113ae14b9b08b3bbba9ea61638
+size 14244

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1795/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc5eddd3ffbc863ad0ba139936cdc56c0591e61c0ab81d7445efbc74ba0c8f5a
+size 1064

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1795/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1795/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1795/trainer_state.json

@@ -0,0 +1,1342 @@
+{
+  "best_metric": 0.4561765789985657,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1795",
+  "epoch": 6.9980506822612085,
+  "eval_steps": 10,
+  "global_step": 1795,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.03898635477582846,
+      "grad_norm": 1.1630783081054688,
+      "learning_rate": 0.0002,
+      "loss": 1.7486,
+      "step": 10
+    },
+    {
+      "epoch": 0.07797270955165692,
+      "grad_norm": 1.3587305545806885,
+      "learning_rate": 0.0002,
+      "loss": 1.1818,
+      "step": 20
+    },
+    {
+      "epoch": 0.11695906432748537,
+      "grad_norm": 0.9663119912147522,
+      "learning_rate": 0.0002,
+      "loss": 1.0017,
+      "step": 30
+    },
+    {
+      "epoch": 0.15594541910331383,
+      "grad_norm": 1.0442273616790771,
+      "learning_rate": 0.0002,
+      "loss": 0.945,
+      "step": 40
+    },
+    {
+      "epoch": 0.1949317738791423,
+      "grad_norm": 0.9459889531135559,
+      "learning_rate": 0.0002,
+      "loss": 0.7863,
+      "step": 50
+    },
+    {
+      "epoch": 0.23391812865497075,
+      "grad_norm": 0.9623780846595764,
+      "learning_rate": 0.0002,
+      "loss": 0.7287,
+      "step": 60
+    },
+    {
+      "epoch": 0.2729044834307992,
+      "grad_norm": 0.8872420787811279,
+      "learning_rate": 0.0002,
+      "loss": 0.6969,
+      "step": 70
+    },
+    {
+      "epoch": 0.31189083820662766,
+      "grad_norm": 0.7173621654510498,
+      "learning_rate": 0.0002,
+      "loss": 0.6828,
+      "step": 80
+    },
+    {
+      "epoch": 0.3508771929824561,
+      "grad_norm": 0.7221336364746094,
+      "learning_rate": 0.0002,
+      "loss": 0.6651,
+      "step": 90
+    },
+    {
+      "epoch": 0.3898635477582846,
+      "grad_norm": 0.7142595648765564,
+      "learning_rate": 0.0002,
+      "loss": 0.621,
+      "step": 100
+    },
+    {
+      "epoch": 0.42884990253411304,
+      "grad_norm": 0.6138886213302612,
+      "learning_rate": 0.0002,
+      "loss": 0.6601,
+      "step": 110
+    },
+    {
+      "epoch": 0.4678362573099415,
+      "grad_norm": 0.8129922151565552,
+      "learning_rate": 0.0002,
+      "loss": 0.6182,
+      "step": 120
+    },
+    {
+      "epoch": 0.50682261208577,
+      "grad_norm": 0.8280786275863647,
+      "learning_rate": 0.0002,
+      "loss": 0.6426,
+      "step": 130
+    },
+    {
+      "epoch": 0.5458089668615984,
+      "grad_norm": 0.5899930596351624,
+      "learning_rate": 0.0002,
+      "loss": 0.6125,
+      "step": 140
+    },
+    {
+      "epoch": 0.5847953216374269,
+      "grad_norm": 0.7101946473121643,
+      "learning_rate": 0.0002,
+      "loss": 0.6444,
+      "step": 150
+    },
+    {
+      "epoch": 0.6237816764132553,
+      "grad_norm": 0.6261710524559021,
+      "learning_rate": 0.0002,
+      "loss": 0.6374,
+      "step": 160
+    },
+    {
+      "epoch": 0.6627680311890838,
+      "grad_norm": 0.7684038877487183,
+      "learning_rate": 0.0002,
+      "loss": 0.594,
+      "step": 170
+    },
+    {
+      "epoch": 0.7017543859649122,
+      "grad_norm": 0.581214189529419,
+      "learning_rate": 0.0002,
+      "loss": 0.5992,
+      "step": 180
+    },
+    {
+      "epoch": 0.7407407407407407,
+      "grad_norm": 0.5658173561096191,
+      "learning_rate": 0.0002,
+      "loss": 0.6179,
+      "step": 190
+    },
+    {
+      "epoch": 0.7797270955165692,
+      "grad_norm": 0.7603134512901306,
+      "learning_rate": 0.0002,
+      "loss": 0.6417,
+      "step": 200
+    },
+    {
+      "epoch": 0.8187134502923976,
+      "grad_norm": 0.5359350442886353,
+      "learning_rate": 0.0002,
+      "loss": 0.5906,
+      "step": 210
+    },
+    {
+      "epoch": 0.8576998050682261,
+      "grad_norm": 0.7978444695472717,
+      "learning_rate": 0.0002,
+      "loss": 0.6125,
+      "step": 220
+    },
+    {
+      "epoch": 0.8966861598440545,
+      "grad_norm": 0.5791468620300293,
+      "learning_rate": 0.0002,
+      "loss": 0.5984,
+      "step": 230
+    },
+    {
+      "epoch": 0.935672514619883,
+      "grad_norm": 0.635198712348938,
+      "learning_rate": 0.0002,
+      "loss": 0.6007,
+      "step": 240
+    },
+    {
+      "epoch": 0.9746588693957114,
+      "grad_norm": 0.5510166883468628,
+      "learning_rate": 0.0002,
+      "loss": 0.5912,
+      "step": 250
+    },
+    {
+      "epoch": 0.9980506822612085,
+      "eval_loss": 0.5850377678871155,
+      "eval_runtime": 72.8118,
+      "eval_samples_per_second": 5.013,
+      "eval_steps_per_second": 0.632,
+      "step": 256
+    },
+    {
+      "epoch": 1.01364522417154,
+      "grad_norm": 0.5623628497123718,
+      "learning_rate": 0.0002,
+      "loss": 0.575,
+      "step": 260
+    },
+    {
+      "epoch": 1.0526315789473684,
+      "grad_norm": 0.6872021555900574,
+      "learning_rate": 0.0002,
+      "loss": 0.5498,
+      "step": 270
+    },
+    {
+      "epoch": 1.0916179337231968,
+      "grad_norm": 0.5543047785758972,
+      "learning_rate": 0.0002,
+      "loss": 0.5373,
+      "step": 280
+    },
+    {
+      "epoch": 1.1306042884990253,
+      "grad_norm": 0.498542845249176,
+      "learning_rate": 0.0002,
+      "loss": 0.5272,
+      "step": 290
+    },
+    {
+      "epoch": 1.1695906432748537,
+      "grad_norm": 0.5940739512443542,
+      "learning_rate": 0.0002,
+      "loss": 0.5528,
+      "step": 300
+    },
+    {
+      "epoch": 1.2085769980506822,
+      "grad_norm": 0.5854314565658569,
+      "learning_rate": 0.0002,
+      "loss": 0.5308,
+      "step": 310
+    },
+    {
+      "epoch": 1.2475633528265107,
+      "grad_norm": 0.5293670892715454,
+      "learning_rate": 0.0002,
+      "loss": 0.5452,
+      "step": 320
+    },
+    {
+      "epoch": 1.286549707602339,
+      "grad_norm": 0.5736058950424194,
+      "learning_rate": 0.0002,
+      "loss": 0.5451,
+      "step": 330
+    },
+    {
+      "epoch": 1.3255360623781676,
+      "grad_norm": 0.7912607789039612,
+      "learning_rate": 0.0002,
+      "loss": 0.5344,
+      "step": 340
+    },
+    {
+      "epoch": 1.364522417153996,
+      "grad_norm": 0.781843364238739,
+      "learning_rate": 0.0002,
+      "loss": 0.531,
+      "step": 350
+    },
+    {
+      "epoch": 1.4035087719298245,
+      "grad_norm": 0.7365643978118896,
+      "learning_rate": 0.0002,
+      "loss": 0.5396,
+      "step": 360
+    },
+    {
+      "epoch": 1.442495126705653,
+      "grad_norm": 0.5565363764762878,
+      "learning_rate": 0.0002,
+      "loss": 0.5363,
+      "step": 370
+    },
+    {
+      "epoch": 1.4814814814814814,
+      "grad_norm": 0.5427217483520508,
+      "learning_rate": 0.0002,
+      "loss": 0.5235,
+      "step": 380
+    },
+    {
+      "epoch": 1.52046783625731,
+      "grad_norm": 0.7150877118110657,
+      "learning_rate": 0.0002,
+      "loss": 0.5173,
+      "step": 390
+    },
+    {
+      "epoch": 1.5594541910331383,
+      "grad_norm": 0.5850045084953308,
+      "learning_rate": 0.0002,
+      "loss": 0.5394,
+      "step": 400
+    },
+    {
+      "epoch": 1.598440545808967,
+      "grad_norm": 0.5376710295677185,
+      "learning_rate": 0.0002,
+      "loss": 0.5045,
+      "step": 410
+    },
+    {
+      "epoch": 1.6374269005847952,
+      "grad_norm": 0.6234501004219055,
+      "learning_rate": 0.0002,
+      "loss": 0.5444,
+      "step": 420
+    },
+    {
+      "epoch": 1.676413255360624,
+      "grad_norm": 0.7774652242660522,
+      "learning_rate": 0.0002,
+      "loss": 0.5186,
+      "step": 430
+    },
+    {
+      "epoch": 1.7153996101364521,
+      "grad_norm": 0.6821911334991455,
+      "learning_rate": 0.0002,
+      "loss": 0.5168,
+      "step": 440
+    },
+    {
+      "epoch": 1.7543859649122808,
+      "grad_norm": 0.7210469841957092,
+      "learning_rate": 0.0002,
+      "loss": 0.5118,
+      "step": 450
+    },
+    {
+      "epoch": 1.793372319688109,
+      "grad_norm": 0.6935250759124756,
+      "learning_rate": 0.0002,
+      "loss": 0.5499,
+      "step": 460
+    },
+    {
+      "epoch": 1.8323586744639377,
+      "grad_norm": 0.6940227150917053,
+      "learning_rate": 0.0002,
+      "loss": 0.516,
+      "step": 470
+    },
+    {
+      "epoch": 1.871345029239766,
+      "grad_norm": 0.6918772459030151,
+      "learning_rate": 0.0002,
+      "loss": 0.4839,
+      "step": 480
+    },
+    {
+      "epoch": 1.9103313840155947,
+      "grad_norm": 0.5278885364532471,
+      "learning_rate": 0.0002,
+      "loss": 0.4906,
+      "step": 490
+    },
+    {
+      "epoch": 1.949317738791423,
+      "grad_norm": 0.655097246170044,
+      "learning_rate": 0.0002,
+      "loss": 0.4954,
+      "step": 500
+    },
+    {
+      "epoch": 1.9883040935672516,
+      "grad_norm": 0.728690505027771,
+      "learning_rate": 0.0002,
+      "loss": 0.4981,
+      "step": 510
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 0.5312528610229492,
+      "eval_runtime": 67.5284,
+      "eval_samples_per_second": 5.405,
+      "eval_steps_per_second": 0.681,
+      "step": 513
+    },
+    {
+      "epoch": 2.02729044834308,
+      "grad_norm": 0.7993459105491638,
+      "learning_rate": 0.0002,
+      "loss": 0.473,
+      "step": 520
+    },
+    {
+      "epoch": 2.0662768031189085,
+      "grad_norm": 0.6997111439704895,
+      "learning_rate": 0.0002,
+      "loss": 0.4514,
+      "step": 530
+    },
+    {
+      "epoch": 2.1052631578947367,
+      "grad_norm": 0.9680729508399963,
+      "learning_rate": 0.0002,
+      "loss": 0.4537,
+      "step": 540
+    },
+    {
+      "epoch": 2.1442495126705654,
+      "grad_norm": 0.67018723487854,
+      "learning_rate": 0.0002,
+      "loss": 0.4278,
+      "step": 550
+    },
+    {
+      "epoch": 2.1832358674463936,
+      "grad_norm": 0.7760737538337708,
+      "learning_rate": 0.0002,
+      "loss": 0.4424,
+      "step": 560
+    },
+    {
+      "epoch": 2.2222222222222223,
+      "grad_norm": 1.9475411176681519,
+      "learning_rate": 0.0002,
+      "loss": 0.4554,
+      "step": 570
+    },
+    {
+      "epoch": 2.2612085769980506,
+      "grad_norm": 0.810126006603241,
+      "learning_rate": 0.0002,
+      "loss": 0.4321,
+      "step": 580
+    },
+    {
+      "epoch": 2.3001949317738792,
+      "grad_norm": 0.8697490692138672,
+      "learning_rate": 0.0002,
+      "loss": 0.4413,
+      "step": 590
+    },
+    {
+      "epoch": 2.3391812865497075,
+      "grad_norm": 0.5910632014274597,
+      "learning_rate": 0.0002,
+      "loss": 0.445,
+      "step": 600
+    },
+    {
+      "epoch": 2.378167641325536,
+      "grad_norm": 1.0113483667373657,
+      "learning_rate": 0.0002,
+      "loss": 0.43,
+      "step": 610
+    },
+    {
+      "epoch": 2.4171539961013644,
+      "grad_norm": 0.5548872351646423,
+      "learning_rate": 0.0002,
+      "loss": 0.4491,
+      "step": 620
+    },
+    {
+      "epoch": 2.456140350877193,
+      "grad_norm": 0.7542755603790283,
+      "learning_rate": 0.0002,
+      "loss": 0.4654,
+      "step": 630
+    },
+    {
+      "epoch": 2.4951267056530213,
+      "grad_norm": 0.6450021862983704,
+      "learning_rate": 0.0002,
+      "loss": 0.4269,
+      "step": 640
+    },
+    {
+      "epoch": 2.53411306042885,
+      "grad_norm": 0.8181225657463074,
+      "learning_rate": 0.0002,
+      "loss": 0.4321,
+      "step": 650
+    },
+    {
+      "epoch": 2.573099415204678,
+      "grad_norm": 0.8882361650466919,
+      "learning_rate": 0.0002,
+      "loss": 0.4476,
+      "step": 660
+    },
+    {
+      "epoch": 2.612085769980507,
+      "grad_norm": 0.9545704126358032,
+      "learning_rate": 0.0002,
+      "loss": 0.4398,
+      "step": 670
+    },
+    {
+      "epoch": 2.651072124756335,
+      "grad_norm": 0.633364200592041,
+      "learning_rate": 0.0002,
+      "loss": 0.4302,
+      "step": 680
+    },
+    {
+      "epoch": 2.690058479532164,
+      "grad_norm": 0.6909534931182861,
+      "learning_rate": 0.0002,
+      "loss": 0.4265,
+      "step": 690
+    },
+    {
+      "epoch": 2.729044834307992,
+      "grad_norm": 0.7342118620872498,
+      "learning_rate": 0.0002,
+      "loss": 0.4508,
+      "step": 700
+    },
+    {
+      "epoch": 2.7680311890838207,
+      "grad_norm": 0.6567948460578918,
+      "learning_rate": 0.0002,
+      "loss": 0.4495,
+      "step": 710
+    },
+    {
+      "epoch": 2.807017543859649,
+      "grad_norm": 0.81608647108078,
+      "learning_rate": 0.0002,
+      "loss": 0.4521,
+      "step": 720
+    },
+    {
+      "epoch": 2.8460038986354776,
+      "grad_norm": 0.7075870633125305,
+      "learning_rate": 0.0002,
+      "loss": 0.4584,
+      "step": 730
+    },
+    {
+      "epoch": 2.884990253411306,
+      "grad_norm": 0.5666054487228394,
+      "learning_rate": 0.0002,
+      "loss": 0.4332,
+      "step": 740
+    },
+    {
+      "epoch": 2.9239766081871346,
+      "grad_norm": 0.5457495450973511,
+      "learning_rate": 0.0002,
+      "loss": 0.423,
+      "step": 750
+    },
+    {
+      "epoch": 2.962962962962963,
+      "grad_norm": 0.8959799408912659,
+      "learning_rate": 0.0002,
+      "loss": 0.4136,
+      "step": 760
+    },
+    {
+      "epoch": 2.9980506822612085,
+      "eval_loss": 0.49315401911735535,
+      "eval_runtime": 67.9039,
+      "eval_samples_per_second": 5.375,
+      "eval_steps_per_second": 0.677,
+      "step": 769
+    },
+    {
+      "epoch": 3.0019493177387915,
+      "grad_norm": 0.8162972927093506,
+      "learning_rate": 0.0002,
+      "loss": 0.4332,
+      "step": 770
+    },
+    {
+      "epoch": 3.0409356725146197,
+      "grad_norm": 0.8814733028411865,
+      "learning_rate": 0.0002,
+      "loss": 0.3778,
+      "step": 780
+    },
+    {
+      "epoch": 3.0799220272904484,
+      "grad_norm": 0.6659106612205505,
+      "learning_rate": 0.0002,
+      "loss": 0.3775,
+      "step": 790
+    },
+    {
+      "epoch": 3.1189083820662766,
+      "grad_norm": 0.7013096213340759,
+      "learning_rate": 0.0002,
+      "loss": 0.3718,
+      "step": 800
+    },
+    {
+      "epoch": 3.1578947368421053,
+      "grad_norm": 0.5946340560913086,
+      "learning_rate": 0.0002,
+      "loss": 0.3799,
+      "step": 810
+    },
+    {
+      "epoch": 3.1968810916179335,
+      "grad_norm": 1.1263338327407837,
+      "learning_rate": 0.0002,
+      "loss": 0.3752,
+      "step": 820
+    },
+    {
+      "epoch": 3.2358674463937622,
+      "grad_norm": 1.028369426727295,
+      "learning_rate": 0.0002,
+      "loss": 0.3819,
+      "step": 830
+    },
+    {
+      "epoch": 3.2748538011695905,
+      "grad_norm": 1.1113781929016113,
+      "learning_rate": 0.0002,
+      "loss": 0.3816,
+      "step": 840
+    },
+    {
+      "epoch": 3.313840155945419,
+      "grad_norm": 0.7735888361930847,
+      "learning_rate": 0.0002,
+      "loss": 0.383,
+      "step": 850
+    },
+    {
+      "epoch": 3.3528265107212474,
+      "grad_norm": 0.8967649936676025,
+      "learning_rate": 0.0002,
+      "loss": 0.3703,
+      "step": 860
+    },
+    {
+      "epoch": 3.391812865497076,
+      "grad_norm": 0.7354698777198792,
+      "learning_rate": 0.0002,
+      "loss": 0.3884,
+      "step": 870
+    },
+    {
+      "epoch": 3.4307992202729043,
+      "grad_norm": 0.8721298575401306,
+      "learning_rate": 0.0002,
+      "loss": 0.3762,
+      "step": 880
+    },
+    {
+      "epoch": 3.469785575048733,
+      "grad_norm": 0.7276861667633057,
+      "learning_rate": 0.0002,
+      "loss": 0.3758,
+      "step": 890
+    },
+    {
+      "epoch": 3.5087719298245617,
+      "grad_norm": 0.8296075463294983,
+      "learning_rate": 0.0002,
+      "loss": 0.3866,
+      "step": 900
+    },
+    {
+      "epoch": 3.54775828460039,
+      "grad_norm": 0.8955655097961426,
+      "learning_rate": 0.0002,
+      "loss": 0.3909,
+      "step": 910
+    },
+    {
+      "epoch": 3.586744639376218,
+      "grad_norm": 0.8008049130439758,
+      "learning_rate": 0.0002,
+      "loss": 0.3882,
+      "step": 920
+    },
+    {
+      "epoch": 3.625730994152047,
+      "grad_norm": 0.7791480422019958,
+      "learning_rate": 0.0002,
+      "loss": 0.3837,
+      "step": 930
+    },
+    {
+      "epoch": 3.6647173489278755,
+      "grad_norm": 0.8780500292778015,
+      "learning_rate": 0.0002,
+      "loss": 0.3877,
+      "step": 940
+    },
+    {
+      "epoch": 3.7037037037037037,
+      "grad_norm": 0.8411193490028381,
+      "learning_rate": 0.0002,
+      "loss": 0.3852,
+      "step": 950
+    },
+    {
+      "epoch": 3.742690058479532,
+      "grad_norm": 0.7564510703086853,
+      "learning_rate": 0.0002,
+      "loss": 0.3789,
+      "step": 960
+    },
+    {
+      "epoch": 3.7816764132553606,
+      "grad_norm": 0.9967873692512512,
+      "learning_rate": 0.0002,
+      "loss": 0.3854,
+      "step": 970
+    },
+    {
+      "epoch": 3.8206627680311893,
+      "grad_norm": 1.0118954181671143,
+      "learning_rate": 0.0002,
+      "loss": 0.3664,
+      "step": 980
+    },
+    {
+      "epoch": 3.8596491228070176,
+      "grad_norm": 0.7355602383613586,
+      "learning_rate": 0.0002,
+      "loss": 0.367,
+      "step": 990
+    },
+    {
+      "epoch": 3.898635477582846,
+      "grad_norm": 0.7059919834136963,
+      "learning_rate": 0.0002,
+      "loss": 0.3802,
+      "step": 1000
+    },
+    {
+      "epoch": 3.9376218323586745,
+      "grad_norm": 0.6720611453056335,
+      "learning_rate": 0.0002,
+      "loss": 0.3907,
+      "step": 1010
+    },
+    {
+      "epoch": 3.976608187134503,
+      "grad_norm": 0.664450466632843,
+      "learning_rate": 0.0002,
+      "loss": 0.392,
+      "step": 1020
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 0.4771515429019928,
+      "eval_runtime": 68.2605,
+      "eval_samples_per_second": 5.347,
+      "eval_steps_per_second": 0.674,
+      "step": 1026
+    },
+    {
+      "epoch": 4.015594541910332,
+      "grad_norm": 0.8478823304176331,
+      "learning_rate": 0.0002,
+      "loss": 0.3554,
+      "step": 1030
+    },
+    {
+      "epoch": 4.05458089668616,
+      "grad_norm": 0.7147605419158936,
+      "learning_rate": 0.0002,
+      "loss": 0.3353,
+      "step": 1040
+    },
+    {
+      "epoch": 4.093567251461988,
+      "grad_norm": 0.8668870329856873,
+      "learning_rate": 0.0002,
+      "loss": 0.3318,
+      "step": 1050
+    },
+    {
+      "epoch": 4.132553606237817,
+      "grad_norm": 0.9445207715034485,
+      "learning_rate": 0.0002,
+      "loss": 0.3284,
+      "step": 1060
+    },
+    {
+      "epoch": 4.171539961013645,
+      "grad_norm": 0.7062109112739563,
+      "learning_rate": 0.0002,
+      "loss": 0.3453,
+      "step": 1070
+    },
+    {
+      "epoch": 4.2105263157894735,
+      "grad_norm": 0.8336073160171509,
+      "learning_rate": 0.0002,
+      "loss": 0.3345,
+      "step": 1080
+    },
+    {
+      "epoch": 4.249512670565302,
+      "grad_norm": 1.3078413009643555,
+      "learning_rate": 0.0002,
+      "loss": 0.3435,
+      "step": 1090
+    },
+    {
+      "epoch": 4.288499025341131,
+      "grad_norm": 0.8631044030189514,
+      "learning_rate": 0.0002,
+      "loss": 0.3443,
+      "step": 1100
+    },
+    {
+      "epoch": 4.3274853801169595,
+      "grad_norm": 0.7545914053916931,
+      "learning_rate": 0.0002,
+      "loss": 0.3269,
+      "step": 1110
+    },
+    {
+      "epoch": 4.366471734892787,
+      "grad_norm": 0.7546519637107849,
+      "learning_rate": 0.0002,
+      "loss": 0.3504,
+      "step": 1120
+    },
+    {
+      "epoch": 4.405458089668616,
+      "grad_norm": 0.8977745175361633,
+      "learning_rate": 0.0002,
+      "loss": 0.3419,
+      "step": 1130
+    },
+    {
+      "epoch": 4.444444444444445,
+      "grad_norm": 1.1983102560043335,
+      "learning_rate": 0.0002,
+      "loss": 0.3472,
+      "step": 1140
+    },
+    {
+      "epoch": 4.483430799220272,
+      "grad_norm": 0.9402135610580444,
+      "learning_rate": 0.0002,
+      "loss": 0.3409,
+      "step": 1150
+    },
+    {
+      "epoch": 4.522417153996101,
+      "grad_norm": 0.7405855655670166,
+      "learning_rate": 0.0002,
+      "loss": 0.3372,
+      "step": 1160
+    },
+    {
+      "epoch": 4.56140350877193,
+      "grad_norm": 0.6910150647163391,
+      "learning_rate": 0.0002,
+      "loss": 0.344,
+      "step": 1170
+    },
+    {
+      "epoch": 4.6003898635477585,
+      "grad_norm": 0.8426865339279175,
+      "learning_rate": 0.0002,
+      "loss": 0.3402,
+      "step": 1180
+    },
+    {
+      "epoch": 4.639376218323587,
+      "grad_norm": 0.6767265200614929,
+      "learning_rate": 0.0002,
+      "loss": 0.3361,
+      "step": 1190
+    },
+    {
+      "epoch": 4.678362573099415,
+      "grad_norm": 0.7790605425834656,
+      "learning_rate": 0.0002,
+      "loss": 0.3417,
+      "step": 1200
+    },
+    {
+      "epoch": 4.717348927875244,
+      "grad_norm": 0.7993232011795044,
+      "learning_rate": 0.0002,
+      "loss": 0.3513,
+      "step": 1210
+    },
+    {
+      "epoch": 4.756335282651072,
+      "grad_norm": 0.6861053705215454,
+      "learning_rate": 0.0002,
+      "loss": 0.346,
+      "step": 1220
+    },
+    {
+      "epoch": 4.7953216374269,
+      "grad_norm": 0.8599495887756348,
+      "learning_rate": 0.0002,
+      "loss": 0.3426,
+      "step": 1230
+    },
+    {
+      "epoch": 4.834307992202729,
+      "grad_norm": 0.8054423332214355,
+      "learning_rate": 0.0002,
+      "loss": 0.3622,
+      "step": 1240
+    },
+    {
+      "epoch": 4.8732943469785575,
+      "grad_norm": 0.949587881565094,
+      "learning_rate": 0.0002,
+      "loss": 0.343,
+      "step": 1250
+    },
+    {
+      "epoch": 4.912280701754386,
+      "grad_norm": 0.6002761125564575,
+      "learning_rate": 0.0002,
+      "loss": 0.3364,
+      "step": 1260
+    },
+    {
+      "epoch": 4.951267056530215,
+      "grad_norm": 0.7506934404373169,
+      "learning_rate": 0.0002,
+      "loss": 0.3417,
+      "step": 1270
+    },
+    {
+      "epoch": 4.990253411306043,
+      "grad_norm": 0.7621583938598633,
+      "learning_rate": 0.0002,
+      "loss": 0.3379,
+      "step": 1280
+    },
+    {
+      "epoch": 4.9980506822612085,
+      "eval_loss": 0.4660908877849579,
+      "eval_runtime": 67.4734,
+      "eval_samples_per_second": 5.41,
+      "eval_steps_per_second": 0.682,
+      "step": 1282
+    },
+    {
+      "epoch": 5.029239766081871,
+      "grad_norm": 0.6844434142112732,
+      "learning_rate": 0.0002,
+      "loss": 0.3272,
+      "step": 1290
+    },
+    {
+      "epoch": 5.0682261208577,
+      "grad_norm": 0.8217318058013916,
+      "learning_rate": 0.0002,
+      "loss": 0.3092,
+      "step": 1300
+    },
+    {
+      "epoch": 5.107212475633529,
+      "grad_norm": 1.0060460567474365,
+      "learning_rate": 0.0002,
+      "loss": 0.3131,
+      "step": 1310
+    },
+    {
+      "epoch": 5.146198830409356,
+      "grad_norm": 0.7044107913970947,
+      "learning_rate": 0.0002,
+      "loss": 0.31,
+      "step": 1320
+    },
+    {
+      "epoch": 5.185185185185185,
+      "grad_norm": 0.9584488868713379,
+      "learning_rate": 0.0002,
+      "loss": 0.3179,
+      "step": 1330
+    },
+    {
+      "epoch": 5.224171539961014,
+      "grad_norm": 0.9667648077011108,
+      "learning_rate": 0.0002,
+      "loss": 0.3138,
+      "step": 1340
+    },
+    {
+      "epoch": 5.2631578947368425,
+      "grad_norm": 0.938228189945221,
+      "learning_rate": 0.0002,
+      "loss": 0.3076,
+      "step": 1350
+    },
+    {
+      "epoch": 5.30214424951267,
+      "grad_norm": 0.7234814167022705,
+      "learning_rate": 0.0002,
+      "loss": 0.2999,
+      "step": 1360
+    },
+    {
+      "epoch": 5.341130604288499,
+      "grad_norm": 0.9198869466781616,
+      "learning_rate": 0.0002,
+      "loss": 0.3112,
+      "step": 1370
+    },
+    {
+      "epoch": 5.380116959064328,
+      "grad_norm": 0.6611506342887878,
+      "learning_rate": 0.0002,
+      "loss": 0.3131,
+      "step": 1380
+    },
+    {
+      "epoch": 5.419103313840156,
+      "grad_norm": 0.7446638345718384,
+      "learning_rate": 0.0002,
+      "loss": 0.3133,
+      "step": 1390
+    },
+    {
+      "epoch": 5.458089668615984,
+      "grad_norm": 1.050093412399292,
+      "learning_rate": 0.0002,
+      "loss": 0.3213,
+      "step": 1400
+    },
+    {
+      "epoch": 5.497076023391813,
+      "grad_norm": 0.7504354119300842,
+      "learning_rate": 0.0002,
+      "loss": 0.3156,
+      "step": 1410
+    },
+    {
+      "epoch": 5.5360623781676415,
+      "grad_norm": 0.6047531962394714,
+      "learning_rate": 0.0002,
+      "loss": 0.3172,
+      "step": 1420
+    },
+    {
+      "epoch": 5.57504873294347,
+      "grad_norm": 0.835421085357666,
+      "learning_rate": 0.0002,
+      "loss": 0.3118,
+      "step": 1430
+    },
+    {
+      "epoch": 5.614035087719298,
+      "grad_norm": 0.8842822313308716,
+      "learning_rate": 0.0002,
+      "loss": 0.3183,
+      "step": 1440
+    },
+    {
+      "epoch": 5.653021442495127,
+      "grad_norm": 0.782640814781189,
+      "learning_rate": 0.0002,
+      "loss": 0.3107,
+      "step": 1450
+    },
+    {
+      "epoch": 5.692007797270955,
+      "grad_norm": 0.7722012996673584,
+      "learning_rate": 0.0002,
+      "loss": 0.3157,
+      "step": 1460
+    },
+    {
+      "epoch": 5.730994152046784,
+      "grad_norm": 0.7458067536354065,
+      "learning_rate": 0.0002,
+      "loss": 0.3168,
+      "step": 1470
+    },
+    {
+      "epoch": 5.769980506822612,
+      "grad_norm": 0.7182063460350037,
+      "learning_rate": 0.0002,
+      "loss": 0.3219,
+      "step": 1480
+    },
+    {
+      "epoch": 5.8089668615984404,
+      "grad_norm": 1.0041741132736206,
+      "learning_rate": 0.0002,
+      "loss": 0.3152,
+      "step": 1490
+    },
+    {
+      "epoch": 5.847953216374269,
+      "grad_norm": 0.894087016582489,
+      "learning_rate": 0.0002,
+      "loss": 0.3148,
+      "step": 1500
+    },
+    {
+      "epoch": 5.886939571150098,
+      "grad_norm": 1.177829384803772,
+      "learning_rate": 0.0002,
+      "loss": 0.3073,
+      "step": 1510
+    },
+    {
+      "epoch": 5.925925925925926,
+      "grad_norm": 1.2548596858978271,
+      "learning_rate": 0.0002,
+      "loss": 0.3088,
+      "step": 1520
+    },
+    {
+      "epoch": 5.964912280701754,
+      "grad_norm": 0.9875953197479248,
+      "learning_rate": 0.0002,
+      "loss": 0.3107,
+      "step": 1530
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 0.4633646011352539,
+      "eval_runtime": 67.64,
+      "eval_samples_per_second": 5.396,
+      "eval_steps_per_second": 0.68,
+      "step": 1539
+    },
+    {
+      "epoch": 6.003898635477583,
+      "grad_norm": 0.7251299619674683,
+      "learning_rate": 0.0002,
+      "loss": 0.3155,
+      "step": 1540
+    },
+    {
+      "epoch": 6.042884990253412,
+      "grad_norm": 0.7206684947013855,
+      "learning_rate": 0.0002,
+      "loss": 0.2645,
+      "step": 1550
+    },
+    {
+      "epoch": 6.081871345029239,
+      "grad_norm": 0.727130651473999,
+      "learning_rate": 0.0002,
+      "loss": 0.2863,
+      "step": 1560
+    },
+    {
+      "epoch": 6.120857699805068,
+      "grad_norm": 0.8526223301887512,
+      "learning_rate": 0.0002,
+      "loss": 0.2945,
+      "step": 1570
+    },
+    {
+      "epoch": 6.159844054580897,
+      "grad_norm": 0.9564275741577148,
+      "learning_rate": 0.0002,
+      "loss": 0.2774,
+      "step": 1580
+    },
+    {
+      "epoch": 6.1988304093567255,
+      "grad_norm": 0.6997653245925903,
+      "learning_rate": 0.0002,
+      "loss": 0.2802,
+      "step": 1590
+    },
+    {
+      "epoch": 6.237816764132553,
+      "grad_norm": 0.549583911895752,
+      "learning_rate": 0.0002,
+      "loss": 0.2911,
+      "step": 1600
+    },
+    {
+      "epoch": 6.276803118908382,
+      "grad_norm": 0.9782546758651733,
+      "learning_rate": 0.0002,
+      "loss": 0.2855,
+      "step": 1610
+    },
+    {
+      "epoch": 6.315789473684211,
+      "grad_norm": 1.0026962757110596,
+      "learning_rate": 0.0002,
+      "loss": 0.2971,
+      "step": 1620
+    },
+    {
+      "epoch": 6.354775828460039,
+      "grad_norm": 0.9684990048408508,
+      "learning_rate": 0.0002,
+      "loss": 0.2933,
+      "step": 1630
+    },
+    {
+      "epoch": 6.393762183235867,
+      "grad_norm": 0.7317994832992554,
+      "learning_rate": 0.0002,
+      "loss": 0.2918,
+      "step": 1640
+    },
+    {
+      "epoch": 6.432748538011696,
+      "grad_norm": 0.8299538493156433,
+      "learning_rate": 0.0002,
+      "loss": 0.2946,
+      "step": 1650
+    },
+    {
+      "epoch": 6.4717348927875245,
+      "grad_norm": 0.8719546794891357,
+      "learning_rate": 0.0002,
+      "loss": 0.2909,
+      "step": 1660
+    },
+    {
+      "epoch": 6.510721247563353,
+      "grad_norm": 0.7498378753662109,
+      "learning_rate": 0.0002,
+      "loss": 0.2826,
+      "step": 1670
+    },
+    {
+      "epoch": 6.549707602339181,
+      "grad_norm": 0.8273977041244507,
+      "learning_rate": 0.0002,
+      "loss": 0.2827,
+      "step": 1680
+    },
+    {
+      "epoch": 6.58869395711501,
+      "grad_norm": 0.6725714802742004,
+      "learning_rate": 0.0002,
+      "loss": 0.2913,
+      "step": 1690
+    },
+    {
+      "epoch": 6.627680311890838,
+      "grad_norm": 0.752943217754364,
+      "learning_rate": 0.0002,
+      "loss": 0.2879,
+      "step": 1700
+    },
+    {
+      "epoch": 6.666666666666667,
+      "grad_norm": 0.8002777099609375,
+      "learning_rate": 0.0002,
+      "loss": 0.3006,
+      "step": 1710
+    },
+    {
+      "epoch": 6.705653021442495,
+      "grad_norm": 0.5702647566795349,
+      "learning_rate": 0.0002,
+      "loss": 0.2934,
+      "step": 1720
+    },
+    {
+      "epoch": 6.744639376218323,
+      "grad_norm": 1.0377178192138672,
+      "learning_rate": 0.0002,
+      "loss": 0.3001,
+      "step": 1730
+    },
+    {
+      "epoch": 6.783625730994152,
+      "grad_norm": 0.9070322513580322,
+      "learning_rate": 0.0002,
+      "loss": 0.2958,
+      "step": 1740
+    },
+    {
+      "epoch": 6.822612085769981,
+      "grad_norm": 0.8154694437980652,
+      "learning_rate": 0.0002,
+      "loss": 0.2935,
+      "step": 1750
+    },
+    {
+      "epoch": 6.861598440545809,
+      "grad_norm": 1.035827875137329,
+      "learning_rate": 0.0002,
+      "loss": 0.3025,
+      "step": 1760
+    },
+    {
+      "epoch": 6.900584795321637,
+      "grad_norm": 0.6619474291801453,
+      "learning_rate": 0.0002,
+      "loss": 0.3071,
+      "step": 1770
+    },
+    {
+      "epoch": 6.939571150097466,
+      "grad_norm": 1.0016711950302124,
+      "learning_rate": 0.0002,
+      "loss": 0.3001,
+      "step": 1780
+    },
+    {
+      "epoch": 6.978557504873295,
+      "grad_norm": 1.1134511232376099,
+      "learning_rate": 0.0002,
+      "loss": 0.2833,
+      "step": 1790
+    },
+    {
+      "epoch": 6.9980506822612085,
+      "eval_loss": 0.4561765789985657,
+      "eval_runtime": 67.345,
+      "eval_samples_per_second": 5.42,
+      "eval_steps_per_second": 0.683,
+      "step": 1795
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 2048,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 7.87770718027776e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}