Upload folder using huggingface_hub

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/README.md

@@ -0,0 +1,202 @@
+---
+base_model: meta-llama/Meta-Llama-3-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fbf404f317d6d677df0262caf60cbf2efe36581fc8bd30e6225cb5db1e47cfba
+size 109069176

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1090/README.md

@@ -0,0 +1,202 @@
+---
+base_model: meta-llama/Meta-Llama-3-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1090/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1090/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:655e1ccc9660d104a8b6d82811fcf77f5dc248ab08fd06844778455121b5b452
+size 109069176

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1090/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:340efda11de515f18704bf210edf64b93a8a02dccf754d02696f4fcd008ffc59
+size 55532666

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1090/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5f55a2c845f4e0f61a728edd27313eb6af48b6f35a5e769221ae9f62567449c
+size 14244

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1090/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:78e1caff834a14f98ef0174c0ce41b886afcb8a2675b1ffa9215ddb764a25db7
+size 1064

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1090/special_tokens_map.json

@@ -0,0 +1,17 @@
+{
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|eot_id|>"
+}

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1090/tokenizer_config.json

@@ -0,0 +1,2063 @@
+{
+  "added_tokens_decoder": {
+    "128000": {
+      "content": "<|begin_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128001": {
+      "content": "<|end_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128002": {
+      "content": "<|reserved_special_token_0|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128003": {
+      "content": "<|reserved_special_token_1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128004": {
+      "content": "<|reserved_special_token_2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128005": {
+      "content": "<|reserved_special_token_3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128006": {
+      "content": "<|start_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128007": {
+      "content": "<|end_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128008": {
+      "content": "<|reserved_special_token_4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128009": {
+      "content": "<|eot_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128010": {
+      "content": "<|reserved_special_token_5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128011": {
+      "content": "<|reserved_special_token_6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128012": {
+      "content": "<|reserved_special_token_7|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128013": {
+      "content": "<|reserved_special_token_8|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128014": {
+      "content": "<|reserved_special_token_9|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128015": {
+      "content": "<|reserved_special_token_10|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128016": {
+      "content": "<|reserved_special_token_11|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128017": {
+      "content": "<|reserved_special_token_12|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128018": {
+      "content": "<|reserved_special_token_13|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128019": {
+      "content": "<|reserved_special_token_14|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128020": {
+      "content": "<|reserved_special_token_15|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128021": {
+      "content": "<|reserved_special_token_16|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128022": {
+      "content": "<|reserved_special_token_17|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128023": {
+      "content": "<|reserved_special_token_18|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128024": {
+      "content": "<|reserved_special_token_19|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128025": {
+      "content": "<|reserved_special_token_20|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128026": {
+      "content": "<|reserved_special_token_21|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128027": {
+      "content": "<|reserved_special_token_22|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128028": {
+      "content": "<|reserved_special_token_23|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128029": {
+      "content": "<|reserved_special_token_24|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128030": {
+      "content": "<|reserved_special_token_25|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128031": {
+      "content": "<|reserved_special_token_26|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128032": {
+      "content": "<|reserved_special_token_27|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128033": {
+      "content": "<|reserved_special_token_28|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128034": {
+      "content": "<|reserved_special_token_29|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128035": {
+      "content": "<|reserved_special_token_30|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128036": {
+      "content": "<|reserved_special_token_31|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128037": {
+      "content": "<|reserved_special_token_32|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128038": {
+      "content": "<|reserved_special_token_33|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128039": {
+      "content": "<|reserved_special_token_34|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128040": {
+      "content": "<|reserved_special_token_35|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128041": {
+      "content": "<|reserved_special_token_36|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128042": {
+      "content": "<|reserved_special_token_37|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128043": {
+      "content": "<|reserved_special_token_38|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128044": {
+      "content": "<|reserved_special_token_39|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128045": {
+      "content": "<|reserved_special_token_40|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128046": {
+      "content": "<|reserved_special_token_41|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128047": {
+      "content": "<|reserved_special_token_42|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128048": {
+      "content": "<|reserved_special_token_43|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128049": {
+      "content": "<|reserved_special_token_44|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128050": {
+      "content": "<|reserved_special_token_45|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128051": {
+      "content": "<|reserved_special_token_46|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128052": {
+      "content": "<|reserved_special_token_47|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128053": {
+      "content": "<|reserved_special_token_48|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128054": {
+      "content": "<|reserved_special_token_49|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128055": {
+      "content": "<|reserved_special_token_50|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128056": {
+      "content": "<|reserved_special_token_51|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128057": {
+      "content": "<|reserved_special_token_52|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128058": {
+      "content": "<|reserved_special_token_53|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128059": {
+      "content": "<|reserved_special_token_54|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128060": {
+      "content": "<|reserved_special_token_55|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128061": {
+      "content": "<|reserved_special_token_56|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128062": {
+      "content": "<|reserved_special_token_57|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128063": {
+      "content": "<|reserved_special_token_58|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128064": {
+      "content": "<|reserved_special_token_59|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128065": {
+      "content": "<|reserved_special_token_60|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128066": {
+      "content": "<|reserved_special_token_61|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128067": {
+      "content": "<|reserved_special_token_62|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128068": {
+      "content": "<|reserved_special_token_63|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128069": {
+      "content": "<|reserved_special_token_64|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128070": {
+      "content": "<|reserved_special_token_65|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128071": {
+      "content": "<|reserved_special_token_66|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128072": {
+      "content": "<|reserved_special_token_67|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128073": {
+      "content": "<|reserved_special_token_68|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128074": {
+      "content": "<|reserved_special_token_69|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128075": {
+      "content": "<|reserved_special_token_70|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128076": {
+      "content": "<|reserved_special_token_71|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128077": {
+      "content": "<|reserved_special_token_72|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128078": {
+      "content": "<|reserved_special_token_73|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128079": {
+      "content": "<|reserved_special_token_74|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128080": {
+      "content": "<|reserved_special_token_75|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128081": {
+      "content": "<|reserved_special_token_76|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128082": {
+      "content": "<|reserved_special_token_77|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128083": {
+      "content": "<|reserved_special_token_78|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128084": {
+      "content": "<|reserved_special_token_79|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128085": {
+      "content": "<|reserved_special_token_80|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128086": {
+      "content": "<|reserved_special_token_81|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128087": {
+      "content": "<|reserved_special_token_82|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128088": {
+      "content": "<|reserved_special_token_83|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128089": {
+      "content": "<|reserved_special_token_84|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128090": {
+      "content": "<|reserved_special_token_85|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128091": {
+      "content": "<|reserved_special_token_86|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128092": {
+      "content": "<|reserved_special_token_87|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128093": {
+      "content": "<|reserved_special_token_88|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128094": {
+      "content": "<|reserved_special_token_89|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128095": {
+      "content": "<|reserved_special_token_90|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128096": {
+      "content": "<|reserved_special_token_91|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128097": {
+      "content": "<|reserved_special_token_92|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128098": {
+      "content": "<|reserved_special_token_93|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128099": {
+      "content": "<|reserved_special_token_94|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128100": {
+      "content": "<|reserved_special_token_95|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128101": {
+      "content": "<|reserved_special_token_96|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128102": {
+      "content": "<|reserved_special_token_97|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128103": {
+      "content": "<|reserved_special_token_98|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128104": {
+      "content": "<|reserved_special_token_99|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128105": {
+      "content": "<|reserved_special_token_100|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128106": {
+      "content": "<|reserved_special_token_101|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128107": {
+      "content": "<|reserved_special_token_102|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128108": {
+      "content": "<|reserved_special_token_103|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128109": {
+      "content": "<|reserved_special_token_104|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128110": {
+      "content": "<|reserved_special_token_105|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128111": {
+      "content": "<|reserved_special_token_106|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128112": {
+      "content": "<|reserved_special_token_107|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128113": {
+      "content": "<|reserved_special_token_108|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128114": {
+      "content": "<|reserved_special_token_109|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128115": {
+      "content": "<|reserved_special_token_110|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128116": {
+      "content": "<|reserved_special_token_111|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128117": {
+      "content": "<|reserved_special_token_112|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128118": {
+      "content": "<|reserved_special_token_113|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128119": {
+      "content": "<|reserved_special_token_114|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128120": {
+      "content": "<|reserved_special_token_115|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128121": {
+      "content": "<|reserved_special_token_116|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128122": {
+      "content": "<|reserved_special_token_117|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128123": {
+      "content": "<|reserved_special_token_118|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128124": {
+      "content": "<|reserved_special_token_119|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128125": {
+      "content": "<|reserved_special_token_120|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128126": {
+      "content": "<|reserved_special_token_121|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128127": {
+      "content": "<|reserved_special_token_122|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128128": {
+      "content": "<|reserved_special_token_123|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128129": {
+      "content": "<|reserved_special_token_124|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128130": {
+      "content": "<|reserved_special_token_125|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128131": {
+      "content": "<|reserved_special_token_126|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128132": {
+      "content": "<|reserved_special_token_127|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128133": {
+      "content": "<|reserved_special_token_128|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128134": {
+      "content": "<|reserved_special_token_129|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128135": {
+      "content": "<|reserved_special_token_130|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128136": {
+      "content": "<|reserved_special_token_131|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128137": {
+      "content": "<|reserved_special_token_132|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128138": {
+      "content": "<|reserved_special_token_133|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128139": {
+      "content": "<|reserved_special_token_134|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128140": {
+      "content": "<|reserved_special_token_135|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128141": {
+      "content": "<|reserved_special_token_136|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128142": {
+      "content": "<|reserved_special_token_137|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128143": {
+      "content": "<|reserved_special_token_138|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128144": {
+      "content": "<|reserved_special_token_139|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128145": {
+      "content": "<|reserved_special_token_140|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128146": {
+      "content": "<|reserved_special_token_141|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128147": {
+      "content": "<|reserved_special_token_142|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128148": {
+      "content": "<|reserved_special_token_143|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128149": {
+      "content": "<|reserved_special_token_144|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128150": {
+      "content": "<|reserved_special_token_145|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128151": {
+      "content": "<|reserved_special_token_146|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128152": {
+      "content": "<|reserved_special_token_147|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128153": {
+      "content": "<|reserved_special_token_148|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128154": {
+      "content": "<|reserved_special_token_149|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128155": {
+      "content": "<|reserved_special_token_150|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128156": {
+      "content": "<|reserved_special_token_151|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128157": {
+      "content": "<|reserved_special_token_152|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128158": {
+      "content": "<|reserved_special_token_153|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128159": {
+      "content": "<|reserved_special_token_154|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128160": {
+      "content": "<|reserved_special_token_155|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128161": {
+      "content": "<|reserved_special_token_156|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128162": {
+      "content": "<|reserved_special_token_157|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128163": {
+      "content": "<|reserved_special_token_158|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128164": {
+      "content": "<|reserved_special_token_159|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128165": {
+      "content": "<|reserved_special_token_160|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128166": {
+      "content": "<|reserved_special_token_161|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128167": {
+      "content": "<|reserved_special_token_162|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128168": {
+      "content": "<|reserved_special_token_163|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128169": {
+      "content": "<|reserved_special_token_164|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128170": {
+      "content": "<|reserved_special_token_165|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128171": {
+      "content": "<|reserved_special_token_166|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128172": {
+      "content": "<|reserved_special_token_167|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128173": {
+      "content": "<|reserved_special_token_168|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128174": {
+      "content": "<|reserved_special_token_169|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128175": {
+      "content": "<|reserved_special_token_170|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128176": {
+      "content": "<|reserved_special_token_171|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128177": {
+      "content": "<|reserved_special_token_172|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128178": {
+      "content": "<|reserved_special_token_173|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128179": {
+      "content": "<|reserved_special_token_174|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128180": {
+      "content": "<|reserved_special_token_175|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128181": {
+      "content": "<|reserved_special_token_176|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128182": {
+      "content": "<|reserved_special_token_177|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128183": {
+      "content": "<|reserved_special_token_178|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128184": {
+      "content": "<|reserved_special_token_179|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128185": {
+      "content": "<|reserved_special_token_180|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128186": {
+      "content": "<|reserved_special_token_181|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128187": {
+      "content": "<|reserved_special_token_182|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128188": {
+      "content": "<|reserved_special_token_183|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128189": {
+      "content": "<|reserved_special_token_184|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128190": {
+      "content": "<|reserved_special_token_185|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128191": {
+      "content": "<|reserved_special_token_186|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128192": {
+      "content": "<|reserved_special_token_187|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128193": {
+      "content": "<|reserved_special_token_188|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128194": {
+      "content": "<|reserved_special_token_189|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128195": {
+      "content": "<|reserved_special_token_190|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128196": {
+      "content": "<|reserved_special_token_191|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128197": {
+      "content": "<|reserved_special_token_192|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128198": {
+      "content": "<|reserved_special_token_193|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128199": {
+      "content": "<|reserved_special_token_194|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128200": {
+      "content": "<|reserved_special_token_195|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128201": {
+      "content": "<|reserved_special_token_196|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128202": {
+      "content": "<|reserved_special_token_197|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128203": {
+      "content": "<|reserved_special_token_198|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128204": {
+      "content": "<|reserved_special_token_199|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128205": {
+      "content": "<|reserved_special_token_200|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128206": {
+      "content": "<|reserved_special_token_201|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128207": {
+      "content": "<|reserved_special_token_202|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128208": {
+      "content": "<|reserved_special_token_203|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128209": {
+      "content": "<|reserved_special_token_204|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128210": {
+      "content": "<|reserved_special_token_205|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128211": {
+      "content": "<|reserved_special_token_206|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128212": {
+      "content": "<|reserved_special_token_207|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128213": {
+      "content": "<|reserved_special_token_208|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128214": {
+      "content": "<|reserved_special_token_209|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128215": {
+      "content": "<|reserved_special_token_210|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128216": {
+      "content": "<|reserved_special_token_211|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128217": {
+      "content": "<|reserved_special_token_212|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128218": {
+      "content": "<|reserved_special_token_213|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128219": {
+      "content": "<|reserved_special_token_214|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128220": {
+      "content": "<|reserved_special_token_215|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128221": {
+      "content": "<|reserved_special_token_216|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128222": {
+      "content": "<|reserved_special_token_217|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128223": {
+      "content": "<|reserved_special_token_218|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128224": {
+      "content": "<|reserved_special_token_219|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128225": {
+      "content": "<|reserved_special_token_220|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128226": {
+      "content": "<|reserved_special_token_221|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128227": {
+      "content": "<|reserved_special_token_222|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128228": {
+      "content": "<|reserved_special_token_223|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128229": {
+      "content": "<|reserved_special_token_224|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128230": {
+      "content": "<|reserved_special_token_225|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128231": {
+      "content": "<|reserved_special_token_226|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128232": {
+      "content": "<|reserved_special_token_227|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128233": {
+      "content": "<|reserved_special_token_228|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128234": {
+      "content": "<|reserved_special_token_229|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128235": {
+      "content": "<|reserved_special_token_230|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128236": {
+      "content": "<|reserved_special_token_231|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128237": {
+      "content": "<|reserved_special_token_232|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128238": {
+      "content": "<|reserved_special_token_233|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128239": {
+      "content": "<|reserved_special_token_234|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128240": {
+      "content": "<|reserved_special_token_235|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128241": {
+      "content": "<|reserved_special_token_236|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128242": {
+      "content": "<|reserved_special_token_237|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128243": {
+      "content": "<|reserved_special_token_238|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128244": {
+      "content": "<|reserved_special_token_239|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128245": {
+      "content": "<|reserved_special_token_240|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128246": {
+      "content": "<|reserved_special_token_241|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128247": {
+      "content": "<|reserved_special_token_242|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128248": {
+      "content": "<|reserved_special_token_243|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128249": {
+      "content": "<|reserved_special_token_244|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128250": {
+      "content": "<|reserved_special_token_245|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128251": {
+      "content": "<|reserved_special_token_246|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128252": {
+      "content": "<|reserved_special_token_247|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128253": {
+      "content": "<|reserved_special_token_248|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128254": {
+      "content": "<|reserved_special_token_249|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128255": {
+      "content": "<|reserved_special_token_250|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|begin_of_text|>",
+  "chat_template": "{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|eot_id|>",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<|eot_id|>",
+  "tokenizer_class": "PreTrainedTokenizerFast"
+}

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1090/trainer_state.json

@@ -0,0 +1,836 @@
+{
+  "best_metric": 1.0338033437728882,
+  "best_model_checkpoint": "outputs-001/Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-654",
+  "epoch": 5.0,
+  "eval_steps": 10,
+  "global_step": 1090,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.045871559633027525,
+      "grad_norm": 0.7936908602714539,
+      "learning_rate": 0.0001,
+      "loss": 2.1542,
+      "step": 10
+    },
+    {
+      "epoch": 0.09174311926605505,
+      "grad_norm": 0.8541925549507141,
+      "learning_rate": 0.0001,
+      "loss": 1.7148,
+      "step": 20
+    },
+    {
+      "epoch": 0.13761467889908258,
+      "grad_norm": 0.6849298477172852,
+      "learning_rate": 0.0001,
+      "loss": 1.4102,
+      "step": 30
+    },
+    {
+      "epoch": 0.1834862385321101,
+      "grad_norm": 0.6280460953712463,
+      "learning_rate": 0.0001,
+      "loss": 1.2936,
+      "step": 40
+    },
+    {
+      "epoch": 0.22935779816513763,
+      "grad_norm": 0.699965238571167,
+      "learning_rate": 0.0001,
+      "loss": 1.2116,
+      "step": 50
+    },
+    {
+      "epoch": 0.27522935779816515,
+      "grad_norm": 0.5185195803642273,
+      "learning_rate": 0.0001,
+      "loss": 1.2337,
+      "step": 60
+    },
+    {
+      "epoch": 0.3211009174311927,
+      "grad_norm": 0.5821201801300049,
+      "learning_rate": 0.0001,
+      "loss": 1.2317,
+      "step": 70
+    },
+    {
+      "epoch": 0.3669724770642202,
+      "grad_norm": 0.4655168652534485,
+      "learning_rate": 0.0001,
+      "loss": 1.1355,
+      "step": 80
+    },
+    {
+      "epoch": 0.41284403669724773,
+      "grad_norm": 0.5342468023300171,
+      "learning_rate": 0.0001,
+      "loss": 1.1571,
+      "step": 90
+    },
+    {
+      "epoch": 0.45871559633027525,
+      "grad_norm": 0.42904630303382874,
+      "learning_rate": 0.0001,
+      "loss": 1.1333,
+      "step": 100
+    },
+    {
+      "epoch": 0.5045871559633027,
+      "grad_norm": 0.5210477709770203,
+      "learning_rate": 0.0001,
+      "loss": 1.1596,
+      "step": 110
+    },
+    {
+      "epoch": 0.5504587155963303,
+      "grad_norm": 0.38087233901023865,
+      "learning_rate": 0.0001,
+      "loss": 1.1089,
+      "step": 120
+    },
+    {
+      "epoch": 0.5963302752293578,
+      "grad_norm": 0.4819198250770569,
+      "learning_rate": 0.0001,
+      "loss": 1.1458,
+      "step": 130
+    },
+    {
+      "epoch": 0.6422018348623854,
+      "grad_norm": 0.6635904908180237,
+      "learning_rate": 0.0001,
+      "loss": 1.1834,
+      "step": 140
+    },
+    {
+      "epoch": 0.6880733944954128,
+      "grad_norm": 0.5125461220741272,
+      "learning_rate": 0.0001,
+      "loss": 1.07,
+      "step": 150
+    },
+    {
+      "epoch": 0.7339449541284404,
+      "grad_norm": 0.5035977959632874,
+      "learning_rate": 0.0001,
+      "loss": 1.093,
+      "step": 160
+    },
+    {
+      "epoch": 0.7798165137614679,
+      "grad_norm": 0.5379819869995117,
+      "learning_rate": 0.0001,
+      "loss": 1.0297,
+      "step": 170
+    },
+    {
+      "epoch": 0.8256880733944955,
+      "grad_norm": 0.4491284191608429,
+      "learning_rate": 0.0001,
+      "loss": 1.1212,
+      "step": 180
+    },
+    {
+      "epoch": 0.8715596330275229,
+      "grad_norm": 0.5257289409637451,
+      "learning_rate": 0.0001,
+      "loss": 1.139,
+      "step": 190
+    },
+    {
+      "epoch": 0.9174311926605505,
+      "grad_norm": 0.5361109375953674,
+      "learning_rate": 0.0001,
+      "loss": 1.0379,
+      "step": 200
+    },
+    {
+      "epoch": 0.963302752293578,
+      "grad_norm": 0.5124537348747253,
+      "learning_rate": 0.0001,
+      "loss": 1.0588,
+      "step": 210
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.070561170578003,
+      "eval_runtime": 8.31,
+      "eval_samples_per_second": 13.117,
+      "eval_steps_per_second": 1.685,
+      "step": 218
+    },
+    {
+      "epoch": 1.0091743119266054,
+      "grad_norm": 0.44702231884002686,
+      "learning_rate": 0.0001,
+      "loss": 1.0394,
+      "step": 220
+    },
+    {
+      "epoch": 1.0550458715596331,
+      "grad_norm": 0.48708081245422363,
+      "learning_rate": 0.0001,
+      "loss": 1.1075,
+      "step": 230
+    },
+    {
+      "epoch": 1.1009174311926606,
+      "grad_norm": 0.6492213010787964,
+      "learning_rate": 0.0001,
+      "loss": 1.0769,
+      "step": 240
+    },
+    {
+      "epoch": 1.146788990825688,
+      "grad_norm": 0.48868921399116516,
+      "learning_rate": 0.0001,
+      "loss": 1.0165,
+      "step": 250
+    },
+    {
+      "epoch": 1.1926605504587156,
+      "grad_norm": 0.6291283369064331,
+      "learning_rate": 0.0001,
+      "loss": 0.9982,
+      "step": 260
+    },
+    {
+      "epoch": 1.238532110091743,
+      "grad_norm": 0.6115853190422058,
+      "learning_rate": 0.0001,
+      "loss": 1.0501,
+      "step": 270
+    },
+    {
+      "epoch": 1.2844036697247707,
+      "grad_norm": 0.6975598931312561,
+      "learning_rate": 0.0001,
+      "loss": 1.0874,
+      "step": 280
+    },
+    {
+      "epoch": 1.3302752293577982,
+      "grad_norm": 0.5336413383483887,
+      "learning_rate": 0.0001,
+      "loss": 1.0721,
+      "step": 290
+    },
+    {
+      "epoch": 1.3761467889908257,
+      "grad_norm": 0.5163336992263794,
+      "learning_rate": 0.0001,
+      "loss": 1.0405,
+      "step": 300
+    },
+    {
+      "epoch": 1.4220183486238533,
+      "grad_norm": 0.6026386022567749,
+      "learning_rate": 0.0001,
+      "loss": 1.0119,
+      "step": 310
+    },
+    {
+      "epoch": 1.4678899082568808,
+      "grad_norm": 0.6290488839149475,
+      "learning_rate": 0.0001,
+      "loss": 1.0327,
+      "step": 320
+    },
+    {
+      "epoch": 1.5137614678899083,
+      "grad_norm": 0.5215861797332764,
+      "learning_rate": 0.0001,
+      "loss": 1.0725,
+      "step": 330
+    },
+    {
+      "epoch": 1.5596330275229358,
+      "grad_norm": 0.6104291677474976,
+      "learning_rate": 0.0001,
+      "loss": 1.0364,
+      "step": 340
+    },
+    {
+      "epoch": 1.6055045871559632,
+      "grad_norm": 0.5163600444793701,
+      "learning_rate": 0.0001,
+      "loss": 1.0412,
+      "step": 350
+    },
+    {
+      "epoch": 1.6513761467889907,
+      "grad_norm": 0.4934627413749695,
+      "learning_rate": 0.0001,
+      "loss": 1.0661,
+      "step": 360
+    },
+    {
+      "epoch": 1.6972477064220184,
+      "grad_norm": 0.6802699565887451,
+      "learning_rate": 0.0001,
+      "loss": 1.0644,
+      "step": 370
+    },
+    {
+      "epoch": 1.7431192660550459,
+      "grad_norm": 0.5180800557136536,
+      "learning_rate": 0.0001,
+      "loss": 1.0175,
+      "step": 380
+    },
+    {
+      "epoch": 1.7889908256880735,
+      "grad_norm": 0.6208664178848267,
+      "learning_rate": 0.0001,
+      "loss": 1.0099,
+      "step": 390
+    },
+    {
+      "epoch": 1.834862385321101,
+      "grad_norm": 0.5667423605918884,
+      "learning_rate": 0.0001,
+      "loss": 0.9917,
+      "step": 400
+    },
+    {
+      "epoch": 1.8807339449541285,
+      "grad_norm": 0.7282969355583191,
+      "learning_rate": 0.0001,
+      "loss": 1.0028,
+      "step": 410
+    },
+    {
+      "epoch": 1.926605504587156,
+      "grad_norm": 0.5308305025100708,
+      "learning_rate": 0.0001,
+      "loss": 1.062,
+      "step": 420
+    },
+    {
+      "epoch": 1.9724770642201834,
+      "grad_norm": 0.6123433113098145,
+      "learning_rate": 0.0001,
+      "loss": 1.012,
+      "step": 430
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.0371460914611816,
+      "eval_runtime": 8.3019,
+      "eval_samples_per_second": 13.13,
+      "eval_steps_per_second": 1.686,
+      "step": 436
+    },
+    {
+      "epoch": 2.018348623853211,
+      "grad_norm": 0.5201505422592163,
+      "learning_rate": 0.0001,
+      "loss": 1.0126,
+      "step": 440
+    },
+    {
+      "epoch": 2.0642201834862384,
+      "grad_norm": 0.5720698237419128,
+      "learning_rate": 0.0001,
+      "loss": 0.9625,
+      "step": 450
+    },
+    {
+      "epoch": 2.1100917431192663,
+      "grad_norm": 0.6407686471939087,
+      "learning_rate": 0.0001,
+      "loss": 0.9256,
+      "step": 460
+    },
+    {
+      "epoch": 2.1559633027522938,
+      "grad_norm": 0.5237308740615845,
+      "learning_rate": 0.0001,
+      "loss": 0.9982,
+      "step": 470
+    },
+    {
+      "epoch": 2.2018348623853212,
+      "grad_norm": 0.6064776182174683,
+      "learning_rate": 0.0001,
+      "loss": 0.9806,
+      "step": 480
+    },
+    {
+      "epoch": 2.2477064220183487,
+      "grad_norm": 0.5534603595733643,
+      "learning_rate": 0.0001,
+      "loss": 0.9223,
+      "step": 490
+    },
+    {
+      "epoch": 2.293577981651376,
+      "grad_norm": 0.610197126865387,
+      "learning_rate": 0.0001,
+      "loss": 0.9339,
+      "step": 500
+    },
+    {
+      "epoch": 2.3394495412844036,
+      "grad_norm": 0.637509286403656,
+      "learning_rate": 0.0001,
+      "loss": 0.9669,
+      "step": 510
+    },
+    {
+      "epoch": 2.385321100917431,
+      "grad_norm": 0.6705873608589172,
+      "learning_rate": 0.0001,
+      "loss": 0.9798,
+      "step": 520
+    },
+    {
+      "epoch": 2.4311926605504586,
+      "grad_norm": 0.6272872090339661,
+      "learning_rate": 0.0001,
+      "loss": 0.9868,
+      "step": 530
+    },
+    {
+      "epoch": 2.477064220183486,
+      "grad_norm": 0.6892614364624023,
+      "learning_rate": 0.0001,
+      "loss": 1.0059,
+      "step": 540
+    },
+    {
+      "epoch": 2.522935779816514,
+      "grad_norm": 0.6316735744476318,
+      "learning_rate": 0.0001,
+      "loss": 0.9838,
+      "step": 550
+    },
+    {
+      "epoch": 2.5688073394495414,
+      "grad_norm": 0.5902363061904907,
+      "learning_rate": 0.0001,
+      "loss": 0.9548,
+      "step": 560
+    },
+    {
+      "epoch": 2.614678899082569,
+      "grad_norm": 0.6575316786766052,
+      "learning_rate": 0.0001,
+      "loss": 0.9859,
+      "step": 570
+    },
+    {
+      "epoch": 2.6605504587155964,
+      "grad_norm": 0.574419379234314,
+      "learning_rate": 0.0001,
+      "loss": 0.9976,
+      "step": 580
+    },
+    {
+      "epoch": 2.706422018348624,
+      "grad_norm": 0.7602782249450684,
+      "learning_rate": 0.0001,
+      "loss": 1.0267,
+      "step": 590
+    },
+    {
+      "epoch": 2.7522935779816513,
+      "grad_norm": 0.5764957070350647,
+      "learning_rate": 0.0001,
+      "loss": 0.9966,
+      "step": 600
+    },
+    {
+      "epoch": 2.7981651376146788,
+      "grad_norm": 0.68736332654953,
+      "learning_rate": 0.0001,
+      "loss": 0.9717,
+      "step": 610
+    },
+    {
+      "epoch": 2.8440366972477067,
+      "grad_norm": 0.5835479497909546,
+      "learning_rate": 0.0001,
+      "loss": 0.9393,
+      "step": 620
+    },
+    {
+      "epoch": 2.8899082568807337,
+      "grad_norm": 0.7003713250160217,
+      "learning_rate": 0.0001,
+      "loss": 0.9751,
+      "step": 630
+    },
+    {
+      "epoch": 2.9357798165137616,
+      "grad_norm": 0.5811060070991516,
+      "learning_rate": 0.0001,
+      "loss": 0.9803,
+      "step": 640
+    },
+    {
+      "epoch": 2.981651376146789,
+      "grad_norm": 0.5394262671470642,
+      "learning_rate": 0.0001,
+      "loss": 1.0262,
+      "step": 650
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 1.0338033437728882,
+      "eval_runtime": 8.3,
+      "eval_samples_per_second": 13.133,
+      "eval_steps_per_second": 1.687,
+      "step": 654
+    },
+    {
+      "epoch": 3.0275229357798166,
+      "grad_norm": 0.6349695920944214,
+      "learning_rate": 0.0001,
+      "loss": 0.9139,
+      "step": 660
+    },
+    {
+      "epoch": 3.073394495412844,
+      "grad_norm": 0.65309077501297,
+      "learning_rate": 0.0001,
+      "loss": 0.8632,
+      "step": 670
+    },
+    {
+      "epoch": 3.1192660550458715,
+      "grad_norm": 0.6830633878707886,
+      "learning_rate": 0.0001,
+      "loss": 0.9452,
+      "step": 680
+    },
+    {
+      "epoch": 3.165137614678899,
+      "grad_norm": 0.7120541334152222,
+      "learning_rate": 0.0001,
+      "loss": 0.9386,
+      "step": 690
+    },
+    {
+      "epoch": 3.2110091743119265,
+      "grad_norm": 0.6665357947349548,
+      "learning_rate": 0.0001,
+      "loss": 0.8971,
+      "step": 700
+    },
+    {
+      "epoch": 3.2568807339449544,
+      "grad_norm": 0.6179565191268921,
+      "learning_rate": 0.0001,
+      "loss": 0.8872,
+      "step": 710
+    },
+    {
+      "epoch": 3.302752293577982,
+      "grad_norm": 0.7118347883224487,
+      "learning_rate": 0.0001,
+      "loss": 0.9486,
+      "step": 720
+    },
+    {
+      "epoch": 3.3486238532110093,
+      "grad_norm": 0.6336777806282043,
+      "learning_rate": 0.0001,
+      "loss": 0.9389,
+      "step": 730
+    },
+    {
+      "epoch": 3.3944954128440368,
+      "grad_norm": 0.7117331027984619,
+      "learning_rate": 0.0001,
+      "loss": 0.89,
+      "step": 740
+    },
+    {
+      "epoch": 3.4403669724770642,
+      "grad_norm": 0.7455915808677673,
+      "learning_rate": 0.0001,
+      "loss": 0.8708,
+      "step": 750
+    },
+    {
+      "epoch": 3.4862385321100917,
+      "grad_norm": 0.7114333510398865,
+      "learning_rate": 0.0001,
+      "loss": 0.9671,
+      "step": 760
+    },
+    {
+      "epoch": 3.532110091743119,
+      "grad_norm": 0.9367598295211792,
+      "learning_rate": 0.0001,
+      "loss": 0.9005,
+      "step": 770
+    },
+    {
+      "epoch": 3.5779816513761467,
+      "grad_norm": 0.6726359128952026,
+      "learning_rate": 0.0001,
+      "loss": 0.8876,
+      "step": 780
+    },
+    {
+      "epoch": 3.623853211009174,
+      "grad_norm": 0.7526884078979492,
+      "learning_rate": 0.0001,
+      "loss": 0.9137,
+      "step": 790
+    },
+    {
+      "epoch": 3.669724770642202,
+      "grad_norm": 0.8159477114677429,
+      "learning_rate": 0.0001,
+      "loss": 0.9261,
+      "step": 800
+    },
+    {
+      "epoch": 3.7155963302752295,
+      "grad_norm": 0.630791425704956,
+      "learning_rate": 0.0001,
+      "loss": 0.9389,
+      "step": 810
+    },
+    {
+      "epoch": 3.761467889908257,
+      "grad_norm": 0.7433311343193054,
+      "learning_rate": 0.0001,
+      "loss": 0.91,
+      "step": 820
+    },
+    {
+      "epoch": 3.8073394495412844,
+      "grad_norm": 0.7488676905632019,
+      "learning_rate": 0.0001,
+      "loss": 0.9171,
+      "step": 830
+    },
+    {
+      "epoch": 3.853211009174312,
+      "grad_norm": 0.7516036629676819,
+      "learning_rate": 0.0001,
+      "loss": 0.9384,
+      "step": 840
+    },
+    {
+      "epoch": 3.8990825688073394,
+      "grad_norm": 0.705337643623352,
+      "learning_rate": 0.0001,
+      "loss": 0.9306,
+      "step": 850
+    },
+    {
+      "epoch": 3.944954128440367,
+      "grad_norm": 0.8075425624847412,
+      "learning_rate": 0.0001,
+      "loss": 0.9641,
+      "step": 860
+    },
+    {
+      "epoch": 3.9908256880733948,
+      "grad_norm": 0.6263691782951355,
+      "learning_rate": 0.0001,
+      "loss": 0.8892,
+      "step": 870
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.0362694263458252,
+      "eval_runtime": 8.3051,
+      "eval_samples_per_second": 13.124,
+      "eval_steps_per_second": 1.686,
+      "step": 872
+    },
+    {
+      "epoch": 4.036697247706422,
+      "grad_norm": 0.6914173364639282,
+      "learning_rate": 0.0001,
+      "loss": 0.8789,
+      "step": 880
+    },
+    {
+      "epoch": 4.08256880733945,
+      "grad_norm": 0.8341014385223389,
+      "learning_rate": 0.0001,
+      "loss": 0.8824,
+      "step": 890
+    },
+    {
+      "epoch": 4.128440366972477,
+      "grad_norm": 1.025681972503662,
+      "learning_rate": 0.0001,
+      "loss": 0.8467,
+      "step": 900
+    },
+    {
+      "epoch": 4.174311926605505,
+      "grad_norm": 0.7425392270088196,
+      "learning_rate": 0.0001,
+      "loss": 0.8526,
+      "step": 910
+    },
+    {
+      "epoch": 4.220183486238533,
+      "grad_norm": 0.8017181158065796,
+      "learning_rate": 0.0001,
+      "loss": 0.8456,
+      "step": 920
+    },
+    {
+      "epoch": 4.26605504587156,
+      "grad_norm": 0.7509899139404297,
+      "learning_rate": 0.0001,
+      "loss": 0.8627,
+      "step": 930
+    },
+    {
+      "epoch": 4.3119266055045875,
+      "grad_norm": 0.8284080624580383,
+      "learning_rate": 0.0001,
+      "loss": 0.8489,
+      "step": 940
+    },
+    {
+      "epoch": 4.3577981651376145,
+      "grad_norm": 0.983587920665741,
+      "learning_rate": 0.0001,
+      "loss": 0.8805,
+      "step": 950
+    },
+    {
+      "epoch": 4.4036697247706424,
+      "grad_norm": 0.7172166109085083,
+      "learning_rate": 0.0001,
+      "loss": 0.8659,
+      "step": 960
+    },
+    {
+      "epoch": 4.4495412844036695,
+      "grad_norm": 0.8484768271446228,
+      "learning_rate": 0.0001,
+      "loss": 0.8489,
+      "step": 970
+    },
+    {
+      "epoch": 4.495412844036697,
+      "grad_norm": 1.2530163526535034,
+      "learning_rate": 0.0001,
+      "loss": 0.8486,
+      "step": 980
+    },
+    {
+      "epoch": 4.541284403669724,
+      "grad_norm": 0.6953453421592712,
+      "learning_rate": 0.0001,
+      "loss": 0.8712,
+      "step": 990
+    },
+    {
+      "epoch": 4.587155963302752,
+      "grad_norm": 0.9675477743148804,
+      "learning_rate": 0.0001,
+      "loss": 0.8496,
+      "step": 1000
+    },
+    {
+      "epoch": 4.63302752293578,
+      "grad_norm": 0.6756150126457214,
+      "learning_rate": 0.0001,
+      "loss": 0.8696,
+      "step": 1010
+    },
+    {
+      "epoch": 4.678899082568807,
+      "grad_norm": 0.7769085764884949,
+      "learning_rate": 0.0001,
+      "loss": 0.8433,
+      "step": 1020
+    },
+    {
+      "epoch": 4.724770642201835,
+      "grad_norm": 0.7071849703788757,
+      "learning_rate": 0.0001,
+      "loss": 0.8315,
+      "step": 1030
+    },
+    {
+      "epoch": 4.770642201834862,
+      "grad_norm": 0.7788539528846741,
+      "learning_rate": 0.0001,
+      "loss": 0.8305,
+      "step": 1040
+    },
+    {
+      "epoch": 4.81651376146789,
+      "grad_norm": 0.8174529671669006,
+      "learning_rate": 0.0001,
+      "loss": 0.793,
+      "step": 1050
+    },
+    {
+      "epoch": 4.862385321100917,
+      "grad_norm": 0.9944226145744324,
+      "learning_rate": 0.0001,
+      "loss": 0.8871,
+      "step": 1060
+    },
+    {
+      "epoch": 4.908256880733945,
+      "grad_norm": 1.0104237794876099,
+      "learning_rate": 0.0001,
+      "loss": 0.8497,
+      "step": 1070
+    },
+    {
+      "epoch": 4.954128440366972,
+      "grad_norm": 0.7839273810386658,
+      "learning_rate": 0.0001,
+      "loss": 0.8683,
+      "step": 1080
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 1.135035753250122,
+      "learning_rate": 0.0001,
+      "loss": 0.8428,
+      "step": 1090
+    },
+    {
+      "epoch": 5.0,
+      "eval_loss": 1.0589717626571655,
+      "eval_runtime": 8.3188,
+      "eval_samples_per_second": 13.103,
+      "eval_steps_per_second": 1.683,
+      "step": 1090
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1744,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.044275939115008e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1090/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fbd275a27ab08d2e5458e423282c9f72d2ca0deb32b3380f16bbc843fea6e0d
+size 5560

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1308/README.md

@@ -0,0 +1,202 @@
+---
+base_model: meta-llama/Meta-Llama-3-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1308/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1308/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3d9c2d8348c62a452057f6df334661a4f6a482b3e538da3d22d81b24315e54b0
+size 109069176

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1308/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2acdd4efbb3267c1845452c95a33497b503f2ec7ebbd9a81526092bbff2d3525
+size 55532666

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1308/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:edada20bdb2b0df7ce8b42a5fb4ac2b9171ef5d50fe52613fea94124231d7f31
+size 14244

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1308/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:30005049ea217ed44bc29fa8d61cff1587fd682ff3484a96392a22e295aea7bf
+size 1064

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1308/special_tokens_map.json

@@ -0,0 +1,17 @@
+{
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|eot_id|>"
+}

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1308/tokenizer_config.json

@@ -0,0 +1,2063 @@
+{
+  "added_tokens_decoder": {
+    "128000": {
+      "content": "<|begin_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128001": {
+      "content": "<|end_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128002": {
+      "content": "<|reserved_special_token_0|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128003": {
+      "content": "<|reserved_special_token_1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128004": {
+      "content": "<|reserved_special_token_2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128005": {
+      "content": "<|reserved_special_token_3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128006": {
+      "content": "<|start_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128007": {
+      "content": "<|end_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128008": {
+      "content": "<|reserved_special_token_4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128009": {
+      "content": "<|eot_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128010": {
+      "content": "<|reserved_special_token_5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128011": {
+      "content": "<|reserved_special_token_6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128012": {
+      "content": "<|reserved_special_token_7|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128013": {
+      "content": "<|reserved_special_token_8|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128014": {
+      "content": "<|reserved_special_token_9|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128015": {
+      "content": "<|reserved_special_token_10|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128016": {
+      "content": "<|reserved_special_token_11|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128017": {
+      "content": "<|reserved_special_token_12|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128018": {
+      "content": "<|reserved_special_token_13|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128019": {
+      "content": "<|reserved_special_token_14|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128020": {
+      "content": "<|reserved_special_token_15|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128021": {
+      "content": "<|reserved_special_token_16|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128022": {
+      "content": "<|reserved_special_token_17|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128023": {
+      "content": "<|reserved_special_token_18|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128024": {
+      "content": "<|reserved_special_token_19|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128025": {
+      "content": "<|reserved_special_token_20|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128026": {
+      "content": "<|reserved_special_token_21|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128027": {
+      "content": "<|reserved_special_token_22|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128028": {
+      "content": "<|reserved_special_token_23|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128029": {
+      "content": "<|reserved_special_token_24|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128030": {
+      "content": "<|reserved_special_token_25|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128031": {
+      "content": "<|reserved_special_token_26|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128032": {
+      "content": "<|reserved_special_token_27|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128033": {
+      "content": "<|reserved_special_token_28|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128034": {
+      "content": "<|reserved_special_token_29|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128035": {
+      "content": "<|reserved_special_token_30|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128036": {
+      "content": "<|reserved_special_token_31|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128037": {
+      "content": "<|reserved_special_token_32|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128038": {
+      "content": "<|reserved_special_token_33|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128039": {
+      "content": "<|reserved_special_token_34|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128040": {
+      "content": "<|reserved_special_token_35|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128041": {
+      "content": "<|reserved_special_token_36|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128042": {
+      "content": "<|reserved_special_token_37|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128043": {
+      "content": "<|reserved_special_token_38|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128044": {
+      "content": "<|reserved_special_token_39|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128045": {
+      "content": "<|reserved_special_token_40|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128046": {
+      "content": "<|reserved_special_token_41|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128047": {
+      "content": "<|reserved_special_token_42|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128048": {
+      "content": "<|reserved_special_token_43|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128049": {
+      "content": "<|reserved_special_token_44|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128050": {
+      "content": "<|reserved_special_token_45|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128051": {
+      "content": "<|reserved_special_token_46|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128052": {
+      "content": "<|reserved_special_token_47|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128053": {
+      "content": "<|reserved_special_token_48|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128054": {
+      "content": "<|reserved_special_token_49|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128055": {
+      "content": "<|reserved_special_token_50|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128056": {
+      "content": "<|reserved_special_token_51|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128057": {
+      "content": "<|reserved_special_token_52|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128058": {
+      "content": "<|reserved_special_token_53|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128059": {
+      "content": "<|reserved_special_token_54|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128060": {
+      "content": "<|reserved_special_token_55|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128061": {
+      "content": "<|reserved_special_token_56|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128062": {
+      "content": "<|reserved_special_token_57|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128063": {
+      "content": "<|reserved_special_token_58|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128064": {
+      "content": "<|reserved_special_token_59|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128065": {
+      "content": "<|reserved_special_token_60|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128066": {
+      "content": "<|reserved_special_token_61|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128067": {
+      "content": "<|reserved_special_token_62|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128068": {
+      "content": "<|reserved_special_token_63|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128069": {
+      "content": "<|reserved_special_token_64|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128070": {
+      "content": "<|reserved_special_token_65|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128071": {
+      "content": "<|reserved_special_token_66|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128072": {
+      "content": "<|reserved_special_token_67|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128073": {
+      "content": "<|reserved_special_token_68|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128074": {
+      "content": "<|reserved_special_token_69|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128075": {
+      "content": "<|reserved_special_token_70|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128076": {
+      "content": "<|reserved_special_token_71|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128077": {
+      "content": "<|reserved_special_token_72|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128078": {
+      "content": "<|reserved_special_token_73|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128079": {
+      "content": "<|reserved_special_token_74|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128080": {
+      "content": "<|reserved_special_token_75|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128081": {
+      "content": "<|reserved_special_token_76|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128082": {
+      "content": "<|reserved_special_token_77|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128083": {
+      "content": "<|reserved_special_token_78|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128084": {
+      "content": "<|reserved_special_token_79|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128085": {
+      "content": "<|reserved_special_token_80|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128086": {
+      "content": "<|reserved_special_token_81|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128087": {
+      "content": "<|reserved_special_token_82|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128088": {
+      "content": "<|reserved_special_token_83|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128089": {
+      "content": "<|reserved_special_token_84|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128090": {
+      "content": "<|reserved_special_token_85|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128091": {
+      "content": "<|reserved_special_token_86|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128092": {
+      "content": "<|reserved_special_token_87|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128093": {
+      "content": "<|reserved_special_token_88|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128094": {
+      "content": "<|reserved_special_token_89|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128095": {
+      "content": "<|reserved_special_token_90|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128096": {
+      "content": "<|reserved_special_token_91|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128097": {
+      "content": "<|reserved_special_token_92|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128098": {
+      "content": "<|reserved_special_token_93|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128099": {
+      "content": "<|reserved_special_token_94|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128100": {
+      "content": "<|reserved_special_token_95|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128101": {
+      "content": "<|reserved_special_token_96|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128102": {
+      "content": "<|reserved_special_token_97|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128103": {
+      "content": "<|reserved_special_token_98|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128104": {
+      "content": "<|reserved_special_token_99|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128105": {
+      "content": "<|reserved_special_token_100|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128106": {
+      "content": "<|reserved_special_token_101|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128107": {
+      "content": "<|reserved_special_token_102|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128108": {
+      "content": "<|reserved_special_token_103|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128109": {
+      "content": "<|reserved_special_token_104|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128110": {
+      "content": "<|reserved_special_token_105|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128111": {
+      "content": "<|reserved_special_token_106|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128112": {
+      "content": "<|reserved_special_token_107|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128113": {
+      "content": "<|reserved_special_token_108|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128114": {
+      "content": "<|reserved_special_token_109|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128115": {
+      "content": "<|reserved_special_token_110|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128116": {
+      "content": "<|reserved_special_token_111|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128117": {
+      "content": "<|reserved_special_token_112|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128118": {
+      "content": "<|reserved_special_token_113|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128119": {
+      "content": "<|reserved_special_token_114|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128120": {
+      "content": "<|reserved_special_token_115|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128121": {
+      "content": "<|reserved_special_token_116|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128122": {
+      "content": "<|reserved_special_token_117|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128123": {
+      "content": "<|reserved_special_token_118|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128124": {
+      "content": "<|reserved_special_token_119|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128125": {
+      "content": "<|reserved_special_token_120|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128126": {
+      "content": "<|reserved_special_token_121|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128127": {
+      "content": "<|reserved_special_token_122|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128128": {
+      "content": "<|reserved_special_token_123|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128129": {
+      "content": "<|reserved_special_token_124|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128130": {
+      "content": "<|reserved_special_token_125|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128131": {
+      "content": "<|reserved_special_token_126|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128132": {
+      "content": "<|reserved_special_token_127|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128133": {
+      "content": "<|reserved_special_token_128|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128134": {
+      "content": "<|reserved_special_token_129|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128135": {
+      "content": "<|reserved_special_token_130|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128136": {
+      "content": "<|reserved_special_token_131|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128137": {
+      "content": "<|reserved_special_token_132|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128138": {
+      "content": "<|reserved_special_token_133|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128139": {
+      "content": "<|reserved_special_token_134|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128140": {
+      "content": "<|reserved_special_token_135|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128141": {
+      "content": "<|reserved_special_token_136|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128142": {
+      "content": "<|reserved_special_token_137|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128143": {
+      "content": "<|reserved_special_token_138|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128144": {
+      "content": "<|reserved_special_token_139|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128145": {
+      "content": "<|reserved_special_token_140|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128146": {
+      "content": "<|reserved_special_token_141|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128147": {
+      "content": "<|reserved_special_token_142|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128148": {
+      "content": "<|reserved_special_token_143|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128149": {
+      "content": "<|reserved_special_token_144|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128150": {
+      "content": "<|reserved_special_token_145|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128151": {
+      "content": "<|reserved_special_token_146|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128152": {
+      "content": "<|reserved_special_token_147|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128153": {
+      "content": "<|reserved_special_token_148|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128154": {
+      "content": "<|reserved_special_token_149|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128155": {
+      "content": "<|reserved_special_token_150|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128156": {
+      "content": "<|reserved_special_token_151|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128157": {
+      "content": "<|reserved_special_token_152|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128158": {
+      "content": "<|reserved_special_token_153|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128159": {
+      "content": "<|reserved_special_token_154|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128160": {
+      "content": "<|reserved_special_token_155|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128161": {
+      "content": "<|reserved_special_token_156|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128162": {
+      "content": "<|reserved_special_token_157|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128163": {
+      "content": "<|reserved_special_token_158|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128164": {
+      "content": "<|reserved_special_token_159|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128165": {
+      "content": "<|reserved_special_token_160|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128166": {
+      "content": "<|reserved_special_token_161|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128167": {
+      "content": "<|reserved_special_token_162|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128168": {
+      "content": "<|reserved_special_token_163|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128169": {
+      "content": "<|reserved_special_token_164|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128170": {
+      "content": "<|reserved_special_token_165|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128171": {
+      "content": "<|reserved_special_token_166|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128172": {
+      "content": "<|reserved_special_token_167|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128173": {
+      "content": "<|reserved_special_token_168|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128174": {
+      "content": "<|reserved_special_token_169|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128175": {
+      "content": "<|reserved_special_token_170|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128176": {
+      "content": "<|reserved_special_token_171|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128177": {
+      "content": "<|reserved_special_token_172|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128178": {
+      "content": "<|reserved_special_token_173|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128179": {
+      "content": "<|reserved_special_token_174|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128180": {
+      "content": "<|reserved_special_token_175|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128181": {
+      "content": "<|reserved_special_token_176|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128182": {
+      "content": "<|reserved_special_token_177|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128183": {
+      "content": "<|reserved_special_token_178|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128184": {
+      "content": "<|reserved_special_token_179|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128185": {
+      "content": "<|reserved_special_token_180|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128186": {
+      "content": "<|reserved_special_token_181|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128187": {
+      "content": "<|reserved_special_token_182|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128188": {
+      "content": "<|reserved_special_token_183|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128189": {
+      "content": "<|reserved_special_token_184|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128190": {
+      "content": "<|reserved_special_token_185|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128191": {
+      "content": "<|reserved_special_token_186|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128192": {
+      "content": "<|reserved_special_token_187|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128193": {
+      "content": "<|reserved_special_token_188|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128194": {
+      "content": "<|reserved_special_token_189|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128195": {
+      "content": "<|reserved_special_token_190|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128196": {
+      "content": "<|reserved_special_token_191|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128197": {
+      "content": "<|reserved_special_token_192|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128198": {
+      "content": "<|reserved_special_token_193|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128199": {
+      "content": "<|reserved_special_token_194|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128200": {
+      "content": "<|reserved_special_token_195|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128201": {
+      "content": "<|reserved_special_token_196|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128202": {
+      "content": "<|reserved_special_token_197|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128203": {
+      "content": "<|reserved_special_token_198|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128204": {
+      "content": "<|reserved_special_token_199|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128205": {
+      "content": "<|reserved_special_token_200|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128206": {
+      "content": "<|reserved_special_token_201|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128207": {
+      "content": "<|reserved_special_token_202|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128208": {
+      "content": "<|reserved_special_token_203|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128209": {
+      "content": "<|reserved_special_token_204|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128210": {
+      "content": "<|reserved_special_token_205|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128211": {
+      "content": "<|reserved_special_token_206|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128212": {
+      "content": "<|reserved_special_token_207|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128213": {
+      "content": "<|reserved_special_token_208|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128214": {
+      "content": "<|reserved_special_token_209|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128215": {
+      "content": "<|reserved_special_token_210|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128216": {
+      "content": "<|reserved_special_token_211|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128217": {
+      "content": "<|reserved_special_token_212|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128218": {
+      "content": "<|reserved_special_token_213|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128219": {
+      "content": "<|reserved_special_token_214|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128220": {
+      "content": "<|reserved_special_token_215|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128221": {
+      "content": "<|reserved_special_token_216|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128222": {
+      "content": "<|reserved_special_token_217|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128223": {
+      "content": "<|reserved_special_token_218|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128224": {
+      "content": "<|reserved_special_token_219|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128225": {
+      "content": "<|reserved_special_token_220|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128226": {
+      "content": "<|reserved_special_token_221|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128227": {
+      "content": "<|reserved_special_token_222|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128228": {
+      "content": "<|reserved_special_token_223|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128229": {
+      "content": "<|reserved_special_token_224|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128230": {
+      "content": "<|reserved_special_token_225|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128231": {
+      "content": "<|reserved_special_token_226|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128232": {
+      "content": "<|reserved_special_token_227|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128233": {
+      "content": "<|reserved_special_token_228|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128234": {
+      "content": "<|reserved_special_token_229|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128235": {
+      "content": "<|reserved_special_token_230|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128236": {
+      "content": "<|reserved_special_token_231|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128237": {
+      "content": "<|reserved_special_token_232|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128238": {
+      "content": "<|reserved_special_token_233|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128239": {
+      "content": "<|reserved_special_token_234|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128240": {
+      "content": "<|reserved_special_token_235|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128241": {
+      "content": "<|reserved_special_token_236|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128242": {
+      "content": "<|reserved_special_token_237|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128243": {
+      "content": "<|reserved_special_token_238|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128244": {
+      "content": "<|reserved_special_token_239|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128245": {
+      "content": "<|reserved_special_token_240|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128246": {
+      "content": "<|reserved_special_token_241|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128247": {
+      "content": "<|reserved_special_token_242|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128248": {
+      "content": "<|reserved_special_token_243|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128249": {
+      "content": "<|reserved_special_token_244|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128250": {
+      "content": "<|reserved_special_token_245|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128251": {
+      "content": "<|reserved_special_token_246|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128252": {
+      "content": "<|reserved_special_token_247|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128253": {
+      "content": "<|reserved_special_token_248|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128254": {
+      "content": "<|reserved_special_token_249|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128255": {
+      "content": "<|reserved_special_token_250|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|begin_of_text|>",
+  "chat_template": "{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|eot_id|>",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<|eot_id|>",
+  "tokenizer_class": "PreTrainedTokenizerFast"
+}

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1308/trainer_state.json

@@ -0,0 +1,991 @@
+{
+  "best_metric": 1.0338033437728882,
+  "best_model_checkpoint": "outputs-001/Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-654",
+  "epoch": 6.0,
+  "eval_steps": 10,
+  "global_step": 1308,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.045871559633027525,
+      "grad_norm": 0.7936908602714539,
+      "learning_rate": 0.0001,
+      "loss": 2.1542,
+      "step": 10
+    },
+    {
+      "epoch": 0.09174311926605505,
+      "grad_norm": 0.8541925549507141,
+      "learning_rate": 0.0001,
+      "loss": 1.7148,
+      "step": 20
+    },
+    {
+      "epoch": 0.13761467889908258,
+      "grad_norm": 0.6849298477172852,
+      "learning_rate": 0.0001,
+      "loss": 1.4102,
+      "step": 30
+    },
+    {
+      "epoch": 0.1834862385321101,
+      "grad_norm": 0.6280460953712463,
+      "learning_rate": 0.0001,
+      "loss": 1.2936,
+      "step": 40
+    },
+    {
+      "epoch": 0.22935779816513763,
+      "grad_norm": 0.699965238571167,
+      "learning_rate": 0.0001,
+      "loss": 1.2116,
+      "step": 50
+    },
+    {
+      "epoch": 0.27522935779816515,
+      "grad_norm": 0.5185195803642273,
+      "learning_rate": 0.0001,
+      "loss": 1.2337,
+      "step": 60
+    },
+    {
+      "epoch": 0.3211009174311927,
+      "grad_norm": 0.5821201801300049,
+      "learning_rate": 0.0001,
+      "loss": 1.2317,
+      "step": 70
+    },
+    {
+      "epoch": 0.3669724770642202,
+      "grad_norm": 0.4655168652534485,
+      "learning_rate": 0.0001,
+      "loss": 1.1355,
+      "step": 80
+    },
+    {
+      "epoch": 0.41284403669724773,
+      "grad_norm": 0.5342468023300171,
+      "learning_rate": 0.0001,
+      "loss": 1.1571,
+      "step": 90
+    },
+    {
+      "epoch": 0.45871559633027525,
+      "grad_norm": 0.42904630303382874,
+      "learning_rate": 0.0001,
+      "loss": 1.1333,
+      "step": 100
+    },
+    {
+      "epoch": 0.5045871559633027,
+      "grad_norm": 0.5210477709770203,
+      "learning_rate": 0.0001,
+      "loss": 1.1596,
+      "step": 110
+    },
+    {
+      "epoch": 0.5504587155963303,
+      "grad_norm": 0.38087233901023865,
+      "learning_rate": 0.0001,
+      "loss": 1.1089,
+      "step": 120
+    },
+    {
+      "epoch": 0.5963302752293578,
+      "grad_norm": 0.4819198250770569,
+      "learning_rate": 0.0001,
+      "loss": 1.1458,
+      "step": 130
+    },
+    {
+      "epoch": 0.6422018348623854,
+      "grad_norm": 0.6635904908180237,
+      "learning_rate": 0.0001,
+      "loss": 1.1834,
+      "step": 140
+    },
+    {
+      "epoch": 0.6880733944954128,
+      "grad_norm": 0.5125461220741272,
+      "learning_rate": 0.0001,
+      "loss": 1.07,
+      "step": 150
+    },
+    {
+      "epoch": 0.7339449541284404,
+      "grad_norm": 0.5035977959632874,
+      "learning_rate": 0.0001,
+      "loss": 1.093,
+      "step": 160
+    },
+    {
+      "epoch": 0.7798165137614679,
+      "grad_norm": 0.5379819869995117,
+      "learning_rate": 0.0001,
+      "loss": 1.0297,
+      "step": 170
+    },
+    {
+      "epoch": 0.8256880733944955,
+      "grad_norm": 0.4491284191608429,
+      "learning_rate": 0.0001,
+      "loss": 1.1212,
+      "step": 180
+    },
+    {
+      "epoch": 0.8715596330275229,
+      "grad_norm": 0.5257289409637451,
+      "learning_rate": 0.0001,
+      "loss": 1.139,
+      "step": 190
+    },
+    {
+      "epoch": 0.9174311926605505,
+      "grad_norm": 0.5361109375953674,
+      "learning_rate": 0.0001,
+      "loss": 1.0379,
+      "step": 200
+    },
+    {
+      "epoch": 0.963302752293578,
+      "grad_norm": 0.5124537348747253,
+      "learning_rate": 0.0001,
+      "loss": 1.0588,
+      "step": 210
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.070561170578003,
+      "eval_runtime": 8.31,
+      "eval_samples_per_second": 13.117,
+      "eval_steps_per_second": 1.685,
+      "step": 218
+    },
+    {
+      "epoch": 1.0091743119266054,
+      "grad_norm": 0.44702231884002686,
+      "learning_rate": 0.0001,
+      "loss": 1.0394,
+      "step": 220
+    },
+    {
+      "epoch": 1.0550458715596331,
+      "grad_norm": 0.48708081245422363,
+      "learning_rate": 0.0001,
+      "loss": 1.1075,
+      "step": 230
+    },
+    {
+      "epoch": 1.1009174311926606,
+      "grad_norm": 0.6492213010787964,
+      "learning_rate": 0.0001,
+      "loss": 1.0769,
+      "step": 240
+    },
+    {
+      "epoch": 1.146788990825688,
+      "grad_norm": 0.48868921399116516,
+      "learning_rate": 0.0001,
+      "loss": 1.0165,
+      "step": 250
+    },
+    {
+      "epoch": 1.1926605504587156,
+      "grad_norm": 0.6291283369064331,
+      "learning_rate": 0.0001,
+      "loss": 0.9982,
+      "step": 260
+    },
+    {
+      "epoch": 1.238532110091743,
+      "grad_norm": 0.6115853190422058,
+      "learning_rate": 0.0001,
+      "loss": 1.0501,
+      "step": 270
+    },
+    {
+      "epoch": 1.2844036697247707,
+      "grad_norm": 0.6975598931312561,
+      "learning_rate": 0.0001,
+      "loss": 1.0874,
+      "step": 280
+    },
+    {
+      "epoch": 1.3302752293577982,
+      "grad_norm": 0.5336413383483887,
+      "learning_rate": 0.0001,
+      "loss": 1.0721,
+      "step": 290
+    },
+    {
+      "epoch": 1.3761467889908257,
+      "grad_norm": 0.5163336992263794,
+      "learning_rate": 0.0001,
+      "loss": 1.0405,
+      "step": 300
+    },
+    {
+      "epoch": 1.4220183486238533,
+      "grad_norm": 0.6026386022567749,
+      "learning_rate": 0.0001,
+      "loss": 1.0119,
+      "step": 310
+    },
+    {
+      "epoch": 1.4678899082568808,
+      "grad_norm": 0.6290488839149475,
+      "learning_rate": 0.0001,
+      "loss": 1.0327,
+      "step": 320
+    },
+    {
+      "epoch": 1.5137614678899083,
+      "grad_norm": 0.5215861797332764,
+      "learning_rate": 0.0001,
+      "loss": 1.0725,
+      "step": 330
+    },
+    {
+      "epoch": 1.5596330275229358,
+      "grad_norm": 0.6104291677474976,
+      "learning_rate": 0.0001,
+      "loss": 1.0364,
+      "step": 340
+    },
+    {
+      "epoch": 1.6055045871559632,
+      "grad_norm": 0.5163600444793701,
+      "learning_rate": 0.0001,
+      "loss": 1.0412,
+      "step": 350
+    },
+    {
+      "epoch": 1.6513761467889907,
+      "grad_norm": 0.4934627413749695,
+      "learning_rate": 0.0001,
+      "loss": 1.0661,
+      "step": 360
+    },
+    {
+      "epoch": 1.6972477064220184,
+      "grad_norm": 0.6802699565887451,
+      "learning_rate": 0.0001,
+      "loss": 1.0644,
+      "step": 370
+    },
+    {
+      "epoch": 1.7431192660550459,
+      "grad_norm": 0.5180800557136536,
+      "learning_rate": 0.0001,
+      "loss": 1.0175,
+      "step": 380
+    },
+    {
+      "epoch": 1.7889908256880735,
+      "grad_norm": 0.6208664178848267,
+      "learning_rate": 0.0001,
+      "loss": 1.0099,
+      "step": 390
+    },
+    {
+      "epoch": 1.834862385321101,
+      "grad_norm": 0.5667423605918884,
+      "learning_rate": 0.0001,
+      "loss": 0.9917,
+      "step": 400
+    },
+    {
+      "epoch": 1.8807339449541285,
+      "grad_norm": 0.7282969355583191,
+      "learning_rate": 0.0001,
+      "loss": 1.0028,
+      "step": 410
+    },
+    {
+      "epoch": 1.926605504587156,
+      "grad_norm": 0.5308305025100708,
+      "learning_rate": 0.0001,
+      "loss": 1.062,
+      "step": 420
+    },
+    {
+      "epoch": 1.9724770642201834,
+      "grad_norm": 0.6123433113098145,
+      "learning_rate": 0.0001,
+      "loss": 1.012,
+      "step": 430
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.0371460914611816,
+      "eval_runtime": 8.3019,
+      "eval_samples_per_second": 13.13,
+      "eval_steps_per_second": 1.686,
+      "step": 436
+    },
+    {
+      "epoch": 2.018348623853211,
+      "grad_norm": 0.5201505422592163,
+      "learning_rate": 0.0001,
+      "loss": 1.0126,
+      "step": 440
+    },
+    {
+      "epoch": 2.0642201834862384,
+      "grad_norm": 0.5720698237419128,
+      "learning_rate": 0.0001,
+      "loss": 0.9625,
+      "step": 450
+    },
+    {
+      "epoch": 2.1100917431192663,
+      "grad_norm": 0.6407686471939087,
+      "learning_rate": 0.0001,
+      "loss": 0.9256,
+      "step": 460
+    },
+    {
+      "epoch": 2.1559633027522938,
+      "grad_norm": 0.5237308740615845,
+      "learning_rate": 0.0001,
+      "loss": 0.9982,
+      "step": 470
+    },
+    {
+      "epoch": 2.2018348623853212,
+      "grad_norm": 0.6064776182174683,
+      "learning_rate": 0.0001,
+      "loss": 0.9806,
+      "step": 480
+    },
+    {
+      "epoch": 2.2477064220183487,
+      "grad_norm": 0.5534603595733643,
+      "learning_rate": 0.0001,
+      "loss": 0.9223,
+      "step": 490
+    },
+    {
+      "epoch": 2.293577981651376,
+      "grad_norm": 0.610197126865387,
+      "learning_rate": 0.0001,
+      "loss": 0.9339,
+      "step": 500
+    },
+    {
+      "epoch": 2.3394495412844036,
+      "grad_norm": 0.637509286403656,
+      "learning_rate": 0.0001,
+      "loss": 0.9669,
+      "step": 510
+    },
+    {
+      "epoch": 2.385321100917431,
+      "grad_norm": 0.6705873608589172,
+      "learning_rate": 0.0001,
+      "loss": 0.9798,
+      "step": 520
+    },
+    {
+      "epoch": 2.4311926605504586,
+      "grad_norm": 0.6272872090339661,
+      "learning_rate": 0.0001,
+      "loss": 0.9868,
+      "step": 530
+    },
+    {
+      "epoch": 2.477064220183486,
+      "grad_norm": 0.6892614364624023,
+      "learning_rate": 0.0001,
+      "loss": 1.0059,
+      "step": 540
+    },
+    {
+      "epoch": 2.522935779816514,
+      "grad_norm": 0.6316735744476318,
+      "learning_rate": 0.0001,
+      "loss": 0.9838,
+      "step": 550
+    },
+    {
+      "epoch": 2.5688073394495414,
+      "grad_norm": 0.5902363061904907,
+      "learning_rate": 0.0001,
+      "loss": 0.9548,
+      "step": 560
+    },
+    {
+      "epoch": 2.614678899082569,
+      "grad_norm": 0.6575316786766052,
+      "learning_rate": 0.0001,
+      "loss": 0.9859,
+      "step": 570
+    },
+    {
+      "epoch": 2.6605504587155964,
+      "grad_norm": 0.574419379234314,
+      "learning_rate": 0.0001,
+      "loss": 0.9976,
+      "step": 580
+    },
+    {
+      "epoch": 2.706422018348624,
+      "grad_norm": 0.7602782249450684,
+      "learning_rate": 0.0001,
+      "loss": 1.0267,
+      "step": 590
+    },
+    {
+      "epoch": 2.7522935779816513,
+      "grad_norm": 0.5764957070350647,
+      "learning_rate": 0.0001,
+      "loss": 0.9966,
+      "step": 600
+    },
+    {
+      "epoch": 2.7981651376146788,
+      "grad_norm": 0.68736332654953,
+      "learning_rate": 0.0001,
+      "loss": 0.9717,
+      "step": 610
+    },
+    {
+      "epoch": 2.8440366972477067,
+      "grad_norm": 0.5835479497909546,
+      "learning_rate": 0.0001,
+      "loss": 0.9393,
+      "step": 620
+    },
+    {
+      "epoch": 2.8899082568807337,
+      "grad_norm": 0.7003713250160217,
+      "learning_rate": 0.0001,
+      "loss": 0.9751,
+      "step": 630
+    },
+    {
+      "epoch": 2.9357798165137616,
+      "grad_norm": 0.5811060070991516,
+      "learning_rate": 0.0001,
+      "loss": 0.9803,
+      "step": 640
+    },
+    {
+      "epoch": 2.981651376146789,
+      "grad_norm": 0.5394262671470642,
+      "learning_rate": 0.0001,
+      "loss": 1.0262,
+      "step": 650
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 1.0338033437728882,
+      "eval_runtime": 8.3,
+      "eval_samples_per_second": 13.133,
+      "eval_steps_per_second": 1.687,
+      "step": 654
+    },
+    {
+      "epoch": 3.0275229357798166,
+      "grad_norm": 0.6349695920944214,
+      "learning_rate": 0.0001,
+      "loss": 0.9139,
+      "step": 660
+    },
+    {
+      "epoch": 3.073394495412844,
+      "grad_norm": 0.65309077501297,
+      "learning_rate": 0.0001,
+      "loss": 0.8632,
+      "step": 670
+    },
+    {
+      "epoch": 3.1192660550458715,
+      "grad_norm": 0.6830633878707886,
+      "learning_rate": 0.0001,
+      "loss": 0.9452,
+      "step": 680
+    },
+    {
+      "epoch": 3.165137614678899,
+      "grad_norm": 0.7120541334152222,
+      "learning_rate": 0.0001,
+      "loss": 0.9386,
+      "step": 690
+    },
+    {
+      "epoch": 3.2110091743119265,
+      "grad_norm": 0.6665357947349548,
+      "learning_rate": 0.0001,
+      "loss": 0.8971,
+      "step": 700
+    },
+    {
+      "epoch": 3.2568807339449544,
+      "grad_norm": 0.6179565191268921,
+      "learning_rate": 0.0001,
+      "loss": 0.8872,
+      "step": 710
+    },
+    {
+      "epoch": 3.302752293577982,
+      "grad_norm": 0.7118347883224487,
+      "learning_rate": 0.0001,
+      "loss": 0.9486,
+      "step": 720
+    },
+    {
+      "epoch": 3.3486238532110093,
+      "grad_norm": 0.6336777806282043,
+      "learning_rate": 0.0001,
+      "loss": 0.9389,
+      "step": 730
+    },
+    {
+      "epoch": 3.3944954128440368,
+      "grad_norm": 0.7117331027984619,
+      "learning_rate": 0.0001,
+      "loss": 0.89,
+      "step": 740
+    },
+    {
+      "epoch": 3.4403669724770642,
+      "grad_norm": 0.7455915808677673,
+      "learning_rate": 0.0001,
+      "loss": 0.8708,
+      "step": 750
+    },
+    {
+      "epoch": 3.4862385321100917,
+      "grad_norm": 0.7114333510398865,
+      "learning_rate": 0.0001,
+      "loss": 0.9671,
+      "step": 760
+    },
+    {
+      "epoch": 3.532110091743119,
+      "grad_norm": 0.9367598295211792,
+      "learning_rate": 0.0001,
+      "loss": 0.9005,
+      "step": 770
+    },
+    {
+      "epoch": 3.5779816513761467,
+      "grad_norm": 0.6726359128952026,
+      "learning_rate": 0.0001,
+      "loss": 0.8876,
+      "step": 780
+    },
+    {
+      "epoch": 3.623853211009174,
+      "grad_norm": 0.7526884078979492,
+      "learning_rate": 0.0001,
+      "loss": 0.9137,
+      "step": 790
+    },
+    {
+      "epoch": 3.669724770642202,
+      "grad_norm": 0.8159477114677429,
+      "learning_rate": 0.0001,
+      "loss": 0.9261,
+      "step": 800
+    },
+    {
+      "epoch": 3.7155963302752295,
+      "grad_norm": 0.630791425704956,
+      "learning_rate": 0.0001,
+      "loss": 0.9389,
+      "step": 810
+    },
+    {
+      "epoch": 3.761467889908257,
+      "grad_norm": 0.7433311343193054,
+      "learning_rate": 0.0001,
+      "loss": 0.91,
+      "step": 820
+    },
+    {
+      "epoch": 3.8073394495412844,
+      "grad_norm": 0.7488676905632019,
+      "learning_rate": 0.0001,
+      "loss": 0.9171,
+      "step": 830
+    },
+    {
+      "epoch": 3.853211009174312,
+      "grad_norm": 0.7516036629676819,
+      "learning_rate": 0.0001,
+      "loss": 0.9384,
+      "step": 840
+    },
+    {
+      "epoch": 3.8990825688073394,
+      "grad_norm": 0.705337643623352,
+      "learning_rate": 0.0001,
+      "loss": 0.9306,
+      "step": 850
+    },
+    {
+      "epoch": 3.944954128440367,
+      "grad_norm": 0.8075425624847412,
+      "learning_rate": 0.0001,
+      "loss": 0.9641,
+      "step": 860
+    },
+    {
+      "epoch": 3.9908256880733948,
+      "grad_norm": 0.6263691782951355,
+      "learning_rate": 0.0001,
+      "loss": 0.8892,
+      "step": 870
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.0362694263458252,
+      "eval_runtime": 8.3051,
+      "eval_samples_per_second": 13.124,
+      "eval_steps_per_second": 1.686,
+      "step": 872
+    },
+    {
+      "epoch": 4.036697247706422,
+      "grad_norm": 0.6914173364639282,
+      "learning_rate": 0.0001,
+      "loss": 0.8789,
+      "step": 880
+    },
+    {
+      "epoch": 4.08256880733945,
+      "grad_norm": 0.8341014385223389,
+      "learning_rate": 0.0001,
+      "loss": 0.8824,
+      "step": 890
+    },
+    {
+      "epoch": 4.128440366972477,
+      "grad_norm": 1.025681972503662,
+      "learning_rate": 0.0001,
+      "loss": 0.8467,
+      "step": 900
+    },
+    {
+      "epoch": 4.174311926605505,
+      "grad_norm": 0.7425392270088196,
+      "learning_rate": 0.0001,
+      "loss": 0.8526,
+      "step": 910
+    },
+    {
+      "epoch": 4.220183486238533,
+      "grad_norm": 0.8017181158065796,
+      "learning_rate": 0.0001,
+      "loss": 0.8456,
+      "step": 920
+    },
+    {
+      "epoch": 4.26605504587156,
+      "grad_norm": 0.7509899139404297,
+      "learning_rate": 0.0001,
+      "loss": 0.8627,
+      "step": 930
+    },
+    {
+      "epoch": 4.3119266055045875,
+      "grad_norm": 0.8284080624580383,
+      "learning_rate": 0.0001,
+      "loss": 0.8489,
+      "step": 940
+    },
+    {
+      "epoch": 4.3577981651376145,
+      "grad_norm": 0.983587920665741,
+      "learning_rate": 0.0001,
+      "loss": 0.8805,
+      "step": 950
+    },
+    {
+      "epoch": 4.4036697247706424,
+      "grad_norm": 0.7172166109085083,
+      "learning_rate": 0.0001,
+      "loss": 0.8659,
+      "step": 960
+    },
+    {
+      "epoch": 4.4495412844036695,
+      "grad_norm": 0.8484768271446228,
+      "learning_rate": 0.0001,
+      "loss": 0.8489,
+      "step": 970
+    },
+    {
+      "epoch": 4.495412844036697,
+      "grad_norm": 1.2530163526535034,
+      "learning_rate": 0.0001,
+      "loss": 0.8486,
+      "step": 980
+    },
+    {
+      "epoch": 4.541284403669724,
+      "grad_norm": 0.6953453421592712,
+      "learning_rate": 0.0001,
+      "loss": 0.8712,
+      "step": 990
+    },
+    {
+      "epoch": 4.587155963302752,
+      "grad_norm": 0.9675477743148804,
+      "learning_rate": 0.0001,
+      "loss": 0.8496,
+      "step": 1000
+    },
+    {
+      "epoch": 4.63302752293578,
+      "grad_norm": 0.6756150126457214,
+      "learning_rate": 0.0001,
+      "loss": 0.8696,
+      "step": 1010
+    },
+    {
+      "epoch": 4.678899082568807,
+      "grad_norm": 0.7769085764884949,
+      "learning_rate": 0.0001,
+      "loss": 0.8433,
+      "step": 1020
+    },
+    {
+      "epoch": 4.724770642201835,
+      "grad_norm": 0.7071849703788757,
+      "learning_rate": 0.0001,
+      "loss": 0.8315,
+      "step": 1030
+    },
+    {
+      "epoch": 4.770642201834862,
+      "grad_norm": 0.7788539528846741,
+      "learning_rate": 0.0001,
+      "loss": 0.8305,
+      "step": 1040
+    },
+    {
+      "epoch": 4.81651376146789,
+      "grad_norm": 0.8174529671669006,
+      "learning_rate": 0.0001,
+      "loss": 0.793,
+      "step": 1050
+    },
+    {
+      "epoch": 4.862385321100917,
+      "grad_norm": 0.9944226145744324,
+      "learning_rate": 0.0001,
+      "loss": 0.8871,
+      "step": 1060
+    },
+    {
+      "epoch": 4.908256880733945,
+      "grad_norm": 1.0104237794876099,
+      "learning_rate": 0.0001,
+      "loss": 0.8497,
+      "step": 1070
+    },
+    {
+      "epoch": 4.954128440366972,
+      "grad_norm": 0.7839273810386658,
+      "learning_rate": 0.0001,
+      "loss": 0.8683,
+      "step": 1080
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 1.135035753250122,
+      "learning_rate": 0.0001,
+      "loss": 0.8428,
+      "step": 1090
+    },
+    {
+      "epoch": 5.0,
+      "eval_loss": 1.0589717626571655,
+      "eval_runtime": 8.3188,
+      "eval_samples_per_second": 13.103,
+      "eval_steps_per_second": 1.683,
+      "step": 1090
+    },
+    {
+      "epoch": 5.045871559633028,
+      "grad_norm": 0.8675414323806763,
+      "learning_rate": 0.0001,
+      "loss": 0.7846,
+      "step": 1100
+    },
+    {
+      "epoch": 5.091743119266055,
+      "grad_norm": 0.9152688980102539,
+      "learning_rate": 0.0001,
+      "loss": 0.7525,
+      "step": 1110
+    },
+    {
+      "epoch": 5.137614678899083,
+      "grad_norm": 1.057682752609253,
+      "learning_rate": 0.0001,
+      "loss": 0.8033,
+      "step": 1120
+    },
+    {
+      "epoch": 5.18348623853211,
+      "grad_norm": 0.9883639812469482,
+      "learning_rate": 0.0001,
+      "loss": 0.8064,
+      "step": 1130
+    },
+    {
+      "epoch": 5.229357798165138,
+      "grad_norm": 0.908318817615509,
+      "learning_rate": 0.0001,
+      "loss": 0.8043,
+      "step": 1140
+    },
+    {
+      "epoch": 5.275229357798165,
+      "grad_norm": 0.9290405511856079,
+      "learning_rate": 0.0001,
+      "loss": 0.7897,
+      "step": 1150
+    },
+    {
+      "epoch": 5.321100917431193,
+      "grad_norm": 1.0225027799606323,
+      "learning_rate": 0.0001,
+      "loss": 0.7678,
+      "step": 1160
+    },
+    {
+      "epoch": 5.36697247706422,
+      "grad_norm": 0.723305344581604,
+      "learning_rate": 0.0001,
+      "loss": 0.7998,
+      "step": 1170
+    },
+    {
+      "epoch": 5.412844036697248,
+      "grad_norm": 0.9404224753379822,
+      "learning_rate": 0.0001,
+      "loss": 0.7691,
+      "step": 1180
+    },
+    {
+      "epoch": 5.458715596330276,
+      "grad_norm": 1.2413196563720703,
+      "learning_rate": 0.0001,
+      "loss": 0.7553,
+      "step": 1190
+    },
+    {
+      "epoch": 5.504587155963303,
+      "grad_norm": 0.9953755140304565,
+      "learning_rate": 0.0001,
+      "loss": 0.7473,
+      "step": 1200
+    },
+    {
+      "epoch": 5.5504587155963305,
+      "grad_norm": 0.8769338130950928,
+      "learning_rate": 0.0001,
+      "loss": 0.7919,
+      "step": 1210
+    },
+    {
+      "epoch": 5.5963302752293576,
+      "grad_norm": 0.7918237447738647,
+      "learning_rate": 0.0001,
+      "loss": 0.7754,
+      "step": 1220
+    },
+    {
+      "epoch": 5.6422018348623855,
+      "grad_norm": 0.8943650126457214,
+      "learning_rate": 0.0001,
+      "loss": 0.7726,
+      "step": 1230
+    },
+    {
+      "epoch": 5.6880733944954125,
+      "grad_norm": 0.9250347018241882,
+      "learning_rate": 0.0001,
+      "loss": 0.8007,
+      "step": 1240
+    },
+    {
+      "epoch": 5.73394495412844,
+      "grad_norm": 1.034350037574768,
+      "learning_rate": 0.0001,
+      "loss": 0.8033,
+      "step": 1250
+    },
+    {
+      "epoch": 5.779816513761467,
+      "grad_norm": 1.0044519901275635,
+      "learning_rate": 0.0001,
+      "loss": 0.7616,
+      "step": 1260
+    },
+    {
+      "epoch": 5.825688073394495,
+      "grad_norm": 0.8655275702476501,
+      "learning_rate": 0.0001,
+      "loss": 0.8549,
+      "step": 1270
+    },
+    {
+      "epoch": 5.871559633027523,
+      "grad_norm": 0.8263447284698486,
+      "learning_rate": 0.0001,
+      "loss": 0.7996,
+      "step": 1280
+    },
+    {
+      "epoch": 5.91743119266055,
+      "grad_norm": 0.679572582244873,
+      "learning_rate": 0.0001,
+      "loss": 0.7603,
+      "step": 1290
+    },
+    {
+      "epoch": 5.963302752293578,
+      "grad_norm": 0.9471953511238098,
+      "learning_rate": 0.0001,
+      "loss": 0.7782,
+      "step": 1300
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 1.0743499994277954,
+      "eval_runtime": 8.2992,
+      "eval_samples_per_second": 13.134,
+      "eval_steps_per_second": 1.687,
+      "step": 1308
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1744,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 6.05313112693801e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1308/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fbd275a27ab08d2e5458e423282c9f72d2ca0deb32b3380f16bbc843fea6e0d
+size 5560

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1526/README.md

@@ -0,0 +1,202 @@
+---
+base_model: meta-llama/Meta-Llama-3-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1526/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1526/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:86b441496ac8bd9f8649a2c31b2246935cdb689d7ac826db4caecf86ad0efb54
+size 109069176

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1526/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5053d4a1da960996cd9d7add4f3913a166b66f343d4e0b3b3d801978bca0ca10
+size 55532666

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1526/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dcdbac8f40004a342dfbf76263406e449ae7332adf031b73ab2ef30d40b26744
+size 14244

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1526/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7cb6223c1f3c30c5e042264874721a8a773a42c2456fba853cba89b049d843d9
+size 1064

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1526/special_tokens_map.json

@@ -0,0 +1,17 @@
+{
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|eot_id|>"
+}

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1526/tokenizer_config.json

@@ -0,0 +1,2063 @@
+{
+  "added_tokens_decoder": {
+    "128000": {
+      "content": "<|begin_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128001": {
+      "content": "<|end_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128002": {
+      "content": "<|reserved_special_token_0|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128003": {
+      "content": "<|reserved_special_token_1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128004": {
+      "content": "<|reserved_special_token_2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128005": {
+      "content": "<|reserved_special_token_3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128006": {
+      "content": "<|start_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128007": {
+      "content": "<|end_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128008": {
+      "content": "<|reserved_special_token_4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128009": {
+      "content": "<|eot_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128010": {
+      "content": "<|reserved_special_token_5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128011": {
+      "content": "<|reserved_special_token_6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128012": {
+      "content": "<|reserved_special_token_7|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128013": {
+      "content": "<|reserved_special_token_8|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128014": {
+      "content": "<|reserved_special_token_9|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128015": {
+      "content": "<|reserved_special_token_10|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128016": {
+      "content": "<|reserved_special_token_11|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128017": {
+      "content": "<|reserved_special_token_12|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128018": {
+      "content": "<|reserved_special_token_13|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128019": {
+      "content": "<|reserved_special_token_14|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128020": {
+      "content": "<|reserved_special_token_15|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128021": {
+      "content": "<|reserved_special_token_16|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128022": {
+      "content": "<|reserved_special_token_17|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128023": {
+      "content": "<|reserved_special_token_18|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128024": {
+      "content": "<|reserved_special_token_19|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128025": {
+      "content": "<|reserved_special_token_20|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128026": {
+      "content": "<|reserved_special_token_21|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128027": {
+      "content": "<|reserved_special_token_22|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128028": {
+      "content": "<|reserved_special_token_23|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128029": {
+      "content": "<|reserved_special_token_24|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128030": {
+      "content": "<|reserved_special_token_25|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128031": {
+      "content": "<|reserved_special_token_26|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128032": {
+      "content": "<|reserved_special_token_27|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128033": {
+      "content": "<|reserved_special_token_28|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128034": {
+      "content": "<|reserved_special_token_29|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128035": {
+      "content": "<|reserved_special_token_30|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128036": {
+      "content": "<|reserved_special_token_31|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128037": {
+      "content": "<|reserved_special_token_32|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128038": {
+      "content": "<|reserved_special_token_33|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128039": {
+      "content": "<|reserved_special_token_34|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128040": {
+      "content": "<|reserved_special_token_35|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128041": {
+      "content": "<|reserved_special_token_36|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128042": {
+      "content": "<|reserved_special_token_37|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128043": {
+      "content": "<|reserved_special_token_38|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128044": {
+      "content": "<|reserved_special_token_39|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128045": {
+      "content": "<|reserved_special_token_40|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128046": {
+      "content": "<|reserved_special_token_41|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128047": {
+      "content": "<|reserved_special_token_42|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128048": {
+      "content": "<|reserved_special_token_43|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128049": {
+      "content": "<|reserved_special_token_44|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128050": {
+      "content": "<|reserved_special_token_45|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128051": {
+      "content": "<|reserved_special_token_46|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128052": {
+      "content": "<|reserved_special_token_47|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128053": {
+      "content": "<|reserved_special_token_48|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128054": {
+      "content": "<|reserved_special_token_49|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128055": {
+      "content": "<|reserved_special_token_50|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128056": {
+      "content": "<|reserved_special_token_51|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128057": {
+      "content": "<|reserved_special_token_52|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128058": {
+      "content": "<|reserved_special_token_53|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128059": {
+      "content": "<|reserved_special_token_54|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128060": {
+      "content": "<|reserved_special_token_55|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128061": {
+      "content": "<|reserved_special_token_56|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128062": {
+      "content": "<|reserved_special_token_57|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128063": {
+      "content": "<|reserved_special_token_58|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128064": {
+      "content": "<|reserved_special_token_59|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128065": {
+      "content": "<|reserved_special_token_60|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128066": {
+      "content": "<|reserved_special_token_61|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128067": {
+      "content": "<|reserved_special_token_62|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128068": {
+      "content": "<|reserved_special_token_63|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128069": {
+      "content": "<|reserved_special_token_64|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128070": {
+      "content": "<|reserved_special_token_65|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128071": {
+      "content": "<|reserved_special_token_66|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128072": {
+      "content": "<|reserved_special_token_67|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128073": {
+      "content": "<|reserved_special_token_68|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128074": {
+      "content": "<|reserved_special_token_69|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128075": {
+      "content": "<|reserved_special_token_70|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128076": {
+      "content": "<|reserved_special_token_71|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128077": {
+      "content": "<|reserved_special_token_72|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128078": {
+      "content": "<|reserved_special_token_73|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128079": {
+      "content": "<|reserved_special_token_74|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128080": {
+      "content": "<|reserved_special_token_75|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128081": {
+      "content": "<|reserved_special_token_76|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128082": {
+      "content": "<|reserved_special_token_77|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128083": {
+      "content": "<|reserved_special_token_78|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128084": {
+      "content": "<|reserved_special_token_79|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128085": {
+      "content": "<|reserved_special_token_80|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128086": {
+      "content": "<|reserved_special_token_81|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128087": {
+      "content": "<|reserved_special_token_82|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128088": {
+      "content": "<|reserved_special_token_83|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128089": {
+      "content": "<|reserved_special_token_84|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128090": {
+      "content": "<|reserved_special_token_85|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128091": {
+      "content": "<|reserved_special_token_86|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128092": {
+      "content": "<|reserved_special_token_87|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128093": {
+      "content": "<|reserved_special_token_88|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128094": {
+      "content": "<|reserved_special_token_89|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128095": {
+      "content": "<|reserved_special_token_90|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128096": {
+      "content": "<|reserved_special_token_91|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128097": {
+      "content": "<|reserved_special_token_92|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128098": {
+      "content": "<|reserved_special_token_93|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128099": {
+      "content": "<|reserved_special_token_94|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128100": {
+      "content": "<|reserved_special_token_95|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128101": {
+      "content": "<|reserved_special_token_96|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128102": {
+      "content": "<|reserved_special_token_97|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128103": {
+      "content": "<|reserved_special_token_98|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128104": {
+      "content": "<|reserved_special_token_99|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128105": {
+      "content": "<|reserved_special_token_100|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128106": {
+      "content": "<|reserved_special_token_101|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128107": {
+      "content": "<|reserved_special_token_102|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128108": {
+      "content": "<|reserved_special_token_103|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128109": {
+      "content": "<|reserved_special_token_104|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128110": {
+      "content": "<|reserved_special_token_105|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128111": {
+      "content": "<|reserved_special_token_106|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128112": {
+      "content": "<|reserved_special_token_107|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128113": {
+      "content": "<|reserved_special_token_108|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128114": {
+      "content": "<|reserved_special_token_109|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128115": {
+      "content": "<|reserved_special_token_110|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128116": {
+      "content": "<|reserved_special_token_111|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128117": {
+      "content": "<|reserved_special_token_112|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128118": {
+      "content": "<|reserved_special_token_113|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128119": {
+      "content": "<|reserved_special_token_114|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128120": {
+      "content": "<|reserved_special_token_115|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128121": {
+      "content": "<|reserved_special_token_116|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128122": {
+      "content": "<|reserved_special_token_117|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128123": {
+      "content": "<|reserved_special_token_118|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128124": {
+      "content": "<|reserved_special_token_119|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128125": {
+      "content": "<|reserved_special_token_120|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128126": {
+      "content": "<|reserved_special_token_121|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128127": {
+      "content": "<|reserved_special_token_122|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128128": {
+      "content": "<|reserved_special_token_123|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128129": {
+      "content": "<|reserved_special_token_124|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128130": {
+      "content": "<|reserved_special_token_125|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128131": {
+      "content": "<|reserved_special_token_126|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128132": {
+      "content": "<|reserved_special_token_127|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128133": {
+      "content": "<|reserved_special_token_128|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128134": {
+      "content": "<|reserved_special_token_129|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128135": {
+      "content": "<|reserved_special_token_130|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128136": {
+      "content": "<|reserved_special_token_131|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128137": {
+      "content": "<|reserved_special_token_132|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128138": {
+      "content": "<|reserved_special_token_133|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128139": {
+      "content": "<|reserved_special_token_134|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128140": {
+      "content": "<|reserved_special_token_135|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128141": {
+      "content": "<|reserved_special_token_136|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128142": {
+      "content": "<|reserved_special_token_137|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128143": {
+      "content": "<|reserved_special_token_138|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128144": {
+      "content": "<|reserved_special_token_139|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128145": {
+      "content": "<|reserved_special_token_140|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128146": {
+      "content": "<|reserved_special_token_141|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128147": {
+      "content": "<|reserved_special_token_142|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128148": {
+      "content": "<|reserved_special_token_143|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128149": {
+      "content": "<|reserved_special_token_144|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128150": {
+      "content": "<|reserved_special_token_145|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128151": {
+      "content": "<|reserved_special_token_146|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128152": {
+      "content": "<|reserved_special_token_147|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128153": {
+      "content": "<|reserved_special_token_148|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128154": {
+      "content": "<|reserved_special_token_149|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128155": {
+      "content": "<|reserved_special_token_150|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128156": {
+      "content": "<|reserved_special_token_151|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128157": {
+      "content": "<|reserved_special_token_152|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128158": {
+      "content": "<|reserved_special_token_153|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128159": {
+      "content": "<|reserved_special_token_154|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128160": {
+      "content": "<|reserved_special_token_155|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128161": {
+      "content": "<|reserved_special_token_156|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128162": {
+      "content": "<|reserved_special_token_157|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128163": {
+      "content": "<|reserved_special_token_158|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128164": {
+      "content": "<|reserved_special_token_159|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128165": {
+      "content": "<|reserved_special_token_160|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128166": {
+      "content": "<|reserved_special_token_161|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128167": {
+      "content": "<|reserved_special_token_162|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128168": {
+      "content": "<|reserved_special_token_163|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128169": {
+      "content": "<|reserved_special_token_164|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128170": {
+      "content": "<|reserved_special_token_165|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128171": {
+      "content": "<|reserved_special_token_166|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128172": {
+      "content": "<|reserved_special_token_167|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128173": {
+      "content": "<|reserved_special_token_168|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128174": {
+      "content": "<|reserved_special_token_169|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128175": {
+      "content": "<|reserved_special_token_170|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128176": {
+      "content": "<|reserved_special_token_171|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128177": {
+      "content": "<|reserved_special_token_172|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128178": {
+      "content": "<|reserved_special_token_173|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128179": {
+      "content": "<|reserved_special_token_174|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128180": {
+      "content": "<|reserved_special_token_175|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128181": {
+      "content": "<|reserved_special_token_176|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128182": {
+      "content": "<|reserved_special_token_177|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128183": {
+      "content": "<|reserved_special_token_178|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128184": {
+      "content": "<|reserved_special_token_179|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128185": {
+      "content": "<|reserved_special_token_180|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128186": {
+      "content": "<|reserved_special_token_181|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128187": {
+      "content": "<|reserved_special_token_182|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128188": {
+      "content": "<|reserved_special_token_183|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128189": {
+      "content": "<|reserved_special_token_184|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128190": {
+      "content": "<|reserved_special_token_185|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128191": {
+      "content": "<|reserved_special_token_186|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128192": {
+      "content": "<|reserved_special_token_187|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128193": {
+      "content": "<|reserved_special_token_188|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128194": {
+      "content": "<|reserved_special_token_189|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128195": {
+      "content": "<|reserved_special_token_190|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128196": {
+      "content": "<|reserved_special_token_191|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128197": {
+      "content": "<|reserved_special_token_192|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128198": {
+      "content": "<|reserved_special_token_193|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128199": {
+      "content": "<|reserved_special_token_194|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128200": {
+      "content": "<|reserved_special_token_195|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128201": {
+      "content": "<|reserved_special_token_196|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128202": {
+      "content": "<|reserved_special_token_197|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128203": {
+      "content": "<|reserved_special_token_198|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128204": {
+      "content": "<|reserved_special_token_199|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128205": {
+      "content": "<|reserved_special_token_200|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128206": {
+      "content": "<|reserved_special_token_201|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128207": {
+      "content": "<|reserved_special_token_202|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128208": {
+      "content": "<|reserved_special_token_203|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128209": {
+      "content": "<|reserved_special_token_204|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128210": {
+      "content": "<|reserved_special_token_205|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128211": {
+      "content": "<|reserved_special_token_206|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128212": {
+      "content": "<|reserved_special_token_207|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128213": {
+      "content": "<|reserved_special_token_208|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128214": {
+      "content": "<|reserved_special_token_209|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128215": {
+      "content": "<|reserved_special_token_210|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128216": {
+      "content": "<|reserved_special_token_211|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128217": {
+      "content": "<|reserved_special_token_212|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128218": {
+      "content": "<|reserved_special_token_213|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128219": {
+      "content": "<|reserved_special_token_214|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128220": {
+      "content": "<|reserved_special_token_215|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128221": {
+      "content": "<|reserved_special_token_216|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128222": {
+      "content": "<|reserved_special_token_217|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128223": {
+      "content": "<|reserved_special_token_218|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128224": {
+      "content": "<|reserved_special_token_219|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128225": {
+      "content": "<|reserved_special_token_220|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128226": {
+      "content": "<|reserved_special_token_221|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128227": {
+      "content": "<|reserved_special_token_222|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128228": {
+      "content": "<|reserved_special_token_223|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128229": {
+      "content": "<|reserved_special_token_224|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128230": {
+      "content": "<|reserved_special_token_225|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128231": {
+      "content": "<|reserved_special_token_226|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128232": {
+      "content": "<|reserved_special_token_227|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128233": {
+      "content": "<|reserved_special_token_228|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128234": {
+      "content": "<|reserved_special_token_229|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128235": {
+      "content": "<|reserved_special_token_230|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128236": {
+      "content": "<|reserved_special_token_231|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128237": {
+      "content": "<|reserved_special_token_232|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128238": {
+      "content": "<|reserved_special_token_233|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128239": {
+      "content": "<|reserved_special_token_234|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128240": {
+      "content": "<|reserved_special_token_235|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128241": {
+      "content": "<|reserved_special_token_236|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128242": {
+      "content": "<|reserved_special_token_237|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128243": {
+      "content": "<|reserved_special_token_238|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128244": {
+      "content": "<|reserved_special_token_239|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128245": {
+      "content": "<|reserved_special_token_240|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128246": {
+      "content": "<|reserved_special_token_241|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128247": {
+      "content": "<|reserved_special_token_242|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128248": {
+      "content": "<|reserved_special_token_243|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128249": {
+      "content": "<|reserved_special_token_244|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128250": {
+      "content": "<|reserved_special_token_245|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128251": {
+      "content": "<|reserved_special_token_246|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128252": {
+      "content": "<|reserved_special_token_247|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128253": {
+      "content": "<|reserved_special_token_248|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128254": {
+      "content": "<|reserved_special_token_249|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128255": {
+      "content": "<|reserved_special_token_250|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|begin_of_text|>",
+  "chat_template": "{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|eot_id|>",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<|eot_id|>",
+  "tokenizer_class": "PreTrainedTokenizerFast"
+}

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1526/trainer_state.json

@@ -0,0 +1,1153 @@
+{
+  "best_metric": 1.0338033437728882,
+  "best_model_checkpoint": "outputs-001/Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-654",
+  "epoch": 7.0,
+  "eval_steps": 10,
+  "global_step": 1526,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.045871559633027525,
+      "grad_norm": 0.7936908602714539,
+      "learning_rate": 0.0001,
+      "loss": 2.1542,
+      "step": 10
+    },
+    {
+      "epoch": 0.09174311926605505,
+      "grad_norm": 0.8541925549507141,
+      "learning_rate": 0.0001,
+      "loss": 1.7148,
+      "step": 20
+    },
+    {
+      "epoch": 0.13761467889908258,
+      "grad_norm": 0.6849298477172852,
+      "learning_rate": 0.0001,
+      "loss": 1.4102,
+      "step": 30
+    },
+    {
+      "epoch": 0.1834862385321101,
+      "grad_norm": 0.6280460953712463,
+      "learning_rate": 0.0001,
+      "loss": 1.2936,
+      "step": 40
+    },
+    {
+      "epoch": 0.22935779816513763,
+      "grad_norm": 0.699965238571167,
+      "learning_rate": 0.0001,
+      "loss": 1.2116,
+      "step": 50
+    },
+    {
+      "epoch": 0.27522935779816515,
+      "grad_norm": 0.5185195803642273,
+      "learning_rate": 0.0001,
+      "loss": 1.2337,
+      "step": 60
+    },
+    {
+      "epoch": 0.3211009174311927,
+      "grad_norm": 0.5821201801300049,
+      "learning_rate": 0.0001,
+      "loss": 1.2317,
+      "step": 70
+    },
+    {
+      "epoch": 0.3669724770642202,
+      "grad_norm": 0.4655168652534485,
+      "learning_rate": 0.0001,
+      "loss": 1.1355,
+      "step": 80
+    },
+    {
+      "epoch": 0.41284403669724773,
+      "grad_norm": 0.5342468023300171,
+      "learning_rate": 0.0001,
+      "loss": 1.1571,
+      "step": 90
+    },
+    {
+      "epoch": 0.45871559633027525,
+      "grad_norm": 0.42904630303382874,
+      "learning_rate": 0.0001,
+      "loss": 1.1333,
+      "step": 100
+    },
+    {
+      "epoch": 0.5045871559633027,
+      "grad_norm": 0.5210477709770203,
+      "learning_rate": 0.0001,
+      "loss": 1.1596,
+      "step": 110
+    },
+    {
+      "epoch": 0.5504587155963303,
+      "grad_norm": 0.38087233901023865,
+      "learning_rate": 0.0001,
+      "loss": 1.1089,
+      "step": 120
+    },
+    {
+      "epoch": 0.5963302752293578,
+      "grad_norm": 0.4819198250770569,
+      "learning_rate": 0.0001,
+      "loss": 1.1458,
+      "step": 130
+    },
+    {
+      "epoch": 0.6422018348623854,
+      "grad_norm": 0.6635904908180237,
+      "learning_rate": 0.0001,
+      "loss": 1.1834,
+      "step": 140
+    },
+    {
+      "epoch": 0.6880733944954128,
+      "grad_norm": 0.5125461220741272,
+      "learning_rate": 0.0001,
+      "loss": 1.07,
+      "step": 150
+    },
+    {
+      "epoch": 0.7339449541284404,
+      "grad_norm": 0.5035977959632874,
+      "learning_rate": 0.0001,
+      "loss": 1.093,
+      "step": 160
+    },
+    {
+      "epoch": 0.7798165137614679,
+      "grad_norm": 0.5379819869995117,
+      "learning_rate": 0.0001,
+      "loss": 1.0297,
+      "step": 170
+    },
+    {
+      "epoch": 0.8256880733944955,
+      "grad_norm": 0.4491284191608429,
+      "learning_rate": 0.0001,
+      "loss": 1.1212,
+      "step": 180
+    },
+    {
+      "epoch": 0.8715596330275229,
+      "grad_norm": 0.5257289409637451,
+      "learning_rate": 0.0001,
+      "loss": 1.139,
+      "step": 190
+    },
+    {
+      "epoch": 0.9174311926605505,
+      "grad_norm": 0.5361109375953674,
+      "learning_rate": 0.0001,
+      "loss": 1.0379,
+      "step": 200
+    },
+    {
+      "epoch": 0.963302752293578,
+      "grad_norm": 0.5124537348747253,
+      "learning_rate": 0.0001,
+      "loss": 1.0588,
+      "step": 210
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.070561170578003,
+      "eval_runtime": 8.31,
+      "eval_samples_per_second": 13.117,
+      "eval_steps_per_second": 1.685,
+      "step": 218
+    },
+    {
+      "epoch": 1.0091743119266054,
+      "grad_norm": 0.44702231884002686,
+      "learning_rate": 0.0001,
+      "loss": 1.0394,
+      "step": 220
+    },
+    {
+      "epoch": 1.0550458715596331,
+      "grad_norm": 0.48708081245422363,
+      "learning_rate": 0.0001,
+      "loss": 1.1075,
+      "step": 230
+    },
+    {
+      "epoch": 1.1009174311926606,
+      "grad_norm": 0.6492213010787964,
+      "learning_rate": 0.0001,
+      "loss": 1.0769,
+      "step": 240
+    },
+    {
+      "epoch": 1.146788990825688,
+      "grad_norm": 0.48868921399116516,
+      "learning_rate": 0.0001,
+      "loss": 1.0165,
+      "step": 250
+    },
+    {
+      "epoch": 1.1926605504587156,
+      "grad_norm": 0.6291283369064331,
+      "learning_rate": 0.0001,
+      "loss": 0.9982,
+      "step": 260
+    },
+    {
+      "epoch": 1.238532110091743,
+      "grad_norm": 0.6115853190422058,
+      "learning_rate": 0.0001,
+      "loss": 1.0501,
+      "step": 270
+    },
+    {
+      "epoch": 1.2844036697247707,
+      "grad_norm": 0.6975598931312561,
+      "learning_rate": 0.0001,
+      "loss": 1.0874,
+      "step": 280
+    },
+    {
+      "epoch": 1.3302752293577982,
+      "grad_norm": 0.5336413383483887,
+      "learning_rate": 0.0001,
+      "loss": 1.0721,
+      "step": 290
+    },
+    {
+      "epoch": 1.3761467889908257,
+      "grad_norm": 0.5163336992263794,
+      "learning_rate": 0.0001,
+      "loss": 1.0405,
+      "step": 300
+    },
+    {
+      "epoch": 1.4220183486238533,
+      "grad_norm": 0.6026386022567749,
+      "learning_rate": 0.0001,
+      "loss": 1.0119,
+      "step": 310
+    },
+    {
+      "epoch": 1.4678899082568808,
+      "grad_norm": 0.6290488839149475,
+      "learning_rate": 0.0001,
+      "loss": 1.0327,
+      "step": 320
+    },
+    {
+      "epoch": 1.5137614678899083,
+      "grad_norm": 0.5215861797332764,
+      "learning_rate": 0.0001,
+      "loss": 1.0725,
+      "step": 330
+    },
+    {
+      "epoch": 1.5596330275229358,
+      "grad_norm": 0.6104291677474976,
+      "learning_rate": 0.0001,
+      "loss": 1.0364,
+      "step": 340
+    },
+    {
+      "epoch": 1.6055045871559632,
+      "grad_norm": 0.5163600444793701,
+      "learning_rate": 0.0001,
+      "loss": 1.0412,
+      "step": 350
+    },
+    {
+      "epoch": 1.6513761467889907,
+      "grad_norm": 0.4934627413749695,
+      "learning_rate": 0.0001,
+      "loss": 1.0661,
+      "step": 360
+    },
+    {
+      "epoch": 1.6972477064220184,
+      "grad_norm": 0.6802699565887451,
+      "learning_rate": 0.0001,
+      "loss": 1.0644,
+      "step": 370
+    },
+    {
+      "epoch": 1.7431192660550459,
+      "grad_norm": 0.5180800557136536,
+      "learning_rate": 0.0001,
+      "loss": 1.0175,
+      "step": 380
+    },
+    {
+      "epoch": 1.7889908256880735,
+      "grad_norm": 0.6208664178848267,
+      "learning_rate": 0.0001,
+      "loss": 1.0099,
+      "step": 390
+    },
+    {
+      "epoch": 1.834862385321101,
+      "grad_norm": 0.5667423605918884,
+      "learning_rate": 0.0001,
+      "loss": 0.9917,
+      "step": 400
+    },
+    {
+      "epoch": 1.8807339449541285,
+      "grad_norm": 0.7282969355583191,
+      "learning_rate": 0.0001,
+      "loss": 1.0028,
+      "step": 410
+    },
+    {
+      "epoch": 1.926605504587156,
+      "grad_norm": 0.5308305025100708,
+      "learning_rate": 0.0001,
+      "loss": 1.062,
+      "step": 420
+    },
+    {
+      "epoch": 1.9724770642201834,
+      "grad_norm": 0.6123433113098145,
+      "learning_rate": 0.0001,
+      "loss": 1.012,
+      "step": 430
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.0371460914611816,
+      "eval_runtime": 8.3019,
+      "eval_samples_per_second": 13.13,
+      "eval_steps_per_second": 1.686,
+      "step": 436
+    },
+    {
+      "epoch": 2.018348623853211,
+      "grad_norm": 0.5201505422592163,
+      "learning_rate": 0.0001,
+      "loss": 1.0126,
+      "step": 440
+    },
+    {
+      "epoch": 2.0642201834862384,
+      "grad_norm": 0.5720698237419128,
+      "learning_rate": 0.0001,
+      "loss": 0.9625,
+      "step": 450
+    },
+    {
+      "epoch": 2.1100917431192663,
+      "grad_norm": 0.6407686471939087,
+      "learning_rate": 0.0001,
+      "loss": 0.9256,
+      "step": 460
+    },
+    {
+      "epoch": 2.1559633027522938,
+      "grad_norm": 0.5237308740615845,
+      "learning_rate": 0.0001,
+      "loss": 0.9982,
+      "step": 470
+    },
+    {
+      "epoch": 2.2018348623853212,
+      "grad_norm": 0.6064776182174683,
+      "learning_rate": 0.0001,
+      "loss": 0.9806,
+      "step": 480
+    },
+    {
+      "epoch": 2.2477064220183487,
+      "grad_norm": 0.5534603595733643,
+      "learning_rate": 0.0001,
+      "loss": 0.9223,
+      "step": 490
+    },
+    {
+      "epoch": 2.293577981651376,
+      "grad_norm": 0.610197126865387,
+      "learning_rate": 0.0001,
+      "loss": 0.9339,
+      "step": 500
+    },
+    {
+      "epoch": 2.3394495412844036,
+      "grad_norm": 0.637509286403656,
+      "learning_rate": 0.0001,
+      "loss": 0.9669,
+      "step": 510
+    },
+    {
+      "epoch": 2.385321100917431,
+      "grad_norm": 0.6705873608589172,
+      "learning_rate": 0.0001,
+      "loss": 0.9798,
+      "step": 520
+    },
+    {
+      "epoch": 2.4311926605504586,
+      "grad_norm": 0.6272872090339661,
+      "learning_rate": 0.0001,
+      "loss": 0.9868,
+      "step": 530
+    },
+    {
+      "epoch": 2.477064220183486,
+      "grad_norm": 0.6892614364624023,
+      "learning_rate": 0.0001,
+      "loss": 1.0059,
+      "step": 540
+    },
+    {
+      "epoch": 2.522935779816514,
+      "grad_norm": 0.6316735744476318,
+      "learning_rate": 0.0001,
+      "loss": 0.9838,
+      "step": 550
+    },
+    {
+      "epoch": 2.5688073394495414,
+      "grad_norm": 0.5902363061904907,
+      "learning_rate": 0.0001,
+      "loss": 0.9548,
+      "step": 560
+    },
+    {
+      "epoch": 2.614678899082569,
+      "grad_norm": 0.6575316786766052,
+      "learning_rate": 0.0001,
+      "loss": 0.9859,
+      "step": 570
+    },
+    {
+      "epoch": 2.6605504587155964,
+      "grad_norm": 0.574419379234314,
+      "learning_rate": 0.0001,
+      "loss": 0.9976,
+      "step": 580
+    },
+    {
+      "epoch": 2.706422018348624,
+      "grad_norm": 0.7602782249450684,
+      "learning_rate": 0.0001,
+      "loss": 1.0267,
+      "step": 590
+    },
+    {
+      "epoch": 2.7522935779816513,
+      "grad_norm": 0.5764957070350647,
+      "learning_rate": 0.0001,
+      "loss": 0.9966,
+      "step": 600
+    },
+    {
+      "epoch": 2.7981651376146788,
+      "grad_norm": 0.68736332654953,
+      "learning_rate": 0.0001,
+      "loss": 0.9717,
+      "step": 610
+    },
+    {
+      "epoch": 2.8440366972477067,
+      "grad_norm": 0.5835479497909546,
+      "learning_rate": 0.0001,
+      "loss": 0.9393,
+      "step": 620
+    },
+    {
+      "epoch": 2.8899082568807337,
+      "grad_norm": 0.7003713250160217,
+      "learning_rate": 0.0001,
+      "loss": 0.9751,
+      "step": 630
+    },
+    {
+      "epoch": 2.9357798165137616,
+      "grad_norm": 0.5811060070991516,
+      "learning_rate": 0.0001,
+      "loss": 0.9803,
+      "step": 640
+    },
+    {
+      "epoch": 2.981651376146789,
+      "grad_norm": 0.5394262671470642,
+      "learning_rate": 0.0001,
+      "loss": 1.0262,
+      "step": 650
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 1.0338033437728882,
+      "eval_runtime": 8.3,
+      "eval_samples_per_second": 13.133,
+      "eval_steps_per_second": 1.687,
+      "step": 654
+    },
+    {
+      "epoch": 3.0275229357798166,
+      "grad_norm": 0.6349695920944214,
+      "learning_rate": 0.0001,
+      "loss": 0.9139,
+      "step": 660
+    },
+    {
+      "epoch": 3.073394495412844,
+      "grad_norm": 0.65309077501297,
+      "learning_rate": 0.0001,
+      "loss": 0.8632,
+      "step": 670
+    },
+    {
+      "epoch": 3.1192660550458715,
+      "grad_norm": 0.6830633878707886,
+      "learning_rate": 0.0001,
+      "loss": 0.9452,
+      "step": 680
+    },
+    {
+      "epoch": 3.165137614678899,
+      "grad_norm": 0.7120541334152222,
+      "learning_rate": 0.0001,
+      "loss": 0.9386,
+      "step": 690
+    },
+    {
+      "epoch": 3.2110091743119265,
+      "grad_norm": 0.6665357947349548,
+      "learning_rate": 0.0001,
+      "loss": 0.8971,
+      "step": 700
+    },
+    {
+      "epoch": 3.2568807339449544,
+      "grad_norm": 0.6179565191268921,
+      "learning_rate": 0.0001,
+      "loss": 0.8872,
+      "step": 710
+    },
+    {
+      "epoch": 3.302752293577982,
+      "grad_norm": 0.7118347883224487,
+      "learning_rate": 0.0001,
+      "loss": 0.9486,
+      "step": 720
+    },
+    {
+      "epoch": 3.3486238532110093,
+      "grad_norm": 0.6336777806282043,
+      "learning_rate": 0.0001,
+      "loss": 0.9389,
+      "step": 730
+    },
+    {
+      "epoch": 3.3944954128440368,
+      "grad_norm": 0.7117331027984619,
+      "learning_rate": 0.0001,
+      "loss": 0.89,
+      "step": 740
+    },
+    {
+      "epoch": 3.4403669724770642,
+      "grad_norm": 0.7455915808677673,
+      "learning_rate": 0.0001,
+      "loss": 0.8708,
+      "step": 750
+    },
+    {
+      "epoch": 3.4862385321100917,
+      "grad_norm": 0.7114333510398865,
+      "learning_rate": 0.0001,
+      "loss": 0.9671,
+      "step": 760
+    },
+    {
+      "epoch": 3.532110091743119,
+      "grad_norm": 0.9367598295211792,
+      "learning_rate": 0.0001,
+      "loss": 0.9005,
+      "step": 770
+    },
+    {
+      "epoch": 3.5779816513761467,
+      "grad_norm": 0.6726359128952026,
+      "learning_rate": 0.0001,
+      "loss": 0.8876,
+      "step": 780
+    },
+    {
+      "epoch": 3.623853211009174,
+      "grad_norm": 0.7526884078979492,
+      "learning_rate": 0.0001,
+      "loss": 0.9137,
+      "step": 790
+    },
+    {
+      "epoch": 3.669724770642202,
+      "grad_norm": 0.8159477114677429,
+      "learning_rate": 0.0001,
+      "loss": 0.9261,
+      "step": 800
+    },
+    {
+      "epoch": 3.7155963302752295,
+      "grad_norm": 0.630791425704956,
+      "learning_rate": 0.0001,
+      "loss": 0.9389,
+      "step": 810
+    },
+    {
+      "epoch": 3.761467889908257,
+      "grad_norm": 0.7433311343193054,
+      "learning_rate": 0.0001,
+      "loss": 0.91,
+      "step": 820
+    },
+    {
+      "epoch": 3.8073394495412844,
+      "grad_norm": 0.7488676905632019,
+      "learning_rate": 0.0001,
+      "loss": 0.9171,
+      "step": 830
+    },
+    {
+      "epoch": 3.853211009174312,
+      "grad_norm": 0.7516036629676819,
+      "learning_rate": 0.0001,
+      "loss": 0.9384,
+      "step": 840
+    },
+    {
+      "epoch": 3.8990825688073394,
+      "grad_norm": 0.705337643623352,
+      "learning_rate": 0.0001,
+      "loss": 0.9306,
+      "step": 850
+    },
+    {
+      "epoch": 3.944954128440367,
+      "grad_norm": 0.8075425624847412,
+      "learning_rate": 0.0001,
+      "loss": 0.9641,
+      "step": 860
+    },
+    {
+      "epoch": 3.9908256880733948,
+      "grad_norm": 0.6263691782951355,
+      "learning_rate": 0.0001,
+      "loss": 0.8892,
+      "step": 870
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.0362694263458252,
+      "eval_runtime": 8.3051,
+      "eval_samples_per_second": 13.124,
+      "eval_steps_per_second": 1.686,
+      "step": 872
+    },
+    {
+      "epoch": 4.036697247706422,
+      "grad_norm": 0.6914173364639282,
+      "learning_rate": 0.0001,
+      "loss": 0.8789,
+      "step": 880
+    },
+    {
+      "epoch": 4.08256880733945,
+      "grad_norm": 0.8341014385223389,
+      "learning_rate": 0.0001,
+      "loss": 0.8824,
+      "step": 890
+    },
+    {
+      "epoch": 4.128440366972477,
+      "grad_norm": 1.025681972503662,
+      "learning_rate": 0.0001,
+      "loss": 0.8467,
+      "step": 900
+    },
+    {
+      "epoch": 4.174311926605505,
+      "grad_norm": 0.7425392270088196,
+      "learning_rate": 0.0001,
+      "loss": 0.8526,
+      "step": 910
+    },
+    {
+      "epoch": 4.220183486238533,
+      "grad_norm": 0.8017181158065796,
+      "learning_rate": 0.0001,
+      "loss": 0.8456,
+      "step": 920
+    },
+    {
+      "epoch": 4.26605504587156,
+      "grad_norm": 0.7509899139404297,
+      "learning_rate": 0.0001,
+      "loss": 0.8627,
+      "step": 930
+    },
+    {
+      "epoch": 4.3119266055045875,
+      "grad_norm": 0.8284080624580383,
+      "learning_rate": 0.0001,
+      "loss": 0.8489,
+      "step": 940
+    },
+    {
+      "epoch": 4.3577981651376145,
+      "grad_norm": 0.983587920665741,
+      "learning_rate": 0.0001,
+      "loss": 0.8805,
+      "step": 950
+    },
+    {
+      "epoch": 4.4036697247706424,
+      "grad_norm": 0.7172166109085083,
+      "learning_rate": 0.0001,
+      "loss": 0.8659,
+      "step": 960
+    },
+    {
+      "epoch": 4.4495412844036695,
+      "grad_norm": 0.8484768271446228,
+      "learning_rate": 0.0001,
+      "loss": 0.8489,
+      "step": 970
+    },
+    {
+      "epoch": 4.495412844036697,
+      "grad_norm": 1.2530163526535034,
+      "learning_rate": 0.0001,
+      "loss": 0.8486,
+      "step": 980
+    },
+    {
+      "epoch": 4.541284403669724,
+      "grad_norm": 0.6953453421592712,
+      "learning_rate": 0.0001,
+      "loss": 0.8712,
+      "step": 990
+    },
+    {
+      "epoch": 4.587155963302752,
+      "grad_norm": 0.9675477743148804,
+      "learning_rate": 0.0001,
+      "loss": 0.8496,
+      "step": 1000
+    },
+    {
+      "epoch": 4.63302752293578,
+      "grad_norm": 0.6756150126457214,
+      "learning_rate": 0.0001,
+      "loss": 0.8696,
+      "step": 1010
+    },
+    {
+      "epoch": 4.678899082568807,
+      "grad_norm": 0.7769085764884949,
+      "learning_rate": 0.0001,
+      "loss": 0.8433,
+      "step": 1020
+    },
+    {
+      "epoch": 4.724770642201835,
+      "grad_norm": 0.7071849703788757,
+      "learning_rate": 0.0001,
+      "loss": 0.8315,
+      "step": 1030
+    },
+    {
+      "epoch": 4.770642201834862,
+      "grad_norm": 0.7788539528846741,
+      "learning_rate": 0.0001,
+      "loss": 0.8305,
+      "step": 1040
+    },
+    {
+      "epoch": 4.81651376146789,
+      "grad_norm": 0.8174529671669006,
+      "learning_rate": 0.0001,
+      "loss": 0.793,
+      "step": 1050
+    },
+    {
+      "epoch": 4.862385321100917,
+      "grad_norm": 0.9944226145744324,
+      "learning_rate": 0.0001,
+      "loss": 0.8871,
+      "step": 1060
+    },
+    {
+      "epoch": 4.908256880733945,
+      "grad_norm": 1.0104237794876099,
+      "learning_rate": 0.0001,
+      "loss": 0.8497,
+      "step": 1070
+    },
+    {
+      "epoch": 4.954128440366972,
+      "grad_norm": 0.7839273810386658,
+      "learning_rate": 0.0001,
+      "loss": 0.8683,
+      "step": 1080
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 1.135035753250122,
+      "learning_rate": 0.0001,
+      "loss": 0.8428,
+      "step": 1090
+    },
+    {
+      "epoch": 5.0,
+      "eval_loss": 1.0589717626571655,
+      "eval_runtime": 8.3188,
+      "eval_samples_per_second": 13.103,
+      "eval_steps_per_second": 1.683,
+      "step": 1090
+    },
+    {
+      "epoch": 5.045871559633028,
+      "grad_norm": 0.8675414323806763,
+      "learning_rate": 0.0001,
+      "loss": 0.7846,
+      "step": 1100
+    },
+    {
+      "epoch": 5.091743119266055,
+      "grad_norm": 0.9152688980102539,
+      "learning_rate": 0.0001,
+      "loss": 0.7525,
+      "step": 1110
+    },
+    {
+      "epoch": 5.137614678899083,
+      "grad_norm": 1.057682752609253,
+      "learning_rate": 0.0001,
+      "loss": 0.8033,
+      "step": 1120
+    },
+    {
+      "epoch": 5.18348623853211,
+      "grad_norm": 0.9883639812469482,
+      "learning_rate": 0.0001,
+      "loss": 0.8064,
+      "step": 1130
+    },
+    {
+      "epoch": 5.229357798165138,
+      "grad_norm": 0.908318817615509,
+      "learning_rate": 0.0001,
+      "loss": 0.8043,
+      "step": 1140
+    },
+    {
+      "epoch": 5.275229357798165,
+      "grad_norm": 0.9290405511856079,
+      "learning_rate": 0.0001,
+      "loss": 0.7897,
+      "step": 1150
+    },
+    {
+      "epoch": 5.321100917431193,
+      "grad_norm": 1.0225027799606323,
+      "learning_rate": 0.0001,
+      "loss": 0.7678,
+      "step": 1160
+    },
+    {
+      "epoch": 5.36697247706422,
+      "grad_norm": 0.723305344581604,
+      "learning_rate": 0.0001,
+      "loss": 0.7998,
+      "step": 1170
+    },
+    {
+      "epoch": 5.412844036697248,
+      "grad_norm": 0.9404224753379822,
+      "learning_rate": 0.0001,
+      "loss": 0.7691,
+      "step": 1180
+    },
+    {
+      "epoch": 5.458715596330276,
+      "grad_norm": 1.2413196563720703,
+      "learning_rate": 0.0001,
+      "loss": 0.7553,
+      "step": 1190
+    },
+    {
+      "epoch": 5.504587155963303,
+      "grad_norm": 0.9953755140304565,
+      "learning_rate": 0.0001,
+      "loss": 0.7473,
+      "step": 1200
+    },
+    {
+      "epoch": 5.5504587155963305,
+      "grad_norm": 0.8769338130950928,
+      "learning_rate": 0.0001,
+      "loss": 0.7919,
+      "step": 1210
+    },
+    {
+      "epoch": 5.5963302752293576,
+      "grad_norm": 0.7918237447738647,
+      "learning_rate": 0.0001,
+      "loss": 0.7754,
+      "step": 1220
+    },
+    {
+      "epoch": 5.6422018348623855,
+      "grad_norm": 0.8943650126457214,
+      "learning_rate": 0.0001,
+      "loss": 0.7726,
+      "step": 1230
+    },
+    {
+      "epoch": 5.6880733944954125,
+      "grad_norm": 0.9250347018241882,
+      "learning_rate": 0.0001,
+      "loss": 0.8007,
+      "step": 1240
+    },
+    {
+      "epoch": 5.73394495412844,
+      "grad_norm": 1.034350037574768,
+      "learning_rate": 0.0001,
+      "loss": 0.8033,
+      "step": 1250
+    },
+    {
+      "epoch": 5.779816513761467,
+      "grad_norm": 1.0044519901275635,
+      "learning_rate": 0.0001,
+      "loss": 0.7616,
+      "step": 1260
+    },
+    {
+      "epoch": 5.825688073394495,
+      "grad_norm": 0.8655275702476501,
+      "learning_rate": 0.0001,
+      "loss": 0.8549,
+      "step": 1270
+    },
+    {
+      "epoch": 5.871559633027523,
+      "grad_norm": 0.8263447284698486,
+      "learning_rate": 0.0001,
+      "loss": 0.7996,
+      "step": 1280
+    },
+    {
+      "epoch": 5.91743119266055,
+      "grad_norm": 0.679572582244873,
+      "learning_rate": 0.0001,
+      "loss": 0.7603,
+      "step": 1290
+    },
+    {
+      "epoch": 5.963302752293578,
+      "grad_norm": 0.9471953511238098,
+      "learning_rate": 0.0001,
+      "loss": 0.7782,
+      "step": 1300
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 1.0743499994277954,
+      "eval_runtime": 8.2992,
+      "eval_samples_per_second": 13.134,
+      "eval_steps_per_second": 1.687,
+      "step": 1308
+    },
+    {
+      "epoch": 6.009174311926605,
+      "grad_norm": 1.0349316596984863,
+      "learning_rate": 0.0001,
+      "loss": 0.7788,
+      "step": 1310
+    },
+    {
+      "epoch": 6.055045871559633,
+      "grad_norm": 1.4172323942184448,
+      "learning_rate": 0.0001,
+      "loss": 0.7197,
+      "step": 1320
+    },
+    {
+      "epoch": 6.10091743119266,
+      "grad_norm": 1.2426704168319702,
+      "learning_rate": 0.0001,
+      "loss": 0.7256,
+      "step": 1330
+    },
+    {
+      "epoch": 6.146788990825688,
+      "grad_norm": 1.2230033874511719,
+      "learning_rate": 0.0001,
+      "loss": 0.693,
+      "step": 1340
+    },
+    {
+      "epoch": 6.192660550458716,
+      "grad_norm": 0.9299599528312683,
+      "learning_rate": 0.0001,
+      "loss": 0.7081,
+      "step": 1350
+    },
+    {
+      "epoch": 6.238532110091743,
+      "grad_norm": 1.1219823360443115,
+      "learning_rate": 0.0001,
+      "loss": 0.6719,
+      "step": 1360
+    },
+    {
+      "epoch": 6.284403669724771,
+      "grad_norm": 1.0289041996002197,
+      "learning_rate": 0.0001,
+      "loss": 0.745,
+      "step": 1370
+    },
+    {
+      "epoch": 6.330275229357798,
+      "grad_norm": 1.0673019886016846,
+      "learning_rate": 0.0001,
+      "loss": 0.7207,
+      "step": 1380
+    },
+    {
+      "epoch": 6.376146788990826,
+      "grad_norm": 1.1395819187164307,
+      "learning_rate": 0.0001,
+      "loss": 0.7073,
+      "step": 1390
+    },
+    {
+      "epoch": 6.422018348623853,
+      "grad_norm": 0.9172726273536682,
+      "learning_rate": 0.0001,
+      "loss": 0.7328,
+      "step": 1400
+    },
+    {
+      "epoch": 6.467889908256881,
+      "grad_norm": 1.0613523721694946,
+      "learning_rate": 0.0001,
+      "loss": 0.6788,
+      "step": 1410
+    },
+    {
+      "epoch": 6.513761467889909,
+      "grad_norm": 0.9698522090911865,
+      "learning_rate": 0.0001,
+      "loss": 0.7287,
+      "step": 1420
+    },
+    {
+      "epoch": 6.559633027522936,
+      "grad_norm": 1.0680975914001465,
+      "learning_rate": 0.0001,
+      "loss": 0.7286,
+      "step": 1430
+    },
+    {
+      "epoch": 6.605504587155964,
+      "grad_norm": 0.8172698020935059,
+      "learning_rate": 0.0001,
+      "loss": 0.7336,
+      "step": 1440
+    },
+    {
+      "epoch": 6.651376146788991,
+      "grad_norm": 1.1772243976593018,
+      "learning_rate": 0.0001,
+      "loss": 0.7084,
+      "step": 1450
+    },
+    {
+      "epoch": 6.697247706422019,
+      "grad_norm": 1.00876784324646,
+      "learning_rate": 0.0001,
+      "loss": 0.7194,
+      "step": 1460
+    },
+    {
+      "epoch": 6.743119266055046,
+      "grad_norm": 1.3571248054504395,
+      "learning_rate": 0.0001,
+      "loss": 0.7332,
+      "step": 1470
+    },
+    {
+      "epoch": 6.7889908256880735,
+      "grad_norm": 1.093043565750122,
+      "learning_rate": 0.0001,
+      "loss": 0.7401,
+      "step": 1480
+    },
+    {
+      "epoch": 6.834862385321101,
+      "grad_norm": 1.4730535745620728,
+      "learning_rate": 0.0001,
+      "loss": 0.7009,
+      "step": 1490
+    },
+    {
+      "epoch": 6.8807339449541285,
+      "grad_norm": 0.9187682271003723,
+      "learning_rate": 0.0001,
+      "loss": 0.737,
+      "step": 1500
+    },
+    {
+      "epoch": 6.926605504587156,
+      "grad_norm": 1.406137466430664,
+      "learning_rate": 0.0001,
+      "loss": 0.6528,
+      "step": 1510
+    },
+    {
+      "epoch": 6.972477064220183,
+      "grad_norm": 1.1134064197540283,
+      "learning_rate": 0.0001,
+      "loss": 0.7074,
+      "step": 1520
+    },
+    {
+      "epoch": 7.0,
+      "eval_loss": 1.1300649642944336,
+      "eval_runtime": 8.3014,
+      "eval_samples_per_second": 13.13,
+      "eval_steps_per_second": 1.686,
+      "step": 1526
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1744,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 7.061986314761011e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1526/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fbd275a27ab08d2e5458e423282c9f72d2ca0deb32b3380f16bbc843fea6e0d
+size 5560

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1744/README.md

@@ -0,0 +1,202 @@
+---
+base_model: meta-llama/Meta-Llama-3-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1744/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1744/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c5960c549e4041697be1d539e7c3d8d5e2951859298ca7226dc3c04113b0c8e
+size 109069176

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1744/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f681eed420da8a37a5526e2480109c1df117dcc66e8f37d42de5a91f37a31ab4
+size 55532666

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1744/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d36f02e0ce788bad7fdf10e69088ac8ae9c0c004780bb0ec55d2ce8e9e8ef417
+size 14244

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1744/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a05d8057c6ceb3333ea3a2834b9b0ea56a183cd63022683207407ff1f33436a
+size 1064

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1744/special_tokens_map.json

@@ -0,0 +1,17 @@
+{
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|eot_id|>"
+}

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1744/tokenizer_config.json

@@ -0,0 +1,2063 @@
+{
+  "added_tokens_decoder": {
+    "128000": {
+      "content": "<|begin_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128001": {
+      "content": "<|end_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128002": {
+      "content": "<|reserved_special_token_0|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128003": {
+      "content": "<|reserved_special_token_1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128004": {
+      "content": "<|reserved_special_token_2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128005": {
+      "content": "<|reserved_special_token_3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128006": {
+      "content": "<|start_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128007": {
+      "content": "<|end_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128008": {
+      "content": "<|reserved_special_token_4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128009": {
+      "content": "<|eot_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128010": {
+      "content": "<|reserved_special_token_5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128011": {
+      "content": "<|reserved_special_token_6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128012": {
+      "content": "<|reserved_special_token_7|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128013": {
+      "content": "<|reserved_special_token_8|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128014": {
+      "content": "<|reserved_special_token_9|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128015": {
+      "content": "<|reserved_special_token_10|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128016": {
+      "content": "<|reserved_special_token_11|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128017": {
+      "content": "<|reserved_special_token_12|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128018": {
+      "content": "<|reserved_special_token_13|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128019": {
+      "content": "<|reserved_special_token_14|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128020": {
+      "content": "<|reserved_special_token_15|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128021": {
+      "content": "<|reserved_special_token_16|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128022": {
+      "content": "<|reserved_special_token_17|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128023": {
+      "content": "<|reserved_special_token_18|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128024": {
+      "content": "<|reserved_special_token_19|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128025": {
+      "content": "<|reserved_special_token_20|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128026": {
+      "content": "<|reserved_special_token_21|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128027": {
+      "content": "<|reserved_special_token_22|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128028": {
+      "content": "<|reserved_special_token_23|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128029": {
+      "content": "<|reserved_special_token_24|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128030": {
+      "content": "<|reserved_special_token_25|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128031": {
+      "content": "<|reserved_special_token_26|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128032": {
+      "content": "<|reserved_special_token_27|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128033": {
+      "content": "<|reserved_special_token_28|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128034": {
+      "content": "<|reserved_special_token_29|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128035": {
+      "content": "<|reserved_special_token_30|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128036": {
+      "content": "<|reserved_special_token_31|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128037": {
+      "content": "<|reserved_special_token_32|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128038": {
+      "content": "<|reserved_special_token_33|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128039": {
+      "content": "<|reserved_special_token_34|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128040": {
+      "content": "<|reserved_special_token_35|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128041": {
+      "content": "<|reserved_special_token_36|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128042": {
+      "content": "<|reserved_special_token_37|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128043": {
+      "content": "<|reserved_special_token_38|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128044": {
+      "content": "<|reserved_special_token_39|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128045": {
+      "content": "<|reserved_special_token_40|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128046": {
+      "content": "<|reserved_special_token_41|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128047": {
+      "content": "<|reserved_special_token_42|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128048": {
+      "content": "<|reserved_special_token_43|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128049": {
+      "content": "<|reserved_special_token_44|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128050": {
+      "content": "<|reserved_special_token_45|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128051": {
+      "content": "<|reserved_special_token_46|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128052": {
+      "content": "<|reserved_special_token_47|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128053": {
+      "content": "<|reserved_special_token_48|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128054": {
+      "content": "<|reserved_special_token_49|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128055": {
+      "content": "<|reserved_special_token_50|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128056": {
+      "content": "<|reserved_special_token_51|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128057": {
+      "content": "<|reserved_special_token_52|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128058": {
+      "content": "<|reserved_special_token_53|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128059": {
+      "content": "<|reserved_special_token_54|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128060": {
+      "content": "<|reserved_special_token_55|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128061": {
+      "content": "<|reserved_special_token_56|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128062": {
+      "content": "<|reserved_special_token_57|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128063": {
+      "content": "<|reserved_special_token_58|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128064": {
+      "content": "<|reserved_special_token_59|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128065": {
+      "content": "<|reserved_special_token_60|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128066": {
+      "content": "<|reserved_special_token_61|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128067": {
+      "content": "<|reserved_special_token_62|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128068": {
+      "content": "<|reserved_special_token_63|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128069": {
+      "content": "<|reserved_special_token_64|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128070": {
+      "content": "<|reserved_special_token_65|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128071": {
+      "content": "<|reserved_special_token_66|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128072": {
+      "content": "<|reserved_special_token_67|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128073": {
+      "content": "<|reserved_special_token_68|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128074": {
+      "content": "<|reserved_special_token_69|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128075": {
+      "content": "<|reserved_special_token_70|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128076": {
+      "content": "<|reserved_special_token_71|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128077": {
+      "content": "<|reserved_special_token_72|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128078": {
+      "content": "<|reserved_special_token_73|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128079": {
+      "content": "<|reserved_special_token_74|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128080": {
+      "content": "<|reserved_special_token_75|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128081": {
+      "content": "<|reserved_special_token_76|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128082": {
+      "content": "<|reserved_special_token_77|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128083": {
+      "content": "<|reserved_special_token_78|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128084": {
+      "content": "<|reserved_special_token_79|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128085": {
+      "content": "<|reserved_special_token_80|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128086": {
+      "content": "<|reserved_special_token_81|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128087": {
+      "content": "<|reserved_special_token_82|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128088": {
+      "content": "<|reserved_special_token_83|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128089": {
+      "content": "<|reserved_special_token_84|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128090": {
+      "content": "<|reserved_special_token_85|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128091": {
+      "content": "<|reserved_special_token_86|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128092": {
+      "content": "<|reserved_special_token_87|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128093": {
+      "content": "<|reserved_special_token_88|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128094": {
+      "content": "<|reserved_special_token_89|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128095": {
+      "content": "<|reserved_special_token_90|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128096": {
+      "content": "<|reserved_special_token_91|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128097": {
+      "content": "<|reserved_special_token_92|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128098": {
+      "content": "<|reserved_special_token_93|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128099": {
+      "content": "<|reserved_special_token_94|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128100": {
+      "content": "<|reserved_special_token_95|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128101": {
+      "content": "<|reserved_special_token_96|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128102": {
+      "content": "<|reserved_special_token_97|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128103": {
+      "content": "<|reserved_special_token_98|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128104": {
+      "content": "<|reserved_special_token_99|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128105": {
+      "content": "<|reserved_special_token_100|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128106": {
+      "content": "<|reserved_special_token_101|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128107": {
+      "content": "<|reserved_special_token_102|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128108": {
+      "content": "<|reserved_special_token_103|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128109": {
+      "content": "<|reserved_special_token_104|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128110": {
+      "content": "<|reserved_special_token_105|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128111": {
+      "content": "<|reserved_special_token_106|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128112": {
+      "content": "<|reserved_special_token_107|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128113": {
+      "content": "<|reserved_special_token_108|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128114": {
+      "content": "<|reserved_special_token_109|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128115": {
+      "content": "<|reserved_special_token_110|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128116": {
+      "content": "<|reserved_special_token_111|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128117": {
+      "content": "<|reserved_special_token_112|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128118": {
+      "content": "<|reserved_special_token_113|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128119": {
+      "content": "<|reserved_special_token_114|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128120": {
+      "content": "<|reserved_special_token_115|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128121": {
+      "content": "<|reserved_special_token_116|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128122": {
+      "content": "<|reserved_special_token_117|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128123": {
+      "content": "<|reserved_special_token_118|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128124": {
+      "content": "<|reserved_special_token_119|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128125": {
+      "content": "<|reserved_special_token_120|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128126": {
+      "content": "<|reserved_special_token_121|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128127": {
+      "content": "<|reserved_special_token_122|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128128": {
+      "content": "<|reserved_special_token_123|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128129": {
+      "content": "<|reserved_special_token_124|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128130": {
+      "content": "<|reserved_special_token_125|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128131": {
+      "content": "<|reserved_special_token_126|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128132": {
+      "content": "<|reserved_special_token_127|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128133": {
+      "content": "<|reserved_special_token_128|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128134": {
+      "content": "<|reserved_special_token_129|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128135": {
+      "content": "<|reserved_special_token_130|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128136": {
+      "content": "<|reserved_special_token_131|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128137": {
+      "content": "<|reserved_special_token_132|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128138": {
+      "content": "<|reserved_special_token_133|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128139": {
+      "content": "<|reserved_special_token_134|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128140": {
+      "content": "<|reserved_special_token_135|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128141": {
+      "content": "<|reserved_special_token_136|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128142": {
+      "content": "<|reserved_special_token_137|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128143": {
+      "content": "<|reserved_special_token_138|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128144": {
+      "content": "<|reserved_special_token_139|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128145": {
+      "content": "<|reserved_special_token_140|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128146": {
+      "content": "<|reserved_special_token_141|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128147": {
+      "content": "<|reserved_special_token_142|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128148": {
+      "content": "<|reserved_special_token_143|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128149": {
+      "content": "<|reserved_special_token_144|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128150": {
+      "content": "<|reserved_special_token_145|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128151": {
+      "content": "<|reserved_special_token_146|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128152": {
+      "content": "<|reserved_special_token_147|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128153": {
+      "content": "<|reserved_special_token_148|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128154": {
+      "content": "<|reserved_special_token_149|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128155": {
+      "content": "<|reserved_special_token_150|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128156": {
+      "content": "<|reserved_special_token_151|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128157": {
+      "content": "<|reserved_special_token_152|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128158": {
+      "content": "<|reserved_special_token_153|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128159": {
+      "content": "<|reserved_special_token_154|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128160": {
+      "content": "<|reserved_special_token_155|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128161": {
+      "content": "<|reserved_special_token_156|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128162": {
+      "content": "<|reserved_special_token_157|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128163": {
+      "content": "<|reserved_special_token_158|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128164": {
+      "content": "<|reserved_special_token_159|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128165": {
+      "content": "<|reserved_special_token_160|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128166": {
+      "content": "<|reserved_special_token_161|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128167": {
+      "content": "<|reserved_special_token_162|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128168": {
+      "content": "<|reserved_special_token_163|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128169": {
+      "content": "<|reserved_special_token_164|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128170": {
+      "content": "<|reserved_special_token_165|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128171": {
+      "content": "<|reserved_special_token_166|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128172": {
+      "content": "<|reserved_special_token_167|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128173": {
+      "content": "<|reserved_special_token_168|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128174": {
+      "content": "<|reserved_special_token_169|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128175": {
+      "content": "<|reserved_special_token_170|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128176": {
+      "content": "<|reserved_special_token_171|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128177": {
+      "content": "<|reserved_special_token_172|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128178": {
+      "content": "<|reserved_special_token_173|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128179": {
+      "content": "<|reserved_special_token_174|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128180": {
+      "content": "<|reserved_special_token_175|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128181": {
+      "content": "<|reserved_special_token_176|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128182": {
+      "content": "<|reserved_special_token_177|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128183": {
+      "content": "<|reserved_special_token_178|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128184": {
+      "content": "<|reserved_special_token_179|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128185": {
+      "content": "<|reserved_special_token_180|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128186": {
+      "content": "<|reserved_special_token_181|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128187": {
+      "content": "<|reserved_special_token_182|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128188": {
+      "content": "<|reserved_special_token_183|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128189": {
+      "content": "<|reserved_special_token_184|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128190": {
+      "content": "<|reserved_special_token_185|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128191": {
+      "content": "<|reserved_special_token_186|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128192": {
+      "content": "<|reserved_special_token_187|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128193": {
+      "content": "<|reserved_special_token_188|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128194": {
+      "content": "<|reserved_special_token_189|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128195": {
+      "content": "<|reserved_special_token_190|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128196": {
+      "content": "<|reserved_special_token_191|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128197": {
+      "content": "<|reserved_special_token_192|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128198": {
+      "content": "<|reserved_special_token_193|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128199": {
+      "content": "<|reserved_special_token_194|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128200": {
+      "content": "<|reserved_special_token_195|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128201": {
+      "content": "<|reserved_special_token_196|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128202": {
+      "content": "<|reserved_special_token_197|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128203": {
+      "content": "<|reserved_special_token_198|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128204": {
+      "content": "<|reserved_special_token_199|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128205": {
+      "content": "<|reserved_special_token_200|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128206": {
+      "content": "<|reserved_special_token_201|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128207": {
+      "content": "<|reserved_special_token_202|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128208": {
+      "content": "<|reserved_special_token_203|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128209": {
+      "content": "<|reserved_special_token_204|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128210": {
+      "content": "<|reserved_special_token_205|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128211": {
+      "content": "<|reserved_special_token_206|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128212": {
+      "content": "<|reserved_special_token_207|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128213": {
+      "content": "<|reserved_special_token_208|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128214": {
+      "content": "<|reserved_special_token_209|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128215": {
+      "content": "<|reserved_special_token_210|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128216": {
+      "content": "<|reserved_special_token_211|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128217": {
+      "content": "<|reserved_special_token_212|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128218": {
+      "content": "<|reserved_special_token_213|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128219": {
+      "content": "<|reserved_special_token_214|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128220": {
+      "content": "<|reserved_special_token_215|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128221": {
+      "content": "<|reserved_special_token_216|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128222": {
+      "content": "<|reserved_special_token_217|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128223": {
+      "content": "<|reserved_special_token_218|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128224": {
+      "content": "<|reserved_special_token_219|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128225": {
+      "content": "<|reserved_special_token_220|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128226": {
+      "content": "<|reserved_special_token_221|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128227": {
+      "content": "<|reserved_special_token_222|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128228": {
+      "content": "<|reserved_special_token_223|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128229": {
+      "content": "<|reserved_special_token_224|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128230": {
+      "content": "<|reserved_special_token_225|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128231": {
+      "content": "<|reserved_special_token_226|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128232": {
+      "content": "<|reserved_special_token_227|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128233": {
+      "content": "<|reserved_special_token_228|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128234": {
+      "content": "<|reserved_special_token_229|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128235": {
+      "content": "<|reserved_special_token_230|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128236": {
+      "content": "<|reserved_special_token_231|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128237": {
+      "content": "<|reserved_special_token_232|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128238": {
+      "content": "<|reserved_special_token_233|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128239": {
+      "content": "<|reserved_special_token_234|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128240": {
+      "content": "<|reserved_special_token_235|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128241": {
+      "content": "<|reserved_special_token_236|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128242": {
+      "content": "<|reserved_special_token_237|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128243": {
+      "content": "<|reserved_special_token_238|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128244": {
+      "content": "<|reserved_special_token_239|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128245": {
+      "content": "<|reserved_special_token_240|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128246": {
+      "content": "<|reserved_special_token_241|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128247": {
+      "content": "<|reserved_special_token_242|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128248": {
+      "content": "<|reserved_special_token_243|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128249": {
+      "content": "<|reserved_special_token_244|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128250": {
+      "content": "<|reserved_special_token_245|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128251": {
+      "content": "<|reserved_special_token_246|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128252": {
+      "content": "<|reserved_special_token_247|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128253": {
+      "content": "<|reserved_special_token_248|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128254": {
+      "content": "<|reserved_special_token_249|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128255": {
+      "content": "<|reserved_special_token_250|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|begin_of_text|>",
+  "chat_template": "{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|eot_id|>",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<|eot_id|>",
+  "tokenizer_class": "PreTrainedTokenizerFast"
+}

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1744/trainer_state.json

@@ -0,0 +1,1315 @@
+{
+  "best_metric": 1.0338033437728882,
+  "best_model_checkpoint": "outputs-001/Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-654",
+  "epoch": 8.0,
+  "eval_steps": 10,
+  "global_step": 1744,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.045871559633027525,
+      "grad_norm": 0.7936908602714539,
+      "learning_rate": 0.0001,
+      "loss": 2.1542,
+      "step": 10
+    },
+    {
+      "epoch": 0.09174311926605505,
+      "grad_norm": 0.8541925549507141,
+      "learning_rate": 0.0001,
+      "loss": 1.7148,
+      "step": 20
+    },
+    {
+      "epoch": 0.13761467889908258,
+      "grad_norm": 0.6849298477172852,
+      "learning_rate": 0.0001,
+      "loss": 1.4102,
+      "step": 30
+    },
+    {
+      "epoch": 0.1834862385321101,
+      "grad_norm": 0.6280460953712463,
+      "learning_rate": 0.0001,
+      "loss": 1.2936,
+      "step": 40
+    },
+    {
+      "epoch": 0.22935779816513763,
+      "grad_norm": 0.699965238571167,
+      "learning_rate": 0.0001,
+      "loss": 1.2116,
+      "step": 50
+    },
+    {
+      "epoch": 0.27522935779816515,
+      "grad_norm": 0.5185195803642273,
+      "learning_rate": 0.0001,
+      "loss": 1.2337,
+      "step": 60
+    },
+    {
+      "epoch": 0.3211009174311927,
+      "grad_norm": 0.5821201801300049,
+      "learning_rate": 0.0001,
+      "loss": 1.2317,
+      "step": 70
+    },
+    {
+      "epoch": 0.3669724770642202,
+      "grad_norm": 0.4655168652534485,
+      "learning_rate": 0.0001,
+      "loss": 1.1355,
+      "step": 80
+    },
+    {
+      "epoch": 0.41284403669724773,
+      "grad_norm": 0.5342468023300171,
+      "learning_rate": 0.0001,
+      "loss": 1.1571,
+      "step": 90
+    },
+    {
+      "epoch": 0.45871559633027525,
+      "grad_norm": 0.42904630303382874,
+      "learning_rate": 0.0001,
+      "loss": 1.1333,
+      "step": 100
+    },
+    {
+      "epoch": 0.5045871559633027,
+      "grad_norm": 0.5210477709770203,
+      "learning_rate": 0.0001,
+      "loss": 1.1596,
+      "step": 110
+    },
+    {
+      "epoch": 0.5504587155963303,
+      "grad_norm": 0.38087233901023865,
+      "learning_rate": 0.0001,
+      "loss": 1.1089,
+      "step": 120
+    },
+    {
+      "epoch": 0.5963302752293578,
+      "grad_norm": 0.4819198250770569,
+      "learning_rate": 0.0001,
+      "loss": 1.1458,
+      "step": 130
+    },
+    {
+      "epoch": 0.6422018348623854,
+      "grad_norm": 0.6635904908180237,
+      "learning_rate": 0.0001,
+      "loss": 1.1834,
+      "step": 140
+    },
+    {
+      "epoch": 0.6880733944954128,
+      "grad_norm": 0.5125461220741272,
+      "learning_rate": 0.0001,
+      "loss": 1.07,
+      "step": 150
+    },
+    {
+      "epoch": 0.7339449541284404,
+      "grad_norm": 0.5035977959632874,
+      "learning_rate": 0.0001,
+      "loss": 1.093,
+      "step": 160
+    },
+    {
+      "epoch": 0.7798165137614679,
+      "grad_norm": 0.5379819869995117,
+      "learning_rate": 0.0001,
+      "loss": 1.0297,
+      "step": 170
+    },
+    {
+      "epoch": 0.8256880733944955,
+      "grad_norm": 0.4491284191608429,
+      "learning_rate": 0.0001,
+      "loss": 1.1212,
+      "step": 180
+    },
+    {
+      "epoch": 0.8715596330275229,
+      "grad_norm": 0.5257289409637451,
+      "learning_rate": 0.0001,
+      "loss": 1.139,
+      "step": 190
+    },
+    {
+      "epoch": 0.9174311926605505,
+      "grad_norm": 0.5361109375953674,
+      "learning_rate": 0.0001,
+      "loss": 1.0379,
+      "step": 200
+    },
+    {
+      "epoch": 0.963302752293578,
+      "grad_norm": 0.5124537348747253,
+      "learning_rate": 0.0001,
+      "loss": 1.0588,
+      "step": 210
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.070561170578003,
+      "eval_runtime": 8.31,
+      "eval_samples_per_second": 13.117,
+      "eval_steps_per_second": 1.685,
+      "step": 218
+    },
+    {
+      "epoch": 1.0091743119266054,
+      "grad_norm": 0.44702231884002686,
+      "learning_rate": 0.0001,
+      "loss": 1.0394,
+      "step": 220
+    },
+    {
+      "epoch": 1.0550458715596331,
+      "grad_norm": 0.48708081245422363,
+      "learning_rate": 0.0001,
+      "loss": 1.1075,
+      "step": 230
+    },
+    {
+      "epoch": 1.1009174311926606,
+      "grad_norm": 0.6492213010787964,
+      "learning_rate": 0.0001,
+      "loss": 1.0769,
+      "step": 240
+    },
+    {
+      "epoch": 1.146788990825688,
+      "grad_norm": 0.48868921399116516,
+      "learning_rate": 0.0001,
+      "loss": 1.0165,
+      "step": 250
+    },
+    {
+      "epoch": 1.1926605504587156,
+      "grad_norm": 0.6291283369064331,
+      "learning_rate": 0.0001,
+      "loss": 0.9982,
+      "step": 260
+    },
+    {
+      "epoch": 1.238532110091743,
+      "grad_norm": 0.6115853190422058,
+      "learning_rate": 0.0001,
+      "loss": 1.0501,
+      "step": 270
+    },
+    {
+      "epoch": 1.2844036697247707,
+      "grad_norm": 0.6975598931312561,
+      "learning_rate": 0.0001,
+      "loss": 1.0874,
+      "step": 280
+    },
+    {
+      "epoch": 1.3302752293577982,
+      "grad_norm": 0.5336413383483887,
+      "learning_rate": 0.0001,
+      "loss": 1.0721,
+      "step": 290
+    },
+    {
+      "epoch": 1.3761467889908257,
+      "grad_norm": 0.5163336992263794,
+      "learning_rate": 0.0001,
+      "loss": 1.0405,
+      "step": 300
+    },
+    {
+      "epoch": 1.4220183486238533,
+      "grad_norm": 0.6026386022567749,
+      "learning_rate": 0.0001,
+      "loss": 1.0119,
+      "step": 310
+    },
+    {
+      "epoch": 1.4678899082568808,
+      "grad_norm": 0.6290488839149475,
+      "learning_rate": 0.0001,
+      "loss": 1.0327,
+      "step": 320
+    },
+    {
+      "epoch": 1.5137614678899083,
+      "grad_norm": 0.5215861797332764,
+      "learning_rate": 0.0001,
+      "loss": 1.0725,
+      "step": 330
+    },
+    {
+      "epoch": 1.5596330275229358,
+      "grad_norm": 0.6104291677474976,
+      "learning_rate": 0.0001,
+      "loss": 1.0364,
+      "step": 340
+    },
+    {
+      "epoch": 1.6055045871559632,
+      "grad_norm": 0.5163600444793701,
+      "learning_rate": 0.0001,
+      "loss": 1.0412,
+      "step": 350
+    },
+    {
+      "epoch": 1.6513761467889907,
+      "grad_norm": 0.4934627413749695,
+      "learning_rate": 0.0001,
+      "loss": 1.0661,
+      "step": 360
+    },
+    {
+      "epoch": 1.6972477064220184,
+      "grad_norm": 0.6802699565887451,
+      "learning_rate": 0.0001,
+      "loss": 1.0644,
+      "step": 370
+    },
+    {
+      "epoch": 1.7431192660550459,
+      "grad_norm": 0.5180800557136536,
+      "learning_rate": 0.0001,
+      "loss": 1.0175,
+      "step": 380
+    },
+    {
+      "epoch": 1.7889908256880735,
+      "grad_norm": 0.6208664178848267,
+      "learning_rate": 0.0001,
+      "loss": 1.0099,
+      "step": 390
+    },
+    {
+      "epoch": 1.834862385321101,
+      "grad_norm": 0.5667423605918884,
+      "learning_rate": 0.0001,
+      "loss": 0.9917,
+      "step": 400
+    },
+    {
+      "epoch": 1.8807339449541285,
+      "grad_norm": 0.7282969355583191,
+      "learning_rate": 0.0001,
+      "loss": 1.0028,
+      "step": 410
+    },
+    {
+      "epoch": 1.926605504587156,
+      "grad_norm": 0.5308305025100708,
+      "learning_rate": 0.0001,
+      "loss": 1.062,
+      "step": 420
+    },
+    {
+      "epoch": 1.9724770642201834,
+      "grad_norm": 0.6123433113098145,
+      "learning_rate": 0.0001,
+      "loss": 1.012,
+      "step": 430
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.0371460914611816,
+      "eval_runtime": 8.3019,
+      "eval_samples_per_second": 13.13,
+      "eval_steps_per_second": 1.686,
+      "step": 436
+    },
+    {
+      "epoch": 2.018348623853211,
+      "grad_norm": 0.5201505422592163,
+      "learning_rate": 0.0001,
+      "loss": 1.0126,
+      "step": 440
+    },
+    {
+      "epoch": 2.0642201834862384,
+      "grad_norm": 0.5720698237419128,
+      "learning_rate": 0.0001,
+      "loss": 0.9625,
+      "step": 450
+    },
+    {
+      "epoch": 2.1100917431192663,
+      "grad_norm": 0.6407686471939087,
+      "learning_rate": 0.0001,
+      "loss": 0.9256,
+      "step": 460
+    },
+    {
+      "epoch": 2.1559633027522938,
+      "grad_norm": 0.5237308740615845,
+      "learning_rate": 0.0001,
+      "loss": 0.9982,
+      "step": 470
+    },
+    {
+      "epoch": 2.2018348623853212,
+      "grad_norm": 0.6064776182174683,
+      "learning_rate": 0.0001,
+      "loss": 0.9806,
+      "step": 480
+    },
+    {
+      "epoch": 2.2477064220183487,
+      "grad_norm": 0.5534603595733643,
+      "learning_rate": 0.0001,
+      "loss": 0.9223,
+      "step": 490
+    },
+    {
+      "epoch": 2.293577981651376,
+      "grad_norm": 0.610197126865387,
+      "learning_rate": 0.0001,
+      "loss": 0.9339,
+      "step": 500
+    },
+    {
+      "epoch": 2.3394495412844036,
+      "grad_norm": 0.637509286403656,
+      "learning_rate": 0.0001,
+      "loss": 0.9669,
+      "step": 510
+    },
+    {
+      "epoch": 2.385321100917431,
+      "grad_norm": 0.6705873608589172,
+      "learning_rate": 0.0001,
+      "loss": 0.9798,
+      "step": 520
+    },
+    {
+      "epoch": 2.4311926605504586,
+      "grad_norm": 0.6272872090339661,
+      "learning_rate": 0.0001,
+      "loss": 0.9868,
+      "step": 530
+    },
+    {
+      "epoch": 2.477064220183486,
+      "grad_norm": 0.6892614364624023,
+      "learning_rate": 0.0001,
+      "loss": 1.0059,
+      "step": 540
+    },
+    {
+      "epoch": 2.522935779816514,
+      "grad_norm": 0.6316735744476318,
+      "learning_rate": 0.0001,
+      "loss": 0.9838,
+      "step": 550
+    },
+    {
+      "epoch": 2.5688073394495414,
+      "grad_norm": 0.5902363061904907,
+      "learning_rate": 0.0001,
+      "loss": 0.9548,
+      "step": 560
+    },
+    {
+      "epoch": 2.614678899082569,
+      "grad_norm": 0.6575316786766052,
+      "learning_rate": 0.0001,
+      "loss": 0.9859,
+      "step": 570
+    },
+    {
+      "epoch": 2.6605504587155964,
+      "grad_norm": 0.574419379234314,
+      "learning_rate": 0.0001,
+      "loss": 0.9976,
+      "step": 580
+    },
+    {
+      "epoch": 2.706422018348624,
+      "grad_norm": 0.7602782249450684,
+      "learning_rate": 0.0001,
+      "loss": 1.0267,
+      "step": 590
+    },
+    {
+      "epoch": 2.7522935779816513,
+      "grad_norm": 0.5764957070350647,
+      "learning_rate": 0.0001,
+      "loss": 0.9966,
+      "step": 600
+    },
+    {
+      "epoch": 2.7981651376146788,
+      "grad_norm": 0.68736332654953,
+      "learning_rate": 0.0001,
+      "loss": 0.9717,
+      "step": 610
+    },
+    {
+      "epoch": 2.8440366972477067,
+      "grad_norm": 0.5835479497909546,
+      "learning_rate": 0.0001,
+      "loss": 0.9393,
+      "step": 620
+    },
+    {
+      "epoch": 2.8899082568807337,
+      "grad_norm": 0.7003713250160217,
+      "learning_rate": 0.0001,
+      "loss": 0.9751,
+      "step": 630
+    },
+    {
+      "epoch": 2.9357798165137616,
+      "grad_norm": 0.5811060070991516,
+      "learning_rate": 0.0001,
+      "loss": 0.9803,
+      "step": 640
+    },
+    {
+      "epoch": 2.981651376146789,
+      "grad_norm": 0.5394262671470642,
+      "learning_rate": 0.0001,
+      "loss": 1.0262,
+      "step": 650
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 1.0338033437728882,
+      "eval_runtime": 8.3,
+      "eval_samples_per_second": 13.133,
+      "eval_steps_per_second": 1.687,
+      "step": 654
+    },
+    {
+      "epoch": 3.0275229357798166,
+      "grad_norm": 0.6349695920944214,
+      "learning_rate": 0.0001,
+      "loss": 0.9139,
+      "step": 660
+    },
+    {
+      "epoch": 3.073394495412844,
+      "grad_norm": 0.65309077501297,
+      "learning_rate": 0.0001,
+      "loss": 0.8632,
+      "step": 670
+    },
+    {
+      "epoch": 3.1192660550458715,
+      "grad_norm": 0.6830633878707886,
+      "learning_rate": 0.0001,
+      "loss": 0.9452,
+      "step": 680
+    },
+    {
+      "epoch": 3.165137614678899,
+      "grad_norm": 0.7120541334152222,
+      "learning_rate": 0.0001,
+      "loss": 0.9386,
+      "step": 690
+    },
+    {
+      "epoch": 3.2110091743119265,
+      "grad_norm": 0.6665357947349548,
+      "learning_rate": 0.0001,
+      "loss": 0.8971,
+      "step": 700
+    },
+    {
+      "epoch": 3.2568807339449544,
+      "grad_norm": 0.6179565191268921,
+      "learning_rate": 0.0001,
+      "loss": 0.8872,
+      "step": 710
+    },
+    {
+      "epoch": 3.302752293577982,
+      "grad_norm": 0.7118347883224487,
+      "learning_rate": 0.0001,
+      "loss": 0.9486,
+      "step": 720
+    },
+    {
+      "epoch": 3.3486238532110093,
+      "grad_norm": 0.6336777806282043,
+      "learning_rate": 0.0001,
+      "loss": 0.9389,
+      "step": 730
+    },
+    {
+      "epoch": 3.3944954128440368,
+      "grad_norm": 0.7117331027984619,
+      "learning_rate": 0.0001,
+      "loss": 0.89,
+      "step": 740
+    },
+    {
+      "epoch": 3.4403669724770642,
+      "grad_norm": 0.7455915808677673,
+      "learning_rate": 0.0001,
+      "loss": 0.8708,
+      "step": 750
+    },
+    {
+      "epoch": 3.4862385321100917,
+      "grad_norm": 0.7114333510398865,
+      "learning_rate": 0.0001,
+      "loss": 0.9671,
+      "step": 760
+    },
+    {
+      "epoch": 3.532110091743119,
+      "grad_norm": 0.9367598295211792,
+      "learning_rate": 0.0001,
+      "loss": 0.9005,
+      "step": 770
+    },
+    {
+      "epoch": 3.5779816513761467,
+      "grad_norm": 0.6726359128952026,
+      "learning_rate": 0.0001,
+      "loss": 0.8876,
+      "step": 780
+    },
+    {
+      "epoch": 3.623853211009174,
+      "grad_norm": 0.7526884078979492,
+      "learning_rate": 0.0001,
+      "loss": 0.9137,
+      "step": 790
+    },
+    {
+      "epoch": 3.669724770642202,
+      "grad_norm": 0.8159477114677429,
+      "learning_rate": 0.0001,
+      "loss": 0.9261,
+      "step": 800
+    },
+    {
+      "epoch": 3.7155963302752295,
+      "grad_norm": 0.630791425704956,
+      "learning_rate": 0.0001,
+      "loss": 0.9389,
+      "step": 810
+    },
+    {
+      "epoch": 3.761467889908257,
+      "grad_norm": 0.7433311343193054,
+      "learning_rate": 0.0001,
+      "loss": 0.91,
+      "step": 820
+    },
+    {
+      "epoch": 3.8073394495412844,
+      "grad_norm": 0.7488676905632019,
+      "learning_rate": 0.0001,
+      "loss": 0.9171,
+      "step": 830
+    },
+    {
+      "epoch": 3.853211009174312,
+      "grad_norm": 0.7516036629676819,
+      "learning_rate": 0.0001,
+      "loss": 0.9384,
+      "step": 840
+    },
+    {
+      "epoch": 3.8990825688073394,
+      "grad_norm": 0.705337643623352,
+      "learning_rate": 0.0001,
+      "loss": 0.9306,
+      "step": 850
+    },
+    {
+      "epoch": 3.944954128440367,
+      "grad_norm": 0.8075425624847412,
+      "learning_rate": 0.0001,
+      "loss": 0.9641,
+      "step": 860
+    },
+    {
+      "epoch": 3.9908256880733948,
+      "grad_norm": 0.6263691782951355,
+      "learning_rate": 0.0001,
+      "loss": 0.8892,
+      "step": 870
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.0362694263458252,
+      "eval_runtime": 8.3051,
+      "eval_samples_per_second": 13.124,
+      "eval_steps_per_second": 1.686,
+      "step": 872
+    },
+    {
+      "epoch": 4.036697247706422,
+      "grad_norm": 0.6914173364639282,
+      "learning_rate": 0.0001,
+      "loss": 0.8789,
+      "step": 880
+    },
+    {
+      "epoch": 4.08256880733945,
+      "grad_norm": 0.8341014385223389,
+      "learning_rate": 0.0001,
+      "loss": 0.8824,
+      "step": 890
+    },
+    {
+      "epoch": 4.128440366972477,
+      "grad_norm": 1.025681972503662,
+      "learning_rate": 0.0001,
+      "loss": 0.8467,
+      "step": 900
+    },
+    {
+      "epoch": 4.174311926605505,
+      "grad_norm": 0.7425392270088196,
+      "learning_rate": 0.0001,
+      "loss": 0.8526,
+      "step": 910
+    },
+    {
+      "epoch": 4.220183486238533,
+      "grad_norm": 0.8017181158065796,
+      "learning_rate": 0.0001,
+      "loss": 0.8456,
+      "step": 920
+    },
+    {
+      "epoch": 4.26605504587156,
+      "grad_norm": 0.7509899139404297,
+      "learning_rate": 0.0001,
+      "loss": 0.8627,
+      "step": 930
+    },
+    {
+      "epoch": 4.3119266055045875,
+      "grad_norm": 0.8284080624580383,
+      "learning_rate": 0.0001,
+      "loss": 0.8489,
+      "step": 940
+    },
+    {
+      "epoch": 4.3577981651376145,
+      "grad_norm": 0.983587920665741,
+      "learning_rate": 0.0001,
+      "loss": 0.8805,
+      "step": 950
+    },
+    {
+      "epoch": 4.4036697247706424,
+      "grad_norm": 0.7172166109085083,
+      "learning_rate": 0.0001,
+      "loss": 0.8659,
+      "step": 960
+    },
+    {
+      "epoch": 4.4495412844036695,
+      "grad_norm": 0.8484768271446228,
+      "learning_rate": 0.0001,
+      "loss": 0.8489,
+      "step": 970
+    },
+    {
+      "epoch": 4.495412844036697,
+      "grad_norm": 1.2530163526535034,
+      "learning_rate": 0.0001,
+      "loss": 0.8486,
+      "step": 980
+    },
+    {
+      "epoch": 4.541284403669724,
+      "grad_norm": 0.6953453421592712,
+      "learning_rate": 0.0001,
+      "loss": 0.8712,
+      "step": 990
+    },
+    {
+      "epoch": 4.587155963302752,
+      "grad_norm": 0.9675477743148804,
+      "learning_rate": 0.0001,
+      "loss": 0.8496,
+      "step": 1000
+    },
+    {
+      "epoch": 4.63302752293578,
+      "grad_norm": 0.6756150126457214,
+      "learning_rate": 0.0001,
+      "loss": 0.8696,
+      "step": 1010
+    },
+    {
+      "epoch": 4.678899082568807,
+      "grad_norm": 0.7769085764884949,
+      "learning_rate": 0.0001,
+      "loss": 0.8433,
+      "step": 1020
+    },
+    {
+      "epoch": 4.724770642201835,
+      "grad_norm": 0.7071849703788757,
+      "learning_rate": 0.0001,
+      "loss": 0.8315,
+      "step": 1030
+    },
+    {
+      "epoch": 4.770642201834862,
+      "grad_norm": 0.7788539528846741,
+      "learning_rate": 0.0001,
+      "loss": 0.8305,
+      "step": 1040
+    },
+    {
+      "epoch": 4.81651376146789,
+      "grad_norm": 0.8174529671669006,
+      "learning_rate": 0.0001,
+      "loss": 0.793,
+      "step": 1050
+    },
+    {
+      "epoch": 4.862385321100917,
+      "grad_norm": 0.9944226145744324,
+      "learning_rate": 0.0001,
+      "loss": 0.8871,
+      "step": 1060
+    },
+    {
+      "epoch": 4.908256880733945,
+      "grad_norm": 1.0104237794876099,
+      "learning_rate": 0.0001,
+      "loss": 0.8497,
+      "step": 1070
+    },
+    {
+      "epoch": 4.954128440366972,
+      "grad_norm": 0.7839273810386658,
+      "learning_rate": 0.0001,
+      "loss": 0.8683,
+      "step": 1080
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 1.135035753250122,
+      "learning_rate": 0.0001,
+      "loss": 0.8428,
+      "step": 1090
+    },
+    {
+      "epoch": 5.0,
+      "eval_loss": 1.0589717626571655,
+      "eval_runtime": 8.3188,
+      "eval_samples_per_second": 13.103,
+      "eval_steps_per_second": 1.683,
+      "step": 1090
+    },
+    {
+      "epoch": 5.045871559633028,
+      "grad_norm": 0.8675414323806763,
+      "learning_rate": 0.0001,
+      "loss": 0.7846,
+      "step": 1100
+    },
+    {
+      "epoch": 5.091743119266055,
+      "grad_norm": 0.9152688980102539,
+      "learning_rate": 0.0001,
+      "loss": 0.7525,
+      "step": 1110
+    },
+    {
+      "epoch": 5.137614678899083,
+      "grad_norm": 1.057682752609253,
+      "learning_rate": 0.0001,
+      "loss": 0.8033,
+      "step": 1120
+    },
+    {
+      "epoch": 5.18348623853211,
+      "grad_norm": 0.9883639812469482,
+      "learning_rate": 0.0001,
+      "loss": 0.8064,
+      "step": 1130
+    },
+    {
+      "epoch": 5.229357798165138,
+      "grad_norm": 0.908318817615509,
+      "learning_rate": 0.0001,
+      "loss": 0.8043,
+      "step": 1140
+    },
+    {
+      "epoch": 5.275229357798165,
+      "grad_norm": 0.9290405511856079,
+      "learning_rate": 0.0001,
+      "loss": 0.7897,
+      "step": 1150
+    },
+    {
+      "epoch": 5.321100917431193,
+      "grad_norm": 1.0225027799606323,
+      "learning_rate": 0.0001,
+      "loss": 0.7678,
+      "step": 1160
+    },
+    {
+      "epoch": 5.36697247706422,
+      "grad_norm": 0.723305344581604,
+      "learning_rate": 0.0001,
+      "loss": 0.7998,
+      "step": 1170
+    },
+    {
+      "epoch": 5.412844036697248,
+      "grad_norm": 0.9404224753379822,
+      "learning_rate": 0.0001,
+      "loss": 0.7691,
+      "step": 1180
+    },
+    {
+      "epoch": 5.458715596330276,
+      "grad_norm": 1.2413196563720703,
+      "learning_rate": 0.0001,
+      "loss": 0.7553,
+      "step": 1190
+    },
+    {
+      "epoch": 5.504587155963303,
+      "grad_norm": 0.9953755140304565,
+      "learning_rate": 0.0001,
+      "loss": 0.7473,
+      "step": 1200
+    },
+    {
+      "epoch": 5.5504587155963305,
+      "grad_norm": 0.8769338130950928,
+      "learning_rate": 0.0001,
+      "loss": 0.7919,
+      "step": 1210
+    },
+    {
+      "epoch": 5.5963302752293576,
+      "grad_norm": 0.7918237447738647,
+      "learning_rate": 0.0001,
+      "loss": 0.7754,
+      "step": 1220
+    },
+    {
+      "epoch": 5.6422018348623855,
+      "grad_norm": 0.8943650126457214,
+      "learning_rate": 0.0001,
+      "loss": 0.7726,
+      "step": 1230
+    },
+    {
+      "epoch": 5.6880733944954125,
+      "grad_norm": 0.9250347018241882,
+      "learning_rate": 0.0001,
+      "loss": 0.8007,
+      "step": 1240
+    },
+    {
+      "epoch": 5.73394495412844,
+      "grad_norm": 1.034350037574768,
+      "learning_rate": 0.0001,
+      "loss": 0.8033,
+      "step": 1250
+    },
+    {
+      "epoch": 5.779816513761467,
+      "grad_norm": 1.0044519901275635,
+      "learning_rate": 0.0001,
+      "loss": 0.7616,
+      "step": 1260
+    },
+    {
+      "epoch": 5.825688073394495,
+      "grad_norm": 0.8655275702476501,
+      "learning_rate": 0.0001,
+      "loss": 0.8549,
+      "step": 1270
+    },
+    {
+      "epoch": 5.871559633027523,
+      "grad_norm": 0.8263447284698486,
+      "learning_rate": 0.0001,
+      "loss": 0.7996,
+      "step": 1280
+    },
+    {
+      "epoch": 5.91743119266055,
+      "grad_norm": 0.679572582244873,
+      "learning_rate": 0.0001,
+      "loss": 0.7603,
+      "step": 1290
+    },
+    {
+      "epoch": 5.963302752293578,
+      "grad_norm": 0.9471953511238098,
+      "learning_rate": 0.0001,
+      "loss": 0.7782,
+      "step": 1300
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 1.0743499994277954,
+      "eval_runtime": 8.2992,
+      "eval_samples_per_second": 13.134,
+      "eval_steps_per_second": 1.687,
+      "step": 1308
+    },
+    {
+      "epoch": 6.009174311926605,
+      "grad_norm": 1.0349316596984863,
+      "learning_rate": 0.0001,
+      "loss": 0.7788,
+      "step": 1310
+    },
+    {
+      "epoch": 6.055045871559633,
+      "grad_norm": 1.4172323942184448,
+      "learning_rate": 0.0001,
+      "loss": 0.7197,
+      "step": 1320
+    },
+    {
+      "epoch": 6.10091743119266,
+      "grad_norm": 1.2426704168319702,
+      "learning_rate": 0.0001,
+      "loss": 0.7256,
+      "step": 1330
+    },
+    {
+      "epoch": 6.146788990825688,
+      "grad_norm": 1.2230033874511719,
+      "learning_rate": 0.0001,
+      "loss": 0.693,
+      "step": 1340
+    },
+    {
+      "epoch": 6.192660550458716,
+      "grad_norm": 0.9299599528312683,
+      "learning_rate": 0.0001,
+      "loss": 0.7081,
+      "step": 1350
+    },
+    {
+      "epoch": 6.238532110091743,
+      "grad_norm": 1.1219823360443115,
+      "learning_rate": 0.0001,
+      "loss": 0.6719,
+      "step": 1360
+    },
+    {
+      "epoch": 6.284403669724771,
+      "grad_norm": 1.0289041996002197,
+      "learning_rate": 0.0001,
+      "loss": 0.745,
+      "step": 1370
+    },
+    {
+      "epoch": 6.330275229357798,
+      "grad_norm": 1.0673019886016846,
+      "learning_rate": 0.0001,
+      "loss": 0.7207,
+      "step": 1380
+    },
+    {
+      "epoch": 6.376146788990826,
+      "grad_norm": 1.1395819187164307,
+      "learning_rate": 0.0001,
+      "loss": 0.7073,
+      "step": 1390
+    },
+    {
+      "epoch": 6.422018348623853,
+      "grad_norm": 0.9172726273536682,
+      "learning_rate": 0.0001,
+      "loss": 0.7328,
+      "step": 1400
+    },
+    {
+      "epoch": 6.467889908256881,
+      "grad_norm": 1.0613523721694946,
+      "learning_rate": 0.0001,
+      "loss": 0.6788,
+      "step": 1410
+    },
+    {
+      "epoch": 6.513761467889909,
+      "grad_norm": 0.9698522090911865,
+      "learning_rate": 0.0001,
+      "loss": 0.7287,
+      "step": 1420
+    },
+    {
+      "epoch": 6.559633027522936,
+      "grad_norm": 1.0680975914001465,
+      "learning_rate": 0.0001,
+      "loss": 0.7286,
+      "step": 1430
+    },
+    {
+      "epoch": 6.605504587155964,
+      "grad_norm": 0.8172698020935059,
+      "learning_rate": 0.0001,
+      "loss": 0.7336,
+      "step": 1440
+    },
+    {
+      "epoch": 6.651376146788991,
+      "grad_norm": 1.1772243976593018,
+      "learning_rate": 0.0001,
+      "loss": 0.7084,
+      "step": 1450
+    },
+    {
+      "epoch": 6.697247706422019,
+      "grad_norm": 1.00876784324646,
+      "learning_rate": 0.0001,
+      "loss": 0.7194,
+      "step": 1460
+    },
+    {
+      "epoch": 6.743119266055046,
+      "grad_norm": 1.3571248054504395,
+      "learning_rate": 0.0001,
+      "loss": 0.7332,
+      "step": 1470
+    },
+    {
+      "epoch": 6.7889908256880735,
+      "grad_norm": 1.093043565750122,
+      "learning_rate": 0.0001,
+      "loss": 0.7401,
+      "step": 1480
+    },
+    {
+      "epoch": 6.834862385321101,
+      "grad_norm": 1.4730535745620728,
+      "learning_rate": 0.0001,
+      "loss": 0.7009,
+      "step": 1490
+    },
+    {
+      "epoch": 6.8807339449541285,
+      "grad_norm": 0.9187682271003723,
+      "learning_rate": 0.0001,
+      "loss": 0.737,
+      "step": 1500
+    },
+    {
+      "epoch": 6.926605504587156,
+      "grad_norm": 1.406137466430664,
+      "learning_rate": 0.0001,
+      "loss": 0.6528,
+      "step": 1510
+    },
+    {
+      "epoch": 6.972477064220183,
+      "grad_norm": 1.1134064197540283,
+      "learning_rate": 0.0001,
+      "loss": 0.7074,
+      "step": 1520
+    },
+    {
+      "epoch": 7.0,
+      "eval_loss": 1.1300649642944336,
+      "eval_runtime": 8.3014,
+      "eval_samples_per_second": 13.13,
+      "eval_steps_per_second": 1.686,
+      "step": 1526
+    },
+    {
+      "epoch": 7.018348623853211,
+      "grad_norm": 1.0100208520889282,
+      "learning_rate": 0.0001,
+      "loss": 0.6884,
+      "step": 1530
+    },
+    {
+      "epoch": 7.064220183486238,
+      "grad_norm": 1.118424654006958,
+      "learning_rate": 0.0001,
+      "loss": 0.616,
+      "step": 1540
+    },
+    {
+      "epoch": 7.110091743119266,
+      "grad_norm": 1.416618824005127,
+      "learning_rate": 0.0001,
+      "loss": 0.6342,
+      "step": 1550
+    },
+    {
+      "epoch": 7.155963302752293,
+      "grad_norm": 1.3433297872543335,
+      "learning_rate": 0.0001,
+      "loss": 0.6165,
+      "step": 1560
+    },
+    {
+      "epoch": 7.201834862385321,
+      "grad_norm": 1.182295799255371,
+      "learning_rate": 0.0001,
+      "loss": 0.6481,
+      "step": 1570
+    },
+    {
+      "epoch": 7.247706422018348,
+      "grad_norm": 1.105743408203125,
+      "learning_rate": 0.0001,
+      "loss": 0.601,
+      "step": 1580
+    },
+    {
+      "epoch": 7.293577981651376,
+      "grad_norm": 1.1156800985336304,
+      "learning_rate": 0.0001,
+      "loss": 0.6558,
+      "step": 1590
+    },
+    {
+      "epoch": 7.339449541284404,
+      "grad_norm": 1.1464107036590576,
+      "learning_rate": 0.0001,
+      "loss": 0.6231,
+      "step": 1600
+    },
+    {
+      "epoch": 7.385321100917431,
+      "grad_norm": 1.4415833950042725,
+      "learning_rate": 0.0001,
+      "loss": 0.6549,
+      "step": 1610
+    },
+    {
+      "epoch": 7.431192660550459,
+      "grad_norm": 1.2493728399276733,
+      "learning_rate": 0.0001,
+      "loss": 0.6411,
+      "step": 1620
+    },
+    {
+      "epoch": 7.477064220183486,
+      "grad_norm": 1.0307130813598633,
+      "learning_rate": 0.0001,
+      "loss": 0.6331,
+      "step": 1630
+    },
+    {
+      "epoch": 7.522935779816514,
+      "grad_norm": 1.0896414518356323,
+      "learning_rate": 0.0001,
+      "loss": 0.6351,
+      "step": 1640
+    },
+    {
+      "epoch": 7.568807339449541,
+      "grad_norm": 1.2772048711776733,
+      "learning_rate": 0.0001,
+      "loss": 0.6995,
+      "step": 1650
+    },
+    {
+      "epoch": 7.614678899082569,
+      "grad_norm": 1.341843605041504,
+      "learning_rate": 0.0001,
+      "loss": 0.6376,
+      "step": 1660
+    },
+    {
+      "epoch": 7.660550458715596,
+      "grad_norm": 1.1451807022094727,
+      "learning_rate": 0.0001,
+      "loss": 0.6727,
+      "step": 1670
+    },
+    {
+      "epoch": 7.706422018348624,
+      "grad_norm": 1.1993539333343506,
+      "learning_rate": 0.0001,
+      "loss": 0.6645,
+      "step": 1680
+    },
+    {
+      "epoch": 7.752293577981652,
+      "grad_norm": 1.0638046264648438,
+      "learning_rate": 0.0001,
+      "loss": 0.646,
+      "step": 1690
+    },
+    {
+      "epoch": 7.798165137614679,
+      "grad_norm": 1.1782898902893066,
+      "learning_rate": 0.0001,
+      "loss": 0.6317,
+      "step": 1700
+    },
+    {
+      "epoch": 7.844036697247707,
+      "grad_norm": 1.1371848583221436,
+      "learning_rate": 0.0001,
+      "loss": 0.6367,
+      "step": 1710
+    },
+    {
+      "epoch": 7.889908256880734,
+      "grad_norm": 1.3090063333511353,
+      "learning_rate": 0.0001,
+      "loss": 0.6617,
+      "step": 1720
+    },
+    {
+      "epoch": 7.935779816513762,
+      "grad_norm": 1.1786032915115356,
+      "learning_rate": 0.0001,
+      "loss": 0.6529,
+      "step": 1730
+    },
+    {
+      "epoch": 7.981651376146789,
+      "grad_norm": 1.4673367738723755,
+      "learning_rate": 0.0001,
+      "loss": 0.681,
+      "step": 1740
+    },
+    {
+      "epoch": 8.0,
+      "eval_loss": 1.1897032260894775,
+      "eval_runtime": 8.2973,
+      "eval_samples_per_second": 13.137,
+      "eval_steps_per_second": 1.687,
+      "step": 1744
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1744,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 8.070841502584013e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-1744/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fbd275a27ab08d2e5458e423282c9f72d2ca0deb32b3380f16bbc843fea6e0d
+size 5560

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-218/README.md

@@ -0,0 +1,202 @@
+---
+base_model: meta-llama/Meta-Llama-3-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-218/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Meta-Llama-3-8B-Instruct_int4_flare-en-fpb_lr-0.0001_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-1.0-num-3100-sd-1/checkpoint-218/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3a06d7fd179f7c2ea0607cd764032b37c9611e0b4ba27b07258c552e977644cb
+size 109069176