Training in progress, step 345, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: katuni4ka/tiny-random-falcon-40b
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,31 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "katuni4ka/tiny-random-falcon-40b",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "dense",
+    "dense_h_to_4h",
+    "dense_4h_to_h",
+    "query_key_value"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:26c2732ae22d6babda155af79b5cb91f4d4e11b94c72cb4d54373dc84e41a8dc
+size 125040

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b470787b406db98f576038d7ba8ee1fa63eef5334e0bea32ae3aaba4312559e
+size 162868

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f45c92add258b0e6eb71d53b54445c891dd1b453c771bfe824e8b6a5a1ecf4f7
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f549cd01d3025604157e79707eeb03b302e046e33bf895a4b0d7a56ddff8dec2
+size 1064

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,29 @@
+{
+  "additional_special_tokens": [
+    ">>TITLE<<",
+    ">>ABSTRACT<<",
+    ">>INTRODUCTION<<",
+    ">>SUMMARY<<",
+    ">>COMMENT<<",
+    ">>ANSWER<<",
+    ">>QUESTION<<",
+    ">>DOMAIN<<",
+    ">>PREFIX<<",
+    ">>SUFFIX<<",
+    ">>MIDDLE<<"
+  ],
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,124 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": ">>TITLE<<",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": ">>ABSTRACT<<",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": ">>INTRODUCTION<<",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": ">>SUMMARY<<",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": ">>COMMENT<<",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "5": {
+      "content": ">>ANSWER<<",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "6": {
+      "content": ">>QUESTION<<",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "7": {
+      "content": ">>DOMAIN<<",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "8": {
+      "content": ">>PREFIX<<",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "9": {
+      "content": ">>SUFFIX<<",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "10": {
+      "content": ">>MIDDLE<<",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "11": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    ">>TITLE<<",
+    ">>ABSTRACT<<",
+    ">>INTRODUCTION<<",
+    ">>SUMMARY<<",
+    ">>COMMENT<<",
+    ">>ANSWER<<",
+    ">>QUESTION<<",
+    ">>DOMAIN<<",
+    ">>PREFIX<<",
+    ">>SUFFIX<<",
+    ">>MIDDLE<<"
+  ],
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 2048,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "PreTrainedTokenizerFast"
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,2464 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.03421940091251736,
+  "eval_steps": 345,
+  "global_step": 345,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 9.918666931164451e-05,
+      "grad_norm": 1.620390772819519,
+      "learning_rate": 2e-05,
+      "loss": 44.4878,
+      "step": 1
+    },
+    {
+      "epoch": 9.918666931164451e-05,
+      "eval_loss": 11.11313247680664,
+      "eval_runtime": 12.6618,
+      "eval_samples_per_second": 335.261,
+      "eval_steps_per_second": 167.67,
+      "step": 1
+    },
+    {
+      "epoch": 0.00019837333862328903,
+      "grad_norm": 1.4469047784805298,
+      "learning_rate": 4e-05,
+      "loss": 44.5058,
+      "step": 2
+    },
+    {
+      "epoch": 0.00029756000793493357,
+      "grad_norm": 1.5632233619689941,
+      "learning_rate": 6e-05,
+      "loss": 44.4768,
+      "step": 3
+    },
+    {
+      "epoch": 0.00039674667724657806,
+      "grad_norm": 1.4555176496505737,
+      "learning_rate": 8e-05,
+      "loss": 44.4999,
+      "step": 4
+    },
+    {
+      "epoch": 0.0004959333465582226,
+      "grad_norm": 1.6239758729934692,
+      "learning_rate": 0.0001,
+      "loss": 44.3626,
+      "step": 5
+    },
+    {
+      "epoch": 0.0005951200158698671,
+      "grad_norm": 1.8492672443389893,
+      "learning_rate": 0.00012,
+      "loss": 44.5033,
+      "step": 6
+    },
+    {
+      "epoch": 0.0006943066851815116,
+      "grad_norm": 1.629564881324768,
+      "learning_rate": 0.00014,
+      "loss": 44.5071,
+      "step": 7
+    },
+    {
+      "epoch": 0.0007934933544931561,
+      "grad_norm": 1.5436326265335083,
+      "learning_rate": 0.00016,
+      "loss": 44.5455,
+      "step": 8
+    },
+    {
+      "epoch": 0.0008926800238048007,
+      "grad_norm": 1.712166666984558,
+      "learning_rate": 0.00018,
+      "loss": 44.3016,
+      "step": 9
+    },
+    {
+      "epoch": 0.0009918666931164452,
+      "grad_norm": 1.554046392440796,
+      "learning_rate": 0.0002,
+      "loss": 44.4516,
+      "step": 10
+    },
+    {
+      "epoch": 0.0010910533624280897,
+      "grad_norm": 1.6125948429107666,
+      "learning_rate": 0.00019999973592181317,
+      "loss": 44.3897,
+      "step": 11
+    },
+    {
+      "epoch": 0.0011902400317397343,
+      "grad_norm": 1.6851387023925781,
+      "learning_rate": 0.00019999894368864744,
+      "loss": 44.5396,
+      "step": 12
+    },
+    {
+      "epoch": 0.0012894267010513786,
+      "grad_norm": 1.5964475870132446,
+      "learning_rate": 0.00019999762330468704,
+      "loss": 44.5239,
+      "step": 13
+    },
+    {
+      "epoch": 0.0013886133703630231,
+      "grad_norm": 1.717020034790039,
+      "learning_rate": 0.00019999577477690568,
+      "loss": 44.5325,
+      "step": 14
+    },
+    {
+      "epoch": 0.0014878000396746677,
+      "grad_norm": 1.6545541286468506,
+      "learning_rate": 0.00019999339811506642,
+      "loss": 44.3654,
+      "step": 15
+    },
+    {
+      "epoch": 0.0015869867089863122,
+      "grad_norm": 1.6294705867767334,
+      "learning_rate": 0.00019999049333172182,
+      "loss": 44.5449,
+      "step": 16
+    },
+    {
+      "epoch": 0.0016861733782979568,
+      "grad_norm": 1.6682461500167847,
+      "learning_rate": 0.0001999870604422136,
+      "loss": 44.3407,
+      "step": 17
+    },
+    {
+      "epoch": 0.0017853600476096013,
+      "grad_norm": 1.4981224536895752,
+      "learning_rate": 0.0001999830994646729,
+      "loss": 44.407,
+      "step": 18
+    },
+    {
+      "epoch": 0.0018845467169212459,
+      "grad_norm": 1.670657753944397,
+      "learning_rate": 0.00019997861042001972,
+      "loss": 44.3801,
+      "step": 19
+    },
+    {
+      "epoch": 0.0019837333862328904,
+      "grad_norm": 1.7518388032913208,
+      "learning_rate": 0.00019997359333196339,
+      "loss": 44.4139,
+      "step": 20
+    },
+    {
+      "epoch": 0.002082920055544535,
+      "grad_norm": 1.7747282981872559,
+      "learning_rate": 0.0001999680482270019,
+      "loss": 44.3652,
+      "step": 21
+    },
+    {
+      "epoch": 0.0021821067248561795,
+      "grad_norm": 1.973069667816162,
+      "learning_rate": 0.00019996197513442204,
+      "loss": 44.4699,
+      "step": 22
+    },
+    {
+      "epoch": 0.002281293394167824,
+      "grad_norm": 1.9039539098739624,
+      "learning_rate": 0.00019995537408629932,
+      "loss": 44.5052,
+      "step": 23
+    },
+    {
+      "epoch": 0.0023804800634794686,
+      "grad_norm": 1.5412516593933105,
+      "learning_rate": 0.00019994824511749757,
+      "loss": 44.3189,
+      "step": 24
+    },
+    {
+      "epoch": 0.0024796667327911127,
+      "grad_norm": 1.517510175704956,
+      "learning_rate": 0.00019994058826566886,
+      "loss": 44.269,
+      "step": 25
+    },
+    {
+      "epoch": 0.002578853402102757,
+      "grad_norm": 1.7756372690200806,
+      "learning_rate": 0.00019993240357125337,
+      "loss": 44.2606,
+      "step": 26
+    },
+    {
+      "epoch": 0.0026780400714144018,
+      "grad_norm": 1.606436014175415,
+      "learning_rate": 0.00019992369107747906,
+      "loss": 44.4704,
+      "step": 27
+    },
+    {
+      "epoch": 0.0027772267407260463,
+      "grad_norm": 2.018287181854248,
+      "learning_rate": 0.00019991445083036155,
+      "loss": 44.3927,
+      "step": 28
+    },
+    {
+      "epoch": 0.002876413410037691,
+      "grad_norm": 2.0493083000183105,
+      "learning_rate": 0.0001999046828787038,
+      "loss": 44.2501,
+      "step": 29
+    },
+    {
+      "epoch": 0.0029756000793493354,
+      "grad_norm": 1.8427711725234985,
+      "learning_rate": 0.00019989438727409584,
+      "loss": 44.4513,
+      "step": 30
+    },
+    {
+      "epoch": 0.00307478674866098,
+      "grad_norm": 1.6964459419250488,
+      "learning_rate": 0.00019988356407091455,
+      "loss": 44.2702,
+      "step": 31
+    },
+    {
+      "epoch": 0.0031739734179726245,
+      "grad_norm": 1.9234622716903687,
+      "learning_rate": 0.00019987221332632343,
+      "loss": 44.4652,
+      "step": 32
+    },
+    {
+      "epoch": 0.003273160087284269,
+      "grad_norm": 1.8713277578353882,
+      "learning_rate": 0.0001998603351002721,
+      "loss": 44.2959,
+      "step": 33
+    },
+    {
+      "epoch": 0.0033723467565959135,
+      "grad_norm": 2.15635085105896,
+      "learning_rate": 0.0001998479294554962,
+      "loss": 44.2018,
+      "step": 34
+    },
+    {
+      "epoch": 0.003471533425907558,
+      "grad_norm": 2.2415690422058105,
+      "learning_rate": 0.00019983499645751692,
+      "loss": 44.387,
+      "step": 35
+    },
+    {
+      "epoch": 0.0035707200952192026,
+      "grad_norm": 1.9001964330673218,
+      "learning_rate": 0.00019982153617464072,
+      "loss": 44.2773,
+      "step": 36
+    },
+    {
+      "epoch": 0.003669906764530847,
+      "grad_norm": 2.3937878608703613,
+      "learning_rate": 0.00019980754867795898,
+      "loss": 44.372,
+      "step": 37
+    },
+    {
+      "epoch": 0.0037690934338424917,
+      "grad_norm": 2.18562650680542,
+      "learning_rate": 0.00019979303404134746,
+      "loss": 44.206,
+      "step": 38
+    },
+    {
+      "epoch": 0.0038682801031541363,
+      "grad_norm": 2.0082168579101562,
+      "learning_rate": 0.00019977799234146618,
+      "loss": 44.2083,
+      "step": 39
+    },
+    {
+      "epoch": 0.003967466772465781,
+      "grad_norm": 2.294412612915039,
+      "learning_rate": 0.0001997624236577589,
+      "loss": 44.2663,
+      "step": 40
+    },
+    {
+      "epoch": 0.004066653441777425,
+      "grad_norm": 2.418440818786621,
+      "learning_rate": 0.00019974632807245253,
+      "loss": 44.0445,
+      "step": 41
+    },
+    {
+      "epoch": 0.00416584011108907,
+      "grad_norm": 2.312135934829712,
+      "learning_rate": 0.00019972970567055698,
+      "loss": 44.0673,
+      "step": 42
+    },
+    {
+      "epoch": 0.004265026780400714,
+      "grad_norm": 2.2133700847625732,
+      "learning_rate": 0.00019971255653986452,
+      "loss": 44.2475,
+      "step": 43
+    },
+    {
+      "epoch": 0.004364213449712359,
+      "grad_norm": 2.2712738513946533,
+      "learning_rate": 0.00019969488077094933,
+      "loss": 44.1074,
+      "step": 44
+    },
+    {
+      "epoch": 0.0044634001190240035,
+      "grad_norm": 2.1439149379730225,
+      "learning_rate": 0.0001996766784571672,
+      "loss": 44.037,
+      "step": 45
+    },
+    {
+      "epoch": 0.004562586788335648,
+      "grad_norm": 2.561309814453125,
+      "learning_rate": 0.00019965794969465473,
+      "loss": 44.0506,
+      "step": 46
+    },
+    {
+      "epoch": 0.004661773457647293,
+      "grad_norm": 2.5531442165374756,
+      "learning_rate": 0.00019963869458232907,
+      "loss": 44.2289,
+      "step": 47
+    },
+    {
+      "epoch": 0.004760960126958937,
+      "grad_norm": 2.356276512145996,
+      "learning_rate": 0.0001996189132218874,
+      "loss": 44.224,
+      "step": 48
+    },
+    {
+      "epoch": 0.004860146796270581,
+      "grad_norm": 2.1195573806762695,
+      "learning_rate": 0.00019959860571780615,
+      "loss": 44.2144,
+      "step": 49
+    },
+    {
+      "epoch": 0.004959333465582225,
+      "grad_norm": 2.272167205810547,
+      "learning_rate": 0.00019957777217734078,
+      "loss": 44.1248,
+      "step": 50
+    },
+    {
+      "epoch": 0.00505852013489387,
+      "grad_norm": 2.4642817974090576,
+      "learning_rate": 0.00019955641271052487,
+      "loss": 43.9688,
+      "step": 51
+    },
+    {
+      "epoch": 0.005157706804205514,
+      "grad_norm": 2.2487308979034424,
+      "learning_rate": 0.00019953452743016987,
+      "loss": 43.9119,
+      "step": 52
+    },
+    {
+      "epoch": 0.005256893473517159,
+      "grad_norm": 2.326240062713623,
+      "learning_rate": 0.00019951211645186426,
+      "loss": 44.0474,
+      "step": 53
+    },
+    {
+      "epoch": 0.0053560801428288035,
+      "grad_norm": 2.248692035675049,
+      "learning_rate": 0.00019948917989397308,
+      "loss": 44.0475,
+      "step": 54
+    },
+    {
+      "epoch": 0.005455266812140448,
+      "grad_norm": 2.3791592121124268,
+      "learning_rate": 0.00019946571787763718,
+      "loss": 43.9322,
+      "step": 55
+    },
+    {
+      "epoch": 0.005554453481452093,
+      "grad_norm": 2.520576000213623,
+      "learning_rate": 0.00019944173052677272,
+      "loss": 43.7626,
+      "step": 56
+    },
+    {
+      "epoch": 0.005653640150763737,
+      "grad_norm": 2.3382210731506348,
+      "learning_rate": 0.00019941721796807042,
+      "loss": 43.8322,
+      "step": 57
+    },
+    {
+      "epoch": 0.005752826820075382,
+      "grad_norm": 2.1466667652130127,
+      "learning_rate": 0.0001993921803309949,
+      "loss": 43.9254,
+      "step": 58
+    },
+    {
+      "epoch": 0.005852013489387026,
+      "grad_norm": 2.225789785385132,
+      "learning_rate": 0.00019936661774778408,
+      "loss": 43.9358,
+      "step": 59
+    },
+    {
+      "epoch": 0.005951200158698671,
+      "grad_norm": 2.6257212162017822,
+      "learning_rate": 0.00019934053035344836,
+      "loss": 43.694,
+      "step": 60
+    },
+    {
+      "epoch": 0.006050386828010315,
+      "grad_norm": 2.5033576488494873,
+      "learning_rate": 0.00019931391828576991,
+      "loss": 43.6864,
+      "step": 61
+    },
+    {
+      "epoch": 0.00614957349732196,
+      "grad_norm": 2.434095621109009,
+      "learning_rate": 0.00019928678168530212,
+      "loss": 43.7584,
+      "step": 62
+    },
+    {
+      "epoch": 0.006248760166633604,
+      "grad_norm": 2.2750728130340576,
+      "learning_rate": 0.0001992591206953687,
+      "loss": 43.7666,
+      "step": 63
+    },
+    {
+      "epoch": 0.006347946835945249,
+      "grad_norm": 1.9984159469604492,
+      "learning_rate": 0.0001992309354620629,
+      "loss": 43.7153,
+      "step": 64
+    },
+    {
+      "epoch": 0.0064471335052568935,
+      "grad_norm": 2.116001844406128,
+      "learning_rate": 0.00019920222613424682,
+      "loss": 43.8651,
+      "step": 65
+    },
+    {
+      "epoch": 0.006546320174568538,
+      "grad_norm": 2.1209514141082764,
+      "learning_rate": 0.00019917299286355058,
+      "loss": 43.7459,
+      "step": 66
+    },
+    {
+      "epoch": 0.0066455068438801826,
+      "grad_norm": 2.773214340209961,
+      "learning_rate": 0.00019914323580437162,
+      "loss": 43.6404,
+      "step": 67
+    },
+    {
+      "epoch": 0.006744693513191827,
+      "grad_norm": 2.400967836380005,
+      "learning_rate": 0.00019911295511387372,
+      "loss": 43.6164,
+      "step": 68
+    },
+    {
+      "epoch": 0.006843880182503472,
+      "grad_norm": 2.4684884548187256,
+      "learning_rate": 0.00019908215095198626,
+      "loss": 43.68,
+      "step": 69
+    },
+    {
+      "epoch": 0.006943066851815116,
+      "grad_norm": 2.086573600769043,
+      "learning_rate": 0.00019905082348140341,
+      "loss": 43.7257,
+      "step": 70
+    },
+    {
+      "epoch": 0.007042253521126761,
+      "grad_norm": 2.0327978134155273,
+      "learning_rate": 0.00019901897286758318,
+      "loss": 43.5133,
+      "step": 71
+    },
+    {
+      "epoch": 0.007141440190438405,
+      "grad_norm": 2.020599842071533,
+      "learning_rate": 0.00019898659927874662,
+      "loss": 43.5116,
+      "step": 72
+    },
+    {
+      "epoch": 0.00724062685975005,
+      "grad_norm": 2.3334968090057373,
+      "learning_rate": 0.00019895370288587695,
+      "loss": 43.5966,
+      "step": 73
+    },
+    {
+      "epoch": 0.007339813529061694,
+      "grad_norm": 2.053218126296997,
+      "learning_rate": 0.0001989202838627185,
+      "loss": 43.74,
+      "step": 74
+    },
+    {
+      "epoch": 0.007439000198373339,
+      "grad_norm": 2.083590269088745,
+      "learning_rate": 0.000198886342385776,
+      "loss": 43.6054,
+      "step": 75
+    },
+    {
+      "epoch": 0.007538186867684983,
+      "grad_norm": 2.0403592586517334,
+      "learning_rate": 0.00019885187863431352,
+      "loss": 43.5867,
+      "step": 76
+    },
+    {
+      "epoch": 0.007637373536996628,
+      "grad_norm": 2.0573623180389404,
+      "learning_rate": 0.00019881689279035354,
+      "loss": 43.6566,
+      "step": 77
+    },
+    {
+      "epoch": 0.0077365602063082725,
+      "grad_norm": 1.7056894302368164,
+      "learning_rate": 0.00019878138503867608,
+      "loss": 43.604,
+      "step": 78
+    },
+    {
+      "epoch": 0.007835746875619916,
+      "grad_norm": 2.111693859100342,
+      "learning_rate": 0.00019874535556681756,
+      "loss": 43.4557,
+      "step": 79
+    },
+    {
+      "epoch": 0.007934933544931562,
+      "grad_norm": 2.048046350479126,
+      "learning_rate": 0.0001987088045650699,
+      "loss": 43.5062,
+      "step": 80
+    },
+    {
+      "epoch": 0.008034120214243205,
+      "grad_norm": 2.2594175338745117,
+      "learning_rate": 0.0001986717322264796,
+      "loss": 43.5631,
+      "step": 81
+    },
+    {
+      "epoch": 0.00813330688355485,
+      "grad_norm": 2.0121192932128906,
+      "learning_rate": 0.00019863413874684653,
+      "loss": 43.681,
+      "step": 82
+    },
+    {
+      "epoch": 0.008232493552866494,
+      "grad_norm": 1.7314908504486084,
+      "learning_rate": 0.0001985960243247231,
+      "loss": 43.601,
+      "step": 83
+    },
+    {
+      "epoch": 0.00833168022217814,
+      "grad_norm": 1.969624638557434,
+      "learning_rate": 0.00019855738916141303,
+      "loss": 43.3999,
+      "step": 84
+    },
+    {
+      "epoch": 0.008430866891489783,
+      "grad_norm": 1.8517192602157593,
+      "learning_rate": 0.0001985182334609704,
+      "loss": 43.6391,
+      "step": 85
+    },
+    {
+      "epoch": 0.008530053560801429,
+      "grad_norm": 1.8946924209594727,
+      "learning_rate": 0.00019847855743019858,
+      "loss": 43.5514,
+      "step": 86
+    },
+    {
+      "epoch": 0.008629240230113073,
+      "grad_norm": 1.6315033435821533,
+      "learning_rate": 0.00019843836127864895,
+      "loss": 43.4945,
+      "step": 87
+    },
+    {
+      "epoch": 0.008728426899424718,
+      "grad_norm": 2.0259015560150146,
+      "learning_rate": 0.00019839764521862015,
+      "loss": 43.5663,
+      "step": 88
+    },
+    {
+      "epoch": 0.008827613568736362,
+      "grad_norm": 2.0214736461639404,
+      "learning_rate": 0.0001983564094651566,
+      "loss": 43.4939,
+      "step": 89
+    },
+    {
+      "epoch": 0.008926800238048007,
+      "grad_norm": 2.1720495223999023,
+      "learning_rate": 0.00019831465423604752,
+      "loss": 43.4768,
+      "step": 90
+    },
+    {
+      "epoch": 0.00902598690735965,
+      "grad_norm": 1.972532868385315,
+      "learning_rate": 0.00019827237975182592,
+      "loss": 43.5539,
+      "step": 91
+    },
+    {
+      "epoch": 0.009125173576671296,
+      "grad_norm": 1.7981455326080322,
+      "learning_rate": 0.00019822958623576714,
+      "loss": 43.5811,
+      "step": 92
+    },
+    {
+      "epoch": 0.00922436024598294,
+      "grad_norm": 1.8357104063034058,
+      "learning_rate": 0.0001981862739138878,
+      "loss": 43.5306,
+      "step": 93
+    },
+    {
+      "epoch": 0.009323546915294585,
+      "grad_norm": 1.6894700527191162,
+      "learning_rate": 0.00019814244301494474,
+      "loss": 43.5594,
+      "step": 94
+    },
+    {
+      "epoch": 0.009422733584606229,
+      "grad_norm": 1.7075917720794678,
+      "learning_rate": 0.00019809809377043368,
+      "loss": 43.6353,
+      "step": 95
+    },
+    {
+      "epoch": 0.009521920253917874,
+      "grad_norm": 1.8798929452896118,
+      "learning_rate": 0.0001980532264145879,
+      "loss": 43.4567,
+      "step": 96
+    },
+    {
+      "epoch": 0.009621106923229518,
+      "grad_norm": 1.9370898008346558,
+      "learning_rate": 0.00019800784118437727,
+      "loss": 43.5642,
+      "step": 97
+    },
+    {
+      "epoch": 0.009720293592541162,
+      "grad_norm": 1.7264550924301147,
+      "learning_rate": 0.00019796193831950673,
+      "loss": 43.5234,
+      "step": 98
+    },
+    {
+      "epoch": 0.009819480261852807,
+      "grad_norm": 2.1001346111297607,
+      "learning_rate": 0.0001979155180624152,
+      "loss": 43.3069,
+      "step": 99
+    },
+    {
+      "epoch": 0.00991866693116445,
+      "grad_norm": 1.971402883529663,
+      "learning_rate": 0.00019786858065827425,
+      "loss": 43.2957,
+      "step": 100
+    },
+    {
+      "epoch": 0.010017853600476096,
+      "grad_norm": 1.9676079750061035,
+      "learning_rate": 0.00019782112635498676,
+      "loss": 43.3939,
+      "step": 101
+    },
+    {
+      "epoch": 0.01011704026978774,
+      "grad_norm": 1.6693346500396729,
+      "learning_rate": 0.00019777315540318565,
+      "loss": 43.4189,
+      "step": 102
+    },
+    {
+      "epoch": 0.010216226939099385,
+      "grad_norm": 1.7066096067428589,
+      "learning_rate": 0.00019772466805623252,
+      "loss": 43.5922,
+      "step": 103
+    },
+    {
+      "epoch": 0.010315413608411029,
+      "grad_norm": 1.8379673957824707,
+      "learning_rate": 0.00019767566457021648,
+      "loss": 43.5591,
+      "step": 104
+    },
+    {
+      "epoch": 0.010414600277722674,
+      "grad_norm": 1.9390805959701538,
+      "learning_rate": 0.00019762614520395247,
+      "loss": 43.5645,
+      "step": 105
+    },
+    {
+      "epoch": 0.010513786947034318,
+      "grad_norm": 1.8997091054916382,
+      "learning_rate": 0.00019757611021898024,
+      "loss": 43.6896,
+      "step": 106
+    },
+    {
+      "epoch": 0.010612973616345963,
+      "grad_norm": 1.7241935729980469,
+      "learning_rate": 0.0001975255598795627,
+      "loss": 43.6881,
+      "step": 107
+    },
+    {
+      "epoch": 0.010712160285657607,
+      "grad_norm": 1.5904136896133423,
+      "learning_rate": 0.00019747449445268474,
+      "loss": 43.4718,
+      "step": 108
+    },
+    {
+      "epoch": 0.010811346954969252,
+      "grad_norm": 1.940871238708496,
+      "learning_rate": 0.00019742291420805165,
+      "loss": 43.3384,
+      "step": 109
+    },
+    {
+      "epoch": 0.010910533624280896,
+      "grad_norm": 1.9792636632919312,
+      "learning_rate": 0.0001973708194180878,
+      "loss": 43.5913,
+      "step": 110
+    },
+    {
+      "epoch": 0.011009720293592542,
+      "grad_norm": 2.008183240890503,
+      "learning_rate": 0.00019731821035793507,
+      "loss": 43.4205,
+      "step": 111
+    },
+    {
+      "epoch": 0.011108906962904185,
+      "grad_norm": 2.022135019302368,
+      "learning_rate": 0.00019726508730545162,
+      "loss": 43.385,
+      "step": 112
+    },
+    {
+      "epoch": 0.01120809363221583,
+      "grad_norm": 1.4658271074295044,
+      "learning_rate": 0.00019721145054121028,
+      "loss": 43.5396,
+      "step": 113
+    },
+    {
+      "epoch": 0.011307280301527474,
+      "grad_norm": 1.9923700094223022,
+      "learning_rate": 0.00019715730034849696,
+      "loss": 43.4149,
+      "step": 114
+    },
+    {
+      "epoch": 0.01140646697083912,
+      "grad_norm": 2.1149795055389404,
+      "learning_rate": 0.00019710263701330937,
+      "loss": 43.4565,
+      "step": 115
+    },
+    {
+      "epoch": 0.011505653640150763,
+      "grad_norm": 1.8142424821853638,
+      "learning_rate": 0.0001970474608243554,
+      "loss": 43.4097,
+      "step": 116
+    },
+    {
+      "epoch": 0.011604840309462409,
+      "grad_norm": 1.8109841346740723,
+      "learning_rate": 0.00019699177207305166,
+      "loss": 43.3104,
+      "step": 117
+    },
+    {
+      "epoch": 0.011704026978774052,
+      "grad_norm": 1.83271062374115,
+      "learning_rate": 0.0001969355710535218,
+      "loss": 43.5675,
+      "step": 118
+    },
+    {
+      "epoch": 0.011803213648085698,
+      "grad_norm": 2.103610038757324,
+      "learning_rate": 0.00019687885806259504,
+      "loss": 43.1547,
+      "step": 119
+    },
+    {
+      "epoch": 0.011902400317397342,
+      "grad_norm": 1.7136248350143433,
+      "learning_rate": 0.00019682163339980474,
+      "loss": 43.618,
+      "step": 120
+    },
+    {
+      "epoch": 0.012001586986708987,
+      "grad_norm": 1.6430232524871826,
+      "learning_rate": 0.00019676389736738656,
+      "loss": 43.5748,
+      "step": 121
+    },
+    {
+      "epoch": 0.01210077365602063,
+      "grad_norm": 1.9517197608947754,
+      "learning_rate": 0.00019670565027027706,
+      "loss": 43.4926,
+      "step": 122
+    },
+    {
+      "epoch": 0.012199960325332276,
+      "grad_norm": 2.109898567199707,
+      "learning_rate": 0.00019664689241611196,
+      "loss": 43.6146,
+      "step": 123
+    },
+    {
+      "epoch": 0.01229914699464392,
+      "grad_norm": 1.8593815565109253,
+      "learning_rate": 0.00019658762411522464,
+      "loss": 43.3389,
+      "step": 124
+    },
+    {
+      "epoch": 0.012398333663955565,
+      "grad_norm": 1.5332725048065186,
+      "learning_rate": 0.0001965278456806444,
+      "loss": 43.4073,
+      "step": 125
+    },
+    {
+      "epoch": 0.012497520333267209,
+      "grad_norm": 2.055220603942871,
+      "learning_rate": 0.00019646755742809487,
+      "loss": 43.051,
+      "step": 126
+    },
+    {
+      "epoch": 0.012596707002578854,
+      "grad_norm": 1.8248649835586548,
+      "learning_rate": 0.00019640675967599224,
+      "loss": 43.3472,
+      "step": 127
+    },
+    {
+      "epoch": 0.012695893671890498,
+      "grad_norm": 1.7612577676773071,
+      "learning_rate": 0.0001963454527454438,
+      "loss": 43.4539,
+      "step": 128
+    },
+    {
+      "epoch": 0.012795080341202143,
+      "grad_norm": 1.922441005706787,
+      "learning_rate": 0.00019628363696024592,
+      "loss": 43.3149,
+      "step": 129
+    },
+    {
+      "epoch": 0.012894267010513787,
+      "grad_norm": 1.77986478805542,
+      "learning_rate": 0.00019622131264688267,
+      "loss": 43.2937,
+      "step": 130
+    },
+    {
+      "epoch": 0.01299345367982543,
+      "grad_norm": 1.7966907024383545,
+      "learning_rate": 0.00019615848013452387,
+      "loss": 43.373,
+      "step": 131
+    },
+    {
+      "epoch": 0.013092640349137076,
+      "grad_norm": 1.9414621591567993,
+      "learning_rate": 0.00019609513975502342,
+      "loss": 43.2283,
+      "step": 132
+    },
+    {
+      "epoch": 0.01319182701844872,
+      "grad_norm": 1.8673967123031616,
+      "learning_rate": 0.0001960312918429176,
+      "loss": 43.5471,
+      "step": 133
+    },
+    {
+      "epoch": 0.013291013687760365,
+      "grad_norm": 2.33743953704834,
+      "learning_rate": 0.00019596693673542322,
+      "loss": 43.4404,
+      "step": 134
+    },
+    {
+      "epoch": 0.013390200357072009,
+      "grad_norm": 1.9352983236312866,
+      "learning_rate": 0.00019590207477243588,
+      "loss": 43.4078,
+      "step": 135
+    },
+    {
+      "epoch": 0.013489387026383654,
+      "grad_norm": 1.6548969745635986,
+      "learning_rate": 0.00019583670629652814,
+      "loss": 43.5059,
+      "step": 136
+    },
+    {
+      "epoch": 0.013588573695695298,
+      "grad_norm": 1.8436064720153809,
+      "learning_rate": 0.0001957708316529478,
+      "loss": 43.4167,
+      "step": 137
+    },
+    {
+      "epoch": 0.013687760365006943,
+      "grad_norm": 1.878476619720459,
+      "learning_rate": 0.00019570445118961602,
+      "loss": 43.4777,
+      "step": 138
+    },
+    {
+      "epoch": 0.013786947034318587,
+      "grad_norm": 1.7631700038909912,
+      "learning_rate": 0.0001956375652571254,
+      "loss": 43.4291,
+      "step": 139
+    },
+    {
+      "epoch": 0.013886133703630232,
+      "grad_norm": 1.9526875019073486,
+      "learning_rate": 0.00019557017420873825,
+      "loss": 43.6412,
+      "step": 140
+    },
+    {
+      "epoch": 0.013985320372941876,
+      "grad_norm": 1.9807161092758179,
+      "learning_rate": 0.00019550227840038476,
+      "loss": 43.1551,
+      "step": 141
+    },
+    {
+      "epoch": 0.014084507042253521,
+      "grad_norm": 1.9654165506362915,
+      "learning_rate": 0.0001954338781906609,
+      "loss": 43.4855,
+      "step": 142
+    },
+    {
+      "epoch": 0.014183693711565165,
+      "grad_norm": 1.8886873722076416,
+      "learning_rate": 0.00019536497394082676,
+      "loss": 43.4846,
+      "step": 143
+    },
+    {
+      "epoch": 0.01428288038087681,
+      "grad_norm": 1.9163804054260254,
+      "learning_rate": 0.0001952955660148045,
+      "loss": 43.5417,
+      "step": 144
+    },
+    {
+      "epoch": 0.014382067050188454,
+      "grad_norm": 1.6911556720733643,
+      "learning_rate": 0.00019522565477917653,
+      "loss": 43.737,
+      "step": 145
+    },
+    {
+      "epoch": 0.0144812537195001,
+      "grad_norm": 1.8554767370224,
+      "learning_rate": 0.0001951552406031835,
+      "loss": 43.5507,
+      "step": 146
+    },
+    {
+      "epoch": 0.014580440388811743,
+      "grad_norm": 1.7839596271514893,
+      "learning_rate": 0.00019508432385872238,
+      "loss": 43.5163,
+      "step": 147
+    },
+    {
+      "epoch": 0.014679627058123389,
+      "grad_norm": 1.5847241878509521,
+      "learning_rate": 0.00019501290492034444,
+      "loss": 43.4539,
+      "step": 148
+    },
+    {
+      "epoch": 0.014778813727435032,
+      "grad_norm": 2.139988422393799,
+      "learning_rate": 0.00019494098416525336,
+      "loss": 43.457,
+      "step": 149
+    },
+    {
+      "epoch": 0.014878000396746678,
+      "grad_norm": 1.82491934299469,
+      "learning_rate": 0.00019486856197330324,
+      "loss": 43.4277,
+      "step": 150
+    },
+    {
+      "epoch": 0.014977187066058321,
+      "grad_norm": 1.5014609098434448,
+      "learning_rate": 0.00019479563872699647,
+      "loss": 43.3014,
+      "step": 151
+    },
+    {
+      "epoch": 0.015076373735369967,
+      "grad_norm": 1.6398491859436035,
+      "learning_rate": 0.0001947222148114818,
+      "loss": 43.2079,
+      "step": 152
+    },
+    {
+      "epoch": 0.01517556040468161,
+      "grad_norm": 1.5315080881118774,
+      "learning_rate": 0.00019464829061455234,
+      "loss": 43.3428,
+      "step": 153
+    },
+    {
+      "epoch": 0.015274747073993256,
+      "grad_norm": 1.8612087965011597,
+      "learning_rate": 0.00019457386652664346,
+      "loss": 43.672,
+      "step": 154
+    },
+    {
+      "epoch": 0.0153739337433049,
+      "grad_norm": 1.6463797092437744,
+      "learning_rate": 0.0001944989429408307,
+      "loss": 43.3032,
+      "step": 155
+    },
+    {
+      "epoch": 0.015473120412616545,
+      "grad_norm": 1.8886929750442505,
+      "learning_rate": 0.00019442352025282776,
+      "loss": 43.409,
+      "step": 156
+    },
+    {
+      "epoch": 0.015572307081928189,
+      "grad_norm": 1.5860872268676758,
+      "learning_rate": 0.00019434759886098442,
+      "loss": 43.5002,
+      "step": 157
+    },
+    {
+      "epoch": 0.015671493751239832,
+      "grad_norm": 1.5877327919006348,
+      "learning_rate": 0.0001942711791662843,
+      "loss": 43.1642,
+      "step": 158
+    },
+    {
+      "epoch": 0.015770680420551478,
+      "grad_norm": 1.9605953693389893,
+      "learning_rate": 0.00019419426157234288,
+      "loss": 43.5132,
+      "step": 159
+    },
+    {
+      "epoch": 0.015869867089863123,
+      "grad_norm": 1.751679539680481,
+      "learning_rate": 0.00019411684648540538,
+      "loss": 43.5054,
+      "step": 160
+    },
+    {
+      "epoch": 0.01596905375917477,
+      "grad_norm": 1.4614412784576416,
+      "learning_rate": 0.00019403893431434445,
+      "loss": 43.3814,
+      "step": 161
+    },
+    {
+      "epoch": 0.01606824042848641,
+      "grad_norm": 1.5248690843582153,
+      "learning_rate": 0.00019396052547065827,
+      "loss": 43.5447,
+      "step": 162
+    },
+    {
+      "epoch": 0.016167427097798056,
+      "grad_norm": 1.5921061038970947,
+      "learning_rate": 0.0001938816203684681,
+      "loss": 43.5961,
+      "step": 163
+    },
+    {
+      "epoch": 0.0162666137671097,
+      "grad_norm": 1.978134036064148,
+      "learning_rate": 0.00019380221942451623,
+      "loss": 43.318,
+      "step": 164
+    },
+    {
+      "epoch": 0.016365800436421343,
+      "grad_norm": 1.8620237112045288,
+      "learning_rate": 0.00019372232305816387,
+      "loss": 43.4081,
+      "step": 165
+    },
+    {
+      "epoch": 0.01646498710573299,
+      "grad_norm": 1.890067219734192,
+      "learning_rate": 0.00019364193169138876,
+      "loss": 43.37,
+      "step": 166
+    },
+    {
+      "epoch": 0.016564173775044634,
+      "grad_norm": 1.7698861360549927,
+      "learning_rate": 0.000193561045748783,
+      "loss": 43.2327,
+      "step": 167
+    },
+    {
+      "epoch": 0.01666336044435628,
+      "grad_norm": 2.023280382156372,
+      "learning_rate": 0.00019347966565755084,
+      "loss": 43.5565,
+      "step": 168
+    },
+    {
+      "epoch": 0.01676254711366792,
+      "grad_norm": 1.60719895362854,
+      "learning_rate": 0.00019339779184750647,
+      "loss": 43.4494,
+      "step": 169
+    },
+    {
+      "epoch": 0.016861733782979567,
+      "grad_norm": 1.7296546697616577,
+      "learning_rate": 0.0001933154247510716,
+      "loss": 43.4502,
+      "step": 170
+    },
+    {
+      "epoch": 0.016960920452291212,
+      "grad_norm": 1.568786382675171,
+      "learning_rate": 0.00019323256480327335,
+      "loss": 43.53,
+      "step": 171
+    },
+    {
+      "epoch": 0.017060107121602858,
+      "grad_norm": 1.7664810419082642,
+      "learning_rate": 0.00019314921244174173,
+      "loss": 43.3674,
+      "step": 172
+    },
+    {
+      "epoch": 0.0171592937909145,
+      "grad_norm": 1.7805827856063843,
+      "learning_rate": 0.0001930653681067076,
+      "loss": 43.3056,
+      "step": 173
+    },
+    {
+      "epoch": 0.017258480460226145,
+      "grad_norm": 1.9585574865341187,
+      "learning_rate": 0.00019298103224100018,
+      "loss": 43.4174,
+      "step": 174
+    },
+    {
+      "epoch": 0.01735766712953779,
+      "grad_norm": 1.9280157089233398,
+      "learning_rate": 0.00019289620529004467,
+      "loss": 43.4921,
+      "step": 175
+    },
+    {
+      "epoch": 0.017456853798849436,
+      "grad_norm": 1.6859203577041626,
+      "learning_rate": 0.00019281088770186004,
+      "loss": 43.2668,
+      "step": 176
+    },
+    {
+      "epoch": 0.017556040468161078,
+      "grad_norm": 1.5108959674835205,
+      "learning_rate": 0.00019272507992705657,
+      "loss": 43.4231,
+      "step": 177
+    },
+    {
+      "epoch": 0.017655227137472723,
+      "grad_norm": 1.8462445735931396,
+      "learning_rate": 0.00019263878241883352,
+      "loss": 43.5939,
+      "step": 178
+    },
+    {
+      "epoch": 0.01775441380678437,
+      "grad_norm": 2.0469777584075928,
+      "learning_rate": 0.00019255199563297665,
+      "loss": 43.4041,
+      "step": 179
+    },
+    {
+      "epoch": 0.017853600476096014,
+      "grad_norm": 1.8307596445083618,
+      "learning_rate": 0.0001924647200278559,
+      "loss": 43.568,
+      "step": 180
+    },
+    {
+      "epoch": 0.017952787145407656,
+      "grad_norm": 1.7826749086380005,
+      "learning_rate": 0.00019237695606442293,
+      "loss": 43.4189,
+      "step": 181
+    },
+    {
+      "epoch": 0.0180519738147193,
+      "grad_norm": 1.6939207315444946,
+      "learning_rate": 0.00019228870420620874,
+      "loss": 43.7362,
+      "step": 182
+    },
+    {
+      "epoch": 0.018151160484030947,
+      "grad_norm": 1.5782077312469482,
+      "learning_rate": 0.00019219996491932114,
+      "loss": 43.4713,
+      "step": 183
+    },
+    {
+      "epoch": 0.018250347153342592,
+      "grad_norm": 1.732811450958252,
+      "learning_rate": 0.00019211073867244228,
+      "loss": 43.2893,
+      "step": 184
+    },
+    {
+      "epoch": 0.018349533822654234,
+      "grad_norm": 2.1155354976654053,
+      "learning_rate": 0.00019202102593682632,
+      "loss": 43.6014,
+      "step": 185
+    },
+    {
+      "epoch": 0.01844872049196588,
+      "grad_norm": 1.980027675628662,
+      "learning_rate": 0.00019193082718629677,
+      "loss": 43.2281,
+      "step": 186
+    },
+    {
+      "epoch": 0.018547907161277525,
+      "grad_norm": 1.9374451637268066,
+      "learning_rate": 0.0001918401428972441,
+      "loss": 43.6718,
+      "step": 187
+    },
+    {
+      "epoch": 0.01864709383058917,
+      "grad_norm": 2.091322898864746,
+      "learning_rate": 0.00019174897354862317,
+      "loss": 43.4641,
+      "step": 188
+    },
+    {
+      "epoch": 0.018746280499900812,
+      "grad_norm": 1.642065167427063,
+      "learning_rate": 0.00019165731962195065,
+      "loss": 43.4111,
+      "step": 189
+    },
+    {
+      "epoch": 0.018845467169212458,
+      "grad_norm": 1.905786156654358,
+      "learning_rate": 0.00019156518160130267,
+      "loss": 43.5117,
+      "step": 190
+    },
+    {
+      "epoch": 0.018944653838524103,
+      "grad_norm": 1.964237928390503,
+      "learning_rate": 0.00019147255997331198,
+      "loss": 43.5668,
+      "step": 191
+    },
+    {
+      "epoch": 0.01904384050783575,
+      "grad_norm": 1.7513357400894165,
+      "learning_rate": 0.00019137945522716568,
+      "loss": 43.5371,
+      "step": 192
+    },
+    {
+      "epoch": 0.01914302717714739,
+      "grad_norm": 1.5807865858078003,
+      "learning_rate": 0.00019128586785460236,
+      "loss": 43.4721,
+      "step": 193
+    },
+    {
+      "epoch": 0.019242213846459036,
+      "grad_norm": 1.9611173868179321,
+      "learning_rate": 0.00019119179834990973,
+      "loss": 43.5727,
+      "step": 194
+    },
+    {
+      "epoch": 0.01934140051577068,
+      "grad_norm": 1.5683705806732178,
+      "learning_rate": 0.0001910972472099219,
+      "loss": 43.5582,
+      "step": 195
+    },
+    {
+      "epoch": 0.019440587185082323,
+      "grad_norm": 1.8240303993225098,
+      "learning_rate": 0.00019100221493401672,
+      "loss": 43.5326,
+      "step": 196
+    },
+    {
+      "epoch": 0.01953977385439397,
+      "grad_norm": 1.6050409078598022,
+      "learning_rate": 0.00019090670202411315,
+      "loss": 43.5931,
+      "step": 197
+    },
+    {
+      "epoch": 0.019638960523705614,
+      "grad_norm": 1.769248127937317,
+      "learning_rate": 0.00019081070898466882,
+      "loss": 43.4457,
+      "step": 198
+    },
+    {
+      "epoch": 0.01973814719301726,
+      "grad_norm": 1.4461135864257812,
+      "learning_rate": 0.000190714236322677,
+      "loss": 43.5051,
+      "step": 199
+    },
+    {
+      "epoch": 0.0198373338623289,
+      "grad_norm": 1.6308741569519043,
+      "learning_rate": 0.00019061728454766423,
+      "loss": 43.3338,
+      "step": 200
+    },
+    {
+      "epoch": 0.019936520531640547,
+      "grad_norm": 1.7925598621368408,
+      "learning_rate": 0.0001905198541716875,
+      "loss": 43.4649,
+      "step": 201
+    },
+    {
+      "epoch": 0.020035707200952192,
+      "grad_norm": 1.7756211757659912,
+      "learning_rate": 0.00019042194570933156,
+      "loss": 43.5132,
+      "step": 202
+    },
+    {
+      "epoch": 0.020134893870263838,
+      "grad_norm": 1.5081807374954224,
+      "learning_rate": 0.00019032355967770617,
+      "loss": 43.4499,
+      "step": 203
+    },
+    {
+      "epoch": 0.02023408053957548,
+      "grad_norm": 1.9493539333343506,
+      "learning_rate": 0.00019022469659644344,
+      "loss": 43.4622,
+      "step": 204
+    },
+    {
+      "epoch": 0.020333267208887125,
+      "grad_norm": 1.603097915649414,
+      "learning_rate": 0.00019012535698769502,
+      "loss": 43.2635,
+      "step": 205
+    },
+    {
+      "epoch": 0.02043245387819877,
+      "grad_norm": 1.8952986001968384,
+      "learning_rate": 0.0001900255413761294,
+      "loss": 43.3947,
+      "step": 206
+    },
+    {
+      "epoch": 0.020531640547510416,
+      "grad_norm": 1.8197367191314697,
+      "learning_rate": 0.0001899252502889291,
+      "loss": 43.4419,
+      "step": 207
+    },
+    {
+      "epoch": 0.020630827216822058,
+      "grad_norm": 1.6402533054351807,
+      "learning_rate": 0.00018982448425578787,
+      "loss": 43.422,
+      "step": 208
+    },
+    {
+      "epoch": 0.020730013886133703,
+      "grad_norm": 1.760541558265686,
+      "learning_rate": 0.00018972324380890794,
+      "loss": 43.2571,
+      "step": 209
+    },
+    {
+      "epoch": 0.02082920055544535,
+      "grad_norm": 1.8905282020568848,
+      "learning_rate": 0.0001896215294829972,
+      "loss": 43.4353,
+      "step": 210
+    },
+    {
+      "epoch": 0.020928387224756994,
+      "grad_norm": 1.965397834777832,
+      "learning_rate": 0.0001895193418152663,
+      "loss": 43.3452,
+      "step": 211
+    },
+    {
+      "epoch": 0.021027573894068636,
+      "grad_norm": 1.6746777296066284,
+      "learning_rate": 0.00018941668134542602,
+      "loss": 43.4485,
+      "step": 212
+    },
+    {
+      "epoch": 0.02112676056338028,
+      "grad_norm": 1.8389016389846802,
+      "learning_rate": 0.00018931354861568406,
+      "loss": 43.3101,
+      "step": 213
+    },
+    {
+      "epoch": 0.021225947232691927,
+      "grad_norm": 1.922573208808899,
+      "learning_rate": 0.00018920994417074256,
+      "loss": 43.3094,
+      "step": 214
+    },
+    {
+      "epoch": 0.021325133902003572,
+      "grad_norm": 1.7113759517669678,
+      "learning_rate": 0.000189105868557795,
+      "loss": 43.437,
+      "step": 215
+    },
+    {
+      "epoch": 0.021424320571315214,
+      "grad_norm": 1.8494682312011719,
+      "learning_rate": 0.00018900132232652336,
+      "loss": 43.6071,
+      "step": 216
+    },
+    {
+      "epoch": 0.02152350724062686,
+      "grad_norm": 1.8459663391113281,
+      "learning_rate": 0.00018889630602909521,
+      "loss": 43.3704,
+      "step": 217
+    },
+    {
+      "epoch": 0.021622693909938505,
+      "grad_norm": 1.892861008644104,
+      "learning_rate": 0.00018879082022016085,
+      "loss": 43.4787,
+      "step": 218
+    },
+    {
+      "epoch": 0.02172188057925015,
+      "grad_norm": 1.9636040925979614,
+      "learning_rate": 0.00018868486545685028,
+      "loss": 43.2111,
+      "step": 219
+    },
+    {
+      "epoch": 0.021821067248561792,
+      "grad_norm": 2.4153854846954346,
+      "learning_rate": 0.00018857844229877032,
+      "loss": 43.5971,
+      "step": 220
+    },
+    {
+      "epoch": 0.021920253917873438,
+      "grad_norm": 1.7833011150360107,
+      "learning_rate": 0.00018847155130800167,
+      "loss": 43.5453,
+      "step": 221
+    },
+    {
+      "epoch": 0.022019440587185083,
+      "grad_norm": 2.046844720840454,
+      "learning_rate": 0.00018836419304909594,
+      "loss": 43.5533,
+      "step": 222
+    },
+    {
+      "epoch": 0.02211862725649673,
+      "grad_norm": 1.7641832828521729,
+      "learning_rate": 0.00018825636808907258,
+      "loss": 43.493,
+      "step": 223
+    },
+    {
+      "epoch": 0.02221781392580837,
+      "grad_norm": 1.7998050451278687,
+      "learning_rate": 0.00018814807699741603,
+      "loss": 43.4208,
+      "step": 224
+    },
+    {
+      "epoch": 0.022317000595120016,
+      "grad_norm": 1.6221156120300293,
+      "learning_rate": 0.00018803932034607256,
+      "loss": 43.6161,
+      "step": 225
+    },
+    {
+      "epoch": 0.02241618726443166,
+      "grad_norm": 1.863451600074768,
+      "learning_rate": 0.00018793009870944738,
+      "loss": 43.3304,
+      "step": 226
+    },
+    {
+      "epoch": 0.022515373933743307,
+      "grad_norm": 2.0428097248077393,
+      "learning_rate": 0.00018782041266440148,
+      "loss": 43.3551,
+      "step": 227
+    },
+    {
+      "epoch": 0.02261456060305495,
+      "grad_norm": 1.7376773357391357,
+      "learning_rate": 0.00018771026279024877,
+      "loss": 43.4487,
+      "step": 228
+    },
+    {
+      "epoch": 0.022713747272366594,
+      "grad_norm": 2.0446527004241943,
+      "learning_rate": 0.00018759964966875273,
+      "loss": 43.5901,
+      "step": 229
+    },
+    {
+      "epoch": 0.02281293394167824,
+      "grad_norm": 2.2305171489715576,
+      "learning_rate": 0.0001874885738841237,
+      "loss": 43.6051,
+      "step": 230
+    },
+    {
+      "epoch": 0.02291212061098988,
+      "grad_norm": 1.8739280700683594,
+      "learning_rate": 0.0001873770360230155,
+      "loss": 43.397,
+      "step": 231
+    },
+    {
+      "epoch": 0.023011307280301527,
+      "grad_norm": 1.6180644035339355,
+      "learning_rate": 0.00018726503667452243,
+      "loss": 43.5229,
+      "step": 232
+    },
+    {
+      "epoch": 0.023110493949613172,
+      "grad_norm": 2.046315908432007,
+      "learning_rate": 0.00018715257643017621,
+      "loss": 43.6235,
+      "step": 233
+    },
+    {
+      "epoch": 0.023209680618924818,
+      "grad_norm": 1.8750463724136353,
+      "learning_rate": 0.00018703965588394276,
+      "loss": 43.5124,
+      "step": 234
+    },
+    {
+      "epoch": 0.02330886728823646,
+      "grad_norm": 1.9105528593063354,
+      "learning_rate": 0.00018692627563221916,
+      "loss": 43.467,
+      "step": 235
+    },
+    {
+      "epoch": 0.023408053957548105,
+      "grad_norm": 1.710308313369751,
+      "learning_rate": 0.00018681243627383042,
+      "loss": 43.4374,
+      "step": 236
+    },
+    {
+      "epoch": 0.02350724062685975,
+      "grad_norm": 1.8155282735824585,
+      "learning_rate": 0.0001866981384100264,
+      "loss": 43.3374,
+      "step": 237
+    },
+    {
+      "epoch": 0.023606427296171396,
+      "grad_norm": 1.9001719951629639,
+      "learning_rate": 0.0001865833826444785,
+      "loss": 43.3604,
+      "step": 238
+    },
+    {
+      "epoch": 0.023705613965483038,
+      "grad_norm": 1.7378791570663452,
+      "learning_rate": 0.00018646816958327667,
+      "loss": 43.3533,
+      "step": 239
+    },
+    {
+      "epoch": 0.023804800634794683,
+      "grad_norm": 1.938234806060791,
+      "learning_rate": 0.00018635249983492603,
+      "loss": 43.3329,
+      "step": 240
+    },
+    {
+      "epoch": 0.02390398730410633,
+      "grad_norm": 1.8653101921081543,
+      "learning_rate": 0.00018623637401034366,
+      "loss": 43.4116,
+      "step": 241
+    },
+    {
+      "epoch": 0.024003173973417974,
+      "grad_norm": 2.307813882827759,
+      "learning_rate": 0.00018611979272285556,
+      "loss": 43.2887,
+      "step": 242
+    },
+    {
+      "epoch": 0.024102360642729616,
+      "grad_norm": 1.9324138164520264,
+      "learning_rate": 0.00018600275658819324,
+      "loss": 43.3422,
+      "step": 243
+    },
+    {
+      "epoch": 0.02420154731204126,
+      "grad_norm": 1.9545015096664429,
+      "learning_rate": 0.00018588526622449048,
+      "loss": 43.2783,
+      "step": 244
+    },
+    {
+      "epoch": 0.024300733981352907,
+      "grad_norm": 2.312723159790039,
+      "learning_rate": 0.00018576732225228013,
+      "loss": 43.4625,
+      "step": 245
+    },
+    {
+      "epoch": 0.024399920650664552,
+      "grad_norm": 1.7571669816970825,
+      "learning_rate": 0.00018564892529449078,
+      "loss": 43.2928,
+      "step": 246
+    },
+    {
+      "epoch": 0.024499107319976194,
+      "grad_norm": 1.896710753440857,
+      "learning_rate": 0.00018553007597644353,
+      "loss": 43.2533,
+      "step": 247
+    },
+    {
+      "epoch": 0.02459829398928784,
+      "grad_norm": 1.737964153289795,
+      "learning_rate": 0.00018541077492584864,
+      "loss": 43.638,
+      "step": 248
+    },
+    {
+      "epoch": 0.024697480658599485,
+      "grad_norm": 1.7832790613174438,
+      "learning_rate": 0.0001852910227728022,
+      "loss": 43.5125,
+      "step": 249
+    },
+    {
+      "epoch": 0.02479666732791113,
+      "grad_norm": 1.793320655822754,
+      "learning_rate": 0.00018517082014978282,
+      "loss": 43.5349,
+      "step": 250
+    },
+    {
+      "epoch": 0.024895853997222772,
+      "grad_norm": 1.9972788095474243,
+      "learning_rate": 0.00018505016769164833,
+      "loss": 43.3333,
+      "step": 251
+    },
+    {
+      "epoch": 0.024995040666534418,
+      "grad_norm": 1.9738582372665405,
+      "learning_rate": 0.00018492906603563238,
+      "loss": 43.4897,
+      "step": 252
+    },
+    {
+      "epoch": 0.025094227335846063,
+      "grad_norm": 1.8204716444015503,
+      "learning_rate": 0.0001848075158213411,
+      "loss": 43.285,
+      "step": 253
+    },
+    {
+      "epoch": 0.02519341400515771,
+      "grad_norm": 1.6776862144470215,
+      "learning_rate": 0.00018468551769074964,
+      "loss": 43.523,
+      "step": 254
+    },
+    {
+      "epoch": 0.02529260067446935,
+      "grad_norm": 1.6923521757125854,
+      "learning_rate": 0.00018456307228819897,
+      "loss": 43.5013,
+      "step": 255
+    },
+    {
+      "epoch": 0.025391787343780996,
+      "grad_norm": 2.2982516288757324,
+      "learning_rate": 0.00018444018026039224,
+      "loss": 43.2198,
+      "step": 256
+    },
+    {
+      "epoch": 0.02549097401309264,
+      "grad_norm": 1.6277918815612793,
+      "learning_rate": 0.00018431684225639155,
+      "loss": 43.6655,
+      "step": 257
+    },
+    {
+      "epoch": 0.025590160682404287,
+      "grad_norm": 1.9118983745574951,
+      "learning_rate": 0.0001841930589276144,
+      "loss": 43.4623,
+      "step": 258
+    },
+    {
+      "epoch": 0.02568934735171593,
+      "grad_norm": 1.7027877569198608,
+      "learning_rate": 0.0001840688309278304,
+      "loss": 43.3977,
+      "step": 259
+    },
+    {
+      "epoch": 0.025788534021027574,
+      "grad_norm": 2.1675608158111572,
+      "learning_rate": 0.00018394415891315758,
+      "loss": 43.1506,
+      "step": 260
+    },
+    {
+      "epoch": 0.02588772069033922,
+      "grad_norm": 1.6530122756958008,
+      "learning_rate": 0.00018381904354205914,
+      "loss": 43.545,
+      "step": 261
+    },
+    {
+      "epoch": 0.02598690735965086,
+      "grad_norm": 2.0166702270507812,
+      "learning_rate": 0.0001836934854753399,
+      "loss": 43.4604,
+      "step": 262
+    },
+    {
+      "epoch": 0.026086094028962507,
+      "grad_norm": 1.9425437450408936,
+      "learning_rate": 0.00018356748537614283,
+      "loss": 43.2025,
+      "step": 263
+    },
+    {
+      "epoch": 0.026185280698274152,
+      "grad_norm": 1.4773979187011719,
+      "learning_rate": 0.00018344104390994543,
+      "loss": 43.4627,
+      "step": 264
+    },
+    {
+      "epoch": 0.026284467367585797,
+      "grad_norm": 1.9007498025894165,
+      "learning_rate": 0.00018331416174455635,
+      "loss": 43.3737,
+      "step": 265
+    },
+    {
+      "epoch": 0.02638365403689744,
+      "grad_norm": 2.1626980304718018,
+      "learning_rate": 0.00018318683955011192,
+      "loss": 43.5208,
+      "step": 266
+    },
+    {
+      "epoch": 0.026482840706209085,
+      "grad_norm": 2.390268564224243,
+      "learning_rate": 0.00018305907799907233,
+      "loss": 43.3615,
+      "step": 267
+    },
+    {
+      "epoch": 0.02658202737552073,
+      "grad_norm": 1.826054573059082,
+      "learning_rate": 0.00018293087776621836,
+      "loss": 43.3623,
+      "step": 268
+    },
+    {
+      "epoch": 0.026681214044832376,
+      "grad_norm": 1.793283224105835,
+      "learning_rate": 0.00018280223952864781,
+      "loss": 43.2613,
+      "step": 269
+    },
+    {
+      "epoch": 0.026780400714144018,
+      "grad_norm": 1.8169645071029663,
+      "learning_rate": 0.00018267316396577166,
+      "loss": 43.3729,
+      "step": 270
+    },
+    {
+      "epoch": 0.026879587383455663,
+      "grad_norm": 1.6684504747390747,
+      "learning_rate": 0.0001825436517593107,
+      "loss": 43.4644,
+      "step": 271
+    },
+    {
+      "epoch": 0.02697877405276731,
+      "grad_norm": 1.916520357131958,
+      "learning_rate": 0.00018241370359329192,
+      "loss": 43.6573,
+      "step": 272
+    },
+    {
+      "epoch": 0.027077960722078954,
+      "grad_norm": 1.9397066831588745,
+      "learning_rate": 0.0001822833201540449,
+      "loss": 43.4218,
+      "step": 273
+    },
+    {
+      "epoch": 0.027177147391390596,
+      "grad_norm": 1.697509765625,
+      "learning_rate": 0.000182152502130198,
+      "loss": 43.4535,
+      "step": 274
+    },
+    {
+      "epoch": 0.02727633406070224,
+      "grad_norm": 1.9047552347183228,
+      "learning_rate": 0.000182021250212675,
+      "loss": 43.4582,
+      "step": 275
+    },
+    {
+      "epoch": 0.027375520730013887,
+      "grad_norm": 2.150933265686035,
+      "learning_rate": 0.00018188956509469125,
+      "loss": 43.3068,
+      "step": 276
+    },
+    {
+      "epoch": 0.027474707399325532,
+      "grad_norm": 2.0963590145111084,
+      "learning_rate": 0.00018175744747175008,
+      "loss": 43.3796,
+      "step": 277
+    },
+    {
+      "epoch": 0.027573894068637174,
+      "grad_norm": 2.0096163749694824,
+      "learning_rate": 0.0001816248980416392,
+      "loss": 43.4927,
+      "step": 278
+    },
+    {
+      "epoch": 0.02767308073794882,
+      "grad_norm": 2.438795566558838,
+      "learning_rate": 0.0001814919175044268,
+      "loss": 43.2995,
+      "step": 279
+    },
+    {
+      "epoch": 0.027772267407260465,
+      "grad_norm": 1.8637646436691284,
+      "learning_rate": 0.00018135850656245808,
+      "loss": 43.4106,
+      "step": 280
+    },
+    {
+      "epoch": 0.02787145407657211,
+      "grad_norm": 1.6904587745666504,
+      "learning_rate": 0.00018122466592035148,
+      "loss": 43.3585,
+      "step": 281
+    },
+    {
+      "epoch": 0.027970640745883752,
+      "grad_norm": 1.963031530380249,
+      "learning_rate": 0.00018109039628499483,
+      "loss": 43.326,
+      "step": 282
+    },
+    {
+      "epoch": 0.028069827415195397,
+      "grad_norm": 1.8556482791900635,
+      "learning_rate": 0.00018095569836554178,
+      "loss": 43.4717,
+      "step": 283
+    },
+    {
+      "epoch": 0.028169014084507043,
+      "grad_norm": 2.059662103652954,
+      "learning_rate": 0.000180820572873408,
+      "loss": 43.3201,
+      "step": 284
+    },
+    {
+      "epoch": 0.02826820075381869,
+      "grad_norm": 1.9710227251052856,
+      "learning_rate": 0.00018068502052226733,
+      "loss": 43.548,
+      "step": 285
+    },
+    {
+      "epoch": 0.02836738742313033,
+      "grad_norm": 1.7940763235092163,
+      "learning_rate": 0.0001805490420280482,
+      "loss": 43.4886,
+      "step": 286
+    },
+    {
+      "epoch": 0.028466574092441976,
+      "grad_norm": 1.9778364896774292,
+      "learning_rate": 0.00018041263810892973,
+      "loss": 43.5431,
+      "step": 287
+    },
+    {
+      "epoch": 0.02856576076175362,
+      "grad_norm": 2.030728340148926,
+      "learning_rate": 0.0001802758094853378,
+      "loss": 43.2431,
+      "step": 288
+    },
+    {
+      "epoch": 0.028664947431065266,
+      "grad_norm": 2.230532169342041,
+      "learning_rate": 0.00018013855687994164,
+      "loss": 43.2208,
+      "step": 289
+    },
+    {
+      "epoch": 0.02876413410037691,
+      "grad_norm": 1.9482969045639038,
+      "learning_rate": 0.00018000088101764953,
+      "loss": 43.5798,
+      "step": 290
+    },
+    {
+      "epoch": 0.028863320769688554,
+      "grad_norm": 1.7520207166671753,
+      "learning_rate": 0.00017986278262560536,
+      "loss": 43.5962,
+      "step": 291
+    },
+    {
+      "epoch": 0.0289625074390002,
+      "grad_norm": 1.9360411167144775,
+      "learning_rate": 0.00017972426243318456,
+      "loss": 43.5776,
+      "step": 292
+    },
+    {
+      "epoch": 0.02906169410831184,
+      "grad_norm": 1.9918673038482666,
+      "learning_rate": 0.0001795853211719904,
+      "loss": 43.3685,
+      "step": 293
+    },
+    {
+      "epoch": 0.029160880777623487,
+      "grad_norm": 1.8499594926834106,
+      "learning_rate": 0.00017944595957584999,
+      "loss": 43.2545,
+      "step": 294
+    },
+    {
+      "epoch": 0.029260067446935132,
+      "grad_norm": 1.7617238759994507,
+      "learning_rate": 0.00017930617838081046,
+      "loss": 43.2486,
+      "step": 295
+    },
+    {
+      "epoch": 0.029359254116246777,
+      "grad_norm": 1.8923050165176392,
+      "learning_rate": 0.0001791659783251351,
+      "loss": 43.1631,
+      "step": 296
+    },
+    {
+      "epoch": 0.02945844078555842,
+      "grad_norm": 2.038088798522949,
+      "learning_rate": 0.00017902536014929946,
+      "loss": 43.249,
+      "step": 297
+    },
+    {
+      "epoch": 0.029557627454870065,
+      "grad_norm": 1.8649237155914307,
+      "learning_rate": 0.00017888432459598744,
+      "loss": 43.5624,
+      "step": 298
+    },
+    {
+      "epoch": 0.02965681412418171,
+      "grad_norm": 2.608098268508911,
+      "learning_rate": 0.0001787428724100872,
+      "loss": 43.3078,
+      "step": 299
+    },
+    {
+      "epoch": 0.029756000793493356,
+      "grad_norm": 2.03300404548645,
+      "learning_rate": 0.00017860100433868755,
+      "loss": 43.3856,
+      "step": 300
+    },
+    {
+      "epoch": 0.029855187462804997,
+      "grad_norm": 1.9915575981140137,
+      "learning_rate": 0.0001784587211310737,
+      "loss": 43.5435,
+      "step": 301
+    },
+    {
+      "epoch": 0.029954374132116643,
+      "grad_norm": 1.8422963619232178,
+      "learning_rate": 0.00017831602353872357,
+      "loss": 43.4246,
+      "step": 302
+    },
+    {
+      "epoch": 0.03005356080142829,
+      "grad_norm": 2.0159566402435303,
+      "learning_rate": 0.00017817291231530348,
+      "loss": 43.4693,
+      "step": 303
+    },
+    {
+      "epoch": 0.030152747470739934,
+      "grad_norm": 2.0639212131500244,
+      "learning_rate": 0.00017802938821666458,
+      "loss": 43.3722,
+      "step": 304
+    },
+    {
+      "epoch": 0.030251934140051576,
+      "grad_norm": 1.985589623451233,
+      "learning_rate": 0.00017788545200083847,
+      "loss": 43.1785,
+      "step": 305
+    },
+    {
+      "epoch": 0.03035112080936322,
+      "grad_norm": 1.6914057731628418,
+      "learning_rate": 0.00017774110442803347,
+      "loss": 43.4961,
+      "step": 306
+    },
+    {
+      "epoch": 0.030450307478674866,
+      "grad_norm": 2.404210329055786,
+      "learning_rate": 0.0001775963462606305,
+      "loss": 43.4228,
+      "step": 307
+    },
+    {
+      "epoch": 0.030549494147986512,
+      "grad_norm": 2.215900182723999,
+      "learning_rate": 0.000177451178263179,
+      "loss": 43.4482,
+      "step": 308
+    },
+    {
+      "epoch": 0.030648680817298154,
+      "grad_norm": 1.7335083484649658,
+      "learning_rate": 0.00017730560120239307,
+      "loss": 43.5255,
+      "step": 309
+    },
+    {
+      "epoch": 0.0307478674866098,
+      "grad_norm": 2.051217794418335,
+      "learning_rate": 0.0001771596158471472,
+      "loss": 43.3276,
+      "step": 310
+    },
+    {
+      "epoch": 0.030847054155921445,
+      "grad_norm": 1.7068490982055664,
+      "learning_rate": 0.00017701322296847236,
+      "loss": 43.2431,
+      "step": 311
+    },
+    {
+      "epoch": 0.03094624082523309,
+      "grad_norm": 1.6466596126556396,
+      "learning_rate": 0.00017686642333955183,
+      "loss": 43.3234,
+      "step": 312
+    },
+    {
+      "epoch": 0.031045427494544732,
+      "grad_norm": 1.9981932640075684,
+      "learning_rate": 0.00017671921773571727,
+      "loss": 43.3187,
+      "step": 313
+    },
+    {
+      "epoch": 0.031144614163856377,
+      "grad_norm": 1.7652865648269653,
+      "learning_rate": 0.0001765716069344444,
+      "loss": 43.1768,
+      "step": 314
+    },
+    {
+      "epoch": 0.031243800833168023,
+      "grad_norm": 1.8989441394805908,
+      "learning_rate": 0.0001764235917153491,
+      "loss": 43.2334,
+      "step": 315
+    },
+    {
+      "epoch": 0.031342987502479665,
+      "grad_norm": 2.141063928604126,
+      "learning_rate": 0.00017627517286018321,
+      "loss": 43.288,
+      "step": 316
+    },
+    {
+      "epoch": 0.031442174171791314,
+      "grad_norm": 2.336796522140503,
+      "learning_rate": 0.0001761263511528303,
+      "loss": 43.5281,
+      "step": 317
+    },
+    {
+      "epoch": 0.031541360841102956,
+      "grad_norm": 2.18377685546875,
+      "learning_rate": 0.00017597712737930178,
+      "loss": 43.5268,
+      "step": 318
+    },
+    {
+      "epoch": 0.0316405475104146,
+      "grad_norm": 2.292229652404785,
+      "learning_rate": 0.00017582750232773248,
+      "loss": 43.4562,
+      "step": 319
+    },
+    {
+      "epoch": 0.031739734179726246,
+      "grad_norm": 1.9269510507583618,
+      "learning_rate": 0.0001756774767883767,
+      "loss": 43.35,
+      "step": 320
+    },
+    {
+      "epoch": 0.03183892084903789,
+      "grad_norm": 1.8633321523666382,
+      "learning_rate": 0.00017552705155360378,
+      "loss": 43.3391,
+      "step": 321
+    },
+    {
+      "epoch": 0.03193810751834954,
+      "grad_norm": 2.1785457134246826,
+      "learning_rate": 0.00017537622741789429,
+      "loss": 43.6481,
+      "step": 322
+    },
+    {
+      "epoch": 0.03203729418766118,
+      "grad_norm": 1.988545298576355,
+      "learning_rate": 0.00017522500517783547,
+      "loss": 43.1767,
+      "step": 323
+    },
+    {
+      "epoch": 0.03213648085697282,
+      "grad_norm": 1.80686354637146,
+      "learning_rate": 0.0001750733856321172,
+      "loss": 43.2304,
+      "step": 324
+    },
+    {
+      "epoch": 0.03223566752628447,
+      "grad_norm": 1.8045567274093628,
+      "learning_rate": 0.00017492136958152784,
+      "loss": 43.4061,
+      "step": 325
+    },
+    {
+      "epoch": 0.03233485419559611,
+      "grad_norm": 1.9179420471191406,
+      "learning_rate": 0.00017476895782894979,
+      "loss": 43.2181,
+      "step": 326
+    },
+    {
+      "epoch": 0.032434040864907754,
+      "grad_norm": 1.8775529861450195,
+      "learning_rate": 0.00017461615117935546,
+      "loss": 43.5342,
+      "step": 327
+    },
+    {
+      "epoch": 0.0325332275342194,
+      "grad_norm": 1.6033300161361694,
+      "learning_rate": 0.0001744629504398029,
+      "loss": 43.4323,
+      "step": 328
+    },
+    {
+      "epoch": 0.032632414203531045,
+      "grad_norm": 1.9226906299591064,
+      "learning_rate": 0.00017430935641943163,
+      "loss": 43.5733,
+      "step": 329
+    },
+    {
+      "epoch": 0.03273160087284269,
+      "grad_norm": 2.0332751274108887,
+      "learning_rate": 0.00017415536992945816,
+      "loss": 43.1741,
+      "step": 330
+    },
+    {
+      "epoch": 0.032830787542154335,
+      "grad_norm": 2.020566701889038,
+      "learning_rate": 0.00017400099178317203,
+      "loss": 43.4961,
+      "step": 331
+    },
+    {
+      "epoch": 0.03292997421146598,
+      "grad_norm": 2.0364789962768555,
+      "learning_rate": 0.00017384622279593122,
+      "loss": 43.143,
+      "step": 332
+    },
+    {
+      "epoch": 0.033029160880777626,
+      "grad_norm": 1.9677212238311768,
+      "learning_rate": 0.000173691063785158,
+      "loss": 43.5331,
+      "step": 333
+    },
+    {
+      "epoch": 0.03312834755008927,
+      "grad_norm": 1.6932072639465332,
+      "learning_rate": 0.0001735355155703346,
+      "loss": 43.3948,
+      "step": 334
+    },
+    {
+      "epoch": 0.03322753421940091,
+      "grad_norm": 2.192953109741211,
+      "learning_rate": 0.00017337957897299884,
+      "loss": 43.5575,
+      "step": 335
+    },
+    {
+      "epoch": 0.03332672088871256,
+      "grad_norm": 1.7866209745407104,
+      "learning_rate": 0.00017322325481673975,
+      "loss": 43.2401,
+      "step": 336
+    },
+    {
+      "epoch": 0.0334259075580242,
+      "grad_norm": 1.863152265548706,
+      "learning_rate": 0.00017306654392719337,
+      "loss": 43.4706,
+      "step": 337
+    },
+    {
+      "epoch": 0.03352509422733584,
+      "grad_norm": 1.6541751623153687,
+      "learning_rate": 0.00017290944713203822,
+      "loss": 43.2841,
+      "step": 338
+    },
+    {
+      "epoch": 0.03362428089664749,
+      "grad_norm": 2.455803394317627,
+      "learning_rate": 0.00017275196526099108,
+      "loss": 43.0331,
+      "step": 339
+    },
+    {
+      "epoch": 0.033723467565959134,
+      "grad_norm": 1.9039475917816162,
+      "learning_rate": 0.00017259409914580242,
+      "loss": 43.3408,
+      "step": 340
+    },
+    {
+      "epoch": 0.03382265423527078,
+      "grad_norm": 1.642409324645996,
+      "learning_rate": 0.00017243584962025224,
+      "loss": 43.1996,
+      "step": 341
+    },
+    {
+      "epoch": 0.033921840904582425,
+      "grad_norm": 1.591845154762268,
+      "learning_rate": 0.00017227721752014548,
+      "loss": 43.2826,
+      "step": 342
+    },
+    {
+      "epoch": 0.034021027573894067,
+      "grad_norm": 1.970920443534851,
+      "learning_rate": 0.0001721182036833077,
+      "loss": 43.5663,
+      "step": 343
+    },
+    {
+      "epoch": 0.034120214243205715,
+      "grad_norm": 2.2533814907073975,
+      "learning_rate": 0.00017195880894958063,
+      "loss": 42.8656,
+      "step": 344
+    },
+    {
+      "epoch": 0.03421940091251736,
+      "grad_norm": 1.8976967334747314,
+      "learning_rate": 0.00017179903416081763,
+      "loss": 43.4036,
+      "step": 345
+    },
+    {
+      "epoch": 0.03421940091251736,
+      "eval_loss": 10.84935188293457,
+      "eval_runtime": 11.4279,
+      "eval_samples_per_second": 371.46,
+      "eval_steps_per_second": 185.774,
+      "step": 345
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 1377,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 345,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3361213513728.0,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f15b7133227076cd1867948b59f82ed8ae1cf3bea5fc6355dcfecaff9e397731
+size 6776