Training in progress, step 100, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-0.5B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-0.5B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "v_proj",
+    "q_proj",
+    "down_proj",
+    "k_proj",
+    "gate_proj",
+    "o_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:91a71c9a8f14fa185f30e113fe7dd2b410471324601afda0c6d8d27d0ba40e0b
+size 35237104

last-checkpoint/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95bc372b0ddbb5e90316cb0cfd8adba993a88e37ab67b0c2e059510d1105650c
+size 18810036

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1fa37a7cf7f7e0a1b6bcff48efc4571c205b16dd75c990c58de904af2ad7e32e
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:945b7d05a395e582b997db2d4fcf1ed9355970dd1a4de12540b001e50ac6ebcd
+size 1064

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,20 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bcfe42da0a4497e8b2b172c1f9f4ec423a46dc12907f4349c55025f670422ba9
+size 11418266

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,758 @@
+{
+  "best_metric": 3.3947603702545166,
+  "best_model_checkpoint": "miner_id_24/checkpoint-100",
+  "epoch": 0.06486135884546781,
+  "eval_steps": 100,
+  "global_step": 100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0006486135884546781,
+      "grad_norm": 3.839925765991211,
+      "learning_rate": 2e-05,
+      "loss": 5.3671,
+      "step": 1
+    },
+    {
+      "epoch": 0.0006486135884546781,
+      "eval_loss": 5.341527462005615,
+      "eval_runtime": 35.046,
+      "eval_samples_per_second": 58.666,
+      "eval_steps_per_second": 14.666,
+      "step": 1
+    },
+    {
+      "epoch": 0.0012972271769093563,
+      "grad_norm": 3.8855299949645996,
+      "learning_rate": 4e-05,
+      "loss": 5.2735,
+      "step": 2
+    },
+    {
+      "epoch": 0.0019458407653640344,
+      "grad_norm": 3.7578446865081787,
+      "learning_rate": 6e-05,
+      "loss": 4.8599,
+      "step": 3
+    },
+    {
+      "epoch": 0.0025944543538187126,
+      "grad_norm": 2.840263843536377,
+      "learning_rate": 8e-05,
+      "loss": 4.3011,
+      "step": 4
+    },
+    {
+      "epoch": 0.0032430679422733905,
+      "grad_norm": 3.2963178157806396,
+      "learning_rate": 0.0001,
+      "loss": 4.7049,
+      "step": 5
+    },
+    {
+      "epoch": 0.003891681530728069,
+      "grad_norm": 2.3765435218811035,
+      "learning_rate": 0.00012,
+      "loss": 4.4563,
+      "step": 6
+    },
+    {
+      "epoch": 0.004540295119182747,
+      "grad_norm": 3.072453022003174,
+      "learning_rate": 0.00014,
+      "loss": 4.4638,
+      "step": 7
+    },
+    {
+      "epoch": 0.005188908707637425,
+      "grad_norm": 3.7620046138763428,
+      "learning_rate": 0.00016,
+      "loss": 4.8635,
+      "step": 8
+    },
+    {
+      "epoch": 0.0058375222960921035,
+      "grad_norm": 2.274141311645508,
+      "learning_rate": 0.00018,
+      "loss": 4.0879,
+      "step": 9
+    },
+    {
+      "epoch": 0.006486135884546781,
+      "grad_norm": 2.1749279499053955,
+      "learning_rate": 0.0002,
+      "loss": 4.0116,
+      "step": 10
+    },
+    {
+      "epoch": 0.007134749473001459,
+      "grad_norm": 2.7090888023376465,
+      "learning_rate": 0.0001999999477769797,
+      "loss": 4.1352,
+      "step": 11
+    },
+    {
+      "epoch": 0.007783363061456138,
+      "grad_norm": 2.8192594051361084,
+      "learning_rate": 0.00019999979110797331,
+      "loss": 3.7506,
+      "step": 12
+    },
+    {
+      "epoch": 0.008431976649910815,
+      "grad_norm": 2.1979215145111084,
+      "learning_rate": 0.00019999952999314444,
+      "loss": 4.0342,
+      "step": 13
+    },
+    {
+      "epoch": 0.009080590238365494,
+      "grad_norm": 1.9726805686950684,
+      "learning_rate": 0.0001999991644327659,
+      "loss": 3.6903,
+      "step": 14
+    },
+    {
+      "epoch": 0.009729203826820172,
+      "grad_norm": 2.4161455631256104,
+      "learning_rate": 0.0001999986944272194,
+      "loss": 3.2464,
+      "step": 15
+    },
+    {
+      "epoch": 0.01037781741527485,
+      "grad_norm": 1.8676812648773193,
+      "learning_rate": 0.00019999811997699593,
+      "loss": 3.7741,
+      "step": 16
+    },
+    {
+      "epoch": 0.011026431003729529,
+      "grad_norm": 1.7903848886489868,
+      "learning_rate": 0.00019999744108269542,
+      "loss": 3.7455,
+      "step": 17
+    },
+    {
+      "epoch": 0.011675044592184207,
+      "grad_norm": 2.043623924255371,
+      "learning_rate": 0.00019999665774502696,
+      "loss": 3.8827,
+      "step": 18
+    },
+    {
+      "epoch": 0.012323658180638884,
+      "grad_norm": 1.8445640802383423,
+      "learning_rate": 0.00019999576996480872,
+      "loss": 3.9371,
+      "step": 19
+    },
+    {
+      "epoch": 0.012972271769093562,
+      "grad_norm": 1.5927119255065918,
+      "learning_rate": 0.000199994777742968,
+      "loss": 3.5422,
+      "step": 20
+    },
+    {
+      "epoch": 0.01362088535754824,
+      "grad_norm": 1.3268831968307495,
+      "learning_rate": 0.0001999936810805411,
+      "loss": 3.286,
+      "step": 21
+    },
+    {
+      "epoch": 0.014269498946002919,
+      "grad_norm": 1.617830514907837,
+      "learning_rate": 0.0001999924799786734,
+      "loss": 3.3049,
+      "step": 22
+    },
+    {
+      "epoch": 0.014918112534457597,
+      "grad_norm": 1.592024803161621,
+      "learning_rate": 0.00019999117443861942,
+      "loss": 3.3464,
+      "step": 23
+    },
+    {
+      "epoch": 0.015566726122912275,
+      "grad_norm": 1.5516711473464966,
+      "learning_rate": 0.00019998976446174277,
+      "loss": 3.399,
+      "step": 24
+    },
+    {
+      "epoch": 0.016215339711366954,
+      "grad_norm": 1.7870945930480957,
+      "learning_rate": 0.00019998825004951612,
+      "loss": 3.5775,
+      "step": 25
+    },
+    {
+      "epoch": 0.01686395329982163,
+      "grad_norm": 1.6130069494247437,
+      "learning_rate": 0.00019998663120352118,
+      "loss": 3.6604,
+      "step": 26
+    },
+    {
+      "epoch": 0.01751256688827631,
+      "grad_norm": 1.4511877298355103,
+      "learning_rate": 0.00019998490792544883,
+      "loss": 3.243,
+      "step": 27
+    },
+    {
+      "epoch": 0.018161180476730987,
+      "grad_norm": 1.6252033710479736,
+      "learning_rate": 0.0001999830802170989,
+      "loss": 3.6021,
+      "step": 28
+    },
+    {
+      "epoch": 0.018809794065185667,
+      "grad_norm": 1.7277567386627197,
+      "learning_rate": 0.00019998114808038043,
+      "loss": 3.5944,
+      "step": 29
+    },
+    {
+      "epoch": 0.019458407653640344,
+      "grad_norm": 1.6731386184692383,
+      "learning_rate": 0.00019997911151731134,
+      "loss": 3.3969,
+      "step": 30
+    },
+    {
+      "epoch": 0.02010702124209502,
+      "grad_norm": 1.6243277788162231,
+      "learning_rate": 0.00019997697053001886,
+      "loss": 3.035,
+      "step": 31
+    },
+    {
+      "epoch": 0.0207556348305497,
+      "grad_norm": 1.773876667022705,
+      "learning_rate": 0.00019997472512073912,
+      "loss": 3.6559,
+      "step": 32
+    },
+    {
+      "epoch": 0.021404248419004377,
+      "grad_norm": 1.47771155834198,
+      "learning_rate": 0.00019997237529181737,
+      "loss": 3.103,
+      "step": 33
+    },
+    {
+      "epoch": 0.022052862007459057,
+      "grad_norm": 1.3901262283325195,
+      "learning_rate": 0.0001999699210457079,
+      "loss": 2.8133,
+      "step": 34
+    },
+    {
+      "epoch": 0.022701475595913734,
+      "grad_norm": 1.6069719791412354,
+      "learning_rate": 0.00019996736238497406,
+      "loss": 3.3517,
+      "step": 35
+    },
+    {
+      "epoch": 0.023350089184368414,
+      "grad_norm": 1.5524790287017822,
+      "learning_rate": 0.0001999646993122883,
+      "loss": 3.3211,
+      "step": 36
+    },
+    {
+      "epoch": 0.02399870277282309,
+      "grad_norm": 1.6193372011184692,
+      "learning_rate": 0.0001999619318304321,
+      "loss": 3.5321,
+      "step": 37
+    },
+    {
+      "epoch": 0.024647316361277767,
+      "grad_norm": 1.6295925378799438,
+      "learning_rate": 0.00019995905994229593,
+      "loss": 3.4257,
+      "step": 38
+    },
+    {
+      "epoch": 0.025295929949732447,
+      "grad_norm": 1.9165164232254028,
+      "learning_rate": 0.00019995608365087946,
+      "loss": 3.9521,
+      "step": 39
+    },
+    {
+      "epoch": 0.025944543538187124,
+      "grad_norm": 1.5593171119689941,
+      "learning_rate": 0.0001999530029592912,
+      "loss": 3.4656,
+      "step": 40
+    },
+    {
+      "epoch": 0.026593157126641804,
+      "grad_norm": 1.691826581954956,
+      "learning_rate": 0.0001999498178707489,
+      "loss": 3.2846,
+      "step": 41
+    },
+    {
+      "epoch": 0.02724177071509648,
+      "grad_norm": 1.709882140159607,
+      "learning_rate": 0.00019994652838857917,
+      "loss": 3.4973,
+      "step": 42
+    },
+    {
+      "epoch": 0.02789038430355116,
+      "grad_norm": 1.5011334419250488,
+      "learning_rate": 0.00019994313451621783,
+      "loss": 3.0221,
+      "step": 43
+    },
+    {
+      "epoch": 0.028538997892005837,
+      "grad_norm": 1.3794457912445068,
+      "learning_rate": 0.0001999396362572096,
+      "loss": 3.5346,
+      "step": 44
+    },
+    {
+      "epoch": 0.029187611480460517,
+      "grad_norm": 1.6306489706039429,
+      "learning_rate": 0.00019993603361520828,
+      "loss": 3.4094,
+      "step": 45
+    },
+    {
+      "epoch": 0.029836225068915194,
+      "grad_norm": 1.2462743520736694,
+      "learning_rate": 0.00019993232659397666,
+      "loss": 3.1227,
+      "step": 46
+    },
+    {
+      "epoch": 0.03048483865736987,
+      "grad_norm": 1.4848414659500122,
+      "learning_rate": 0.00019992851519738664,
+      "loss": 3.2734,
+      "step": 47
+    },
+    {
+      "epoch": 0.03113345224582455,
+      "grad_norm": 1.7278729677200317,
+      "learning_rate": 0.00019992459942941906,
+      "loss": 3.3184,
+      "step": 48
+    },
+    {
+      "epoch": 0.03178206583427923,
+      "grad_norm": 1.4307690858840942,
+      "learning_rate": 0.00019992057929416371,
+      "loss": 3.5227,
+      "step": 49
+    },
+    {
+      "epoch": 0.03243067942273391,
+      "grad_norm": 1.530920147895813,
+      "learning_rate": 0.00019991645479581956,
+      "loss": 3.1131,
+      "step": 50
+    },
+    {
+      "epoch": 0.03307929301118859,
+      "grad_norm": 1.5259414911270142,
+      "learning_rate": 0.00019991222593869444,
+      "loss": 3.7062,
+      "step": 51
+    },
+    {
+      "epoch": 0.03372790659964326,
+      "grad_norm": 1.562098503112793,
+      "learning_rate": 0.0001999078927272052,
+      "loss": 3.3501,
+      "step": 52
+    },
+    {
+      "epoch": 0.03437652018809794,
+      "grad_norm": 1.539969801902771,
+      "learning_rate": 0.00019990345516587775,
+      "loss": 3.7154,
+      "step": 53
+    },
+    {
+      "epoch": 0.03502513377655262,
+      "grad_norm": 1.8281248807907104,
+      "learning_rate": 0.00019989891325934692,
+      "loss": 3.5978,
+      "step": 54
+    },
+    {
+      "epoch": 0.035673747365007294,
+      "grad_norm": 1.5411529541015625,
+      "learning_rate": 0.00019989426701235653,
+      "loss": 3.3103,
+      "step": 55
+    },
+    {
+      "epoch": 0.036322360953461974,
+      "grad_norm": 1.3391532897949219,
+      "learning_rate": 0.00019988951642975947,
+      "loss": 2.9469,
+      "step": 56
+    },
+    {
+      "epoch": 0.036970974541916654,
+      "grad_norm": 1.5546962022781372,
+      "learning_rate": 0.00019988466151651748,
+      "loss": 3.5001,
+      "step": 57
+    },
+    {
+      "epoch": 0.037619588130371334,
+      "grad_norm": 1.673667550086975,
+      "learning_rate": 0.00019987970227770135,
+      "loss": 3.5981,
+      "step": 58
+    },
+    {
+      "epoch": 0.03826820171882601,
+      "grad_norm": 1.5357543230056763,
+      "learning_rate": 0.00019987463871849078,
+      "loss": 3.2209,
+      "step": 59
+    },
+    {
+      "epoch": 0.03891681530728069,
+      "grad_norm": 1.5452758073806763,
+      "learning_rate": 0.0001998694708441745,
+      "loss": 2.9202,
+      "step": 60
+    },
+    {
+      "epoch": 0.03956542889573537,
+      "grad_norm": 1.5281524658203125,
+      "learning_rate": 0.00019986419866015013,
+      "loss": 3.1143,
+      "step": 61
+    },
+    {
+      "epoch": 0.04021404248419004,
+      "grad_norm": 1.6060402393341064,
+      "learning_rate": 0.00019985882217192423,
+      "loss": 3.5452,
+      "step": 62
+    },
+    {
+      "epoch": 0.04086265607264472,
+      "grad_norm": 1.6011849641799927,
+      "learning_rate": 0.00019985334138511237,
+      "loss": 3.5304,
+      "step": 63
+    },
+    {
+      "epoch": 0.0415112696610994,
+      "grad_norm": 1.8889275789260864,
+      "learning_rate": 0.00019984775630543902,
+      "loss": 3.2174,
+      "step": 64
+    },
+    {
+      "epoch": 0.04215988324955408,
+      "grad_norm": 1.7960286140441895,
+      "learning_rate": 0.00019984206693873753,
+      "loss": 3.6113,
+      "step": 65
+    },
+    {
+      "epoch": 0.042808496838008754,
+      "grad_norm": 1.4898135662078857,
+      "learning_rate": 0.00019983627329095028,
+      "loss": 3.2973,
+      "step": 66
+    },
+    {
+      "epoch": 0.043457110426463434,
+      "grad_norm": 1.5716447830200195,
+      "learning_rate": 0.00019983037536812842,
+      "loss": 3.011,
+      "step": 67
+    },
+    {
+      "epoch": 0.044105724014918114,
+      "grad_norm": 1.4708287715911865,
+      "learning_rate": 0.00019982437317643217,
+      "loss": 3.3225,
+      "step": 68
+    },
+    {
+      "epoch": 0.04475433760337279,
+      "grad_norm": 1.4985527992248535,
+      "learning_rate": 0.00019981826672213054,
+      "loss": 3.1602,
+      "step": 69
+    },
+    {
+      "epoch": 0.04540295119182747,
+      "grad_norm": 1.4717998504638672,
+      "learning_rate": 0.0001998120560116015,
+      "loss": 3.2304,
+      "step": 70
+    },
+    {
+      "epoch": 0.04605156478028215,
+      "grad_norm": 1.9946448802947998,
+      "learning_rate": 0.00019980574105133187,
+      "loss": 3.3665,
+      "step": 71
+    },
+    {
+      "epoch": 0.04670017836873683,
+      "grad_norm": 1.5488526821136475,
+      "learning_rate": 0.00019979932184791742,
+      "loss": 3.0899,
+      "step": 72
+    },
+    {
+      "epoch": 0.0473487919571915,
+      "grad_norm": 1.3588461875915527,
+      "learning_rate": 0.00019979279840806271,
+      "loss": 3.0037,
+      "step": 73
+    },
+    {
+      "epoch": 0.04799740554564618,
+      "grad_norm": 1.368888258934021,
+      "learning_rate": 0.00019978617073858123,
+      "loss": 3.1688,
+      "step": 74
+    },
+    {
+      "epoch": 0.04864601913410086,
+      "grad_norm": 1.7766389846801758,
+      "learning_rate": 0.00019977943884639534,
+      "loss": 3.0139,
+      "step": 75
+    },
+    {
+      "epoch": 0.049294632722555534,
+      "grad_norm": 1.4920904636383057,
+      "learning_rate": 0.0001997726027385362,
+      "loss": 3.2154,
+      "step": 76
+    },
+    {
+      "epoch": 0.049943246311010214,
+      "grad_norm": 1.707010269165039,
+      "learning_rate": 0.00019976566242214388,
+      "loss": 3.3891,
+      "step": 77
+    },
+    {
+      "epoch": 0.050591859899464894,
+      "grad_norm": 1.7228715419769287,
+      "learning_rate": 0.00019975861790446722,
+      "loss": 3.4335,
+      "step": 78
+    },
+    {
+      "epoch": 0.051240473487919574,
+      "grad_norm": 1.423966646194458,
+      "learning_rate": 0.000199751469192864,
+      "loss": 3.0397,
+      "step": 79
+    },
+    {
+      "epoch": 0.05188908707637425,
+      "grad_norm": 1.5038292407989502,
+      "learning_rate": 0.00019974421629480075,
+      "loss": 3.0318,
+      "step": 80
+    },
+    {
+      "epoch": 0.05253770066482893,
+      "grad_norm": 1.4808934926986694,
+      "learning_rate": 0.00019973685921785282,
+      "loss": 3.1422,
+      "step": 81
+    },
+    {
+      "epoch": 0.05318631425328361,
+      "grad_norm": 1.4777363538742065,
+      "learning_rate": 0.00019972939796970436,
+      "loss": 3.3775,
+      "step": 82
+    },
+    {
+      "epoch": 0.05383492784173829,
+      "grad_norm": 1.7979999780654907,
+      "learning_rate": 0.00019972183255814843,
+      "loss": 3.3686,
+      "step": 83
+    },
+    {
+      "epoch": 0.05448354143019296,
+      "grad_norm": 1.5056769847869873,
+      "learning_rate": 0.00019971416299108672,
+      "loss": 3.2013,
+      "step": 84
+    },
+    {
+      "epoch": 0.05513215501864764,
+      "grad_norm": 1.389157772064209,
+      "learning_rate": 0.0001997063892765298,
+      "loss": 3.0217,
+      "step": 85
+    },
+    {
+      "epoch": 0.05578076860710232,
+      "grad_norm": 1.7881611585617065,
+      "learning_rate": 0.00019969851142259706,
+      "loss": 3.1829,
+      "step": 86
+    },
+    {
+      "epoch": 0.056429382195556994,
+      "grad_norm": 1.3970147371292114,
+      "learning_rate": 0.0001996905294375166,
+      "loss": 3.059,
+      "step": 87
+    },
+    {
+      "epoch": 0.057077995784011674,
+      "grad_norm": 1.655731439590454,
+      "learning_rate": 0.0001996824433296252,
+      "loss": 3.3041,
+      "step": 88
+    },
+    {
+      "epoch": 0.057726609372466355,
+      "grad_norm": 1.7357268333435059,
+      "learning_rate": 0.0001996742531073686,
+      "loss": 3.4154,
+      "step": 89
+    },
+    {
+      "epoch": 0.058375222960921035,
+      "grad_norm": 1.6906143426895142,
+      "learning_rate": 0.00019966595877930106,
+      "loss": 3.186,
+      "step": 90
+    },
+    {
+      "epoch": 0.05902383654937571,
+      "grad_norm": 1.3434598445892334,
+      "learning_rate": 0.00019965756035408573,
+      "loss": 3.0845,
+      "step": 91
+    },
+    {
+      "epoch": 0.05967245013783039,
+      "grad_norm": 1.5761163234710693,
+      "learning_rate": 0.00019964905784049442,
+      "loss": 3.2079,
+      "step": 92
+    },
+    {
+      "epoch": 0.06032106372628507,
+      "grad_norm": 1.366127371788025,
+      "learning_rate": 0.00019964045124740772,
+      "loss": 3.0052,
+      "step": 93
+    },
+    {
+      "epoch": 0.06096967731473974,
+      "grad_norm": 1.5109351873397827,
+      "learning_rate": 0.0001996317405838148,
+      "loss": 3.0047,
+      "step": 94
+    },
+    {
+      "epoch": 0.06161829090319442,
+      "grad_norm": 1.2576245069503784,
+      "learning_rate": 0.0001996229258588136,
+      "loss": 3.1236,
+      "step": 95
+    },
+    {
+      "epoch": 0.0622669044916491,
+      "grad_norm": 1.4992951154708862,
+      "learning_rate": 0.0001996140070816108,
+      "loss": 3.1032,
+      "step": 96
+    },
+    {
+      "epoch": 0.06291551808010377,
+      "grad_norm": 1.4879511594772339,
+      "learning_rate": 0.0001996049842615217,
+      "loss": 3.2732,
+      "step": 97
+    },
+    {
+      "epoch": 0.06356413166855845,
+      "grad_norm": 1.2676115036010742,
+      "learning_rate": 0.00019959585740797028,
+      "loss": 3.0975,
+      "step": 98
+    },
+    {
+      "epoch": 0.06421274525701313,
+      "grad_norm": 1.826397180557251,
+      "learning_rate": 0.00019958662653048913,
+      "loss": 3.3263,
+      "step": 99
+    },
+    {
+      "epoch": 0.06486135884546781,
+      "grad_norm": 1.3498917818069458,
+      "learning_rate": 0.00019957729163871962,
+      "loss": 3.1325,
+      "step": 100
+    },
+    {
+      "epoch": 0.06486135884546781,
+      "eval_loss": 3.3947603702545166,
+      "eval_runtime": 35.0846,
+      "eval_samples_per_second": 58.601,
+      "eval_steps_per_second": 14.65,
+      "step": 100
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 3084,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "EarlyStoppingCallback": {
+      "args": {
+        "early_stopping_patience": 2,
+        "early_stopping_threshold": 0.0
+      },
+      "attributes": {
+        "early_stopping_patience_counter": 0
+      }
+    },
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 7209543008256000.0,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3bc81bac9a771772e15a7ff8d91c6031d56c2b99b3df1c652f1619417809f8bc
+size 6776