Upload folder using huggingface_hub

adapter_config.json

@@ -20,9 +20,9 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "up_proj",
     "o_proj",
     "down_proj",
+    "up_proj",
     "gate_proj",
     "Wqkv"
   ],

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8cc369bd50d6638a1cb511201fa2a7311eb09eaba9474129b6038722ccf333b1
+oid sha256:b2d43d47b46d25c72215b5d4ffd66d7a49c408c465c4eee0e1c6a5dac531ca3e
 size 681596224

phi_sft/checkpoint-145/README.md

@@ -0,0 +1,202 @@
+---
+base_model: microsoft/Phi-3-medium-4k-instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

phi_sft/checkpoint-145/adapter_config.json

@@ -0,0 +1,32 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "microsoft/Phi-3-medium-4k-instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "down_proj",
+    "up_proj",
+    "gate_proj",
+    "Wqkv"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

phi_sft/checkpoint-145/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b2d43d47b46d25c72215b5d4ffd66d7a49c408c465c4eee0e1c6a5dac531ca3e
+size 681596224

phi_sft/checkpoint-145/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4256f4dc1f8083ce55dddc28c2a9d755483c4736b6c7bba72b6b4c6d208bad43
+size 346305466

phi_sft/checkpoint-145/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f9137a602b56427676d11fe56428e1e8af945cbad348e7ff2829e615c339c495
+size 14244

phi_sft/checkpoint-145/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee838dfba322d506fceea33abd86a87dabc113c9164438a0ebd2f9eb277c4896
+size 1064

phi_sft/checkpoint-145/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

phi_sft/checkpoint-145/tokenizer_config.json

@@ -0,0 +1,131 @@
+{
+  "add_bos_token": false,
+  "add_eos_token": false,
+  "add_prefix_space": null,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": false
+    },
+    "32000": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32001": {
+      "content": "<|assistant|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32002": {
+      "content": "<|placeholder1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32003": {
+      "content": "<|placeholder2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32004": {
+      "content": "<|placeholder3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32005": {
+      "content": "<|placeholder4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32006": {
+      "content": "<|system|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32007": {
+      "content": "<|end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32008": {
+      "content": "<|placeholder5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32009": {
+      "content": "<|placeholder6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32010": {
+      "content": "<|user|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "chat_template": "{% for message in messages %}{% if (message['role'] == 'user') %}{{'<|user|>' + '\n' + message['content'] + '<|end|>' + '\n' + '<|assistant|>' + '\n'}}{% elif (message['role'] == 'assistant') %}{{message['content'] + '<|end|>' + '\n'}}{% endif %}{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "legacy": false,
+  "model_max_length": 4096,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "left",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

phi_sft/checkpoint-145/trainer_state.json

@@ -0,0 +1,1048 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 5.0,
+  "eval_steps": 500,
+  "global_step": 145,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.034482758620689655,
+      "grad_norm": 0.18263371288776398,
+      "learning_rate": 2.0000000000000003e-06,
+      "loss": 1.1722,
+      "step": 1
+    },
+    {
+      "epoch": 0.06896551724137931,
+      "grad_norm": 0.14676940441131592,
+      "learning_rate": 4.000000000000001e-06,
+      "loss": 1.1382,
+      "step": 2
+    },
+    {
+      "epoch": 0.10344827586206896,
+      "grad_norm": 0.15175683796405792,
+      "learning_rate": 6e-06,
+      "loss": 1.0623,
+      "step": 3
+    },
+    {
+      "epoch": 0.13793103448275862,
+      "grad_norm": 0.22508814930915833,
+      "learning_rate": 8.000000000000001e-06,
+      "loss": 1.0764,
+      "step": 4
+    },
+    {
+      "epoch": 0.1724137931034483,
+      "grad_norm": 0.23835930228233337,
+      "learning_rate": 1e-05,
+      "loss": 1.1267,
+      "step": 5
+    },
+    {
+      "epoch": 0.20689655172413793,
+      "grad_norm": 0.4437412917613983,
+      "learning_rate": 1.2e-05,
+      "loss": 1.0839,
+      "step": 6
+    },
+    {
+      "epoch": 0.2413793103448276,
+      "grad_norm": 0.22235934436321259,
+      "learning_rate": 1.4e-05,
+      "loss": 1.0636,
+      "step": 7
+    },
+    {
+      "epoch": 0.27586206896551724,
+      "grad_norm": 0.19867576658725739,
+      "learning_rate": 1.6000000000000003e-05,
+      "loss": 1.3298,
+      "step": 8
+    },
+    {
+      "epoch": 0.3103448275862069,
+      "grad_norm": 0.24894282221794128,
+      "learning_rate": 1.8e-05,
+      "loss": 1.2449,
+      "step": 9
+    },
+    {
+      "epoch": 0.3448275862068966,
+      "grad_norm": 0.14895613491535187,
+      "learning_rate": 2e-05,
+      "loss": 1.0393,
+      "step": 10
+    },
+    {
+      "epoch": 0.3793103448275862,
+      "grad_norm": 0.1951148360967636,
+      "learning_rate": 1.999729241179462e-05,
+      "loss": 0.9705,
+      "step": 11
+    },
+    {
+      "epoch": 0.41379310344827586,
+      "grad_norm": 0.24513274431228638,
+      "learning_rate": 1.998917111338525e-05,
+      "loss": 1.0221,
+      "step": 12
+    },
+    {
+      "epoch": 0.4482758620689655,
+      "grad_norm": 0.37853339314460754,
+      "learning_rate": 1.9975640502598243e-05,
+      "loss": 1.0526,
+      "step": 13
+    },
+    {
+      "epoch": 0.4827586206896552,
+      "grad_norm": 0.25096696615219116,
+      "learning_rate": 1.9956707906498046e-05,
+      "loss": 1.0337,
+      "step": 14
+    },
+    {
+      "epoch": 0.5172413793103449,
+      "grad_norm": 0.32762548327445984,
+      "learning_rate": 1.9932383577419432e-05,
+      "loss": 1.1386,
+      "step": 15
+    },
+    {
+      "epoch": 0.5517241379310345,
+      "grad_norm": 0.2336706519126892,
+      "learning_rate": 1.9902680687415704e-05,
+      "loss": 1.0962,
+      "step": 16
+    },
+    {
+      "epoch": 0.5862068965517241,
+      "grad_norm": 0.25260019302368164,
+      "learning_rate": 1.9867615321125796e-05,
+      "loss": 1.0128,
+      "step": 17
+    },
+    {
+      "epoch": 0.6206896551724138,
+      "grad_norm": 0.208200603723526,
+      "learning_rate": 1.9827206467064133e-05,
+      "loss": 1.1418,
+      "step": 18
+    },
+    {
+      "epoch": 0.6551724137931034,
+      "grad_norm": 0.24314813315868378,
+      "learning_rate": 1.9781476007338058e-05,
+      "loss": 1.0557,
+      "step": 19
+    },
+    {
+      "epoch": 0.6896551724137931,
+      "grad_norm": 0.2378815859556198,
+      "learning_rate": 1.973044870579824e-05,
+      "loss": 0.9015,
+      "step": 20
+    },
+    {
+      "epoch": 0.7241379310344828,
+      "grad_norm": 0.4887104332447052,
+      "learning_rate": 1.967415219462864e-05,
+      "loss": 1.1282,
+      "step": 21
+    },
+    {
+      "epoch": 0.7586206896551724,
+      "grad_norm": 0.26686033606529236,
+      "learning_rate": 1.961261695938319e-05,
+      "loss": 1.0028,
+      "step": 22
+    },
+    {
+      "epoch": 0.7931034482758621,
+      "grad_norm": 0.20646223425865173,
+      "learning_rate": 1.954587632247732e-05,
+      "loss": 1.0274,
+      "step": 23
+    },
+    {
+      "epoch": 0.8275862068965517,
+      "grad_norm": 0.26502159237861633,
+      "learning_rate": 1.9473966425143292e-05,
+      "loss": 1.1362,
+      "step": 24
+    },
+    {
+      "epoch": 0.8620689655172413,
+      "grad_norm": 0.23833607137203217,
+      "learning_rate": 1.9396926207859085e-05,
+      "loss": 0.9086,
+      "step": 25
+    },
+    {
+      "epoch": 0.896551724137931,
+      "grad_norm": 0.24085558950901031,
+      "learning_rate": 1.9314797389261426e-05,
+      "loss": 0.9698,
+      "step": 26
+    },
+    {
+      "epoch": 0.9310344827586207,
+      "grad_norm": 0.188435897231102,
+      "learning_rate": 1.9227624443554425e-05,
+      "loss": 1.0137,
+      "step": 27
+    },
+    {
+      "epoch": 0.9655172413793104,
+      "grad_norm": 0.22470542788505554,
+      "learning_rate": 1.913545457642601e-05,
+      "loss": 0.9346,
+      "step": 28
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.3263242840766907,
+      "learning_rate": 1.9038337699485207e-05,
+      "loss": 0.948,
+      "step": 29
+    },
+    {
+      "epoch": 1.0344827586206897,
+      "grad_norm": 0.257449746131897,
+      "learning_rate": 1.8936326403234125e-05,
+      "loss": 0.9209,
+      "step": 30
+    },
+    {
+      "epoch": 1.0689655172413792,
+      "grad_norm": 0.21045814454555511,
+      "learning_rate": 1.8829475928589272e-05,
+      "loss": 0.9512,
+      "step": 31
+    },
+    {
+      "epoch": 1.103448275862069,
+      "grad_norm": 0.2047726809978485,
+      "learning_rate": 1.8717844136967626e-05,
+      "loss": 1.0788,
+      "step": 32
+    },
+    {
+      "epoch": 1.1379310344827587,
+      "grad_norm": 0.22279374301433563,
+      "learning_rate": 1.860149147895366e-05,
+      "loss": 0.8921,
+      "step": 33
+    },
+    {
+      "epoch": 1.1724137931034484,
+      "grad_norm": 0.24843072891235352,
+      "learning_rate": 1.848048096156426e-05,
+      "loss": 0.9741,
+      "step": 34
+    },
+    {
+      "epoch": 1.206896551724138,
+      "grad_norm": 0.25808510184288025,
+      "learning_rate": 1.8354878114129368e-05,
+      "loss": 1.0236,
+      "step": 35
+    },
+    {
+      "epoch": 1.2413793103448276,
+      "grad_norm": 0.2522240877151489,
+      "learning_rate": 1.8224750952806626e-05,
+      "loss": 0.8968,
+      "step": 36
+    },
+    {
+      "epoch": 1.2758620689655173,
+      "grad_norm": 0.19142690300941467,
+      "learning_rate": 1.8090169943749477e-05,
+      "loss": 0.8529,
+      "step": 37
+    },
+    {
+      "epoch": 1.3103448275862069,
+      "grad_norm": 0.21294455230236053,
+      "learning_rate": 1.795120796494848e-05,
+      "loss": 0.857,
+      "step": 38
+    },
+    {
+      "epoch": 1.3448275862068966,
+      "grad_norm": 0.22687983512878418,
+      "learning_rate": 1.7807940266766595e-05,
+      "loss": 0.8623,
+      "step": 39
+    },
+    {
+      "epoch": 1.3793103448275863,
+      "grad_norm": 0.3414287269115448,
+      "learning_rate": 1.766044443118978e-05,
+      "loss": 1.0595,
+      "step": 40
+    },
+    {
+      "epoch": 1.4137931034482758,
+      "grad_norm": 0.23636585474014282,
+      "learning_rate": 1.7508800329814993e-05,
+      "loss": 0.8035,
+      "step": 41
+    },
+    {
+      "epoch": 1.4482758620689655,
+      "grad_norm": 0.20115771889686584,
+      "learning_rate": 1.735309008059829e-05,
+      "loss": 0.8461,
+      "step": 42
+    },
+    {
+      "epoch": 1.4827586206896552,
+      "grad_norm": 0.24040541052818298,
+      "learning_rate": 1.7193398003386514e-05,
+      "loss": 0.8927,
+      "step": 43
+    },
+    {
+      "epoch": 1.5172413793103448,
+      "grad_norm": 0.24060678482055664,
+      "learning_rate": 1.702981057425662e-05,
+      "loss": 0.9177,
+      "step": 44
+    },
+    {
+      "epoch": 1.5517241379310345,
+      "grad_norm": 0.20893888175487518,
+      "learning_rate": 1.686241637868734e-05,
+      "loss": 0.9427,
+      "step": 45
+    },
+    {
+      "epoch": 1.5862068965517242,
+      "grad_norm": 0.2027370184659958,
+      "learning_rate": 1.6691306063588583e-05,
+      "loss": 1.0462,
+      "step": 46
+    },
+    {
+      "epoch": 1.6206896551724137,
+      "grad_norm": 0.23139198124408722,
+      "learning_rate": 1.6516572288214555e-05,
+      "loss": 0.9829,
+      "step": 47
+    },
+    {
+      "epoch": 1.6551724137931034,
+      "grad_norm": 0.20256613194942474,
+      "learning_rate": 1.63383096739871e-05,
+      "loss": 0.9218,
+      "step": 48
+    },
+    {
+      "epoch": 1.6896551724137931,
+      "grad_norm": 0.26435062289237976,
+      "learning_rate": 1.6156614753256583e-05,
+      "loss": 0.9617,
+      "step": 49
+    },
+    {
+      "epoch": 1.7241379310344827,
+      "grad_norm": 0.2765027582645416,
+      "learning_rate": 1.5971585917027864e-05,
+      "loss": 0.9572,
+      "step": 50
+    },
+    {
+      "epoch": 1.7586206896551724,
+      "grad_norm": 0.5229343175888062,
+      "learning_rate": 1.5783323361679865e-05,
+      "loss": 0.9205,
+      "step": 51
+    },
+    {
+      "epoch": 1.793103448275862,
+      "grad_norm": 0.2305651754140854,
+      "learning_rate": 1.5591929034707468e-05,
+      "loss": 0.9161,
+      "step": 52
+    },
+    {
+      "epoch": 1.8275862068965516,
+      "grad_norm": 0.2311047613620758,
+      "learning_rate": 1.539750657951513e-05,
+      "loss": 0.847,
+      "step": 53
+    },
+    {
+      "epoch": 1.8620689655172413,
+      "grad_norm": 0.21977312862873077,
+      "learning_rate": 1.5200161279292154e-05,
+      "loss": 0.8169,
+      "step": 54
+    },
+    {
+      "epoch": 1.896551724137931,
+      "grad_norm": 0.22656522691249847,
+      "learning_rate": 1.5000000000000002e-05,
+      "loss": 0.9383,
+      "step": 55
+    },
+    {
+      "epoch": 1.9310344827586206,
+      "grad_norm": 0.2748524844646454,
+      "learning_rate": 1.4797131132502464e-05,
+      "loss": 1.0618,
+      "step": 56
+    },
+    {
+      "epoch": 1.9655172413793105,
+      "grad_norm": 0.22263024747371674,
+      "learning_rate": 1.4591664533870118e-05,
+      "loss": 0.9597,
+      "step": 57
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.3616458773612976,
+      "learning_rate": 1.4383711467890776e-05,
+      "loss": 0.9062,
+      "step": 58
+    },
+    {
+      "epoch": 2.0344827586206895,
+      "grad_norm": 0.2252507507801056,
+      "learning_rate": 1.417338454481818e-05,
+      "loss": 0.9398,
+      "step": 59
+    },
+    {
+      "epoch": 2.0689655172413794,
+      "grad_norm": 0.2219877392053604,
+      "learning_rate": 1.396079766039157e-05,
+      "loss": 0.8734,
+      "step": 60
+    },
+    {
+      "epoch": 2.103448275862069,
+      "grad_norm": 0.2383543848991394,
+      "learning_rate": 1.3746065934159123e-05,
+      "loss": 0.9004,
+      "step": 61
+    },
+    {
+      "epoch": 2.1379310344827585,
+      "grad_norm": 0.21969589591026306,
+      "learning_rate": 1.3529305647138689e-05,
+      "loss": 0.8606,
+      "step": 62
+    },
+    {
+      "epoch": 2.1724137931034484,
+      "grad_norm": 0.22909048199653625,
+      "learning_rate": 1.3310634178849583e-05,
+      "loss": 0.8885,
+      "step": 63
+    },
+    {
+      "epoch": 2.206896551724138,
+      "grad_norm": 0.25766170024871826,
+      "learning_rate": 1.3090169943749475e-05,
+      "loss": 0.7432,
+      "step": 64
+    },
+    {
+      "epoch": 2.2413793103448274,
+      "grad_norm": 0.23315741121768951,
+      "learning_rate": 1.2868032327110904e-05,
+      "loss": 0.9039,
+      "step": 65
+    },
+    {
+      "epoch": 2.2758620689655173,
+      "grad_norm": 0.26347848773002625,
+      "learning_rate": 1.2644341620372025e-05,
+      "loss": 0.8992,
+      "step": 66
+    },
+    {
+      "epoch": 2.310344827586207,
+      "grad_norm": 0.26184648275375366,
+      "learning_rate": 1.2419218955996677e-05,
+      "loss": 0.8213,
+      "step": 67
+    },
+    {
+      "epoch": 2.344827586206897,
+      "grad_norm": 0.2565184533596039,
+      "learning_rate": 1.2192786241879033e-05,
+      "loss": 1.0208,
+      "step": 68
+    },
+    {
+      "epoch": 2.3793103448275863,
+      "grad_norm": 0.2696502208709717,
+      "learning_rate": 1.1965166095328302e-05,
+      "loss": 0.8618,
+      "step": 69
+    },
+    {
+      "epoch": 2.413793103448276,
+      "grad_norm": 0.22688528895378113,
+      "learning_rate": 1.1736481776669307e-05,
+      "loss": 0.9013,
+      "step": 70
+    },
+    {
+      "epoch": 2.4482758620689653,
+      "grad_norm": 0.2907235622406006,
+      "learning_rate": 1.1506857122494832e-05,
+      "loss": 0.8826,
+      "step": 71
+    },
+    {
+      "epoch": 2.4827586206896552,
+      "grad_norm": 0.2706524431705475,
+      "learning_rate": 1.127641647860595e-05,
+      "loss": 0.8441,
+      "step": 72
+    },
+    {
+      "epoch": 2.5172413793103448,
+      "grad_norm": 0.37649792432785034,
+      "learning_rate": 1.1045284632676535e-05,
+      "loss": 0.9164,
+      "step": 73
+    },
+    {
+      "epoch": 2.5517241379310347,
+      "grad_norm": 0.271186888217926,
+      "learning_rate": 1.0813586746678584e-05,
+      "loss": 0.764,
+      "step": 74
+    },
+    {
+      "epoch": 2.586206896551724,
+      "grad_norm": 0.2484758198261261,
+      "learning_rate": 1.0581448289104759e-05,
+      "loss": 0.8553,
+      "step": 75
+    },
+    {
+      "epoch": 2.6206896551724137,
+      "grad_norm": 0.34356212615966797,
+      "learning_rate": 1.0348994967025012e-05,
+      "loss": 0.7467,
+      "step": 76
+    },
+    {
+      "epoch": 2.655172413793103,
+      "grad_norm": 0.2672325074672699,
+      "learning_rate": 1.0116352658013973e-05,
+      "loss": 0.7631,
+      "step": 77
+    },
+    {
+      "epoch": 2.689655172413793,
+      "grad_norm": 0.2786653935909271,
+      "learning_rate": 9.883647341986032e-06,
+      "loss": 1.01,
+      "step": 78
+    },
+    {
+      "epoch": 2.7241379310344827,
+      "grad_norm": 0.241670161485672,
+      "learning_rate": 9.651005032974994e-06,
+      "loss": 0.908,
+      "step": 79
+    },
+    {
+      "epoch": 2.7586206896551726,
+      "grad_norm": 0.2924453020095825,
+      "learning_rate": 9.418551710895243e-06,
+      "loss": 0.873,
+      "step": 80
+    },
+    {
+      "epoch": 2.793103448275862,
+      "grad_norm": 0.29062798619270325,
+      "learning_rate": 9.18641325332142e-06,
+      "loss": 1.1795,
+      "step": 81
+    },
+    {
+      "epoch": 2.8275862068965516,
+      "grad_norm": 0.26050105690956116,
+      "learning_rate": 8.954715367323468e-06,
+      "loss": 0.7916,
+      "step": 82
+    },
+    {
+      "epoch": 2.862068965517241,
+      "grad_norm": 0.23967131972312927,
+      "learning_rate": 8.723583521394054e-06,
+      "loss": 0.9865,
+      "step": 83
+    },
+    {
+      "epoch": 2.896551724137931,
+      "grad_norm": 0.2599307596683502,
+      "learning_rate": 8.49314287750517e-06,
+      "loss": 1.0798,
+      "step": 84
+    },
+    {
+      "epoch": 2.9310344827586206,
+      "grad_norm": 0.3155495524406433,
+      "learning_rate": 8.263518223330698e-06,
+      "loss": 0.8233,
+      "step": 85
+    },
+    {
+      "epoch": 2.9655172413793105,
+      "grad_norm": 0.24418021738529205,
+      "learning_rate": 8.034833904671698e-06,
+      "loss": 0.752,
+      "step": 86
+    },
+    {
+      "epoch": 3.0,
+      "grad_norm": 0.32657715678215027,
+      "learning_rate": 7.807213758120965e-06,
+      "loss": 0.7967,
+      "step": 87
+    },
+    {
+      "epoch": 3.0344827586206895,
+      "grad_norm": 0.23531968891620636,
+      "learning_rate": 7.580781044003324e-06,
+      "loss": 0.8511,
+      "step": 88
+    },
+    {
+      "epoch": 3.0689655172413794,
+      "grad_norm": 0.26011955738067627,
+      "learning_rate": 7.355658379627981e-06,
+      "loss": 0.7617,
+      "step": 89
+    },
+    {
+      "epoch": 3.103448275862069,
+      "grad_norm": 0.2797260880470276,
+      "learning_rate": 7.131967672889101e-06,
+      "loss": 0.932,
+      "step": 90
+    },
+    {
+      "epoch": 3.1379310344827585,
+      "grad_norm": 0.2513308525085449,
+      "learning_rate": 6.909830056250527e-06,
+      "loss": 0.9837,
+      "step": 91
+    },
+    {
+      "epoch": 3.1724137931034484,
+      "grad_norm": 0.31923943758010864,
+      "learning_rate": 6.689365821150421e-06,
+      "loss": 0.8026,
+      "step": 92
+    },
+    {
+      "epoch": 3.206896551724138,
+      "grad_norm": 0.2611626982688904,
+      "learning_rate": 6.4706943528613135e-06,
+      "loss": 0.8217,
+      "step": 93
+    },
+    {
+      "epoch": 3.2413793103448274,
+      "grad_norm": 0.24943020939826965,
+      "learning_rate": 6.25393406584088e-06,
+      "loss": 0.7969,
+      "step": 94
+    },
+    {
+      "epoch": 3.2758620689655173,
+      "grad_norm": 0.22827626764774323,
+      "learning_rate": 6.039202339608432e-06,
+      "loss": 0.6983,
+      "step": 95
+    },
+    {
+      "epoch": 3.310344827586207,
+      "grad_norm": 0.28360244631767273,
+      "learning_rate": 5.8266154551818225e-06,
+      "loss": 0.8632,
+      "step": 96
+    },
+    {
+      "epoch": 3.344827586206897,
+      "grad_norm": 0.26782166957855225,
+      "learning_rate": 5.616288532109225e-06,
+      "loss": 0.9549,
+      "step": 97
+    },
+    {
+      "epoch": 3.3793103448275863,
+      "grad_norm": 0.24422940611839294,
+      "learning_rate": 5.4083354661298816e-06,
+      "loss": 0.7525,
+      "step": 98
+    },
+    {
+      "epoch": 3.413793103448276,
+      "grad_norm": 0.30532753467559814,
+      "learning_rate": 5.202868867497542e-06,
+      "loss": 0.8082,
+      "step": 99
+    },
+    {
+      "epoch": 3.4482758620689653,
+      "grad_norm": 0.2639884054660797,
+      "learning_rate": 5.000000000000003e-06,
+      "loss": 0.7849,
+      "step": 100
+    },
+    {
+      "epoch": 3.4827586206896552,
+      "grad_norm": 0.3023616075515747,
+      "learning_rate": 4.799838720707847e-06,
+      "loss": 0.7912,
+      "step": 101
+    },
+    {
+      "epoch": 3.5172413793103448,
+      "grad_norm": 0.2704639732837677,
+      "learning_rate": 4.6024934204848745e-06,
+      "loss": 0.8371,
+      "step": 102
+    },
+    {
+      "epoch": 3.5517241379310347,
+      "grad_norm": 0.2659608721733093,
+      "learning_rate": 4.408070965292534e-06,
+      "loss": 0.8002,
+      "step": 103
+    },
+    {
+      "epoch": 3.586206896551724,
+      "grad_norm": 0.4289320409297943,
+      "learning_rate": 4.216676638320135e-06,
+      "loss": 0.8571,
+      "step": 104
+    },
+    {
+      "epoch": 3.6206896551724137,
+      "grad_norm": 0.28088241815567017,
+      "learning_rate": 4.028414082972141e-06,
+      "loss": 0.9002,
+      "step": 105
+    },
+    {
+      "epoch": 3.655172413793103,
+      "grad_norm": 0.24444082379341125,
+      "learning_rate": 3.8433852467434175e-06,
+      "loss": 0.7742,
+      "step": 106
+    },
+    {
+      "epoch": 3.689655172413793,
+      "grad_norm": 0.33060815930366516,
+      "learning_rate": 3.661690326012897e-06,
+      "loss": 0.8304,
+      "step": 107
+    },
+    {
+      "epoch": 3.7241379310344827,
+      "grad_norm": 0.30594876408576965,
+      "learning_rate": 3.483427711785449e-06,
+      "loss": 0.9203,
+      "step": 108
+    },
+    {
+      "epoch": 3.7586206896551726,
+      "grad_norm": 0.26667875051498413,
+      "learning_rate": 3.308693936411421e-06,
+      "loss": 0.913,
+      "step": 109
+    },
+    {
+      "epoch": 3.793103448275862,
+      "grad_norm": 0.28447291254997253,
+      "learning_rate": 3.1375836213126653e-06,
+      "loss": 0.8509,
+      "step": 110
+    },
+    {
+      "epoch": 3.8275862068965516,
+      "grad_norm": 0.24133624136447906,
+      "learning_rate": 2.970189425743383e-06,
+      "loss": 0.9497,
+      "step": 111
+    },
+    {
+      "epoch": 3.862068965517241,
+      "grad_norm": 0.2903217077255249,
+      "learning_rate": 2.8066019966134907e-06,
+      "loss": 1.0794,
+      "step": 112
+    },
+    {
+      "epoch": 3.896551724137931,
+      "grad_norm": 0.3111764192581177,
+      "learning_rate": 2.6469099194017144e-06,
+      "loss": 0.9053,
+      "step": 113
+    },
+    {
+      "epoch": 3.9310344827586206,
+      "grad_norm": 0.24572409689426422,
+      "learning_rate": 2.4911996701850083e-06,
+      "loss": 0.8585,
+      "step": 114
+    },
+    {
+      "epoch": 3.9655172413793105,
+      "grad_norm": 0.3079777657985687,
+      "learning_rate": 2.339555568810221e-06,
+      "loss": 1.0361,
+      "step": 115
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 0.37165409326553345,
+      "learning_rate": 2.192059733233408e-06,
+      "loss": 0.8723,
+      "step": 116
+    },
+    {
+      "epoch": 4.0344827586206895,
+      "grad_norm": 0.26258090138435364,
+      "learning_rate": 2.048792035051521e-06,
+      "loss": 0.9485,
+      "step": 117
+    },
+    {
+      "epoch": 4.068965517241379,
+      "grad_norm": 0.30095139145851135,
+      "learning_rate": 1.9098300562505266e-06,
+      "loss": 0.8825,
+      "step": 118
+    },
+    {
+      "epoch": 4.103448275862069,
+      "grad_norm": 0.26902303099632263,
+      "learning_rate": 1.7752490471933769e-06,
+      "loss": 0.8113,
+      "step": 119
+    },
+    {
+      "epoch": 4.137931034482759,
+      "grad_norm": 0.31948816776275635,
+      "learning_rate": 1.6451218858706374e-06,
+      "loss": 0.7612,
+      "step": 120
+    },
+    {
+      "epoch": 4.172413793103448,
+      "grad_norm": 0.24395278096199036,
+      "learning_rate": 1.5195190384357405e-06,
+      "loss": 0.811,
+      "step": 121
+    },
+    {
+      "epoch": 4.206896551724138,
+      "grad_norm": 0.2477354109287262,
+      "learning_rate": 1.3985085210463479e-06,
+      "loss": 0.8972,
+      "step": 122
+    },
+    {
+      "epoch": 4.241379310344827,
+      "grad_norm": 0.2926355004310608,
+      "learning_rate": 1.282155863032377e-06,
+      "loss": 0.8915,
+      "step": 123
+    },
+    {
+      "epoch": 4.275862068965517,
+      "grad_norm": 0.2731974720954895,
+      "learning_rate": 1.1705240714107301e-06,
+      "loss": 0.6849,
+      "step": 124
+    },
+    {
+      "epoch": 4.310344827586207,
+      "grad_norm": 0.2553686201572418,
+      "learning_rate": 1.0636735967658785e-06,
+      "loss": 0.8023,
+      "step": 125
+    },
+    {
+      "epoch": 4.344827586206897,
+      "grad_norm": 0.2760959267616272,
+      "learning_rate": 9.616623005147952e-07,
+      "loss": 0.8025,
+      "step": 126
+    },
+    {
+      "epoch": 4.379310344827586,
+      "grad_norm": 0.3258477747440338,
+      "learning_rate": 8.645454235739903e-07,
+      "loss": 0.824,
+      "step": 127
+    },
+    {
+      "epoch": 4.413793103448276,
+      "grad_norm": 0.26479798555374146,
+      "learning_rate": 7.723755564455771e-07,
+      "loss": 0.91,
+      "step": 128
+    },
+    {
+      "epoch": 4.448275862068965,
+      "grad_norm": 0.25864148139953613,
+      "learning_rate": 6.852026107385756e-07,
+      "loss": 0.8891,
+      "step": 129
+    },
+    {
+      "epoch": 4.482758620689655,
+      "grad_norm": 0.25734126567840576,
+      "learning_rate": 6.030737921409169e-07,
+      "loss": 0.7896,
+      "step": 130
+    },
+    {
+      "epoch": 4.517241379310345,
+      "grad_norm": 0.26618000864982605,
+      "learning_rate": 5.26033574856708e-07,
+      "loss": 0.8261,
+      "step": 131
+    },
+    {
+      "epoch": 4.551724137931035,
+      "grad_norm": 0.27741914987564087,
+      "learning_rate": 4.5412367752268094e-07,
+      "loss": 0.7687,
+      "step": 132
+    },
+    {
+      "epoch": 4.586206896551724,
+      "grad_norm": 0.37320151925086975,
+      "learning_rate": 3.8738304061681107e-07,
+      "loss": 1.101,
+      "step": 133
+    },
+    {
+      "epoch": 4.620689655172414,
+      "grad_norm": 0.2821623384952545,
+      "learning_rate": 3.2584780537136206e-07,
+      "loss": 0.8905,
+      "step": 134
+    },
+    {
+      "epoch": 4.655172413793103,
+      "grad_norm": 0.2923290729522705,
+      "learning_rate": 2.6955129420176193e-07,
+      "loss": 0.958,
+      "step": 135
+    },
+    {
+      "epoch": 4.689655172413794,
+      "grad_norm": 0.2523677945137024,
+      "learning_rate": 2.1852399266194312e-07,
+      "loss": 0.7831,
+      "step": 136
+    },
+    {
+      "epoch": 4.724137931034483,
+      "grad_norm": 0.26920151710510254,
+      "learning_rate": 1.7279353293586765e-07,
+      "loss": 0.7747,
+      "step": 137
+    },
+    {
+      "epoch": 4.758620689655173,
+      "grad_norm": 0.25129997730255127,
+      "learning_rate": 1.323846788742078e-07,
+      "loss": 0.7621,
+      "step": 138
+    },
+    {
+      "epoch": 4.793103448275862,
+      "grad_norm": 0.2483283281326294,
+      "learning_rate": 9.731931258429638e-08,
+      "loss": 0.8085,
+      "step": 139
+    },
+    {
+      "epoch": 4.827586206896552,
+      "grad_norm": 0.24231112003326416,
+      "learning_rate": 6.761642258056977e-08,
+      "loss": 0.8761,
+      "step": 140
+    },
+    {
+      "epoch": 4.862068965517241,
+      "grad_norm": 0.2538662254810333,
+      "learning_rate": 4.329209350195651e-08,
+      "loss": 0.944,
+      "step": 141
+    },
+    {
+      "epoch": 4.896551724137931,
+      "grad_norm": 0.28175920248031616,
+      "learning_rate": 2.4359497401758026e-08,
+      "loss": 0.8917,
+      "step": 142
+    },
+    {
+      "epoch": 4.931034482758621,
+      "grad_norm": 0.28999796509742737,
+      "learning_rate": 1.0828886614754342e-08,
+      "loss": 0.804,
+      "step": 143
+    },
+    {
+      "epoch": 4.9655172413793105,
+      "grad_norm": 0.2674841284751892,
+      "learning_rate": 2.7075882053828605e-09,
+      "loss": 0.9698,
+      "step": 144
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 0.37377139925956726,
+      "learning_rate": 0.0,
+      "loss": 0.813,
+      "step": 145
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 145,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 5,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 7.5619687486464e+16,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

phi_sft/checkpoint-145/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e6a6acde7c9c646707b9822328c2133636a7e7784d24cdb1e849387e4d2d708e
+size 5432

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fd8a63b1846ede38a9473a4010be006301fe7f7e3d7f796f9b1fd0416a2bfadc
+oid sha256:e6a6acde7c9c646707b9822328c2133636a7e7784d24cdb1e849387e4d2d708e
 size 5432