Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,9 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+assets/data_pipelines.png filter=lfs diff=lfs merge=lfs -text
+assets/distribution_level.png filter=lfs diff=lfs merge=lfs -text
+assets/webwatcher_logo.png filter=lfs diff=lfs merge=lfs -text
+assets/webwatcher_main.png filter=lfs diff=lfs merge=lfs -text
+assets/webwatcher_performance_general.png filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,92 @@
----
-license: apache-2.0
----
+# WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent
+
+<p align="center">
+  <img src="./assets/webwatcher_logo.png" alt="logo" width="30%"/>
+</p>
+
+![version](https://img.shields.io/badge/version-1.0.0-blue)
+
+## 🥇 Introduction
+In this paper, we introduce **WebWatcher**, a multimodal agent for deep research that possesses enhanced visual-language reasoning capabilities. Our work presents a unified framework that combines complex vision-language reasoning with multi-tool interaction.
+
+<p align="center">
+  <img src="./assets/webwatcher_main.png" alt="logo" width="80%"/>
+</p>
+
+
+Key features of our approach include:
+
+<p align="center">
+  <img src="./assets/distribution_level.png" alt="logo" width="80%"/>
+</p>
+
+- BrowseComp-VL Benchmark: We propose a new benchmark, BrowseComp-VL, to evaluate the capabilities of multimodal agents. This challenging dataset is designed for in-depth multimodal reasoning and strategic planning, mirroring the complexity of BrowseComp but extending it into the visual domain. It emphasizes tasks that require both visual perception and advanced information-gathering abilities.
+
+<p align="center">
+  <img src="./assets/data_pipelines.png" alt="logo" width="80%"/>
+</p>
+
+- Automated Trajectory Generation: To provide robust tool-use capabilities, we developed an automated pipeline to generate high-quality, multi-step reasoning trajectories. These trajectories, which are grounded in actual tool-use behavior and reflect procedural decision-making, are used for efficient cold-start training and further optimization via reinforcement learning. The agent is equipped with several tools, including Web Image Search, Web Text Search, Webpage Visit, Code Interpreter, and an internal OCR tool.
+
+- Superior Performance: WebWatcher significantly outperforms proprietary baselines, RAG workflows, and other open-source agents across four challenging VQA benchmarks: Humanity's Last Exam (HLE)-VL, BrowseComp-VL, LiveVQA, and MMSearch. The WebWatcher-32B model, in particular, achieves an average score of 18.2% on HLE, surpassing the GPT-4o-based OmniSearch baseline. It also achieves top-tier performance on LiveVQA (58.7%) and MMSearch (55.3%), demonstrating stable and superior results on demanding, real-world visual search benchmarks.
+
+## 🚀 Performance Highlights
+<p align="center">
+  <img src="./assets/webwatcher_performance_general.png" alt="logo" width="80%"/>
+</p>
+
+1. Complex Reasoning (HLE-VL): On the Human Life Exam (HLE-VL), a benchmark for multi-step complex reasoning, WebWatcher achieved a commanding lead with a Pass@1 score of 13.6%, substantially outperforming representative models including GPT-4o (9.8%), Gemini2.5-flash (9.2%), and Qwen2.5-VL-72B (8.6%).
+
+2. Information Retrieval (MMSearch): In the MMSearch evaluation, WebWatcher demonstrated exceptional retrieval accuracy with a Pass@1 score of 55.3%, significantly surpassing Gemini2.5-flash (43.9%) and GPT-4o (24.1%), showcasing superior precision in retrieval tasks and robust information aggregation capabilities in complex scenarios.
+
+3. Knowledge-Retrieval Integration (LiveVQA): On the LiveVQA benchmark, WebWatcher achieved a Pass@1 score of 58.7%, outperforming Gemini2.5-flash (41.3%), Qwen2.5-VL-72B (35.7%), and GPT-4o (34.0%).
+
+4. Information Optimization and Aggregation (BrowseComp-VL): On BrowseComp-VL, the most comprehensively challenging benchmark, WebWatcher dominated with an average score of 27.0%, more than doubling the performance of mainstream models including GPT-4o (13.4%), Gemini2.5-flash (13.0%), and Claude-3.7 (11.2%).
+
+
+## 🔧 Quick Start
+
+### Step 1: Download the WebWatcher model
+
+You can download WebWatcher via Hugging Face [🤗 HuggingFace](https://huggingface.co/Alibaba-NLP/).
+
+### Step 2: Data Preparation
+
+Before running inference, test set images need to be downloaded to the `infer/scripts_eval/images` folder. This can be accomplished by running `infer/scripts_eval/download_image.py`. If you encounter issues downloading images from our provided OSS URLs, please obtain the images from the original dataset source and place them in the corresponding `infer/scripts_eval/images` folder.
+
+### Step 3: Inference
+
+Run `infer/scripts_eval/scripts/eval.sh` with the following required parameters:
+
+- **benchmark**: Name of the dataset to test. Available options: `'hle'`, `'gaia'`, `'livevqa'`, `'mmsearch'`, `'simplevqa'`, `'bc_vl_v1'`, `'bc_vl_v2'`. These test sets should be pre-stored in `infer/vl_search_r1/eval_data` with naming convention like `hle.jsonl`. We have provided format examples for some datasets in `infer/vl_search_r1/eval_data`. If extending to new datasets, please ensure consistent formatting.
+- **EXPERIMENT_NAME**: Name for this experiment (user-defined)
+- **MODEL_PATH**: Path to the trained model
+- **DASHSCOPE_API_KEY**: GPT API key
+- **IMG_SEARCH_KEY**: Google SerpApi key for image search
+- **JINA_API_KEY**: Jina API key
+- **SCRAPERAPI_KEY**: Scraper API key
+- **QWEN_SEARCH_KEY**: Google SerpApi key for text search
+
+**Note**: For image search tools, if you need to upload searched images to OSS, the following are required:
+- **ALIBABA_CLOUD_ACCESS_KEY_ID**: Alibaba Cloud OSS access key ID
+- **ALIBABA_CLOUD_ACCESS_KEY_SECRET**: Alibaba Cloud OSS access key secret
+
+### Step 4: Evaluation
+
+Run `infer/vl_search_r1/pass3.sh` to use LLM-as-judge for evaluating Pass@3 and Pass@1 metrics. Parameters:
+
+- **DIRECTORY**: Path to the folder containing JSONL files generated from inference
+- **DASHSCOPE_API_KEY**: GPT API key
+
+
+## 📑 Citation
+
+If this work is helpful, please kindly cite as:
+
+```bigquery
+@article{geng2025webwatcher,
+  title={WebWatcher: Breaking New Frontiers of Vision-Language Deep Research Agent},
+  author={Geng, Xinyu and Xia, Peng and Zhang, Zhen and Wang, Xinyu and Wang, Qiuchen and Ding, Ruixue and Wang, Chenxi and Wu, Jialong and Zhao, Yida and Li, Kuan and others},
+  journal={arXiv preprint arXiv:2508.05748},
+  year={2025}
+}

added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

all_results.json

@@ -0,0 +1,8 @@
+{
+    "epoch": 2.9533678756476682,
+    "total_flos": 887572825899008.0,
+    "train_loss": 0.6164978990952173,
+    "train_runtime": 16386.6242,
+    "train_samples_per_second": 1.127,
+    "train_steps_per_second": 0.009
+}

chat_template.json

@@ -0,0 +1,3 @@
+{
+  "chat_template": "{% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n{% endif %}<|im_start|>{{ message['role'] }}\n{% if message['content'] is string %}{{ message['content'] }}<|im_end|>\n{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>\n{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant\n{% endif %}"
+}

config.json

@@ -0,0 +1,65 @@
+{
+  "architectures": [
+    "Qwen2_5_VLForConditionalGeneration"
+  ],
+  "attention_dropout": 0.0,
+  "eos_token_id": 151645,
+  "hidden_act": "silu",
+  "hidden_size": 5120,
+  "image_token_id": 151655,
+  "initializer_range": 0.02,
+  "intermediate_size": 27648,
+  "max_position_embeddings": 128000,
+  "max_window_layers": 64,
+  "model_type": "qwen2_5_vl",
+  "num_attention_heads": 40,
+  "num_hidden_layers": 64,
+  "num_key_value_heads": 8,
+  "pad_token_id": 151643,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": {
+    "mrope_section": [
+      16,
+      24,
+      24
+    ],
+    "rope_type": "default",
+    "type": "default"
+  },
+  "rope_theta": 1000000.0,
+  "sliding_window": 32768,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.51.3",
+  "use_cache": false,
+  "use_sliding_window": false,
+  "video_token_id": 151656,
+  "vision_config": {
+    "depth": 32,
+    "fullatt_block_indexes": [
+      7,
+      15,
+      23,
+      31
+    ],
+    "hidden_act": "silu",
+    "hidden_size": 1280,
+    "in_channels": 3,
+    "in_chans": 3,
+    "intermediate_size": 3456,
+    "model_type": "qwen2_5_vl",
+    "num_heads": 16,
+    "out_hidden_size": 5120,
+    "patch_size": 14,
+    "spatial_merge_size": 2,
+    "spatial_patch_size": 14,
+    "temporal_patch_size": 2,
+    "tokens_per_second": 2,
+    "torch_dtype": "float32",
+    "window_size": 112
+  },
+  "vision_end_token_id": 151653,
+  "vision_start_token_id": 151652,
+  "vision_token_id": 151654,
+  "vocab_size": 152064
+}

generation_config.json

@@ -0,0 +1,12 @@
+{
+  "bos_token_id": 151643,
+  "do_sample": true,
+  "eos_token_id": [
+    151645,
+    151643
+  ],
+  "pad_token_id": 151643,
+  "repetition_penalty": 1.05,
+  "temperature": 1e-06,
+  "transformers_version": "4.51.3"
+}

model-00001-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:26cf3705f4c09922692057d62443a965bec6552748560b8dcc12a53a517430d3
+size 4958700928

model-00002-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d05a3ec201b050bb728e9df870bfe6ba7e8ec86bbb875e5d2a44cc2819cbccf7
+size 4928488272

model-00003-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2102d5d1b37388d8793e9a28c8b7afb8e703e25630940eb4274c0608c652286f
+size 4876059368

model-00004-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d0576421623dbc851e3cfcdbe1891970472b2e343a105b563bb0e22eed491f71
+size 4876059416

model-00005-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6a6a3c2e3ae93a52f4273308a3357aa7c74d079299ef4258c92499cba1d0958
+size 4876059416

model-00006-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:94f01b17a55853f80611e9ab0e76786f898c5d968b081e6c9e7c2fc35fbd1db3
+size 4876059416

model-00007-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e465a05da179b4483130a393d6f3a730067606b4605818ca0d30e5682dc64571
+size 4876059416

model-00008-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:adae06eaf08c164dfe03f9d158058050d5ac6150fcb016d62ca82dd23486f85a
+size 4876059416

model-00009-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e2d19307739502bd376d70ebf112846a96cfdfda6fd6cbd090481e3f12c8ba82
+size 4876059416

model-00010-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:abccb5925565daa00a7000d19a2c692547486510574a7defbd800ce28eb990df
+size 4876059416

model-00011-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:78271f305af5398777af037efc51a5f20a6bd68ff39bd850b3ded933fb1d79ff
+size 4876059416

model-00012-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b358a6f667a6cf479da8a68b6f3f44648c4d319db6bb45b723fce6ecddd821ed
+size 4876059416

model-00013-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:15c6732a83b2801a979f5c10fd7b22410f9bf23864739e21a733efe9ae07a196
+size 4876059416

model-00014-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f7db9abed0299390255ff1c8e89b44d0347440256e4e8e46a3e94f6d7575a51
+size 3381725320

model.safetensors.index.json

@@ -0,0 +1,1168 @@
+{
+  "metadata": {
+    "total_size": 66905436672
+  },
+  "weight_map": {
+    "lm_head.weight": "model-00014-of-00014.safetensors",
+    "model.embed_tokens.weight": "model-00001-of-00014.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00001-of-00014.safetensors",
+    "model.layers.0.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.0.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00014.safetensors",
+    "model.layers.0.self_attn.k_proj.bias": "model-00001-of-00014.safetensors",
+    "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.0.self_attn.q_proj.bias": "model-00001-of-00014.safetensors",
+    "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.0.self_attn.v_proj.bias": "model-00001-of-00014.safetensors",
+    "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00014.safetensors",
+    "model.layers.1.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.1.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00014.safetensors",
+    "model.layers.1.self_attn.k_proj.bias": "model-00001-of-00014.safetensors",
+    "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.1.self_attn.q_proj.bias": "model-00001-of-00014.safetensors",
+    "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.1.self_attn.v_proj.bias": "model-00001-of-00014.safetensors",
+    "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.10.mlp.down_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.10.mlp.gate_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.10.mlp.up_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.10.self_attn.k_proj.bias": "model-00003-of-00014.safetensors",
+    "model.layers.10.self_attn.k_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.10.self_attn.q_proj.bias": "model-00003-of-00014.safetensors",
+    "model.layers.10.self_attn.q_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.10.self_attn.v_proj.bias": "model-00003-of-00014.safetensors",
+    "model.layers.10.self_attn.v_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.11.mlp.down_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.11.mlp.gate_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.11.mlp.up_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.11.self_attn.k_proj.bias": "model-00003-of-00014.safetensors",
+    "model.layers.11.self_attn.k_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.11.self_attn.q_proj.bias": "model-00003-of-00014.safetensors",
+    "model.layers.11.self_attn.q_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.11.self_attn.v_proj.bias": "model-00003-of-00014.safetensors",
+    "model.layers.11.self_attn.v_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.12.mlp.down_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.12.mlp.gate_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.12.mlp.up_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.12.self_attn.k_proj.bias": "model-00003-of-00014.safetensors",
+    "model.layers.12.self_attn.k_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.12.self_attn.q_proj.bias": "model-00003-of-00014.safetensors",
+    "model.layers.12.self_attn.q_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.12.self_attn.v_proj.bias": "model-00003-of-00014.safetensors",
+    "model.layers.12.self_attn.v_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.13.mlp.down_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.13.mlp.gate_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.13.mlp.up_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.13.self_attn.k_proj.bias": "model-00004-of-00014.safetensors",
+    "model.layers.13.self_attn.k_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.13.self_attn.q_proj.bias": "model-00004-of-00014.safetensors",
+    "model.layers.13.self_attn.q_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.13.self_attn.v_proj.bias": "model-00004-of-00014.safetensors",
+    "model.layers.13.self_attn.v_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.14.mlp.down_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.14.mlp.gate_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.14.mlp.up_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.14.self_attn.k_proj.bias": "model-00004-of-00014.safetensors",
+    "model.layers.14.self_attn.k_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.14.self_attn.q_proj.bias": "model-00004-of-00014.safetensors",
+    "model.layers.14.self_attn.q_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.14.self_attn.v_proj.bias": "model-00004-of-00014.safetensors",
+    "model.layers.14.self_attn.v_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.15.mlp.down_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.15.mlp.gate_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.15.mlp.up_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.15.self_attn.k_proj.bias": "model-00004-of-00014.safetensors",
+    "model.layers.15.self_attn.k_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.15.self_attn.q_proj.bias": "model-00004-of-00014.safetensors",
+    "model.layers.15.self_attn.q_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.15.self_attn.v_proj.bias": "model-00004-of-00014.safetensors",
+    "model.layers.15.self_attn.v_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.16.mlp.down_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.16.mlp.gate_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.16.mlp.up_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00004-of-00014.safetensors",
+    "model.layers.16.self_attn.k_proj.bias": "model-00004-of-00014.safetensors",
+    "model.layers.16.self_attn.k_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.16.self_attn.q_proj.bias": "model-00004-of-00014.safetensors",
+    "model.layers.16.self_attn.q_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.16.self_attn.v_proj.bias": "model-00004-of-00014.safetensors",
+    "model.layers.16.self_attn.v_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.17.mlp.down_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.17.mlp.gate_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.17.mlp.up_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.17.self_attn.k_proj.bias": "model-00004-of-00014.safetensors",
+    "model.layers.17.self_attn.k_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.17.self_attn.q_proj.bias": "model-00004-of-00014.safetensors",
+    "model.layers.17.self_attn.q_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.17.self_attn.v_proj.bias": "model-00004-of-00014.safetensors",
+    "model.layers.17.self_attn.v_proj.weight": "model-00004-of-00014.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.18.mlp.down_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.18.mlp.gate_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.18.mlp.up_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.18.self_attn.k_proj.bias": "model-00005-of-00014.safetensors",
+    "model.layers.18.self_attn.k_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.18.self_attn.q_proj.bias": "model-00005-of-00014.safetensors",
+    "model.layers.18.self_attn.q_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.18.self_attn.v_proj.bias": "model-00005-of-00014.safetensors",
+    "model.layers.18.self_attn.v_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.19.mlp.down_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.19.mlp.gate_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.19.mlp.up_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.19.self_attn.k_proj.bias": "model-00005-of-00014.safetensors",
+    "model.layers.19.self_attn.k_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.19.self_attn.q_proj.bias": "model-00005-of-00014.safetensors",
+    "model.layers.19.self_attn.q_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.19.self_attn.v_proj.bias": "model-00005-of-00014.safetensors",
+    "model.layers.19.self_attn.v_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.2.mlp.down_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.2.mlp.gate_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.2.mlp.up_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.2.self_attn.k_proj.bias": "model-00001-of-00014.safetensors",
+    "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.2.self_attn.q_proj.bias": "model-00001-of-00014.safetensors",
+    "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.2.self_attn.v_proj.bias": "model-00001-of-00014.safetensors",
+    "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00014.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.20.mlp.down_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.20.mlp.gate_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.20.mlp.up_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.20.self_attn.k_proj.bias": "model-00005-of-00014.safetensors",
+    "model.layers.20.self_attn.k_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.20.self_attn.q_proj.bias": "model-00005-of-00014.safetensors",
+    "model.layers.20.self_attn.q_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.20.self_attn.v_proj.bias": "model-00005-of-00014.safetensors",
+    "model.layers.20.self_attn.v_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.21.mlp.down_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.21.mlp.gate_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.21.mlp.up_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00005-of-00014.safetensors",
+    "model.layers.21.self_attn.k_proj.bias": "model-00005-of-00014.safetensors",
+    "model.layers.21.self_attn.k_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.21.self_attn.q_proj.bias": "model-00005-of-00014.safetensors",
+    "model.layers.21.self_attn.q_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.21.self_attn.v_proj.bias": "model-00005-of-00014.safetensors",
+    "model.layers.21.self_attn.v_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.22.mlp.down_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.22.mlp.gate_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.22.mlp.up_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.22.self_attn.k_proj.bias": "model-00005-of-00014.safetensors",
+    "model.layers.22.self_attn.k_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.22.self_attn.q_proj.bias": "model-00005-of-00014.safetensors",
+    "model.layers.22.self_attn.q_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.22.self_attn.v_proj.bias": "model-00005-of-00014.safetensors",
+    "model.layers.22.self_attn.v_proj.weight": "model-00005-of-00014.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.23.mlp.down_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.23.mlp.gate_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.23.mlp.up_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.23.self_attn.k_proj.bias": "model-00006-of-00014.safetensors",
+    "model.layers.23.self_attn.k_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.23.self_attn.q_proj.bias": "model-00006-of-00014.safetensors",
+    "model.layers.23.self_attn.q_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.23.self_attn.v_proj.bias": "model-00006-of-00014.safetensors",
+    "model.layers.23.self_attn.v_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.24.mlp.down_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.24.mlp.gate_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.24.mlp.up_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.24.self_attn.k_proj.bias": "model-00006-of-00014.safetensors",
+    "model.layers.24.self_attn.k_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.24.self_attn.o_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.24.self_attn.q_proj.bias": "model-00006-of-00014.safetensors",
+    "model.layers.24.self_attn.q_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.24.self_attn.v_proj.bias": "model-00006-of-00014.safetensors",
+    "model.layers.24.self_attn.v_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.25.mlp.down_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.25.mlp.gate_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.25.mlp.up_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.25.self_attn.k_proj.bias": "model-00006-of-00014.safetensors",
+    "model.layers.25.self_attn.k_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.25.self_attn.q_proj.bias": "model-00006-of-00014.safetensors",
+    "model.layers.25.self_attn.q_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.25.self_attn.v_proj.bias": "model-00006-of-00014.safetensors",
+    "model.layers.25.self_attn.v_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.26.mlp.down_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.26.mlp.gate_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.26.mlp.up_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "model-00006-of-00014.safetensors",
+    "model.layers.26.self_attn.k_proj.bias": "model-00006-of-00014.safetensors",
+    "model.layers.26.self_attn.k_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.26.self_attn.o_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.26.self_attn.q_proj.bias": "model-00006-of-00014.safetensors",
+    "model.layers.26.self_attn.q_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.26.self_attn.v_proj.bias": "model-00006-of-00014.safetensors",
+    "model.layers.26.self_attn.v_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.27.mlp.down_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.27.mlp.gate_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.27.mlp.up_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.27.self_attn.k_proj.bias": "model-00006-of-00014.safetensors",
+    "model.layers.27.self_attn.k_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.27.self_attn.o_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.27.self_attn.q_proj.bias": "model-00006-of-00014.safetensors",
+    "model.layers.27.self_attn.q_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.27.self_attn.v_proj.bias": "model-00006-of-00014.safetensors",
+    "model.layers.27.self_attn.v_proj.weight": "model-00006-of-00014.safetensors",
+    "model.layers.28.input_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.28.mlp.down_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.28.mlp.gate_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.28.mlp.up_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.28.post_attention_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.28.self_attn.k_proj.bias": "model-00007-of-00014.safetensors",
+    "model.layers.28.self_attn.k_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.28.self_attn.o_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.28.self_attn.q_proj.bias": "model-00007-of-00014.safetensors",
+    "model.layers.28.self_attn.q_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.28.self_attn.v_proj.bias": "model-00007-of-00014.safetensors",
+    "model.layers.28.self_attn.v_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.29.input_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.29.mlp.down_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.29.mlp.gate_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.29.mlp.up_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.29.post_attention_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.29.self_attn.k_proj.bias": "model-00007-of-00014.safetensors",
+    "model.layers.29.self_attn.k_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.29.self_attn.o_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.29.self_attn.q_proj.bias": "model-00007-of-00014.safetensors",
+    "model.layers.29.self_attn.q_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.29.self_attn.v_proj.bias": "model-00007-of-00014.safetensors",
+    "model.layers.29.self_attn.v_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.3.mlp.down_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.3.mlp.gate_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.3.mlp.up_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.3.self_attn.k_proj.bias": "model-00002-of-00014.safetensors",
+    "model.layers.3.self_attn.k_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.3.self_attn.q_proj.bias": "model-00002-of-00014.safetensors",
+    "model.layers.3.self_attn.q_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.3.self_attn.v_proj.bias": "model-00002-of-00014.safetensors",
+    "model.layers.3.self_attn.v_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.30.input_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.30.mlp.down_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.30.mlp.gate_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.30.mlp.up_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.30.post_attention_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.30.self_attn.k_proj.bias": "model-00007-of-00014.safetensors",
+    "model.layers.30.self_attn.k_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.30.self_attn.o_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.30.self_attn.q_proj.bias": "model-00007-of-00014.safetensors",
+    "model.layers.30.self_attn.q_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.30.self_attn.v_proj.bias": "model-00007-of-00014.safetensors",
+    "model.layers.30.self_attn.v_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.31.input_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.31.mlp.down_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.31.mlp.gate_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.31.mlp.up_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.31.post_attention_layernorm.weight": "model-00007-of-00014.safetensors",
+    "model.layers.31.self_attn.k_proj.bias": "model-00007-of-00014.safetensors",
+    "model.layers.31.self_attn.k_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.31.self_attn.o_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.31.self_attn.q_proj.bias": "model-00007-of-00014.safetensors",
+    "model.layers.31.self_attn.q_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.31.self_attn.v_proj.bias": "model-00007-of-00014.safetensors",
+    "model.layers.31.self_attn.v_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.32.input_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.32.mlp.down_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.32.mlp.gate_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.32.mlp.up_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.32.post_attention_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.32.self_attn.k_proj.bias": "model-00007-of-00014.safetensors",
+    "model.layers.32.self_attn.k_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.32.self_attn.o_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.32.self_attn.q_proj.bias": "model-00007-of-00014.safetensors",
+    "model.layers.32.self_attn.q_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.32.self_attn.v_proj.bias": "model-00007-of-00014.safetensors",
+    "model.layers.32.self_attn.v_proj.weight": "model-00007-of-00014.safetensors",
+    "model.layers.33.input_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.33.mlp.down_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.33.mlp.gate_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.33.mlp.up_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.33.post_attention_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.33.self_attn.k_proj.bias": "model-00008-of-00014.safetensors",
+    "model.layers.33.self_attn.k_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.33.self_attn.o_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.33.self_attn.q_proj.bias": "model-00008-of-00014.safetensors",
+    "model.layers.33.self_attn.q_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.33.self_attn.v_proj.bias": "model-00008-of-00014.safetensors",
+    "model.layers.33.self_attn.v_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.34.input_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.34.mlp.down_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.34.mlp.gate_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.34.mlp.up_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.34.post_attention_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.34.self_attn.k_proj.bias": "model-00008-of-00014.safetensors",
+    "model.layers.34.self_attn.k_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.34.self_attn.o_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.34.self_attn.q_proj.bias": "model-00008-of-00014.safetensors",
+    "model.layers.34.self_attn.q_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.34.self_attn.v_proj.bias": "model-00008-of-00014.safetensors",
+    "model.layers.34.self_attn.v_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.35.input_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.35.mlp.down_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.35.mlp.gate_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.35.mlp.up_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.35.post_attention_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.35.self_attn.k_proj.bias": "model-00008-of-00014.safetensors",
+    "model.layers.35.self_attn.k_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.35.self_attn.o_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.35.self_attn.q_proj.bias": "model-00008-of-00014.safetensors",
+    "model.layers.35.self_attn.q_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.35.self_attn.v_proj.bias": "model-00008-of-00014.safetensors",
+    "model.layers.35.self_attn.v_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.36.input_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.36.mlp.down_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.36.mlp.gate_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.36.mlp.up_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.36.post_attention_layernorm.weight": "model-00008-of-00014.safetensors",
+    "model.layers.36.self_attn.k_proj.bias": "model-00008-of-00014.safetensors",
+    "model.layers.36.self_attn.k_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.36.self_attn.o_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.36.self_attn.q_proj.bias": "model-00008-of-00014.safetensors",
+    "model.layers.36.self_attn.q_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.36.self_attn.v_proj.bias": "model-00008-of-00014.safetensors",
+    "model.layers.36.self_attn.v_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.37.input_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.37.mlp.down_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.37.mlp.gate_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.37.mlp.up_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.37.post_attention_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.37.self_attn.k_proj.bias": "model-00008-of-00014.safetensors",
+    "model.layers.37.self_attn.k_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.37.self_attn.o_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.37.self_attn.q_proj.bias": "model-00008-of-00014.safetensors",
+    "model.layers.37.self_attn.q_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.37.self_attn.v_proj.bias": "model-00008-of-00014.safetensors",
+    "model.layers.37.self_attn.v_proj.weight": "model-00008-of-00014.safetensors",
+    "model.layers.38.input_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.38.mlp.down_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.38.mlp.gate_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.38.mlp.up_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.38.post_attention_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.38.self_attn.k_proj.bias": "model-00009-of-00014.safetensors",
+    "model.layers.38.self_attn.k_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.38.self_attn.o_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.38.self_attn.q_proj.bias": "model-00009-of-00014.safetensors",
+    "model.layers.38.self_attn.q_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.38.self_attn.v_proj.bias": "model-00009-of-00014.safetensors",
+    "model.layers.38.self_attn.v_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.39.input_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.39.mlp.down_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.39.mlp.gate_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.39.mlp.up_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.39.post_attention_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.39.self_attn.k_proj.bias": "model-00009-of-00014.safetensors",
+    "model.layers.39.self_attn.k_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.39.self_attn.o_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.39.self_attn.q_proj.bias": "model-00009-of-00014.safetensors",
+    "model.layers.39.self_attn.q_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.39.self_attn.v_proj.bias": "model-00009-of-00014.safetensors",
+    "model.layers.39.self_attn.v_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.4.mlp.down_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.4.mlp.gate_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.4.mlp.up_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.4.self_attn.k_proj.bias": "model-00002-of-00014.safetensors",
+    "model.layers.4.self_attn.k_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.4.self_attn.q_proj.bias": "model-00002-of-00014.safetensors",
+    "model.layers.4.self_attn.q_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.4.self_attn.v_proj.bias": "model-00002-of-00014.safetensors",
+    "model.layers.4.self_attn.v_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.40.input_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.40.mlp.down_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.40.mlp.gate_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.40.mlp.up_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.40.post_attention_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.40.self_attn.k_proj.bias": "model-00009-of-00014.safetensors",
+    "model.layers.40.self_attn.k_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.40.self_attn.o_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.40.self_attn.q_proj.bias": "model-00009-of-00014.safetensors",
+    "model.layers.40.self_attn.q_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.40.self_attn.v_proj.bias": "model-00009-of-00014.safetensors",
+    "model.layers.40.self_attn.v_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.41.input_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.41.mlp.down_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.41.mlp.gate_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.41.mlp.up_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.41.post_attention_layernorm.weight": "model-00009-of-00014.safetensors",
+    "model.layers.41.self_attn.k_proj.bias": "model-00009-of-00014.safetensors",
+    "model.layers.41.self_attn.k_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.41.self_attn.o_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.41.self_attn.q_proj.bias": "model-00009-of-00014.safetensors",
+    "model.layers.41.self_attn.q_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.41.self_attn.v_proj.bias": "model-00009-of-00014.safetensors",
+    "model.layers.41.self_attn.v_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.42.input_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.42.mlp.down_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.42.mlp.gate_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.42.mlp.up_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.42.post_attention_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.42.self_attn.k_proj.bias": "model-00009-of-00014.safetensors",
+    "model.layers.42.self_attn.k_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.42.self_attn.o_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.42.self_attn.q_proj.bias": "model-00009-of-00014.safetensors",
+    "model.layers.42.self_attn.q_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.42.self_attn.v_proj.bias": "model-00009-of-00014.safetensors",
+    "model.layers.42.self_attn.v_proj.weight": "model-00009-of-00014.safetensors",
+    "model.layers.43.input_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.43.mlp.down_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.43.mlp.gate_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.43.mlp.up_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.43.post_attention_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.43.self_attn.k_proj.bias": "model-00010-of-00014.safetensors",
+    "model.layers.43.self_attn.k_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.43.self_attn.o_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.43.self_attn.q_proj.bias": "model-00010-of-00014.safetensors",
+    "model.layers.43.self_attn.q_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.43.self_attn.v_proj.bias": "model-00010-of-00014.safetensors",
+    "model.layers.43.self_attn.v_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.44.input_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.44.mlp.down_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.44.mlp.gate_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.44.mlp.up_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.44.post_attention_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.44.self_attn.k_proj.bias": "model-00010-of-00014.safetensors",
+    "model.layers.44.self_attn.k_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.44.self_attn.o_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.44.self_attn.q_proj.bias": "model-00010-of-00014.safetensors",
+    "model.layers.44.self_attn.q_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.44.self_attn.v_proj.bias": "model-00010-of-00014.safetensors",
+    "model.layers.44.self_attn.v_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.45.input_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.45.mlp.down_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.45.mlp.gate_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.45.mlp.up_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.45.post_attention_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.45.self_attn.k_proj.bias": "model-00010-of-00014.safetensors",
+    "model.layers.45.self_attn.k_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.45.self_attn.o_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.45.self_attn.q_proj.bias": "model-00010-of-00014.safetensors",
+    "model.layers.45.self_attn.q_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.45.self_attn.v_proj.bias": "model-00010-of-00014.safetensors",
+    "model.layers.45.self_attn.v_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.46.input_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.46.mlp.down_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.46.mlp.gate_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.46.mlp.up_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.46.post_attention_layernorm.weight": "model-00010-of-00014.safetensors",
+    "model.layers.46.self_attn.k_proj.bias": "model-00010-of-00014.safetensors",
+    "model.layers.46.self_attn.k_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.46.self_attn.o_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.46.self_attn.q_proj.bias": "model-00010-of-00014.safetensors",
+    "model.layers.46.self_attn.q_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.46.self_attn.v_proj.bias": "model-00010-of-00014.safetensors",
+    "model.layers.46.self_attn.v_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.47.input_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.47.mlp.down_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.47.mlp.gate_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.47.mlp.up_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.47.post_attention_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.47.self_attn.k_proj.bias": "model-00010-of-00014.safetensors",
+    "model.layers.47.self_attn.k_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.47.self_attn.o_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.47.self_attn.q_proj.bias": "model-00010-of-00014.safetensors",
+    "model.layers.47.self_attn.q_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.47.self_attn.v_proj.bias": "model-00010-of-00014.safetensors",
+    "model.layers.47.self_attn.v_proj.weight": "model-00010-of-00014.safetensors",
+    "model.layers.48.input_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.48.mlp.down_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.48.mlp.gate_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.48.mlp.up_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.48.post_attention_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.48.self_attn.k_proj.bias": "model-00011-of-00014.safetensors",
+    "model.layers.48.self_attn.k_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.48.self_attn.o_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.48.self_attn.q_proj.bias": "model-00011-of-00014.safetensors",
+    "model.layers.48.self_attn.q_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.48.self_attn.v_proj.bias": "model-00011-of-00014.safetensors",
+    "model.layers.48.self_attn.v_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.49.input_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.49.mlp.down_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.49.mlp.gate_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.49.mlp.up_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.49.post_attention_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.49.self_attn.k_proj.bias": "model-00011-of-00014.safetensors",
+    "model.layers.49.self_attn.k_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.49.self_attn.o_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.49.self_attn.q_proj.bias": "model-00011-of-00014.safetensors",
+    "model.layers.49.self_attn.q_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.49.self_attn.v_proj.bias": "model-00011-of-00014.safetensors",
+    "model.layers.49.self_attn.v_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.5.mlp.down_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.5.mlp.gate_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.5.mlp.up_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.5.self_attn.k_proj.bias": "model-00002-of-00014.safetensors",
+    "model.layers.5.self_attn.k_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.5.self_attn.q_proj.bias": "model-00002-of-00014.safetensors",
+    "model.layers.5.self_attn.q_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.5.self_attn.v_proj.bias": "model-00002-of-00014.safetensors",
+    "model.layers.5.self_attn.v_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.50.input_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.50.mlp.down_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.50.mlp.gate_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.50.mlp.up_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.50.post_attention_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.50.self_attn.k_proj.bias": "model-00011-of-00014.safetensors",
+    "model.layers.50.self_attn.k_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.50.self_attn.o_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.50.self_attn.q_proj.bias": "model-00011-of-00014.safetensors",
+    "model.layers.50.self_attn.q_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.50.self_attn.v_proj.bias": "model-00011-of-00014.safetensors",
+    "model.layers.50.self_attn.v_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.51.input_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.51.mlp.down_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.51.mlp.gate_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.51.mlp.up_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.51.post_attention_layernorm.weight": "model-00011-of-00014.safetensors",
+    "model.layers.51.self_attn.k_proj.bias": "model-00011-of-00014.safetensors",
+    "model.layers.51.self_attn.k_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.51.self_attn.o_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.51.self_attn.q_proj.bias": "model-00011-of-00014.safetensors",
+    "model.layers.51.self_attn.q_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.51.self_attn.v_proj.bias": "model-00011-of-00014.safetensors",
+    "model.layers.51.self_attn.v_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.52.input_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.52.mlp.down_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.52.mlp.gate_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.52.mlp.up_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.52.post_attention_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.52.self_attn.k_proj.bias": "model-00011-of-00014.safetensors",
+    "model.layers.52.self_attn.k_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.52.self_attn.o_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.52.self_attn.q_proj.bias": "model-00011-of-00014.safetensors",
+    "model.layers.52.self_attn.q_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.52.self_attn.v_proj.bias": "model-00011-of-00014.safetensors",
+    "model.layers.52.self_attn.v_proj.weight": "model-00011-of-00014.safetensors",
+    "model.layers.53.input_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.53.mlp.down_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.53.mlp.gate_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.53.mlp.up_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.53.post_attention_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.53.self_attn.k_proj.bias": "model-00012-of-00014.safetensors",
+    "model.layers.53.self_attn.k_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.53.self_attn.o_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.53.self_attn.q_proj.bias": "model-00012-of-00014.safetensors",
+    "model.layers.53.self_attn.q_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.53.self_attn.v_proj.bias": "model-00012-of-00014.safetensors",
+    "model.layers.53.self_attn.v_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.54.input_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.54.mlp.down_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.54.mlp.gate_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.54.mlp.up_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.54.post_attention_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.54.self_attn.k_proj.bias": "model-00012-of-00014.safetensors",
+    "model.layers.54.self_attn.k_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.54.self_attn.o_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.54.self_attn.q_proj.bias": "model-00012-of-00014.safetensors",
+    "model.layers.54.self_attn.q_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.54.self_attn.v_proj.bias": "model-00012-of-00014.safetensors",
+    "model.layers.54.self_attn.v_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.55.input_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.55.mlp.down_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.55.mlp.gate_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.55.mlp.up_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.55.post_attention_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.55.self_attn.k_proj.bias": "model-00012-of-00014.safetensors",
+    "model.layers.55.self_attn.k_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.55.self_attn.o_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.55.self_attn.q_proj.bias": "model-00012-of-00014.safetensors",
+    "model.layers.55.self_attn.q_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.55.self_attn.v_proj.bias": "model-00012-of-00014.safetensors",
+    "model.layers.55.self_attn.v_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.56.input_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.56.mlp.down_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.56.mlp.gate_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.56.mlp.up_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.56.post_attention_layernorm.weight": "model-00012-of-00014.safetensors",
+    "model.layers.56.self_attn.k_proj.bias": "model-00012-of-00014.safetensors",
+    "model.layers.56.self_attn.k_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.56.self_attn.o_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.56.self_attn.q_proj.bias": "model-00012-of-00014.safetensors",
+    "model.layers.56.self_attn.q_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.56.self_attn.v_proj.bias": "model-00012-of-00014.safetensors",
+    "model.layers.56.self_attn.v_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.57.input_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.57.mlp.down_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.57.mlp.gate_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.57.mlp.up_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.57.post_attention_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.57.self_attn.k_proj.bias": "model-00012-of-00014.safetensors",
+    "model.layers.57.self_attn.k_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.57.self_attn.o_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.57.self_attn.q_proj.bias": "model-00012-of-00014.safetensors",
+    "model.layers.57.self_attn.q_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.57.self_attn.v_proj.bias": "model-00012-of-00014.safetensors",
+    "model.layers.57.self_attn.v_proj.weight": "model-00012-of-00014.safetensors",
+    "model.layers.58.input_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.58.mlp.down_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.58.mlp.gate_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.58.mlp.up_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.58.post_attention_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.58.self_attn.k_proj.bias": "model-00013-of-00014.safetensors",
+    "model.layers.58.self_attn.k_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.58.self_attn.o_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.58.self_attn.q_proj.bias": "model-00013-of-00014.safetensors",
+    "model.layers.58.self_attn.q_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.58.self_attn.v_proj.bias": "model-00013-of-00014.safetensors",
+    "model.layers.58.self_attn.v_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.59.input_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.59.mlp.down_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.59.mlp.gate_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.59.mlp.up_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.59.post_attention_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.59.self_attn.k_proj.bias": "model-00013-of-00014.safetensors",
+    "model.layers.59.self_attn.k_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.59.self_attn.o_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.59.self_attn.q_proj.bias": "model-00013-of-00014.safetensors",
+    "model.layers.59.self_attn.q_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.59.self_attn.v_proj.bias": "model-00013-of-00014.safetensors",
+    "model.layers.59.self_attn.v_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.6.mlp.down_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.6.mlp.gate_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.6.mlp.up_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00002-of-00014.safetensors",
+    "model.layers.6.self_attn.k_proj.bias": "model-00002-of-00014.safetensors",
+    "model.layers.6.self_attn.k_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.6.self_attn.q_proj.bias": "model-00002-of-00014.safetensors",
+    "model.layers.6.self_attn.q_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.6.self_attn.v_proj.bias": "model-00002-of-00014.safetensors",
+    "model.layers.6.self_attn.v_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.60.input_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.60.mlp.down_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.60.mlp.gate_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.60.mlp.up_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.60.post_attention_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.60.self_attn.k_proj.bias": "model-00013-of-00014.safetensors",
+    "model.layers.60.self_attn.k_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.60.self_attn.o_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.60.self_attn.q_proj.bias": "model-00013-of-00014.safetensors",
+    "model.layers.60.self_attn.q_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.60.self_attn.v_proj.bias": "model-00013-of-00014.safetensors",
+    "model.layers.60.self_attn.v_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.61.input_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.61.mlp.down_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.61.mlp.gate_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.61.mlp.up_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.61.post_attention_layernorm.weight": "model-00013-of-00014.safetensors",
+    "model.layers.61.self_attn.k_proj.bias": "model-00013-of-00014.safetensors",
+    "model.layers.61.self_attn.k_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.61.self_attn.o_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.61.self_attn.q_proj.bias": "model-00013-of-00014.safetensors",
+    "model.layers.61.self_attn.q_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.61.self_attn.v_proj.bias": "model-00013-of-00014.safetensors",
+    "model.layers.61.self_attn.v_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.62.input_layernorm.weight": "model-00014-of-00014.safetensors",
+    "model.layers.62.mlp.down_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.62.mlp.gate_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.62.mlp.up_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.62.post_attention_layernorm.weight": "model-00014-of-00014.safetensors",
+    "model.layers.62.self_attn.k_proj.bias": "model-00013-of-00014.safetensors",
+    "model.layers.62.self_attn.k_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.62.self_attn.o_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.62.self_attn.q_proj.bias": "model-00013-of-00014.safetensors",
+    "model.layers.62.self_attn.q_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.62.self_attn.v_proj.bias": "model-00013-of-00014.safetensors",
+    "model.layers.62.self_attn.v_proj.weight": "model-00013-of-00014.safetensors",
+    "model.layers.63.input_layernorm.weight": "model-00014-of-00014.safetensors",
+    "model.layers.63.mlp.down_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.63.mlp.gate_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.63.mlp.up_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.63.post_attention_layernorm.weight": "model-00014-of-00014.safetensors",
+    "model.layers.63.self_attn.k_proj.bias": "model-00014-of-00014.safetensors",
+    "model.layers.63.self_attn.k_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.63.self_attn.o_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.63.self_attn.q_proj.bias": "model-00014-of-00014.safetensors",
+    "model.layers.63.self_attn.q_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.63.self_attn.v_proj.bias": "model-00014-of-00014.safetensors",
+    "model.layers.63.self_attn.v_proj.weight": "model-00014-of-00014.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.7.mlp.down_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.7.mlp.gate_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.7.mlp.up_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.7.self_attn.k_proj.bias": "model-00002-of-00014.safetensors",
+    "model.layers.7.self_attn.k_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.7.self_attn.q_proj.bias": "model-00002-of-00014.safetensors",
+    "model.layers.7.self_attn.q_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.7.self_attn.v_proj.bias": "model-00002-of-00014.safetensors",
+    "model.layers.7.self_attn.v_proj.weight": "model-00002-of-00014.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.8.mlp.down_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.8.mlp.gate_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.8.mlp.up_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.8.self_attn.k_proj.bias": "model-00003-of-00014.safetensors",
+    "model.layers.8.self_attn.k_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.8.self_attn.q_proj.bias": "model-00003-of-00014.safetensors",
+    "model.layers.8.self_attn.q_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.8.self_attn.v_proj.bias": "model-00003-of-00014.safetensors",
+    "model.layers.8.self_attn.v_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.9.mlp.down_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.9.mlp.gate_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.9.mlp.up_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00003-of-00014.safetensors",
+    "model.layers.9.self_attn.k_proj.bias": "model-00003-of-00014.safetensors",
+    "model.layers.9.self_attn.k_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.9.self_attn.q_proj.bias": "model-00003-of-00014.safetensors",
+    "model.layers.9.self_attn.q_proj.weight": "model-00003-of-00014.safetensors",
+    "model.layers.9.self_attn.v_proj.bias": "model-00003-of-00014.safetensors",
+    "model.layers.9.self_attn.v_proj.weight": "model-00003-of-00014.safetensors",
+    "model.norm.weight": "model-00014-of-00014.safetensors",
+    "visual.blocks.0.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.0.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.0.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.0.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.0.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.0.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.0.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.0.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.0.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.0.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.0.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.0.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.1.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.1.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.1.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.1.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.1.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.1.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.1.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.1.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.1.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.1.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.1.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.1.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.10.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.10.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.10.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.10.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.10.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.10.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.10.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.10.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.10.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.10.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.10.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.10.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.11.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.11.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.11.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.11.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.11.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.11.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.11.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.11.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.11.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.11.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.11.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.11.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.12.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.12.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.12.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.12.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.12.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.12.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.12.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.12.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.12.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.12.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.12.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.12.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.13.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.13.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.13.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.13.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.13.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.13.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.13.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.13.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.13.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.13.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.13.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.13.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.14.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.14.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.14.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.14.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.14.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.14.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.14.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.14.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.14.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.14.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.14.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.14.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.15.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.15.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.15.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.15.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.15.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.15.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.15.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.15.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.15.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.15.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.15.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.15.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.16.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.16.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.16.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.16.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.16.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.16.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.16.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.16.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.16.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.16.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.16.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.16.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.17.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.17.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.17.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.17.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.17.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.17.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.17.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.17.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.17.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.17.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.17.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.17.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.18.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.18.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.18.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.18.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.18.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.18.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.18.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.18.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.18.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.18.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.18.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.18.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.19.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.19.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.19.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.19.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.19.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.19.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.19.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.19.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.19.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.19.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.19.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.19.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.2.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.2.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.2.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.2.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.2.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.2.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.2.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.2.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.2.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.2.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.2.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.2.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.20.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.20.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.20.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.20.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.20.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.20.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.20.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.20.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.20.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.20.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.20.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.20.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.21.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.21.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.21.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.21.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.21.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.21.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.21.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.21.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.21.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.21.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.21.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.21.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.22.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.22.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.22.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.22.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.22.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.22.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.22.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.22.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.22.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.22.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.22.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.22.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.23.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.23.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.23.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.23.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.23.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.23.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.23.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.23.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.23.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.23.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.23.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.23.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.24.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.24.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.24.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.24.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.24.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.24.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.24.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.24.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.24.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.24.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.24.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.24.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.25.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.25.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.25.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.25.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.25.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.25.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.25.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.25.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.25.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.25.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.25.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.25.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.26.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.26.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.26.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.26.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.26.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.26.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.26.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.26.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.26.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.26.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.26.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.26.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.27.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.27.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.27.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.27.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.27.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.27.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.27.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.27.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.27.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.27.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.27.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.27.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.28.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.28.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.28.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.28.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.28.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.28.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.28.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.28.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.28.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.28.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.28.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.28.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.29.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.29.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.29.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.29.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.29.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.29.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.29.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.29.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.29.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.29.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.29.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.29.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.3.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.3.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.3.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.3.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.3.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.3.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.3.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.3.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.3.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.3.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.3.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.3.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.30.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.30.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.30.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.30.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.30.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.30.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.30.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.30.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.30.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.30.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.30.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.30.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.31.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.31.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.31.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.31.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.31.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.31.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.31.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.31.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.31.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.31.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.31.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.31.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.4.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.4.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.4.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.4.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.4.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.4.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.4.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.4.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.4.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.4.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.4.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.4.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.5.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.5.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.5.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.5.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.5.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.5.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.5.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.5.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.5.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.5.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.5.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.5.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.6.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.6.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.6.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.6.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.6.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.6.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.6.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.6.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.6.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.6.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.6.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.6.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.7.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.7.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.7.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.7.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.7.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.7.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.7.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.7.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.7.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.7.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.7.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.7.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.8.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.8.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.8.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.8.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.8.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.8.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.8.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.8.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.8.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.8.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.8.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.8.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.9.attn.proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.9.attn.proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.9.attn.qkv.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.9.attn.qkv.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.9.mlp.down_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.9.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.9.mlp.gate_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.9.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.9.mlp.up_proj.bias": "model-00001-of-00014.safetensors",
+    "visual.blocks.9.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.9.norm1.weight": "model-00001-of-00014.safetensors",
+    "visual.blocks.9.norm2.weight": "model-00001-of-00014.safetensors",
+    "visual.merger.ln_q.weight": "model-00001-of-00014.safetensors",
+    "visual.merger.mlp.0.bias": "model-00001-of-00014.safetensors",
+    "visual.merger.mlp.0.weight": "model-00001-of-00014.safetensors",
+    "visual.merger.mlp.2.bias": "model-00001-of-00014.safetensors",
+    "visual.merger.mlp.2.weight": "model-00001-of-00014.safetensors",
+    "visual.patch_embed.proj.weight": "model-00001-of-00014.safetensors"
+  }
+}

preprocessor_config.json

@@ -0,0 +1,29 @@
+{
+  "do_convert_rgb": true,
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.48145466,
+    0.4578275,
+    0.40821073
+  ],
+  "image_processor_type": "Qwen2VLImageProcessor",
+  "image_std": [
+    0.26862954,
+    0.26130258,
+    0.27577711
+  ],
+  "max_pixels": 12845056,
+  "merge_size": 2,
+  "min_pixels": 3136,
+  "patch_size": 14,
+  "processor_class": "Qwen2_5_VLProcessor",
+  "resample": 3,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "longest_edge": 12845056,
+    "shortest_edge": 3136
+  },
+  "temporal_patch_size": 2
+}

runs/Jul16_01-57-58_dlc1kpv0nc8u9kz9-master-0/events.out.tfevents.1752603136.dlc1kpv0nc8u9kz9-master-0.36.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:939eda3d442a59773c948b75893389d4cce594a4fcedd276670978dc2502ec82
+size 9581

special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c5ae00e602b8860cbd784ba82a8aa14e8feecec692e7076590d014d7b7fdafa
+size 11421896

tokenizer_config.json

@@ -0,0 +1,210 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n{% endif %}<|im_start|>{{ message['role'] }}\n{% if message['content'] is string %}{{ message['content'] }}<|im_end|>\n{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>\n{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant\n{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "processor_class": "Qwen2_5_VLProcessor",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

train_results.json

@@ -0,0 +1,8 @@
+{
+    "epoch": 2.9533678756476682,
+    "total_flos": 887572825899008.0,
+    "train_loss": 0.6164978990952173,
+    "train_runtime": 16386.6242,
+    "train_samples_per_second": 1.127,
+    "train_steps_per_second": 0.009
+}

trainer_log.jsonl

@@ -0,0 +1,15 @@
+{"current_steps": 10, "total_steps": 144, "loss": 0.9129, "lr": 3e-06, "epoch": 0.20725388601036268, "percentage": 6.94, "elapsed_time": "0:16:44", "remaining_time": "3:44:23"}
+{"current_steps": 20, "total_steps": 144, "loss": 0.7274, "lr": 4.98814758473309e-06, "epoch": 0.41450777202072536, "percentage": 13.89, "elapsed_time": "0:33:16", "remaining_time": "3:26:18"}
+{"current_steps": 30, "total_steps": 144, "loss": 0.6513, "lr": 4.856095207084287e-06, "epoch": 0.6217616580310881, "percentage": 20.83, "elapsed_time": "0:50:27", "remaining_time": "3:11:43"}
+{"current_steps": 40, "total_steps": 144, "loss": 0.6373, "lr": 4.584994544724695e-06, "epoch": 0.8290155440414507, "percentage": 27.78, "elapsed_time": "1:09:11", "remaining_time": "2:59:54"}
+{"current_steps": 50, "total_steps": 144, "loss": 0.5683, "lr": 4.190844985952257e-06, "epoch": 1.0207253886010363, "percentage": 34.72, "elapsed_time": "1:34:03", "remaining_time": "2:56:50"}
+{"current_steps": 60, "total_steps": 144, "loss": 0.6012, "lr": 3.6969078233930235e-06, "epoch": 1.2279792746113989, "percentage": 41.67, "elapsed_time": "1:51:31", "remaining_time": "2:36:07"}
+{"current_steps": 70, "total_steps": 144, "loss": 0.5866, "lr": 3.132333455979202e-06, "epoch": 1.4352331606217616, "percentage": 48.61, "elapsed_time": "2:08:54", "remaining_time": "2:16:16"}
+{"current_steps": 80, "total_steps": 144, "loss": 0.5958, "lr": 2.530441037004994e-06, "epoch": 1.6424870466321244, "percentage": 55.56, "elapsed_time": "2:26:46", "remaining_time": "1:57:25"}
+{"current_steps": 90, "total_steps": 144, "loss": 0.5755, "lr": 1.9267520972587754e-06, "epoch": 1.849740932642487, "percentage": 62.5, "elapsed_time": "2:44:07", "remaining_time": "1:38:28"}
+{"current_steps": 100, "total_steps": 144, "loss": 0.5319, "lr": 1.3568941917384038e-06, "epoch": 2.0414507772020727, "percentage": 69.44, "elapsed_time": "3:08:52", "remaining_time": "1:23:06"}
+{"current_steps": 110, "total_steps": 144, "loss": 0.5579, "lr": 8.544982891981304e-07, "epoch": 2.2487046632124352, "percentage": 76.39, "elapsed_time": "3:26:41", "remaining_time": "1:03:53"}
+{"current_steps": 120, "total_steps": 144, "loss": 0.5555, "lr": 4.4921399305122264e-07, "epoch": 2.4559585492227978, "percentage": 83.33, "elapsed_time": "3:43:13", "remaining_time": "0:44:38"}
+{"current_steps": 130, "total_steps": 144, "loss": 0.5625, "lr": 1.649597281686302e-07, "epoch": 2.6632124352331608, "percentage": 90.28, "elapsed_time": "4:02:05", "remaining_time": "0:26:04"}
+{"current_steps": 140, "total_steps": 144, "loss": 0.5588, "lr": 1.851116126993402e-08, "epoch": 2.8704663212435233, "percentage": 97.22, "elapsed_time": "4:19:40", "remaining_time": "0:07:25"}
+{"current_steps": 144, "total_steps": 144, "epoch": 2.9533678756476682, "percentage": 100.0, "elapsed_time": "4:33:03", "remaining_time": "0:00:00"}

trainer_state.json

@@ -0,0 +1,141 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.9533678756476682,
+  "eval_steps": 500,
+  "global_step": 144,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.20725388601036268,
+      "grad_norm": 1.8093507425794213,
+      "learning_rate": 3e-06,
+      "loss": 0.9129,
+      "step": 10
+    },
+    {
+      "epoch": 0.41450777202072536,
+      "grad_norm": 0.7996825526135815,
+      "learning_rate": 4.98814758473309e-06,
+      "loss": 0.7274,
+      "step": 20
+    },
+    {
+      "epoch": 0.6217616580310881,
+      "grad_norm": 0.5373843764254267,
+      "learning_rate": 4.856095207084287e-06,
+      "loss": 0.6513,
+      "step": 30
+    },
+    {
+      "epoch": 0.8290155440414507,
+      "grad_norm": 0.4621175956467346,
+      "learning_rate": 4.584994544724695e-06,
+      "loss": 0.6373,
+      "step": 40
+    },
+    {
+      "epoch": 1.0207253886010363,
+      "grad_norm": 0.47882928868708474,
+      "learning_rate": 4.190844985952257e-06,
+      "loss": 0.5683,
+      "step": 50
+    },
+    {
+      "epoch": 1.2279792746113989,
+      "grad_norm": 0.4492344877710059,
+      "learning_rate": 3.6969078233930235e-06,
+      "loss": 0.6012,
+      "step": 60
+    },
+    {
+      "epoch": 1.4352331606217616,
+      "grad_norm": 0.4358797089748146,
+      "learning_rate": 3.132333455979202e-06,
+      "loss": 0.5866,
+      "step": 70
+    },
+    {
+      "epoch": 1.6424870466321244,
+      "grad_norm": 0.41894841115210807,
+      "learning_rate": 2.530441037004994e-06,
+      "loss": 0.5958,
+      "step": 80
+    },
+    {
+      "epoch": 1.849740932642487,
+      "grad_norm": 0.4253754435893472,
+      "learning_rate": 1.9267520972587754e-06,
+      "loss": 0.5755,
+      "step": 90
+    },
+    {
+      "epoch": 2.0414507772020727,
+      "grad_norm": 0.40951381645086515,
+      "learning_rate": 1.3568941917384038e-06,
+      "loss": 0.5319,
+      "step": 100
+    },
+    {
+      "epoch": 2.2487046632124352,
+      "grad_norm": 0.41726111462999993,
+      "learning_rate": 8.544982891981304e-07,
+      "loss": 0.5579,
+      "step": 110
+    },
+    {
+      "epoch": 2.4559585492227978,
+      "grad_norm": 0.4374441335411058,
+      "learning_rate": 4.4921399305122264e-07,
+      "loss": 0.5555,
+      "step": 120
+    },
+    {
+      "epoch": 2.6632124352331608,
+      "grad_norm": 0.4146727228301452,
+      "learning_rate": 1.649597281686302e-07,
+      "loss": 0.5625,
+      "step": 130
+    },
+    {
+      "epoch": 2.8704663212435233,
+      "grad_norm": 0.4451658295553321,
+      "learning_rate": 1.851116126993402e-08,
+      "loss": 0.5588,
+      "step": 140
+    },
+    {
+      "epoch": 2.9533678756476682,
+      "step": 144,
+      "total_flos": 887572825899008.0,
+      "train_loss": 0.6164978990952173,
+      "train_runtime": 16386.6242,
+      "train_samples_per_second": 1.127,
+      "train_steps_per_second": 0.009
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 144,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 887572825899008.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:970688ab55c0c587129af5a0ddfb771ba3ce02cd4ca79d5810fc009bd1cbbe75
+size 7608