Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,200 @@
----
-license: apache-2.0
----
+---
+tags:
+- unsloth
+base_model:
+- Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8
+library_name: transformers
+license: apache-2.0
+license_link: https://huggingface.co/Qwen/Qwen3-Coder-480B-A35B-Instruct/blob/main/LICENSE
+pipeline_tag: text-generation
+---
+> [!NOTE]
+>  Includes Unsloth **chat template fixes**! <br> For `llama.cpp`, use `--jinja`
+>
+
+<div>
+<p style="margin-top: 0;margin-bottom: 0;">
+    <em><a href="https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-gguf">Unsloth Dynamic 2.0</a> achieves superior accuracy & outperforms other leading quants.</em>
+  </p>
+  <div style="display: flex; gap: 5px; align-items: center; ">
+    <a href="https://github.com/unslothai/unsloth/">
+      <img src="https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png" width="133">
+    </a>
+    <a href="https://discord.gg/unsloth">
+      <img src="https://github.com/unslothai/unsloth/raw/main/images/Discord%20button.png" width="173">
+    </a>
+    <a href="https://docs.unsloth.ai/">
+      <img src="https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/images/documentation%20green%20button.png" width="143">
+    </a>
+  </div>
+</div>
+
+
+# Qwen3-Coder-480B-A35B-Instruct
+<a href="https://chat.qwen.ai/" target="_blank" style="margin: 2px;">
+    <img alt="Chat" src="https://img.shields.io/badge/%F0%9F%92%9C%EF%B8%8F%20Qwen%20Chat%20-536af5" style="display: inline-block; vertical-align: middle;"/>
+</a>
+
+## Highlights
+
+Today, we're announcing **Qwen3-Coder**, our most agentic code model to date. **Qwen3-Coder** is available in multiple sizes, but we're excited to introduce its most powerful variant first: **Qwen3-Coder-480B-A35B-Instruct**. featuring the following key enhancements:  
+
+- **Significant Performance** among open models on **Agentic Coding**, **Agentic Browser-Use**, and other foundational coding tasks, achieving results comparable to Claude Sonnet.
+- **Long-context Capabilities** with native support for **256K** tokens, extendable up to **1M** tokens using Yarn, optimized for repository-scale understanding.
+- **Agentic Coding** supporting for most platfrom such as **Qwen Code**, **CLINE**, featuring a specially designed function call format.
+
+![image/jpeg](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Coder/qwen3-coder-main.jpg)
+
+## Model Overview
+
+**Qwen3-480B-A35B-Instruct** has the following features:
+- Type: Causal Language Models
+- Training Stage: Pretraining & Post-training
+- Number of Parameters: 480B in total and 35B activated
+- Number of Layers: 62
+- Number of Attention Heads (GQA): 96 for Q and 8 for KV
+- Number of Experts: 160
+- Number of Activated Experts: 8
+- Context Length: **262,144 natively**. 
+
+**NOTE: This model supports only non-thinking mode and does not generate ``<think></think>`` blocks in its output. Meanwhile, specifying `enable_thinking=False` is no longer required.**
+
+For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our [blog](https://qwenlm.github.io/blog/qwen3-coder/), [GitHub](https://github.com/QwenLM/Qwen3-Coder), and [Documentation](https://qwen.readthedocs.io/en/latest/).
+
+
+## Quickstart
+
+We advise you to use the latest version of `transformers`.
+
+With `transformers<4.51.0`, you will encounter the following error:
+```
+KeyError: 'qwen3_moe'
+```
+
+The following contains a code snippet illustrating how to use the model generate content based on given inputs. 
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "Qwen/Qwen3-480B-A35B-Instruct"
+
+# load the tokenizer and the model
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype="auto",
+    device_map="auto"
+)
+
+# prepare the model input
+prompt = "Write a quick sort algorithm."
+messages = [
+    {"role": "user", "content": prompt}
+]
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True,
+)
+model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
+
+# conduct text completion
+generated_ids = model.generate(
+    **model_inputs,
+    max_new_tokens=65536
+)
+output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() 
+
+content = tokenizer.decode(output_ids, skip_special_tokens=True)
+
+print("content:", content)
+```
+
+**Note: If you encounter out-of-memory (OOM) issues, consider reducing the context length to a shorter value, such as `32,768`.**
+
+
+## Note on FP8
+
+For convenience and performance, we have provided `fp8`-quantized model checkpoint for Qwen3, whose name ends with `-FP8`. The quantization method is fine-grained `fp8` quantization with block size of 128. You can find more details in the `quantization_config` field in `config.json`.
+
+You can use the Qwen3-480B-A35B-Instruct-FP8 model with serveral inference frameworks, including `transformers`, `sglang`, and `vllm`, as the original bfloat16 model.
+However, please pay attention to the following known issues:
+- `transformers`:
+    - there are currently issues with the "fine-grained fp8" method in `transformers` for distributed inference. You may need to set the environment variable `CUDA_LAUNCH_BLOCKING=1` if multiple devices are used in inference.
+
+
+## Agentic Coding
+
+Qwen3-Coder excels in tool calling capabilities. 
+
+You can simply define or use any tools as following example.
+```python
+# Your tool implementation
+def square_the_number(num: float) -> dict:
+    return num ** 2
+
+# Define Tools
+tools=[
+    {
+        "type":"function",
+        "function":{
+            "name": "square_the_number",
+            "description": "output the square of the number.",
+            "parameters": {
+                "type": "object",
+                "required": ["input_num"],
+                "properties": {
+                    'input_num': {
+                        'type': 'number', 
+                        'description': 'input_num is a number that will be squared'
+                        }
+                },
+            }
+        }
+    }
+]
+
+import OpenAI
+# Define LLM
+client = OpenAI(
+    # Use a custom endpoint compatible with OpenAI API
+    base_url='http://localhost:8000/v1',  # api_base
+    api_key="EMPTY"
+)
+ 
+messages = [{'role': 'user', 'content': 'square the number 1024'}]
+
+completion = client.chat.completions.create(
+    messages=messages,
+    model="Qwen3-480B-A35B-Instruct",
+    max_tokens=65536,
+    tools=tools,
+)
+
+print(completion.choice[0])
+```
+
+## Best Practices
+
+To achieve optimal performance, we recommend the following settings:
+
+1. **Sampling Parameters**:
+   - We suggest using `temperature=0.7`, `top_p=0.8`, `top_k=20`, `repetition_penalty=1.05`.
+
+2. **Adequate Output Length**: We recommend using an output length of 65,536 tokens for most queries, which is adequate for instruct models.
+
+
+### Citation
+
+If you find our work helpful, feel free to give us a cite.
+
+```
+@misc{qwen3technicalreport,
+      title={Qwen3 Technical Report}, 
+      author={Qwen Team},
+      year={2025},
+      eprint={2505.09388},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2505.09388}, 
+}
+```

added_tokens.json

@@ -0,0 +1,28 @@
+{
+  "</think>": 151668,
+  "</tool_call>": 151658,
+  "</tool_response>": 151666,
+  "<think>": 151667,
+  "<tool_call>": 151657,
+  "<tool_response>": 151665,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

chat_template.jinja

@@ -0,0 +1,131 @@
+{% macro render_item_list(item_list, tag_name='required') %}
+    {%- if item_list is defined and item_list is iterable and item_list | length > 0 %}
+        {%- if tag_name %}{{- '\n<' ~ tag_name ~ '>' -}}{% endif %}
+            {{- '[' }}
+                {%- for item in item_list -%}
+                    {%- if loop.index > 1 %}{{- ", "}}{% endif -%}
+                    {%- if item is string -%}
+                        {{ "`" ~ item ~ "`" }}
+                    {%- else -%}
+                        {{ item }}
+                    {%- endif -%}
+                {%- endfor -%}
+            {{- ']' }}
+        {%- if tag_name %}{{- '</' ~ tag_name ~ '>' -}}{% endif %}
+    {%- endif %}
+{% endmacro %}
+
+{%- if messages[0]["role"] == "system" %}
+    {%- set system_message = messages[0]["content"] %}
+    {%- set loop_messages = messages[1:] %}
+{%- else %}
+    {%- set loop_messages = messages %}
+{%- endif %}
+
+{%- if not tools is defined %}
+    {%- set tools = [] %}
+{%- endif %}
+
+{%- if system_message is defined %}
+    {{- "<|im_start|>system\n" + system_message }}
+{%- else %}
+    {%- if tools is iterable and tools | length > 0 %}
+        {{- "<|im_start|>system\nYou are Qwen, a helpful AI assistant that can interact with a computer to solve tasks." }}
+    {%- endif %}
+{%- endif %}
+{%- if tools is iterable and tools | length > 0 %}
+    {{- "\n\nYou have access to the following functions:\n\n" }}
+    {{- "<tools>" }}
+    {%- for tool in tools %}
+        {%- if tool.function is defined %}
+            {%- set tool = tool.function %}
+        {%- endif %}
+        {{- "\n<function>\n<name>" ~ tool.name ~ "</name>" }}
+        {{- '\n<description>' ~ (tool.description | trim) ~ '</description>' }}
+        {{- '\n<parameters>' }}
+        {%- for param_name, param_fields in tool.parameters.properties|items %}
+            {{- '\n<parameter>' }}
+            {{- '\n<name>' ~ param_name ~ '</name>' }}
+            {%- if param_fields.type is defined %}
+                {{- '\n<type>' ~ (param_fields.type | string) ~ '</type>' }}
+            {%- endif %}
+            {%- if param_fields.description is defined %}
+                {{- '\n<description>' ~ (param_fields.description | trim) ~ '</description>' }}
+            {%- endif %}
+            {{- render_item_list(param_fields.enum, 'enum') }}
+            {%- set handled_keys = ['type', 'description', 'enum', 'required'] %}
+            {%- for json_key in param_fields.keys() | reject("in", handled_keys) %}
+                {%- set normed_json_key = json_key | replace("-", "_") | replace(" ", "_") | replace("$", "") %}
+                {%- if param_fields[json_key] is mapping %}
+                    {{- '\n<' ~ normed_json_key ~ '>' ~ (param_fields[json_key] | tojson | safe) ~ '</' ~ normed_json_key ~ '>' }}
+                {%- else %}
+                    {{-'\n<' ~ normed_json_key ~ '>' ~ (param_fields[json_key] | string) ~ '</' ~ normed_json_key ~ '>' }}
+                {%- endif %}
+            {%- endfor %}
+            {{- render_item_list(param_fields.required, 'required') }}
+            {{- '\n</parameter>' }}
+        {%- endfor %}
+        {{- render_item_list(tool.parameters.required, 'required') }}
+        {{- '\n</parameters>' }}
+        {%- if tool.return is defined %}
+            {%- if tool.return is mapping %}
+                {{- '\n<return>' ~ (tool.return | tojson | safe) ~ '</return>' }}
+            {%- else %}
+                {{- '\n<return>' ~ (tool.return | string) ~ '</return>' }}
+            {%- endif %}
+        {%- endif %}
+        {{- '\n</function>' }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+{%- endif %}
+{%- if system_message is defined %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if tools is iterable and tools | length > 0 %}
+        {{- '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in loop_messages %}
+    {%- if message.role == "assistant" and message.tool_calls is defined and message.tool_calls is iterable and message.tool_calls | length > 0 %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content is defined and message.content is string and message.content | trim | length > 0 %}
+            {{- '\n' + message.content | trim + '\n' }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+            {%- if tool_call.arguments is defined %}
+                {%- for args_name, args_value in tool_call.arguments|items %}
+                    {{- '<parameter=' + args_name + '>\n' }}
+                    {%- set args_value = args_value if args_value is string else args_value | string %}
+                    {{- args_value }}
+                    {{- '\n</parameter>\n' }}
+                {%- endfor %}
+            {%- endif %}
+            {{- '</function>\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "user" or message.role == "system" or message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user\n' }}
+        {%- endif %}
+        {{- '<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>\n' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

config.json

@@ -0,0 +1,239 @@
+{
+  "architectures": [
+    "Qwen3MoeForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "decoder_sparse_step": 1,
+  "eos_token_id": 151645,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 6144,
+  "initializer_range": 0.02,
+  "intermediate_size": 8192,
+  "max_position_embeddings": 262144,
+  "max_window_layers": 28,
+  "mlp_only_layers": [],
+  "model_type": "qwen3_moe",
+  "moe_intermediate_size": 2560,
+  "norm_topk_prob": true,
+  "num_attention_heads": 96,
+  "num_experts": 160,
+  "num_experts_per_tok": 8,
+  "num_hidden_layers": 62,
+  "num_key_value_heads": 8,
+  "output_router_logits": false,
+  "pad_token_id": 151654,
+  "qkv_bias": false,
+  "quantization_config": {
+    "activation_scheme": "dynamic",
+    "modules_to_not_convert": [
+      "lm_head",
+      "model.layers.0.input_layernorm",
+      "model.layers.0.mlp.gate",
+      "model.layers.0.post_attention_layernorm",
+      "model.layers.1.input_layernorm",
+      "model.layers.1.mlp.gate",
+      "model.layers.1.post_attention_layernorm",
+      "model.layers.2.input_layernorm",
+      "model.layers.2.mlp.gate",
+      "model.layers.2.post_attention_layernorm",
+      "model.layers.3.input_layernorm",
+      "model.layers.3.mlp.gate",
+      "model.layers.3.post_attention_layernorm",
+      "model.layers.4.input_layernorm",
+      "model.layers.4.mlp.gate",
+      "model.layers.4.post_attention_layernorm",
+      "model.layers.5.input_layernorm",
+      "model.layers.5.mlp.gate",
+      "model.layers.5.post_attention_layernorm",
+      "model.layers.6.input_layernorm",
+      "model.layers.6.mlp.gate",
+      "model.layers.6.post_attention_layernorm",
+      "model.layers.7.input_layernorm",
+      "model.layers.7.mlp.gate",
+      "model.layers.7.post_attention_layernorm",
+      "model.layers.8.input_layernorm",
+      "model.layers.8.mlp.gate",
+      "model.layers.8.post_attention_layernorm",
+      "model.layers.9.input_layernorm",
+      "model.layers.9.mlp.gate",
+      "model.layers.9.post_attention_layernorm",
+      "model.layers.10.input_layernorm",
+      "model.layers.10.mlp.gate",
+      "model.layers.10.post_attention_layernorm",
+      "model.layers.11.input_layernorm",
+      "model.layers.11.mlp.gate",
+      "model.layers.11.post_attention_layernorm",
+      "model.layers.12.input_layernorm",
+      "model.layers.12.mlp.gate",
+      "model.layers.12.post_attention_layernorm",
+      "model.layers.13.input_layernorm",
+      "model.layers.13.mlp.gate",
+      "model.layers.13.post_attention_layernorm",
+      "model.layers.14.input_layernorm",
+      "model.layers.14.mlp.gate",
+      "model.layers.14.post_attention_layernorm",
+      "model.layers.15.input_layernorm",
+      "model.layers.15.mlp.gate",
+      "model.layers.15.post_attention_layernorm",
+      "model.layers.16.input_layernorm",
+      "model.layers.16.mlp.gate",
+      "model.layers.16.post_attention_layernorm",
+      "model.layers.17.input_layernorm",
+      "model.layers.17.mlp.gate",
+      "model.layers.17.post_attention_layernorm",
+      "model.layers.18.input_layernorm",
+      "model.layers.18.mlp.gate",
+      "model.layers.18.post_attention_layernorm",
+      "model.layers.19.input_layernorm",
+      "model.layers.19.mlp.gate",
+      "model.layers.19.post_attention_layernorm",
+      "model.layers.20.input_layernorm",
+      "model.layers.20.mlp.gate",
+      "model.layers.20.post_attention_layernorm",
+      "model.layers.21.input_layernorm",
+      "model.layers.21.mlp.gate",
+      "model.layers.21.post_attention_layernorm",
+      "model.layers.22.input_layernorm",
+      "model.layers.22.mlp.gate",
+      "model.layers.22.post_attention_layernorm",
+      "model.layers.23.input_layernorm",
+      "model.layers.23.mlp.gate",
+      "model.layers.23.post_attention_layernorm",
+      "model.layers.24.input_layernorm",
+      "model.layers.24.mlp.gate",
+      "model.layers.24.post_attention_layernorm",
+      "model.layers.25.input_layernorm",
+      "model.layers.25.mlp.gate",
+      "model.layers.25.post_attention_layernorm",
+      "model.layers.26.input_layernorm",
+      "model.layers.26.mlp.gate",
+      "model.layers.26.post_attention_layernorm",
+      "model.layers.27.input_layernorm",
+      "model.layers.27.mlp.gate",
+      "model.layers.27.post_attention_layernorm",
+      "model.layers.28.input_layernorm",
+      "model.layers.28.mlp.gate",
+      "model.layers.28.post_attention_layernorm",
+      "model.layers.29.input_layernorm",
+      "model.layers.29.mlp.gate",
+      "model.layers.29.post_attention_layernorm",
+      "model.layers.30.input_layernorm",
+      "model.layers.30.mlp.gate",
+      "model.layers.30.post_attention_layernorm",
+      "model.layers.31.input_layernorm",
+      "model.layers.31.mlp.gate",
+      "model.layers.31.post_attention_layernorm",
+      "model.layers.32.input_layernorm",
+      "model.layers.32.mlp.gate",
+      "model.layers.32.post_attention_layernorm",
+      "model.layers.33.input_layernorm",
+      "model.layers.33.mlp.gate",
+      "model.layers.33.post_attention_layernorm",
+      "model.layers.34.input_layernorm",
+      "model.layers.34.mlp.gate",
+      "model.layers.34.post_attention_layernorm",
+      "model.layers.35.input_layernorm",
+      "model.layers.35.mlp.gate",
+      "model.layers.35.post_attention_layernorm",
+      "model.layers.36.input_layernorm",
+      "model.layers.36.mlp.gate",
+      "model.layers.36.post_attention_layernorm",
+      "model.layers.37.input_layernorm",
+      "model.layers.37.mlp.gate",
+      "model.layers.37.post_attention_layernorm",
+      "model.layers.38.input_layernorm",
+      "model.layers.38.mlp.gate",
+      "model.layers.38.post_attention_layernorm",
+      "model.layers.39.input_layernorm",
+      "model.layers.39.mlp.gate",
+      "model.layers.39.post_attention_layernorm",
+      "model.layers.40.input_layernorm",
+      "model.layers.40.mlp.gate",
+      "model.layers.40.post_attention_layernorm",
+      "model.layers.41.input_layernorm",
+      "model.layers.41.mlp.gate",
+      "model.layers.41.post_attention_layernorm",
+      "model.layers.42.input_layernorm",
+      "model.layers.42.mlp.gate",
+      "model.layers.42.post_attention_layernorm",
+      "model.layers.43.input_layernorm",
+      "model.layers.43.mlp.gate",
+      "model.layers.43.post_attention_layernorm",
+      "model.layers.44.input_layernorm",
+      "model.layers.44.mlp.gate",
+      "model.layers.44.post_attention_layernorm",
+      "model.layers.45.input_layernorm",
+      "model.layers.45.mlp.gate",
+      "model.layers.45.post_attention_layernorm",
+      "model.layers.46.input_layernorm",
+      "model.layers.46.mlp.gate",
+      "model.layers.46.post_attention_layernorm",
+      "model.layers.47.input_layernorm",
+      "model.layers.47.mlp.gate",
+      "model.layers.47.post_attention_layernorm",
+      "model.layers.48.input_layernorm",
+      "model.layers.48.mlp.gate",
+      "model.layers.48.post_attention_layernorm",
+      "model.layers.49.input_layernorm",
+      "model.layers.49.mlp.gate",
+      "model.layers.49.post_attention_layernorm",
+      "model.layers.50.input_layernorm",
+      "model.layers.50.mlp.gate",
+      "model.layers.50.post_attention_layernorm",
+      "model.layers.51.input_layernorm",
+      "model.layers.51.mlp.gate",
+      "model.layers.51.post_attention_layernorm",
+      "model.layers.52.input_layernorm",
+      "model.layers.52.mlp.gate",
+      "model.layers.52.post_attention_layernorm",
+      "model.layers.53.input_layernorm",
+      "model.layers.53.mlp.gate",
+      "model.layers.53.post_attention_layernorm",
+      "model.layers.54.input_layernorm",
+      "model.layers.54.mlp.gate",
+      "model.layers.54.post_attention_layernorm",
+      "model.layers.55.input_layernorm",
+      "model.layers.55.mlp.gate",
+      "model.layers.55.post_attention_layernorm",
+      "model.layers.56.input_layernorm",
+      "model.layers.56.mlp.gate",
+      "model.layers.56.post_attention_layernorm",
+      "model.layers.57.input_layernorm",
+      "model.layers.57.mlp.gate",
+      "model.layers.57.post_attention_layernorm",
+      "model.layers.58.input_layernorm",
+      "model.layers.58.mlp.gate",
+      "model.layers.58.post_attention_layernorm",
+      "model.layers.59.input_layernorm",
+      "model.layers.59.mlp.gate",
+      "model.layers.59.post_attention_layernorm",
+      "model.layers.60.input_layernorm",
+      "model.layers.60.mlp.gate",
+      "model.layers.60.post_attention_layernorm",
+      "model.layers.61.input_layernorm",
+      "model.layers.61.mlp.gate",
+      "model.layers.61.post_attention_layernorm"
+    ],
+    "quant_method": "fp8",
+    "weight_block_size": [
+      128,
+      128
+    ]
+  },
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": null,
+  "rope_theta": 10000000,
+  "router_aux_loss_coef": 0.0,
+  "shared_expert_intermediate_size": 0,
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.53.3",
+  "unsloth_fixed": true,
+  "use_cache": true,
+  "use_qk_norm": true,
+  "use_sliding_window": false,
+  "vocab_size": 151936
+}

generation_config.json

@@ -0,0 +1,12 @@
+{
+  "pad_token_id": 151643,
+  "do_sample": true,
+  "eos_token_id": [
+    151645,
+    151643
+  ],
+  "repetition_penalty": 1.05,
+  "temperature": 0.7,
+  "top_p": 0.8,
+  "top_k": 20
+}

model-00001-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da5e1fb3695be02926ec09f8d46d5fe0499ab5bad792973e20467936c157f965
+size 9984900968

model-00002-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4e89f3eeb7bc743905a6fc9bc8a6ef27d7f2b5f47d85d0fcbb8e6ce549971304
+size 9997346816

model-00003-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:08c86cf377a571560bf36f73c2bab548557d2c5336b014a30d4ace255289b44f
+size 9997346856

model-00004-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e47c1d020c9e3944b735b96fc6877a2f9ee5501306d62dead8a7cc0ab228fa0
+size 9997347136

model-00005-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3da6f32b9fac7215e24be62021f0a9ccd7e0674c9f51f7fb08b1c734823098ae
+size 9989898424

model-00006-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8688e8d26a7ca9e3dbef62a7dae247d85014978d558e91e7bba8d7a611548bc7
+size 9997346848

model-00007-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5bf602baa71a1e711362e06057dc5b01ce08e11c03bd70596f2fa57b783cf9ba
+size 9997347040

model-00008-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5e3583fc9ac98459f995ce00de1e0f20f69e5825c879604fef262dae76f2de8d
+size 9989899528

model-00009-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a0054be18235f5a680668868a8b7839da83575148b69ab8de6c671eba1e9671
+size 9997348112

model-00010-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9bfc85be495ebc6ab7cc2efb2e5e262de478844324537b3cd836d96aec8369e5
+size 9997348184

model-00011-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9a2c1a5a489db5b7217c69e65ee344ba461c908ead79b0676ce033d9900c4864
+size 9950180760

model-00012-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:41897c33c3ffea539e39796f5b606d119ad6e9ec72d80b3dac0c49e4970e69dd
+size 9989874840

model-00013-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e05692ebfe61aa8a7d7479f1ee5df8e416a4ae21576074afddaa2d8a545c779c
+size 9997348112

model-00014-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:81f310ebd39dbc0280c6f920cfb4ed1896aec85b39781b0d8df84f82cf53a166
+size 9997348360

model-00015-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:66f0a10a49f8db2aa5c08e5a9a15379f9a9bdffeffc40d6576124199ee62e5c8
+size 9989899704

model-00016-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b5105a58532352a9802e4fd12d7345f1d467fe866d21fd973abad9d641095e4
+size 9997348112

model-00017-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d69954e8e319b404e230c9d76cc9c68ed304d91ee994406e2259993e3e343b22
+size 9997348240

model-00018-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f87ae0f24a92020b0d2ad5c962d1ce4ae9a859bfdac383f2cbd59ef7185059c
+size 9989899840

model-00019-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:669d3cfd31b76fcd4556e0afdff75ed512a3c592d88905fadcf0a46ac19a703b
+size 9997348088

model-00020-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b6e111d07e3a7b4f796998a08d46900be19c67a8a10f0b2b20acdcc03d98c67
+size 9997348136

model-00021-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3f9b50185c94eb55fb0aaae147febfebe0fc8e97aeafa437372b3363e8331d56
+size 9997348400

model-00022-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8f58ac8051a00a8b4f07a0c6a6f28e714c37715f072bcb2bc2c1f9a8231c20f
+size 9989899664

model-00023-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee024a2cb8ce6a45274960c2a61279de1aad85834a6a3ff207c832d229cabc6f
+size 9997348112

model-00024-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b657832f896619c326270bccd232cfc5d95a7ea328a6bf750559c19dcc62aee
+size 9997348312

model-00025-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b003320079404a5ef55cb253611f8cfc097802e601447848473dc13996571af7
+size 9989899760

model-00026-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c86ed89803f3ef7e89b0f81c299026a33d66ce7de1ad15e536fc6edcafe6ef2b
+size 9997348104

model-00027-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ce3adf6ead7f0273c6cd74fa77095a5d6074ff828868fc4ceeb20810db1da2b
+size 9997348200

model-00028-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:668628f53a82dceaced90396a7919debbf19befbc94efacee5fc65a79047c1f1
+size 9928156904

model-00029-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a3566cebf32e22c5e48be0b94605314a49caeb0f6356fe4a2a318387b52b287f
+size 9996167872

model-00030-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf1d968c1b5736a96bc80c0e434fd0f3669434f15dc3e183cdbdc208a6e2ad5a
+size 9997348112

model-00031-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b1f4796d7f92d7cf78b7ea54c009a99fe5b64e2a6368a676ab3cb7d09d34c2ac
+size 9997348368

model-00032-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1b728be17d6855173188b5c94866afb39a2ff5cf95c09d10cdec7270d58a5bd8
+size 9989899696

model-00033-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f1b10f22d419dfcc4fde4ad34b699015c3b6f4fdc125cf93e340585b5b2d194
+size 9997348112

model-00034-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:96649317f0644ac78f561ed0d9dee30a8e7b9f3bea2e37785dadebd679ff1f25
+size 9997348256

model-00035-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e37a3a14349ee4907cba7396144c0637fb10280b10824d88e9b5e3e263de4c9
+size 9989899816

model-00036-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37e5b2c17e702d17a11aed1a2123ef3a0fc04536fbe2700df00490b980f27f83
+size 9997348104

model-00037-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:640b809ee36dc6c14390623b6e06e694fc803e3aaa0275eb2d5419f5427a2789
+size 9997348144

model-00038-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b82a4474c3aef95967ac69178c4179baa7b5f562974c5e6c5a677c4ffecdad88
+size 9997348400

model-00039-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:67862f44ff6b1456482a5eaab52b52b848db5fe20eba2298c2994e8b63de421a
+size 9989899656

model-00040-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dca78652b591dbdbec2e5a36d2667e33de181507dc6a5c07cf037b3cbdea9b93
+size 9997348112

model-00041-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5ceb5403e7debb467bad47d887a070e2776d1cde7ffa8f604c2927f040f3626
+size 9997348320

model-00042-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:57df19df663cceeb6ee3e300979c2c58a0f735ed78e838248ea54909e33940f8
+size 9989899744

model-00043-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:24e2251ad9431ae9d8e234f99dab905a1dd7b2cef10eceba1f07aa4a53aa067f
+size 9997348104