aws-neuron
/

optimum-neuron-cache

dacorvo HF Staff commited on 6 days ago

Commit

5e828c9

verified ·

1 Parent(s): 2fdfa36

Update inference-cache-config/qwen3-moe.json

Files changed (1) hide show

inference-cache-config/qwen3-moe.json CHANGED Viewed

@@ -9,13 +9,13 @@
     {
       "batch_size": 4,
       "sequence_length": 4096,
-      "num_cores": 2,
       "auto_cast_type": "bf16"
     },
     {
       "batch_size": 8,
       "sequence_length": 4096,
-      "num_cores": 2,
       "auto_cast_type": "bf16"
     }
   ]

     {
       "batch_size": 4,
       "sequence_length": 4096,
+      "num_cores": 8,
       "auto_cast_type": "bf16"
     },
     {
       "batch_size": 8,
       "sequence_length": 4096,
+      "num_cores": 8,
       "auto_cast_type": "bf16"
     }
   ]