This model was converted to the OpenVINO IR format in compliance with the documentation for use with NPU devices.

I used this command:

optimum-cli export openvino -m "input/path" --weight-format int4 --sym --group-size -1 --ratio 1.0 --awq --scale-estimation --dataset wikitext2  "output/path"
Downloads last month
14
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no library tag.

Collection including Echo9Zulu/Hermes-3-Llama-3.2-3B-int4-awq-se-ns-NPU-ov