README.md

metadata

license: apache-2.0

This model is an int4 model with group_size 128 and symmetric quantization of Qwen/Qwen2-0.5B-Instruct generated by intel/auto-round algorithm.

Mainly for vllm ut