As correctly questioned in community section, your recipe contradicts with quantized model parameters in recipe.yaml. I guess the correct version of the recipe is the one that is demonstrated in llm-compressor github.

The proposed change is made accordingly.

Cannot merge
This branch has merge conflicts in the following files:
  • README.md
Your need to confirm your account before you can post a new comment.

Sign up or log in to comment