Possible to get exl2 quants?
#1
by
Shoukupan
- opened
This model has been my favorite so far, I was wondering if it would be possible to get an exl2 version for the speed benefits?
Thanks!
Should already have it. On the model page just below the picture are links to the EXL2 quants.
https://huggingface.co/Statuo/LemonKunoichiWizardv3_4bpw
https://huggingface.co/Statuo/LemonKunoichiWizardv3_6bpw
https://huggingface.co/Statuo/LemonKunoichiWizardv3_8bpw