Bartowski! 0.0!!!! You are on double-secret probation for this jinja error!
Just joking and thanks dude for all you do for the community and opening AI and this bizarro world up for non-rocket scientists, but I am getting a jinja error in LM-studio with your Q5KL quant? Here is the error: "Failed to parse Jinja template: Parser Error: Expected closing statement token. OpenSquareBracket !== CloseStatement." Can you see whats up with this and I am gonna try Iq4NL with my 4080 right now too since I still don't know wtf to pick, or which is better but thanks!!
Same issue here, I tried a few GGUFs including the official one but they all fail with that error in LM Studio (had a similar issue with arcee-blitz)
You can either use the official lmstudio-community ones here: https://huggingface.co/lmstudio-community/QwQ-32B-GGUF
or you can apply the fix from here: https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/479#issuecomment-2701947624
Thanks, and since you use iMatrix and LM-studio quants don't I stick to yours but my gratiude to the devs of that program is due as well. I hope u got the Animal House reference and idk why your name always makes me think of it? But it appears to be working now except he made random tool calls now?- in LM-Studio and not saying that's your fault but for others who read this just be careful and I will test it further and follow up but my sincere gratitude for contributing the way you do! Also did u ever figure out what was going on with your 3bit quants?
since you use iMatrix and LM-studio quants don't I stick to yours
Same - and just curious since you're (Bartowski) the official curator for their models, is there any reason they don't?
iMatrix always seem to be a straight-up upgrade.
I agree, but there was circumstantial evidence (that I don't agree with) showing degradation in performance when imatrix was applied
The lack of imatrix also makes the results more "pure" and impossible to accuse of dataset biasing
Again, I still hold that imatrix is universally better, but there was enough to convince to switch to static. Plus static is way faster to make so the lmstudio ones can be first out the door while I crunch the numbers for my own behind the scenes