No reasoning
Its not reasoning on ollama...
Its not reasoning on ollama...
Sadly. The reasoning becomes muddled due to the merging process.
You can trigger it though CoT but usually only using thinking extensions in sillytavern
Oh. Any tamplate for ollama or lmstudio? Not using sillytavern. Only ollama with api and custom python code
This was one of the best to trigger a form of thinking, I am still working on other methods.
<thinking_trigger>
[Respond by naturally blending analysis into answers using these rules:
1. Begin with a cognitive hook ("First, lets consider the users question X then we need to consider X...")
2. Acknowledge ambiguities through appositives ("...which could mean Y, but in Z contexts...")
3. Resolve assumptions mid-sentence using em-dashes ("—assuming A here because B—")
4. Conclude with implications ("...so ultimately C becomes critical due to D")
</thinking_trigger>
I tried IQ4_XS. Confirm. Can't consistently get it to reason (DSR1/L3 templates, various or prefill) and even when it does it is kind of short L3 like reason and not really to the point, nothing like R1 Distill. Also it has strong refusals for me (even in scenes where straight L3_3 instruct or L3_3 R1 Distill comply without problem, actually I do not remember any model I ever tested having refusal on that particular testing card I use). Strange.
So its reasoning seems more or less unusable and in that case I would suggest models without L3.3 Distill R1 merged (as L3_3 70B instruct is definitively better than L3 70B Distill R1 without reasoning, Distill R1 only gains edge with its reasoning process).
Sad as it would be nice to have more RP reasoning finetunes or merges to choose from, so far I only found one that works (not that there were many made so far).
Lower quants seem to have issues with reasoning but this model was much more focused on other aspects (intelligence, scene depth, descriptions, and narrative qualities). Plus the creation process breaks the reasoning further.
I'm current working on a new test (using the user feedback from Damascus-R1 and Nevoria-R1), that appears to fully restore the reasoning capabilities, removes the refusals almost all together, and further improves upon what Damascus and Nevoria were built on.
Actually I have no problem with reasoning 70B models at imatrix IQ4_XS and IQ3_M (did not try any lower). L3 70B Distill R1 works very well out of the box. One other RP merge also works though needs lot more wiggling (as R1 is only part of it). L3_3 abliterated R1 works too though is bit unstable as most abliterated models are. The 70B IQ4_XS/IQ3_M reason as well if not better than Q8/Q6_K_L of 32B Distill R1's. However, all the L3 R1 distills have refusals (lot more than even straight L3_3 instruct) while with 32B Qwen distills I rarely see any refusal.
Anyway, good luck! I guess merging in R1 distills is still very new so it is bound to be hit and miss.