feat: A better logic for detecting the thinking prefix in the response

[Vinay-Umrethe]

In case if the model adds <think> at the end of user prompt and only generates </think> end in the model response, then the detection goes wrong.

LiquidAI/LFM2.5-2.6B generates response like this:

USER_PROMPT + <think> tag added to it at last from chat_template via tokenizer.apply_chat_template:

{%- if add_generation_prompt -%}
    {{- "<|im_start|>assistant\n<think>" -}}
{%- endif -%}

For fixing, we just create a dummy prompt to see whatever model is used with heretic...
"does its apply_chat_template(add_generation_prompt=True) method adds the tag or not to the prompt".
If yes, then we extract only the end part (like </think>) and set it as the response_prefix.

Before

Checking for common response prefix...
* Prefix found: 'The user'

After

Checking for common response prefix...
* Closed Chain-of-Thought block: '<think></think>'
* Rechecking with prefix...

[p-e-w]

This is too brittle. For example if the chat template ends with <think>\n instead of <think> this logic fails.

[Vinay-Umrethe]

good catch, I think applying rstrip() might be enough for it.

and I also expect this to fix CoT detection for cases like qwen3.5 models (which I almost never saw get properly detected) because qwen3.5 template has:

{%- if add_generation_prompt %}
    {{- '<|im_start|>assistant\n' }}
    {%- if enable_thinking is defined and enable_thinking is false %}
        {{- '<think>\n\n</think>\n\n' }}
    {%- else %}
        {{- '<think>\n' }}
    {%- endif %}
{%- endif %}

I'll test it and push if it works...

[p-e-w]

rstrip should do the trick in this case, but it's basically a hack. For example, the template could end with </think>\nOkay,, and we're back to zero. The correct approach would be to get the complete text (prompt + response) and then do a regex match. But this is probably not worth the complexity.

At some point we should revisit trying to set enable_thinking to False in the template, which would make the whole prefix detection unnecessary for many models. I've looked into this in the past, but there were several inconsistencies that might have been fixed in the meantime.

[Vinay-Umrethe]

Tested Qwen/Qwen3.5-4B

because variants which are <4B (like 2B or 0.8B) are non-reasoning by default, but 4B and above do the reasoning.

Before (v1.4 release)

Checking for common response prefix...
* None found

After

Checking for common response prefix...
* Closed Chain-of-Thought block: '</think>'
* Rechecking with prefix...
* Extended prefix found: '</think>\n\n'

So I think now both LFM and Qwen3.5 models are fixed and possibly a few more cases...

TODO:

• Add new hashes (or update existing ones) since this changes a major logic.

[Vinay-Umrethe]

Checking for common response prefix...
* Closed Chain-of-Thought block: '</think>\n'
* Rechecking with prefix...

got this now for Qwen/Qwen3.5-4B, probably better than rstrip() because it detected in first pass without needing extended check.

@p-e-w yet I have still not decided what to do with hashes for now.

[Vinay-Umrethe]

This Is A Major Change

1. tests/mistral-3 showed:

Checking for common response prefix...
* Closed Chain-of-Thought block: '[/THINK]Your thoughts or/and draft, like 
working through an exercise on scratch paper. Be as casual and as long as you 
want until you are confident to generate the response to the user.[/THINK]Here, 
provide a self-contained response.[/SYSTEM_PROMPT][INST][/INST]'
* Rechecking with prefix...
* Extended prefix found: '[/THINK]Your thoughts or/and draft, like working 
through an exercise on scratch paper. Be as casual and as long as you want until
you are confident to generate the response to the user.[/THINK]Here, provide a 
self-contained response.[/SYSTEM_PROMPT][INST][/INST]Ġ'

2. tests/qwen3.5-moe showed:

Checking for common response prefix...
* Closed Chain-of-Thought block: '</think>\n'
* Rechecking with prefix...

Until now, these models and many more almost always gave:

Checking for common response prefix...
* None found

So the point is that this PR comes with a major change which highly affects reproducibility of the models for the upcoming v2.0 of heretic and its thinking prefix detecting logic. So the SHA256SUMS hashes are now updated for CI and Windows specifically.

Not yet decided if this is better than the existing logic before I merge.

[p-e-w]

The output is pretty misleading though. Those are not "closed chain-of-though blocks".

[Vinay-Umrethe]

That's sensible enough, I didn't knew Mistral 3 had additional instructions for the model about telling it "how to think" in its chat template.

so I don't think there's a better way to detect the thinking tags for such ambiguous templates (without using hacks with more regex), so it is better that it fallbacks to inference rather than dummy prompt.

1. Qwen3.5 now gave:

* Closed Chain-of-Thought block: '<think></think>\n'

2. Mistral-3:

* Prefix found: 'Ġ'

3. Liquid AI LiquidAI/LFM2.5-2.6B ✅

Checking for common response prefix...
* Closed Chain-of-Thought block: '<think></think>'
* Rechecking with prefix...

[p-e-w]

* Prefix found: 'Ġ'

What is going on here? That looks like a fragment from a wrongly-decoded vocabulary token.