diff options
| author | Danilo M. <danix@danix.xyz> | 2026-08-01 20:03:56 +0200 |
|---|---|---|
| committer | Danilo M. <danix@danix.xyz> | 2026-08-01 20:03:56 +0200 |
| commit | 2fae45408489594d009215c82c2ca9acdc00e394 (patch) | |
| tree | ed5587690429b360e898433eadc85e00158760f5 /llamachat.py | |
| parent | fe335db0cdc72cc190c290479d24c6469e2c07ab (diff) | |
| download | llamachat-2fae45408489594d009215c82c2ca9acdc00e394.tar.gz llamachat-2fae45408489594d009215c82c2ca9acdc00e394.zip | |
feat: model-written titles, fix empty replies after search
Titles: once a reply lands in a session still wearing its placeholder
title, the exchange goes back to the model in a short side request asking
for six words or fewer. Keyed on the placeholder rather than the turn
number, so an empty first reply does not forfeit titling for the session
and a one-shot window is titled from the question that was answered.
Thinking is disabled for that request, or a reasoning model spends the
whole budget thinking and returns nothing.
Empty replies: the final search round now says so in the tool result.
Withdrawing the tool schema is invisible to the model, which asks for
another search regardless; the request then surfaces as literal
<tool_call> text or vanishes into the thinking block, leaving the reply
empty either way. The note rides on the tool result because Qwen3.5's
template rejects a trailing system message outright.
max_searches now defaults to 2. The other half of that bug was
llama-server's quantized KV cache: with cache-type-k/v = q8_0 three of
five turns broke, and f16 answered six of six.
Measured against a live router, not mocked.
Diffstat (limited to 'llamachat.py')
0 files changed, 0 insertions, 0 deletions
