diff options
| author | Danilo M. <danix@danix.xyz> | 2026-10-16 16:25:08 +0200 |
|---|---|---|
| committer | Danilo M. <danix@danix.xyz> | 2026-10-16 16:25:08 +0200 |
| commit | b25909ebea879bf6d72f0c41a9ec9d3ec40178dc (patch) | |
| tree | 5601a7235be7a3118313accac52decab2e6a712b /examples/tools/weather.toml | |
| parent | 0386d999403f78248aeda91d0cee1bd4716b8dda (diff) | |
| download | desktop-assistant-b25909ebea879bf6d72f0c41a9ec9d3ec40178dc.tar.gz desktop-assistant-b25909ebea879bf6d72f0c41a9ec9d3ec40178dc.zip | |
Hang up on llama-server at once when a reply is stopped
A stop was noticed at the reply's next chunk, and llama-server stops a
task only when its client hangs up. Through a long prompt pass, measured
at 12232 tokens for a file with page images, no chunk comes, so the GPU
kept working after Stop until the first one. interrupt() now shuts the
reply's socket from the stop handler; the request goes through
http.client, which exposes it. A fake server that stays silent records the
hang-up within a second of the stop.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Diffstat (limited to 'examples/tools/weather.toml')
0 files changed, 0 insertions, 0 deletions
