diff options
Diffstat (limited to 'AGENTS.md')
| -rw-r--r-- | AGENTS.md | 17 |
1 files changed, 17 insertions, 0 deletions
@@ -393,6 +393,23 @@ phase 2. - Whisper wrote "build system" as two words; the model mapped it to the VM name `buildsystem` without help. +### Context size (2026-09-26) + +Gemma's context is 32768 tokens. Measured against the running model: +system prompt + question 106 prompt tokens, + the 10 tool definitions +946, + a full 20-message history of typical exchanges 1576. Worst case +per question is ~9k (20 long history messages ~3k, up to 3 tool rounds +of 2000-character results ~4k), so the 20-message window and the +2000-character tool cap keep it well inside; each extra tool costs ~85. + +- The assistant logs `context: N prompt tokens` for every LLM round + (`stream_options.include_usage`). Live: a weather question 969, then + 1115 with the tool result. +- The server log (`/var/log/llama-server/server.log`) also prints + `stop processing: n_tokens = N, truncated = 0` per request, but it is + shared with every router client, so it cannot be attributed to the + assistant. `truncated = 1` there would mean an overflow. + ### Barge-in (2026-09-26) A button press cuts the current reply. A global `turn` counter goes up on |