diff options
| author | Danilo M. <danix@danix.xyz> | 2026-08-11 11:22:32 +0200 |
|---|---|---|
| committer | Danilo M. <danix@danix.xyz> | 2026-08-11 11:22:32 +0200 |
| commit | 2cae7f19ac90e6df9d4008e6e364bae30e8b389a (patch) | |
| tree | 045eae84caefd1ebd97836fe0a5b97ae181022a2 /CHANGELOG.md | |
| parent | 3f984ccb8ec25ef3b06109d638eed4022db82d45 (diff) | |
| download | llamachat-2cae7f19ac90e6df9d4008e6e364bae30e8b389a.tar.gz llamachat-2cae7f19ac90e6df9d4008e6e364bae30e8b389a.zip | |
fix: prevent cloud reasoning models from exhausting default max_tokens
- Send max_tokens = 32768 for non-local providers so reasoning models have room for both chain-of-thought and answer.
- Add per-provider thinking_budget option for endpoints (currently SiliconFlow) that cap reasoning tokens separately.
- Accept bare JSON arrays from /v1/models; some OpenAI-compatible endpoints omit the {"data": [...]} envelope.
Diffstat (limited to 'CHANGELOG.md')
| -rw-r--r-- | CHANGELOG.md | 14 |
1 files changed, 14 insertions, 0 deletions
diff --git a/CHANGELOG.md b/CHANGELOG.md index d3df977..6e9a831 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -34,6 +34,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 `model`. Token counts come from the stream's final usage chunk and power the cost readout; the model column records which model actually replied, so a switched conversation prices each turn correctly. +- Per-provider `thinking_budget` option, for endpoints (currently SiliconFlow) + that cap chain-of-thought tokens separately from the final answer. Unset + providers skip the key entirely so other endpoints do not receive an unknown + parameter. + +### Fixed + +- Model listing crashed when a provider's `/v1/models` endpoint returned a + bare JSON array instead of the standard `{"data": [...]}` envelope. The + parser now accepts both shapes. +- Cloud chat requests no longer rely on the provider's default `max_tokens`, + which could be too small for a reasoning model. Non-local providers now send + `max_tokens = 32768`, leaving room for both chain-of-thought reasoning and + the answer. Tool calling and reasoning output vary between providers. Web search on a cloud model may not work as reliably as it does with a local router. |
