mirror of
https://github.com/Crosstalk-Solutions/project-nomad.git
synced 2026-10-17 13:19:18 +02:00
27be71b46e
Chat sent num_ctx, num_predict and nothing else, so every turn decoded at whatever the backend picked: temperature 0.8, top_p 0.9, no min_p. On a heavily quantized 1-3B model that is the wandering, repetitive output people read as "the local model is bad". Adds `ai.responseStyle` with four profiles. min_p is the lever, not temperature: it floors relative to the top token, so it truncates hard on a confident distribution and barely at all on a flat one, which is where small models start to wander. top_p stays at 1.0 rather than stacked on top of it. repeat_penalty moves with the style, because a grounded answer legitimately repeats a part number inside the 64-token window: `focused` relaxes it to 1.05, `creative` tightens it to 1.15. `auto` is the only style that defers to the model. Plenty of library models ship author-tuned samplers (qwen3 sets 0.6/0.95/20), so those win key by key and NOMAD fills only the gap, which in practice is min_p. A style the user picked by name overrides them, or it would silently do nothing on half their models. The compat transport drops min_p, so a profile carries a separate compatTopP. Passing top_p 1.0 to a backend with no min_p behind it would sample the raw distribution, which is worse than the 0.9 it replaced. Resolved in the controller, so the grammar-constrained task calls keep their temperature 0. An explicit temperature still outranks a profile's, which is what keeps the eval harness deterministic. `eval:generation --style` is new and load-bearing: temperature 0 makes min_p, top_p and top_k inert, so a default run cannot see this change at all. `--style=off` sends no samplers and no temperature, and is the control. Verified against Ollama 0.32.9, min_p accepted. 22 unit tests. |
||
|---|---|---|
| .. | ||
| app_auto_update | ||
| auto_update | ||
| benchmark | ||
| content_auto_update | ||
| eval | ||
| queue | ||