mirror of
https://github.com/Crosstalk-Solutions/project-nomad.git
synced 2026-10-17 13:19:18 +02:00
27be71b46e
Chat sent num_ctx, num_predict and nothing else, so every turn decoded at whatever the backend picked: temperature 0.8, top_p 0.9, no min_p. On a heavily quantized 1-3B model that is the wandering, repetitive output people read as "the local model is bad". Adds `ai.responseStyle` with four profiles. min_p is the lever, not temperature: it floors relative to the top token, so it truncates hard on a confident distribution and barely at all on a flat one, which is where small models start to wander. top_p stays at 1.0 rather than stacked on top of it. repeat_penalty moves with the style, because a grounded answer legitimately repeats a part number inside the 64-token window: `focused` relaxes it to 1.05, `creative` tightens it to 1.15. `auto` is the only style that defers to the model. Plenty of library models ship author-tuned samplers (qwen3 sets 0.6/0.95/20), so those win key by key and NOMAD fills only the gap, which in practice is min_p. A style the user picked by name overrides them, or it would silently do nothing on half their models. The compat transport drops min_p, so a profile carries a separate compatTopP. Passing top_p 1.0 to a backend with no min_p behind it would sample the raw distribution, which is worse than the 0.9 it replaced. Resolved in the controller, so the grammar-constrained task calls keep their temperature 0. An explicit temperature still outranks a profile's, which is what keeps the eval harness deterministic. `eval:generation --style` is new and load-bearing: temperature 0 makes min_p, top_p and top_k inert, so a default run cannot see this change at all. `--style=off` sends no samplers and no temperature, and is the control. Verified against Ollama 0.32.9, min_p accepted. 22 unit tests. |
||
|---|---|---|
| .. | ||
| eval | ||
| affirmative_remedies.ts | ||
| amd_hsa_override.ts | ||
| chat_images.ts | ||
| content_auto_update_backoff.ts | ||
| content_reindex_decision.ts | ||
| context_budget.ts | ||
| context_window.ts | ||
| downloads.ts | ||
| fs.ts | ||
| gpu_model.ts | ||
| hosted_content_auth.ts | ||
| hosted_content.ts | ||
| image_disk_preflight.ts | ||
| kb_ingest_decision.ts | ||
| kb_job_health.ts | ||
| kb_ratio_lookup.ts | ||
| kb_warning_decision.ts | ||
| misc.ts | ||
| model_capabilities.ts | ||
| platform_metadata.ts | ||
| queue_stall_options.ts | ||
| rag_prompt.ts | ||
| rag_relevance.ts | ||
| response_style.ts | ||
| sampler.ts | ||
| service_catalog_merge.ts | ||
| structured_output.ts | ||
| superseded_resource.ts | ||
| tasks_model.ts | ||
| think_stream.ts | ||
| token_estimate.ts | ||
| update_window.ts | ||
| version.ts | ||
| zim_batch_decision.ts | ||
| zim_download_resolution.ts | ||
| zim_filename.ts | ||
| zim_html.ts | ||