mirror of
https://github.com/Crosstalk-Solutions/project-nomad.git
synced 2026-10-17 05:14:19 +02:00
27be71b46e
Chat sent num_ctx, num_predict and nothing else, so every turn decoded at whatever the backend picked: temperature 0.8, top_p 0.9, no min_p. On a heavily quantized 1-3B model that is the wandering, repetitive output people read as "the local model is bad". Adds `ai.responseStyle` with four profiles. min_p is the lever, not temperature: it floors relative to the top token, so it truncates hard on a confident distribution and barely at all on a flat one, which is where small models start to wander. top_p stays at 1.0 rather than stacked on top of it. repeat_penalty moves with the style, because a grounded answer legitimately repeats a part number inside the 64-token window: `focused` relaxes it to 1.05, `creative` tightens it to 1.15. `auto` is the only style that defers to the model. Plenty of library models ship author-tuned samplers (qwen3 sets 0.6/0.95/20), so those win key by key and NOMAD fills only the gap, which in practice is min_p. A style the user picked by name overrides them, or it would silently do nothing on half their models. The compat transport drops min_p, so a profile carries a separate compatTopP. Passing top_p 1.0 to a backend with no min_p behind it would sample the raw distribution, which is worse than the 0.9 it replaced. Resolved in the controller, so the grammar-constrained task calls keep their temperature 0. An explicit temperature still outranks a profile's, which is what keeps the eval harness deterministic. `eval:generation --style` is new and load-bearing: temperature 0 makes min_p, top_p and top_k inert, so a default run cannot see this change at all. `--style=off` sends no samplers and no temperature, and is the control. Verified against Ollama 0.32.9, min_p accepted. 22 unit tests. |
||
|---|---|---|
| .. | ||
| amd_hsa_override.spec.ts | ||
| app_auto_update.spec.ts | ||
| chat_images.spec.ts | ||
| chat_request_compat.spec.ts | ||
| chat_stream_abort.spec.ts | ||
| chat_stream.spec.ts | ||
| cloud_metadata_url.spec.ts | ||
| content_auto_update_backoff.spec.ts | ||
| content_auto_update.spec.ts | ||
| content_reindex_decision.spec.ts | ||
| context_budget.spec.ts | ||
| context_window.spec.ts | ||
| curated_resource_schema.spec.ts | ||
| custom_app_guard.spec.ts | ||
| drug_ingest_status.spec.ts | ||
| drug_interactions.spec.ts | ||
| drug_labels.spec.ts | ||
| eval_generation_metrics.spec.ts | ||
| eval_golden_set.spec.ts | ||
| eval_report.spec.ts | ||
| eval_retrieval_metrics.spec.ts | ||
| eval_source_guard.spec.ts | ||
| global_map_banner.spec.ts | ||
| gpu_model.spec.ts | ||
| kb_file_grouping.spec.ts | ||
| kb_guardrail.spec.ts | ||
| kb_ingest_decision.spec.ts | ||
| kb_job_health.spec.ts | ||
| kb_ratio_lookup.spec.ts | ||
| kb_warning_decision.spec.ts | ||
| link_tile.spec.ts | ||
| ollama_compute.spec.ts | ||
| ollama_controller_chat.spec.ts | ||
| platform_metadata.spec.ts | ||
| private_url.spec.ts | ||
| queue_stall_options.spec.ts | ||
| rag_citations.spec.ts | ||
| rag_pipeline_prompt.spec.ts | ||
| rag_relevance.spec.ts | ||
| rag_retrieval_toggle.spec.ts | ||
| sampler.spec.ts | ||
| service_catalog_merge.spec.ts | ||
| structured_output.spec.ts | ||
| superseded_resource.spec.ts | ||
| tasks_model.spec.ts | ||
| think_stream.spec.ts | ||
| token_estimate.spec.ts | ||
| vision_guidance.spec.ts | ||
| zim_batch_decision.spec.ts | ||
| zim_download_resolution.spec.ts | ||
| zim_filename.spec.ts | ||
| zim_html.spec.ts | ||