project-nomad/admin/tests/unit
Matt Moyles 94fb876f1b
feat(rag): make the knowledge base embedding model configurable
Adds a `rag.embeddingModel` setting that selects the Ollama model the
knowledge base is embedded and searched with. EMBEDDING_MODELS lists
each supported model with its vector width, task prefixes and a
calibrated default relevance floor:

  nomic-embed-text:v1.5    768   default, unchanged
  bge-m3                  1024   multilingual
  qwen3-embedding:0.6b    1024   multilingual
  snowflake-arctic-embed2 1024   multilingual
  embeddinggemma           768   multilingual

An unset setting behaves exactly as before.

Score scales differ between models, so the relevance floor is now per
model: an unset `rag.minRelevance` resolves to the configured model's
floor, and the eval harness pins to it. Each floor was calibrated the
way RAG_MIN_FINAL_SCORE was, with `eval:retrieval --min-final-score`
sweeps. Under nomic's 0.62, bge-m3 loses a tenth of the golden set.

Switching models is followed by the existing Reset & Rebuild. The model
a collection was built with is recorded when it is created (collections
from before this change count as nomic), and until they match embed
jobs fail once with UnrecoverableError instead of retrying 30 times, and
search returns no context with a warning. This also catches switches
between models of the same width, which Qdrant would accept silently.

Embedding models whose names lack "embed" (bge-m3) are also kept out of
the chat model lists and recognised by GPU pacing, the configured model
is exempt from the VRAM unload sweep, and only nomic keeps the any-tag
installed-model fallback, since other tags of a family can differ in
width (qwen3-embedding:8b is 4096-d).
2026-10-09 13:54:32 -04:00
..
amd_hsa_override.spec.ts fix(gpu): keep an existing AMD HSA override across reinstall and update 2026-09-29 15:54:41 -07:00
app_auto_update.spec.ts feat(supply-depot): opt-in automatic updates for installed apps 2026-06-23 04:46:52 +00:00
chat_images.spec.ts fix(ai): probe a model the list reports without vision before disabling images 2026-10-05 14:25:28 -07:00
chat_request_compat.spec.ts feat(ai): support local vision attachments 2026-09-29 15:54:41 -07:00
chat_stream_abort.spec.ts fix(chat): abort abandoned response streams 2026-09-29 15:54:41 -07:00
chat_stream.spec.ts feat(ai): support local vision attachments 2026-09-29 15:54:41 -07:00
cloud_metadata_url.spec.ts fix(security): canonicalize hostnames to block IPv4-mapped IPv6 IMDS bypass 2026-05-20 10:16:00 -07:00
content_auto_update_backoff.spec.ts feat(content): opt-in automatic updates for installed ZIM & map content 2026-06-23 04:47:01 +00:00
content_auto_update.spec.ts feat(content): opt-in automatic updates for installed ZIM & map content 2026-06-23 04:47:01 +00:00
content_reindex_decision.spec.ts feat(content): opt-in automatic updates for installed ZIM & map content 2026-06-23 04:47:01 +00:00
context_budget.spec.ts fix(chat): stop capping answers at 1024 tokens and surface truncation 2026-09-29 15:54:41 -07:00
context_window.spec.ts feat(AI): budget the prompt and actually set the context window (#1253) 2026-09-29 15:54:41 -07:00
curated_resource_schema.spec.ts feat: support gated downloads for self-hosted curated content (#1172) 2026-08-04 03:39:55 +00:00
custom_app_guard.spec.ts feat: supply depot 2026-06-23 04:46:45 +00:00
custom_url.spec.ts fix: bound custom URL length before persistence 2026-09-29 15:54:41 -07:00
dark_mode_contrast.spec.ts fix(ui): cover dark accent interaction states 2026-09-29 15:54:41 -07:00
drug_ingest_status.spec.ts Add offline FDA drug reference (labels, interaction view, conditions, remedies) (#1040) 2026-08-04 03:38:33 +00:00
drug_install_row_reconcile.spec.ts fix(drug-reference): backfill the missing dataset install row on boot 2026-09-29 15:54:41 -07:00
drug_interactions.spec.ts Add offline FDA drug reference (labels, interaction view, conditions, remedies) (#1040) 2026-08-04 03:38:33 +00:00
drug_labels.spec.ts Add offline FDA drug reference (labels, interaction view, conditions, remedies) (#1040) 2026-08-04 03:38:33 +00:00
embedding_model.spec.ts feat(rag): make the knowledge base embedding model configurable 2026-10-09 13:54:32 -04:00
eval_generation_metrics.spec.ts feat(eval): standardized RAG quality evaluation harness (#1233) 2026-09-29 15:54:41 -07:00
eval_golden_set.spec.ts feat(eval): standardized RAG quality evaluation harness (#1233) 2026-09-29 15:54:41 -07:00
eval_report.spec.ts feat(eval): standardized RAG quality evaluation harness (#1233) 2026-09-29 15:54:41 -07:00
eval_retrieval_metrics.spec.ts feat(AI): let retrieval decline instead of making the prompt paper over it 2026-09-29 15:54:41 -07:00
eval_source_guard.spec.ts feat(eval): standardized RAG quality evaluation harness (#1233) 2026-09-29 15:54:41 -07:00
global_map_banner.spec.ts fix(UI): improve global map banner display logic (#702) 2026-05-20 10:16:00 -07:00
gpu_model.spec.ts fix(system): resolve a placeholder GPU name on the System page too 2026-09-29 15:54:41 -07:00
kb_file_grouping.spec.ts feat(kb): per-file active toggle + collection grouping/bulk toggle (#1286) 2026-09-29 15:54:41 -07:00
kb_guardrail.spec.ts feat(KB): guardrail modal at 50GB / 10%-free thresholds (RFC #883 §7) 2026-05-20 10:16:00 -07:00
kb_ingest_decision.spec.ts feat(KB): Always/Manual ingest policy toggle (RFC #883 §1/§4) (#894) 2026-05-20 10:16:00 -07:00
kb_job_health.spec.ts feat(KB): status pill + last-activity timestamp on Processing Queue (RFC #883 §5/§10) 2026-05-20 10:16:00 -07:00
kb_orphan_decision.spec.ts fix(kb): stop orphan sweep from purging an empty but present scan root 2026-09-29 15:54:41 -07:00
kb_ratio_lookup.spec.ts fix(KB): stop partial_stall warning firing on atypical ZIMs (link-out/PDF-heavy) 2026-06-23 04:46:54 +00:00
kb_warning_decision.spec.ts feat(KB): conditional warnings A + B on Stored Files (RFC #883 §6) 2026-05-20 10:16:00 -07:00
link_tile.spec.ts fix(dashboard): show the color picker swatches solid 2026-09-29 15:54:41 -07:00
ollama_compute.spec.ts fix(gpu): keep an existing AMD HSA override across reinstall and update 2026-09-29 15:54:41 -07:00
ollama_controller_chat.spec.ts test(ai): run the controller vision spec under Japa 2026-09-29 15:54:41 -07:00
ollama_done_reason.spec.ts fix(chat): stop capping answers at 1024 tokens and surface truncation 2026-09-29 15:54:41 -07:00
platform_metadata.spec.ts feat(benchmark): official multi-arch sysbench, resolved digest, platform metadata (#1158) 2026-08-04 03:39:53 +00:00
private_url.spec.ts fix(security): harden assertNotPrivateUrl with ipaddr.js + host normalization 2026-06-23 04:46:55 +00:00
queue_stall_options.spec.ts fix(queue): stop stalled-job recovery forking the embedding chain 2026-09-29 15:54:41 -07:00
rag_citations.spec.ts feat: show source document and date as citations under RAG chat answers 2026-09-29 15:54:41 -07:00
rag_pipeline_prompt.spec.ts feat(AI): budget the prompt and actually set the context window (#1253) 2026-09-29 15:54:41 -07:00
rag_relevance.spec.ts feat(AI): let retrieval decline instead of making the prompt paper over it 2026-09-29 15:54:41 -07:00
rag_retrieval_toggle.spec.ts feat(AI): add opt-in relevance check for off-topic retrieval 2026-09-29 15:54:41 -07:00
relevance_judge.spec.ts feat(AI): add opt-in relevance check for off-topic retrieval 2026-09-29 15:54:41 -07:00
sampler.spec.ts feat(AI): give chat a sampler instead of the backend's defaults 2026-09-29 15:54:41 -07:00
service_catalog_merge.spec.ts fix(seeder): derive ui_location from the merged config, not the live row 2026-09-29 15:54:41 -07:00
settings_amd_hsa_override.spec.ts fix(gpu): detect Ollama CPU fallback on AMD and offer the right fix 2026-09-29 15:54:41 -07:00
structured_output.spec.ts fix(AI): tell a broken grammar apart from a backend that never had one 2026-09-29 15:54:41 -07:00
superseded_resource.spec.ts fix(content): remove superseded curated map/ZIM files when a new version installs 2026-06-23 04:47:01 +00:00
tasks_model.spec.ts feat(AI): specify a model for ancillary tasks (#1244) 2026-09-29 15:54:41 -07:00
think_stream.spec.ts fix(AI): stop <think> output leaking into titles, chips, and the RAG query (#1254) 2026-09-29 15:54:41 -07:00
token_estimate.spec.ts feat(AI): budget the prompt and actually set the context window (#1253) 2026-09-29 15:54:41 -07:00
vision_guidance.spec.ts feat(ai): support local vision attachments 2026-09-29 15:54:41 -07:00
zim_batch_decision.spec.ts fix(rag): don't abandon a ZIM after one sparse batch 2026-09-29 15:54:41 -07:00
zim_download_resolution.spec.ts feat: support gated downloads for self-hosted curated content (#1172) 2026-08-04 03:39:55 +00:00
zim_filename.spec.ts fix(ZIM): preserve co-existing Wikipedia corpora on cleanup (#884) 2026-05-20 10:16:00 -07:00
zim_html.spec.ts chore(KB): filter non-content sections + render tables in ZIM extraction (#1044) 2026-08-04 03:37:30 +00:00