project-nomad/admin/app/utils
jakeaturner ae1eae78e7 fix(kb): stop orphan sweep from purging an empty but present scan root
Boot runs ensureDirectoryExists() on the zim root, so a separate volume
that failed to mount leaves an empty mountpoint behind. The scan walks it
without error and reports it as scanned. One upload in kb_uploads keeps
the overall scan non-empty, so the next manual sync purged the vectors
for every ZIM under that root.

decideOrphans() now decides per scanned root and returns
{ orphans, withheld }. A root keeps its indexed sources when:

- it holds no embeddable files (kiwix-library.xml does not count)
- the purge would remove more than 50% of its indexed sources and at
  least 5 of them, which points to a wrong or stale mount rather than
  hand deletions

scanAndSyncStorage() logs each withheld root and appends a note to the
sync result message, so the user sees a likely unmounted drive instead
of losing hours of embeddings.

Replaces the unit test that asserted the old empty-root purge with
regression coverage for #1378 and both guards.

Closes #1378
2026-09-29 15:54:41 -07:00
..
eval feat(AI): add opt-in relevance check for off-topic retrieval 2026-09-29 15:54:41 -07:00
affirmative_remedies.ts Add offline FDA drug reference (labels, interaction view, conditions, remedies) (#1040) 2026-08-04 03:38:33 +00:00
amd_hsa_override.ts fix(gpu): keep an existing AMD HSA override across reinstall and update 2026-09-29 15:54:41 -07:00
chat_images.ts feat(ai): support local vision attachments 2026-09-29 15:54:41 -07:00
content_auto_update_backoff.ts feat(content): opt-in automatic updates for installed ZIM & map content 2026-06-23 04:47:01 +00:00
content_reindex_decision.ts feat(content): opt-in automatic updates for installed ZIM & map content 2026-06-23 04:47:01 +00:00
context_budget.ts fix(chat): stop capping answers at 1024 tokens and surface truncation 2026-09-29 15:54:41 -07:00
context_window.ts feat(AI): budget the prompt and actually set the context window (#1253) 2026-09-29 15:54:41 -07:00
downloads.ts fix(downloads): don't retry a rejected entitlement for four hours (#1205) 2026-08-04 03:40:02 +00:00
drug_install_row_reconcile.ts fix(drug-reference): backfill the missing dataset install row on boot 2026-09-29 15:54:41 -07:00
fs.ts fix(rag): add proper .docx text extraction via mammoth (#1100) 2026-08-04 03:37:42 +00:00
gpu_model.ts fix(system): resolve a placeholder GPU name on the System page too 2026-09-29 15:54:41 -07:00
hosted_content_auth.ts feat: support gated downloads for self-hosted curated content (#1172) 2026-08-04 03:39:55 +00:00
hosted_content.ts feat: support gated downloads for self-hosted curated content (#1172) 2026-08-04 03:39:55 +00:00
image_disk_preflight.ts feat(supply-depot): opt-in automatic updates for installed apps 2026-06-23 04:46:52 +00:00
kb_ingest_decision.ts feat(KB): Always/Manual ingest policy toggle (RFC #883 §1/§4) (#894) 2026-05-20 10:16:00 -07:00
kb_job_health.ts feat(KB): status pill + last-activity timestamp on Processing Queue (RFC #883 §5/§10) 2026-05-20 10:16:00 -07:00
kb_orphan_decision.ts fix(kb): stop orphan sweep from purging an empty but present scan root 2026-09-29 15:54:41 -07:00
kb_ratio_lookup.ts fix(KB): stop partial_stall warning firing on atypical ZIMs (link-out/PDF-heavy) 2026-06-23 04:46:54 +00:00
kb_warning_decision.ts refactor(KB): move FileWarning to shared types/rag following existing convention 2026-05-20 10:16:00 -07:00
misc.ts feat(AI): let retrieval decline instead of making the prompt paper over it 2026-09-29 15:54:41 -07:00
model_capabilities.ts feat(ai): support local vision attachments 2026-09-29 15:54:41 -07:00
platform_metadata.ts feat(benchmark): official multi-arch sysbench, resolved digest, platform metadata (#1158) 2026-08-04 03:39:53 +00:00
queue_stall_options.ts fix(queue): stop stalled-job recovery forking the embedding chain 2026-09-29 15:54:41 -07:00
rag_prompt.ts feat: show source document and date as citations under RAG chat answers 2026-09-29 15:54:41 -07:00
rag_relevance.ts feat(AI): let retrieval decline instead of making the prompt paper over it 2026-09-29 15:54:41 -07:00
relevance_judge.ts feat(AI): add opt-in relevance check for off-topic retrieval 2026-09-29 15:54:41 -07:00
response_style.ts feat(AI): give chat a sampler instead of the backend's defaults 2026-09-29 15:54:41 -07:00
sampler.ts feat(AI): give chat a sampler instead of the backend's defaults 2026-09-29 15:54:41 -07:00
service_catalog_merge.ts fix(seeder): derive ui_location from the merged config, not the live row 2026-09-29 15:54:41 -07:00
structured_output.ts fix(AI): tell a broken grammar apart from a backend that never had one 2026-09-29 15:54:41 -07:00
superseded_resource.ts fix(content): remove superseded curated map/ZIM files when a new version installs 2026-06-23 04:47:01 +00:00
tasks_model.ts feat(AI): budget the prompt and actually set the context window (#1253) 2026-09-29 15:54:41 -07:00
think_stream.ts fix(AI): stop <think> output leaking into titles, chips, and the RAG query (#1254) 2026-09-29 15:54:41 -07:00
token_estimate.ts feat(AI): budget the prompt and actually set the context window (#1253) 2026-09-29 15:54:41 -07:00
update_window.ts feat(supply-depot): opt-in automatic updates for installed apps 2026-06-23 04:46:52 +00:00
version.ts feat: support for updating services 2026-03-11 14:08:09 -07:00
zim_batch_decision.ts fix(rag): don't abandon a ZIM after one sparse batch 2026-09-29 15:54:41 -07:00
zim_download_resolution.ts feat: support gated downloads for self-hosted curated content (#1172) 2026-08-04 03:39:55 +00:00
zim_filename.ts fix(ZIM): preserve co-existing Wikipedia corpora on cleanup (#884) 2026-05-20 10:16:00 -07:00
zim_html.ts chore(KB): filter non-content sections + render tables in ZIM extraction (#1044) 2026-08-04 03:37:30 +00:00