project-nomad/admin/app
Chris Sherwood 52d12a222e fix(kb): confine the orphan sweep to roots the scan actually walked
decideOrphans guards on the disk scan coming back empty, but the failure
mode it needs to defend against is per-root. _discoverKbFiles() walks
kb_uploads and zim in a loop and skips a missing root rather than
failing, because a fresh install legitimately has no kb_uploads yet.

So if the zim root is relocated, renamed, or not yet mounted (#1050),
the scan still returns a non-empty list from kb_uploads. The empty-scan
guard never fires, filterOrphanCandidates has no way to tell a root that
scanned clean from one that was never there, and every ZIM in the index
is classified as an orphan and purged in a single batch. On a
Wikipedia-scale knowledge base that is millions of vectors and a
multi-hour re-embed. It is reachable from the UI: scanAndSyncStorage is
POST /rag/sync, so the button pressed when the knowledge base looks
wrong is the button that empties it.

_discoverKbFilesWithRoots() now reports which roots were walked, and
filterOrphanCandidates allowlists against that list instead of the
configured roots. A root that wasn't there contributes no candidates; a
root that scanned clean and held no files still yields its orphans, so
the legitimate purge path is unchanged. The empty-scan guard becomes the
degenerate case of the same rule rather than a separate one.

Also builds the unit-test paths with join() instead of POSIX literals.
Written as '/data/...' the filterOrphanCandidates cases pass on Linux CI
and fail on Windows, where sep is backslash and no source ever matches a
prefix, which is exactly where they would be run while changing this.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-29 15:54:41 -07:00
..
controllers feat(AI): give chat a sampler instead of the backend's defaults 2026-09-29 15:54:41 -07:00
data refactor(drug-reference): make collections JSON the single source for curated data (#1130) 2026-08-04 03:38:35 +00:00
exceptions fix(Docs): documentation renderer fixes 2025-12-23 16:00:33 -08:00
jobs fix(drug-reference): backfill the missing dataset install row on boot 2026-09-29 15:54:41 -07:00
middleware fix(API): skip compression for Server-Sent Events (#798) 2026-05-20 10:16:00 -07:00
models feat(kb): per-file active toggle + collection grouping/bulk toggle (#1286) 2026-09-29 15:54:41 -07:00
services fix(kb): confine the orphan sweep to roots the scan actually walked 2026-09-29 15:54:41 -07:00
utils fix(kb): confine the orphan sweep to roots the scan actually walked 2026-09-29 15:54:41 -07:00
validators feat(AI): give chat a sampler instead of the backend's defaults 2026-09-29 15:54:41 -07:00