mirror of
https://github.com/Crosstalk-Solutions/project-nomad.git
synced 2026-07-29 11:44:38 +02:00
abed40a8c4
The Stored Files "partial stall" warning compares chunks in Qdrant against an expected count from the ratio registry. The registry has an empty-pattern catch-all (100 chunks/MB) that matches any filename, so a ZIM that matches no specific pattern still gets a size-based estimate. For archives that are mostly PDFs, images, or link-out stubs (e.g. irp.fas.org military-medicine), byte size wildly over-predicts embeddable text: a 75 MB ZIM estimates ~7,236 chunks but produces ~1, tripping a false "ingestion may have stalled" warning that re-embed can't clear. The catch-all is fine for rough aggregate disk-cost estimates, but it should not drive a per-file stall signal. Add an `ignoreCatchAll` option to the ratio lookup that excludes the empty-pattern row (returning null when only the fallback would match), and use it in the warnings path so partial_stall only fires when the registry has a *specific* expectation for the file. Files that match a real pattern (wikipedia_, devdocs_, ifixit_, ...) are unaffected; disk-cost/batch estimates keep using the fallback. Closes #913 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| downloads.ts | ||
| fs.ts | ||
| image_disk_preflight.ts | ||
| kb_ingest_decision.ts | ||
| kb_job_health.ts | ||
| kb_ratio_lookup.ts | ||
| kb_warning_decision.ts | ||
| misc.ts | ||
| update_window.ts | ||
| version.ts | ||
| zim_filename.ts | ||