mirror of
https://github.com/Crosstalk-Solutions/project-nomad.git
synced 2026-10-17 05:14:19 +02:00
Surfaces two silent failure modes that the prior binary "any-chunks-in-Qdrant ⇒ embedded" check could not distinguish from healthy ingestion: - **Warning A — Zero-chunk file** (file_size > 100 MB, chunks = 0) Fires on video-only / image-only ZIMs (`lrnselfreliance_en_all`, TED talks, etc.) that the pipeline completes "successfully" with no extractable text. AI Assistant literally cannot reference these. - **Warning B — Partial-embed stall** (chunks < 50% of expected from the ratio registry). Surfaces the simple_wiki "266 of 600,000 chunks" case observed during NOMAD1 ingestion testing — previously these looked identical to fully-completed embeds in the UI. Both warnings render only when their condition is met (silent by default; noisy only on real problems). Base is `feat/kb-ratio-registry` (#891) because Warning B's "expected chunks" estimate comes from `KbRatioRegistry.estimateChunks()`. GitHub fast-forwards to `rc` once #891 merges. - `app/utils/kb_warning_decision.ts` — pure `decideWarnings(inputs)` with thresholds (`100 MB`, `0.5×`) as exported constants. 10 unit tests cover the healthy case, both warnings, the under/at/over boundary, the registry-miss suppression, and the video-only registry case (`expectedChunks: 0` correctly skips Warning B). - `RagService.computeFileWarnings()` — single Qdrant scroll tallies chunks per source, filesystem walk fills in zero-chunk files, ratio registry estimates the expectation, decision function emits. - New endpoint `GET /api/rag/file-warnings` returns `Record<source, FileWarning[]>` (sources with no warnings are omitted, so the frontend can `warnings[source] ?? []` for clean defaults). - KB modal: warnings render inline under the file name as amber-tinted pills. Polled every 30s alongside the existing health check. - Warning C — chunks skipped due to length. PR #890 (#881 fix) prevents the silent drop at the embed boundary, so the underlying condition shouldn't fire anymore. If we still want to surface "we truncated N chunks to fit", that needs separate `skipped_count` tracking in EmbedFileJob — a Phase 2 follow-up. - Suppressing Warning B during active mid-ingestion. The user can cross- reference the Processing Queue to know it's in-flight; suppressing warnings while a job runs would mask real stalls where the job died mid-batch. Will revisit when per-card status is wired through. - Use of `kb_ingest_state.chunks_embedded` (#888) as the chunk count source. This PR uses Qdrant scroll directly so it can land independently of #888. - 10 new unit tests on `decideWarnings`, all pass - Type-check clean - Hot-patch + browser smoke test deferred until #891 lands (the ratio registry needs to exist in the DB for `estimateChunks()` to return non-null estimates — without it, only Warning A fires which is still useful but Warning B stays dormant)
126 lines
3.4 KiB
TypeScript
126 lines
3.4 KiB
TypeScript
import * as assert from 'node:assert/strict'
|
|
import { test } from 'node:test'
|
|
|
|
import { decideWarnings } from '../../app/utils/kb_warning_decision.js'
|
|
|
|
const MB = 1024 * 1024
|
|
|
|
test('healthy file: chunks present and on-target → no warnings', () => {
|
|
assert.deepEqual(
|
|
decideWarnings({
|
|
fileSizeBytes: 100 * MB,
|
|
chunksInQdrant: 11_000,
|
|
expectedChunks: 11_000,
|
|
}),
|
|
[]
|
|
)
|
|
})
|
|
|
|
test('healthy file: chunks slightly above expectation → no warnings', () => {
|
|
assert.deepEqual(
|
|
decideWarnings({
|
|
fileSizeBytes: 100 * MB,
|
|
chunksInQdrant: 12_000,
|
|
expectedChunks: 11_000,
|
|
}),
|
|
[]
|
|
)
|
|
})
|
|
|
|
test('Warning A: large file with 0 chunks (video-only ZIM)', () => {
|
|
assert.deepEqual(
|
|
decideWarnings({
|
|
fileSizeBytes: 5 * 1024 * MB,
|
|
chunksInQdrant: 0,
|
|
expectedChunks: 0,
|
|
}),
|
|
[{ kind: 'zero_chunks', fileSizeBytes: 5 * 1024 * MB }]
|
|
)
|
|
})
|
|
|
|
test('Warning A: small empty file is silently ignored (under 100 MB threshold)', () => {
|
|
// A user uploads a 5 KB placeholder.txt that produces nothing → not worth a banner
|
|
assert.deepEqual(
|
|
decideWarnings({
|
|
fileSizeBytes: 5 * 1024, // 5 KB
|
|
chunksInQdrant: 0,
|
|
expectedChunks: null,
|
|
}),
|
|
[]
|
|
)
|
|
})
|
|
|
|
test('Warning B: partial stall — chunks well below expectation', () => {
|
|
assert.deepEqual(
|
|
decideWarnings({
|
|
fileSizeBytes: 1000 * MB,
|
|
chunksInQdrant: 266,
|
|
expectedChunks: 600_000,
|
|
}),
|
|
[{ kind: 'partial_stall', chunksEmbedded: 266, chunksExpected: 600_000 }]
|
|
)
|
|
})
|
|
|
|
test('Warning B: chunks just under 50% of expected → triggers', () => {
|
|
assert.deepEqual(
|
|
decideWarnings({
|
|
fileSizeBytes: 100 * MB,
|
|
chunksInQdrant: 4_999,
|
|
expectedChunks: 10_000,
|
|
}),
|
|
[{ kind: 'partial_stall', chunksEmbedded: 4_999, chunksExpected: 10_000 }]
|
|
)
|
|
})
|
|
|
|
test('Warning B: chunks at exactly 50% of expected → does NOT trigger', () => {
|
|
// Strict less-than threshold leaves room for the boundary
|
|
assert.deepEqual(
|
|
decideWarnings({
|
|
fileSizeBytes: 100 * MB,
|
|
chunksInQdrant: 5_000,
|
|
expectedChunks: 10_000,
|
|
}),
|
|
[]
|
|
)
|
|
})
|
|
|
|
test('Warning B suppressed when expectedChunks is null (registry miss)', () => {
|
|
// Better to be silent than show a meaningless "266 of unknown" comparison
|
|
assert.deepEqual(
|
|
decideWarnings({
|
|
fileSizeBytes: 100 * MB,
|
|
chunksInQdrant: 266,
|
|
expectedChunks: null,
|
|
}),
|
|
[]
|
|
)
|
|
})
|
|
|
|
test('Warning B suppressed when expectedChunks is 0 (video-only registry entry)', () => {
|
|
// A `lrnselfreliance_` row in the registry says "expect 0 chunks". A real
|
|
// file matching it correctly producing 0 chunks must not trigger Warning B.
|
|
assert.deepEqual(
|
|
decideWarnings({
|
|
fileSizeBytes: 500 * MB,
|
|
chunksInQdrant: 0,
|
|
expectedChunks: 0,
|
|
}),
|
|
// Note: Warning A triggers here because file > 100 MB and chunks = 0
|
|
[{ kind: 'zero_chunks', fileSizeBytes: 500 * MB }]
|
|
)
|
|
})
|
|
|
|
test('Both warnings can fire on the same file in principle', () => {
|
|
// Edge case: huge file, 0 chunks, but ratio registry expected 100k.
|
|
// Warning A fires (large + zero), Warning B suppressed (chunksInQdrant must be > 0).
|
|
// This documents the chunksInQdrant > 0 guard on Warning B.
|
|
assert.deepEqual(
|
|
decideWarnings({
|
|
fileSizeBytes: 1000 * MB,
|
|
chunksInQdrant: 0,
|
|
expectedChunks: 100_000,
|
|
}),
|
|
[{ kind: 'zero_chunks', fileSizeBytes: 1000 * MB }]
|
|
)
|
|
})
|