project-nomad/admin/tests/unit/kb_warning_decision.spec.ts
Chris Sherwood 563f86a22b feat(KB): conditional warnings A + B on Stored Files (RFC #883 §6)
Surfaces two silent failure modes that the prior binary
"any-chunks-in-Qdrant ⇒ embedded" check could not distinguish from
healthy ingestion:

- **Warning A — Zero-chunk file** (file_size > 100 MB, chunks = 0)
  Fires on video-only / image-only ZIMs (`lrnselfreliance_en_all`,
  TED talks, etc.) that the pipeline completes "successfully" with no
  extractable text. AI Assistant literally cannot reference these.

- **Warning B — Partial-embed stall** (chunks < 50% of expected from
  the ratio registry). Surfaces the simple_wiki "266 of 600,000 chunks"
  case observed during NOMAD1 ingestion testing — previously these
  looked identical to fully-completed embeds in the UI.

Both warnings render only when their condition is met (silent by
default; noisy only on real problems).

Base is `feat/kb-ratio-registry` (#891) because Warning B's "expected
chunks" estimate comes from `KbRatioRegistry.estimateChunks()`. GitHub
fast-forwards to `rc` once #891 merges.

- `app/utils/kb_warning_decision.ts` — pure `decideWarnings(inputs)`
  with thresholds (`100 MB`, `0.5×`) as exported constants. 10 unit
  tests cover the healthy case, both warnings, the under/at/over
  boundary, the registry-miss suppression, and the video-only registry
  case (`expectedChunks: 0` correctly skips Warning B).
- `RagService.computeFileWarnings()` — single Qdrant scroll tallies
  chunks per source, filesystem walk fills in zero-chunk files,
  ratio registry estimates the expectation, decision function emits.
- New endpoint `GET /api/rag/file-warnings` returns
  `Record<source, FileWarning[]>` (sources with no warnings are
  omitted, so the frontend can `warnings[source] ?? []` for clean
  defaults).
- KB modal: warnings render inline under the file name as amber-tinted
  pills. Polled every 30s alongside the existing health check.

- Warning C — chunks skipped due to length. PR #890 (#881 fix) prevents
  the silent drop at the embed boundary, so the underlying condition
  shouldn't fire anymore. If we still want to surface "we truncated
  N chunks to fit", that needs separate `skipped_count` tracking in
  EmbedFileJob — a Phase 2 follow-up.
- Suppressing Warning B during active mid-ingestion. The user can cross-
  reference the Processing Queue to know it's in-flight; suppressing
  warnings while a job runs would mask real stalls where the job died
  mid-batch. Will revisit when per-card status is wired through.
- Use of `kb_ingest_state.chunks_embedded` (#888) as the chunk count
  source. This PR uses Qdrant scroll directly so it can land
  independently of #888.

- 10 new unit tests on `decideWarnings`, all pass
- Type-check clean
- Hot-patch + browser smoke test deferred until #891 lands (the ratio
  registry needs to exist in the DB for `estimateChunks()` to return
  non-null estimates — without it, only Warning A fires which is still
  useful but Warning B stays dormant)
2026-05-20 10:16:00 -07:00

126 lines
3.4 KiB
TypeScript

import * as assert from 'node:assert/strict'
import { test } from 'node:test'
import { decideWarnings } from '../../app/utils/kb_warning_decision.js'
const MB = 1024 * 1024
test('healthy file: chunks present and on-target → no warnings', () => {
assert.deepEqual(
decideWarnings({
fileSizeBytes: 100 * MB,
chunksInQdrant: 11_000,
expectedChunks: 11_000,
}),
[]
)
})
test('healthy file: chunks slightly above expectation → no warnings', () => {
assert.deepEqual(
decideWarnings({
fileSizeBytes: 100 * MB,
chunksInQdrant: 12_000,
expectedChunks: 11_000,
}),
[]
)
})
test('Warning A: large file with 0 chunks (video-only ZIM)', () => {
assert.deepEqual(
decideWarnings({
fileSizeBytes: 5 * 1024 * MB,
chunksInQdrant: 0,
expectedChunks: 0,
}),
[{ kind: 'zero_chunks', fileSizeBytes: 5 * 1024 * MB }]
)
})
test('Warning A: small empty file is silently ignored (under 100 MB threshold)', () => {
// A user uploads a 5 KB placeholder.txt that produces nothing → not worth a banner
assert.deepEqual(
decideWarnings({
fileSizeBytes: 5 * 1024, // 5 KB
chunksInQdrant: 0,
expectedChunks: null,
}),
[]
)
})
test('Warning B: partial stall — chunks well below expectation', () => {
assert.deepEqual(
decideWarnings({
fileSizeBytes: 1000 * MB,
chunksInQdrant: 266,
expectedChunks: 600_000,
}),
[{ kind: 'partial_stall', chunksEmbedded: 266, chunksExpected: 600_000 }]
)
})
test('Warning B: chunks just under 50% of expected → triggers', () => {
assert.deepEqual(
decideWarnings({
fileSizeBytes: 100 * MB,
chunksInQdrant: 4_999,
expectedChunks: 10_000,
}),
[{ kind: 'partial_stall', chunksEmbedded: 4_999, chunksExpected: 10_000 }]
)
})
test('Warning B: chunks at exactly 50% of expected → does NOT trigger', () => {
// Strict less-than threshold leaves room for the boundary
assert.deepEqual(
decideWarnings({
fileSizeBytes: 100 * MB,
chunksInQdrant: 5_000,
expectedChunks: 10_000,
}),
[]
)
})
test('Warning B suppressed when expectedChunks is null (registry miss)', () => {
// Better to be silent than show a meaningless "266 of unknown" comparison
assert.deepEqual(
decideWarnings({
fileSizeBytes: 100 * MB,
chunksInQdrant: 266,
expectedChunks: null,
}),
[]
)
})
test('Warning B suppressed when expectedChunks is 0 (video-only registry entry)', () => {
// A `lrnselfreliance_` row in the registry says "expect 0 chunks". A real
// file matching it correctly producing 0 chunks must not trigger Warning B.
assert.deepEqual(
decideWarnings({
fileSizeBytes: 500 * MB,
chunksInQdrant: 0,
expectedChunks: 0,
}),
// Note: Warning A triggers here because file > 100 MB and chunks = 0
[{ kind: 'zero_chunks', fileSizeBytes: 500 * MB }]
)
})
test('Both warnings can fire on the same file in principle', () => {
// Edge case: huge file, 0 chunks, but ratio registry expected 100k.
// Warning A fires (large + zero), Warning B suppressed (chunksInQdrant must be > 0).
// This documents the chunksInQdrant > 0 guard on Warning B.
assert.deepEqual(
decideWarnings({
fileSizeBytes: 1000 * MB,
chunksInQdrant: 0,
expectedChunks: 100_000,
}),
[{ kind: 'zero_chunks', fileSizeBytes: 1000 * MB }]
)
})