project-nomad/admin/util/conditions.ts
caweis 762a4a12ad
Add offline FDA drug reference (labels, interaction view, conditions, remedies) (#1040)
* feat(drug-reference): offline FDA drug labels, conditions, and remedies

Adds an offline medical-reference feature with three coupled layers:

- Drug Reference: full-text search over openFDA drug-label indications,
  a detail page per label, and a side-by-side single-drug comparison view.
  A two-phase background pipeline downloads the openFDA label parts to the
  storage volume (resumable) and ingests them into the search table.
- Conditions ("When to use what"): a curated spine of first-aid situations
  that maps each situation to matching OTC drugs, each linking back to its
  Drug Reference detail page.
- Curated remedies: hand-authored natural and home-remedy entries drawn from
  US-government public-domain sources (NCCIH, CDC, MedlinePlus, FDA), shown
  with their source links and the same safety disclaimers as the rest of the
  feature.

The drug-reference and conditions layers are intentionally coupled: the drug
detail page shows the situations a drug treats, and the conditions controller
reads the same drug_labels table.

Safety surfaces ship as written. The amber SafetyBanner ("informational only,
not medical advice, not an FDA endorsement, not a drug-interaction checker, in
an emergency call emergency services") renders on the condition pages and the
search page; the detail and comparison pages carry their own "not a cross-drug
interaction checker" callout; and every page carries the openFDA CC0 source
citation and no-FDA-affiliation footer.

Wiring on this branch:
- start/routes.ts: the /drug-reference and /conditions page GETs plus their
  /api/* groups.
- commands/queue/work.ts: the drug-download and drug-ingest queues, both at
  concurrency 1. The two drug queues get a per-queue stall override
  (lockDuration 1_800_000, maxStalledCount 3) because each part is one long
  stream; every other queue keeps the existing 300000 default.
- inertia/pages/home.tsx: Drug Reference and "When to use what" tiles. The
  icon and display_order are a starting point, open to change.
- types/kv_store.ts: the two drugReference.* keys the pipeline reads and writes.
- package.json: yauzl and stream-json (plus their @types), used by the ingest
  job to stream the label JSON out of the downloaded zips.

The app reads the conditions and remedy data from the compiled TS constants in
app/data/; the repo-root collections/*.json files are the browseable mirrors.
The natural-remedies standalone test reads collections/natural_remedies.json to
assert the two stay in sync, so that file is also a test fixture.

The four standalone tests pass (drug_interactions, drug_ingest_status,
conditions, natural_remedies). tsc reports no errors in the feature code.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore(drug-reference): drop a fork-internal comment reference

* fix(drug-reference): address review — defer FULLTEXT, fallback remedies prop, strip fork refs

- migration: wrap the FULLTEXT ALTER in this.defer so it runs after the
  deferred createTable (was silently swallowed, index never created)
- controller: add remedies:[] to the index() error fallback (required prop)
- strip fork-internal issue/spec references from ported comments
- tsconfig: exclude tests/standalone (node --experimental-strip-types only)
- correct the varchar(768) byte-math comment; extend remedy-spine test

* fix(drug-reference): make the interaction comparison readable at five drugs

The comparison view laid its columns out on an equal-fraction CSS grid
(repeat(N, minmax(0, 1fr))), so each added drug shrank every column; at the
five-drug maximum the FDA interaction text was squeezed into unreadable slivers.

Lay the columns out with flex instead: full-width and stacked on phones, then
fixed-width columns that scroll sideways from the sm: breakpoint up, so they
never shrink below a readable width. Theme the columns with the same palette as
the rest of the page (they were on stock gray), and give the headers a fixed
min-height so columns line up when drug names wrap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(collections): route a 'dataset' tier resource to the drug pipeline

Add an optional `type` discriminator to SpecResource ('zim' | 'dataset',
absent == 'zim'), so the tier installer can carry a DB-ingested resource
alongside ZIM files. ZimService.downloadCategoryTier branches on it: a
'dataset' resource dispatches the existing FDA download+ingest pipeline
instead of RunDownloadJob, guarded against duplicate dispatch by the drug
ingest status. Every existing manifest entry has no `type` and keeps the
exact ZIM path.

Widen InstalledResource.resource_type to include 'dataset' and exclude
dataset rows from the ZIM/map catalog-update scan (datasets aren't
filename-versioned; their freshness path is separate). No dataset rows are
written yet: the InstalledResource 'dataset' row on ingest-ready, the
manifest entry, install-gating, and the downloads-aggregator integration
are follow-up commits.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(drug-reference): rework into an opt-in medicine-standard tier dataset

Reshapes the offline FDA drug reference from a built-in feature into curated
content installed by selecting the Medicine / Standard tier, per maintainer
direction.

- install-state: the ingest writes an installed_resources 'dataset' row on
  ready (version = the openFDA export_date), threaded installer to download to
  ingest; the tier-status math and the home-tile gate read it. Manual ingests
  write no row, so install-state stays tied to the curated path.
- manifest: declare the dataset in the medicine-standard tier (runtime fetches
  the remote manifest, so this also needs to land upstream).
- install-gating: the drug-reference home tiles render only when installed.
- uninstall: DrugReferenceService.uninstall() stops the two drug queues, deletes
  the on-disk parts, truncates drug_labels (schema kept), clears the KV markers,
  and drops the install row. Best-effort, logged, scoped to drug data only.
- downloads: the download phase reports the canonical {percent, downloadedBytes,
  totalBytes} shape as one drug-data card in the Active Downloads aggregator with
  cancel/remove; the heavy ingest stays in the IngestStatus surface with an
  Indexing handoff on the card.
- auto-update: a daily DrugAutoUpdateJob compares the manifest export_date and
  re-downloads when newer, gated on installed + no active job.

typecheck clean; the drug standalone suites pass. Three points are flagged in
code for the maintainer: the InstalledResource 'dataset' approach, the tier
home, and the export_date string format.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(downloads): keep the drug download card live across all parts

The Active Downloads card filtered to the deterministic jobId, but the
download's continuations run under auto-generated jobIds (only part 0 uses the
deterministic one). So the card tracked part 0 and then vanished while parts
2..N kept downloading. The queue is concurrency 1, so collapse to whichever
single part is in flight and report the deterministic jobId: one card tracks
aggregate progress through the whole download and cancel/remove still routes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(drug-reference): co-locate a persistent safety note with affirmative remedy guidance

Add RemedySafetyNote at the head of every natural-remedy section — the two on
Drug Reference and the one on "When to use what" — so the "informational only,
not medical advice, seek real medical care in an emergency" framing appears with
the guidance itself, not only in the page-top banner. Replaces the terse
per-section caveat with the same amber alert language as SafetyBanner.

Addresses the upstream #1040 review request that the disclaimer be unmistakable
and present wherever affirmative self-care guidance appears, not a one-time banner.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(drug-reference): fold dataset freshness into the content-auto-update path

The drug dataset auto-updated on its own daily cron that ignored the
contentAutoUpdate.* master switch, so it would refresh even with content
auto-update turned off, and it didn't ride the content-update path the way
ZIMs and maps do.

Move the export_date freshness/apply orchestration onto
DrugReferenceService.attemptAutoUpdate(), add
ContentAutoUpdateService.attemptDrugDataset() gated on the same enabled +
window config, and have the hourly ContentAutoUpdateJob drive both. Retire
the standalone DrugAutoUpdateJob. The ZIM/map attempt() path is unchanged.

Addresses the upstream #1040 request to wire the openFDA export_date check
into the content updater so the dataset updates alongside ZIMs and maps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(drug-reference): strengthen the remedy note to consult a clinician before combining with meds

Widen the affirmative-remedy safety note from "talk to a clinician before use"
to explicitly cover using a remedy AND combining one with a medication the user
already takes — the interaction case is the higher-risk path for an off-grid
user self-treating.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(drug-reference): gate affirmative remedy content behind an off-by-default flag

Add drugReference.remediesEnabled (default off), independent of the tier
install. When off, the server emits no remedy data at any boundary — the
drug-reference page prop, conditions show, and the /api/conditions/drugs
situation search — and the "Natural" filter is hidden, so installing the
medicine-standard tier lights up the verbatim FDA label search and the
condition-to-OTC matching but not the hand-authored self-care and herbal
sections. No user-facing toggle: it is flipped on after a clinician content-pass.

Implements the upstream #1040 split-by-risk request: the regulated label content
ships with the tier; the authored remedy guidance stays gated until sign-off.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Jake Turner <52841588+jakeaturner@users.noreply.github.com>
2026-08-04 03:38:33 +00:00

357 lines
13 KiB
TypeScript

/**
* "When to use what" — pure, unit-testable helpers (Phase 1 + Phase 2).
*
* NO Lucid / AdonisJS / HTTP imports. These take plain objects and return plain
* objects so they run standalone under `node --experimental-strip-types` without
* booting MySQL or Redis. Mirrors the `drug_labels.ts` helper convention.
*/
import type {
Condition,
ConditionsFile,
ConditionSummary,
NaturalRemedy,
NaturalRemediesFile,
} from '../types/conditions.js'
import type { DrugSearchResult } from '../types/drug_reference.js'
/**
* Canonical FDA product_type strings, mirrored from `PRODUCT_TYPES` in
* types/drug_reference.ts. Inlined here (not imported) so this module stays a
* pure, value-import-free helper runnable under `node --experimental-strip-types`
* (which does not rewrite `.js` specifiers to `.ts` for runtime value imports).
* The standalone test asserts these equal PRODUCT_TYPES.OTC / .RX, so they
* cannot drift from the canonical source.
*/
export const OTC_PRODUCT_TYPE = 'HUMAN OTC DRUG'
export const RX_PRODUCT_TYPE = 'HUMAN PRESCRIPTION DRUG'
// ─── Spine parsing (fail-soft) ────────────────────────────────────────────────
/** Trim a value to a non-empty string, or return null. */
function nonEmptyString(value: unknown): string | null {
if (typeof value !== 'string') return null
const trimmed = value.trim()
return trimmed.length > 0 ? trimmed : null
}
/**
* Validate + coerce one raw condition entry into a typed Condition.
*
* Returns null when the entry is structurally unusable (missing slug/label/
* category, or no usable searchTerms) so the caller can skip it rather than
* surface a broken row. searchTerms are de-duped (case-insensitive) and
* stripped of empties; an entry with zero usable terms is dropped (it could
* never match anything).
*/
export function parseConditionEntry(raw: unknown): Condition | null {
if (typeof raw !== 'object' || raw === null) return null
const r = raw as Record<string, unknown>
const slug = nonEmptyString(r.slug)
const label = nonEmptyString(r.label)
const category = nonEmptyString(r.category)
if (!slug || !label || !category) return null
if (!Array.isArray(r.searchTerms)) return null
const seen = new Set<string>()
const searchTerms: string[] = []
for (const term of r.searchTerms) {
const clean = nonEmptyString(term)
if (!clean) continue
const key = clean.toLowerCase()
if (seen.has(key)) continue
seen.add(key)
searchTerms.push(clean)
}
if (searchTerms.length === 0) return null
return { slug, label, category, searchTerms }
}
/**
* Defensively parse a ConditionsFile-shaped object.
*
* A non-object, a missing/empty `conditions` array, or every entry being
* malformed all yield an empty `conditions` list (with a best-effort version)
* rather than throwing — the page then degrades to free-text-only search
* instead of crashing. Duplicate slugs keep the FIRST occurrence (the curated
* order is intentional).
*/
export function parseConditionsFile(json: unknown): ConditionsFile {
if (typeof json !== 'object' || json === null) {
return { version: 'unknown', conditions: [] }
}
const root = json as Record<string, unknown>
const version = nonEmptyString(root.version) ?? 'unknown'
if (!Array.isArray(root.conditions)) {
return { version, conditions: [] }
}
const seenSlugs = new Set<string>()
const conditions: Condition[] = []
for (const raw of root.conditions) {
const entry = parseConditionEntry(raw)
if (!entry) continue
if (seenSlugs.has(entry.slug)) continue
seenSlugs.add(entry.slug)
conditions.push(entry)
}
return { version, conditions }
}
// ─── Lookup + projection ──────────────────────────────────────────────────────
/** Find a condition by its slug (exact match). Returns null when absent. */
export function findConditionBySlug(conditions: Condition[], slug: string): Condition | null {
const target = slug.trim()
if (!target) return null
return conditions.find((c) => c.slug === target) ?? null
}
/** Project a Condition to the client-facing summary (drops searchTerms). */
export function toConditionSummary(condition: Condition): ConditionSummary {
return {
slug: condition.slug,
label: condition.label,
category: condition.category,
}
}
// ─── Reverse link: indications text → curated situations ──────────────────────
/**
* The other direction of the symbiotic relationship: given a single drug's
* indications-and-usage text, find which curated situations it treats.
*
* A condition matches when ANY of its searchTerms appears as a case-insensitive
* substring of the indications text. Substring (not word-boundary or FULLTEXT)
* matching mirrors the FULLTEXT/LIKE intent of the forward search closely enough
* for a "Commonly used for" affordance, while staying a pure string operation
* with no DB. Curated order is preserved and results are deduped by slug.
*
* Blank/empty input → [] (a label with no indications treats nothing here).
*/
export function situationsForIndications(
indicationsText: string | null | undefined,
conditions: Condition[]
): ConditionSummary[] {
if (typeof indicationsText !== 'string') return []
const haystack = indicationsText.toLowerCase()
if (haystack.trim().length === 0) return []
const matched: ConditionSummary[] = []
const seen = new Set<string>()
for (const condition of conditions) {
if (seen.has(condition.slug)) continue
const hit = condition.searchTerms.some((term) => {
const needle = term.trim().toLowerCase()
return needle.length > 0 && haystack.includes(needle)
})
if (hit) {
seen.add(condition.slug)
matched.push(toConditionSummary(condition))
}
}
return matched
}
// ─── FULLTEXT query construction ──────────────────────────────────────────────
/**
* Build a MySQL NATURAL LANGUAGE MODE query string from a condition's
* searchTerms.
*
* - Multi-word terms are wrapped in double quotes so they match as a phrase
* ("sore throat") rather than as two loose tokens — this trades a little
* recall for precision on multi-word conditions.
* - Single-word terms are passed through bare.
* - Internal double-quotes inside a term are stripped (they would break the
* phrase quoting).
* - Terms are de-duped (case-insensitive) and empties dropped. An all-empty
* input yields '' so the caller can short-circuit to the LIKE fallback.
*
* NATURAL LANGUAGE MODE treats the space-joined terms as an OR-ish relevance
* query, which is what we want: any synonym can surface a matching label, ranked
* by relevance.
*/
export function buildIndicationQuery(searchTerms: string[]): string {
const seen = new Set<string>()
const parts: string[] = []
for (const raw of searchTerms) {
if (typeof raw !== 'string') continue
const term = raw.trim().replace(/"/g, '')
if (term.length === 0) continue
const key = term.toLowerCase()
if (seen.has(key)) continue
seen.add(key)
parts.push(/\s/.test(term) ? `"${term}"` : term)
}
return parts.join(' ')
}
// ─── Natural remedies (Phase 2) — parsing + lookup ───────────────────────────
/**
* Validate + coerce one raw remedy entry into a typed NaturalRemedy.
*
* Returns null when the entry is structurally unusable (missing required string
* fields or conditions array) so the caller can skip it rather than surface a
* broken row. Mirrors the defensive posture of parseConditionEntry.
*/
export function parseRemedyEntry(raw: unknown): NaturalRemedy | null {
if (typeof raw !== 'object' || raw === null) return null
const r = raw as Record<string, unknown>
const slug = nonEmptyString(r.slug)
const name = nonEmptyString(r.name)
const uses = nonEmptyString(r.uses)
const evidence = nonEmptyString(r.evidence)
const cautions = nonEmptyString(r.cautions)
const sourceUrl = nonEmptyString(r.sourceUrl)
if (!slug || !name || !uses || !evidence || !cautions || !sourceUrl) return null
if (!Array.isArray(r.commonNames)) return null
const commonNames: string[] = []
for (const cn of r.commonNames) {
const clean = nonEmptyString(cn)
if (clean) commonNames.push(clean)
}
if (!Array.isArray(r.conditions)) return null
const conditions: string[] = []
for (const c of r.conditions) {
const clean = nonEmptyString(c)
if (clean) conditions.push(clean)
}
// A remedy with no conditions never surfaces — drop it.
if (conditions.length === 0) return null
// Optional practical how-to (the offline-mission field) — carried when present.
const how = nonEmptyString(r.how)
return how
? { slug, name, commonNames, conditions, uses, how, evidence, cautions, sourceUrl }
: { slug, name, commonNames, conditions, uses, evidence, cautions, sourceUrl }
}
/**
* Defensively parse a NaturalRemediesFile-shaped object.
*
* A non-object, missing/empty `remedies` array, or all malformed entries yield
* an empty `remedies` list (with a best-effort version) rather than throwing —
* the UI degrades to "no natural remedies for this condition" instead of
* crashing. Duplicate remedy slugs keep the FIRST occurrence.
*/
export function parseNaturalRemediesFile(json: unknown): NaturalRemediesFile {
const empty: NaturalRemediesFile = {
version: 'unknown',
source: { name: '', url: '', license: '' },
remedies: [],
}
if (typeof json !== 'object' || json === null) return empty
const root = json as Record<string, unknown>
const version = nonEmptyString(root.version) ?? 'unknown'
let source = { name: '', url: '', license: '' }
if (typeof root.source === 'object' && root.source !== null) {
const s = root.source as Record<string, unknown>
source = {
name: nonEmptyString(s.name) ?? '',
url: nonEmptyString(s.url) ?? '',
license: nonEmptyString(s.license) ?? '',
}
}
if (!Array.isArray(root.remedies)) return { version, source, remedies: [] }
const seenSlugs = new Set<string>()
const remedies: NaturalRemedy[] = []
for (const raw of root.remedies) {
const entry = parseRemedyEntry(raw)
if (!entry) continue
if (seenSlugs.has(entry.slug)) continue
seenSlugs.add(entry.slug)
remedies.push(entry)
}
return { version, source, remedies }
}
/**
* Return all remedies whose `conditions` array includes the given slug.
*
* Stable order (preserves the curated order of the remedies array). Pure O(remedies)
* filter — no DB, no fuzzy matching. Returns [] when nothing matches or when file
* has no remedies.
*/
export function remediesForCondition(file: NaturalRemediesFile, slug: string): NaturalRemedy[] {
const target = slug.trim()
if (!target) return []
return file.remedies.filter((r) => r.conditions.includes(target))
}
/**
* Short, display-friendly source attribution for a remedy — plain text, NOT a
* link. NOMAD is offline-first: remedy cards carry everything needed to act and
* never link out to the internet; this credit satisfies the public-domain
* attribution without implying connectivity.
*/
export function remedySourceName(remedy: { kind?: 'herb' | 'self-care'; sourceUrl: string }): string {
if ((remedy.kind ?? 'herb') === 'herb') return 'NCCIH'
const url = remedy.sourceUrl.toLowerCase()
if (url.includes('cdc.gov')) return 'CDC'
if (url.includes('fda.gov')) return 'FDA'
if (url.includes('medlineplus.gov')) return 'MedlinePlus (NLM)'
if (url.includes('nih.gov')) return 'NIH'
return 'US government source'
}
/**
* Return all remedies matching a free-text query as a secondary fallback.
*
* Case-insensitive substring match over `name` and `uses`. Used when a free-text
* query does not resolve to a curated condition slug. Deduped by slug.
*/
export function remediesForFreeText(file: NaturalRemediesFile, query: string): NaturalRemedy[] {
const q = query.trim().toLowerCase()
if (!q) return []
const seen = new Set<string>()
const results: NaturalRemedy[] = []
for (const r of file.remedies) {
if (seen.has(r.slug)) continue
if (r.name.toLowerCase().includes(q) || r.uses.toLowerCase().includes(q)) {
seen.add(r.slug)
results.push(r)
}
}
return results
}
// ─── Result ordering ──────────────────────────────────────────────────────────
/**
* Stable-sort drug results OTC-first.
*
* The condition browser is consumer-facing, so over-the-counter products are
* the most actionable and lead. Order within each tier (OTC → other → Rx) is
* preserved from the input (which the service already relevance-ranks), so this
* only re-buckets by product_type without disturbing relevance order.
*
* Bucketing: OTC = 0, anything not Rx-and-not-OTC = 1, Rx = 2. Returns a new
* array; the input is not mutated.
*/
export function orderOtcFirst(drugs: DrugSearchResult[]): DrugSearchResult[] {
const rank = (d: DrugSearchResult): number => {
if (d.product_type === OTC_PRODUCT_TYPE) return 0
if (d.product_type === RX_PRODUCT_TYPE) return 2
return 1
}
return drugs
.map((d, i) => ({ d, i }))
.sort((a, b) => rank(a.d) - rank(b.d) || a.i - b.i)
.map((x) => x.d)
}