mirror of
https://github.com/Crosstalk-Solutions/project-nomad.git
synced 2026-10-17 13:19:18 +02:00
* feat(drug-reference): offline FDA drug labels, conditions, and remedies
Adds an offline medical-reference feature with three coupled layers:
- Drug Reference: full-text search over openFDA drug-label indications,
a detail page per label, and a side-by-side single-drug comparison view.
A two-phase background pipeline downloads the openFDA label parts to the
storage volume (resumable) and ingests them into the search table.
- Conditions ("When to use what"): a curated spine of first-aid situations
that maps each situation to matching OTC drugs, each linking back to its
Drug Reference detail page.
- Curated remedies: hand-authored natural and home-remedy entries drawn from
US-government public-domain sources (NCCIH, CDC, MedlinePlus, FDA), shown
with their source links and the same safety disclaimers as the rest of the
feature.
The drug-reference and conditions layers are intentionally coupled: the drug
detail page shows the situations a drug treats, and the conditions controller
reads the same drug_labels table.
Safety surfaces ship as written. The amber SafetyBanner ("informational only,
not medical advice, not an FDA endorsement, not a drug-interaction checker, in
an emergency call emergency services") renders on the condition pages and the
search page; the detail and comparison pages carry their own "not a cross-drug
interaction checker" callout; and every page carries the openFDA CC0 source
citation and no-FDA-affiliation footer.
Wiring on this branch:
- start/routes.ts: the /drug-reference and /conditions page GETs plus their
/api/* groups.
- commands/queue/work.ts: the drug-download and drug-ingest queues, both at
concurrency 1. The two drug queues get a per-queue stall override
(lockDuration 1_800_000, maxStalledCount 3) because each part is one long
stream; every other queue keeps the existing 300000 default.
- inertia/pages/home.tsx: Drug Reference and "When to use what" tiles. The
icon and display_order are a starting point, open to change.
- types/kv_store.ts: the two drugReference.* keys the pipeline reads and writes.
- package.json: yauzl and stream-json (plus their @types), used by the ingest
job to stream the label JSON out of the downloaded zips.
The app reads the conditions and remedy data from the compiled TS constants in
app/data/; the repo-root collections/*.json files are the browseable mirrors.
The natural-remedies standalone test reads collections/natural_remedies.json to
assert the two stay in sync, so that file is also a test fixture.
The four standalone tests pass (drug_interactions, drug_ingest_status,
conditions, natural_remedies). tsc reports no errors in the feature code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore(drug-reference): drop a fork-internal comment reference
* fix(drug-reference): address review — defer FULLTEXT, fallback remedies prop, strip fork refs
- migration: wrap the FULLTEXT ALTER in this.defer so it runs after the
deferred createTable (was silently swallowed, index never created)
- controller: add remedies:[] to the index() error fallback (required prop)
- strip fork-internal issue/spec references from ported comments
- tsconfig: exclude tests/standalone (node --experimental-strip-types only)
- correct the varchar(768) byte-math comment; extend remedy-spine test
* fix(drug-reference): make the interaction comparison readable at five drugs
The comparison view laid its columns out on an equal-fraction CSS grid
(repeat(N, minmax(0, 1fr))), so each added drug shrank every column; at the
five-drug maximum the FDA interaction text was squeezed into unreadable slivers.
Lay the columns out with flex instead: full-width and stacked on phones, then
fixed-width columns that scroll sideways from the sm: breakpoint up, so they
never shrink below a readable width. Theme the columns with the same palette as
the rest of the page (they were on stock gray), and give the headers a fixed
min-height so columns line up when drug names wrap.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(collections): route a 'dataset' tier resource to the drug pipeline
Add an optional `type` discriminator to SpecResource ('zim' | 'dataset',
absent == 'zim'), so the tier installer can carry a DB-ingested resource
alongside ZIM files. ZimService.downloadCategoryTier branches on it: a
'dataset' resource dispatches the existing FDA download+ingest pipeline
instead of RunDownloadJob, guarded against duplicate dispatch by the drug
ingest status. Every existing manifest entry has no `type` and keeps the
exact ZIM path.
Widen InstalledResource.resource_type to include 'dataset' and exclude
dataset rows from the ZIM/map catalog-update scan (datasets aren't
filename-versioned; their freshness path is separate). No dataset rows are
written yet: the InstalledResource 'dataset' row on ingest-ready, the
manifest entry, install-gating, and the downloads-aggregator integration
are follow-up commits.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(drug-reference): rework into an opt-in medicine-standard tier dataset
Reshapes the offline FDA drug reference from a built-in feature into curated
content installed by selecting the Medicine / Standard tier, per maintainer
direction.
- install-state: the ingest writes an installed_resources 'dataset' row on
ready (version = the openFDA export_date), threaded installer to download to
ingest; the tier-status math and the home-tile gate read it. Manual ingests
write no row, so install-state stays tied to the curated path.
- manifest: declare the dataset in the medicine-standard tier (runtime fetches
the remote manifest, so this also needs to land upstream).
- install-gating: the drug-reference home tiles render only when installed.
- uninstall: DrugReferenceService.uninstall() stops the two drug queues, deletes
the on-disk parts, truncates drug_labels (schema kept), clears the KV markers,
and drops the install row. Best-effort, logged, scoped to drug data only.
- downloads: the download phase reports the canonical {percent, downloadedBytes,
totalBytes} shape as one drug-data card in the Active Downloads aggregator with
cancel/remove; the heavy ingest stays in the IngestStatus surface with an
Indexing handoff on the card.
- auto-update: a daily DrugAutoUpdateJob compares the manifest export_date and
re-downloads when newer, gated on installed + no active job.
typecheck clean; the drug standalone suites pass. Three points are flagged in
code for the maintainer: the InstalledResource 'dataset' approach, the tier
home, and the export_date string format.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(downloads): keep the drug download card live across all parts
The Active Downloads card filtered to the deterministic jobId, but the
download's continuations run under auto-generated jobIds (only part 0 uses the
deterministic one). So the card tracked part 0 and then vanished while parts
2..N kept downloading. The queue is concurrency 1, so collapse to whichever
single part is in flight and report the deterministic jobId: one card tracks
aggregate progress through the whole download and cancel/remove still routes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(drug-reference): co-locate a persistent safety note with affirmative remedy guidance
Add RemedySafetyNote at the head of every natural-remedy section — the two on
Drug Reference and the one on "When to use what" — so the "informational only,
not medical advice, seek real medical care in an emergency" framing appears with
the guidance itself, not only in the page-top banner. Replaces the terse
per-section caveat with the same amber alert language as SafetyBanner.
Addresses the upstream #1040 review request that the disclaimer be unmistakable
and present wherever affirmative self-care guidance appears, not a one-time banner.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(drug-reference): fold dataset freshness into the content-auto-update path
The drug dataset auto-updated on its own daily cron that ignored the
contentAutoUpdate.* master switch, so it would refresh even with content
auto-update turned off, and it didn't ride the content-update path the way
ZIMs and maps do.
Move the export_date freshness/apply orchestration onto
DrugReferenceService.attemptAutoUpdate(), add
ContentAutoUpdateService.attemptDrugDataset() gated on the same enabled +
window config, and have the hourly ContentAutoUpdateJob drive both. Retire
the standalone DrugAutoUpdateJob. The ZIM/map attempt() path is unchanged.
Addresses the upstream #1040 request to wire the openFDA export_date check
into the content updater so the dataset updates alongside ZIMs and maps.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(drug-reference): strengthen the remedy note to consult a clinician before combining with meds
Widen the affirmative-remedy safety note from "talk to a clinician before use"
to explicitly cover using a remedy AND combining one with a medication the user
already takes — the interaction case is the higher-risk path for an off-grid
user self-treating.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(drug-reference): gate affirmative remedy content behind an off-by-default flag
Add drugReference.remediesEnabled (default off), independent of the tier
install. When off, the server emits no remedy data at any boundary — the
drug-reference page prop, conditions show, and the /api/conditions/drugs
situation search — and the "Natural" filter is hidden, so installing the
medicine-standard tier lights up the verbatim FDA label search and the
condition-to-OTC matching but not the hand-authored self-care and herbal
sections. No user-facing toggle: it is flipped on after a clinician content-pass.
Implements the upstream #1040 split-by-risk request: the regulated label content
ships with the tier; the authored remedy guidance stays gated until sign-off.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Jake Turner <52841588+jakeaturner@users.noreply.github.com>
330 lines
12 KiB
TypeScript
330 lines
12 KiB
TypeScript
import db from '@adonisjs/lucid/services/db'
|
|
import logger from '@adonisjs/core/services/logger'
|
|
import { CONDITIONS_FILE } from '../data/conditions.js'
|
|
import { NATURAL_REMEDIES_FILE } from '../data/natural_remedies.js'
|
|
import { HOME_REMEDIES_FILE } from '../data/home_remedies.js'
|
|
import {
|
|
parseConditionsFile,
|
|
parseNaturalRemediesFile,
|
|
findConditionBySlug,
|
|
toConditionSummary,
|
|
buildIndicationQuery,
|
|
orderOtcFirst,
|
|
remediesForCondition,
|
|
remediesForFreeText,
|
|
} from '../../util/conditions.js'
|
|
import { PRODUCT_TYPES } from '../../types/drug_reference.js'
|
|
import type {
|
|
Condition,
|
|
ConditionSummary,
|
|
ConditionDrugsResult,
|
|
NaturalRemedy,
|
|
NaturalRemediesFile,
|
|
} from '../../types/conditions.js'
|
|
import type { DrugSearchResult } from '../../types/drug_reference.js'
|
|
|
|
/**
|
|
* "When to use what" — condition-first service (Phase 1 + Phase 2).
|
|
*
|
|
* Resolves a curated condition (or a free-text situation) to the OTC drugs whose
|
|
* FDA label indications match it, plus the natural remedies (NCCIH) whose curated
|
|
* condition mapping includes the resolved slug.
|
|
*
|
|
* Reuses the Drug Reference indication-search machinery: the same combined-FULLTEXT
|
|
* index (ft_drug_labels_name_indications), the same MAX(MATCH …) aggregate pattern
|
|
* required under MySQL 8 ONLY_FULL_GROUP_BY, and the same brand+generic collapse —
|
|
* then re-buckets results OTC-first.
|
|
*
|
|
* Natural remedies are in-memory (no DB table, no migration) — one module-level
|
|
* parse of NATURAL_REMEDIES_FILE, reused across all requests.
|
|
*/
|
|
|
|
/**
|
|
* Module-level merged remedies corpus (fail-soft): the NCCIH herbs plus the
|
|
* non-herbal home-care measures (CDC/NIH/FDA), each entry tagged with its kind
|
|
* so the UI can badge them apart. Slugs are disjoint between the two files
|
|
* (validated at curation time).
|
|
*/
|
|
const HERB_FILE = parseNaturalRemediesFile(NATURAL_REMEDIES_FILE)
|
|
const HOME_FILE = parseNaturalRemediesFile(HOME_REMEDIES_FILE)
|
|
const REMEDIES_FILE: NaturalRemediesFile = {
|
|
version: HERB_FILE.version,
|
|
source: HERB_FILE.source,
|
|
remedies: [
|
|
...HERB_FILE.remedies.map((r) => ({ ...r, kind: 'herb' as const })),
|
|
...HOME_FILE.remedies.map((r) => ({ ...r, kind: 'self-care' as const })),
|
|
],
|
|
}
|
|
|
|
export class ConditionService {
|
|
/** Parsed (fail-soft) curated spine — bad entries are dropped, not fatal. */
|
|
private get spine(): Condition[] {
|
|
return parseConditionsFile(CONDITIONS_FILE).conditions
|
|
}
|
|
|
|
/** All curated conditions as client-facing summaries (no searchTerms). */
|
|
listConditions(): ConditionSummary[] {
|
|
return this.spine.map(toConditionSummary)
|
|
}
|
|
|
|
/**
|
|
* The full curated natural-remedies list (NCCIH herbs). Small enough to ship
|
|
* as page props, which lets the Drug Reference search match remedies by name
|
|
* client-side and offer a "Natural" browse with no extra round trip.
|
|
*/
|
|
listRemedies(): NaturalRemedy[] {
|
|
return REMEDIES_FILE.remedies
|
|
}
|
|
|
|
/**
|
|
* The full curated spine (WITH searchTerms) for server-side matching such as
|
|
* the drug-detail reverse link (situationsForIndications). Stays server-only —
|
|
* searchTerms are a search-implementation detail the client never receives.
|
|
*/
|
|
allConditions(): Condition[] {
|
|
return this.spine
|
|
}
|
|
|
|
/** Find a curated condition by slug. Returns null when absent. */
|
|
findCondition(slug: string): Condition | null {
|
|
return findConditionBySlug(this.spine, slug)
|
|
}
|
|
|
|
/**
|
|
* Resolve a curated condition (by slug) to its matching OTC drugs and natural
|
|
* remedies. Returns null when the slug is not in the curated spine so the
|
|
* controller can 404. Drugs are OTC-only and OTC-first-ordered.
|
|
*/
|
|
async drugsForSlug(
|
|
slug: string,
|
|
limit = 50,
|
|
opts?: { route?: string; sort?: 'relevance' | 'name' }
|
|
): Promise<ConditionDrugsResult | null> {
|
|
const condition = this.findCondition(slug)
|
|
if (!condition) return null
|
|
|
|
const drugs = await this.searchIndications(condition.searchTerms, limit, opts)
|
|
const remedies: NaturalRemedy[] = remediesForCondition(REMEDIES_FILE, slug)
|
|
return { condition: toConditionSummary(condition), drugs, remedies }
|
|
}
|
|
|
|
/**
|
|
* Resolve a free-text situation (off-list condition) to matching OTC drugs and
|
|
* natural remedies. Treats the raw query as a single search term. Returns a
|
|
* synthetic condition summary echoing the query so the UI can render a consistent
|
|
* header.
|
|
*
|
|
* Remedy resolution for free text:
|
|
* 1. Try to resolve the query to a curated condition slug (exact or substring
|
|
* match on slug/label). If found, use remediesForCondition — this is the
|
|
* primary path (e.g. "burns" typed free-form still finds aloe/tea-tree).
|
|
* 2. Secondary fallback: remediesForFreeText searches remedy name/uses by
|
|
* substring so an unmapped query ("athlete's foot") can still surface a
|
|
* relevant remedy. Results from both paths are unioned and de-duped by slug.
|
|
*/
|
|
async drugsForFreeText(
|
|
query: string,
|
|
limit = 50,
|
|
opts?: { route?: string; sort?: 'relevance' | 'name' }
|
|
): Promise<ConditionDrugsResult> {
|
|
const trimmed = query.trim()
|
|
const drugs = trimmed.length > 0 ? await this.searchIndications([trimmed], limit, opts) : []
|
|
|
|
// Phase 2: union condition-mapped + free-text-matched remedies.
|
|
const remedyMap = new Map<string, NaturalRemedy>()
|
|
if (trimmed.length > 0) {
|
|
// Primary: resolve to a curated condition slug the same way the client-side
|
|
// matchSituation helper does (case-insensitive exact/substring on slug+label).
|
|
const q = trimmed.toLowerCase()
|
|
const matchedCondition =
|
|
this.spine.find((c) => c.slug.toLowerCase() === q || c.label.toLowerCase() === q) ??
|
|
this.spine.find(
|
|
(c) =>
|
|
c.label.toLowerCase().includes(q) || c.slug.replace(/-/g, ' ').toLowerCase().includes(q)
|
|
) ??
|
|
null
|
|
if (matchedCondition) {
|
|
for (const r of remediesForCondition(REMEDIES_FILE, matchedCondition.slug)) {
|
|
remedyMap.set(r.slug, r)
|
|
}
|
|
}
|
|
// Secondary: name/uses substring fallback.
|
|
for (const r of remediesForFreeText(REMEDIES_FILE, trimmed)) {
|
|
if (!remedyMap.has(r.slug)) remedyMap.set(r.slug, r)
|
|
}
|
|
}
|
|
|
|
return {
|
|
condition: { slug: '', label: trimmed, category: 'Search' },
|
|
drugs,
|
|
remedies: Array.from(remedyMap.values()),
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Core search: FULLTEXT over (searchable_name, indications) for the OR-expanded
|
|
* searchTerms, OTC-filtered, collapsed by brand+generic, OTC-first-ordered.
|
|
*
|
|
* Strategy mirrors DrugReferenceService:
|
|
* 1. FULLTEXT NATURAL LANGUAGE MODE on the built query (requires the query
|
|
* to have a >= 3-char token; innodb_ft_min_token_size = 3).
|
|
* 2. LIKE fallback when the built query is empty/too short OR FULLTEXT throws
|
|
* (e.g. the index is absent) — runs each term as a LIKE clause.
|
|
* Both paths force product_type = OTC and then orderOtcFirst (a no-op on the
|
|
* already-OTC set, but kept so a future relaxation of the OTC filter stays
|
|
* correctly ordered).
|
|
*/
|
|
async searchIndications(
|
|
searchTerms: string[],
|
|
limit = 50,
|
|
opts?: { route?: string; sort?: 'relevance' | 'name' }
|
|
): Promise<DrugSearchResult[]> {
|
|
const ftQuery = buildIndicationQuery(searchTerms)
|
|
// A FULLTEXT query needs a token >= innodb_ft_min_token_size (3). If the
|
|
// longest bare token is shorter, NATURAL LANGUAGE MODE returns nothing, so
|
|
// go straight to LIKE.
|
|
const longestToken = ftQuery
|
|
.replace(/"/g, ' ')
|
|
.split(/\s+/)
|
|
.reduce((max, t) => Math.max(max, t.length), 0)
|
|
const useFulltext = ftQuery.length > 0 && longestToken >= 3
|
|
|
|
if (useFulltext) {
|
|
try {
|
|
const rows = await this.searchIndicationFulltext(ftQuery, limit, opts)
|
|
return orderOtcFirst(rows)
|
|
} catch (err) {
|
|
logger.warn(
|
|
`[ConditionService] FULLTEXT indication search failed, falling back to LIKE: ${
|
|
err instanceof Error ? err.message : String(err)
|
|
}`
|
|
)
|
|
}
|
|
}
|
|
|
|
const rows = await this.searchIndicationLike(searchTerms, limit, opts)
|
|
return orderOtcFirst(rows)
|
|
}
|
|
|
|
/**
|
|
* FULLTEXT OTC indication search.
|
|
*
|
|
* MATCHes over (searchable_name, indications) — must exactly match the
|
|
* ft_drug_labels_name_indications index column list. The MAX(MATCH …) wrapper
|
|
* is LOAD-BEARING: MySQL 8 ONLY_FULL_GROUP_BY rejects a bare MATCH() in SELECT
|
|
* under GROUP BY; wrapping it in MAX() makes it an aggregate. Copied verbatim
|
|
* from DrugReferenceService.searchIndicationFulltext, with a fixed OTC filter.
|
|
*/
|
|
private async searchIndicationFulltext(
|
|
ftQuery: string,
|
|
limit: number,
|
|
opts?: { route?: string; sort?: 'relevance' | 'name' }
|
|
): Promise<DrugSearchResult[]> {
|
|
let sql = `
|
|
SELECT
|
|
MIN(id) AS id,
|
|
brand_name,
|
|
generic_name,
|
|
MIN(manufacturer) AS manufacturer,
|
|
MIN(route) AS route,
|
|
MIN(product_type) AS product_type,
|
|
COUNT(*) AS labelCount,
|
|
MAX(MATCH(searchable_name, indications) AGAINST(? IN NATURAL LANGUAGE MODE)) AS relevance
|
|
FROM drug_labels
|
|
WHERE MATCH(searchable_name, indications) AGAINST(? IN NATURAL LANGUAGE MODE)
|
|
AND product_type = ?
|
|
`
|
|
const bindings: unknown[] = [ftQuery, ftQuery, PRODUCT_TYPES.OTC]
|
|
|
|
if (opts?.route) {
|
|
sql += ' AND route LIKE ?'
|
|
bindings.push(`%${opts.route.toUpperCase()}%`)
|
|
}
|
|
|
|
sql += `
|
|
GROUP BY brand_name, generic_name
|
|
ORDER BY ${opts?.sort === 'name' ? 'COALESCE(brand_name, generic_name) ASC' : 'relevance DESC'}
|
|
LIMIT ?
|
|
`
|
|
bindings.push(limit)
|
|
|
|
const rows = await db.rawQuery(sql, bindings)
|
|
return this.mapSearchRows(rows[0])
|
|
}
|
|
|
|
/**
|
|
* LIKE OTC indication fallback (FULLTEXT unavailable or query too short).
|
|
*
|
|
* ORs a LIKE clause per search term over `indications` so any synonym can
|
|
* surface a match. OTC-filtered and collapsed identically to the FULLTEXT path.
|
|
*/
|
|
private async searchIndicationLike(
|
|
searchTerms: string[],
|
|
limit: number,
|
|
opts?: { route?: string; sort?: 'relevance' | 'name' }
|
|
): Promise<DrugSearchResult[]> {
|
|
const terms = searchTerms.map((t) => t.trim()).filter((t) => t.length > 0)
|
|
if (terms.length === 0) return []
|
|
|
|
const likeClauses = terms.map(() => 'indications LIKE ?').join(' OR ')
|
|
let sql = `
|
|
SELECT
|
|
MIN(id) AS id,
|
|
brand_name,
|
|
generic_name,
|
|
MIN(manufacturer) AS manufacturer,
|
|
MIN(route) AS route,
|
|
MIN(product_type) AS product_type,
|
|
COUNT(*) AS labelCount
|
|
FROM drug_labels
|
|
WHERE (${likeClauses})
|
|
AND product_type = ?
|
|
`
|
|
const bindings: unknown[] = [...terms.map((t) => `%${t}%`), PRODUCT_TYPES.OTC]
|
|
|
|
if (opts?.route) {
|
|
sql += ' AND route LIKE ?'
|
|
bindings.push(`%${opts.route.toUpperCase()}%`)
|
|
}
|
|
|
|
sql += `
|
|
GROUP BY brand_name, generic_name
|
|
ORDER BY ${opts?.sort === 'name' ? 'COALESCE(brand_name, generic_name) ASC' : 'brand_name ASC'}
|
|
LIMIT ?
|
|
`
|
|
bindings.push(limit)
|
|
|
|
const rows = await db.rawQuery(sql, bindings)
|
|
return this.mapSearchRows(rows[0])
|
|
}
|
|
|
|
/** Map raw collapsed rows to DrugSearchResult — mirrors DrugReferenceService. */
|
|
private mapSearchRows(rows: any[]): DrugSearchResult[] {
|
|
if (!Array.isArray(rows)) return []
|
|
return rows.map((row) => ({
|
|
id: Number(row.id),
|
|
brand_name: row.brand_name ?? null,
|
|
generic_name: row.generic_name ?? null,
|
|
manufacturer: row.manufacturer ?? null,
|
|
route: row.route ?? null,
|
|
product_type: row.product_type ?? null,
|
|
labelCount: Number(row.labelCount ?? row.labelcount ?? 1),
|
|
}))
|
|
}
|
|
|
|
/**
|
|
* Current drug_labels row count — drives the index page empty-state (no data →
|
|
* point the user to Drug Reference to download first). Mirrors
|
|
* DrugReferenceService.rowCount; returns 0 on any error.
|
|
*/
|
|
async drugRowCount(): Promise<number> {
|
|
try {
|
|
const result = await db.rawQuery('SELECT COUNT(*) AS cnt FROM drug_labels')
|
|
const rows = result[0] as Array<{ cnt: number | string }>
|
|
return Number(rows[0]?.cnt ?? 0)
|
|
} catch {
|
|
return 0
|
|
}
|
|
}
|
|
}
|