project-nomad/admin
Chris Sherwood f8794d2bbf fix(benchmark): warm the AI model before timed runs for reproducible scores
The AI benchmark evicted resident models (forcing the benchmark model cold)
and then went straight into the timed median-of-N loop with no warm-up. On a
cold box the model-load + GPU spin-up cost landed inside the timed runs
(observed: 173s TTFT / 5.83 tok/s vs a ~80 tok/s warm steady-state), and
consecutive runs weren't isolated (a prior run left the model warm). Because
the AI channel is uncapped and ~30% of the composite, the same machine could
post a ~2x-different NOMAD Score depending on warm/cold state (888 vs 1958
observed back-to-back).

Add one discarded warm-up inference after eviction and before the timed loop
so every timed run measures warm, steady-state throughput. Cold and warm
invocations now converge on the same score. Best-effort: a warm-up hiccup
never fails the run.

Closes #1139
2026-07-23 14:50:23 -07:00
..
app fix(benchmark): warm the AI model before timed runs for reproducible scores 2026-07-23 14:50:23 -07:00
bin feat: curated content system overhaul 2026-02-11 15:44:46 -08:00
commands Add offline FDA drug reference (labels, interaction view, conditions, remedies) (#1040) 2026-07-21 11:06:18 -07:00
config fix: lazy-connect + retry for ioredis to avoid blocked startup 2026-06-23 17:40:10 +00:00
constants fix(KVStore): missing apps.homebox key 2026-07-21 16:54:10 +00:00
database Add offline FDA drug reference (labels, interaction view, conditions, remedies) (#1040) 2026-07-21 11:06:18 -07:00
docs docs: update release notes 2026-07-22 20:33:09 +00:00
inertia feat(AI): nomad.md for custom instructions (#1127) 2026-07-21 11:15:58 -07:00
providers fix(kiwix): self-heal a missing or corrupt library XML on startup 2026-06-23 04:46:56 +00:00
public feat(creator-packs): gated per-creator video packs, offline via Kiwix (#1106) 2026-07-20 16:21:18 -07:00
resources chore: standardize brand name to Project NOMAD, retire backronym 2026-07-18 09:56:54 -07:00
scripts refactor(drug-reference): make collections JSON the single source for curated data (#1130) 2026-07-21 15:50:47 -07:00
start feat: auto-generating OpenAPI docs with Scalar UI (#1128) 2026-07-21 12:36:40 -07:00
tests fix(amd): coerce gfx1103 (780M) to HSA_OVERRIDE 11.0.0 so it stays on GPU (#1134) 2026-07-22 11:07:42 -07:00
types Add offline FDA drug reference (labels, interaction view, conditions, remedies) (#1040) 2026-07-21 11:06:18 -07:00
util Add offline FDA drug reference (labels, interaction view, conditions, remedies) (#1040) 2026-07-21 11:06:18 -07:00
views chore: standardize brand name to Project NOMAD, retire backronym 2026-07-18 09:56:54 -07:00
.editorconfig feat: initial commit 2025-06-29 15:51:08 -07:00
.env.example feat(creator-packs): gated per-creator video packs, offline via Kiwix (#1106) 2026-07-20 16:21:18 -07:00
ace.js feat: initial commit 2025-06-29 15:51:08 -07:00
adonisrc.ts feat(GPU): auto-remediate nomad_ollama passthrough loss on admin boot (#755) 2026-05-20 10:16:00 -07:00
eslint.config.js feat: openwebui+ollama and zim management 2025-07-09 09:08:21 -07:00
package-lock.json feat: auto-generating OpenAPI docs with Scalar UI (#1128) 2026-07-21 12:36:40 -07:00
package.json refactor(drug-reference): make collections JSON the single source for curated data (#1130) 2026-07-21 15:50:47 -07:00
tailwind.config.ts feat: initial commit 2025-06-29 15:51:08 -07:00
tsconfig.json Add offline FDA drug reference (labels, interaction view, conditions, remedies) (#1040) 2026-07-21 11:06:18 -07:00
vite.config.ts fix(Maps): ensure proper parsing of hostnames (#640) 2026-04-03 14:26:50 -07:00