Under WSL2 the GPU is reached through /dev/dxg rather than the real adapter, so
si.graphics() returns Microsoft's generic placeholder even while CUDA work runs
on a physical card. isUnresolvedGpuModel() caught blanks, "Device <hex>" and
"Unknown" but not this, so the placeholder was accepted as a legitimate model
name and the authoritative detection path was skipped.
The reporter on #1218 has an RTX 3090 doing real work (92% utilisation, 11.6 GB
VRAM, 352 W, 127.9 tok/s) that reaches the public leaderboard labelled
"Microsoft Basic Render Driver". A wrong label is worse than none: it fragments
per-hardware grouping, which is the same failure #1165 fixed for raw PCI ids.
Adds Microsoft's two placeholder adapter names to the same rejection list.
Scoped so it cannot affect anyone off WSL: these are Microsoft's own placeholder
names and appear nowhere outside a Windows/WSL graphics stack, so on a native
Linux host the function returns exactly what it returns today. Checked against
real product names from all four vendors, including "Microsoft Basic Renderer
Pro 9000" as a near-miss, to confirm the anchored pattern cannot swallow one.
Note this makes the model resolve to unknown rather than to "RTX 3090": the
admin container has no nvidia-smi and WSL's copy lives at /usr/lib/wsl/lib,
outside the container. Honest-unknown is the correct outcome here; naming the
card under WSL is a separate problem.
Refs #1218
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LtGWCVMPYn6qPWVYiycK74
When an AMD GPU is detected, the install switches to the ROCm image and passes
/dev/kfd and /dev/dri into the container. _discoverAMDDevices() returns both
unconditionally, with no check that they exist. Detection reads PCI data, which
says nothing about whether the ROCm kernel interface was ever loaded, so on a
pre-ROCm card or a host without amdgpu the container is rejected:
(HTTP code 500) server error - error gathering device information while
adding custom device "/dev/kfd": no such file or directory
and the install dies. There is an opt-out, ai.amdGpuAcceleration, but it is a
backend-only KV key with no settings toggle, no route and no UI, so an affected
user cannot install the AI Assistant at all through the interface even though
CPU-only would work fine on their machine.
Fall back to the CPU image instead, with a message explaining why.
Two things about the shape of this fix, both measured rather than assumed:
- It hooks start(), not createContainer(). The daemon accepts a container whose
--device path does not exist and only resolves devices on start. Verified
against Docker 29.6.2 over the socket: create returns 201, start returns 500
with the message above.
- It retries rather than pre-checking. The admin runs in its own container, so
stat()ing /dev/kfd describes the admin's namespace, not the host's. Verified
on NOMAD6, a working ROCm box (Ollama running ollama/ollama:rocm with both
devices attached): /dev/kfd is PRESENT on the host and ABSENT inside the admin
container. A pre-check would therefore have stripped GPU passthrough from a
machine where AMD acceleration works.
The matcher keys on the device-gathering phrase rather than a bare "no such file
or directory", which Docker also emits for missing bind-mount sources; falling
back to CPU on one of those would hide a real failure behind a degraded install.
Checked against all three formats the daemon/dockerode produce, and against a
port-conflict message, to confirm it discriminates.
The success path is unchanged: the create payload is built by one helper used by
both attempts, and the fallback only runs after a start() failure that matches.
Fixes#1232
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LtGWCVMPYn6qPWVYiycK74
Ollama's /api/pull returns HTTP 200 and reports failures IN-BAND, as a JSON
line inside the NDJSON stream body:
HTTP_STATUS=200
{"status":"pulling manifest"}
{"error":"pull model manifest: file does not exist"}
The stream handler only ever read completed/total/digest off each line and
silently dropped everything else, including that error line. The stream then
ended normally, on('end') resolved unconditionally, and we logged the model as
downloaded successfully. on('error') never fires for this, because at the HTTP
layer the request succeeded.
Every failed model pull, for any reason, was reported as a success. On #1071
that surfaced as an ~800 GB fp16 model "downloading" in two seconds and nothing
being installed, which had been triaged twice as disk space or a stale blob.
Those explanations produce workarounds that cannot work, since the pull never
reaches the disk at all.
Three changes:
- Check parsed.error in the data handler and destroy the stream with it, so the
existing catch reports a real failure.
- Verify the model is actually in getModels() before returning success. This is
the load-bearing half: it catches silent failures we have not anticipated,
not only the ones Ollama labels. includeEmbeddings must be true or
nomic-embed-text fails verification after a good pull, and names are
normalised because Ollama resolves a bare name to ":latest".
- Treat an in-band "model does not exist" as permanent. It was retryable, so a
bad model name spun through attempts: 10 on exponential backoff and the user
watched a delayed job for hours instead of being told the name was wrong.
Verified on NOMAD3 against the shipped v1.34.0 build, which carries the same
defect:
- bogus tag, before: 'Model "..." downloaded successfully.'
- bogus tag, after: 'Failed to download model "...": pull model manifest:
file does not exist', and the job stops at 1 attempt instead of retrying
- real model (qwen2.5:0.5b): still succeeds, passes verification, and is
present in `ollama list`
Fixes#1273
Refs #1071
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LtGWCVMPYn6qPWVYiycK74
Three dispatch sites pin a deterministic jobId. `queue.add` with an existing
custom jobId returns the existing job rather than throwing, so any retained
terminal record makes later dispatches a silent no-op while still reporting
success to the caller.
- download_model_job: only `failed` records were cleared, so once a model
downloaded successfully its retained completed job blocked every later
request for that model. Deleting the model and reinstalling it did nothing,
and a restart did not help because the record is persisted in Redis. Adds
getActiveByModelName, mirroring RunDownloadJob.getActiveByUrl.
- download_service.retryFailedJob: the model branch dispatched before removing,
so remove() targeted the job just enqueued under the same id instead of the
old record. Flipped to remove-then-dispatch, matching the file branch.
- embed_file_job: a retained failed record made re-indexing that file a no-op,
returning 202 "Indexing queued" with nothing enqueued.
In-flight jobs are still returned as-is, so re-clicking during an active
download or embed stays idempotent.
Diagnosis by @caweis in #1214, including the retry-ordering race and the embed
case. Reimplemented in-house rather than ported, as noted there.
Closes#1214
Fixes#1269. Fixes#1203.
`file-embeddings` and `downloads` both routinely outlive the default
5 minute lock, so BullMQ calls healthy jobs stalled on both. They need
opposite recovery policies, which is why neither was covered by the
existing drug-queue override.
Embedding is a self-continuing chain, so re-queueing a stalled job while
the original is alive runs two chains over the same file. Both write to
Qdrant with freshly minted point ids, so the second cannot overwrite the
first, and the queue's concurrency of 2 means the fork never contends
for a slot. maxStalledCount 0 fails on the first stall instead.
A download is resumable, so recovery is safe there; the default of 1 was
failing jobs outright and bypassing their own attempts: 10 policy.
The decision moves to a pure, unit-tested util alongside the other
decision helpers.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LtGWCVMPYn6qPWVYiycK74
The batch continuation gated on articles that produced chunks rather than
articles the extractor consumed. Articles whose text is empty after HTML
cleaning — redirect stubs, category listings, media and PDF wrappers —
return no chunks and so no documentIds, which made a normal sparse batch
look like the end of the archive. Ingestion stopped after the first batch
and the rest of the file was silently skipped.
Medicine LibreTexts embedded 16 chunks out of 23,171 articles: its first
50 entries are navigation pages, only 10 of which had text, so 10 >= 50
was false and no continuation was ever dispatched. Measured against the
same install, WikiMed reached 28% of its articles, Libre Pathology 31%,
MedlinePlus 84%, and CDC Travelers' Health 2.7%.
The same value was also returned as `articlesProcessed`, which
EmbedFileJob uses to advance the next batch offset. Advancing by the
content-bearing count re-read the overlap on every sparse batch, and a
batch yielding no text at all would have advanced by zero and re-run the
same window indefinitely.
Both now use the extractor's consumed-article count, which it already
computed and logged but discarded. The end-of-archive signal is the
iterator running dry (articlesProcessed < batchSize); archive.articleCount
cannot serve as a bound because iterByPath() legitimately yields more
article entries than that figure for some archives.
The log line now reports both counts (content-bearing/consumed) so a
sparse batch is visible without re-deriving it.
The history table rendered nomad_score for every row, so a v2 run showed its
legacy score while the Benchmark Details card directly above it showed the v2
one. The same run read as 65.0 in the table and 1036.1 in the card.
Rows predating v2 have no v2 score and can only show the legacy number, so the
two scales necessarily share a column. They differ by more than 10x, and
without the scale on screen a v2 run sitting above older runs reads as a
collapse rather than a rescale. Legacy rows now carry a "/ 100" suffix, which
names the scale in place and reuses the wording the details card already uses
("Legacy scale: X / 100").
This compounds the re-run guidance in the release notes: someone re-runs on
1.34 as asked, gets a real v2 score, and their own history then shows the old
number as though the machine got slower.
Frontend only. getAllResults() already returns full rows, so nomad_score_v2
was in the payload the whole time.
Verified on the NOMAD2 dev environment with a seeded history matching a real
mixed case (one v2 run at 1036.1 plus two pre-v2 runs): the v2 row shows
1036.1 with no suffix and agrees with the details card, and the legacy rows
show 71.1 / 100 and 86.5 / 100. Inertia typecheck shows the same 30
pre-existing errors as dev with none in this file.
Adds a security policy so vulnerability reports are routed to the private
advisory form instead of public issues, and defines scope so the intentional
no-auth LAN design is not re-reported as a vulnerability.
Also adds a security contact link to the issue template chooser (blank issues
are disabled, so there was previously no route for a private report), and
extends .gitignore to cover .env variants, key/cert files, .npmrc/.netrc and
local copies of the deployed compose file. A local management compose with real
generated DB passwords was committed to a branch once before; secret scanning
push protection now covers that case too.
No tracked file is affected by the new ignore rules.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was one cutoff, Qdrant's score_threshold at 0.3, and nomic scores
unrelated text well above it, so every question retrieved *something*
(nonEmptyRateOnRefusal 1.0) and rag_context wasted four lines asking a 3B
model to "silently judge" relevance. That is a classification a number
should decide, not a micro model.
Adds `RAG_MIN_FINAL_SCORE`, a floor on the post-rerank score, applied before
source diversity. The floor judges relevance, the 0.85^n penalty judges
redundancy. Nothing clears it and nothing is injected. `score_threshold` stays
at 0.3 as the candidate net. Exposed as `rag.minRelevance` with a module-
scoped TTL cache invalidated from `updateSetting`.
The floor is calibrated and the harness was describing the wrong number:
aggregate() bucketed semanticScore, so its percentiles measured raw cosine
while the new knob filters the reranked score. Both axes now, and the gap
matters: recall falls at 0.60 on the semantic axis, 0.66 on the final one.
floor recall@5 precision@5 emptyOnAnswerable nonEmptyOnRefusal
0.00 0.991 0.221 0% 100%
0.62 0.991 0.524 0% 60%
0.66 0.991 0.695 0% 60%
0.67 0.972 0.704 1% 60%
Default 0.62, not the last-lossless 0.66: the refusal win is complete at
0.62 and 28 documents is a thin basis for spending margin. 0.60 is the
honest floor. Three of five refusal goldens are near-corpus by
construction, where declining is the generation tier's job.
Both eval tiers pin the floor to the constant and ignore the setting, for
the same reason the harness never sets skipRetrieval.
4 metrics improved against baseline, no regressions.
Ollama's format parameter turns a formatting request into a decoder
constraint, and NOMAD's three ancillary calls — chat title, suggestion
chips, RAG query rewrite — were the ones paying for not having it.
The rewrite is particularly important: whatever it returned got embedded
and sent to Qdrant, so a malformed response degraded retrieval for the
rest of the conversation.
Three explicit design choices made:
- Native-only. The compat path sends nothing new and keeps its string
parsing, so a non-Ollama backend that rejects response_format can't
take out title generation entirely.
- parseStructured returning null is the control flow. It's how each
caller falls back to the parser it already had, which is why it's written
to be total and never throw.
- {queries: []} not {query: ""} for the rewrite (tee-ing up future
improvements)
ThinkTagSplitter was wired into both streaming paths but neither
non-streaming one, and every ancillary call goes through non-streaming
chat(). So a reasoning tasks model put its reasoning transcript into the
sidebar title, into the suggestion chips, and — worst — into the string
that gets embedded and sent to Qdrant, silently corrupting retrieval for
the rest of the conversation. splitThinkTags() already existed and was
already unit tested; it just had no production callers.
Fixed in three layers:
1. Strip at the transport. New normalizeNonStreamed() in think_stream.ts
merges structured reasoning (native `thinking`, /v1 `reasoning`) with
anything the splitter pulls out of inline tags, and both _chatNative
and _chatCompat now return through it. That covers every caller at
once, including the legacy controller path and the eval harness.
2. Suppress at the source. _chatNative and _chatStreamNative forwarded
`think` only when truthy, so `think: false` was never sent and Ollama
left reasoning on for capable models. Guard is now definedness. This
also fixes the ai.autoThinking=off setting, which had no effect on
the native streaming transport.
3. Guard the call sites. Title, suggestions, and query rewrite now pass
think:false plus thinkingCapable (from the memoized capability check,
so no extra round-trip), and handle an empty-after-strip result. The
rewrite matters most: QUERY_REWRITE_MAX_TOKENS caps it at 120, which
a reasoning model burns entirely on thinking, so stripping alone
would have turned a corrupt query into an empty one. It now falls
back to the raw user message instead of embedding "".
Verified: think_stream.spec.ts 17/17 (6 new cases, including the
truncated-mid-thought case that drives the fallback), typecheck clean,
full test:unit failures byte-identical to the pre-change baseline.
Chat was run through Ollama's OpenAI-compatible /v1 endpoint, which has no
context-size field, so the num_ctx we sent was silently discarded and the
entire NUM_CTX_LADDER was dead code. Every conversation ran at the backend
default and was truncated from the middle (where history and
retrieved context actually sit). Measured with identical ~7k-token prompts:
2,060 tokens processed against 7,049 on the native endpoint, and with the
needed fact in the discarded middle the old path hallucinated a plausible
wrong number.
Transport: add a native /api/chat path using the already-installed ollama
package, sending a real options object (num_ctx, num_predict, temperature,
seed) plus keep_alive. The OpenAI path is still used for LM Studio and llama.cpp,
minus the ineffective num_ctx and with include_usage so both paths return
prompt token counts. /api/show is now read for trained context, parameter
size, quantization and any modelfile num_ctx. The <think> stream normalizer
moves to app/utils/think_stream.ts so both transports share it; it gains a
flush() that fixes a dangling partial tag being dropped at end of stream.
Token accounting: replace chars/N with a structural segmenter scaled by a
per-model factor learned by EWMA from the prompt_eval_count every response
already returned and threw away. End-to-end estimator error is 4.6% against
18-20% for a fixed divisor — qwen2.5 converges to k=1.26 where llama3 sits
at 1.00, so no single global constant serves both.
Context window: resolved once per model from trained context, the exact KV
cost derived from GGUF metadata (head_count_kv, not head_count — GQA makes
them differ 4x), and available VRAM, then memoized. Memoization is
load-bearing: asking for a different num_ctx makes Ollama unload and reload
the model, stalling the turn and discarding the KV cache. The floor never
overrides the model's trained ceiling — tinyllama trains to 2048, and a
blind 4096 floor pushes RoPE past anything the weights saw.
Prompt budget: planPrompt allocates explicitly instead of delegating
overflow to silent backend truncation — response reserve, then fixed blocks
(system and current query, never dropped), then RAG chunks whole and
best-first, then history in whole user+assistant turns, evicted newest-first
in blocks with an elision marker. Block eviction keeps the prefix stable for
several turns rather than shifting on every one.
KV-cache ordering: the per-turn retrieved block moves behind history to the
tail, so the stable prefix stays byte-identical turn to turn. At turn 14 of
a long conversation, prefill is 231ms at the tail against 539ms at the front
(~1ms/turn growth against ~25ms). The win is conditional — with a large
retrieved block and short history the two measure the same — so both
orderings ship behind RAG_PLACEMENT. The per-turn query rewrite also moves
to the tasks model; on a single-slot Ollama it was evicting the chat model's
cached prefix on every single turn.
Also: OLLAMA_CONTEXT_LENGTH and OLLAMA_KV_CACHE_TYPE=q8_0 on the container
(q8_0 is treated as headroom, never as a budgeting assumption, since
unsupported architectures fall back to f16 silently), a Context Window
setting in Settings > Models, and budget fields on PipelineTrace.
Eval: golden suites (--suite) so the new long_context goldens cannot move
the aggregates the committed baselines were recorded against, plus budget
and estimator-error metrics through the report path. Retrieval is unchanged
and eval:compare reports no metric outside the tolerance band; the
long_context suite scores correctness 1.000 with historyElided 0.500 (the
two eviction cases, by design). Unit tests 337 -> 386 passing, same 7
pre-existing failures.
Ingest-side CHAR_TO_TOKEN_RATIO is deliberately untouched: changing it
re-chunks the corpus and invalidates retrieval baselines, which belongs in
its own measured change.
Allows users to optionally specify a model to use for ancillary tasks (chat suggestions, chat titles, etc.) instead of strictly using the last used model. This means aesthetic/non-critical work can be passed to a small, lightweight model instead of a heavy reasoning model used for chats.
If a model is not selected in the AI Assistant settings (`/settings/models`), the existing behavior of using the last used model is retained.
workflow_dispatch workflows are only dispatchable if the file exists on
the default branch, so this must land on main before the translate proxy
image can be published.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
browse.library.kiwix.org grew an anti-crawler interstitial on 2026-08-29
that answers HTTP 200 with an HTML confirmation page. Content Explorer's
remote listing 500s on it and ZIM auto-update silently reports "no
updates". opds.library.kiwix.org serves the same OPDS API ungated.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
HOLD UNTIL 1.34.0 GA. Merging early breaks v1.33.0 installs.
The manifest is fetched live from main
(collection_manifest_service.ts: refs/heads/main/collections/kiwix-categories.json),
so every install on every version picks this up the moment it merges — not on
their next upgrade.
`type: "dataset"` does not exist anywhere in v1.33.0: zero hits across
admin/app, admin/inertia and admin/types. It is new in 1.34.0. On a v1.33.0 box
this entry would render as an installable resource with no code that
understands its type and a URL that is not a ZIM, so the install path breaks
for users who are not running the version that introduced the feature.
Held back until GA for that reason, which also means 1.34.0 would otherwise
ship a working Drug Reference that cannot be discovered from Content Explorer.
Entry is byte-identical to the one already on rc.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The 401/403 branch added in #1172 threw a plain Error, and
RunDownloadJob is registered attempts: 10 with exponential backoff from
30s. A permanent "this build has no entitlement key" rejection therefore
retried nine more times over roughly 4h15m, during which the job reads
`delayed` rather than `failed` - so the user sees a stuck download
instead of the clear message #1172 was written to give them. Each retry
also re-hits our own rate-limited Worker.
Throw a named GatedContentAuthError from the download util and translate
it to UnrecoverableError at the queue boundary in RunDownloadJob,
alongside the existing cancellation case. Declaring the class in
downloads.ts rather than throwing UnrecoverableError directly keeps
BullMQ out of a module that docker_service and map_service also use.
Harmless today because no catalog entry uses `auth`, but that ends with
the first gated card - see #1204.
Fixes#1195
doResumableDownload already implements resume - stats the .tmp, sends a
Range header, handles a server that ignores it. The code was unreachable
for ZIMs because every dispatch site passed forceNew: true, which skips
the partial-file check and opens the stream with 'w' instead of 'a', so
an interrupted 12.5 GB Wikipedia download truncated and restarted at
byte 0. Maps already do this correctly (map_service.ts:681, "so retries
resume partial downloads"); ZIMs never got the same treatment.
Worse in combination with attempts: 10 - every retry also restarted from
zero, so a flaky connection re-downloaded the whole file up to ten times.
Drop forceNew from the content-download dispatch sites (it already
defaults to false) and add the guard that enabling resume requires: a
.tmp larger than the file now on the server cannot be a prefix of it,
because openZIM re-publishes builds under the same name. Resuming would
request a range past the end, 416 on every attempt, and never delete the
.tmp - so the download could never recover on its own.
Verified on a test appliance: a 470 MB partial survived a container
restart and continued rather than truncating, and a planted oversized
.tmp was discarded with the file then downloading to its exact size.
Refs #1201
Assigning a collection before indexing lost it silently. The per-row
value still showed, but Manage Collections and the Search in dropdown
stayed empty, because getKnowledgeCollections() facets on the Qdrant
payload while only MySQL had been written.
Five gaps on one path:
updateFileCollection() sets the payload filtered on `source`, which
matches nothing before the file is indexed. It also only persisted to
kb_ingest_state `if (row)`, so a file with no row stored the value
nowhere at all and still returned "Moved to ...".
Six of the seven EmbedFileJob.dispatch sites never pass `collection`,
and none read the existing row, so Index dispatched a job with no
knowledge of the assignment. The ZIM branch of processAndEmbedFile then
dropped `collection` even when the job had one, so ZIM content could
never be tagged at embed time by any path. Batch continuations dropped
it too, which would have tagged only batch 1.
Resolve the effective collection once inside EmbedFileJob.handle rather
than at seven call sites, thread it through the ZIM path into the point
payload, carry it across batch continuations, and make the pre-index
assignment durable with getOrCreate.
Verified end to end on a test appliance: assigned a collection to an
unindexed ZIM, indexed it across multiple batch continuations, and all
6106 chunks carry the tag.
Two layout defects in the Knowledge Base modal, both found during
v1.34.0-rc.4 QA.
The collection combobox list was clipped to a single row's height on
every row of the table. The cause is not stacking order - StyledTable
gives each cell `truncate` (overflow:hidden) plus `relative`, so an
absolutely-positioned child cannot escape the cell box. No z-index can
win against a clip. The modal body is also an `overflow-y-auto`
scroller, so simply opting the cell out of `truncate` still left rows
lower down clipped by the scroller instead.
Render the list in a portal with fixed positioning instead, measured
from the input's rect and flipped above when it would run off the
bottom of the viewport. Reposition on scroll and resize (capture
listener, so any ancestor scroller counts). The click-outside handler
now also checks the portaled list, otherwise mousedown on an option
would close it before the click landed.
Separately, the modal was capped at max-w-4xl (896px) while its table
needs 906px, so the Delete button lost its right edge for everyone
regardless of monitor size. max-w-5xl gives the table room with none to
spare wasted.
Verified against a real build on a test appliance.
The banner shipped with two highlights while the 1.34 line delivered 21
features, so it undersold the release. Adds the three with the widest
user-facing surface: Score v2, NOMAD.md and Knowledge Base collections.
Also converts the two existing bullets from em dashes to spaced hyphens
to match project copy convention.
Kept to five bullets - this is a dashboard alert, not release notes.
A job id can outlive its payload hash. BullMQ still returns an entry for
it with empty `data`, so `normalize(job.data.filepath)` threw and took the
whole response with it.
`fetchJobsWithStates()` includes failed jobs, and failed jobs are retained
deliberately, so one orphan made GET /api/downloads/jobs throw on every
call, permanently. Content Explorer polls that endpoint every ~3s, so the
symptom was a 500 loop and a page that never loaded. It survived restarts
and only cleared by editing Redis by hand.
Drop entries with no usable payload at the source, and guard both
normalize() calls (the pmtiles-extract map had the same pattern). With no
payload there is nothing to render anyway.
Seen on NOMAD3 running rc.3: 93 occurrences in 15 minutes.
Closes#1190
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
ZimService.registerLocalUpload() dispatched EmbedFileJob unconditionally,
gated only on whether Ollama was reachable. It never read
rag.defaultIngestPolicy, so a user who deliberately chose Manual still got
sideloaded ZIMs embedded into the knowledge base behind their back.
PR #919 fixed exactly this for the post-download dispatch path. The
local-upload path was missed.
Rather than inline a third copy of the Always/Manual conditional, this
reuses decideScanAction, the same helper the scanner uses. That also means
an existing browse_only or pending_decision row is now honored instead of
being overridden by the act of re-uploading the file, which the inline
version in run_download_job.ts does not do.
Unset policy is still treated as Always, so existing installs keep their
current behavior.
Reported by @just-jbc on #1119, which also proposed disabling ZIM
auto-discovery entirely. That larger change is not included here: turning
discovery off by default would mean a user who downloads Wikipedia through
the curated flow gets no AI answers from it and no explanation why.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The chat modal's pop-out button called window.open('/chat', '_blank').
/chat is served by the admin app itself, so this spawned a second browser
window for a same-origin internal route.
That breaks anyone running NOMAD as an installed web app or in kiosk mode:
clicking it leaves a stray window they then have to get back out of, which
is exactly the complaint in #1123.
Navigate with router.visit instead, and relabel the button from "Open in
New Tab" to "Open Full Chat" so it describes what now happens. IconMessage
isn't in the DynamicIcon registry (deliberately curated for tree-shaking),
so use the already-registered IconArrowRight.
Refs #1123
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds an optional `auth: 'nomad_app_key'` discriminator to curated manifest
resources so a curated collection tier can carry content we host ourselves,
gated to official release builds. Without this, only Creator Packs could use
the entitlement Worker; curated tier installs always downloaded
unauthenticated.
No behaviour change for any existing manifest entry: absent `auth` means
unauthenticated, exactly as today.
- `auth` declared on both the type and the VineJS validator. It has to be on
the validator or VineJS strips it silently on fetch, and the gated download
would then go out with no header and 401 for everyone. A dedicated spec
guards that regression.
- Gated resources are pinned to their manifest URL (resolveZimDownload skips
the catalog comparison) and excluded from catalog update checks, so a
resource-id collision cannot let a third-party mirror overwrite our content.
Consequence, commented rather than implied: gated content does not
auto-update; new versions ship via the manifest.
- 401/403 on a download now reports that an official build is required instead
of a raw axios status, which is what a fork build will hit.
- The pure `isGatedResource` predicate is deliberately split from the
env-reading header builder: importing `#start/env` into
zim_download_resolution triggers env validation at import time and breaks its
unit tests.
Reuses CREATOR_PACKS_APP_KEY rather than minting a second secret — the question
it answers ("is this an official build?") is identical for both content types.
Verified end to end on a test server: `auth` survives validation into the
cached spec, the Bearer header attaches to only the gated resource, the file
lands byte-exact with an installed_resources row and a Kiwix library entry, and
an entry with a gated URL but no `auth` field fails with the intended message.
No catalog entry is included here. Manifests are fetched live from `main`, so a
gated entry must not merge until this ships and is adopted — pre-`auth` builds
strip the field and 401.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The #1157 gate blocks leaderboard submission whenever ai.remoteOllamaUrl is
set, using bare truthiness with no exemption for addresses that point back at
this same box. Someone running Ollama natively on the host while NOMAD runs in
Docker is measuring THIS hardware, yet is permanently blocked from submitting
with no in-app indication of why.
That configuration is not exotic: it is how the AI assistant is expected to
work on macOS, so under the current gate a Mac could never submit at all.
Exempts host.docker.internal (and gateway.docker.internal) plus the loopback
forms. host.docker.internal is the meaningful one — from inside the admin
container it resolves to the host, and it is what a native host install uses.
A LAN address is deliberately NOT exempt. 192.168.1.50 is indistinguishable
from another machine on the same network, and wrongly exempting it would let a
genuinely remote GPU's throughput be attributed to this hardware. A false block
is recoverable by clearing the setting; a false pass silently corrupts the
board.
Unparseable values fall through to blocked rather than allowed, so a malformed
setting cannot be used to slip past the gate.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three changes that together let ARM hardware appear on the leaderboard honestly.
Shipping them separately would leave ARM half-supported either way: without the
image a Pi cannot submit at all, and without the architecture field it submits
but is indistinguishable from x86.
1. PIN THE OFFICIAL MULTI-ARCH SYSBENCH IMAGE
severalnines/sysbench publishes amd64 only, so ARM hosts could not run the
System Benchmark at all — not a graceful failure, the container simply cannot
execute. Apple Silicon could only run it under Rosetta emulation, which distorts
the measurement it is taking, and that is what drove a community macOS fork to
substitute a different benchmark and submit incomparable numbers.
Swaps to ghcr.io/crosstalk-solutions/nomad-sysbench (Debian 12 + sysbench
1.0.20+ds-5, built for linux/amd64 + linux/arm64). One digest covers both
architectures; verified that pulling the pinned manifest-list digest resolves to
arm64 on a Raspberry Pi 5 and amd64 on x86, and that RepoDigests reports the
same manifest-list digest on both — so a single allowlist entry serves both.
No rescoring: 1.0.17 -> 1.0.20 measured 1.25% apart on identical hardware with
identical flags (7170.18 vs 7259.56 events/sec), inside run-to-run noise and
~0.3% on a composite. Both digests are allowlisted server-side, so the fleet can
cross over gradually.
2. REPORT THE DIGEST ACTUALLY RESOLVED
The submission previously sent SYSBENCH_DIGEST, the constant the client was
compiled with. The leaderboard validates that field, but a constant attests to
how a client was BUILT rather than what it RAN, so any build inherits a valid
value simply by carrying the same source.
Now reads it back from the image. Uses RepoDigests (the manifest digest we
pulled by), never Id — Id is the config digest, differs per architecture, and
would never match the allowlist. Falls back to the constant if inspection yields
nothing usable, so a benchmark never fails over provenance metadata.
Still forgeable, and always will be with an open-source client. It moves the bar
from "no effort" to "deliberate", which is the distinction that matters when
judging whether a submission is a mistake or a choice.
3. RECORD CPU ARCHITECTURE AND OS
The leaderboard is a single board across instruction sets by design, with
disclosure as the fairness mechanism. Without an architecture field an ARM result
sits unlabelled beside x86 — exactly what the disclosure exists to prevent.
All three fields come from the Docker daemon, reusing the docker.info() call
_detectRunEnvironment already makes. That is deliberate: inside the admin
container os.arch() and si.osInfo() describe the CONTAINER, not the host being
benchmarked.
cpu_architecture Architecture x86_64 -> amd64, aarch64 -> arm64
os_version OSVersion '24.04' (already structured, no parsing)
os_name OperatingSystem 'Ubuntu 24.04.4 LTS' minus the version
run_environment is kept rather than replaced: "which distro" and "is this
virtualised" are different questions, and WSL2 is a real performance factor.
String handling lives in app/utils/platform_metadata.ts with unit tests, matching
the amd_hsa_override convention, so it is testable without a Docker daemon.
Unknown architectures pass through verbatim rather than being guessed at, and
os_name falls back to the full description whenever the version is missing or
absent from it — an over-long name is harmless, a wrong one is not.
Columns are nullable and the submission fields optional, so results recorded
before this shipped remain submittable.
Closes#1156
Refs #1151
DockerService.getServiceURL() resolves ai.remoteOllamaUrl ahead of the local
container, so when a remote AI host is configured the AI channel measures THAT
machine while every other channel measures this one. The submission then reports
someone else's tok/s under this hardware's CPU, RAM and disk.
Under v2 this matters more than it did under v1: ai_tokens_per_second carries
0.30 of the weight and the score is uncapped, so a remote GPU's throughput is no
longer limited by a clamp.
Adds a guard alongside the existing submit-time checks. Uses the same truthiness
predicate as getServiceURL, so the guard fires exactly when the remote routing
it is guarding against would occur. KVStore.clearValue() nulls the value and
getValue() returns null for that, so a cleared key correctly does not trip it.
Blocks submission only. Running the benchmark locally is still useful to the
operator — it just isn't a result about this box, so it shouldn't go on a board
that ranks hardware.
Known limitation: the check reads the KV at submit time, not at measurement
time, so benchmarking with a remote host and then clearing the setting before
submitting would still get through. That is a deliberate workaround rather than
an accident, and closing it properly needs a column on benchmark_results to
record how inference was reached. Worth doing if it ever shows up in practice;
not worth a migration on the evidence available.
Refs #1151
The modern-rogue pack entry landed in collections/creator-packs.json
(#1145/#1146) but its banner image was never committed, so the app fell
back to /creator-packs/modern-rogue.webp which 404s and the card rendered
without a header.
Adds the 1060x175 banner (RGB webp), matching the existing
project-nomad.webp and crosstalk-solutions.webp, generated from the
pack's source banner.png built on the pack-build workspace.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The MeshCore Web app doc listed meshcore.co.uk as the official site. The
canonical MeshCore project (github.com/meshcore-dev/MeshCore, the firmware +
protocol) declares its homepage as meshcore.io, so update the two references
in the MeshCore Web section to point there.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
