workflow_dispatch workflows are only dispatchable if the file exists on
the default branch, so this must land on main before the translate proxy
image can be published.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
browse.library.kiwix.org grew an anti-crawler interstitial on 2026-08-29
that answers HTTP 200 with an HTML confirmation page. Content Explorer's
remote listing 500s on it and ZIM auto-update silently reports "no
updates". opds.library.kiwix.org serves the same OPDS API ungated.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
HOLD UNTIL 1.34.0 GA. Merging early breaks v1.33.0 installs.
The manifest is fetched live from main
(collection_manifest_service.ts: refs/heads/main/collections/kiwix-categories.json),
so every install on every version picks this up the moment it merges — not on
their next upgrade.
`type: "dataset"` does not exist anywhere in v1.33.0: zero hits across
admin/app, admin/inertia and admin/types. It is new in 1.34.0. On a v1.33.0 box
this entry would render as an installable resource with no code that
understands its type and a URL that is not a ZIM, so the install path breaks
for users who are not running the version that introduced the feature.
Held back until GA for that reason, which also means 1.34.0 would otherwise
ship a working Drug Reference that cannot be discovered from Content Explorer.
Entry is byte-identical to the one already on rc.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The 401/403 branch added in #1172 threw a plain Error, and
RunDownloadJob is registered attempts: 10 with exponential backoff from
30s. A permanent "this build has no entitlement key" rejection therefore
retried nine more times over roughly 4h15m, during which the job reads
`delayed` rather than `failed` - so the user sees a stuck download
instead of the clear message #1172 was written to give them. Each retry
also re-hits our own rate-limited Worker.
Throw a named GatedContentAuthError from the download util and translate
it to UnrecoverableError at the queue boundary in RunDownloadJob,
alongside the existing cancellation case. Declaring the class in
downloads.ts rather than throwing UnrecoverableError directly keeps
BullMQ out of a module that docker_service and map_service also use.
Harmless today because no catalog entry uses `auth`, but that ends with
the first gated card - see #1204.
Fixes#1195
doResumableDownload already implements resume - stats the .tmp, sends a
Range header, handles a server that ignores it. The code was unreachable
for ZIMs because every dispatch site passed forceNew: true, which skips
the partial-file check and opens the stream with 'w' instead of 'a', so
an interrupted 12.5 GB Wikipedia download truncated and restarted at
byte 0. Maps already do this correctly (map_service.ts:681, "so retries
resume partial downloads"); ZIMs never got the same treatment.
Worse in combination with attempts: 10 - every retry also restarted from
zero, so a flaky connection re-downloaded the whole file up to ten times.
Drop forceNew from the content-download dispatch sites (it already
defaults to false) and add the guard that enabling resume requires: a
.tmp larger than the file now on the server cannot be a prefix of it,
because openZIM re-publishes builds under the same name. Resuming would
request a range past the end, 416 on every attempt, and never delete the
.tmp - so the download could never recover on its own.
Verified on a test appliance: a 470 MB partial survived a container
restart and continued rather than truncating, and a planted oversized
.tmp was discarded with the file then downloading to its exact size.
Refs #1201
Assigning a collection before indexing lost it silently. The per-row
value still showed, but Manage Collections and the Search in dropdown
stayed empty, because getKnowledgeCollections() facets on the Qdrant
payload while only MySQL had been written.
Five gaps on one path:
updateFileCollection() sets the payload filtered on `source`, which
matches nothing before the file is indexed. It also only persisted to
kb_ingest_state `if (row)`, so a file with no row stored the value
nowhere at all and still returned "Moved to ...".
Six of the seven EmbedFileJob.dispatch sites never pass `collection`,
and none read the existing row, so Index dispatched a job with no
knowledge of the assignment. The ZIM branch of processAndEmbedFile then
dropped `collection` even when the job had one, so ZIM content could
never be tagged at embed time by any path. Batch continuations dropped
it too, which would have tagged only batch 1.
Resolve the effective collection once inside EmbedFileJob.handle rather
than at seven call sites, thread it through the ZIM path into the point
payload, carry it across batch continuations, and make the pre-index
assignment durable with getOrCreate.
Verified end to end on a test appliance: assigned a collection to an
unindexed ZIM, indexed it across multiple batch continuations, and all
6106 chunks carry the tag.
Two layout defects in the Knowledge Base modal, both found during
v1.34.0-rc.4 QA.
The collection combobox list was clipped to a single row's height on
every row of the table. The cause is not stacking order - StyledTable
gives each cell `truncate` (overflow:hidden) plus `relative`, so an
absolutely-positioned child cannot escape the cell box. No z-index can
win against a clip. The modal body is also an `overflow-y-auto`
scroller, so simply opting the cell out of `truncate` still left rows
lower down clipped by the scroller instead.
Render the list in a portal with fixed positioning instead, measured
from the input's rect and flipped above when it would run off the
bottom of the viewport. Reposition on scroll and resize (capture
listener, so any ancestor scroller counts). The click-outside handler
now also checks the portaled list, otherwise mousedown on an option
would close it before the click landed.
Separately, the modal was capped at max-w-4xl (896px) while its table
needs 906px, so the Delete button lost its right edge for everyone
regardless of monitor size. max-w-5xl gives the table room with none to
spare wasted.
Verified against a real build on a test appliance.
The banner shipped with two highlights while the 1.34 line delivered 21
features, so it undersold the release. Adds the three with the widest
user-facing surface: Score v2, NOMAD.md and Knowledge Base collections.
Also converts the two existing bullets from em dashes to spaced hyphens
to match project copy convention.
Kept to five bullets - this is a dashboard alert, not release notes.
A job id can outlive its payload hash. BullMQ still returns an entry for
it with empty `data`, so `normalize(job.data.filepath)` threw and took the
whole response with it.
`fetchJobsWithStates()` includes failed jobs, and failed jobs are retained
deliberately, so one orphan made GET /api/downloads/jobs throw on every
call, permanently. Content Explorer polls that endpoint every ~3s, so the
symptom was a 500 loop and a page that never loaded. It survived restarts
and only cleared by editing Redis by hand.
Drop entries with no usable payload at the source, and guard both
normalize() calls (the pmtiles-extract map had the same pattern). With no
payload there is nothing to render anyway.
Seen on NOMAD3 running rc.3: 93 occurrences in 15 minutes.
Closes#1190
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
ZimService.registerLocalUpload() dispatched EmbedFileJob unconditionally,
gated only on whether Ollama was reachable. It never read
rag.defaultIngestPolicy, so a user who deliberately chose Manual still got
sideloaded ZIMs embedded into the knowledge base behind their back.
PR #919 fixed exactly this for the post-download dispatch path. The
local-upload path was missed.
Rather than inline a third copy of the Always/Manual conditional, this
reuses decideScanAction, the same helper the scanner uses. That also means
an existing browse_only or pending_decision row is now honored instead of
being overridden by the act of re-uploading the file, which the inline
version in run_download_job.ts does not do.
Unset policy is still treated as Always, so existing installs keep their
current behavior.
Reported by @just-jbc on #1119, which also proposed disabling ZIM
auto-discovery entirely. That larger change is not included here: turning
discovery off by default would mean a user who downloads Wikipedia through
the curated flow gets no AI answers from it and no explanation why.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The chat modal's pop-out button called window.open('/chat', '_blank').
/chat is served by the admin app itself, so this spawned a second browser
window for a same-origin internal route.
That breaks anyone running NOMAD as an installed web app or in kiosk mode:
clicking it leaves a stray window they then have to get back out of, which
is exactly the complaint in #1123.
Navigate with router.visit instead, and relabel the button from "Open in
New Tab" to "Open Full Chat" so it describes what now happens. IconMessage
isn't in the DynamicIcon registry (deliberately curated for tree-shaking),
so use the already-registered IconArrowRight.
Refs #1123
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds an optional `auth: 'nomad_app_key'` discriminator to curated manifest
resources so a curated collection tier can carry content we host ourselves,
gated to official release builds. Without this, only Creator Packs could use
the entitlement Worker; curated tier installs always downloaded
unauthenticated.
No behaviour change for any existing manifest entry: absent `auth` means
unauthenticated, exactly as today.
- `auth` declared on both the type and the VineJS validator. It has to be on
the validator or VineJS strips it silently on fetch, and the gated download
would then go out with no header and 401 for everyone. A dedicated spec
guards that regression.
- Gated resources are pinned to their manifest URL (resolveZimDownload skips
the catalog comparison) and excluded from catalog update checks, so a
resource-id collision cannot let a third-party mirror overwrite our content.
Consequence, commented rather than implied: gated content does not
auto-update; new versions ship via the manifest.
- 401/403 on a download now reports that an official build is required instead
of a raw axios status, which is what a fork build will hit.
- The pure `isGatedResource` predicate is deliberately split from the
env-reading header builder: importing `#start/env` into
zim_download_resolution triggers env validation at import time and breaks its
unit tests.
Reuses CREATOR_PACKS_APP_KEY rather than minting a second secret — the question
it answers ("is this an official build?") is identical for both content types.
Verified end to end on a test server: `auth` survives validation into the
cached spec, the Bearer header attaches to only the gated resource, the file
lands byte-exact with an installed_resources row and a Kiwix library entry, and
an entry with a gated URL but no `auth` field fails with the intended message.
No catalog entry is included here. Manifests are fetched live from `main`, so a
gated entry must not merge until this ships and is adopted — pre-`auth` builds
strip the field and 401.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The #1157 gate blocks leaderboard submission whenever ai.remoteOllamaUrl is
set, using bare truthiness with no exemption for addresses that point back at
this same box. Someone running Ollama natively on the host while NOMAD runs in
Docker is measuring THIS hardware, yet is permanently blocked from submitting
with no in-app indication of why.
That configuration is not exotic: it is how the AI assistant is expected to
work on macOS, so under the current gate a Mac could never submit at all.
Exempts host.docker.internal (and gateway.docker.internal) plus the loopback
forms. host.docker.internal is the meaningful one — from inside the admin
container it resolves to the host, and it is what a native host install uses.
A LAN address is deliberately NOT exempt. 192.168.1.50 is indistinguishable
from another machine on the same network, and wrongly exempting it would let a
genuinely remote GPU's throughput be attributed to this hardware. A false block
is recoverable by clearing the setting; a false pass silently corrupts the
board.
Unparseable values fall through to blocked rather than allowed, so a malformed
setting cannot be used to slip past the gate.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
