The model declares `'zim' | 'map' | 'dataset'` and IngestDrugDataJob writes a
'dataset' row for the openFDA drug labels, but the column was still
enum('zim','map') from the table's original migration. Every write failed:
Data truncated for column 'resource_type' at row 1
insert into `installed_resources` (... `resource_type`, ...) values
(..., 'openfda-drug-labels', 'dataset', ...)
The ingest itself succeeded — 261,671 rows landed in drug_labels — but the
install-state write-back at the end of the job was lost every time. Since
getCategoriesWithStatus() resolves a tier only when every resource id is
accounted for, and it already queries whereIn('resource_type', ['zim',
'dataset']), the missing row kept the Medicine category from ever resolving
its Standard or Comprehensive tier. Only the enum was left behind.
Verified against a live install: after the migration the previously failing
insert succeeds, and Medicine resolves installedTierSlug
'medicine-comprehensive'.
down() drops 'dataset' rows before narrowing the column, since MySQL would
otherwise truncate them to '' or fail under STRICT_TRANS_TABLES.
handleSendMessage awaited createChatSession before the optimistic render,
so on a slow request the message the user just sent did not appear at all,
with no spinner and nothing to retry. A failed session creation was worse:
a bare `return` left no message and no error, so the app looked like it had
ignored the user.
Render first, and surface a failure the same way a failed response is
surfaced. Completes #1211 alongside the stream-abort fix in #1220.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
Chat sent num_ctx, num_predict and nothing else, so every turn decoded at
whatever the backend picked: temperature 0.8, top_p 0.9, no min_p. On a
heavily quantized 1-3B model that is the wandering, repetitive output people
read as "the local model is bad".
Adds `ai.responseStyle` with four profiles. min_p is the lever, not
temperature: it floors relative to the top token, so it truncates hard on a
confident distribution and barely at all on a flat one, which is where small
models start to wander. top_p stays at 1.0 rather than stacked on top of it.
repeat_penalty moves with the style, because a grounded answer legitimately
repeats a part number inside the 64-token window: `focused` relaxes it to
1.05, `creative` tightens it to 1.15.
`auto` is the only style that defers to the model. Plenty of library models
ship author-tuned samplers (qwen3 sets 0.6/0.95/20), so those win key by key
and NOMAD fills only the gap, which in practice is min_p. A style the user
picked by name overrides them, or it would silently do nothing on half
their models.
The compat transport drops min_p, so a profile carries a separate compatTopP.
Passing top_p 1.0 to a backend with no min_p behind it would sample the raw
distribution, which is worse than the 0.9 it replaced.
Resolved in the controller, so the grammar-constrained task calls keep their
temperature 0. An explicit temperature still outranks a profile's, which is
what keeps the eval harness deterministic.
`eval:generation --style` is new and load-bearing: temperature 0 makes min_p,
top_p and top_k inert, so a default run cannot see this change at all.
`--style=off` sends no samplers and no temperature, and is the control.
Verified against Ollama 0.32.9, min_p accepted. 22 unit tests.
getModels() and the _isNativeBackend() probe both set isOllamaNative =
false in their catch blocks for ANY failure of GET /api/tags - including
a plain 5s timeout while Ollama is busy or still starting, or a refused
connection while the container is down. The probe memoized that verdict
for the process lifetime. Every model download then failed with 'Model
is not available in your AI host... (model pulling is only supported
for Ollama backends)' on stock single-Ollama installs (#1349, two
reporters on GA).
The fix separates 'the backend answered and is not Ollama' from 'the
request failed'. Only an actual HTTP response - a non-2xx status, or a
200 whose body fails the Ollama shape check - proves a live non-Ollama
server and classifies false. No response at all leaves the
classification untouched (null, or a previously proven value), the
probe unmemoized for re-probing, and the download path falls through to
the pull attempt, whose existing error handling reports the real
network error as a retryable failure.
Verified live (v1.35.0-rc.1 + this patch, Ollama container stopped):
- stock behavior reproduced verbatim: dispatch fails with the
'only supported for Ollama backends' message
- patched: the same dispatch fails with 'Failed to download model:
getaddrinfo ENOTFOUND nomad_ollama' and stays retryable
- after starting Ollama again, the retrying BullMQ job recovered on
its own and installed the model with no user action
Fixes#1349
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ollama/ollama:0.24.0 never cancels in-flight generation when a client
disconnects: the runner process is left spinning at 300%+ CPU with no
active requests and never recovers. NOMAD triggers this organically via
its own 60s RAG embedding timeouts, so slow CPU-only boxes can wedge
themselves after a single slow request. Reported with a standalone
container repro in #1321; the cancellation fix landed upstream between
0.24 and 0.33.
Regression-tested on 0.33.3 (RTX 5060 box):
- model listing via /api/tags (NOMAD backend detection intact)
- GPU acceleration (100% GPU placement, 82.8 tok/s on llama3.1:8b)
- model pull through the NOMAD download pipeline (NDJSON progress)
- embeddings via /api/embed with nomic-embed-text (768-dim)
- streaming chat end-to-end through the Command Center UI
- the disconnect hang itself: container idles at 0.06% CPU 20s after
a mid-generation client drop
The AMD path is unaffected: it always overrides to ollama/ollama:rocm.
The seeder does not sync container_image on existing rows, so this pin
reaches fresh installs; existing installs pick it up via the opt-in app
auto-update or a manual update.
Fixes#1321
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(translate): offline translation for the Information Library
Implements #1272. Ships as a Supply Depot app so users who do not need
translation never see it.
A path-preserving reverse proxy in front of kiwix-serve. /viewer, /skin/*,
/search, /random and /catalog/* are forwarded byte for byte; only text/html
under /content/ is rewritten. Preserving the path is the whole design:
viewer.js keeps the content iframe in sync with location.hash by comparing
contentWindow.location.pathname, so moving the path starts the ping-pong its
own comment warns about and forces us to patch Kiwix's JavaScript on every
Kiwix bump. Because the path never moves, no Kiwix JS is modified, and Kiwix's
header, search, library and random keep working because they are Kiwix.
Language is a cookie, so links inside the ZIM need no rewriting either.
Bergamot rather than the AI Assistant: measured on the same box and the same
passage, 12,818 words/s on CPU against llama3.1:8b at 8 words/s on an RTX 5060.
It is also more careful with names, where the LLM translated the product name
and a smaller model hallucinated a clause. No GPU, so it works on CPU-only
boxes where the LLM path is unusable.
Models come from Firefox Remote Settings, the endpoint Firefox itself uses,
not the mozilla/firefox-translations-models repo, which was archived on
2026-08-21. MPL-2.0, about 37 MB per direction.
Failure handling is deliberate throughout: a failed model fetch is non-fatal, a
translation error returns the original article with a comment rather than an
error page, and a box with no models still starts and still forwards the
library. Translation being unavailable must never make the Information Library
unreachable.
amd64 only, and pinned as such in the build workflow. Bergamot's intgemm
backend is x86 specific and there is no aarch64 wheel, so a multi-arch build
would produce an ARM image that pulls cleanly and dies at start.
Also adds .gitattributes pinning *.sh to LF. The repo stores LF but Windows
checks out CRLF, and a CRLF after a shebang makes the interpreter
unresolvable inside the image. CI builds on Linux so this never bit, but it
breaks a local image build and would have bitten the existing sidecar script
too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
* fix(translate): vary the cache on the language cookie
Found by clicking the language buttons in a browser rather than trusting curl.
Kiwix sends ETag and Cache-Control: max-age=3600, and the proxy forwarded both
unchanged on a response whose content now varies by cookie. The browser cached
the English article for an hour and served it straight back when the reader
picked a language, so the feature looked like it simply did not work. A shared
cache would have been worse, serving one reader's language to another.
Translatable responses now send Vary: Cookie, and their ETag is namespaced by
language so revalidation still works per language rather than being turned off.
Forwarded paths are untouched, so /viewer, /skin/* and /catalog/* stay byte for
byte identical to Kiwix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
* fix(translate): move off port 8480, which Vaultwarden already uses
Caught by seeing the two cards side by side in Supply Depot with the same port.
My port survey grepped ui_location for a bare number, and Vaultwarden's is
'https:8480' because it needs an https Open link, so it did not match and 8480
looked free. Moved to 8460, which fits the existing spacing between Meshtastic
Web on 8450 and Homebox on 8470.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
* fix(translate): pin the base image to python 3.10
The build fails outright on 3.12: bergamot publishes no wheel for it and pip
resolves no further than 0.1.2. The spike used 3.10 for this reason and I had
modernised it without checking. Caught by building the image rather than
reading it.
Moving off this pin means moving off the 2022 PyPI wheel to a build from
mozilla/translations, which is the follow-up already noted in the Dockerfile.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
* fix(translate): translate text that lives in a bare div
Sotoki-generated StackExchange ZIMs put every question excerpt in a
<div class="...excerpt">, so on a questions list the titles translated and the
descriptions underneath them did not. Reported against the ham radio ZIM.
div could not simply be added to BLOCK_TAGS: the outermost-block rule would
claim a top-level layout wrapper and send the whole page as one job. It is now
claimed speculatively and abandoned the moment anything block-level opens
inside it, so a wrapper yields to the blocks it contains and only a genuine
text leaf is translated.
Also moves the test harness's reporting block back to the end of the file. New
checks had been appended after it, so they executed but their failures were
never reported: verified that removing div from the set now fails 3 checks,
where before it reported success.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
* fix(translate): decode numeric character references before translating
Bergamot's HTML mode decodes named entities (`&` arrives as "&") but not
numeric ones. It treats `'` as literal text and escapes the ampersand on
the way out, so "If I'm in the wilderness" came back as
"Si I&#x27;m dans la nature" and rendered the escape sequence visibly to
the user. Confirmed by round-tripping fragments through the model directly.
Numeric refs are now decoded before the round trip. `&`, `<` and `>` are
deliberately left encoded: decoding those would inject real markup into the
fragment and fail the balance check that gates HTML mode, dropping the block to
plain text and losing its inline links.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
applyUpdate guarded with the raw RunDownloadJob.getByUrl and only bailed
out for active/waiting/delayed states, letting failed/completed states
fall through to dispatch. But dispatch adds the job under a deterministic
jobId (sha256 of the URL) with no removeOnFail, so once a download
exhausts its retries the failed job persists in Redis. The next apply
then hits "job already exists", dispatch returns the stale failed job,
and applyUpdate reports success while nothing is re-downloaded — the
resource is stuck forever.
Use getActiveByUrl instead, which removes any terminal job for the URL
before returning, so a fresh job is dispatched. This matches every other
dispatch site (map_service, zim_service, creator_pack_service).
The GpuPassthroughRemediationProvider probed GPU passthrough by exec'ing
`nvidia-smi` inside nomad_ollama. When Ollama was created CPU-only — most
commonly because the NVIDIA container runtime was registered with Docker
only AFTER the AI Assistant was first installed (DockerService attaches a
GPU DeviceRequest only when 'nvidia' is in docker.info().Runtimes at
install time) — the container ships no nvidia-smi, so the exec returned an
error string whose letters the alphabetic-output check mistook for
"passthrough healthy". That false negative suppressed all auto-remediation,
leaving Ollama silently stuck on CPU (models run at a fraction of a token
per second) until the user manually clicked "Fix: Reinstall AI Assistant".
Replace the nvidia-smi probe with Ollama's own "inference compute" startup
log line — the ground truth for whether a GPU backend actually loaded — via
a new, unit-tested classifyOllamaComputeBackend() helper. One signal covers
both the "created CPU-only" case and the original "DeviceRequests present
but the toolkit binding tore" case. Add a cooldown on gpu.autoRemediatedAt
so a GPU that genuinely cannot be accelerated (an architecture Ollama has
no kernels for) does not trigger a reinstall on every admin boot.
Co-Authored-By: Claude <noreply@anthropic.com>
Content Explorer hardcoded `lang: 'eng'`, so it could only ever show
about 1,300 of the catalog's ~10,900 books. Adds a language selector
sourced live from the catalog's own language feed, with book counts,
defaulting to English and remembered per browser.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
* feat(kb): add per-file active/inactive toggle for search inclusion
Implements the per-file toggle agreed on in #1119, in place of disabling
ZIM auto-discovery by default. Any indexed file (ZIM, upload, or other)
can now be excluded from RAG search results instantly, without deleting
or re-embedding its vectors.
- Vectors stay in Qdrant permanently; each point's payload carries an
active boolean. Toggling flips this payload in place, both directions,
with no re-embedding cost.
- Search filter is a denylist (must_not: active == false), not an
allowlist, so a point missing the field entirely is still found by
default -- search correctness never depends on backfill timing.
- embedAndStoreText() looks up a file's existing active state before
writing new points, so retries, force re-embeds, and replaced-content
reindexing never silently reactivate a file the user explicitly
toggled off.
- Applies uniformly to all non-admin_docs buckets (ZIM, uploads, other),
matching how the rest of this pipeline already treats buckets, rather
than special-casing ZIM.
New POST /api/rag/files/active endpoint and KB panel toggle switch,
following the existing updateFileCollection pattern.
Real vector deletion for genuinely deleted/replaced files remains
#1170's separate concern.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* feat(kb): group personal KB uploads by collection with bulk toggle
Groups personal KB uploads into collapsible per-collection header
rows (with an "Uncategorized" bucket for files with no collection) to
reduce clutter in the file table. Each collection header also exposes
a bulk active/inactive toggle to flip every file in that collection at
once.
Also fixes three pre-existing bugs found in the collection-assignment
dropdown while building this: the popover was clipped by table cell
truncation, clipped again by the table wrapper's overflow, and the
combobox query was stale-filtering results on open.
Refs #1193
* fix(kb): widen the modal for the Active column and coerce active to a boolean
Both found by browser-verifying on NOMAD3.
The Active column adds ~57px, which put the table back over the modal width and
reintroduced the exact defect #1198 fixed: the Delete button falls off the right
edge at every viewport. Measured on the box at 1018px of table against 961px of
usable width at max-w-5xl. Widened to 6xl.
MySQL returns tinyint(1) as 0/1 rather than a boolean, and the raw 1 reached the
DOM as aria-checked="1". That is not a valid ARIA value, so a screen reader is
told nothing useful about whether the toggle is on. Coerced in the service so
every consumer sees a real boolean, with a !! guard in Switch as well.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
* fix(kb): validate the bulk collection toggle's input before sanitising
sanitizeCollectionName ran before the type check, so a form-encoded request
sending collection[] as an array reached .trim() on a non-string and produced
an uncaught TypeError, surfacing as a 500. The vision controller already
guards this the same way.
Also documents why requiring `active` to be a real boolean is load-bearing
rather than mere input hygiene: a cross-site HTML form can only send the string
"true", so that check is what keeps a bulk Qdrant write across every point in
the index out of reach of a form POST from another origin. Worth a comment,
because coercing it would look like a harmless usability fix.
Found in a security review of the release PRs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
---------
Co-authored-by: John Cortright <jcortright@zscaler.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
`q=` searches an entry's title and summary, never its filename, so it
cannot match a resource id. It cost up to 5 paged catalog requests per
unresolved book on every check and resolved none. Verified on a test box:
identical results with and without it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
OPDS indexes a book under its base name with the flavour in a separate
element, so `name=wikibooks_en_all_nopic` matched nothing and no
maxi/mini/nopic book could ever resolve an update. Retry against the
base. Also drop the `lang: 'eng'` filter, which made every non-English
installed ZIM permanently unresolvable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
browse.library.kiwix.org grew an anti-crawler interstitial on 2026-08-29
that answers HTTP 200 with an HTML confirmation page. Content Explorer's
remote listing 500s on it and ZIM auto-update silently reports "no
updates". opds.library.kiwix.org serves the same OPDS API ungated.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
The catalog-sync loop overwrote container_config and ui_location for every
non-user-modified curated service, resetting the published host port of an
install that had been moved off the catalog default. The row then disagreed
with the running container: the Open link pointed at a dead port and the
next recreate collided.
Preserve the live host ports for installed rows and apply every other
catalog change, so these installs keep receiving catalog updates instead of
being opted out of them.
Diagnosis and the original fix are @johno10661's work in #1005.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
The swatches reused the tile tint, which is 10% opacity. That reads well across
a 190px card and makes six 28px squares indistinguishable from one another, so
the picker looked like six empty outlines. Caught by looking at it.
Each color now carries a solid class alongside its tint: the picker shows the
color at full strength, the tile stays subtle. A test asserts the swatch class
is solid rather than an opacity variant, since reaching for the tint again is
the obvious mistake.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
"Address" was vague for a field that takes a URL. "No scheme? We'll default
to http://" named a concept most people do not have a word for, and it was not
needed: the live preview already shows http:// appearing the moment someone
types a bare address, which demonstrates the behaviour rather than describing
it. The preview stays, the sentence goes.
The two link tile validation messages follow the same wording. The custom URL
override message is left alone; it belongs to a different feature.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
"still tellable apart from" read badly; it is now "distinguishable from".
Also switches the spelling I had introduced to American, matching the rest of
the product and the existing schema and constants (custom_color, PIN_COLORS).
Limited to the files this branch adds; the two pre-existing benchmark
components that use the British form are left alone rather than churned.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
A link tile rendered on a flat surface colour, which read as a stark white card
against the filled green app tiles. Tiles now take one of six colours from the
brand palette, chosen when creating or editing one, and the card is tinted from
it.
Tiles stay visually distinct from managed apps regardless of colour: the dashed
border and the external-link marker carry that, not the hue, and the tint is
deliberately light so the distinction survives every option.
Class names are written out in full rather than interpolated, because Tailwind
scans source text for literal class strings and would silently drop a computed
one, leaving the tile untinted. A test asserts that.
Added as a separate migration from the one introducing is_link_tile, since that
has already run on the test box and editing it in place would leave fresh and
upgraded installs with different schemas.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
_syncContainersWithDatabase flips any service marked installed to not-installed
when no container exists for it. A link tile has no container by definition, so
every tile was silently unset on the next sync and disappeared from the
dashboard within seconds of being created.
Caught by creating one in the browser and watching it not appear.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
Users want things they already run to appear on the dashboard: a NAS, a router
UI, a printer, a Pi, a container they started themselves. Adds a tile they
address themselves, with a name, one URL field, an optional description and an
icon from the curated set.
Chosen over adopting containers from the Docker daemon, which is what community
PR #1239 proposed. Adoption only reaches containers on this Docker host and has
to guess: it took the lowest published host port and assumed http, which picks
the wrong port for anything publishing several (Portainer 8000 against 9443,
Jellyfin 8096 against 8920) and breaks entirely for HTTPS. A tile the user
addresses has no guessing in it, covers the whole LAN rather than one Docker
host, and has no lifecycle that can break.
One URL field rather than separate host, port and path, because people paste
URLs and it gets https and sub-paths for free. A bare "192.168.1.50:8080" gains
http automatically, which is what most LAN devices need. normalizeCustomUrl and
getServiceLink already did exactly this work for the custom app URL override, so
both are reused rather than duplicated.
Stored as a service row with no container behind it: is_link_tile marks it,
custom_url carries the destination, and the service_name is namespaced
nomad_link_* so it can never collide with a curated entry. The seeder only
iterates its own DEFAULT_SERVICES and never prunes unknown rows, so a reseed
cannot touch or remove a tile.
The tile is deliberately styled differently from a managed app, outlined and
dashed with an external-link marker rather than filled, because if it looked the
same users would expect Start, Stop and Update and file bugs when those controls
are not there. Removing a tile arms a confirm first, since it is user-entered
data with no undo.
Icons come from a fixed 36 shared by the picker and the server-side validator,
so the two cannot disagree about what is selectable. DynamicIcon renders nothing
at all for a name it does not know, which would leave a silent hole in a tile.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
Drops the condition examples from the dashboard tile. Also brings it in
line with the length and punctuation of the neighbouring tiles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
The UI enforces 500 characters but the validator did not, so the API accepted
up to the MySQL TEXT ceiling. The API needs no authentication by design, so an
uncapped free-text column reachable from any page a user visits is a cheap way
to bloat the database. Matches the limit the form already shows.
Found in a security review of the release PRs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
react-icons was added and then removed again during review, and npm on Windows
rewrote the lockfile in passing, stripping the libc platform annotations that
npm on Linux had written. package.json ends up identical to dev, so the correct
lockfile is dev's -- restored verbatim rather than shipping a Windows-rewritten
one into a file that governs platform-specific binaries.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
Deleting a pin was a single click on a trash icon with no confirmation and no
undo. Pin names are short and repetitive by nature ("Well", "Cache", "Fishing
bridge", "Fishing bridge west"), so the name in the row is not enough to tell
the user which place they are about to lose.
The trash icon now arms rather than deletes: it selects the pin and flies the
map to it, then the row offers Delete or Cancel. The user sees the actual
location before confirming.
Escape cancels. The armed state also clears if that pin stops being listed --
deleted from elsewhere, or filtered out by a search -- so a stray confirm can
never end up pointing at a different pin.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
Dropping w-72 when the picker was rewritten left the absolutely positioned
popover to shrink-wrap its content. The six 1fr columns collapsed to a few
pixels each and the 18px glyphs overflowed and overlapped, so the grid rendered
as a smear. Restored the explicit width, added shrink-0 to the cells so a
narrow container can never squash them again, and centred them in their tracks.
Caught by looking at it. The DOM assertions I had run -- 36 buttons in a
grid-cols-6 -- all passed while the thing was visually broken, which is the
whole reason a frontend change needs eyes on it and not just a query.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
Review changes from looking at the feature running on NOMAD3.
The icon picker offered every Tabler icon *and* every Font Awesome icon via
namespace imports: 160 pages, whose first page included the Amazon and Amazon
Pay logos, an "Ad" badge and text-alignment glyphs. It also took the maps chunk
from 864 kB to 5,774 kB. Replaced with a hand-written set of 36 map-relevant
icons in a 6x6 grid -- water, shelter, medical, power, comms, transport,
terrain, hazards -- with no search box or paging, because 36 fit on screen.
Maps chunk is back to 904 kB, +40 kB over dev rather than +4,910 kB, and the
react-icons dependency is gone entirely since Tabler was already present.
Icon resolution moved into marker_icons.ts so the panel and the pin share one
definition, with a caller-supplied fallback (the pin's inner glyph defaults to
a filled circle, the list to a map pin). An icon later removed from the set
still renders as the fallback rather than blanking the marker.
Dropped the tree view and the List/Tree band: grouping pins by hue or by icon
organises them by an attribute the user picked arbitrarily, and grouping by
first letter is worse than sorting. Search and sort cover the real need. "Hide
all" lived inside the tree view, so it moved up into the header where it
applies to whatever the list shows -- it is the one bulk control worth keeping.
The title bar now closes the panel, matching the "Pins" button that opens it;
clicking a title to open but having to hit a small X to close is an asymmetry
that makes the panel feel fiddly. The list also reclaims the height the removed
band was using.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
Salvages @kennethbrewer3's work from #836, which had drifted onto a stale base
(GitHub showed 231 files; the real change is the map UI plus a migration).
Marker visibility (individually and Hide All), per-marker icon and custom hex
colour with contrast-adjusted icon colour, search, sorting and a tree view over
the marker list, coordinate search and fly-to with URL parameters
(?lat=&lng=&zoom=), creating a marker from typed coordinates, and markers
rendered as SVGs anchored to true coordinates rather than approximately.
Name, notes and preset colour already existed; the new columns are
custom_color, icon, icon_color and visible.
Dropped from the original as stale-base noise rather than his work: unrelated
duplicate changes to the Ollama, Docker, download, RAG and ZIM services, the
API reference and getting-started docs, README/FAQ/CONTRIBUTING, the installer
script, and a revert of AppLayout to a version predating the compact sidebar.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
It constructs OllamaController, which imports the Adonis logger service at
module scope, so it cannot run under the plain node:test runner. It failed
identically on the original branch -- the coverage has never executed. Moving
it to the Japa runner puts it in the same bucket as the other app-importing
specs, where a booted app is available.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
Closes#1176. Salvages @gabegraves's work from #1178, rebased onto the
RagPipelineService refactor that landed after it was written.
Adds JPEG/PNG/WebP attachments to chat over multipart, leaving text-only JSON
requests on their existing path. Images are normalized server-side to bounded
JPEG with metadata stripped, capped on count/size/pixels/output, and the
temporary uploads are cleaned up on every validation path. They are attached
only to the latest user message, are excluded from RAG, titles, logs and
persisted history, and are never written to storage.
Vision support is resolved per model from what the backend actually reports --
OpenAI-compatible /v1/models metadata, then Ollama /api/show, then llama.cpp
/props -- never inferred from a model name. A model that reports no vision
support has attachments disabled with a visible explanation; a backend that
reports nothing is allowed with a warning and its rejection is translated into
plain compatibility guidance rather than a raw error.
Changes from the original:
- Prompt assembly moved into RagPipelineService in the meantime, so the
controller half is rebuilt against the trace and the multipart-aware
collection filter is now the only one (dev's request.input() version would
have silently dropped the collection on every image request).
- Added a native-path converter. dev split chat into native and OpenAI-compat
paths after this was written; Ollama's native /api/chat does not take OpenAI
content parts, it takes a sibling `images` array of bare base64. Without this
every image request on a native Ollama backend would fail.
- Dropped the model-recommendation change (selectRecommendedModels and the
Qwen3-VL fallback entry). Displacing a recommended model at first-time setup
is a product decision and is being taken separately. The "Supports images"
label in Easy Setup is kept: it describes what is already on offer rather
than changing it.
- Dropped the DEFAULT_QUERY_REWRITE_MODEL download prompt his branch carried,
which dev deleted in d022c2d.
- Removed a duplicate checkModelHasThinking; kept dev's memoized getModelInfo
path, since thinking detection is not part of this feature.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
Closes#1179.
Surfaces the documents an answer was drawn from as a "Sources" list under the
assistant message, with the archive date where one is known, and persists it so
reopening a past conversation still shows what backed each answer.
Offline there is no second opinion to check an answer against, so provenance is
the only signal the user has that a claim came from a real document rather than
from the model. This is the other half of retrieval being allowed to decline:
one says when nothing relevant was found, this says what was found.
Citations are built from the chunks actually injected into the prompt, not from
everything retrieval returned. A chunk dropped by the budget planner never
reached the model, so citing it would credit the answer to a document it was
not based on -- the exact failure the list exists to prevent.
The archive title and date have been written into the vector payload by the ZIM
ingestion path all along and were simply never read back, so this works on
already-embedded content with no re-embed.
Rebased onto the RagPipelineService refactor, which landed after this was
written: the citation list now comes off the pipeline trace rather than the
inline retrieval block that used to live in the controller. Entries with
neither a path nor a title are skipped rather than shown as "Unknown source" --
a row the user cannot act on makes an answer look sourced when it is not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
#1165 taught the benchmark path to reject an unresolved PCI id as a GPU
model, but isUnresolvedGpuModel lived inside benchmark_service.ts and the
Settings > System display path never learned the same thing. It still shows
whatever lspci handed back.
The two symptoms of a stale pci.ids database are independent, which is why
the existing guard misses this. system_service already re-probes when a
discrete-GPU controller reports an implausible BAR0 reading as VRAM, and
that happens to fix the name as a side effect. But lspci can return a
perfectly plausible VRAM figure alongside a name it could not resolve, and
then nothing fires.
Confirmed live on an AMD box. The Radeon 780M reports:
model: "Device 1900", vram: 512
512 MiB clears the 256 MiB bogus-VRAM threshold, so no probe runs and the
System page renders "Device 1900". The same box's Ollama startup log carries
description="AMD Radeon 780M Graphics", so the probe the guard declined to
run had the real name available the whole time. NVIDIA hides this because
its path tends to be reached for other reasons; AMD has no equivalent.
Adds an unresolved-name trigger alongside the VRAM one, and moves
isUnresolvedGpuModel to app/utils/gpu_model.ts so the submission path and the
display path share one definition instead of diverging again. Tests cover
both placeholder shapes and, more importantly, a set of real product names
that must never be rejected.
Stacked on #1277, so the shared helper carries the WSL placeholders too.
Closes#1196
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
_embedWithFallback() always tried Ollama's native /api/embed first and only
reached /v1/embeddings from the catch block. Against a backend that only
speaks the OpenAI embeddings API, every batch therefore paid a
guaranteed-failing round trip plus a warn line before doing the real work.
Invisible on a handful of documents. Expensive on a bulk file-embeddings
job: the reporter is indexing several million chunks through an
OpenAI-compatible proxy, which doubles request volume and log noise against
their embedding host for the entire lifetime of the ingest.
Gates the native attempt on _isNativeBackend(), matching how _doDownloadModel
already skips /api/pull. Used the probe rather than reading isOllamaNative
directly: the field is null until something populates it, and an embed job
can easily be the first thing to touch the service after a restart, which is
exactly when a long ingest is starting and the saving matters most. The probe
memoizes for the process lifetime, so this costs one request overall rather
than one per batch.
Behaviour is otherwise unchanged. A native backend still tries /api/embed
first and still falls back if it fails for a non-context reason, and
context-length errors still bubble to embed() for the truncated retry. The
response-shape validation stays: the probe establishes that /api/tags
answered like Ollama, not that /api/embed will.
Reported by @horsleyb.
Closes#1279
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Measured on a running install rather than estimated. The README already
gave 4 GB and 32 GB; what was missing is why the gap is that wide, which
is the model rather than NOMAD, and whether it comes out of VRAM or
system RAM.
Asked for by @mk-pmb on #1271.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
The quickstart never said how much free space is required to install, which is
the first thing anyone plans around on a project whose whole purpose is
downloading large offline libraries. The FAQ's storage question only covered
content sizes, so it answered "how big is Wikipedia" but not "can I install
this at all".
Measured on a v1.34.0 install: admin 2.81GB, MySQL 1.1GB, plus redis, dozzle,
the disk collector and the sidecar updater at roughly 300MB combined. That is
about 4.2GB of images, so ~5GB installed and 10GB free to install comfortably.
The AI Assistant adds the Ollama image at 10.6GB plus a model (llama3.1:8b is
~4.9GB), which is where the ~25GB figure comes from.
Reported in #1271, which proposed the README sentence with the number left
blank.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LtGWCVMPYn6qPWVYiycK74
Under WSL2 the GPU is reached through /dev/dxg rather than the real adapter, so
si.graphics() returns Microsoft's generic placeholder even while CUDA work runs
on a physical card. isUnresolvedGpuModel() caught blanks, "Device <hex>" and
"Unknown" but not this, so the placeholder was accepted as a legitimate model
name and the authoritative detection path was skipped.
The reporter on #1218 has an RTX 3090 doing real work (92% utilisation, 11.6 GB
VRAM, 352 W, 127.9 tok/s) that reaches the public leaderboard labelled
"Microsoft Basic Render Driver". A wrong label is worse than none: it fragments
per-hardware grouping, which is the same failure #1165 fixed for raw PCI ids.
Adds Microsoft's two placeholder adapter names to the same rejection list.
Scoped so it cannot affect anyone off WSL: these are Microsoft's own placeholder
names and appear nowhere outside a Windows/WSL graphics stack, so on a native
Linux host the function returns exactly what it returns today. Checked against
real product names from all four vendors, including "Microsoft Basic Renderer
Pro 9000" as a near-miss, to confirm the anchored pattern cannot swallow one.
Note this makes the model resolve to unknown rather than to "RTX 3090": the
admin container has no nvidia-smi and WSL's copy lives at /usr/lib/wsl/lib,
outside the container. Honest-unknown is the correct outcome here; naming the
card under WSL is a separate problem.
Refs #1218
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LtGWCVMPYn6qPWVYiycK74
When an AMD GPU is detected, the install switches to the ROCm image and passes
/dev/kfd and /dev/dri into the container. _discoverAMDDevices() returns both
unconditionally, with no check that they exist. Detection reads PCI data, which
says nothing about whether the ROCm kernel interface was ever loaded, so on a
pre-ROCm card or a host without amdgpu the container is rejected:
(HTTP code 500) server error - error gathering device information while
adding custom device "/dev/kfd": no such file or directory
and the install dies. There is an opt-out, ai.amdGpuAcceleration, but it is a
backend-only KV key with no settings toggle, no route and no UI, so an affected
user cannot install the AI Assistant at all through the interface even though
CPU-only would work fine on their machine.
Fall back to the CPU image instead, with a message explaining why.
Two things about the shape of this fix, both measured rather than assumed:
- It hooks start(), not createContainer(). The daemon accepts a container whose
--device path does not exist and only resolves devices on start. Verified
against Docker 29.6.2 over the socket: create returns 201, start returns 500
with the message above.
- It retries rather than pre-checking. The admin runs in its own container, so
stat()ing /dev/kfd describes the admin's namespace, not the host's. Verified
on NOMAD6, a working ROCm box (Ollama running ollama/ollama:rocm with both
devices attached): /dev/kfd is PRESENT on the host and ABSENT inside the admin
container. A pre-check would therefore have stripped GPU passthrough from a
machine where AMD acceleration works.
The matcher keys on the device-gathering phrase rather than a bare "no such file
or directory", which Docker also emits for missing bind-mount sources; falling
back to CPU on one of those would hide a real failure behind a degraded install.
Checked against all three formats the daemon/dockerode produce, and against a
port-conflict message, to confirm it discriminates.
The success path is unchanged: the create payload is built by one helper used by
both attempts, and the fallback only runs after a start() failure that matches.
Fixes#1232
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LtGWCVMPYn6qPWVYiycK74
Ollama's /api/pull returns HTTP 200 and reports failures IN-BAND, as a JSON
line inside the NDJSON stream body:
HTTP_STATUS=200
{"status":"pulling manifest"}
{"error":"pull model manifest: file does not exist"}
The stream handler only ever read completed/total/digest off each line and
silently dropped everything else, including that error line. The stream then
ended normally, on('end') resolved unconditionally, and we logged the model as
downloaded successfully. on('error') never fires for this, because at the HTTP
layer the request succeeded.
Every failed model pull, for any reason, was reported as a success. On #1071
that surfaced as an ~800 GB fp16 model "downloading" in two seconds and nothing
being installed, which had been triaged twice as disk space or a stale blob.
Those explanations produce workarounds that cannot work, since the pull never
reaches the disk at all.
Three changes:
- Check parsed.error in the data handler and destroy the stream with it, so the
existing catch reports a real failure.
- Verify the model is actually in getModels() before returning success. This is
the load-bearing half: it catches silent failures we have not anticipated,
not only the ones Ollama labels. includeEmbeddings must be true or
nomic-embed-text fails verification after a good pull, and names are
normalised because Ollama resolves a bare name to ":latest".
- Treat an in-band "model does not exist" as permanent. It was retryable, so a
bad model name spun through attempts: 10 on exponential backoff and the user
watched a delayed job for hours instead of being told the name was wrong.
Verified on NOMAD3 against the shipped v1.34.0 build, which carries the same
defect:
- bogus tag, before: 'Model "..." downloaded successfully.'
- bogus tag, after: 'Failed to download model "...": pull model manifest:
file does not exist', and the job stops at 1 attempt instead of retrying
- real model (qwen2.5:0.5b): still succeeds, passes verification, and is
present in `ollama list`
Fixes#1273
Refs #1071
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LtGWCVMPYn6qPWVYiycK74
Three dispatch sites pin a deterministic jobId. `queue.add` with an existing
custom jobId returns the existing job rather than throwing, so any retained
terminal record makes later dispatches a silent no-op while still reporting
success to the caller.
- download_model_job: only `failed` records were cleared, so once a model
downloaded successfully its retained completed job blocked every later
request for that model. Deleting the model and reinstalling it did nothing,
and a restart did not help because the record is persisted in Redis. Adds
getActiveByModelName, mirroring RunDownloadJob.getActiveByUrl.
- download_service.retryFailedJob: the model branch dispatched before removing,
so remove() targeted the job just enqueued under the same id instead of the
old record. Flipped to remove-then-dispatch, matching the file branch.
- embed_file_job: a retained failed record made re-indexing that file a no-op,
returning 202 "Indexing queued" with nothing enqueued.
In-flight jobs are still returned as-is, so re-clicking during an active
download or embed stays idempotent.
Diagnosis by @caweis in #1214, including the retry-ordering race and the embed
case. Reimplemented in-house rather than ported, as noted there.
Closes#1214
Fixes#1269. Fixes#1203.
`file-embeddings` and `downloads` both routinely outlive the default
5 minute lock, so BullMQ calls healthy jobs stalled on both. They need
opposite recovery policies, which is why neither was covered by the
existing drug-queue override.
Embedding is a self-continuing chain, so re-queueing a stalled job while
the original is alive runs two chains over the same file. Both write to
Qdrant with freshly minted point ids, so the second cannot overwrite the
first, and the queue's concurrency of 2 means the fork never contends
for a slot. maxStalledCount 0 fails on the first stall instead.
A download is resumable, so recovery is safe there; the default of 1 was
failing jobs outright and bypassing their own attempts: 10 policy.
The decision moves to a pure, unit-tested util alongside the other
decision helpers.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LtGWCVMPYn6qPWVYiycK74
The batch continuation gated on articles that produced chunks rather than
articles the extractor consumed. Articles whose text is empty after HTML
cleaning — redirect stubs, category listings, media and PDF wrappers —
return no chunks and so no documentIds, which made a normal sparse batch
look like the end of the archive. Ingestion stopped after the first batch
and the rest of the file was silently skipped.
Medicine LibreTexts embedded 16 chunks out of 23,171 articles: its first
50 entries are navigation pages, only 10 of which had text, so 10 >= 50
was false and no continuation was ever dispatched. Measured against the
same install, WikiMed reached 28% of its articles, Libre Pathology 31%,
MedlinePlus 84%, and CDC Travelers' Health 2.7%.
The same value was also returned as `articlesProcessed`, which
EmbedFileJob uses to advance the next batch offset. Advancing by the
content-bearing count re-read the overlap on every sparse batch, and a
batch yielding no text at all would have advanced by zero and re-run the
same window indefinitely.
Both now use the extractor's consumed-article count, which it already
computed and logged but discarded. The end-of-archive signal is the
iterator running dry (articlesProcessed < batchSize); archive.articleCount
cannot serve as a bound because iterByPath() legitimately yields more
article entries than that figure for some archives.
The log line now reports both counts (content-bearing/consumed) so a
sparse batch is visible without re-deriving it.
The history table rendered nomad_score for every row, so a v2 run showed its
legacy score while the Benchmark Details card directly above it showed the v2
one. The same run read as 65.0 in the table and 1036.1 in the card.
Rows predating v2 have no v2 score and can only show the legacy number, so the
two scales necessarily share a column. They differ by more than 10x, and
without the scale on screen a v2 run sitting above older runs reads as a
collapse rather than a rescale. Legacy rows now carry a "/ 100" suffix, which
names the scale in place and reuses the wording the details card already uses
("Legacy scale: X / 100").
This compounds the re-run guidance in the release notes: someone re-runs on
1.34 as asked, gets a real v2 score, and their own history then shows the old
number as though the machine got slower.
Frontend only. getAllResults() already returns full rows, so nomad_score_v2
was in the payload the whole time.
Verified on the NOMAD2 dev environment with a seeded history matching a real
mixed case (one v2 run at 1036.1 plus two pre-v2 runs): the v2 row shows
1036.1 with no suffix and agrees with the details card, and the legacy rows
show 71.1 / 100 and 86.5 / 100. Inertia typecheck shows the same 30
pre-existing errors as dev with none in this file.
There was one cutoff, Qdrant's score_threshold at 0.3, and nomic scores
unrelated text well above it, so every question retrieved *something*
(nonEmptyRateOnRefusal 1.0) and rag_context wasted four lines asking a 3B
model to "silently judge" relevance. That is a classification a number
should decide, not a micro model.
Adds `RAG_MIN_FINAL_SCORE`, a floor on the post-rerank score, applied before
source diversity. The floor judges relevance, the 0.85^n penalty judges
redundancy. Nothing clears it and nothing is injected. `score_threshold` stays
at 0.3 as the candidate net. Exposed as `rag.minRelevance` with a module-
scoped TTL cache invalidated from `updateSetting`.
The floor is calibrated and the harness was describing the wrong number:
aggregate() bucketed semanticScore, so its percentiles measured raw cosine
while the new knob filters the reranked score. Both axes now, and the gap
matters: recall falls at 0.60 on the semantic axis, 0.66 on the final one.
floor recall@5 precision@5 emptyOnAnswerable nonEmptyOnRefusal
0.00 0.991 0.221 0% 100%
0.62 0.991 0.524 0% 60%
0.66 0.991 0.695 0% 60%
0.67 0.972 0.704 1% 60%
Default 0.62, not the last-lossless 0.66: the refusal win is complete at
0.62 and 28 documents is a thin basis for spending margin. 0.60 is the
honest floor. Three of five refusal goldens are near-corpus by
construction, where declining is the generation tier's job.
Both eval tiers pin the floor to the constant and ignore the setting, for
the same reason the harness never sets skipRetrieval.
4 metrics improved against baseline, no regressions.
