GpuPassthroughRemediationProvider returned early unless the NVIDIA
runtime was registered with Docker. AMD doesn't register one, so the
log-based CPU-fallback detection added in v1.35.0-rc.1 never ran on
AMD boxes. On a Phoenix1 iGPU running ollama:rocm with /dev/kfd and
/dev/dri passed through, Ollama logged library=cpu while gpuHealth
reported status "ok".
The gpuHealth side had its own bug. hasNvidiaRuntime and
hasRocmRuntime were set only inside the lspci probe branch. When lspci
returned a plausible controller, the else branch reported "ok" without
probing and left both flags false. A blind reinstall would not have
helped either: gfx1103 needs HSA_OVERRIDE_GFX_VERSION=11.0.0, and
without the gfx marker or ai.amdHsaOverride a reinstall rebuilds the
same CPU-bound container.
- Add resolveExpectedGpuVendor(). It reads the nomad_ollama container
config first (nvidia DeviceRequest, ROCm image tag, /dev/kfd), then
the nvidia runtime, then the AMD GPU type when no container exists.
/dev/dri alone is not treated as AMD. A CPU container on an AMD box
is trusted as deliberate (#1232 fallback, acceleration disabled).
- Gate the provider on the expected vendor. NVIDIA auto-reinstall is
unchanged but now skips when the container requests a GPU with no
runtime registered. AMD is detect-only: the provider logs whether a
reinstall would change the container and never reinstalls itself.
- In gpuHealth, set the runtime flags up front and let Ollama's
startup log decide health whenever a GPU is expected, whatever lspci
returned. A CPU-only log now reports passthrough_failed and skips
the nvidia-smi exec. lspci stays the display source when clean.
- Parse the first GPU "inference compute" line instead of the last
line, which could be the CPU fallback device, and recognize Vulkan.
Move the startup-log reader into ollama_compute so both callers
share it.
- Report amdReinstallWouldChange, amdGfxTarget, currentHsaOverride and
suggestedHsaOverride on AMD passthrough_failed. The gfx target is
parsed from Ollama's "amdgpu is not supported" line (best-effort).
- Add GpuPassthroughAlert, shared by Settings > System and Settings >
Models. When a reinstall would rebuild the same AMD container, it
offers "Fix: Set GFX Override" instead of a reinstall. The dialog
saves ai.amdHsaOverride and then reinstalls.
- Expose ai.amdHsaOverride through the settings API. The value lands
in the container env, so only x.y.z, "none" or empty is accepted.
- Silence the HSA override resolver's logging on the gpuHealth poll.
The #1325 case where no GPU runtime is expected and lspci reports a
controller still returns "ok" unprobed.
Fixes#1344
Refs #1325, #1232
The post-rerank relevance floor rejects noise but passes coherent
off-topic questions, and it gets worse as the library grows. Adding
Gutenberg prose, the Python tutorial and MDN pages to the eval corpus
took the off-topic leak rate from 22% to 67% at the 0.62 floor.
Score cutoff does not fix it. The weakest correct top hit scores 0.666
while "write me a haiku about autumn leaves" pulls Walden at 0.705.
A cosine floor and a top-vs-median contrast test both cost recall
faster than they cut leaks.
- Add rag.relevanceCheck (default off). After the floor, the tasks
model, or the chat model when none is set, gives one grammar-constrained
on_topic verdict for the whole retrieved set. Off topic means no
context and no citations. Any failure keeps the chunks unjudged.
- Add a Settings > Models toggle recommending an 8B+ tasks model.
On llama3:8b the check cuts off-topic leaks from 67% to 6% for about
5 points of core recall@5 and ~0.6s per turn. On 3B models it rejects
most valid questions, which is why it is opt-in.
- Add 6 distractor documents and an off_topic suite (18 unanswerable
questions, 7 distractor controls) to the eval corpus.
- Report per-tag leak rates and leaked cases with top scores in
eval:retrieval. Add --judge-model to measure the check and --dump to
export pre-floor candidate pools for offline gate simulation.
- Pin the check off in eval:generation so a local setting cannot move
scores.
- Record the distractor sweep on RAG_MIN_FINAL_SCORE. The 0.62 default
stays: 0.65 halves leaks at no measured recall cost but leaves only
0.016 of margin.
The corpus fingerprint changes to b4ebe5dce8699f5d. Retrieval reports
from before this commit are not comparable, and a new baseline is
committed.
Refs #1341
Chat sent num_ctx, num_predict and nothing else, so every turn decoded at
whatever the backend picked: temperature 0.8, top_p 0.9, no min_p. On a
heavily quantized 1-3B model that is the wandering, repetitive output people
read as "the local model is bad".
Adds `ai.responseStyle` with four profiles. min_p is the lever, not
temperature: it floors relative to the top token, so it truncates hard on a
confident distribution and barely at all on a flat one, which is where small
models start to wander. top_p stays at 1.0 rather than stacked on top of it.
repeat_penalty moves with the style, because a grounded answer legitimately
repeats a part number inside the 64-token window: `focused` relaxes it to
1.05, `creative` tightens it to 1.15.
`auto` is the only style that defers to the model. Plenty of library models
ship author-tuned samplers (qwen3 sets 0.6/0.95/20), so those win key by key
and NOMAD fills only the gap, which in practice is min_p. A style the user
picked by name overrides them, or it would silently do nothing on half
their models.
The compat transport drops min_p, so a profile carries a separate compatTopP.
Passing top_p 1.0 to a backend with no min_p behind it would sample the raw
distribution, which is worse than the 0.9 it replaced.
Resolved in the controller, so the grammar-constrained task calls keep their
temperature 0. An explicit temperature still outranks a profile's, which is
what keeps the eval harness deterministic.
`eval:generation --style` is new and load-bearing: temperature 0 makes min_p,
top_p and top_k inert, so a default run cannot see this change at all.
`--style=off` sends no samplers and no temperature, and is the control.
Verified against Ollama 0.32.9, min_p accepted. 22 unit tests.
* feat(translate): offline translation for the Information Library
Implements #1272. Ships as a Supply Depot app so users who do not need
translation never see it.
A path-preserving reverse proxy in front of kiwix-serve. /viewer, /skin/*,
/search, /random and /catalog/* are forwarded byte for byte; only text/html
under /content/ is rewritten. Preserving the path is the whole design:
viewer.js keeps the content iframe in sync with location.hash by comparing
contentWindow.location.pathname, so moving the path starts the ping-pong its
own comment warns about and forces us to patch Kiwix's JavaScript on every
Kiwix bump. Because the path never moves, no Kiwix JS is modified, and Kiwix's
header, search, library and random keep working because they are Kiwix.
Language is a cookie, so links inside the ZIM need no rewriting either.
Bergamot rather than the AI Assistant: measured on the same box and the same
passage, 12,818 words/s on CPU against llama3.1:8b at 8 words/s on an RTX 5060.
It is also more careful with names, where the LLM translated the product name
and a smaller model hallucinated a clause. No GPU, so it works on CPU-only
boxes where the LLM path is unusable.
Models come from Firefox Remote Settings, the endpoint Firefox itself uses,
not the mozilla/firefox-translations-models repo, which was archived on
2026-08-21. MPL-2.0, about 37 MB per direction.
Failure handling is deliberate throughout: a failed model fetch is non-fatal, a
translation error returns the original article with a comment rather than an
error page, and a box with no models still starts and still forwards the
library. Translation being unavailable must never make the Information Library
unreachable.
amd64 only, and pinned as such in the build workflow. Bergamot's intgemm
backend is x86 specific and there is no aarch64 wheel, so a multi-arch build
would produce an ARM image that pulls cleanly and dies at start.
Also adds .gitattributes pinning *.sh to LF. The repo stores LF but Windows
checks out CRLF, and a CRLF after a shebang makes the interpreter
unresolvable inside the image. CI builds on Linux so this never bit, but it
breaks a local image build and would have bitten the existing sidecar script
too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
* fix(translate): vary the cache on the language cookie
Found by clicking the language buttons in a browser rather than trusting curl.
Kiwix sends ETag and Cache-Control: max-age=3600, and the proxy forwarded both
unchanged on a response whose content now varies by cookie. The browser cached
the English article for an hour and served it straight back when the reader
picked a language, so the feature looked like it simply did not work. A shared
cache would have been worse, serving one reader's language to another.
Translatable responses now send Vary: Cookie, and their ETag is namespaced by
language so revalidation still works per language rather than being turned off.
Forwarded paths are untouched, so /viewer, /skin/* and /catalog/* stay byte for
byte identical to Kiwix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
* fix(translate): move off port 8480, which Vaultwarden already uses
Caught by seeing the two cards side by side in Supply Depot with the same port.
My port survey grepped ui_location for a bare number, and Vaultwarden's is
'https:8480' because it needs an https Open link, so it did not match and 8480
looked free. Moved to 8460, which fits the existing spacing between Meshtastic
Web on 8450 and Homebox on 8470.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
* fix(translate): pin the base image to python 3.10
The build fails outright on 3.12: bergamot publishes no wheel for it and pip
resolves no further than 0.1.2. The spike used 3.10 for this reason and I had
modernised it without checking. Caught by building the image rather than
reading it.
Moving off this pin means moving off the 2022 PyPI wheel to a build from
mozilla/translations, which is the follow-up already noted in the Dockerfile.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
* fix(translate): translate text that lives in a bare div
Sotoki-generated StackExchange ZIMs put every question excerpt in a
<div class="...excerpt">, so on a questions list the titles translated and the
descriptions underneath them did not. Reported against the ham radio ZIM.
div could not simply be added to BLOCK_TAGS: the outermost-block rule would
claim a top-level layout wrapper and send the whole page as one job. It is now
claimed speculatively and abandoned the moment anything block-level opens
inside it, so a wrapper yields to the blocks it contains and only a genuine
text leaf is translated.
Also moves the test harness's reporting block back to the end of the file. New
checks had been appended after it, so they executed but their failures were
never reported: verified that removing div from the set now fails 3 checks,
where before it reported success.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
* fix(translate): decode numeric character references before translating
Bergamot's HTML mode decodes named entities (`&` arrives as "&") but not
numeric ones. It treats `'` as literal text and escapes the ampersand on
the way out, so "If I'm in the wilderness" came back as
"Si I&#x27;m dans la nature" and rendered the escape sequence visibly to
the user. Confirmed by round-tripping fragments through the model directly.
Numeric refs are now decoded before the round trip. `&`, `<` and `>` are
deliberately left encoded: decoding those would inject real markup into the
fragment and fail the balance check that gates HTML mode, dropping the block to
plain text and losing its inline links.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The swatches reused the tile tint, which is 10% opacity. That reads well across
a 190px card and makes six 28px squares indistinguishable from one another, so
the picker looked like six empty outlines. Caught by looking at it.
Each color now carries a solid class alongside its tint: the picker shows the
color at full strength, the tile stays subtle. A test asserts the swatch class
is solid rather than an opacity variant, since reaching for the tint again is
the obvious mistake.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
"still tellable apart from" read badly; it is now "distinguishable from".
Also switches the spelling I had introduced to American, matching the rest of
the product and the existing schema and constants (custom_color, PIN_COLORS).
Limited to the files this branch adds; the two pre-existing benchmark
components that use the British form are left alone rather than churned.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
A link tile rendered on a flat surface colour, which read as a stark white card
against the filled green app tiles. Tiles now take one of six colours from the
brand palette, chosen when creating or editing one, and the card is tinted from
it.
Tiles stay visually distinct from managed apps regardless of colour: the dashed
border and the external-link marker carry that, not the hue, and the tint is
deliberately light so the distinction survives every option.
Class names are written out in full rather than interpolated, because Tailwind
scans source text for literal class strings and would silently drop a computed
one, leaving the tile untinted. A test asserts that.
Added as a separate migration from the one introducing is_link_tile, since that
has already run on the test box and editing it in place would leave fresh and
upgraded installs with different schemas.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
Users want things they already run to appear on the dashboard: a NAS, a router
UI, a printer, a Pi, a container they started themselves. Adds a tile they
address themselves, with a name, one URL field, an optional description and an
icon from the curated set.
Chosen over adopting containers from the Docker daemon, which is what community
PR #1239 proposed. Adoption only reaches containers on this Docker host and has
to guess: it took the lowest published host port and assumed http, which picks
the wrong port for anything publishing several (Portainer 8000 against 9443,
Jellyfin 8096 against 8920) and breaks entirely for HTTPS. A tile the user
addresses has no guessing in it, covers the whole LAN rather than one Docker
host, and has no lifecycle that can break.
One URL field rather than separate host, port and path, because people paste
URLs and it gets https and sub-paths for free. A bare "192.168.1.50:8080" gains
http automatically, which is what most LAN devices need. normalizeCustomUrl and
getServiceLink already did exactly this work for the custom app URL override, so
both are reused rather than duplicated.
Stored as a service row with no container behind it: is_link_tile marks it,
custom_url carries the destination, and the service_name is namespaced
nomad_link_* so it can never collide with a curated entry. The seeder only
iterates its own DEFAULT_SERVICES and never prunes unknown rows, so a reseed
cannot touch or remove a tile.
The tile is deliberately styled differently from a managed app, outlined and
dashed with an external-link marker rather than filled, because if it looked the
same users would expect Start, Stop and Update and file bugs when those controls
are not there. Removing a tile arms a confirm first, since it is user-entered
data with no undo.
Icons come from a fixed 36 shared by the picker and the server-side validator,
so the two cannot disagree about what is selectable. DynamicIcon renders nothing
at all for a name it does not know, which would leave a silent hole in a tile.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
There was one cutoff, Qdrant's score_threshold at 0.3, and nomic scores
unrelated text well above it, so every question retrieved *something*
(nonEmptyRateOnRefusal 1.0) and rag_context wasted four lines asking a 3B
model to "silently judge" relevance. That is a classification a number
should decide, not a micro model.
Adds `RAG_MIN_FINAL_SCORE`, a floor on the post-rerank score, applied before
source diversity. The floor judges relevance, the 0.85^n penalty judges
redundancy. Nothing clears it and nothing is injected. `score_threshold` stays
at 0.3 as the candidate net. Exposed as `rag.minRelevance` with a module-
scoped TTL cache invalidated from `updateSetting`.
The floor is calibrated and the harness was describing the wrong number:
aggregate() bucketed semanticScore, so its percentiles measured raw cosine
while the new knob filters the reranked score. Both axes now, and the gap
matters: recall falls at 0.60 on the semantic axis, 0.66 on the final one.
floor recall@5 precision@5 emptyOnAnswerable nonEmptyOnRefusal
0.00 0.991 0.221 0% 100%
0.62 0.991 0.524 0% 60%
0.66 0.991 0.695 0% 60%
0.67 0.972 0.704 1% 60%
Default 0.62, not the last-lossless 0.66: the refusal win is complete at
0.62 and 28 documents is a thin basis for spending margin. 0.60 is the
honest floor. Three of five refusal goldens are near-corpus by
construction, where declining is the generation tier's job.
Both eval tiers pin the floor to the constant and ignore the setting, for
the same reason the harness never sets skipRetrieval.
4 metrics improved against baseline, no regressions.
Ollama's format parameter turns a formatting request into a decoder
constraint, and NOMAD's three ancillary calls — chat title, suggestion
chips, RAG query rewrite — were the ones paying for not having it.
The rewrite is particularly important: whatever it returned got embedded
and sent to Qdrant, so a malformed response degraded retrieval for the
rest of the conversation.
Three explicit design choices made:
- Native-only. The compat path sends nothing new and keeps its string
parsing, so a non-Ollama backend that rejects response_format can't
take out title generation entirely.
- parseStructured returning null is the control flow. It's how each
caller falls back to the parser it already had, which is why it's written
to be total and never throw.
- {queries: []} not {query: ""} for the rewrite (tee-ing up future
improvements)
Chat was run through Ollama's OpenAI-compatible /v1 endpoint, which has no
context-size field, so the num_ctx we sent was silently discarded and the
entire NUM_CTX_LADDER was dead code. Every conversation ran at the backend
default and was truncated from the middle (where history and
retrieved context actually sit). Measured with identical ~7k-token prompts:
2,060 tokens processed against 7,049 on the native endpoint, and with the
needed fact in the discarded middle the old path hallucinated a plausible
wrong number.
Transport: add a native /api/chat path using the already-installed ollama
package, sending a real options object (num_ctx, num_predict, temperature,
seed) plus keep_alive. The OpenAI path is still used for LM Studio and llama.cpp,
minus the ineffective num_ctx and with include_usage so both paths return
prompt token counts. /api/show is now read for trained context, parameter
size, quantization and any modelfile num_ctx. The <think> stream normalizer
moves to app/utils/think_stream.ts so both transports share it; it gains a
flush() that fixes a dangling partial tag being dropped at end of stream.
Token accounting: replace chars/N with a structural segmenter scaled by a
per-model factor learned by EWMA from the prompt_eval_count every response
already returned and threw away. End-to-end estimator error is 4.6% against
18-20% for a fixed divisor — qwen2.5 converges to k=1.26 where llama3 sits
at 1.00, so no single global constant serves both.
Context window: resolved once per model from trained context, the exact KV
cost derived from GGUF metadata (head_count_kv, not head_count — GQA makes
them differ 4x), and available VRAM, then memoized. Memoization is
load-bearing: asking for a different num_ctx makes Ollama unload and reload
the model, stalling the turn and discarding the KV cache. The floor never
overrides the model's trained ceiling — tinyllama trains to 2048, and a
blind 4096 floor pushes RoPE past anything the weights saw.
Prompt budget: planPrompt allocates explicitly instead of delegating
overflow to silent backend truncation — response reserve, then fixed blocks
(system and current query, never dropped), then RAG chunks whole and
best-first, then history in whole user+assistant turns, evicted newest-first
in blocks with an elision marker. Block eviction keeps the prefix stable for
several turns rather than shifting on every one.
KV-cache ordering: the per-turn retrieved block moves behind history to the
tail, so the stable prefix stays byte-identical turn to turn. At turn 14 of
a long conversation, prefill is 231ms at the tail against 539ms at the front
(~1ms/turn growth against ~25ms). The win is conditional — with a large
retrieved block and short history the two measure the same — so both
orderings ship behind RAG_PLACEMENT. The per-turn query rewrite also moves
to the tasks model; on a single-slot Ollama it was evicting the chat model's
cached prefix on every single turn.
Also: OLLAMA_CONTEXT_LENGTH and OLLAMA_KV_CACHE_TYPE=q8_0 on the container
(q8_0 is treated as headroom, never as a budgeting assumption, since
unsupported architectures fall back to f16 silently), a Context Window
setting in Settings > Models, and budget fields on PipelineTrace.
Eval: golden suites (--suite) so the new long_context goldens cannot move
the aggregates the committed baselines were recorded against, plus budget
and estimator-error metrics through the report path. Retrieval is unchanged
and eval:compare reports no metric outside the tolerance band; the
long_context suite scores correctness 1.000 with historyElided 0.500 (the
two eviction cases, by design). Unit tests 337 -> 386 passing, same 7
pre-existing failures.
Ingest-side CHAR_TO_TOKEN_RATIO is deliberately untouched: changing it
re-chunks the corpus and invalidates retrieval baselines, which belongs in
its own measured change.
Allows users to optionally specify a model to use for ancillary tasks (chat suggestions, chat titles, etc.) instead of strictly using the last used model. This means aesthetic/non-critical work can be passed to a small, lightweight model instead of a heavy reasoning model used for chats.
If a model is not selected in the AI Assistant settings (`/settings/models`), the existing behavior of using the last used model is retained.
Shows a dismissible dashboard banner to users who submitted a benchmark under
v1 scoring but don't yet have a Score v2 result, nudging them to re-run so the
community leaderboard gets their v2 number.
- Show logic (benchmark_service.shouldShowRerunBanner): has a result with
submitted_to_repository=true AND no result with nomad_score_v2 AND the
dismiss KV isn't set. Self-clears two ways — dismiss sets the KV, and any v2
run gives a result a nomad_score_v2 so the condition flips off on its own.
- New GET /api/benchmark/rerun-banner + useBenchmarkRerunBanner hook (mirrors
useUpdateAvailable), rendered as a dismissible Alert on the dashboard with a
"Re-run benchmark" CTA to /settings/benchmark.
- New KV key benchmark.rerunBannerDismissed (boolean) in both KV_STORE_SCHEMA
and the SETTINGS_KEYS whitelist (the PATCH /system/settings validator enum);
dismiss writes it via the existing updateSetting endpoint + invalidates the
query.
Sibling to the Score v2 app client (#1094); stacked on it for the
nomad_score_v2 column. Browser-verified on the NOMAD3 dev env: show/dismiss/
reload-persist/self-clear-on-v2 all correct.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(AI): per-model thinking toggle with global default (off)
Stop forcing thinking on for every capable model. Adds a global default
(ai.autoThinking, ships OFF) and a per-model override in the chat window,
shown only for thinking-capable models and remembered client-side.
The /v1 (OpenAI-compat) endpoint ignores `think`; reasoning_effort is the
real lever. The controller resolves per-request preference -> global default
-> OFF (gated on capability), and the service maps that to reasoning_effort
('none' to suppress on a capable model, 'medium' for gpt-oss, unset to let a
capable model default thinking on). A new thinkingCapable flag keeps
non-Ollama backends from ever receiving reasoning_effort.
installed-models is enriched with a `thinking` flag (checkModelHasThinking,
now memoized) so the picker knows which models get the toggle.
Stacks on #1078 (reasoning-field read + client-disconnect abort).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(AI): thinking toggle as NOMAD Switch + info tooltip
Address review feedback on the per-model thinking control:
- Use the shared Switch component (matches AI Assistant settings) instead
of a raw checkbox.
- Label "Thinking:" with a colon to match the adjacent "Model:" label.
- Add an InfoTooltip explaining what thinking does and where the default
lives. Extend InfoTooltip with optional `position` ('top'|'bottom') and
`align` ('center'|'right') so it opens downward and expands leftward from
the right-edge header slot instead of being clipped/crushed against the
viewport edge. Defaults preserve existing (benchmark page) behavior.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(rag): add subject/collection organization to knowledge base
- Add nullable collection field to KbIngestState, propagated through the
embed job, RAG service, and Qdrant point payloads (indexed for filtering)
- Add upload-time category selection and per-file collection reassignment
in the Knowledge Base modal, with a filterable Stored Files table
- Add a 'Search in' collection filter to the chat interface, threaded
through to searchSimilarDocuments as an optional Qdrant filter
- Fix .docx extraction: previously routed through raw-text extraction
(garbage output for a ZIP-based XML format); adds a proper mammoth-based
extractor and a dedicated 'docx' file-type case
* feat(rag): support dynamic KB collection creation, rename, and removal
Extends collection organization with a Manage Collections UI: collections
are created on the fly when a file is assigned to a new name, can be
renamed (bulk-updates every tagged file and Qdrant point), and can be
removed (reassigns tagged files back to Uncategorized rather than
deleting anything).
* fix(rag): use dynamic collections query in chat search filter
chat/index.tsx still imported the static KB_COLLECTIONS constant for its
'Search in' dropdown, inconsistent with KnowledgeBaseModal.tsx which already
uses the live getKnowledgeCollections() query. Renamed/added collections
via the new Manage Collections UI weren't reflected in the chat filter.
* feat(rag): broaden preset tags and add creatable collection combobox
Replaces the survival-specific preset list with general-purpose starter
tags (recipes, diy, health, technology, finance, travel, hobbies,
reference, survival, energy) so the Knowledge Base reads well for
home-lab/reference use, not just prepping.
Adds sanitizeCollectionName() (trim, lowercase, length cap) applied on
every write path server-side, and a dependency-free CollectionCombobox
component replacing the plain <select> + window.prompt pattern for
tagging — autocompletes against presets + tags already in use, with a
'+ Create' option for anything new.
* chore(rag): remove .docx fix from this branch, split into #1100
Per review feedback, the .docx extraction fix is unrelated to the
collections feature and can merge independently. Moved to a standalone
PR (Crosstalk-Solutions/project-nomad#1100) off dev.
* chore: remove unrelated diff noise (lockfile, comments, indentation)
---------
Co-authored-by: John Cortright <jcortright@zscaler.com>
Phase 1 of the live benchmark-run experience. Replaces the opaque (and
in sync mode, simulated) progress bar with a real-time run view driven by
actual host telemetry.
- Async run path: the UI now dispatches to the queue worker and keys off
SSE instead of faking stage progress with client-side timers.
- New BenchmarkTelemetrySampler broadcasts per-core CPU load, CPU temp
(best-effort, hidden when unavailable), and disk MB/s at 1 Hz over a new
benchmark-telemetry SSE channel. Runs in the orchestration process, never
the sysbench container, so it cannot affect scores.
- BenchmarkProgress carries the ordered stage plan + index so the frontend
renders a live stage rail.
- AI benchmark streams /api/generate for live tokens/sec and true TTFT; the
scored numbers still come from Ollama's authoritative final eval fields.
- Frontend: useBenchmarkRun hook owns both subscriptions; self-contained SVG
components (StageRail, CoreGrid, Sparkline, LiveReadout) + BenchmarkRunView,
styled in the desert palette. No chart library added.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The uninstall and update scripts called header_red() and referenced color
variables (GREEN/RED/RESET) that were never defined, so running them printed
"header_red: command not found" instead of the section banners. Also corrects
the update script's success message (it said "installation completed" and
pointed at an undefined ${nomad_dir} path) and fixes two spelling typos.
- uninstall_nomad.sh: add Color Codes block + header_red()
- update_nomad.sh: add header_red(); fix success wording and start_nomad.sh path
- CONTRIBUTING.md, admin/constants/ollama.ts: spelling fixes
Reported by @fix2015 in #1058; reimplemented in-house.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: replace legacy Kolibri image default with latest v19 image
* feat(supply-depot): add content migration instructions for Edu Platform Gen 1 to 2
Adds the MeshCore web client to the Supply Depot catalog (host port 8500),
alongside the existing Meshtastic apps. Uses aXistem's prebuilt image of Liam
Cottle's MeshCore client (MeshCore is a sibling LoRa mesh project to Meshtastic).
The image is stock nginx serving a static Flutter build over HTTP, but the
client reaches radios via Web Bluetooth / Web Serial, which browsers only allow
from a secure (HTTPS) context. So we serve it over HTTPS: a new preinstall hook
generates a self-signed cert + a small SSL nginx config into storage/meshcore-web,
both bind-mounted into the container (the config over the image's default.conf),
publishing 443. Same one-time browser-warning approach as Vaultwarden, whose
openssl cert generation is refactored into a shared _ensureSelfSignedCert helper.
Also adds a NOMAD-specific docs section + Manage>Docs anchor, and registers the
IconAntenna icon. Meshtastic Web left unchanged.
Validated on NOMAD3 (v1.33.0-rc.1): the image + SSL config + self-signed cert
serves the MeshCore Flutter app over HTTPS 200 with working SPA fallback.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a self-update path for the NOMAD admin/core image that runs without a
human in the loop, gated by layered safety checks. Recreation remains the
sidecar's job; this only decides *whether* to update now and requests it.
An hourly job (AutoUpdateJob) evaluates a side-effect-free decision pipeline
(AutoUpdateService.evaluate):
- opt-in: disabled by default
- in a user-configured local-time window (handles midnight wrap)
- an eligible release: same major (major bumps stay manual), strictly newer,
past a configurable cool-off, GA only (no drafts/prereleases), strict semver
- pre-flight: sidecar present, no in-flight system update / downloads / app
installs, and sufficient host disk (estimated from the registry manifest)
When all pass, it drives SystemUpdateService.requestUpdate() with a vetted tag.
Failure backoff auto-disables after 3 genuine update-request failures; transient
release-lookup errors are skips (offline-first appliances are routinely without
connectivity). Re-enabling clears the backoff state.
Settings UI exposes the toggle, window, and cool-off, with live status (eligible
target, in/out of window, last result/error). A `node ace auto-update:dry-run`
command exercises the full pipeline — and a deterministic --scenarios suite the
pure decision logic — without ever triggering an update.
New KVStore keys under autoUpdate.* hold config + last-run state.
Add NOMAD-specific getting-started docs for all 9 curated Supply Depot apps,
the catalog/install fixes each one surfaced, and a way to reach the docs from
each app card.
Docs:
- New in-app Markdoc page admin/docs/supply-depot-apps.md covering all 9 apps
(Stirling PDF, File Browser, Calibre-Web, IT Tools, Excalidraw, Homebox,
Vaultwarden, Jellyfin, Meshtastic Web): first run/login, where data lives,
and offline behaviour. Registered in docs_service DOC_ORDER.
- Manage > Docs dropdown item linking each app to its section
(/docs/supply-depot-apps#<anchor>): anchor map in constants/supply_depot_docs.ts,
heading anchors via Markdoc {% #id %}, and hash-scroll on the docs page.
Install / catalog fixes:
- Stirling PDF: open straight to the tools (SECURITY_ENABLELOGIN=false; the old
v1 DOCKER_ENABLE_SECURITY flag was dead).
- File Browser: seed a known admin/nomad login (bcrypt) instead of a random
log-only password; scope visibility to content folders via mount selection and
move the DB out of the browsable root.
- Calibre-Web: bundle an empty Calibre library and seed it on install so setup
doesn't dead-end at db config (_runPreinstallActions__CalibreWeb).
- Homebox: swap the archived hay-kot image for the maintained sysadminsmedia fork.
- Vaultwarden: generate a self-signed cert on install and serve HTTPS by default
(_runPreinstallActions__Vaultwarden + ROCKET_TLS + ui_location https:8480), so
the web vault has the secure context it requires.
- Jellyfin: pre-create storage/media/{Movies,TV Shows,Music,Photos} so each
library points at its own subfolder, avoiding the overlapping-path issue that
silently hides content (_runPreinstallActions__Jellyfin).
- Seeder run() now also syncs ui_location for non-modified curated services, so a
catalog link/scheme/port change reaches existing installs on update.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Users often saw the assistant disclaim ("Sorry, I couldn't find specific
context regarding X, but here's a general answer...") even when material
that directly answered the query was embedded. Two compounding causes,
both on the read/generation side:
1. The rag_context system prompt explicitly authorized the hedge: it told
the model to fall back to general knowledge and "acknowledge the
limitations," and mandated "According to the information available..."
citations that pushed small models into meta-commentary preambles.
Rewrote it to treat the retrieved context as the primary, authoritative
source, lead with the answer, fall back to general knowledge *silently*,
and never emit "couldn't find specific context" phrasing, while still allowing
the model itself to make some "sanity-check" decisions if the retrieved context
seems wildly incorrect or unrelated to the user's query.
2. The injected context labeled each block with a raw relevance score
(e.g. "Relevance: 42.3%"). nomic-embed cosine scores for genuinely
relevant passages sit ~0.4-0.6, so the number primed the model to
distrust correct context. Dropped the model-visible score (it stays in
the logs) and replaced it with a neutral source-title label.
Also added a conservative, score-scaled heading boost in rerankResults:
when query keywords match a chunk's section/article title (ZIM metadata we
already fetch), nudge its rank. Same diminishing-returns shape as the
existing boosts and gated behind the existing semantic-quality threshold,
so it can't promote a weak match.
Scope note: this removes the visible hedge and improves ranking of
already-retrieved chunks. It does NOT fix retrieval recall — diluted
1500-token chunks that never reach dense search's top-k are unaffected.
That's a follow-up (smaller chunks + BM25/RRF hybrid).
Lift the hardcoded 'nomic-embed-text:v1.5' string out of both
RagService and OllamaService into a shared EMBEDDING_MODEL_NAME
constant in constants/ollama.ts. The duplicate in OllamaService
existed only to dodge a circular import with RagService; the
constants module has no service imports, so a shared constant
eliminates both the duplication and the drift risk called out
in the inline "keep in sync" comment.
* feat(KB): per-file ingest state machine (Phase 1 of RFC #883)
Adds a persistent state machine for AI knowledge-base ingestion so the
scanner can distinguish "fully indexed", "user opted out", "failed", and
"stalled" from each other — none of which were derivable from the prior
binary "any chunks in Qdrant ⇒ embedded" check.
## What lands
- New table `kb_ingest_state` keyed by `file_path` with enum state column
(`pending_decision | indexed | browse_only | failed | stalled`).
Independent of `installed_resources` so it covers both curated downloads
and manually-uploaded KB files.
- New KV key `rag.defaultIngestPolicy` (string: `Always | Manual`).
Registered now but not consumed yet — JIT prompt + wizard step land in
Phase 3 of the RFC.
- `EmbedFileJob.handle` writes state on terminal outcomes:
- Success (final batch) → `indexed` + chunks count
- `UnrecoverableError` → `failed` + error message
- Retryable errors are left to BullMQ's existing retry path
- `scanAndSyncStorage` swaps the binary qdrant check for a state-aware
decision tree (see `decideScanAction`). Existing installs auto-backfill
on first scan: files with chunks in Qdrant but no state row become
`indexed`; new files start as `pending_decision`.
- `deleteFileBySource` drops the state row last, so removed files
disappear entirely instead of leaving an orphan that the next scan
would re-dispatch into nothing.
## What does NOT land here
- Ratio registry (separate PR) — needed for partial-stall detection and
cost estimates, but a separable concern.
- #880 follow-up initial-progress anchor (separate tiny PR).
- Phase 2 UI (status pill, per-card actions, conditional warnings).
- Phase 3 policy surfaces (wizard step, JIT prompt, guardrail modal).
- PR #886's bulk-action hookup — `_deletePointsBySource` / Re-embed All
/ Reset & Rebuild would also want to set state, but #886 isn't merged
yet; that wiring goes in a follow-up once #886 lands.
## Target
This is forward work for v1.40.0 (RFC #883). Branching off `rc` because
that's the current latest base and post-GA Jake will sync rc→dev; a
retarget at PR-open time is a fast-forward if requested.
## Tests
- 9 new unit tests for `decideScanAction` covering all five states plus
the no-row / chunks-present / chunks-missing combinations
- Type-check clean
- Smoke-tested end-to-end on NOMAD3 via hot-patch:
- Backfill: 5 ZIMs + 2 KB uploads with existing chunks in Qdrant all
came back `indexed` on first scan
- Pending dispatch: a video-only ZIM with no chunks (`lrnselfreliance`)
came back `pending_decision` and was correctly re-dispatched (Bull
deduped to its historical `:completed` jobId — bgauger's #886 fix
drains that)
- Delete hook: deleting a KB upload via `DELETE /api/rag/files`
removed both the disk file and the state row
* feat(KB): Always/Manual ingest policy toggle (RFC #883 §1/§4)
Activates the `rag.defaultIngestPolicy` KV registered in Phase 1
(#888) so users on a fresh install (or anyone who picks Manual mode)
no longer get every new ZIM auto-dispatched to the embed pipeline.
## Stacks on #888
This PR's base is `feat/kb-ingest-state-machine` (#888). The state
machine has to be in place for the decision function to be policy-aware;
GitHub will fast-forward the base to `rc` once #888 merges.
## Backend changes
- `decideScanAction` now takes a `policy: 'Always' | 'Manual'` argument
(defaults to `Always` for backward compatibility).
- New `ScanAction` kind: `create_pending`. Manual mode records that the
scanner has seen a new file (so the UI can surface a per-card Index
affordance later) without dispatching an EmbedFileJob.
- `scanAndSyncStorage` reads the KV and passes it through. The scan-result
log line now includes the active policy and a `waiting on user` count
for Manual-mode hits.
- `rag.defaultIngestPolicy` added to `SETTINGS_KEYS` so it's reachable
through the existing `GET/PATCH /api/system/settings` surface — no new
endpoint.
## Frontend changes
- New section in the KB panel between "Why upload" and "Processing Queue":
"Auto-index new content for AI? [Always | Manual]" — segmented radio
with copy explaining the 5-10× disk multiplier. Default Always.
- `useQuery('ingestPolicy')` reads the current value; clicking the
inactive option mutates and shows a notification confirming the new
behavior.
## Tests
- 14 unit tests on `decideScanAction` (was 9) — split into Always-mode
cases (preserves Phase 1's contract) and Manual-mode cases
(`create_pending`, `pending_decision → skip`, etc.).
- Type-check clean.
- Hot-patch + browser verification deferred until #888 lands; the state
machine smoke-tested cleanly on NOMAD3 in #888's PR, and this PR's
decision-tree changes are exhaustively unit-tested.
## RFC open question §3 — policy-change re-trigger
Switching Manual → Always doesn't auto-dispatch existing `pending_decision`
rows immediately. The next scan re-evaluates and dispatches them under
the new policy. This matches the RFC's "treat the switch as I've-
thought-about-it" instinct for the guardrail; full guardrail
implementation lands in Phase 3 task 14.
---------
Co-authored-by: Jake Turner <52841588+jakeaturner@users.noreply.github.com>
* feat(maps): add regional map downloads via go-pmtiles extract
* address Copilot review feedback on PR #780
- auto-refresh preflight on selection/maxzoom change with 400ms debounce and
requestId stale-safety so the confirm button no longer requires a two-step
"Estimate Size" -> "Start Download" dance
- safeUpdateProgress helper replaces fire-and-forget updateProgress().catch()
pattern so cancelled-job errors (code -1) can't surface as unhandled rejections
- gate world basemap source on worldBasemapReady - when ensureWorldBasemap()
fails we already delete world.pmtiles, so emitting the source was producing
404s on every tile request
- verify go-pmtiles binary SHA256 at image build time; upstream doesn't ship a
checksums file so per-arch hashes are pinned as build args with a regenerate
note when bumping PMTILES_VERSION
New defaults:
OLLAMA_NO_CLOUD=1 - "Ollama can run in local only mode by disabling
Ollama’s cloud features. By turning off Ollama’s cloud features, you
will lose the ability to use Ollama’s cloud models and web search."
https://ollama.com/blog/web-searchhttps://docs.ollama.com/faq#how-do-i-disable-ollama%E2%80%99s-cloud-features
example output:
```
ollama run minimax-m2.7:cloud
Error: ollama cloud is disabled: remote model details are unavailable
```
This setting can be safely disabled as you have to click on a link to
login to ollama cloud and theres no real way to do that in nomad outside
of looking at the nomad_ollama logs.
This one can be disabled in settings in case theres a model out there
that doesn't play nice. but that doesnt seem necessary so far.
OLLAMA_FLASH_ATTENTION=1 - "Flash Attention is a feature of most modern
models that can significantly reduce memory usage as the context size
grows. "
Tested with llama3.2:
```
docker logs nomad_ollama --tail 1000 2>&1 |grep --color -i flash_attn
llama_context: flash_attn = enabled
```
And with second_constantine/deepseek-coder-v2 with is based on
https://huggingface.co/lmstudio-community/DeepSeek-Coder-V2-Lite-Instruct-GGUF
which is a model that specifically calls out that you should disable
flash attention, but during testing it seems ollama can do this for you
automatically:
```
docker logs nomad_ollama --tail 1000 2>&1 |grep --color -i flash_attn
llama_context: flash_attn = disabled
```
Add a warm charcoal dark mode ("Night Ops") using CSS variable swapping
under [data-theme="dark"]. All 23 desert palette variables are overridden
with dark-mode counterparts, and ~313 generic Tailwind classes (bg-white,
text-gray-*, border-gray-*) are replaced with semantic tokens.
Infrastructure:
- CSS variable overrides in app.css for both themes
- ThemeProvider + useTheme hook (localStorage + KV store sync)
- ThemeToggle component (moon/sun icons, "Night Ops"/"Day Ops" labels)
- FOUC prevention script in inertia_layout.edge
- Toggle placed in StyledSidebar and Footer for access on every page
Color replacements across 50 files:
- bg-white → bg-surface-primary
- bg-gray-50/100 → bg-surface-secondary
- text-gray-900/800 → text-text-primary
- text-gray-600/500 → text-text-secondary/text-text-muted
- border-gray-200/300 → border-border-subtle/border-border-default
- text-desert-white → text-white (fixes invisible text on colored bg)
- Button hover/active states use dedicated btn-green-hover/active vars
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
