Commit Graph

49 Commits

Author SHA1 Message Date
Chris Sherwood
ee203df485 fix(translate): download every released language, and list them in the docs
fetch_models.py sorted versions with float(), and Firefox Remote Settings
now publishes Nightly pre-releases ("1.0a1") alongside each release. The
first one crashed the whole fetch, so adding Swedish, or 21 of the 34 named
languages, silently did nothing and the button never appeared.

Only take records desktop Firefox would use on the release channel, pick
the newest complete version per direction instead of mixing files across
versions, support pairs that publish a separate source and target vocab
(Japanese), and keep one failing language from stopping the rest.

That opens 50 languages, up from the ~40 the docs claimed, including
Arabic, Japanese, Korean and Thai, which the docs said were unavailable.
All 50 downloaded, loaded and translated on the bergamot 0.4.5 engine in
the shipped image (NOMAD3). Chinese stays out: it is published as
zh-Hans/zh-Hant, which the proxy's two-letter codes cannot carry yet.

Translated pages now send Cache-Control: no-cache and put the installed
language set in the ETag. Kiwix's max-age=3600 kept the old language bar
for up to an hour after a language was added, and a normal refresh does
not reach the article frame.

Docs: step-by-step instructions and the full code table in the Supply
Depot section; sizes corrected to 45-140 MB per language.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 14:19:27 -07:00
Chris Sherwood
eef5b6cd37 fix(translate): forward upstream redirects instead of following them
urlopen() follows redirects silently. Kiwix answers a book root such as
/content/<book> with a 302 to the ZIM's main page, so the proxy served
that page's HTML at the original, shallower URL. Every relative link and
stylesheet then resolved one directory too high: the page lost its
styling and every article link 404'd (e.g. atenolol/ -> /content/atenolol/).
Kiwix's /random is a 302 too and broke the same way.

Use an opener that returns 3xx to the browser like any other non-200
response, and keep an absolute Location on this origin rather than the
upstream container name.

Bumps the catalog image to 0.1.1.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 14:19:27 -07:00
Chris Sherwood
18e403bbaa fix(ai): bump the Ollama pin to 0.33.3 to fix the disconnect hang
ollama/ollama:0.24.0 never cancels in-flight generation when a client
disconnects: the runner process is left spinning at 300%+ CPU with no
active requests and never recovers. NOMAD triggers this organically via
its own 60s RAG embedding timeouts, so slow CPU-only boxes can wedge
themselves after a single slow request. Reported with a standalone
container repro in #1321; the cancellation fix landed upstream between
0.24 and 0.33.

Regression-tested on 0.33.3 (RTX 5060 box):
- model listing via /api/tags (NOMAD backend detection intact)
- GPU acceleration (100% GPU placement, 82.8 tok/s on llama3.1:8b)
- model pull through the NOMAD download pipeline (NDJSON progress)
- embeddings via /api/embed with nomic-embed-text (768-dim)
- streaming chat end-to-end through the Command Center UI
- the disconnect hang itself: container idles at 0.06% CPU 20s after
  a mid-generation client drop

The AMD path is unaffected: it always overrides to ollama/ollama:rocm.
The seeder does not sync container_image on existing rows, so this pin
reaches fresh installs; existing installs pick it up via the opt-in app
auto-update or a manual update.

Fixes #1321

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-29 15:54:41 -07:00
chriscrosstalk
1b67a87254 feat(translate): offline translation for the Information Library (#1292)
* feat(translate): offline translation for the Information Library

Implements #1272. Ships as a Supply Depot app so users who do not need
translation never see it.

A path-preserving reverse proxy in front of kiwix-serve. /viewer, /skin/*,
/search, /random and /catalog/* are forwarded byte for byte; only text/html
under /content/ is rewritten. Preserving the path is the whole design:
viewer.js keeps the content iframe in sync with location.hash by comparing
contentWindow.location.pathname, so moving the path starts the ping-pong its
own comment warns about and forces us to patch Kiwix's JavaScript on every
Kiwix bump. Because the path never moves, no Kiwix JS is modified, and Kiwix's
header, search, library and random keep working because they are Kiwix.
Language is a cookie, so links inside the ZIM need no rewriting either.

Bergamot rather than the AI Assistant: measured on the same box and the same
passage, 12,818 words/s on CPU against llama3.1:8b at 8 words/s on an RTX 5060.
It is also more careful with names, where the LLM translated the product name
and a smaller model hallucinated a clause. No GPU, so it works on CPU-only
boxes where the LLM path is unusable.

Models come from Firefox Remote Settings, the endpoint Firefox itself uses,
not the mozilla/firefox-translations-models repo, which was archived on
2026-08-21. MPL-2.0, about 37 MB per direction.

Failure handling is deliberate throughout: a failed model fetch is non-fatal, a
translation error returns the original article with a comment rather than an
error page, and a box with no models still starts and still forwards the
library. Translation being unavailable must never make the Information Library
unreachable.

amd64 only, and pinned as such in the build workflow. Bergamot's intgemm
backend is x86 specific and there is no aarch64 wheel, so a multi-arch build
would produce an ARM image that pulls cleanly and dies at start.

Also adds .gitattributes pinning *.sh to LF. The repo stores LF but Windows
checks out CRLF, and a CRLF after a shebang makes the interpreter
unresolvable inside the image. CI builds on Linux so this never bit, but it
breaks a local image build and would have bitten the existing sidecar script
too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV

* fix(translate): vary the cache on the language cookie

Found by clicking the language buttons in a browser rather than trusting curl.

Kiwix sends ETag and Cache-Control: max-age=3600, and the proxy forwarded both
unchanged on a response whose content now varies by cookie. The browser cached
the English article for an hour and served it straight back when the reader
picked a language, so the feature looked like it simply did not work. A shared
cache would have been worse, serving one reader's language to another.

Translatable responses now send Vary: Cookie, and their ETag is namespaced by
language so revalidation still works per language rather than being turned off.
Forwarded paths are untouched, so /viewer, /skin/* and /catalog/* stay byte for
byte identical to Kiwix.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV

* fix(translate): move off port 8480, which Vaultwarden already uses

Caught by seeing the two cards side by side in Supply Depot with the same port.

My port survey grepped ui_location for a bare number, and Vaultwarden's is
'https:8480' because it needs an https Open link, so it did not match and 8480
looked free. Moved to 8460, which fits the existing spacing between Meshtastic
Web on 8450 and Homebox on 8470.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV

* fix(translate): pin the base image to python 3.10

The build fails outright on 3.12: bergamot publishes no wheel for it and pip
resolves no further than 0.1.2. The spike used 3.10 for this reason and I had
modernised it without checking. Caught by building the image rather than
reading it.

Moving off this pin means moving off the 2022 PyPI wheel to a build from
mozilla/translations, which is the follow-up already noted in the Dockerfile.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV

* fix(translate): translate text that lives in a bare div

Sotoki-generated StackExchange ZIMs put every question excerpt in a
<div class="...excerpt">, so on a questions list the titles translated and the
descriptions underneath them did not. Reported against the ham radio ZIM.

div could not simply be added to BLOCK_TAGS: the outermost-block rule would
claim a top-level layout wrapper and send the whole page as one job. It is now
claimed speculatively and abandoned the moment anything block-level opens
inside it, so a wrapper yields to the blocks it contains and only a genuine
text leaf is translated.

Also moves the test harness's reporting block back to the end of the file. New
checks had been appended after it, so they executed but their failures were
never reported: verified that removing div from the set now fails 3 checks,
where before it reported success.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV

* fix(translate): decode numeric character references before translating

Bergamot's HTML mode decodes named entities (`&amp;` arrives as "&") but not
numeric ones. It treats `&#x27;` as literal text and escapes the ampersand on
the way out, so "If I&#x27;m in the wilderness" came back as
"Si I&amp;#x27;m dans la nature" and rendered the escape sequence visibly to
the user. Confirmed by round-tripping fragments through the model directly.

Numeric refs are now decoded before the round trip. `&`, `<` and `>` are
deliberately left encoded: decoding those would inject real markup into the
fragment and fail the balance check that gates HTML mode, dropping the block to
plain text and losing its inline links.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-29 15:54:41 -07:00
jakeaturner
e468cf363a fix(seeder): derive ui_location from the merged config, not the live row 2026-09-29 15:54:41 -07:00
Chris Sherwood
32eea2ec22 fix(seeder): keep an installed app's host port across catalog sync
The catalog-sync loop overwrote container_config and ui_location for every
non-user-modified curated service, resetting the published host port of an
install that had been moved off the catalog default. The row then disagreed
with the running container: the Open link pointed at a dead port and the
next recreate collided.

Preserve the live host ports for installed rows and apply every other
catalog change, so these installs keep receiving catalog updates instead of
being opted out of them.

Diagnosis and the original fix are @johno10661's work in #1005.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
2026-09-29 15:54:41 -07:00
Chris Sherwood
400d64ab55 feat(dashboard): let a link tile carry a brand colour
A link tile rendered on a flat surface colour, which read as a stark white card
against the filled green app tiles. Tiles now take one of six colours from the
brand palette, chosen when creating or editing one, and the card is tinted from
it.

Tiles stay visually distinct from managed apps regardless of colour: the dashed
border and the external-link marker carry that, not the hue, and the tint is
deliberately light so the distinction survives every option.

Class names are written out in full rather than interpolated, because Tailwind
scans source text for literal class strings and would silently drop a computed
one, leaving the tile untinted. A test asserts that.

Added as a separate migration from the one introducing is_link_tile, since that
has already run on the test box and editing it in place would leave fresh and
upgraded installs with different schemas.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
2026-09-29 15:54:41 -07:00
Chris Sherwood
b7b973e9b8 feat(dashboard): user-added link tiles for services NOMAD does not manage
Users want things they already run to appear on the dashboard: a NAS, a router
UI, a printer, a Pi, a container they started themselves. Adds a tile they
address themselves, with a name, one URL field, an optional description and an
icon from the curated set.

Chosen over adopting containers from the Docker daemon, which is what community
PR #1239 proposed. Adoption only reaches containers on this Docker host and has
to guess: it took the lowest published host port and assumed http, which picks
the wrong port for anything publishing several (Portainer 8000 against 9443,
Jellyfin 8096 against 8920) and breaks entirely for HTTPS. A tile the user
addresses has no guessing in it, covers the whole LAN rather than one Docker
host, and has no lifecycle that can break.

One URL field rather than separate host, port and path, because people paste
URLs and it gets https and sub-paths for free. A bare "192.168.1.50:8080" gains
http automatically, which is what most LAN devices need. normalizeCustomUrl and
getServiceLink already did exactly this work for the custom app URL override, so
both are reused rather than duplicated.

Stored as a service row with no container behind it: is_link_tile marks it,
custom_url carries the destination, and the service_name is namespaced
nomad_link_* so it can never collide with a curated entry. The seeder only
iterates its own DEFAULT_SERVICES and never prunes unknown rows, so a reseed
cannot touch or remove a tile.

The tile is deliberately styled differently from a managed app, outlined and
dashed with an external-link marker rather than filled, because if it looked the
same users would expect Start, Stop and Update and file bugs when those controls
are not there. Removing a tile arms a confirm first, since it is user-entered
data with no undo.

Icons come from a fixed 36 shared by the picker and the server-side validator,
so the two cannot disagree about what is selectable. DynamicIcon renders nothing
at all for a name it does not know, which would leave a silent hole in a tile.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV
2026-09-29 15:54:41 -07:00
Jake Turner
0143afe7cc
fix(supply-depot): bump default Ollama and CyberChef image versions (#1036) 2026-06-23 04:47:12 +00:00
Jake Turner
5af27e9904
fix(supply-depot): ensure all curated images pinned to specific versions (#1033) 2026-06-23 04:47:12 +00:00
Jake Turner
8982d93a31
feat: replace legacy Kolibri image default with latest v19 image (#1019)
* feat: replace legacy Kolibri image default with latest v19 image
* feat(supply-depot): add content migration instructions for Edu Platform Gen 1 to 2
2026-06-23 04:47:07 +00:00
jakeaturner
1b040c8b9a
fix: pin default meshcore web image 2026-06-23 04:47:07 +00:00
Chris Sherwood
bd65c885be
feat(supply-depot): add MeshCore Web with self-signed HTTPS
Adds the MeshCore web client to the Supply Depot catalog (host port 8500),
alongside the existing Meshtastic apps. Uses aXistem's prebuilt image of Liam
Cottle's MeshCore client (MeshCore is a sibling LoRa mesh project to Meshtastic).

The image is stock nginx serving a static Flutter build over HTTP, but the
client reaches radios via Web Bluetooth / Web Serial, which browsers only allow
from a secure (HTTPS) context. So we serve it over HTTPS: a new preinstall hook
generates a self-signed cert + a small SSL nginx config into storage/meshcore-web,
both bind-mounted into the container (the config over the image's default.conf),
publishing 443. Same one-time browser-warning approach as Vaultwarden, whose
openssl cert generation is refactored into a shared _ensureSelfSignedCert helper.

Also adds a NOMAD-specific docs section + Manage>Docs anchor, and registers the
IconAntenna icon. Meshtastic Web left unchanged.

Validated on NOMAD3 (v1.33.0-rc.1): the image + SSL config + self-signed cert
serves the MeshCore Flutter app over HTTPS 200 with working SPA fallback.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 04:47:06 +00:00
jakeaturner
026ab6df8f
feat(supply-depot): add custom launch URLs for apps
Let users override an app's "Open" link with a reverse-proxy or
local-DNS address (e.g. https://jellyfin.myhomelab.net). Falls back to
the default host+port when unset. Metadata-only — no container changes.
2026-06-23 04:46:56 +00:00
jakeaturner
cf8db6218d
feat(supply-depot): opt-in automatic updates for installed apps 2026-06-23 04:46:52 +00:00
Chris Sherwood
f488f08c96
feat(supply-depot): per-app onboarding docs, install fixes, and in-app Docs links
Add NOMAD-specific getting-started docs for all 9 curated Supply Depot apps,
the catalog/install fixes each one surfaced, and a way to reach the docs from
each app card.

Docs:
- New in-app Markdoc page admin/docs/supply-depot-apps.md covering all 9 apps
  (Stirling PDF, File Browser, Calibre-Web, IT Tools, Excalidraw, Homebox,
  Vaultwarden, Jellyfin, Meshtastic Web): first run/login, where data lives,
  and offline behaviour. Registered in docs_service DOC_ORDER.
- Manage > Docs dropdown item linking each app to its section
  (/docs/supply-depot-apps#<anchor>): anchor map in constants/supply_depot_docs.ts,
  heading anchors via Markdoc {% #id %}, and hash-scroll on the docs page.

Install / catalog fixes:
- Stirling PDF: open straight to the tools (SECURITY_ENABLELOGIN=false; the old
  v1 DOCKER_ENABLE_SECURITY flag was dead).
- File Browser: seed a known admin/nomad login (bcrypt) instead of a random
  log-only password; scope visibility to content folders via mount selection and
  move the DB out of the browsable root.
- Calibre-Web: bundle an empty Calibre library and seed it on install so setup
  doesn't dead-end at db config (_runPreinstallActions__CalibreWeb).
- Homebox: swap the archived hay-kot image for the maintained sysadminsmedia fork.
- Vaultwarden: generate a self-signed cert on install and serve HTTPS by default
  (_runPreinstallActions__Vaultwarden + ROCKET_TLS + ui_location https:8480), so
  the web vault has the secure context it requires.
- Jellyfin: pre-create storage/media/{Movies,TV Shows,Music,Photos} so each
  library points at its own subfolder, avoiding the overlapping-path issue that
  silently hides content (_runPreinstallActions__Jellyfin).
- Seeder run() now also syncs ui_location for non-modified curated services, so a
  catalog link/scheme/port change reaches existing installs on update.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 04:46:47 +00:00
Chris Sherwood
7b1b480c5e
fix(supply-depot): correct Meshtastic Web internal port (80->8080); add catalog port audit script 2026-06-23 04:46:46 +00:00
Chris Sherwood
02c33b277e
feat: edit curated apps + fix dropdown clip + stale _old rollback 2026-06-23 04:46:46 +00:00
Jake Turner
be434d755a
feat: supply depot 2026-06-23 04:46:45 +00:00
chriscrosstalk
0c76a195b9 fix(qdrant): disable anonymous telemetry by default (#747)
Qdrant's upstream default enables anonymous telemetry to telemetry.qdrant.io,
which doesn't match NOMAD's offline-first "zero telemetry" posture. Adding
QDRANT__TELEMETRY_DISABLED=true to the container environment turns it off for
fresh installs and reinstalls.

Existing installs keep their current telemetry-enabled container until the
Qdrant service is force-reinstalled via the Knowledge Base panel or the next
container recreation — Docker bakes Env into containers at create time, so
env changes require a new container.

Closes #742

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 14:26:28 -07:00
Jake Turner
9e3828bcba feat(Kiwix): migrate to Kiwix library mode for improved stability (#622) 2026-04-03 14:26:50 -07:00
Jake Turner
0226e651c7 fix: bump default ollama and cyberchef versions 2026-03-25 16:30:35 -07:00
Jake Turner
58b106f388 feat: support for updating services 2026-03-11 14:08:09 -07:00
Jake Turner
9731ce839d fix(Docker): pin Qdrant and Ollama versions 2026-02-02 00:26:26 +00:00
Jake Turner
31c671bdb5 fix: service name defs and ollama ui location 2026-02-01 05:46:23 +00:00
Jake Turner
adf76d272e fix: remove Open WebUI 2026-01-31 20:39:49 -08:00
Jake Turner
50174d2edb feat(RAG): [wip] RAG capabilities 2026-01-31 20:39:49 -08:00
Jake Turner
8092fb58d8 fix(Benchmark): remove unused seeder definition 2026-01-22 21:48:12 -08:00
Chris Sherwood
755807f95e feat: Add system benchmark feature with NOMAD Score
Add comprehensive benchmarking capability to measure server performance:

Backend:
- BenchmarkService with CPU, memory, disk, and AI benchmarks using sysbench
- Database migrations for benchmark_results and benchmark_settings tables
- REST API endpoints for running benchmarks and retrieving results
- CLI commands: benchmark:run, benchmark:results, benchmark:submit
- BullMQ job for async benchmark execution with SSE progress updates
- Synchronous mode option (?sync=true) for simpler local dev setup

Frontend:
- Benchmark settings page with circular gauges for scores
- NOMAD Score display with weighted composite calculation
- System Performance section (CPU, Memory, Disk Read/Write)
- AI Performance section (tokens/sec, time to first token)
- Hardware Information display
- Expandable Benchmark Details section
- Progress simulation during sync benchmark execution

Easy Setup Integration:
- Added System Benchmark to Additional Tools section
- Built-in capability pattern for non-Docker features
- Click-to-navigate behavior for built-in tools

Fixes:
- Docker log multiplexing issue (Tty: true) for proper output parsing
- Consolidated disk benchmarks into single container execution

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-22 21:48:12 -08:00
Chris Sherwood
24f10ea3d5 feat: Use friendly app names on Dashboard with open source attribution
Updates the Dashboard to use the same user-friendly names as the Easy Setup
Wizard, giving credit to the open source projects powering each capability:

- Kiwix → Information Library (Powered by Kiwix)
- Kolibri → Education Platform (Powered by Kolibri)
- Open WebUI → AI Assistant (Powered by Open WebUI + Ollama)
- FlatNotes → Notes (Powered by FlatNotes)
- CyberChef → Data Tools (Powered by CyberChef)

Also reorders Dashboard cards to prioritize Core Capabilities first, with
Maps promoted to Core Capability status, followed by Additional Tools,
then system items (Easy Setup, Install Apps, Docs, Settings).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-22 16:43:32 -08:00
Chris Sherwood
d86c78dba5 feat: Add Windows Docker Desktop support for local development
- Detect Windows platform and use named pipe (//./pipe/docker_engine)
  instead of Unix socket for Docker Desktop compatibility
- Add NOMAD_STORAGE_PATH environment variable for configurable
  storage paths across different platforms
- Update seeder to use environment variable with Linux default
- Document new environment variable in .env.example

This enables local development on Windows machines with Docker Desktop
while maintaining Linux production compatibility.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-19 10:29:24 -08:00
Jake Turner
bb67bab9a9 feat: improved app cards and custom icons 2026-01-15 15:54:59 -08:00
Jake Turner
c46b75e63d fix(admin): improve duplicate install request handling 2026-01-13 06:58:05 -08:00
Jake Turner
5a19882273 fix(admin): port binding for OpenWebUI 2026-01-13 06:58:05 -08:00
Jake Turner
2ff7b055b5
fix(Kiwix): initial download and setup 2025-12-07 16:04:41 -08:00
Jake Turner
606dd3ad0b
feat: [wip] custom map and zim downloads 2025-12-02 08:25:09 -08:00
Jake Turner
9670a78fb4 feat: kolibri app 2025-11-18 16:35:16 -08:00
Jake Turner
b8eaaa7ac6
feat(Services): friendly names and descriptions 2025-11-18 14:02:22 -08:00
Jake Turner
033cc10420 feat: add flatnotes as app 2025-10-06 23:53:40 -07:00
Jake Turner
1df7c490a6 feat: add cyberchef as app 2025-10-06 23:22:50 -07:00
Jake Turner
85e6b84e32 fix(OSM): use absolute host paths 2025-10-06 21:13:22 -07:00
Jake Turner
4e1377554a fix(OSM): directory paths and access 2025-10-06 21:13:22 -07:00
Jake Turner
2099750e06 fix(OSM): osm installation 2025-08-20 23:05:19 -07:00
Jake Turner
5ee949b96a fix(Docker): [wip] OSM install fixes 2025-08-20 23:05:19 -07:00
Jake Turner
5fc490715d ref: cleanup service seeder 2025-08-08 15:07:32 -07:00
Jake Turner
2373f2c1b2 fix(open-webui): ollama connection 2025-07-11 20:21:44 -07:00
Jake Turner
3b81e00a69 feat: openwebui+ollama and zim management 2025-07-09 09:08:21 -07:00
Jake Turner
39d75c9cdf feat: init app installation and documentation 2025-06-30 01:44:42 -07:00
Jake Turner
b33a1b3e37 feat: initial commit 2025-06-29 15:51:08 -07:00