Commit Graph

494 Commits

Author SHA1 Message Date
gujishh
9d73628ee3 docs(faq): add recovery steps for missing Kiwix library XML 2026-05-04 10:27:50 -07:00
chriscrosstalk
9cbf8c2135
build: write version.json from VERSION build-arg (#754)
The Dockerfile copied root package.json to /app/version.json, which
SystemService.getAppVersion() reads on every render of the app version in
the UI. semantic-release only reliably commits that bump back on the main
branch; on the rc branch it does not, so v1.31.1-rc.1 and v1.31.1-rc.2
both shipped with a version.json still reading 1.31.0. Result: a user who
upgrades to rc.2 sees "1.31.0" in the UI and a persistent "update to
v1.31.1-rc.2 available" prompt.

The build workflow already passes VERSION as a build-arg (used today only
for the OCI image label). Generating version.json from that arg at build
time makes the image tag the single source of truth and eliminates the
drift, regardless of what the committed-back package.json says.

Dev builds (no VERSION override) write "dev", which matches the existing
NODE_ENV=development short-circuit in getAppVersion().

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 00:58:43 +00:00
cosmistack-bot
3117a1be9d docs(release): finalize v1.31.1 release notes [skip ci] 2026-04-21 21:27:53 +00:00
cosmistack-bot
1a81290b31 chore(release): 1.31.1 [skip ci] 2026-04-21 21:27:16 +00:00
Jake Turner
bd20ba87bd ci: ensure tags are force fetched on semantic release 2026-04-21 14:26:28 -07:00
Jake Turner
5cbe6f5203 docs: update release notes 2026-04-21 14:26:28 -07:00
chriscrosstalk
216509ae0d fix(rag): repair ZIM embedding pipeline (sync filter, batch gate, DOM walk) (#745)
Three bugs in the RAG embedding pipeline, diagnosed and patched by @sbruschke
against v1.31.0 with working before/after chunk counts. All three are
root-cause contributors to #388.

1. scanAndSyncStorage queued every file under /storage/zim/ for embedding,
   including Kiwix's generated kiwix-library.xml. EmbedFileJob rejected it
   with "Unsupported file type" and the default 30-attempt retry policy
   kept it looping on every sync, flooding nomad_admin logs. Now gated on
   determineFileType(filePath) !== 'unknown'.

2. hasMoreBatches compared zimChunks.length (section-level chunk count
   under the 'structured' strategy) against ZIM_BATCH_SIZE (an article
   limit). Because articles emit multiple sections, the two are never
   equal for real archives and processing silently stopped after the
   first 50 articles. Now gated on articlesInBatch >= ZIM_BATCH_SIZE.

3. extractStructuredContent walked only direct children of <body>, so any
   ZIM that wraps content in a container div (Devdocs, Wikipedia,
   FreeCodeCamp, React docs, etc.) produced zero sections and silently
   embedded zero chunks while reporting success. Now walks the full DOM
   via $('body').find('h2, h3, h4, p, ul, ol, dl, table'), with a
   whole-body text fallback when the selector walk yields nothing.

Before/after chunk counts confirmed by @sbruschke on v1.31.0:
  devdocs_en_git   0 -> 916
  devdocs_en_react 0 -> 481
  devdocs_en_node  0 -> 423
  libretexts_en_eng 1 -> 35 (climbing)
Wikipedia resumed progressing normally through its 6M articles.

Closes #718
Closes #719
Closes #720
Closes #388

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 14:26:28 -07:00
chriscrosstalk
810a70acb7 fix(ZIM): accumulate across Kiwix pages to prevent empty Content Explorer (#746)
When many ZIMs are already installed locally, a single Kiwix catalog page
(12 items) could return 12 already-installed items, which zim_service
would fully filter out client-side. The endpoint returned items: [] with
has_more: true, and the frontend's infinite-scroll guard
(flatData.length > 0) blocked fetchNextPage — leaving the user with
"No records found" despite plenty of uninstalled ZIMs available.

Backend now accumulates across up to 5 Kiwix fetches (60 items each)
until it has enough post-filter results to return, dedupes by entry id,
advances currentStart by actual entries returned (not requested), and
returns a next_start cursor. The frontend consumes that cursor instead
of computing Kiwix offsets locally, and the flatData.length > 0 guard is
removed so the existing on-mount effect drives bounded auto-fetch when
a short page lands.

The pre-existing has_more off-by-one (compared totalResults against the
input start rather than the post-fetch position) is fixed implicitly.

Diagnosis credit: @johno10661.

Closes #731

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 14:26:28 -07:00
chriscrosstalk
6646b3480b fix(AI): stop local nomad_ollama container when remote Ollama is configured (#744)
When users set a remote Ollama URL via AI Settings, the local nomad_ollama
container continued running and competed with the remote host for port 11434
and GPU access. Now configureRemote stops the local container on set and
restores it on clear (if still present). Container and its models volume are
preserved so the local install can be re-enabled later.

Closes #662

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 14:26:28 -07:00
0xGlitch
33727c744f fix(UI): gate NAS Storage label on network filesystem type (#749)
Closes #743
2026-04-21 14:26:28 -07:00
chriscrosstalk
0c76a195b9 fix(qdrant): disable anonymous telemetry by default (#747)
Qdrant's upstream default enables anonymous telemetry to telemetry.qdrant.io,
which doesn't match NOMAD's offline-first "zero telemetry" posture. Adding
QDRANT__TELEMETRY_DISABLED=true to the container environment turns it off for
fresh installs and reinstalls.

Existing installs keep their current telemetry-enabled container until the
Qdrant service is force-reinstalled via the Knowledge Base panel or the next
container recreation — Docker bakes Env into containers at create time, so
env changes require a new container.

Closes #742

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 14:26:28 -07:00
chriscrosstalk
056556497c docs: add Community Add-Ons page with field manuals + W3Schools packs (#753)
Introduces a dedicated page listing third-party ZIM content packs built
by the community. Launches with the two current add-ons (jrsphoto field
manuals, kennethbrewer W3Schools) and explains how to install a ZIM pack
and where to submit a new one for inclusion.

- New doc at admin/docs/community-add-ons.md
- Wired into DocsService DOC_ORDER (slot 4) and TITLE_OVERRIDES so the
  hyphen in "Add-Ons" is preserved in the sidebar
- README gets a link under Community & Resources

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 14:26:28 -07:00
Jake Turner
b7b3bf00de docs: update release notes 2026-04-21 14:26:28 -07:00
dependabot[bot]
7ec3d790d1 build(deps): bump lodash from 4.17.23 to 4.18.1 in /admin (#643)
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.23 to 4.18.1.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.23...4.18.1)

---
updated-dependencies:
- dependency-name: lodash
  dependency-version: 4.18.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-21 14:26:28 -07:00
dependabot[bot]
b6bb0f2321 build(deps-dev): bump vite from 6.4.1 to 6.4.2 in /admin (#677)
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 6.4.1 to 6.4.2.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v6.4.2/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v6.4.2/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 6.4.2
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-21 14:26:28 -07:00
dependabot[bot]
92b6f3c22f build(deps): bump @adonisjs/http-server from 7.8.0 to 7.8.1 in /admin (#724)
Bumps [@adonisjs/http-server](https://github.com/adonisjs/http-server) from 7.8.0 to 7.8.1.
- [Release notes](https://github.com/adonisjs/http-server/releases)
- [Commits](https://github.com/adonisjs/http-server/compare/v7.8.0...v7.8.1)

---
updated-dependencies:
- dependency-name: "@adonisjs/http-server"
  dependency-version: 7.8.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-21 14:26:28 -07:00
dependabot[bot]
6ec0678752 build(deps): bump protobufjs from 7.5.4 to 7.5.5 in /admin (#737)
Bumps [protobufjs](https://github.com/protobufjs/protobuf.js) from 7.5.4 to 7.5.5.
- [Release notes](https://github.com/protobufjs/protobuf.js/releases)
- [Changelog](https://github.com/protobufjs/protobuf.js/blob/master/CHANGELOG.md)
- [Commits](https://github.com/protobufjs/protobuf.js/compare/protobufjs-v7.5.4...protobufjs-v7.5.5)

---
updated-dependencies:
- dependency-name: protobufjs
  dependency-version: 7.5.5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-21 14:26:28 -07:00
dependabot[bot]
56dbf95c66 build(deps): bump protocol-buffers-schema from 3.6.0 to 3.6.1 in /admin (#736)
Bumps [protocol-buffers-schema](https://github.com/mafintosh/protocol-buffers-schema) from 3.6.0 to 3.6.1.
- [Commits](https://github.com/mafintosh/protocol-buffers-schema/compare/v3.6.0...v3.6.1)

---
updated-dependencies:
- dependency-name: protocol-buffers-schema
  dependency-version: 3.6.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-21 14:26:28 -07:00
dependabot[bot]
5f0463bb08 build(deps): bump axios from 1.13.5 to 1.15.0 in /admin (#708)
Bumps [axios](https://github.com/axios/axios) from 1.13.5 to 1.15.0.
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/v1.x/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v1.13.5...v1.15.0)

---
updated-dependencies:
- dependency-name: axios
  dependency-version: 1.15.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-21 14:26:28 -07:00
dependabot[bot]
540c0abee5 build(deps): bump follow-redirects from 1.15.11 to 1.16.0 in /admin (#729)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.15.11 to 1.16.0.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.15.11...v1.16.0)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-version: 1.16.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-21 14:26:28 -07:00
chriscrosstalk
6c33a96972 fix(AI): allow cancelling in-progress model downloads and ensure consistent progress UI (#701)
Adds a cancel button to in-progress Ollama model downloads and unifies
the Active Model Downloads card layout with the Active Downloads card
used for ZIMs, maps, and pmtiles (byte counts, progress bar, live speed,
status indicator).

Closes #676.
2026-04-21 14:26:28 -07:00
Luís Miguel
806b2c1714 fix(security): SSRF validation for map downloads and error sanitization (CWE-918, CWE-209) (#552)
* fix(security): add SSRF validation to map download URLs from manifest
* fix(security): sanitize verbose error in rag controller scan endpoint
* fix(security): sanitize verbose errors in benchmark controller
* fix(security): sanitize verbose error in system controller version fetch
* fix(security): sanitize verbose errors in chats controller (6 instances)
* fix(security): sanitize verbose errors in docker service (6 instances)
* fix(security): sanitize verbose error in system update service
* fix(security): sanitize verbose errors in collection update service
---------
Co-authored-by: Jake Turner <52841588+jakeaturner@users.noreply.github.com>
2026-04-21 14:26:28 -07:00
Jake Turner
2b8c847295 fix(Downloads): remove duplicate err listnr and improv Range req stability 2026-04-21 14:26:28 -07:00
Aaron Bird
8d026da06e fix(downloads): stage downloads to .tmp to prevent Kiwix loading partial files
Downloads are now written to `filepath + '.tmp'` and atomically renamed
to the final path only on successful completion. Kiwix globs for `*.zim`
and ZimService filters `.endsWith('.zim')`, so `.tmp` files are invisible
to both during download. The same staging applies to `.pmtiles` map files.

Ref #372

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-21 14:26:28 -07:00
Ben Gauger
151b454ad9 fix(disk-display): show NAS Storage label in fsSize fallback path
Co-Authored-By: Ben Smith <bravosierra99@gmail.com>
2026-04-21 14:26:28 -07:00
Ben Gauger
84399b19d9 fix(disk-collector): fix storage reporting for NFS mounts
Co-Authored-By: Ben Smith <bravosierra99@gmail.com>
2026-04-21 14:26:28 -07:00
Jake Turner
c8cb79a3a5 fix: prevent ZIM corrupt file crash and deduplicate Ollama download logs (#741)
Corrupted ZIM files cause a native C++ abort (ZimFileFormatError) that
bypasses JS try/catch and kills the process. Add magic number validation
before passing files to @openzim/libzim so invalid files are skipped
gracefully. Also deduplicate Ollama download progress broadcasts — both
within a single stream (skip unchanged percentages) and across concurrent
callers (share one download promise per model).

Co-authored-by: aegisman <aegis@manicode.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-21 14:26:28 -07:00
Henry Estela
6510f42184 fix(AI): qwen2.5 loading on every chat message (#649)
Use the currently loaded model for chat title generation and query rewrite.
2026-04-21 14:26:28 -07:00
Henry Estela
4d866167a2 fix(AI): add null check to model name (#645)
When the OpenAI-compatible fallback (/v1/models) is used, models are mapped as { name: m.id, size: 0 } with no details field. Accessing model.details.parameter_size throws `TypeError: Cannot read properties of undefined`, which crashes the React render and causes the entire page to go blank.
2026-04-21 14:26:28 -07:00
cosmistack-bot
8dcbf7dbcf docs(release): finalize v1.31.0 release notes [skip ci] 2026-04-03 21:27:57 +00:00
cosmistack-bot
37abad33c9 chore(release): 1.31.0 [skip ci] 2026-04-03 21:27:36 +00:00
Jake Turner
d666b24598 docs: update release notes 2026-04-03 14:26:50 -07:00
chriscrosstalk
a813468949 feat(maps): add imperial/metric toggle for scale bar (#641)
Defaults to metric for global audience. Persists choice in localStorage.
Segmented button styled to match MapLibre controls.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 14:26:50 -07:00
cosmistack-bot
e72268bb1c chore(release): 1.31.0-rc.3 [skip ci] 2026-04-03 14:26:50 -07:00
chriscrosstalk
0183b42d71 feat(maps): add scale bar and location markers (#636)
Add distance scale bar and user-placed location pins to the offline maps viewer.

- Scale bar (bottom-left) shows distance reference that updates with zoom level
- Click anywhere on map to place a named pin with color selection (6 colors)
- Collapsible "Saved Locations" panel lists all pins with fly-to navigation
- Full dark mode support for popups and panel via CSS overrides
- New `map_markers` table with future-proofed columns for routing (marker_type,
  route_id, route_order, notes) to avoid a migration when routes are added later
- CRUD endpoints: GET/POST /api/maps/markers, PATCH/DELETE /api/maps/markers/:id
- VineJS validation on create/update
- MapMarker Lucid model

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 14:26:50 -07:00
Jake Turner
6287755946 fix(Maps): ensure proper parsing of hostnames (#640) 2026-04-03 14:26:50 -07:00
cosmistack-bot
0f9be7c215 chore(release): 1.31.0-rc.2 [skip ci] 2026-04-03 14:26:50 -07:00
Jake Turner
afbe4c42b1 docs: update release notes 2026-04-03 14:26:50 -07:00
0xGlitch
d7e3d9246b fix(downloads): improved handling for large file downloads and user-initiated cancellation (#632)
* fix(downloads): increase retry attempts and backoff for large file downloads
* fix download retry config and abort handling
* use abort reason to detect user-initiated cancels
2026-04-03 14:26:50 -07:00
Jake Turner
cb4fa003a4 fix: cache docker list requests, aiAssistantName fetching, and ensure inertia used properly 2026-04-03 14:26:50 -07:00
Jake Turner
877fb1276a feat: gzip compression by default for all registered routes 2026-04-03 14:26:50 -07:00
Jake Turner
1e4b7aea82 fix(UI): manual import map for DynamicIcon to avoid huge bundle of Tabler icons 2026-04-03 14:26:50 -07:00
Jake Turner
a14dd688fa feat(KnowledgeBase): support up to 5 files upload of 100mb each per req 2026-04-03 14:26:50 -07:00
cosmistack-bot
1bd1811498 chore(release): 1.31.0-rc.1 [skip ci] 2026-04-03 14:26:50 -07:00
Jake Turner
3e922877d2 docs: update release notes 2026-04-03 14:26:50 -07:00
Jake Turner
9964a82240 docs: update CONTRIBUTING.md 2026-04-03 14:26:50 -07:00
Jake Turner
91a0b8bad5 docs: update FAQ 2026-04-03 14:26:50 -07:00
Jake Turner
9e3828bcba feat(Kiwix): migrate to Kiwix library mode for improved stability (#622) 2026-04-03 14:26:50 -07:00
Henry Estela
43c8876f19 feat(docs): add simple API reference (#615)
Adds tables with method,path and description in /docs/api-reference/
2026-04-03 14:26:50 -07:00
Jake Turner
31986d7319 chore(deps): bump yaml, fast-xml-parser, pmtiles, tailwindcss, @types/dockerode 2026-04-03 14:26:50 -07:00