github-mirror/project-nomad
1
0
Fork 0
mirror of https://github.com/Crosstalk-Solutions/project-nomad.git synced 2026-05-26 06:15:07 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph
Project N.O.M.A.D, is a self-contained, offline survival computer packed with critical tools, knowledge, and AI to keep you informed and empowered—anytime, anywhere.
504 Commits 32 Branches 69 Tags 31 MiB
TypeScript 93.7%
Shell 5.6%
CSS 0.3%
Dockerfile 0.3%
73e2115245
Go to file
chriscrosstalk 73e2115245 feat(AI): improved AMD GPU acceleration for Ollama via ROCm + HSA override (#804) 
* feat(AI): re-enable AMD GPU acceleration for Ollama via ROCm + HSA override

Re-enables AMD GPU support that was disabled in 77f1868 pending validation
of the ROCm image and device discovery. Validation done 2026-04-28 on a
Minisforum UM890 Pro (Ryzen 9 PRO 8945HS + Radeon 780M iGPU) — Ollama
correctly offloaded all model layers to the iGPU when the container was
started with /dev/kfd + /dev/dri passthrough and HSA_OVERRIDE_GFX_VERSION=11.0.0.
On llama3.2:1b, GPU inference ran at 51.83 tok/s vs 33.16 tok/s on CPU
(same hardware, same prompt) — a 1.56x speedup confirmed by Ollama logs
showing "load_tensors: offloaded 17/17 layers to GPU".

Changes
-------

docker_service.ts
- Restore _discoverAMDDevices() (simplified — pass /dev/dri as a directory
  entry, mirroring `docker run --device /dev/dri` behavior, instead of the
  prior brittle hardcoded card0/renderD128 fallback that broke on systems
  where the AMD GPU enumerates as card1+).
- Restore the AMD branch in _createContainer():
  - Switches Ollama image to ollama/ollama:rocm
  - Mounts /dev/kfd + /dev/dri via Devices
  - Sets HSA_OVERRIDE_GFX_VERSION=11.0.0 (required for unsupported-but-RDNA3
    iGPUs like gfx1103; harmless on supported discrete cards)
  - KV opt-out via ai.amdGpuAcceleration (default on)
- Mirror the AMD branch in updateContainer():
  - Lifted GPU detection above docker.pull() so AMD updates pull :rocm
    rather than the standard :targetVersion tag (per-version ROCm tags
    aren't always published)
  - Replaces stale HSA_OVERRIDE in the inspect-captured env on update,
    so containers built before this PR pick up the current value

system_service.ts
- New getOllamaInferenceComputeFromLogs() — parses Ollama startup log line
  "msg=\"inference compute\" ... library=CUDA|ROCm ..." which Ollama emits
  for both NVIDIA and AMD. Catches silent CPU fallback (e.g. NVML death
  after update, or HSA_OVERRIDE failure) that the prior nvidia-smi exec
  probe couldn't detect.
- gpuHealth refactored to use log parsing as the primary probe for both
  vendors, with nvidia-smi exec retained as the NVIDIA-only secondary
  path for hardware enrichment when log parsing has no startup line yet.
- AMD path uses gpu.type KV value (persisted by DockerService._detectGPUType)
  + ai.amdGpuAcceleration opt-out to determine hasRocmRuntime.

types/system.ts
- GpuHealthStatus extended additively: hasRocmRuntime + optional gpuVendor.

types/kv_store.ts
- New ai.amdGpuAcceleration boolean (default-on).

settings/models.tsx, settings/system.tsx
- passthrough_failed banner copy now reads vendor from gpuHealth.gpuVendor
  ("an AMD GPU" vs "an NVIDIA GPU"). Same Fix button hits the same
  force-reinstall endpoint, which now configures AMD correctly.

install_nomad.sh
- AMD detection in verify_gpu_setup() upgraded from a strict-positive
  "ROCm not currently available" message to "ROCm acceleration will be
  configured automatically." Also tightens the lspci match to display
  controller classes (avoids false positives from AMD CPU host bridges,
  matching the same fix already in DockerService._detectGPUType).

Auto-remediation
----------------

Issue #755 proposes auto-remediation when gpuHealth.status flips to
passthrough_failed (today the user has to click "Fix: Reinstall AI
Assistant"). When that PR lands, AMD coverage falls out for free since
this PR uses the same passthrough_failed status code via the shared
gpuHealth machinery — #755's guard will need to flip from
hasNvidiaRuntime === true to (hasNvidiaRuntime || hasRocmRuntime).

Closes #124 (AMD GPU support).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(AI): detect AMD GPU presence inside admin container via marker file

The admin container doesn't have lspci installed, and AMD GPUs don't register
a Docker runtime the way NVIDIA does — so DockerService._detectGPUType() and
SystemService.gpuHealth had no way to know an AMD GPU was present.

The previous implementation fell through to lspci, which silently failed inside
the admin container, leaving gpu.type unset and gpuHealth stuck at 'no_gpu'
even on systems with an AMD GPU. (NVIDIA worked because Docker registers the
nvidia runtime, which is reachable via dockerInfo.Runtimes from any container.)

Discovered while testing the AMD acceleration patch on a Minisforum UM890 Pro:
the AMD branch in _createContainer() never fired because _detectGPUType()
returned 'none' even on a host with a working /dev/kfd.

Fix
---

install_nomad.sh writes the host-detected GPU type ('nvidia' | 'amd') to a
marker file in the storage volume the admin container already bind-mounts:

  /opt/project-nomad/storage/.nomad-gpu-type  →  /app/storage/.nomad-gpu-type

DockerService._detectGPUType() reads the marker as a secondary probe (after
the Docker runtime check) — covers AMD detection from inside the container
without requiring lspci or a /dev bind mount.

SystemService falls back to the marker file when KV gpu.type is empty so the
System page reflects AMD presence even before the user installs AI Assistant
for the first time. (Without this, the page would say 'no_gpu' until Ollama
was installed, even on hosts with an AMD GPU detected at install time.)

Verified on NOMAD6 (UM890 Pro, Ubuntu 24.04, 780M iGPU): with the marker file
in place and admin restarted, the patch's AMD branch fires correctly on Force
Reinstall AI Assistant. Resulting nomad_ollama runs ollama/ollama:rocm with
/dev/kfd + /dev/dri passthrough and HSA_OVERRIDE_GFX_VERSION=11.0.0; Ollama
logs show 'library=ROCm compute=gfx1100 ... type=iGPU'. NOMAD's in-product
benchmark on the same hardware climbed from 33.8 tok/s (CPU) to 57.3 tok/s
(GPU) — a 1.69x speedup, with TTFT dropping from 148ms to 66ms.

Migration for existing AMD installs
-----------------------------------

Users on an existing NOMAD install with an AMD GPU have no marker file (the
install script wrote it on a fresh install). Two paths get them on the GPU:
  1. Re-run install_nomad.sh — writes the marker, no other side effects
  2. Manually: echo amd | sudo tee /opt/project-nomad/storage/.nomad-gpu-type

Either then triggers AMD detection on the next AI Assistant install/reinstall.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(AI): pull ollama/ollama:rocm separately when AMD branch overrides image

The pull-if-missing logic in _createContainer ran against service.container_image
(the DB-pinned tag, e.g. ollama/ollama:0.18.2). The AMD branch then overrode
finalImage to ollama/ollama:rocm — but if that image wasn't already local, the
container creation step failed with "no such image: ollama/ollama:rocm".

Caught while validating on NOMAD2 (Ryzen AI 9 HX 370 + Radeon 890M / RDNA 3.5):
the prior end-to-end test on NOMAD6 had silently passed because the rocm image
was already pulled there from an earlier sidecar test, masking the bug.

Fix: inside the AMD branch, after setting finalImage to ollama/ollama:rocm,
run a parallel _checkImageExists + docker.pull dance for the new tag.

Also confirmed via this validation: the same HSA_OVERRIDE_GFX_VERSION=11.0.0
override works on the 890M (gfx1150 / RDNA 3.5) — Ollama logs report
'library=ROCm compute=gfx1100 description="AMD Radeon 890M Graphics"' and
inference runs at 51.68 tok/s (matching the existing X1 Pro published tile
of 51.7 tok/s on the same hardware class). RDNA 3 (780M, gfx1103) and RDNA
3.5 (890M, gfx1150) both use the same override successfully.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* build(Dockerfile): include pciutils for lspci gpu detection fallback

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Jake Turner <jturner@cosmistack.com>
2026-05-20 10:16:00 -07:00
.github ci: ensure tags are force fetched on semantic release 2026-04-21 14:26:28 -07:00
admin feat(AI): improved AMD GPU acceleration for Ollama via ROCm + HSA override (#804) 2026-05-20 10:16:00 -07:00
collections fix(Collections): update ZIM files to latest versions (#332) 2026-03-25 16:30:35 -07:00
install feat(AI): improved AMD GPU acceleration for Ollama via ROCm + HSA override (#804) 2026-05-20 10:16:00 -07:00
.dockerignore build: switch to node:22-slim image for libzim compat 2026-02-09 16:23:47 -08:00
.gitignore feat: openwebui+ollama and zim management 2025-07-09 09:08:21 -07:00
.releaserc.json ops: support RC versions 2026-03-03 20:51:38 -08:00
CODE_OF_CONDUCT.md docs: add Contributor Covenant Code of Conduct 2026-03-11 17:07:41 -07:00
CONTRIBUTING.md docs: require linked issue for non-trivial PRs (#799) 2026-05-20 10:16:00 -07:00
Dockerfile feat(AI): improved AMD GPU acceleration for Ollama via ROCm + HSA override (#804) 2026-05-20 10:16:00 -07:00
FAQ.md docs: update FAQ 2026-04-03 14:26:50 -07:00
LICENSE chore: add Apache 2.0 license 2026-03-04 20:05:14 -08:00
package-lock.json feat(RAG): display embedding queue and improve progress tracking 2026-03-04 20:05:14 -08:00
package.json chore(release): 1.31.1 [skip ci] 2026-04-21 21:27:16 +00:00
README.md docs: add Community Add-Ons page with field manuals + W3Schools packs (#753) 2026-04-21 14:26:28 -07:00

README.md

Project N.O.M.A.D.

Node for Offline Media, Archives, and Data

Knowledge That Never Goes Offline

Website Discord Benchmark

Project N.O.M.A.D. is a self-contained, offline-first knowledge and education server packed with critical tools, knowledge, and AI to keep you informed and empowered—anytime, anywhere.

Installation & Quickstart

Project N.O.M.A.D. can be installed on any Debian-based operating system (we recommend Ubuntu). Installation is completely terminal-based, and all tools and resources are designed to be accessed through the browser, so there's no need for a desktop environment if you'd rather setup N.O.M.A.D. as a "server" and access it through other clients.

Note: sudo/root privileges are required to run the install script

Quick Install (Debian-based OS Only)

sudo apt-get update && \
sudo apt-get install -y curl && \
curl -fsSL https://raw.githubusercontent.com/Crosstalk-Solutions/project-nomad/refs/heads/main/install/install_nomad.sh \
  -o install_nomad.sh && \
sudo bash install_nomad.sh

Project N.O.M.A.D. is now installed on your device! Open a browser and navigate to http://localhost:8080 (or http://DEVICE_IP:8080) to start exploring!

For a complete step-by-step walkthrough (including Ubuntu installation), see the Installation Guide.

Advanced Installation

For more control over the installation process, copy and paste the Docker Compose template into a docker-compose.yml file and customize it to your liking (be sure to replace any placeholders with your actual values). Then, run docker compose up -d to start the Command Center and its dependencies. Note: this method is recommended for advanced users only, as it requires familiarity with Docker and manual configuration before starting.

How It Works

N.O.M.A.D. is a management UI ("Command Center") and API that orchestrates a collection of containerized tools and resources via Docker. It handles installation, configuration, and updates for everything — so you don't have to.

Built-in capabilities include:

  • AI Chat with Knowledge Base — local AI chat powered by Ollama or you can use OpenAI API compatible software such as LM Studio or llama.cpp, with document upload and semantic search (RAG via Qdrant)
  • Information Library — offline Wikipedia, medical references, ebooks, and more via Kiwix
  • Education Platform — Khan Academy courses with progress tracking via Kolibri
  • Offline Maps — downloadable regional maps via ProtoMaps
  • Data Tools — encryption, encoding, and analysis via CyberChef
  • Notes — local note-taking via FlatNotes
  • System Benchmark — hardware scoring with a community leaderboard
  • Easy Setup Wizard — guided first-time configuration with curated content collections

N.O.M.A.D. also includes built-in tools like a Wikipedia content selector, ZIM library manager, and content explorer.

What's Included

Capability Powered By What You Get
Information Library Kiwix Offline Wikipedia, medical references, survival guides, ebooks
AI Assistant Ollama + Qdrant Built-in chat with document upload and semantic search
Education Platform Kolibri Khan Academy courses, progress tracking, multi-user support
Offline Maps ProtoMaps Downloadable regional maps with search and navigation
Data Tools CyberChef Encryption, encoding, hashing, and data analysis
Notes FlatNotes Local note-taking with markdown support
System Benchmark Built-in Hardware scoring, Builder Tags, and community leaderboard

Device Requirements

While many similar offline survival computers are designed to be run on bare-minimum, lightweight hardware, Project N.O.M.A.D. is quite the opposite. To install and run the available AI tools, we highly encourage the use of a beefy, GPU-backed device to make the most of your install.

At it's core, however, N.O.M.A.D. is still very lightweight. For a barebones installation of the management application itself, the following minimal specs are required:

Note: Project N.O.M.A.D. is not sponsored by any hardware manufacturer and is designed to be as hardware-agnostic as possible. The harware listed below is for example/comparison use only

Minimum Specs

  • Processor: 2 GHz dual-core processor or better
  • RAM: 4GB system memory
  • Storage: At least 5 GB free disk space
  • OS: Debian-based (Ubuntu recommended)
  • Stable internet connection (required during install only)

To run LLM's and other included AI tools:

Optimal Specs

  • Processor: AMD Ryzen 7 or Intel Core i7 or better
  • RAM: 32 GB system memory
  • Graphics: NVIDIA RTX 3060 or AMD equivalent or better (more VRAM = run larger models)
  • Storage: At least 250 GB free disk space (preferably on SSD)
  • OS: Debian-based (Ubuntu recommended)
  • Stable internet connection (required during install only)

For detailed build recommendations at three price points ($150–$1,000+), see the Hardware Guide.

Again, Project N.O.M.A.D. itself is quite lightweight - it's the tools and resources you choose to install with N.O.M.A.D. that will determine the specs required for your unique deployment

Running AI models on a different host

By default, N.O.M.A.D.'s installer will attempt to setup Ollama on the host when the AI Assistant is installed. However, if you would like to run the AI model on a different host, you can go to the settings of of the AI assistant and input a URL for either an ollama or OpenAI-compatible API server (such as LM Studio).
Note that if you use Ollama on a different host, you must start the server with this option OLLAMA_HOST=0.0.0.0.
Ollama is the preferred way to use the AI assistant as it has features such as model download that OpenAI API does not support. So when using LM Studio for example, you will have to use LM Studio to download models. You are responsible for the setup of Ollama/OpenAI server on the other host.

Frequently Asked Questions (FAQ)

For answers to common questions about Project N.O.M.A.D., please see our FAQ page.

About Internet Usage & Privacy

Project N.O.M.A.D. is designed for offline usage. An internet connection is only required during the initial installation (to download dependencies) and if you (the user) decide to download additional tools and resources at a later time. Otherwise, N.O.M.A.D. does not require an internet connection and has ZERO built-in telemetry.

To test internet connectivity, N.O.M.A.D. attempts to make a request to Cloudflare's utility endpoint, https://1.1.1.1/cdn-cgi/trace and checks for a successful response.

About Security

By design, Project N.O.M.A.D. is intended to be open and available without hurdles - it includes no authentication. If you decide to connect your device to a local network after install (e.g. for allowing other devices to access it's resources), you can block/open ports to control which services are exposed.

Will authentication be added in the future? Maybe. It's not currently a priority, but if there's enough demand for it, we may consider building in an optional authentication layer in a future release to support uses cases where multiple users need access to the same instance but with different permission levels (e.g. family use with parental controls, classroom use with teacher/admin accounts, etc.). We have a suggestion for this on our public roadmap, so if this is something you'd like to see, please upvote it here: https://roadmap.projectnomad.us/posts/1/user-authentication-please-build-in-user-auth-with-admin-user-roles

For now, we recommend using network-level controls to manage access if you're planning to expose your N.O.M.A.D. instance to other devices on a local network. N.O.M.A.D. is not designed to be exposed directly to the internet, and we strongly advise against doing so unless you really know what you're doing, have taken appropriate security measures, and understand the risks involved.

Contributing

Contributions are welcome and appreciated! Please see CONTRIBUTING.md for guidelines on how to contribute to the project.

Community & Resources

License

Project N.O.M.A.D. is licensed under the Apache License 2.0.

Helper Scripts

Once installed, Project N.O.M.A.D. has a few helper scripts should you ever need to troubleshoot issues or perform maintenance that can't be done through the Command Center. All of these scripts are found in Project N.O.M.A.D.'s install directory, /opt/project-nomad

Start Script - Starts all installed project containers
sudo bash /opt/project-nomad/start_nomad.sh

Stop Script - Stops all installed project containers
sudo bash /opt/project-nomad/stop_nomad.sh

Update Script - Attempts to pull the latest images for the Command Center and its dependencies (i.e. mysql) and recreate the containers. Note: this only updates the Command Center containers. It does not update the installable application containers - that should be done through the Command Center UI
sudo bash /opt/project-nomad/update_nomad.sh
Uninstall Script - Need to start fresh? Use the uninstall script to make your life easy. Note: this cannot be undone!
curl -fsSL https://raw.githubusercontent.com/Crosstalk-Solutions/project-nomad/refs/heads/main/install/uninstall_nomad.sh -o uninstall_nomad.sh && sudo bash uninstall_nomad.sh