Compare commits

...

21 Commits

Author SHA1 Message Date
cosmistack-bot
8eb07808e9 chore(release): 1.29.0-rc.5 [skip ci] 2026-03-11 20:44:32 +00:00
Jake Turner
9adfb9fa33
build: add OCI image labels to Dockerfile 2026-03-11 20:43:02 +00:00
Chris Sherwood
c931c2b961 fix(UI): replace WikiHow reference with DIY repair guides
WikiHow ZIM files were deprecated by Kiwix after WikiHow requested
removal to protect their content from LLM training harvesting.
Replace with "DIY repair guides and how-to content" which accurately
reflects the iFixit, Stack Exchange, and other how-to content
available in NOMAD's curated collections.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 13:33:50 -07:00
Chris Sherwood
a20ea54d12 feat(collections): add Project Gutenberg ZIMs and fix broken education entry
Add Project Gutenberg books from the Library of Congress Classification
to relevant curated collection categories:

- Agriculture Comprehensive: Gutenberg Agriculture (LCC-S, 4.3 GB) —
  classic texts on farming, animal husbandry, and food preservation
- Survival Comprehensive: Gutenberg Military Science (LCC-U, 1.2 GB) —
  classic military strategy, tactics, and field manuals

Remove broken gutenberg_en_education entry from Education Standard tier.
The URL returned 404 — Kiwix only publishes LCC-coded Gutenberg ZIMs,
not topic-named ones. The pre-1928 educational philosophy texts were
also not practical enough for NOMAD's audience.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 13:29:50 -07:00
Jake Turner
4ee08fb344
ci: tag with and without v prefix 2026-03-11 17:20:20 +00:00
cosmistack-bot
20b3af4739 chore(release): 1.29.0-rc.4 [skip ci] 2026-03-11 05:53:38 +00:00
Jake Turner
f574d1cca1
feat(AI Assistant): performance improvements and smarter RAG context usage 2026-03-11 05:52:46 +00:00
Jake Turner
8305e07c9f
feat(AI Assistant): improved state management and performance 2026-03-10 23:50:27 +00:00
Jake Turner
07560d82e6
feat(AI Assistant): remember last model used 2026-03-10 22:35:43 +00:00
cosmistack-bot
b340d84ad0 chore(release): 1.29.0-rc.3 [skip ci] 2026-03-09 05:50:15 +00:00
Jake Turner
64e3ad52f8
fix(System): ensure nomad container image tag resolves correctly 2026-03-09 05:44:37 +00:00
cosmistack-bot
0bc53727f3 chore(release): 1.29.0-rc.2 [skip ci] 2026-03-09 05:13:53 +00:00
Jake Turner
d0fd1cd690
fix(Settings): hide AI Assistant from navigation until installed 2026-03-09 05:13:06 +00:00
Chris Sherwood
20c28cb811 fix(security): narrow SSRF scope to allow RFC1918 LAN addresses
NOMAD is a LAN appliance — blocking RFC1918 private ranges (10.x,
172.16-31.x, 192.168.x) would prevent users from downloading content
from local network mirrors. Narrowed to only block loopback (localhost,
127.x, 0.0.0.0, ::1) and link-local (169.254.x, fe80::) addresses.
Restored require_tld: false for LAN hostnames without TLDs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 22:01:54 -07:00
Chris Sherwood
db1fe84553 fix(security): path traversal and SSRF protections from pre-launch audit
Fixes 4 high-severity findings from a comprehensive security audit:

1. Path traversal on ZIM file delete — resolve()+startsWith() containment
2. Path traversal on Map file delete — same pattern
3. Path traversal on docs read — same pattern (already used in rag_service)
4. SSRF on download endpoints — block private/internal IPs, require TLD

Also adds assertNotPrivateUrl() to content update endpoints.

Full audit report attached as admin/docs/security-audit-v1.md.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 22:01:54 -07:00
Chris Sherwood
9093d06a64 docs: update documentation for recent features and hardware page
- Add hardware guide link (projectnomad.us/hardware) to README, FAQ, and About page
- Add Apache 2.0 license section to README and About page
- Add Early Access Channel FAQ and Getting Started mention
- Add GPU passthrough warning troubleshooting entry to FAQ
- Add Knowledge Base document deletion to FAQ and Getting Started

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 21:56:59 -07:00
Jake Turner
bdeccfa791
feat: support for updating services 2026-03-09 04:55:43 +00:00
cosmistack-bot
b6a32a548c chore(release): 1.29.0-rc.1 [skip ci] 2026-03-05 22:35:35 +00:00
dependabot[bot]
5a35856747 build(deps): bump tar from 7.5.9 to 7.5.10 in /admin
Bumps [tar](https://github.com/isaacs/node-tar) from 7.5.9 to 7.5.10.
- [Release notes](https://github.com/isaacs/node-tar/releases)
- [Changelog](https://github.com/isaacs/node-tar/blob/main/CHANGELOG.md)
- [Commits](https://github.com/isaacs/node-tar/compare/v7.5.9...v7.5.10)

---
updated-dependencies:
- dependency-name: tar
  dependency-version: 7.5.10
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-05 14:34:49 -08:00
Chris Sherwood
6783cda222 feat(GPU): warn when GPU passthrough not working and offer one-click fix
Ollama can silently run on CPU even when the host has an NVIDIA GPU,
resulting in ~3 tok/s instead of ~167 tok/s. This happens when Ollama
was installed before the GPU toolkit, or when the container was
recreated without proper DeviceRequests. Users had zero indication.

Adds a GPU health check to the system info API response that detects
when the host has an NVIDIA runtime but nvidia-smi fails inside the
Ollama container. Shows a warning banner on the System Information
and AI Settings pages with a one-click "Reinstall AI Assistant"
button that force-reinstalls Ollama with GPU passthrough.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 14:34:28 -08:00
Jake Turner
175d63da8b
fix(AI): allow force refresh of models list 2026-03-05 22:31:24 +00:00
56 changed files with 2404 additions and 244 deletions

View File

@ -4,7 +4,7 @@ on:
workflow_dispatch:
inputs:
version:
description: 'Semantic version to label the Docker image under'
description: 'Semantic version to label the Docker image under (no "v" prefix, e.g. "1.2.3")'
required: true
type: string
tag_latest:
@ -46,4 +46,9 @@ jobs:
push: true
tags: |
ghcr.io/crosstalk-solutions/project-nomad:${{ inputs.version }}
ghcr.io/crosstalk-solutions/project-nomad:v${{ inputs.version }}
${{ inputs.tag_latest && 'ghcr.io/crosstalk-solutions/project-nomad:latest' || '' }}
build-args: |
VERSION=${{ inputs.version }}
BUILD_DATE=${{ github.event.workflow_run.created_at }}
VCS_REF=${{ github.sha }}

View File

@ -24,6 +24,21 @@ RUN node ace build
# Production stage
FROM base
ARG VERSION=dev
ARG BUILD_DATE
ARG VCS_REF
# Labels
LABEL org.opencontainers.image.title="Project N.O.M.A.D" \
org.opencontainers.image.description="The Project N.O.M.A.D Official Docker image" \
org.opencontainers.image.version="${VERSION}" \
org.opencontainers.image.created="${BUILD_DATE}" \
org.opencontainers.image.revision="${VCS_REF}" \
org.opencontainers.image.vendor="Crosstalk Solutions, LLC" \
org.opencontainers.image.documentation="https://github.com/CrosstalkSolutions/project-nomad/blob/main/README.md" \
org.opencontainers.image.source="https://github.com/CrosstalkSolutions/project-nomad" \
org.opencontainers.image.licenses="Apache-2.0"
ENV NODE_ENV=production
WORKDIR /app
COPY --from=production-deps /app/node_modules /app/node_modules

View File

@ -82,6 +82,8 @@ To run LLM's and other included AI tools:
- OS: Debian-based (Ubuntu recommended)
- Stable internet connection (required during install only)
**For detailed build recommendations at three price points ($200$800+), see the [Hardware Guide](https://www.projectnomad.us/hardware).**
Again, Project N.O.M.A.D. itself is quite lightweight - it's the tools and resources you choose to install with N.O.M.A.D. that will determine the specs required for your unique deployment
## About Internet Usage & Privacy
@ -136,6 +138,10 @@ Use the format `- **Area**: Description` to stay consistent with existing entrie
- **Discord:** [Join the Community](https://discord.com/invite/crosstalksolutions) - Get help, share your builds, and connect with other NOMAD users
- **Benchmark Leaderboard:** [benchmark.projectnomad.us](https://benchmark.projectnomad.us) - See how your hardware stacks up against other NOMAD builds
## License
Project N.O.M.A.D. is licensed under the [Apache License 2.0](LICENSE).
## Helper Scripts
Once installed, Project N.O.M.A.D. has a few helper scripts should you ever need to troubleshoot issues or perform maintenance that can't be done through the Command Center. All of these scripts are found in Project N.O.M.A.D.'s install directory, `/opt/project-nomad`

View File

@ -1,5 +1,6 @@
import { CollectionUpdateService } from '#services/collection_update_service'
import {
assertNotPrivateUrl,
applyContentUpdateValidator,
applyAllContentUpdatesValidator,
} from '#validators/common'
@ -13,12 +14,16 @@ export default class CollectionUpdatesController {
async applyUpdate({ request }: HttpContext) {
const update = await request.validateUsing(applyContentUpdateValidator)
assertNotPrivateUrl(update.download_url)
const service = new CollectionUpdateService()
return await service.applyUpdate(update)
}
async applyAllUpdates({ request }: HttpContext) {
const { updates } = await request.validateUsing(applyAllContentUpdatesValidator)
for (const update of updates) {
assertNotPrivateUrl(update.download_url)
}
const service = new CollectionUpdateService()
return await service.applyAllUpdates(updates)
}

View File

@ -1,5 +1,6 @@
import { MapService } from '#services/map_service'
import {
assertNotPrivateUrl,
downloadCollectionValidator,
filenameParamValidator,
remoteDownloadValidator,
@ -25,12 +26,14 @@ export default class MapsController {
async downloadBaseAssets({ request }: HttpContext) {
const payload = await request.validateUsing(remoteDownloadValidatorOptional)
if (payload.url) assertNotPrivateUrl(payload.url)
await this.mapService.downloadBaseAssets(payload.url)
return { success: true }
}
async downloadRemote({ request }: HttpContext) {
const payload = await request.validateUsing(remoteDownloadValidator)
assertNotPrivateUrl(payload.url)
const filename = await this.mapService.downloadRemote(payload.url)
return {
message: 'Download started successfully',
@ -52,6 +55,7 @@ export default class MapsController {
// For providing a "preflight" check in the UI before actually starting a background download
async downloadRemotePreflight({ request }: HttpContext) {
const payload = await request.validateUsing(remoteDownloadValidator)
assertNotPrivateUrl(payload.url)
const info = await this.mapService.downloadRemotePreflight(payload.url)
return info
}

View File

@ -1,16 +1,18 @@
import { ChatService } from '#services/chat_service'
import { OllamaService } from '#services/ollama_service'
import { RagService } from '#services/rag_service'
import { modelNameSchema } from '#validators/download'
import { chatSchema, getAvailableModelsSchema } from '#validators/ollama'
import { inject } from '@adonisjs/core'
import type { HttpContext } from '@adonisjs/core/http'
import { DEFAULT_QUERY_REWRITE_MODEL, SYSTEM_PROMPTS } from '../../constants/ollama.js'
import { DEFAULT_QUERY_REWRITE_MODEL, RAG_CONTEXT_LIMITS, SYSTEM_PROMPTS } from '../../constants/ollama.js'
import logger from '@adonisjs/core/services/logger'
import type { Message } from 'ollama'
@inject()
export default class OllamaController {
constructor(
private chatService: ChatService,
private ollamaService: OllamaService,
private ragService: RagService
) { }
@ -22,6 +24,7 @@ export default class OllamaController {
recommendedOnly: reqData.recommendedOnly,
query: reqData.query || null,
limit: reqData.limit || 15,
force: reqData.force,
})
}
@ -63,9 +66,28 @@ export default class OllamaController {
logger.debug(`[RAG] Retrieved ${relevantDocs.length} relevant documents for query: "${rewrittenQuery}"`)
// If relevant context is found, inject as a system message
// If relevant context is found, inject as a system message with adaptive limits
if (relevantDocs.length > 0) {
const contextText = relevantDocs
// Determine context budget based on model size
const { maxResults, maxTokens } = this.getContextLimitsForModel(reqData.model)
let trimmedDocs = relevantDocs.slice(0, maxResults)
// Apply token cap if set (estimate ~4 chars per token)
// Always include the first (most relevant) result — the cap only gates subsequent results
if (maxTokens > 0) {
const charCap = maxTokens * 4
let totalChars = 0
trimmedDocs = trimmedDocs.filter((doc, idx) => {
totalChars += doc.text.length
return idx === 0 || totalChars <= charCap
})
}
logger.debug(
`[RAG] Injecting ${trimmedDocs.length}/${relevantDocs.length} results (model: ${reqData.model}, maxResults: ${maxResults}, maxTokens: ${maxTokens || 'unlimited'})`
)
const contextText = trimmedDocs
.map((doc, idx) => `[Context ${idx + 1}] (Relevance: ${(doc.score * 100).toFixed(1)}%)\n${doc.text}`)
.join('\n\n')
@ -86,19 +108,59 @@ export default class OllamaController {
const thinkingCapability = await this.ollamaService.checkModelHasThinking(reqData.model)
const think: boolean | 'medium' = thinkingCapability ? (reqData.model.startsWith('gpt-oss') ? 'medium' : true) : false
// Separate sessionId from the Ollama request payload — Ollama rejects unknown fields
const { sessionId, ...ollamaRequest } = reqData
// Save user message to DB before streaming if sessionId provided
let userContent: string | null = null
if (sessionId) {
const lastUserMsg = [...reqData.messages].reverse().find((m) => m.role === 'user')
if (lastUserMsg) {
userContent = lastUserMsg.content
await this.chatService.addMessage(sessionId, 'user', userContent)
}
}
if (reqData.stream) {
logger.debug(`[OllamaController] Initiating streaming response for model: "${reqData.model}" with think: ${think}`)
// Headers already flushed above
const stream = await this.ollamaService.chatStream({ ...reqData, think })
const stream = await this.ollamaService.chatStream({ ...ollamaRequest, think })
let fullContent = ''
for await (const chunk of stream) {
if (chunk.message?.content) {
fullContent += chunk.message.content
}
response.response.write(`data: ${JSON.stringify(chunk)}\n\n`)
}
response.response.end()
// Save assistant message and optionally generate title
if (sessionId && fullContent) {
await this.chatService.addMessage(sessionId, 'assistant', fullContent)
const messageCount = await this.chatService.getMessageCount(sessionId)
if (messageCount <= 2 && userContent) {
this.chatService.generateTitle(sessionId, userContent, fullContent).catch((err) => {
logger.error(`[OllamaController] Title generation failed: ${err instanceof Error ? err.message : err}`)
})
}
}
return
}
// Non-streaming (legacy) path
return await this.ollamaService.chat({ ...reqData, think })
const result = await this.ollamaService.chat({ ...ollamaRequest, think })
if (sessionId && result?.message?.content) {
await this.chatService.addMessage(sessionId, 'assistant', result.message.content)
const messageCount = await this.chatService.getMessageCount(sessionId)
if (messageCount <= 2 && userContent) {
this.chatService.generateTitle(sessionId, userContent, result.message.content).catch((err) => {
logger.error(`[OllamaController] Title generation failed: ${err instanceof Error ? err.message : err}`)
})
}
}
return result
} catch (error) {
if (reqData.stream) {
response.response.write(`data: ${JSON.stringify({ error: true })}\n\n`)
@ -131,6 +193,25 @@ export default class OllamaController {
return await this.ollamaService.getModels()
}
/**
* Determines RAG context limits based on model size extracted from the model name.
* Parses size indicators like "1b", "3b", "8b", "70b" from model names/tags.
*/
private getContextLimitsForModel(modelName: string): { maxResults: number; maxTokens: number } {
// Extract parameter count from model name (e.g., "llama3.2:3b", "qwen2.5:1.5b", "gemma:7b")
const sizeMatch = modelName.match(/(\d+\.?\d*)[bB]/)
const paramBillions = sizeMatch ? parseFloat(sizeMatch[1]) : 8 // default to 8B if unknown
for (const tier of RAG_CONTEXT_LIMITS) {
if (paramBillions <= tier.maxParams) {
return { maxResults: tier.maxResults, maxTokens: tier.maxTokens }
}
}
// Fallback: no limits
return { maxResults: 5, maxTokens: 0 }
}
private async rewriteQueryWithContext(
messages: Message[]
): Promise<string | null> {
@ -156,8 +237,8 @@ export default class OllamaController {
})
.join('\n')
const availableModels = await this.ollamaService.getAvailableModels({ query: null, limit: 500 })
const rewriteModelAvailable = availableModels?.models.some(model => model.name === DEFAULT_QUERY_REWRITE_MODEL)
const installedModels = await this.ollamaService.getModels(true)
const rewriteModelAvailable = installedModels?.some(model => model.name === DEFAULT_QUERY_REWRITE_MODEL)
if (!rewriteModelAvailable) {
logger.warn(`[RAG] Query rewrite model "${DEFAULT_QUERY_REWRITE_MODEL}" not available. Skipping query rewriting.`)
const lastUserMessage = [...messages].reverse().find(msg => msg.role === 'user')

View File

@ -1,7 +1,9 @@
import { DockerService } from '#services/docker_service';
import { SystemService } from '#services/system_service'
import { SystemUpdateService } from '#services/system_update_service'
import { affectServiceValidator, checkLatestVersionValidator, installServiceValidator, subscribeToReleaseNotesValidator } from '#validators/system';
import { ContainerRegistryService } from '#services/container_registry_service'
import { CheckServiceUpdatesJob } from '#jobs/check_service_updates_job'
import { affectServiceValidator, checkLatestVersionValidator, installServiceValidator, subscribeToReleaseNotesValidator, updateServiceValidator } from '#validators/system';
import { inject } from '@adonisjs/core'
import type { HttpContext } from '@adonisjs/core/http'
@ -10,7 +12,8 @@ export default class SystemController {
constructor(
private systemService: SystemService,
private dockerService: DockerService,
private systemUpdateService: SystemUpdateService
private systemUpdateService: SystemUpdateService,
private containerRegistryService: ContainerRegistryService
) { }
async getInternetStatus({ }: HttpContext) {
@ -104,9 +107,70 @@ export default class SystemController {
response.send({ logs });
}
async subscribeToReleaseNotes({ request }: HttpContext) {
const reqData = await request.validateUsing(subscribeToReleaseNotesValidator);
return await this.systemService.subscribeToReleaseNotes(reqData.email);
}
async checkServiceUpdates({ response }: HttpContext) {
await CheckServiceUpdatesJob.dispatch()
response.send({ success: true, message: 'Service update check dispatched' })
}
async getAvailableVersions({ params, response }: HttpContext) {
const serviceName = params.name
const service = await (await import('#models/service')).default
.query()
.where('service_name', serviceName)
.where('installed', true)
.first()
if (!service) {
return response.status(404).send({ error: `Service ${serviceName} not found or not installed` })
}
try {
const hostArch = await this.getHostArch()
const updates = await this.containerRegistryService.getAvailableUpdates(
service.container_image,
hostArch,
service.source_repo
)
response.send({ versions: updates })
} catch (error) {
response.status(500).send({ error: `Failed to fetch versions: ${error.message}` })
}
}
async updateService({ request, response }: HttpContext) {
const payload = await request.validateUsing(updateServiceValidator)
const result = await this.dockerService.updateContainer(
payload.service_name,
payload.target_version
)
if (result.success) {
response.send({ success: true, message: result.message })
} else {
response.status(400).send({ error: result.message })
}
}
private async getHostArch(): Promise<string> {
try {
const info = await this.dockerService.docker.info()
const arch = info.Architecture || ''
const archMap: Record<string, string> = {
x86_64: 'amd64',
aarch64: 'arm64',
armv7l: 'arm',
amd64: 'amd64',
arm64: 'arm64',
}
return archMap[arch] || arch.toLowerCase()
} catch {
return 'amd64'
}
}
}

View File

@ -1,5 +1,6 @@
import { ZimService } from '#services/zim_service'
import {
assertNotPrivateUrl,
downloadCategoryTierValidator,
filenameParamValidator,
remoteDownloadWithMetadataValidator,
@ -25,6 +26,7 @@ export default class ZimController {
async downloadRemote({ request }: HttpContext) {
const payload = await request.validateUsing(remoteDownloadWithMetadataValidator)
assertNotPrivateUrl(payload.url)
const { filename, jobId } = await this.zimService.downloadRemote(payload.url)
return {

View File

@ -0,0 +1,134 @@
import { Job } from 'bullmq'
import { QueueService } from '#services/queue_service'
import { DockerService } from '#services/docker_service'
import { ContainerRegistryService } from '#services/container_registry_service'
import Service from '#models/service'
import logger from '@adonisjs/core/services/logger'
import transmit from '@adonisjs/transmit/services/main'
import { BROADCAST_CHANNELS } from '../../constants/broadcast.js'
import { DateTime } from 'luxon'
export class CheckServiceUpdatesJob {
static get queue() {
return 'service-updates'
}
static get key() {
return 'check-service-updates'
}
async handle(_job: Job) {
logger.info('[CheckServiceUpdatesJob] Checking for service updates...')
const dockerService = new DockerService()
const registryService = new ContainerRegistryService()
// Determine host architecture
const hostArch = await this.getHostArch(dockerService)
const installedServices = await Service.query().where('installed', true)
let updatesFound = 0
for (const service of installedServices) {
try {
const updates = await registryService.getAvailableUpdates(
service.container_image,
hostArch,
service.source_repo
)
const latestUpdate = updates.length > 0 ? updates[0].tag : null
service.available_update_version = latestUpdate
service.update_checked_at = DateTime.now()
await service.save()
if (latestUpdate) {
updatesFound++
logger.info(
`[CheckServiceUpdatesJob] Update available for ${service.service_name}: ${service.container_image}${latestUpdate}`
)
}
} catch (error) {
logger.error(
`[CheckServiceUpdatesJob] Failed to check updates for ${service.service_name}: ${error.message}`
)
// Continue checking other services
}
}
logger.info(
`[CheckServiceUpdatesJob] Completed. ${updatesFound} update(s) found for ${installedServices.length} service(s).`
)
// Broadcast completion so the frontend can refresh
transmit.broadcast(BROADCAST_CHANNELS.SERVICE_UPDATES, {
status: 'completed',
updatesFound,
timestamp: new Date().toISOString(),
})
return { updatesFound }
}
private async getHostArch(dockerService: DockerService): Promise<string> {
try {
const info = await dockerService.docker.info()
const arch = info.Architecture || ''
// Map Docker architecture names to OCI names
const archMap: Record<string, string> = {
x86_64: 'amd64',
aarch64: 'arm64',
armv7l: 'arm',
amd64: 'amd64',
arm64: 'arm64',
}
return archMap[arch] || arch.toLowerCase()
} catch (error) {
logger.warn(
`[CheckServiceUpdatesJob] Could not detect host architecture: ${error.message}. Defaulting to amd64.`
)
return 'amd64'
}
}
static async scheduleNightly() {
const queueService = new QueueService()
const queue = queueService.getQueue(this.queue)
await queue.upsertJobScheduler(
'nightly-service-update-check',
{ pattern: '0 3 * * *' },
{
name: this.key,
opts: {
removeOnComplete: { count: 7 },
removeOnFail: { count: 5 },
},
}
)
logger.info('[CheckServiceUpdatesJob] Service update check scheduled with cron: 0 3 * * *')
}
static async dispatch() {
const queueService = new QueueService()
const queue = queueService.getQueue(this.queue)
const job = await queue.add(
this.key,
{},
{
attempts: 3,
backoff: { type: 'exponential', delay: 60000 },
removeOnComplete: { count: 7 },
removeOnFail: { count: 5 },
}
)
logger.info(`[CheckServiceUpdatesJob] Dispatched ad-hoc service update check job ${job.id}`)
return job
}
}

View File

@ -62,6 +62,15 @@ export default class Service extends BaseModel {
@column()
declare metadata: string | null
@column()
declare source_repo: string | null
@column()
declare available_update_version: string | null
@column.dateTime()
declare update_checked_at: DateTime | null
@column.dateTime({ autoCreate: true })
declare created_at: DateTime

View File

@ -4,7 +4,7 @@ import logger from '@adonisjs/core/services/logger'
import { DateTime } from 'luxon'
import { inject } from '@adonisjs/core'
import { OllamaService } from './ollama_service.js'
import { SYSTEM_PROMPTS } from '../../constants/ollama.js'
import { DEFAULT_QUERY_REWRITE_MODEL, SYSTEM_PROMPTS } from '../../constants/ollama.js'
import { toTitleCase } from '../utils/misc.js'
@inject()
@ -220,6 +220,59 @@ export class ChatService {
}
}
async getMessageCount(sessionId: number): Promise<number> {
try {
const count = await ChatMessage.query().where('session_id', sessionId).count('* as total')
return Number(count[0].$extras.total)
} catch (error) {
logger.error(
`[ChatService] Failed to get message count for session ${sessionId}: ${error instanceof Error ? error.message : error}`
)
return 0
}
}
async generateTitle(sessionId: number, userMessage: string, assistantMessage: string) {
try {
const models = await this.ollamaService.getModels()
const titleModelAvailable = models?.some((m) => m.name === DEFAULT_QUERY_REWRITE_MODEL)
let title: string
if (!titleModelAvailable) {
title = userMessage.slice(0, 57) + (userMessage.length > 57 ? '...' : '')
} else {
const response = await this.ollamaService.chat({
model: DEFAULT_QUERY_REWRITE_MODEL,
messages: [
{ role: 'system', content: SYSTEM_PROMPTS.title_generation },
{ role: 'user', content: userMessage },
{ role: 'assistant', content: assistantMessage },
],
})
title = response?.message?.content?.trim()
if (!title) {
title = userMessage.slice(0, 57) + (userMessage.length > 57 ? '...' : '')
}
}
await this.updateSession(sessionId, { title })
logger.info(`[ChatService] Generated title for session ${sessionId}: "${title}"`)
} catch (error) {
logger.error(
`[ChatService] Failed to generate title for session ${sessionId}: ${error instanceof Error ? error.message : error}`
)
// Fall back to truncated user message
try {
const fallbackTitle = userMessage.slice(0, 57) + (userMessage.length > 57 ? '...' : '')
await this.updateSession(sessionId, { title: fallbackTitle })
} catch {
// Silently fail - session keeps "New Chat" title
}
}
}
async deleteAllSessions() {
try {
await ChatSession.query().delete()

View File

@ -0,0 +1,484 @@
import logger from '@adonisjs/core/services/logger'
import { isNewerVersion, parseMajorVersion } from '../utils/version.js'
export interface ParsedImageReference {
registry: string
namespace: string
repo: string
tag: string
/** Full name for registry API calls: namespace/repo */
fullName: string
}
export interface AvailableUpdate {
tag: string
isLatest: boolean
releaseUrl?: string
}
interface TokenCacheEntry {
token: string
expiresAt: number
}
const SEMVER_TAG_PATTERN = /^v?(\d+\.\d+(?:\.\d+)?)$/
const PLATFORM_SUFFIXES = ['-arm64', '-amd64', '-alpine', '-slim', '-cuda', '-rocm']
const REJECTED_TAGS = new Set(['latest', 'nightly', 'edge', 'dev', 'beta', 'alpha', 'canary', 'rc', 'test', 'debug'])
export class ContainerRegistryService {
private tokenCache = new Map<string, TokenCacheEntry>()
private sourceUrlCache = new Map<string, string | null>()
private releaseTagPrefixCache = new Map<string, string>()
/**
* Parse a Docker image reference string into its components.
*/
parseImageReference(image: string): ParsedImageReference {
let registry: string
let remainder: string
let tag = 'latest'
// Split off the tag
const lastColon = image.lastIndexOf(':')
if (lastColon > -1 && !image.substring(lastColon).includes('/')) {
tag = image.substring(lastColon + 1)
image = image.substring(0, lastColon)
}
// Determine registry vs image path
const parts = image.split('/')
if (parts.length === 1) {
// e.g. "nginx" → Docker Hub library image
registry = 'registry-1.docker.io'
remainder = `library/${parts[0]}`
} else if (parts.length === 2 && !parts[0].includes('.') && !parts[0].includes(':')) {
// e.g. "ollama/ollama" → Docker Hub user image
registry = 'registry-1.docker.io'
remainder = image
} else {
// e.g. "ghcr.io/kiwix/kiwix-serve" → custom registry
registry = parts[0]
remainder = parts.slice(1).join('/')
}
const namespaceParts = remainder.split('/')
const repo = namespaceParts.pop()!
const namespace = namespaceParts.join('/')
return {
registry,
namespace,
repo,
tag,
fullName: remainder,
}
}
/**
* Get an anonymous auth token for the given registry and repository.
* NOTE: This could be expanded in the future to support private repo authentication
*/
private async getToken(registry: string, fullName: string): Promise<string> {
const cacheKey = `${registry}/${fullName}`
const cached = this.tokenCache.get(cacheKey)
if (cached && cached.expiresAt > Date.now()) {
return cached.token
}
let tokenUrl: string
if (registry === 'registry-1.docker.io') {
tokenUrl = `https://auth.docker.io/token?service=registry.docker.io&scope=repository:${fullName}:pull`
} else if (registry === 'ghcr.io') {
tokenUrl = `https://ghcr.io/token?service=ghcr.io&scope=repository:${fullName}:pull`
} else {
// For other registries, try the standard v2 token endpoint
tokenUrl = `https://${registry}/token?service=${registry}&scope=repository:${fullName}:pull`
}
const response = await this.fetchWithRetry(tokenUrl)
if (!response.ok) {
throw new Error(`Failed to get auth token from ${registry}: ${response.status}`)
}
const data = (await response.json()) as { token?: string; access_token?: string }
const token = data.token || data.access_token || ''
if (!token) {
throw new Error(`No token returned from ${registry}`)
}
// Cache for 5 minutes (tokens usually last longer, but be conservative)
this.tokenCache.set(cacheKey, {
token,
expiresAt: Date.now() + 5 * 60 * 1000,
})
return token
}
/**
* List all tags for a given image from the registry.
*/
async listTags(parsed: ParsedImageReference): Promise<string[]> {
const token = await this.getToken(parsed.registry, parsed.fullName)
const allTags: string[] = []
let url = `https://${parsed.registry}/v2/${parsed.fullName}/tags/list?n=1000`
while (url) {
const response = await this.fetchWithRetry(url, {
headers: { Authorization: `Bearer ${token}` },
})
if (!response.ok) {
throw new Error(`Failed to list tags for ${parsed.fullName}: ${response.status}`)
}
const data = (await response.json()) as { tags?: string[] }
if (data.tags) {
allTags.push(...data.tags)
}
// Handle pagination via Link header
const linkHeader = response.headers.get('link')
if (linkHeader) {
const match = linkHeader.match(/<([^>]+)>;\s*rel="next"/)
url = match ? match[1] : ''
} else {
url = ''
}
}
return allTags
}
/**
* Check if a specific tag supports the given architecture by fetching its manifest.
*/
async checkArchSupport(parsed: ParsedImageReference, tag: string, hostArch: string): Promise<boolean> {
try {
const token = await this.getToken(parsed.registry, parsed.fullName)
const url = `https://${parsed.registry}/v2/${parsed.fullName}/manifests/${tag}`
const response = await this.fetchWithRetry(url, {
headers: {
Authorization: `Bearer ${token}`,
Accept: [
'application/vnd.oci.image.index.v1+json',
'application/vnd.docker.distribution.manifest.list.v2+json',
'application/vnd.oci.image.manifest.v1+json',
'application/vnd.docker.distribution.manifest.v2+json',
].join(', '),
},
})
if (!response.ok) return true // If we can't check, assume it's compatible
const manifest = (await response.json()) as {
mediaType?: string
manifests?: Array<{ platform?: { architecture?: string } }>
}
const mediaType = manifest.mediaType || response.headers.get('content-type') || ''
// Manifest list — check if any platform matches
if (
mediaType.includes('manifest.list') ||
mediaType.includes('image.index') ||
manifest.manifests
) {
const manifests = manifest.manifests || []
return manifests.some(
(m: any) => m.platform && m.platform.architecture === hostArch
)
}
// Single manifest — assume compatible (can't easily determine arch without fetching config blob)
return true
} catch (error) {
logger.warn(`[ContainerRegistryService] Error checking arch for ${tag}: ${error.message}`)
return true // Assume compatible on error
}
}
/**
* Extract the source repository URL from an image's OCI labels.
* Uses the standardized `org.opencontainers.image.source` label.
* Result is cached per image (not per tag).
*/
async getSourceUrl(parsed: ParsedImageReference): Promise<string | null> {
const cacheKey = `${parsed.registry}/${parsed.fullName}`
if (this.sourceUrlCache.has(cacheKey)) {
return this.sourceUrlCache.get(cacheKey)!
}
try {
const token = await this.getToken(parsed.registry, parsed.fullName)
// First get the manifest to find the config blob digest
const manifestUrl = `https://${parsed.registry}/v2/${parsed.fullName}/manifests/${parsed.tag}`
const manifestRes = await this.fetchWithRetry(manifestUrl, {
headers: {
Authorization: `Bearer ${token}`,
Accept: [
'application/vnd.oci.image.manifest.v1+json',
'application/vnd.docker.distribution.manifest.v2+json',
'application/vnd.oci.image.index.v1+json',
'application/vnd.docker.distribution.manifest.list.v2+json',
].join(', '),
},
})
if (!manifestRes.ok) {
this.sourceUrlCache.set(cacheKey, null)
return null
}
const manifest = (await manifestRes.json()) as {
config?: { digest?: string }
manifests?: Array<{ digest?: string; mediaType?: string; platform?: { architecture?: string } }>
}
// If this is a manifest list, pick the first manifest to get the config
let configDigest = manifest.config?.digest
if (!configDigest && manifest.manifests?.length) {
const firstManifest = manifest.manifests[0]
if (firstManifest.digest) {
const childRes = await this.fetchWithRetry(
`https://${parsed.registry}/v2/${parsed.fullName}/manifests/${firstManifest.digest}`,
{
headers: {
Authorization: `Bearer ${token}`,
Accept: 'application/vnd.oci.image.manifest.v1+json, application/vnd.docker.distribution.manifest.v2+json',
},
}
)
if (childRes.ok) {
const childManifest = (await childRes.json()) as { config?: { digest?: string } }
configDigest = childManifest.config?.digest
}
}
}
if (!configDigest) {
this.sourceUrlCache.set(cacheKey, null)
return null
}
// Fetch the config blob to read labels
const blobUrl = `https://${parsed.registry}/v2/${parsed.fullName}/blobs/${configDigest}`
const blobRes = await this.fetchWithRetry(blobUrl, {
headers: { Authorization: `Bearer ${token}` },
})
if (!blobRes.ok) {
this.sourceUrlCache.set(cacheKey, null)
return null
}
const config = (await blobRes.json()) as {
config?: { Labels?: Record<string, string> }
}
const sourceUrl = config.config?.Labels?.['org.opencontainers.image.source'] || null
this.sourceUrlCache.set(cacheKey, sourceUrl)
return sourceUrl
} catch (error) {
logger.warn(`[ContainerRegistryService] Failed to get source URL for ${cacheKey}: ${error.message}`)
this.sourceUrlCache.set(cacheKey, null)
return null
}
}
/**
* Detect whether a GitHub/GitLab repo uses a 'v' prefix on release tags.
* Probes the GitHub API with the current tag to determine the convention,
* then caches the result per source URL.
*/
async detectReleaseTagPrefix(sourceUrl: string, sampleTag: string): Promise<string> {
if (this.releaseTagPrefixCache.has(sourceUrl)) {
return this.releaseTagPrefixCache.get(sourceUrl)!
}
try {
const url = new URL(sourceUrl)
if (url.hostname !== 'github.com') {
this.releaseTagPrefixCache.set(sourceUrl, '')
return ''
}
const cleanPath = url.pathname.replace(/\.git$/, '').replace(/\/$/, '')
const strippedTag = sampleTag.replace(/^v/, '')
const vTag = `v${strippedTag}`
// Try both variants against GitHub's API — the one that 200s tells us the convention
// Try v-prefixed first since it's more common
const vRes = await this.fetchWithRetry(
`https://api.github.com/repos${cleanPath}/releases/tags/${vTag}`,
{ headers: { Accept: 'application/vnd.github.v3+json', 'User-Agent': 'ProjectNomad' } },
1
)
if (vRes.ok) {
this.releaseTagPrefixCache.set(sourceUrl, 'v')
return 'v'
}
const plainRes = await this.fetchWithRetry(
`https://api.github.com/repos${cleanPath}/releases/tags/${strippedTag}`,
{ headers: { Accept: 'application/vnd.github.v3+json', 'User-Agent': 'ProjectNomad' } },
1
)
if (plainRes.ok) {
this.releaseTagPrefixCache.set(sourceUrl, '')
return ''
}
} catch {
// On error, fall through to default
}
// Default: no prefix modification
this.releaseTagPrefixCache.set(sourceUrl, '')
return ''
}
/**
* Build a release URL for a specific tag given a source repository URL and
* the detected release tag prefix convention.
* Supports GitHub and GitLab URL patterns.
*/
buildReleaseUrl(sourceUrl: string, tag: string, releaseTagPrefix: string): string | undefined {
try {
const url = new URL(sourceUrl)
if (url.hostname === 'github.com' || url.hostname.includes('gitlab')) {
const cleanPath = url.pathname.replace(/\.git$/, '').replace(/\/$/, '')
const strippedTag = tag.replace(/^v/, '')
const releaseTag = releaseTagPrefix ? `${releaseTagPrefix}${strippedTag}` : strippedTag
return `${url.origin}${cleanPath}/releases/tag/${releaseTag}`
}
} catch {
// Invalid URL, skip
}
return undefined
}
/**
* Filter and sort tags to find compatible updates for a service.
*/
filterCompatibleUpdates(
tags: string[],
currentTag: string,
majorVersion: number
): string[] {
return tags
.filter((tag) => {
// Must match semver pattern
if (!SEMVER_TAG_PATTERN.test(tag)) return false
// Reject known non-version tags
if (REJECTED_TAGS.has(tag.toLowerCase())) return false
// Reject platform suffixes
if (PLATFORM_SUFFIXES.some((suffix) => tag.toLowerCase().endsWith(suffix))) return false
// Must be same major version
if (parseMajorVersion(tag) !== majorVersion) return false
// Must be newer than current
return isNewerVersion(tag, currentTag)
})
.sort((a, b) => (isNewerVersion(a, b) ? -1 : 1)) // Newest first
}
/**
* High-level method to get available updates for a service.
* Returns a sorted list of compatible newer versions (newest first).
*/
async getAvailableUpdates(
containerImage: string,
hostArch: string,
fallbackSourceRepo?: string | null
): Promise<AvailableUpdate[]> {
const parsed = this.parseImageReference(containerImage)
const currentTag = parsed.tag
if (currentTag === 'latest') {
logger.warn(
`[ContainerRegistryService] Cannot check updates for ${containerImage} — using :latest tag`
)
return []
}
const majorVersion = parseMajorVersion(currentTag)
// Fetch tags and source URL in parallel
const [tags, ociSourceUrl] = await Promise.all([
this.listTags(parsed),
this.getSourceUrl(parsed),
])
// OCI label takes precedence, fall back to DB-stored source_repo
const sourceUrl = ociSourceUrl || fallbackSourceRepo || null
const compatible = this.filterCompatibleUpdates(tags, currentTag, majorVersion)
// Detect release tag prefix convention (e.g. 'v' vs no prefix) if we have a source URL
let releaseTagPrefix = ''
if (sourceUrl) {
releaseTagPrefix = await this.detectReleaseTagPrefix(sourceUrl, currentTag)
}
// Check architecture support for the top candidates (limit checks to save API calls)
const maxArchChecks = 10
const results: AvailableUpdate[] = []
for (const tag of compatible.slice(0, maxArchChecks)) {
const supported = await this.checkArchSupport(parsed, tag, hostArch)
if (supported) {
results.push({
tag,
isLatest: results.length === 0,
releaseUrl: sourceUrl ? this.buildReleaseUrl(sourceUrl, tag, releaseTagPrefix) : undefined,
})
}
}
// For remaining tags (beyond arch check limit), include them but mark as not latest
for (const tag of compatible.slice(maxArchChecks)) {
results.push({
tag,
isLatest: false,
releaseUrl: sourceUrl ? this.buildReleaseUrl(sourceUrl, tag, releaseTagPrefix) : undefined,
})
}
return results
}
/**
* Fetch with retry and exponential backoff for rate limiting.
*/
private async fetchWithRetry(
url: string,
init?: RequestInit,
maxRetries = 3
): Promise<Response> {
for (let attempt = 0; attempt <= maxRetries; attempt++) {
const response = await fetch(url, init)
if (response.status === 429 && attempt < maxRetries) {
const retryAfter = response.headers.get('retry-after')
const delay = retryAfter
? parseInt(retryAfter, 10) * 1000
: Math.pow(2, attempt) * 1000
logger.warn(
`[ContainerRegistryService] Rate limited on ${url}, retrying in ${delay}ms`
)
await new Promise((resolve) => setTimeout(resolve, delay))
continue
}
return response
}
throw new Error(`Failed to fetch ${url} after ${maxRetries} retries`)
}
}

View File

@ -113,8 +113,8 @@ export class DockerService {
const containers = await this.docker.listContainers({ all: true })
const containerMap = new Map<string, Docker.ContainerInfo>()
containers.forEach((container) => {
const name = container.Names[0].replace('/', '')
if (name.startsWith('nomad_')) {
const name = container.Names[0]?.replace('/', '')
if (name && name.startsWith('nomad_')) {
containerMap.set(name, container)
}
})
@ -792,6 +792,186 @@ export class DockerService {
// }
// }
/**
* Update a service container to a new image version while preserving volumes and data.
* Includes automatic rollback if the new container fails health checks.
*/
async updateContainer(
serviceName: string,
targetVersion: string
): Promise<{ success: boolean; message: string }> {
try {
const service = await Service.query().where('service_name', serviceName).first()
if (!service) {
return { success: false, message: `Service ${serviceName} not found` }
}
if (!service.installed) {
return { success: false, message: `Service ${serviceName} is not installed` }
}
if (this.activeInstallations.has(serviceName)) {
return { success: false, message: `Service ${serviceName} already has an operation in progress` }
}
this.activeInstallations.add(serviceName)
// Compute new image string
const currentImage = service.container_image
const imageBase = currentImage.includes(':')
? currentImage.substring(0, currentImage.lastIndexOf(':'))
: currentImage
const newImage = `${imageBase}:${targetVersion}`
// Step 1: Pull new image
this._broadcast(serviceName, 'update-pulling', `Pulling image ${newImage}...`)
const pullStream = await this.docker.pull(newImage)
await new Promise((res) => this.docker.modem.followProgress(pullStream, res))
// Step 2: Find and stop existing container
this._broadcast(serviceName, 'update-stopping', `Stopping current container...`)
const containers = await this.docker.listContainers({ all: true })
const existingContainer = containers.find((c) => c.Names.includes(`/${serviceName}`))
if (!existingContainer) {
this.activeInstallations.delete(serviceName)
return { success: false, message: `Container for ${serviceName} not found` }
}
const oldContainer = this.docker.getContainer(existingContainer.Id)
// Inspect to capture full config before stopping
const inspectData = await oldContainer.inspect()
if (existingContainer.State === 'running') {
await oldContainer.stop({ t: 15 })
}
// Step 3: Rename old container as safety net
const oldName = `${serviceName}_old`
await oldContainer.rename({ name: oldName })
// Step 4: Create new container with inspected config + new image
this._broadcast(serviceName, 'update-creating', `Creating updated container...`)
const hostConfig = inspectData.HostConfig || {}
const newContainerConfig: any = {
Image: newImage,
name: serviceName,
Env: inspectData.Config?.Env || undefined,
Cmd: inspectData.Config?.Cmd || undefined,
ExposedPorts: inspectData.Config?.ExposedPorts || undefined,
WorkingDir: inspectData.Config?.WorkingDir || undefined,
User: inspectData.Config?.User || undefined,
HostConfig: {
Binds: hostConfig.Binds || undefined,
PortBindings: hostConfig.PortBindings || undefined,
RestartPolicy: hostConfig.RestartPolicy || undefined,
DeviceRequests: hostConfig.DeviceRequests || undefined,
Devices: hostConfig.Devices || undefined,
},
NetworkingConfig: inspectData.NetworkSettings?.Networks
? {
EndpointsConfig: Object.fromEntries(
Object.keys(inspectData.NetworkSettings.Networks).map((net) => [net, {}])
),
}
: undefined,
}
// Remove undefined values from HostConfig
Object.keys(newContainerConfig.HostConfig).forEach((key) => {
if (newContainerConfig.HostConfig[key] === undefined) {
delete newContainerConfig.HostConfig[key]
}
})
let newContainer: any
try {
newContainer = await this.docker.createContainer(newContainerConfig)
} catch (createError) {
// Rollback: rename old container back
this._broadcast(serviceName, 'update-rollback', `Failed to create new container: ${createError.message}. Rolling back...`)
const rollbackContainer = this.docker.getContainer((await this.docker.listContainers({ all: true })).find((c) => c.Names.includes(`/${oldName}`))!.Id)
await rollbackContainer.rename({ name: serviceName })
await rollbackContainer.start()
this.activeInstallations.delete(serviceName)
return { success: false, message: `Failed to create updated container: ${createError.message}` }
}
// Step 5: Start new container
this._broadcast(serviceName, 'update-starting', `Starting updated container...`)
await newContainer.start()
// Step 6: Health check — verify container stays running for 5 seconds
await new Promise((resolve) => setTimeout(resolve, 5000))
const newContainerInfo = await newContainer.inspect()
if (newContainerInfo.State?.Running) {
// Healthy — clean up old container
try {
const oldContainerRef = this.docker.getContainer(
(await this.docker.listContainers({ all: true })).find((c) =>
c.Names.includes(`/${oldName}`)
)?.Id || ''
)
await oldContainerRef.remove({ force: true })
} catch {
// Old container may already be gone
}
// Update DB
service.container_image = newImage
service.available_update_version = null
await service.save()
this.activeInstallations.delete(serviceName)
this._broadcast(
serviceName,
'update-complete',
`Successfully updated ${serviceName} to ${targetVersion}`
)
return { success: true, message: `Service ${serviceName} updated to ${targetVersion}` }
} else {
// Unhealthy — rollback
this._broadcast(
serviceName,
'update-rollback',
`New container failed health check. Rolling back to previous version...`
)
try {
await newContainer.stop({ t: 5 }).catch(() => {})
await newContainer.remove({ force: true })
} catch {
// Best effort cleanup
}
// Restore old container
const oldContainers = await this.docker.listContainers({ all: true })
const oldRef = oldContainers.find((c) => c.Names.includes(`/${oldName}`))
if (oldRef) {
const rollbackContainer = this.docker.getContainer(oldRef.Id)
await rollbackContainer.rename({ name: serviceName })
await rollbackContainer.start()
}
this.activeInstallations.delete(serviceName)
return {
success: false,
message: `Update failed: new container did not stay running. Rolled back to previous version.`,
}
}
} catch (error) {
this.activeInstallations.delete(serviceName)
this._broadcast(
serviceName,
'update-rollback',
`Update failed: ${error.message}`
)
logger.error(`[DockerService] Update failed for ${serviceName}: ${error.message}`)
return { success: false, message: `Update failed: ${error.message}` }
}
}
private _broadcast(service: string, status: string, message: string) {
transmit.broadcast(BROADCAST_CHANNELS.SERVICE_INSTALLATION, {
service_name: service,

View File

@ -66,12 +66,19 @@ export class DocsService {
const filename = _filename.endsWith('.md') ? _filename : `${_filename}.md`
const fileExists = await getFileStatsIfExists(path.join(this.docsPath, filename))
// Prevent path traversal — resolved path must stay within the docs directory
const basePath = path.resolve(this.docsPath)
const fullPath = path.resolve(path.join(this.docsPath, filename))
if (!fullPath.startsWith(basePath + path.sep)) {
throw new Error('Invalid document slug')
}
const fileExists = await getFileStatsIfExists(fullPath)
if (!fileExists) {
throw new Error(`File not found: ${filename}`)
}
const fileStream = await getFile(path.join(this.docsPath, filename), 'stream')
const fileStream = await getFile(fullPath, 'stream')
if (!fileStream) {
throw new Error(`Failed to read file stream: ${filename}`)
}

View File

@ -13,7 +13,7 @@ import {
getFile,
ensureDirectoryExists,
} from '../utils/fs.js'
import { join } from 'path'
import { join, resolve, sep } from 'path'
import urlJoin from 'url-join'
import { RunDownloadJob } from '#jobs/run_download_job'
import logger from '@adonisjs/core/services/logger'
@ -404,7 +404,13 @@ export class MapService implements IMapService {
fileName += '.pmtiles'
}
const fullPath = join(this.baseDirPath, 'pmtiles', fileName)
const basePath = resolve(join(this.baseDirPath, 'pmtiles'))
const fullPath = resolve(join(basePath, fileName))
// Prevent path traversal — resolved path must stay within the storage directory
if (!fullPath.startsWith(basePath + sep)) {
throw new Error('Invalid filename')
}
const exists = await getFileStatsIfExists(fullPath)
if (!exists) {

View File

@ -183,7 +183,7 @@ export class OllamaService {
}
async getAvailableModels(
{ sort, recommendedOnly, query, limit }: { sort?: 'pulls' | 'name'; recommendedOnly?: boolean, query: string | null, limit?: number } = {
{ sort, recommendedOnly, query, limit, force }: { sort?: 'pulls' | 'name'; recommendedOnly?: boolean, query: string | null, limit?: number, force?: boolean } = {
sort: 'pulls',
recommendedOnly: false,
query: null,
@ -191,7 +191,7 @@ export class OllamaService {
}
): Promise<{ models: NomadOllamaModel[], hasMore: boolean } | null> {
try {
const models = await this.retrieveAndRefreshModels(sort)
const models = await this.retrieveAndRefreshModels(sort, force)
if (!models) {
// If we fail to get models from the API, return the fallback recommended models
logger.warn(
@ -244,13 +244,18 @@ export class OllamaService {
}
private async retrieveAndRefreshModels(
sort?: 'pulls' | 'name'
sort?: 'pulls' | 'name',
force?: boolean
): Promise<NomadOllamaModel[] | null> {
try {
const cachedModels = await this.readModelsFromCache()
if (cachedModels) {
logger.info('[OllamaService] Using cached available models data')
return this.sortModels(cachedModels, sort)
if (!force) {
const cachedModels = await this.readModelsFromCache()
if (cachedModels) {
logger.info('[OllamaService] Using cached available models data')
return this.sortModels(cachedModels, sort)
}
} else {
logger.info('[OllamaService] Force refresh requested, bypassing cache')
}
logger.info('[OllamaService] Fetching fresh available models from API')

View File

@ -16,11 +16,13 @@ import { join, resolve, sep } from 'node:path'
import KVStore from '#models/kv_store'
import { ZIMExtractionService } from './zim_extraction_service.js'
import { ZIM_BATCH_SIZE } from '../../constants/zim_extraction.js'
import { ProcessAndEmbedFileResponse, ProcessZIMFileResponse, RAGResult, RerankedRAGResult } from '../../types/rag.js'
@inject()
export class RagService {
private qdrant: QdrantClient | null = null
private qdrantInitPromise: Promise<void> | null = null
private embeddingModelVerified = false
public static UPLOADS_STORAGE_PATH = 'storage/kb_uploads'
public static CONTENT_COLLECTION_NAME = 'nomad_knowledge_base'
public static EMBEDDING_MODEL = 'nomic-embed-text:v1.5'
@ -33,6 +35,7 @@ export class RagService {
// Nomic Embed Text v1.5 uses task-specific prefixes for optimal performance
public static SEARCH_DOCUMENT_PREFIX = 'search_document: '
public static SEARCH_QUERY_PREFIX = 'search_query: '
public static EMBEDDING_BATCH_SIZE = 8 // Conservative batch size for low-end hardware
constructor(
private dockerService: DockerService,
@ -75,6 +78,16 @@ export class RagService {
},
})
}
// Create payload indexes for faster filtering (idempotent — Qdrant ignores duplicates)
await this.qdrant!.createPayloadIndex(collectionName, {
field_name: 'source',
field_schema: 'keyword',
})
await this.qdrant!.createPayloadIndex(collectionName, {
field_name: 'content_type',
field_schema: 'keyword',
})
} catch (error) {
logger.error('Error ensuring Qdrant collection:', error)
throw error
@ -148,14 +161,57 @@ export class RagService {
/**
* Preprocesses a query to improve retrieval by expanding it with context.
* This helps match documents even when using different terminology.
* TODO: We could probably move this to a separate QueryPreprocessor class if it grows more complex, but for now it's manageable here.
*/
private static QUERY_EXPANSION_DICTIONARY: Record<string, string> = {
'bob': 'bug out bag',
'bov': 'bug out vehicle',
'bol': 'bug out location',
'edc': 'every day carry',
'mre': 'meal ready to eat',
'shtf': 'shit hits the fan',
'teotwawki': 'the end of the world as we know it',
'opsec': 'operational security',
'ifak': 'individual first aid kit',
'ghb': 'get home bag',
'ghi': 'get home in',
'wrol': 'without rule of law',
'emp': 'electromagnetic pulse',
'ham': 'ham amateur radio',
'nbr': 'nuclear biological radiological',
'cbrn': 'chemical biological radiological nuclear',
'sar': 'search and rescue',
'comms': 'communications radio',
'fifo': 'first in first out',
'mylar': 'mylar bag food storage',
'paracord': 'paracord 550 cord',
'ferro': 'ferro rod fire starter',
'bivvy': 'bivvy bivy emergency shelter',
'bdu': 'battle dress uniform',
'gmrs': 'general mobile radio service',
'frs': 'family radio service',
'nbc': 'nuclear biological chemical',
}
private preprocessQuery(query: string): string {
// Future: this is a placeholder for more advanced query expansion techniques.
// For now, we simply trim whitespace. Improvements could include:
// - Synonym expansion using a thesaurus
// - Adding related terms based on domain knowledge
// - Using a language model to rephrase or elaborate the query
const expanded = query.trim()
let expanded = query.trim()
// Expand known domain abbreviations/acronyms
const words = expanded.toLowerCase().split(/\s+/)
const expansions: string[] = []
for (const word of words) {
const cleaned = word.replace(/[^\w]/g, '')
if (RagService.QUERY_EXPANSION_DICTIONARY[cleaned]) {
expansions.push(RagService.QUERY_EXPANSION_DICTIONARY[cleaned])
}
}
if (expansions.length > 0) {
expanded = `${expanded} ${expansions.join(' ')}`
logger.debug(`[RAG] Query expanded with domain terms: "${expanded}"`)
}
logger.debug(`[RAG] Original query: "${query}"`)
logger.debug(`[RAG] Preprocessed query: "${expanded}"`)
return expanded
@ -187,22 +243,26 @@ export class RagService {
RagService.EMBEDDING_DIMENSION
)
const allModels = await this.ollamaService.getModels(true)
const embeddingModel = allModels.find((model) => model.name === RagService.EMBEDDING_MODEL)
if (!this.embeddingModelVerified) {
const allModels = await this.ollamaService.getModels(true)
const embeddingModel = allModels.find((model) => model.name === RagService.EMBEDDING_MODEL)
if (!embeddingModel) {
try {
const downloadResult = await this.ollamaService.downloadModel(RagService.EMBEDDING_MODEL)
if (!downloadResult.success) {
throw new Error(downloadResult.message || 'Unknown error during model download')
if (!embeddingModel) {
try {
const downloadResult = await this.ollamaService.downloadModel(RagService.EMBEDDING_MODEL)
if (!downloadResult.success) {
throw new Error(downloadResult.message || 'Unknown error during model download')
}
} catch (modelError) {
logger.error(
`[RAG] Embedding model ${RagService.EMBEDDING_MODEL} not found locally and failed to download:`,
modelError
)
this.embeddingModelVerified = false
return null
}
} catch (modelError) {
logger.error(
`[RAG] Embedding model ${RagService.EMBEDDING_MODEL} not found locally and failed to download:`,
modelError
)
return null
}
this.embeddingModelVerified = true
}
// TokenChunker uses character-based tokenization (1 char = 1 token)
@ -227,7 +287,8 @@ export class RagService {
const ollamaClient = await this.ollamaService.getClient()
const embeddings: number[][] = []
// Prepare all chunk texts with prefix and truncation
const prefixedChunks: string[] = []
for (let i = 0; i < chunks.length; i++) {
let chunkText = chunks[i]
@ -237,7 +298,6 @@ export class RagService {
const estimatedTokens = this.estimateTokenCount(withPrefix)
if (estimatedTokens > RagService.MAX_SAFE_TOKENS) {
// This should be rare - log for debugging if it's occurring frequently
const prefixTokens = this.estimateTokenCount(prefixText)
const maxTokensForText = RagService.MAX_SAFE_TOKENS - prefixTokens
logger.warn(
@ -246,17 +306,30 @@ export class RagService {
chunkText = this.truncateToTokenLimit(chunkText, maxTokensForText)
}
logger.debug(`[RAG] Generating embedding for chunk ${i + 1}/${chunks.length}`)
prefixedChunks.push(RagService.SEARCH_DOCUMENT_PREFIX + chunkText)
}
const response = await ollamaClient.embeddings({
// Batch embed chunks for performance
const embeddings: number[][] = []
const batchSize = RagService.EMBEDDING_BATCH_SIZE
const totalBatches = Math.ceil(prefixedChunks.length / batchSize)
for (let batchIdx = 0; batchIdx < totalBatches; batchIdx++) {
const batchStart = batchIdx * batchSize
const batch = prefixedChunks.slice(batchStart, batchStart + batchSize)
logger.debug(`[RAG] Embedding batch ${batchIdx + 1}/${totalBatches} (${batch.length} chunks)`)
const response = await ollamaClient.embed({
model: RagService.EMBEDDING_MODEL,
prompt: RagService.SEARCH_DOCUMENT_PREFIX + chunkText,
input: batch,
})
embeddings.push(response.embedding)
embeddings.push(...response.embeddings)
if (onProgress) {
await onProgress(((i + 1) / chunks.length) * 100)
const progress = ((batchStart + batch.length) / prefixedChunks.length) * 100
await onProgress(progress)
}
}
@ -395,14 +468,7 @@ export class RagService {
deleteAfterEmbedding: boolean,
batchOffset?: number,
onProgress?: (percent: number) => Promise<void>
): Promise<{
success: boolean
message: string
chunks?: number
hasMoreBatches?: boolean
articlesProcessed?: number
totalArticles?: number
}> {
): Promise<ProcessZIMFileResponse> {
const zimExtractionService = new ZIMExtractionService()
// Process in batches to avoid lock timeout
@ -540,14 +606,7 @@ export class RagService {
deleteAfterEmbedding: boolean = false,
batchOffset?: number,
onProgress?: (percent: number) => Promise<void>
): Promise<{
success: boolean
message: string
chunks?: number
hasMoreBatches?: boolean
articlesProcessed?: number
totalArticles?: number
}> {
): Promise<ProcessAndEmbedFileResponse> {
try {
const fileType = determineFileType(filepath)
logger.debug(`[RAG] Processing file: ${filepath} (detected type: ${fileType})`)
@ -631,14 +690,18 @@ export class RagService {
return []
}
const allModels = await this.ollamaService.getModels(true)
const embeddingModel = allModels.find((model) => model.name === RagService.EMBEDDING_MODEL)
if (!this.embeddingModelVerified) {
const allModels = await this.ollamaService.getModels(true)
const embeddingModel = allModels.find((model) => model.name === RagService.EMBEDDING_MODEL)
if (!embeddingModel) {
logger.warn(
`[RAG] ${RagService.EMBEDDING_MODEL} not found. Cannot perform similarity search.`
)
return []
if (!embeddingModel) {
logger.warn(
`[RAG] ${RagService.EMBEDDING_MODEL} not found. Cannot perform similarity search.`
)
this.embeddingModelVerified = false
return []
}
this.embeddingModelVerified = true
}
// Preprocess query for better matching
@ -666,9 +729,9 @@ export class RagService {
return []
}
const response = await ollamaClient.embeddings({
const response = await ollamaClient.embed({
model: RagService.EMBEDDING_MODEL,
prompt: prefixedQuery,
input: [prefixedQuery],
})
// Perform semantic search with a higher limit to enable reranking
@ -678,7 +741,7 @@ export class RagService {
)
const searchResults = await this.qdrant!.search(RagService.CONTENT_COLLECTION_NAME, {
vector: response.embedding,
vector: response.embeddings[0],
limit: searchLimit,
score_threshold: scoreThreshold,
with_payload: true,
@ -687,7 +750,7 @@ export class RagService {
logger.debug(`[RAG] Found ${searchResults.length} results above threshold ${scoreThreshold}`)
// Map results with metadata for reranking
const resultsWithMetadata = searchResults.map((result) => ({
const resultsWithMetadata: RAGResult[] = searchResults.map((result) => ({
text: (result.payload?.text as string) || '',
score: result.score,
keywords: (result.payload?.keywords as string) || '',
@ -700,6 +763,7 @@ export class RagService {
hierarchy: result.payload?.hierarchy as string | undefined,
document_id: result.payload?.document_id as string | undefined,
content_type: result.payload?.content_type as string | undefined,
source: result.payload?.source as string | undefined,
}))
const rerankedResults = this.rerankResults(resultsWithMetadata, keywords, query)
@ -711,8 +775,11 @@ export class RagService {
)
})
// Apply source diversity penalty to avoid all results from the same document
const diverseResults = this.applySourceDiversity(rerankedResults)
// Return top N results with enhanced metadata
return rerankedResults.slice(0, limit).map((result) => ({
return diverseResults.slice(0, limit).map((result) => ({
text: result.text,
score: result.finalScore,
metadata: {
@ -748,34 +815,10 @@ export class RagService {
* outweigh the overhead.
*/
private rerankResults(
results: Array<{
text: string
score: number
keywords: string
chunk_index: number
created_at: number
article_title?: string
section_title?: string
full_title?: string
hierarchy?: string
document_id?: string
content_type?: string
}>,
results: Array<RAGResult>,
queryKeywords: string[],
originalQuery: string
): Array<{
text: string
score: number
finalScore: number
chunk_index: number
created_at: number
article_title?: string
section_title?: string
full_title?: string
hierarchy?: string
document_id?: string
content_type?: string
}> {
): Array<RerankedRAGResult> {
return results
.map((result) => {
let finalScore = result.score
@ -851,6 +894,37 @@ export class RagService {
.sort((a, b) => b.finalScore - a.finalScore)
}
/**
* Applies a diversity penalty so results from the same source are down-weighted.
* Uses greedy selection: for each result, apply 0.85^n penalty where n is the
* number of results already selected from the same source.
*/
private applySourceDiversity(
results: Array<RerankedRAGResult>
) {
const sourceCounts = new Map<string, number>()
const DIVERSITY_PENALTY = 0.85
return results
.map((result) => {
const sourceKey = result.document_id || result.source || 'unknown'
const count = sourceCounts.get(sourceKey) || 0
const penalty = Math.pow(DIVERSITY_PENALTY, count)
const diverseScore = result.finalScore * penalty
sourceCounts.set(sourceKey, count + 1)
if (count > 0) {
logger.debug(
`[RAG] Source diversity penalty for "${sourceKey}": ${result.finalScore.toFixed(4)}${diverseScore.toFixed(4)} (seen ${count}x)`
)
}
return { ...result, finalScore: diverseScore }
})
.sort((a, b) => b.finalScore - a.finalScore)
}
/**
* Retrieve all unique source files that have been stored in the knowledge base.
* @returns Array of unique full source paths
@ -866,12 +940,12 @@ export class RagService {
let offset: string | number | null | Record<string, unknown> = null
const batchSize = 100
// Scroll through all points in the collection
// Scroll through all points in the collection (only fetch source field)
do {
const scrollResult = await this.qdrant!.scroll(RagService.CONTENT_COLLECTION_NAME, {
limit: batchSize,
offset: offset,
with_payload: true,
with_payload: ['source'],
with_vector: false,
})

View File

@ -4,7 +4,7 @@ import { DockerService } from '#services/docker_service'
import { ServiceSlim } from '../../types/services.js'
import logger from '@adonisjs/core/services/logger'
import si from 'systeminformation'
import { NomadDiskInfo, NomadDiskInfoRaw, SystemInformationResponse } from '../../types/system.js'
import { GpuHealthStatus, NomadDiskInfo, NomadDiskInfoRaw, SystemInformationResponse } from '../../types/system.js'
import { SERVICE_NAMES } from '../../constants/service_names.js'
import { readFileSync } from 'fs'
import path, { join } from 'path'
@ -13,6 +13,7 @@ import axios from 'axios'
import env from '#start/env'
import KVStore from '#models/kv_store'
import { KV_STORE_SCHEMA, KVStoreKey } from '../../types/kv_store.js'
import { isNewerVersion } from '../utils/version.js'
@inject()
@ -142,7 +143,9 @@ export class SystemService {
'description',
'icon',
'powered_by',
'display_order'
'display_order',
'container_image',
'available_update_version'
)
.where('is_dependency_service', false)
if (installedOnly) {
@ -172,6 +175,8 @@ export class SystemService {
ui_location: service.ui_location || '',
powered_by: service.powered_by,
display_order: service.display_order,
container_image: service.container_image,
available_update_version: service.available_update_version,
})
}
@ -235,6 +240,13 @@ export class SystemService {
logger.error('Error reading disk info file:', error)
}
// GPU health tracking — detect when host has NVIDIA GPU but Ollama can't access it
let gpuHealth: GpuHealthStatus = {
status: 'no_gpu',
hasNvidiaRuntime: false,
ollamaGpuAccessible: false,
}
// Query Docker API for host-level info (hostname, OS, GPU runtime)
// si.osInfo() returns the container's info inside Docker, not the host's
try {
@ -255,6 +267,7 @@ export class SystemService {
if (!graphics.controllers || graphics.controllers.length === 0) {
const runtimes = dockerInfo.Runtimes || {}
if ('nvidia' in runtimes) {
gpuHealth.hasNvidiaRuntime = true
const nvidiaInfo = await this.getNvidiaSmiInfo()
if (Array.isArray(nvidiaInfo)) {
graphics.controllers = nvidiaInfo.map((gpu) => ({
@ -264,10 +277,19 @@ export class SystemService {
vram: gpu.vram,
vramDynamic: false, // assume false here, we don't actually use this field for our purposes.
}))
gpuHealth.status = 'ok'
gpuHealth.ollamaGpuAccessible = true
} else if (nvidiaInfo === 'OLLAMA_NOT_FOUND') {
gpuHealth.status = 'ollama_not_installed'
} else {
logger.warn(`NVIDIA runtime detected but failed to get GPU info: ${typeof nvidiaInfo === 'string' ? nvidiaInfo : JSON.stringify(nvidiaInfo)}`)
gpuHealth.status = 'passthrough_failed'
logger.warn(`NVIDIA runtime detected but GPU passthrough failed: ${typeof nvidiaInfo === 'string' ? nvidiaInfo : JSON.stringify(nvidiaInfo)}`)
}
}
} else {
// si.graphics() returned controllers (host install, not Docker) — GPU is working
gpuHealth.status = 'ok'
gpuHealth.ollamaGpuAccessible = true
}
} catch {
// Docker info query failed, skip host-level enrichment
@ -282,6 +304,7 @@ export class SystemService {
fsSize,
uptime,
graphics,
gpuHealth,
}
} catch (error) {
logger.error('Error getting system info:', error)
@ -335,7 +358,7 @@ export class SystemService {
const updateAvailable = process.env.NODE_ENV === 'development'
? false
: this.isNewerVersion(latestVersion, currentVersion.trim())
: isNewerVersion(latestVersion, currentVersion.trim(), earlyAccess)
// Cache the results in KVStore for frontend checks
await KVStore.setValue('system.updateAvailable', updateAvailable)
@ -476,35 +499,4 @@ export class SystemService {
})
}
/**
* Compare two semantic version strings to determine if the first is newer than the second.
* @param version1 - The version to check (e.g., "1.25.0")
* @param version2 - The current version (e.g., "1.24.0")
* @returns true if version1 is newer than version2
*/
private isNewerVersion(version1: string, version2: string): boolean {
const [base1, pre1] = version1.split('-')
const [base2, pre2] = version2.split('-')
const v1Parts = base1.split('.').map((p) => parseInt(p, 10) || 0)
const v2Parts = base2.split('.').map((p) => parseInt(p, 10) || 0)
const maxLen = Math.max(v1Parts.length, v2Parts.length)
for (let i = 0; i < maxLen; i++) {
const a = v1Parts[i] || 0
const b = v2Parts[i] || 0
if (a > b) return true
if (a < b) return false
}
// Base versions equal — GA > RC, RC.n+1 > RC.n
if (!pre1 && pre2) return true // v1 is GA, v2 is RC → v1 is newer
if (pre1 && !pre2) return false // v1 is RC, v2 is GA → v2 is newer
if (!pre1 && !pre2) return false // both GA, equal
// Both prerelease: compare numeric suffix (e.g. "rc.2" vs "rc.1")
const pre1Num = parseInt(pre1.split('.')[1], 10) || 0
const pre2Num = parseInt(pre2.split('.')[1], 10) || 0
return pre1Num > pre2Num
}
}

View File

@ -36,7 +36,7 @@ export class SystemUpdateService {
const requestData = {
requested_at: new Date().toISOString(),
requester: 'admin-api',
target_tag: latestVersion || 'latest', // We should always have a latest version, but fallback to 'latest' just in case
target_tag: latestVersion ? `v${latestVersion}` : 'latest',
}
await writeFile(SystemUpdateService.REQUEST_FILE, JSON.stringify(requestData, null, 2))

View File

@ -16,7 +16,7 @@ import {
listDirectoryContents,
ZIM_STORAGE_PATH,
} from '../utils/fs.js'
import { join } from 'path'
import { join, resolve, sep } from 'path'
import { WikipediaOption, WikipediaState } from '../../types/downloads.js'
import vine from '@vinejs/vine'
import { wikipediaOptionsFileSchema } from '#validators/curated_collections'
@ -332,7 +332,13 @@ export class ZimService {
fileName += '.zim'
}
const fullPath = join(process.cwd(), ZIM_STORAGE_PATH, fileName)
const basePath = resolve(join(process.cwd(), ZIM_STORAGE_PATH))
const fullPath = resolve(join(basePath, fileName))
// Prevent path traversal — resolved path must stay within the storage directory
if (!fullPath.startsWith(basePath + sep)) {
throw new Error('Invalid filename')
}
const exists = await getFileStatsIfExists(fullPath)
if (!exists) {

View File

@ -0,0 +1,49 @@
/**
* Compare two semantic version strings to determine if the first is newer than the second.
* @param version1 - The version to check (e.g., "1.25.0")
* @param version2 - The current version (e.g., "1.24.0")
* @returns true if version1 is newer than version2
*/
export function isNewerVersion(version1: string, version2: string, includePreReleases = false): boolean {
const normalize = (v: string) => v.replace(/^v/, '')
const [base1, pre1] = normalize(version1).split('-')
const [base2, pre2] = normalize(version2).split('-')
// If pre-releases are not included and version1 is a pre-release, don't consider it newer
if (!includePreReleases && pre1) {
return false
}
const v1Parts = base1.split('.').map((p) => parseInt(p, 10) || 0)
const v2Parts = base2.split('.').map((p) => parseInt(p, 10) || 0)
const maxLen = Math.max(v1Parts.length, v2Parts.length)
for (let i = 0; i < maxLen; i++) {
const a = v1Parts[i] || 0
const b = v2Parts[i] || 0
if (a > b) return true
if (a < b) return false
}
// Base versions equal — GA > RC, RC.n+1 > RC.n
if (!pre1 && pre2) return true // v1 is GA, v2 is RC → v1 is newer
if (pre1 && !pre2) return false // v1 is RC, v2 is GA → v2 is newer
if (!pre1 && !pre2) return false // both GA, equal
// Both prerelease: compare numeric suffix (e.g. "rc.2" vs "rc.1")
const pre1Num = parseInt(pre1.split('.')[1], 10) || 0
const pre2Num = parseInt(pre2.split('.')[1], 10) || 0
return pre1Num > pre2Num
}
/**
* Parse the major version number from a tag string.
* Strips the 'v' prefix if present.
* @param tag - Version tag (e.g., "v3.8.1", "10.19.4")
* @returns The major version number
*/
export function parseMajorVersion(tag: string): number {
const normalized = tag.replace(/^v/, '')
const major = parseInt(normalized.split('.')[0], 10)
return isNaN(major) ? 0 : major
}

View File

@ -1,12 +1,39 @@
import vine from '@vinejs/vine'
/**
* Checks whether a URL points to a loopback or link-local address.
* Used to prevent SSRF the server should not fetch from localhost
* or link-local/metadata endpoints (e.g. cloud instance metadata at 169.254.x.x).
*
* RFC1918 private ranges (10.x, 172.16-31.x, 192.168.x) are intentionally
* ALLOWED because NOMAD is a LAN appliance and users may host content
* mirrors on their local network.
*
* Throws an error if the URL is a loopback or link-local address.
*/
export function assertNotPrivateUrl(urlString: string): void {
const parsed = new URL(urlString)
const hostname = parsed.hostname.toLowerCase()
const blockedPatterns = [
/^localhost$/,
/^127\.\d+\.\d+\.\d+$/,
/^0\.0\.0\.0$/,
/^169\.254\.\d+\.\d+$/, // Link-local / cloud metadata
/^\[::1\]$/,
/^\[?fe80:/i, // IPv6 link-local
]
if (blockedPatterns.some((re) => re.test(hostname))) {
throw new Error(`Download URL must not point to a loopback or link-local address: ${hostname}`)
}
}
export const remoteDownloadValidator = vine.compile(
vine.object({
url: vine
.string()
.url({
require_tld: false, // Allow local URLs
})
.url({ require_tld: false }) // Allow LAN URLs (e.g. http://my-nas:8080/file.zim)
.trim(),
})
)
@ -15,9 +42,7 @@ export const remoteDownloadWithMetadataValidator = vine.compile(
vine.object({
url: vine
.string()
.url({
require_tld: false, // Allow local URLs
})
.url({ require_tld: false }) // Allow LAN URLs
.trim(),
metadata: vine
.object({
@ -34,9 +59,7 @@ export const remoteDownloadValidatorOptional = vine.compile(
vine.object({
url: vine
.string()
.url({
require_tld: false, // Allow local URLs
})
.url({ require_tld: false }) // Allow LAN URLs
.trim()
.optional(),
})
@ -74,7 +97,7 @@ const resourceUpdateInfoBase = vine.object({
resource_type: vine.enum(['zim', 'map'] as const),
installed_version: vine.string().trim(),
latest_version: vine.string().trim().minLength(1),
download_url: vine.string().url().trim(),
download_url: vine.string().url({ require_tld: false }).trim(),
})
export const applyContentUpdateValidator = vine.compile(resourceUpdateInfoBase)

View File

@ -10,6 +10,7 @@ export const chatSchema = vine.compile(
})
),
stream: vine.boolean().optional(),
sessionId: vine.number().positive().optional(),
})
)
@ -19,5 +20,6 @@ export const getAvailableModelsSchema = vine.compile(
recommendedOnly: vine.boolean().optional(),
query: vine.string().trim().optional(),
limit: vine.number().positive().optional(),
force: vine.boolean().optional(),
})
)

View File

@ -24,3 +24,10 @@ export const checkLatestVersionValidator = vine.compile(
force: vine.boolean().optional(), // Optional flag to force bypassing cache and checking for updates immediately
})
)
export const updateServiceValidator = vine.compile(
vine.object({
service_name: vine.string().trim(),
target_version: vine.string().trim(),
})
)

View File

@ -7,6 +7,7 @@ import { DownloadModelJob } from '#jobs/download_model_job'
import { RunBenchmarkJob } from '#jobs/run_benchmark_job'
import { EmbedFileJob } from '#jobs/embed_file_job'
import { CheckUpdateJob } from '#jobs/check_update_job'
import { CheckServiceUpdatesJob } from '#jobs/check_service_updates_job'
export default class QueueWork extends BaseCommand {
static commandName = 'queue:work'
@ -76,8 +77,9 @@ export default class QueueWork extends BaseCommand {
this.logger.info(`Worker started for queue: ${queueName}`)
}
// Schedule nightly update check (idempotent, will persist over restarts)
// Schedule nightly update checks (idempotent, will persist over restarts)
await CheckUpdateJob.scheduleNightly()
await CheckServiceUpdatesJob.scheduleNightly()
// Graceful shutdown for all workers
process.on('SIGTERM', async () => {
@ -97,12 +99,14 @@ export default class QueueWork extends BaseCommand {
handlers.set(RunBenchmarkJob.key, new RunBenchmarkJob())
handlers.set(EmbedFileJob.key, new EmbedFileJob())
handlers.set(CheckUpdateJob.key, new CheckUpdateJob())
handlers.set(CheckServiceUpdatesJob.key, new CheckServiceUpdatesJob())
queues.set(RunDownloadJob.key, RunDownloadJob.queue)
queues.set(DownloadModelJob.key, DownloadModelJob.queue)
queues.set(RunBenchmarkJob.key, RunBenchmarkJob.queue)
queues.set(EmbedFileJob.key, EmbedFileJob.queue)
queues.set(CheckUpdateJob.key, CheckUpdateJob.queue)
queues.set(CheckServiceUpdatesJob.key, CheckServiceUpdatesJob.queue)
return [handlers, queues]
}

View File

@ -3,4 +3,5 @@ export const BROADCAST_CHANNELS = {
BENCHMARK_PROGRESS: 'benchmark-progress',
OLLAMA_MODEL_DOWNLOAD: 'ollama-model-download',
SERVICE_INSTALLATION: 'service-installation',
SERVICE_UPDATES: 'service-updates',
}

View File

@ -1,3 +1,3 @@
import { KVStoreKey } from "../types/kv_store.js";
export const SETTINGS_KEYS: KVStoreKey[] = ['chat.suggestionsEnabled', 'ui.hasVisitedEasySetup', 'system.earlyAccess', 'ai.assistantCustomName'];
export const SETTINGS_KEYS: KVStoreKey[] = ['chat.suggestionsEnabled', 'chat.lastModel', 'ui.hasVisitedEasySetup', 'system.earlyAccess', 'ai.assistantCustomName'];

View File

@ -64,6 +64,16 @@ export const FALLBACK_RECOMMENDED_OLLAMA_MODELS: NomadOllamaModel[] = [
export const DEFAULT_QUERY_REWRITE_MODEL = 'qwen2.5:3b' // default to qwen2.5 for query rewriting with good balance of text task performance and resource usage
/**
* Adaptive RAG context limits based on model size.
* Smaller models get overwhelmed with too much context, so we cap it.
*/
export const RAG_CONTEXT_LIMITS: { maxParams: number; maxResults: number; maxTokens: number }[] = [
{ maxParams: 3, maxResults: 2, maxTokens: 1000 }, // 1-3B models
{ maxParams: 8, maxResults: 4, maxTokens: 2500 }, // 4-8B models
{ maxParams: Infinity, maxResults: 5, maxTokens: 0 }, // 13B+ (no cap)
]
export const SYSTEM_PROMPTS = {
default: `
Format all responses using markdown for better readability. Vanilla markdown or GitHub-flavored markdown is preferred.
@ -83,9 +93,9 @@ IMPORTANT INSTRUCTIONS:
1. If the user's question is directly related to the context above, use this information to provide accurate, detailed answers.
2. Always cite or reference the context when using it (e.g., "According to the information available..." or "Based on the knowledge base...").
3. If the context is only partially relevant, combine it with your general knowledge but be clear about what comes from the knowledge base.
4. If the context is not relevant to the user's question, you can respond using your general knowledge without forcing the context into your answer.
4. If the context is not relevant to the user's question, you can respond using your general knowledge without forcing the context into your answer. Do not mention the context if it's not relevant.
5. Never fabricate information that isn't in the context or your training data.
6. If you're unsure or the context doesn't contain enough information, acknowledge the limitations.
6. If you're unsure or you don't have enough information to answer the user's question, acknowledge the limitations.
Format your response using markdown for readability.
`,
@ -113,6 +123,7 @@ Ensure that your suggestions are comma-seperated with no conjunctions like "and"
Do not use line breaks, new lines, or extra spacing to separate the suggestions.
Format: suggestion1, suggestion2, suggestion3
`,
title_generation: `You are a title generator. Given the start of a conversation, generate a concise, descriptive title under 50 characters. Return ONLY the title text with no quotes, punctuation wrapping, or extra formatting.`,
query_rewrite: `
You are a query rewriting assistant. Your task is to reformulate the user's latest question to include relevant context from the conversation history.

View File

@ -0,0 +1,21 @@
import { BaseSchema } from '@adonisjs/lucid/schema'
export default class extends BaseSchema {
protected tableName = 'services'
async up() {
this.schema.alterTable(this.tableName, (table) => {
table.string('source_repo', 255).nullable()
table.string('available_update_version', 50).nullable()
table.timestamp('update_checked_at').nullable()
})
}
async down() {
this.schema.alterTable(this.tableName, (table) => {
table.dropColumn('source_repo')
table.dropColumn('available_update_version')
table.dropColumn('update_checked_at')
})
}
}

View File

@ -0,0 +1,61 @@
import { BaseSchema } from '@adonisjs/lucid/schema'
export default class extends BaseSchema {
protected tableName = 'services'
async up() {
this.defer(async (db) => {
// Pin :latest images to specific versions
await db
.from(this.tableName)
.where('container_image', 'ghcr.io/gchq/cyberchef:latest')
.update({ container_image: 'ghcr.io/gchq/cyberchef:10.19.4' })
await db
.from(this.tableName)
.where('container_image', 'dullage/flatnotes:latest')
.update({ container_image: 'dullage/flatnotes:v5.5.4' })
await db
.from(this.tableName)
.where('container_image', 'treehouses/kolibri:latest')
.update({ container_image: 'treehouses/kolibri:0.12.8' })
// Populate source_repo for services whose images lack the OCI source label
const sourceRepos: Record<string, string> = {
nomad_kiwix_server: 'https://github.com/kiwix/kiwix-tools',
nomad_ollama: 'https://github.com/ollama/ollama',
nomad_qdrant: 'https://github.com/qdrant/qdrant',
nomad_cyberchef: 'https://github.com/gchq/CyberChef',
nomad_flatnotes: 'https://github.com/dullage/flatnotes',
nomad_kolibri: 'https://github.com/learningequality/kolibri',
}
for (const [serviceName, repoUrl] of Object.entries(sourceRepos)) {
await db
.from(this.tableName)
.where('service_name', serviceName)
.update({ source_repo: repoUrl })
}
})
}
async down() {
this.defer(async (db) => {
await db
.from(this.tableName)
.where('container_image', 'ghcr.io/gchq/cyberchef:10.19.4')
.update({ container_image: 'ghcr.io/gchq/cyberchef:latest' })
await db
.from(this.tableName)
.where('container_image', 'dullage/flatnotes:v5.5.4')
.update({ container_image: 'dullage/flatnotes:latest' })
await db
.from(this.tableName)
.where('container_image', 'treehouses/kolibri:0.12.8')
.update({ container_image: 'treehouses/kolibri:latest' })
})
}
}

View File

@ -12,7 +12,7 @@ export default class ServiceSeeder extends BaseSeeder {
)
private static DEFAULT_SERVICES: Omit<
ModelAttributes<Service>,
'created_at' | 'updated_at' | 'metadata' | 'id'
'created_at' | 'updated_at' | 'metadata' | 'id' | 'available_update_version' | 'update_checked_at'
>[] = [
{
service_name: SERVICE_NAMES.KIWIX,
@ -23,6 +23,7 @@ export default class ServiceSeeder extends BaseSeeder {
'Offline access to Wikipedia, medical references, how-to guides, and encyclopedias',
icon: 'IconBooks',
container_image: 'ghcr.io/kiwix/kiwix-serve:3.8.1',
source_repo: 'https://github.com/kiwix/kiwix-tools',
container_command: '*.zim --address=all',
container_config: JSON.stringify({
HostConfig: {
@ -46,6 +47,7 @@ export default class ServiceSeeder extends BaseSeeder {
description: 'Vector database for storing and searching embeddings',
icon: 'IconRobot',
container_image: 'qdrant/qdrant:v1.16',
source_repo: 'https://github.com/qdrant/qdrant',
container_command: null,
container_config: JSON.stringify({
HostConfig: {
@ -69,6 +71,7 @@ export default class ServiceSeeder extends BaseSeeder {
description: 'Local AI chat that runs entirely on your hardware - no internet required',
icon: 'IconWand',
container_image: 'ollama/ollama:0.15.2',
source_repo: 'https://github.com/ollama/ollama',
container_command: 'serve',
container_config: JSON.stringify({
HostConfig: {
@ -91,7 +94,8 @@ export default class ServiceSeeder extends BaseSeeder {
display_order: 11,
description: 'Swiss Army knife for data encoding, encryption, and analysis',
icon: 'IconChefHat',
container_image: 'ghcr.io/gchq/cyberchef:latest',
container_image: 'ghcr.io/gchq/cyberchef:10.19.4',
source_repo: 'https://github.com/gchq/CyberChef',
container_command: null,
container_config: JSON.stringify({
HostConfig: {
@ -113,7 +117,8 @@ export default class ServiceSeeder extends BaseSeeder {
display_order: 10,
description: 'Simple note-taking app with local storage',
icon: 'IconNotes',
container_image: 'dullage/flatnotes:latest',
container_image: 'dullage/flatnotes:v5.5.4',
source_repo: 'https://github.com/dullage/flatnotes',
container_command: null,
container_config: JSON.stringify({
HostConfig: {
@ -137,7 +142,8 @@ export default class ServiceSeeder extends BaseSeeder {
display_order: 2,
description: 'Interactive learning platform with video courses and exercises',
icon: 'IconSchool',
container_image: 'treehouses/kolibri:latest',
container_image: 'treehouses/kolibri:0.12.8',
source_repo: 'https://github.com/learningequality/kolibri',
container_command: null,
container_config: JSON.stringify({
HostConfig: {

View File

@ -2,13 +2,16 @@
Project N.O.M.A.D. (Node for Offline Media, Archives, and Data; "Nomad" for short) is a project started in 2025 by Chris Sherwood of [Crosstalk Solutions, LLC](https://crosstalksolutions.com). The goal of the project is not to create just another utility for storing offline resources, but rather to allow users to run their own ultimate "survival computer".
While many similar offline survival computers are designed to be run on bare-minimum, lightweight hardware, Project N.O.M.A.D. is quite the opposite. To install and run the available AI tools, we highly encourage the use of a beefy, GPU-backed device to make the most of your install.
While many similar offline survival computers are designed to be run on bare-minimum, lightweight hardware, Project N.O.M.A.D. is quite the opposite. To install and run the available AI tools, we highly encourage the use of a beefy, GPU-backed device to make the most of your install. See the [Hardware Guide](https://www.projectnomad.us/hardware) for detailed build recommendations at three price points.
Since its initial release, NOMAD has grown to include built-in AI chat with a Knowledge Base for document-aware responses, a System Benchmark with a community leaderboard, curated content collections with tiered options, and an Easy Setup Wizard to get new users up and running quickly.
Project N.O.M.A.D. is open source, released under the [Apache License 2.0](https://github.com/Crosstalk-Solutions/project-nomad/blob/main/LICENSE).
## Links
- **Website:** [www.projectnomad.us](https://www.projectnomad.us)
- **Hardware Guide:** [www.projectnomad.us/hardware](https://www.projectnomad.us/hardware)
- **Discord:** [Join the Community](https://discord.com/invite/crosstalksolutions)
- **GitHub:** [Crosstalk-Solutions/project-nomad](https://github.com/Crosstalk-Solutions/project-nomad)
- **Benchmark Leaderboard:** [benchmark.projectnomad.us](https://benchmark.projectnomad.us)

View File

@ -18,6 +18,8 @@ N.O.M.A.D. is designed for capable hardware, especially if you want to use the A
- SSD storage (size depends on content — 500GB minimum, 2TB+ recommended)
- NVIDIA or AMD GPU recommended for faster AI responses
**For detailed build recommendations at three price points ($200$800+), see the [Hardware Guide](https://www.projectnomad.us/hardware).**
### How much storage do I need?
It depends on what you download:
- Full Wikipedia: ~95GB
@ -79,6 +81,8 @@ The AI must be installed first — enable it during Easy Setup or install it fro
3. Documents are processed and indexed automatically
4. Ask questions in AI Chat — the AI will reference your uploaded documents when relevant
You can also remove documents from the Knowledge Base when they're no longer needed.
NOMAD documentation is automatically added to the Knowledge Base when the AI Assistant is installed.
### What is the System Benchmark?
@ -86,6 +90,9 @@ The System Benchmark tests your hardware performance and generates a NOMAD Score
Go to **[System Benchmark →](/settings/benchmark)** to run one.
### What is the Early Access Channel?
The Early Access Channel lets you opt in to receive release candidate builds with the latest features and improvements before they hit stable releases. You can enable or disable it from **Settings → Check for Updates**. Early access builds may contain bugs — if you prefer stability, stay on the stable channel.
---
## Troubleshooting
@ -137,6 +144,10 @@ When you add or swap a GPU, N.O.M.A.D. needs to reconfigure the AI container to
Force Reinstall recreates the AI container with GPU support enabled. Without this step, the AI continues to run on CPU only.
### I see a "GPU passthrough not working" warning
N.O.M.A.D. checks whether your GPU is actually accessible inside the AI container. If a GPU is detected on the host but isn't working inside the container, you'll see a warning banner on the System Information and AI Settings pages. Click the **"Fix: Reinstall AI Assistant"** button to recreate the container with proper GPU access. This preserves your downloaded AI models.
### AI Chat not available
The AI Chat page requires the AI Assistant to be installed first:

View File

@ -103,6 +103,7 @@ The Knowledge Base lets you upload documents so the AI can reference them when a
2. Upload your documents (PDFs, text files, etc.)
3. Documents are processed and indexed automatically
4. Ask questions in AI Chat — the AI will reference your uploaded documents when relevant
5. Remove documents you no longer need — they'll be deleted from the index and local storage
**Use cases:**
- Upload emergency plans for quick reference during a crisis
@ -183,6 +184,8 @@ While you have internet, periodically check for updates:
Content updates (Wikipedia, maps, etc.) can be managed separately from software updates.
**Early Access Channel:** Want the latest features before they hit stable? Enable the Early Access Channel from the Check for Updates page to receive release candidate builds. You can switch back to stable anytime.
### Monitoring System Health
Check on your server anytime:

View File

@ -1,5 +1,20 @@
# Release Notes
## Unreleased
### Features
- **AI Assistant**: Added improved user guidance for troubleshooting GPU pass-through issues
- **AI Assistant**: The last used model is now automatically selected when a new chat is started
- **Settings**: Nomad now automatically performs nightly checks for available app updates, and users can select and apply updates from the Apps page in Settings
### Bug Fixes
- **Settings**: Fixed an issue where the AI Assistant settings page would be shown in navigation even if the AI Assistant was not installed, thus causing 404 errors when clicked
- **Security**: Path traversal and SSRF mitigations
- **AI Assistant**: Fixed an issue that was causing intermittent failures saving chat session titles
### Improvements
- **AI Assistant**: Extensive performance improvements and improved RAG intelligence/context usage
## Version 1.28.0 - March 5, 2026
### Features

View File

@ -0,0 +1,281 @@
# Project NOMAD Security Audit Report
**Date:** 2026-03-08
**Version audited:** v1.28.0 (main branch)
**Auditor:** Claude Code (automated + manual review)
**Target:** Pre-launch security review
---
## Executive Summary
Project NOMAD's codebase is **reasonably clean for a LAN appliance**, with no critical authentication bypasses or remote code execution vulnerabilities. However, there are **4 findings that should be fixed before public launch** — all are straightforward path traversal and SSRF issues with known fix patterns already used elsewhere in the codebase.
| Severity | Count | Summary |
|----------|-------|---------|
| **HIGH** | 4 | Path traversal (3), SSRF (1) |
| **MEDIUM** | 5 | Dozzle shell, unvalidated settings read, content update URL injection, verbose errors, no rate limiting |
| **LOW** | 5 | CSRF disabled, CORS wildcard, debug logging, npm dep CVEs, hardcoded HMAC |
| **INFO** | 2 | No auth by design, Docker socket exposure by design |
---
## Scans Performed
| Scan | Tool | Result |
|------|------|--------|
| Dependency audit | `npm audit` | 2 CVEs (1 high, 1 moderate) |
| Secret scan | Manual grep (passwords, keys, tokens, certs) | Clean — all secrets from env vars |
| SAST | Semgrep (security-audit, OWASP, nodejs rulesets) | 0 findings (AdonisJS not in rulesets) |
| Docker config review | Manual review of compose, Dockerfiles, scripts | 2 actionable findings |
| Code review | Manual review of services, controllers, validators | 4 path traversal + 1 SSRF |
| API endpoint audit | Manual review of all 60+ routes | Attack surface documented |
| DAST (OWASP ZAP) | Skipped — Docker Desktop not running | Recommended as follow-up |
---
## FIX BEFORE LAUNCH
### 1. Path Traversal — ZIM File Delete (HIGH)
**File:** `admin/app/services/zim_service.ts:329-342`
**Endpoint:** `DELETE /api/zim/:filename`
The `filename` parameter flows into `path.join()` with no directory containment check. An attacker can delete `.zim` files outside the storage directory:
```
DELETE /api/zim/..%2F..%2Fsome-file.zim
```
**Fix:** Resolve the full path and verify it starts with the expected storage directory:
```typescript
async delete(file: string): Promise<void> {
let fileName = file
if (!fileName.endsWith('.zim')) {
fileName += '.zim'
}
const basePath = join(process.cwd(), ZIM_STORAGE_PATH)
const fullPath = resolve(basePath, fileName)
// Prevent path traversal
if (!fullPath.startsWith(basePath)) {
throw new Error('Invalid filename')
}
// ... rest of delete logic
}
```
This pattern is already used correctly in `rag_service.ts:deleteFileBySource()`.
---
### 2. Path Traversal — Map File Delete (HIGH)
**File:** `admin/app/services/map_service.ts` (delete method)
**Endpoint:** `DELETE /api/maps/:filename`
Identical pattern to the ZIM delete. Same fix — resolve path, verify `startsWith(basePath)`.
---
### 3. Path Traversal — Documentation Read (HIGH)
**File:** `admin/app/services/docs_service.ts:61-83`
**Endpoint:** `GET /docs/:slug`
The `slug` parameter flows into `path.join(this.docsPath, filename)` with no containment check. An attacker can read arbitrary `.md` files on the filesystem:
```
GET /docs/..%2F..%2F..%2Fetc%2Fpasswd
```
Limited by the mandatory `.md` extension, but could still read sensitive markdown files outside the docs directory (like CLAUDE.md, README.md, etc.).
**Fix:**
```typescript
const basePath = this.docsPath
const fullPath = path.resolve(basePath, filename)
if (!fullPath.startsWith(path.resolve(basePath))) {
throw new Error('Invalid document slug')
}
```
---
### 4. SSRF — Download Endpoints (HIGH)
**File:** `admin/app/validators/common.ts`
**Endpoints:** `POST /api/zim/download-remote`, `POST /api/maps/download-remote`, `POST /api/maps/download-base-assets`, `POST /api/maps/download-remote-preflight`
The download endpoints accept user-supplied URLs and the server fetches from them. Without validation, an attacker on the LAN (or via CSRF since `shield.ts` disables CSRF protection) could make NOMAD fetch from co-located services:
- `http://localhost:3306` (MySQL)
- `http://localhost:6379` (Redis)
- `http://169.254.169.254/` (cloud metadata — if NOMAD is ever cloud-hosted)
**Fix:** Added `assertNotPrivateUrl()` that blocks loopback and link-local addresses before any download is initiated. Called in all download controllers.
**Scope note:** RFC1918 private addresses (10.x, 172.16-31.x, 192.168.x) are intentionally **allowed** because NOMAD is a LAN appliance and users may host content mirrors on their local network. The `require_tld: false` VineJS option is preserved so URLs like `http://my-nas:8080/file.zim` remain valid.
```typescript
const blockedPatterns = [
/^localhost$/,
/^127\.\d+\.\d+\.\d+$/,
/^0\.0\.0\.0$/,
/^169\.254\.\d+\.\d+$/, // Link-local / cloud metadata
/^\[::1\]$/,
/^\[?fe80:/i, // IPv6 link-local
]
```
---
## FIX AFTER LAUNCH (Medium Priority)
### 5. Dozzle Web Shell Access (MEDIUM)
**File:** `install/management_compose.yaml:56`
```yaml
- DOZZLE_ENABLE_SHELL=true
```
Dozzle on port 9999 is bound to all interfaces with shell access enabled. Anyone on the LAN can open a web shell into containers, including `nomad_admin` which has the Docker socket mounted. This creates a path from "LAN access" → "container shell" → "Docker socket" → "host root."
**Fix:** Set `DOZZLE_ENABLE_SHELL=false`. Log viewing and container restart functionality are preserved.
---
### 6. Unvalidated Settings Key Read (MEDIUM)
**File:** `admin/app/controllers/settings_controller.ts`
**Endpoint:** `GET /api/system/settings?key=...`
The `updateSetting` endpoint validates the key against an enum, but `getSetting` accepts any arbitrary key string. Currently harmless since the KV store only contains settings data, but could leak sensitive info if new keys are added.
**Fix:** Apply the same enum validation to the read endpoint.
---
### 7. Content Update URL Injection (MEDIUM)
**File:** `admin/app/validators/common.ts:72-88`
**Endpoint:** `POST /api/content-updates/apply`
The `download_url` comes directly from the client request body. An attacker can supply any URL and NOMAD will download from it. The URL should be looked up server-side from the content manifest instead.
**Fix:** Validate `download_url` against the cached manifest, or apply the same loopback/link-local protections as finding #4 (already applied in this PR).
---
### 8. Verbose Error Messages (MEDIUM)
**Files:** `rag_controller.ts`, `docker_service.ts`, `system_update_service.ts`
Several controllers return raw `error.message` in API responses, potentially leaking internal paths, stack details, or Docker error messages to the client.
**Fix:** Return generic error messages in production. Log the details server-side.
---
### 9. No Rate Limiting (MEDIUM)
Zero rate limiting across all 60+ endpoints. While acceptable for a LAN appliance, some endpoints are particularly abusable:
- `POST /api/benchmark/run` — spins up Docker containers for CPU/memory/disk stress tests
- `POST /api/rag/upload` — file uploads (20MB limit per bodyparser config)
- `POST /api/system/services/affect` — can stop/start any service repeatedly
**Fix:** Consider basic rate limiting on the benchmark and service control endpoints (e.g., 1 benchmark per minute, service actions throttled to prevent rapid cycling).
---
## LOW PRIORITY / ACCEPTED RISK
### 10. CSRF Protection Disabled (LOW)
**File:** `admin/config/shield.ts`
CSRF is disabled, meaning any website a LAN user visits could fire requests at NOMAD's API. This amplifies findings 1-4 — path traversal and SSRF could be triggered by a malicious webpage, not just direct LAN access.
**Assessment:** Acceptable for a LAN appliance with no auth system. Enabling CSRF would require significant auth/session infrastructure changes.
### 11. CORS Wildcard with Credentials (LOW)
**File:** `admin/config/cors.ts`
`origin: ['*']` with `credentials: true`. Standard for LAN appliances.
### 12. npm Dependency CVEs (LOW)
```
tar <=7.5.9 HIGH Hardlink Path Traversal via Drive-Relative Linkpath
ajv <6.14.0 MODERATE ReDoS when using $data option
```
Both fixable via `npm audit fix`. Low practical risk since these are build/dev dependencies not directly exposed to user input.
**Fix:** Run `npm audit fix` and commit the updated lockfile.
### 13. Hardcoded HMAC Secret (LOW)
**File:** `admin/app/services/benchmark_service.ts:35`
The benchmark HMAC secret `'nomad-benchmark-v1-2026'` is hardcoded in open-source code. Anyone can forge leaderboard submissions.
**Assessment:** Accepted risk. The leaderboard has compensating controls (rate limiting, plausibility validation, hardware fingerprint dedup). The secret stops casual abuse, not determined attackers.
### 14. Production Debug Logging (LOW)
**File:** `install/management_compose.yaml:22`
```yaml
LOG_LEVEL=debug
```
Debug logging in production can expose internal state in log files.
**Fix:** Change to `LOG_LEVEL=info` for production compose template.
---
## INFORMATIONAL (By Design)
### No Authentication
All 60+ API endpoints are unauthenticated. This is by design — NOMAD is a LAN appliance and the network boundary is the access control. Issue #73 tracks the edge case of public IP interfaces.
### Docker Socket Exposure
The `nomad_admin` container mounts `/var/run/docker.sock`. This is necessary for NOMAD's core functionality (managing Docker containers). The socket is not exposed to the network — only the admin container can use it.
---
## Recommendations Summary
| Priority | Action | Effort |
|----------|--------|--------|
| **Before launch** | Fix 3 path traversals (ZIM delete, Map delete, Docs read) | ~30 min |
| **Before launch** | Add SSRF protection to download URL validators | ~1 hour |
| **Soon after** | Disable Dozzle shell access | 1 line change |
| **Soon after** | Validate settings key on read endpoint | ~15 min |
| **Soon after** | Sanitize error messages in responses | ~30 min |
| **Nice to have** | Run `npm audit fix` | 5 min |
| **Nice to have** | Change production log level to info | 1 line change |
| **Follow-up** | OWASP ZAP dynamic scan against NOMAD3 | ~1 hour |
---
## What Went Right
- **No hardcoded secrets** — all credentials properly use environment variables
- **No command injection** — Docker operations use the Docker API (dockerode), not shell commands
- **No SQL injection** — all database queries use AdonisJS Lucid ORM with parameterized queries
- **No eval/Function** — no dynamic code execution anywhere
- **RAG service already has the correct fix pattern**`deleteFileBySource()` uses `resolve()` + `startsWith()` for path containment
- **Install script generates strong random passwords** — uses `/dev/urandom` for APP_KEY and DB passwords
- **No privileged containers** — GPU passthrough uses DeviceRequests, not --privileged
- **Health checks don't leak data** — internal-only calls

View File

@ -1,4 +1,4 @@
import { IconCircleCheck } from '@tabler/icons-react'
import { IconCircleCheck, IconCircleX } from '@tabler/icons-react'
import classNames from '~/lib/classNames'
export type InstallActivityFeedProps = {
@ -16,6 +16,12 @@ export type InstallActivityFeedProps = {
| 'started'
| 'finalizing'
| 'completed'
| 'update-pulling'
| 'update-stopping'
| 'update-creating'
| 'update-starting'
| 'update-complete'
| 'update-rollback'
timestamp: string
message: string
}>
@ -40,8 +46,10 @@ const InstallActivityFeed: React.FC<InstallActivityFeedProps> = ({ activity, cla
</div>
<>
<div className="relative flex size-6 flex-none items-center justify-center bg-transparent">
{activityItem.type === 'completed' ? (
{activityItem.type === 'completed' || activityItem.type === 'update-complete' ? (
<IconCircleCheck aria-hidden="true" className="size-6 text-indigo-600" />
) : activityItem.type === 'update-rollback' ? (
<IconCircleX aria-hidden="true" className="size-6 text-red-500" />
) : (
<div className="size-1.5 rounded-full bg-gray-100 ring-1 ring-gray-300" />
)}

View File

@ -20,6 +20,7 @@ const StyledButton: React.FC<StyledButtonProps> = ({
size = 'md',
loading = false,
fullWidth = false,
className,
...props
}) => {
const isDisabled = useMemo(() => {
@ -152,7 +153,8 @@ const StyledButton: React.FC<StyledButtonProps> = ({
getSizeClasses(),
getVariantClasses(),
isDisabled ? 'pointer-events-none opacity-60' : 'cursor-pointer',
'items-center justify-center rounded-md font-semibold focus:outline-none focus:ring-2 focus:ring-desert-green-light focus:ring-offset-2 focus:ring-offset-desert-sand disabled:cursor-not-allowed disabled:shadow-none'
'items-center justify-center rounded-md font-semibold focus:outline-none focus:ring-2 focus:ring-desert-green-light focus:ring-offset-2 focus:ring-offset-desert-sand disabled:cursor-not-allowed disabled:shadow-none',
className
)}
{...props}
disabled={isDisabled}

View File

@ -0,0 +1,145 @@
import { useState } from "react"
import { ServiceSlim } from "../../types/services"
import StyledModal from "./StyledModal"
import { IconArrowUp } from "@tabler/icons-react"
import api from "~/lib/api"
interface UpdateServiceModalProps {
record: ServiceSlim
currentTag: string
latestVersion: string
onCancel: () => void
onUpdate: (version: string) => void
showError: (msg: string) => void
}
export default function UpdateServiceModal({
record,
currentTag,
latestVersion,
onCancel,
onUpdate,
showError,
}: UpdateServiceModalProps) {
const [selectedVersion, setSelectedVersion] = useState(latestVersion)
const [showAdvanced, setShowAdvanced] = useState(false)
const [versions, setVersions] = useState<Array<{ tag: string; isLatest: boolean; releaseUrl?: string }>>([])
const [loadingVersions, setLoadingVersions] = useState(false)
async function loadVersions() {
if (versions.length > 0) return
setLoadingVersions(true)
try {
const result = await api.getAvailableVersions(record.service_name)
if (result?.versions) {
setVersions(result.versions)
}
} catch (error) {
showError('Failed to load available versions')
} finally {
setLoadingVersions(false)
}
}
function handleToggleAdvanced() {
const next = !showAdvanced
setShowAdvanced(next)
if (next) loadVersions()
}
return (
<StyledModal
title="Update Service"
onConfirm={() => onUpdate(selectedVersion)}
onCancel={onCancel}
open={true}
confirmText="Update"
cancelText="Cancel"
confirmVariant="primary"
icon={<IconArrowUp className="h-12 w-12 text-desert-green" />}
>
<div className="space-y-4">
<p className="text-gray-700">
Update <strong>{record.friendly_name || record.service_name}</strong> from{' '}
<code className="bg-gray-100 px-1.5 py-0.5 rounded text-sm">{currentTag}</code> to{' '}
<code className="bg-gray-100 px-1.5 py-0.5 rounded text-sm">{selectedVersion}</code>?
</p>
<p className="text-sm text-gray-500">
Your data and configuration will be preserved during the update.
{versions.find((v) => v.tag === selectedVersion)?.releaseUrl && (
<>
{' '}
<a
href={versions.find((v) => v.tag === selectedVersion)!.releaseUrl}
target="_blank"
rel="noopener noreferrer"
className="text-desert-green hover:underline"
>
View release notes
</a>
</>
)}
</p>
<div>
<button
type="button"
onClick={handleToggleAdvanced}
className="text-sm text-desert-green hover:underline font-medium"
>
{showAdvanced ? 'Hide' : 'Show'} available versions
</button>
{showAdvanced && (
<>
<div className="mt-3 max-h-48 overflow-y-auto border rounded-lg divide-y">
{loadingVersions ? (
<div className="p-4 text-center text-gray-500 text-sm">Loading versions...</div>
) : versions.length === 0 ? (
<div className="p-4 text-center text-gray-500 text-sm">No other versions available</div>
) : (
versions.map((v) => (
<label
key={v.tag}
className="flex items-center gap-3 px-4 py-2.5 hover:bg-gray-50 cursor-pointer"
>
<input
type="radio"
name="version"
value={v.tag}
checked={selectedVersion === v.tag}
onChange={() => setSelectedVersion(v.tag)}
className="text-desert-green focus:ring-desert-green"
/>
<span className="text-sm font-medium text-gray-900">{v.tag}</span>
{v.isLatest && (
<span className="text-xs bg-desert-green/10 text-desert-green px-2 py-0.5 rounded-full">
Latest
</span>
)}
{v.releaseUrl && (
<a
href={v.releaseUrl}
target="_blank"
rel="noopener noreferrer"
className="ml-auto text-xs text-desert-green hover:underline"
onClick={(e) => e.stopPropagation()}
>
Release notes
</a>
)}
</label>
))
)}
</div>
<p className="mt-2 text-sm text-gray-500">
It's not recommended to upgrade to a new major version (e.g. 1.8.2 &rarr; 2.0.0) unless you have verified compatibility with your current configuration. Always review the release notes and test in a staging environment if possible.
</p>
</>
)}
</div>
</div>
</StyledModal>
)
}

View File

@ -10,6 +10,7 @@ import { ChatMessage } from '../../../types/chat'
import classNames from '~/lib/classNames'
import { IconX } from '@tabler/icons-react'
import { DEFAULT_QUERY_REWRITE_MODEL } from '../../../constants/ollama'
import { useSystemSetting } from '~/hooks/useSystemSetting'
interface ChatProps {
enabled: boolean
@ -51,6 +52,8 @@ export default function Chat({
const activeSession = sessions.find((s) => s.id === activeSessionId)
const { data: lastModelSetting } = useSystemSetting({ key: 'chat.lastModel', enabled })
const { data: installedModels = [], isLoading: isLoadingModels } = useQuery({
queryKey: ['installedModels'],
queryFn: () => api.getInstalledModels(),
@ -87,8 +90,9 @@ export default function Chat({
mutationFn: (request: {
model: string
messages: Array<{ role: 'system' | 'user' | 'assistant'; content: string }>
sessionId?: number
}) => api.sendChatMessage({ ...request, stream: false }),
onSuccess: async (data, variables) => {
onSuccess: async (data) => {
if (!data || !activeSessionId) {
throw new Error('No response from Ollama')
}
@ -103,17 +107,9 @@ export default function Chat({
setMessages((prev) => [...prev, assistantMessage])
// Save assistant message to backend
await api.addChatMessage(activeSessionId, 'assistant', assistantMessage.content)
// Update session title if it's a new chat
const currentSession = sessions.find((s) => s.id === activeSessionId)
if (currentSession && currentSession.title === 'New Chat') {
const userContent = variables.messages[variables.messages.length - 1].content
const newTitle = userContent.slice(0, 50) + (userContent.length > 50 ? '...' : '')
await api.updateChatSession(activeSessionId, { title: newTitle })
queryClient.invalidateQueries({ queryKey: ['chatSessions'] })
}
// Refresh sessions to pick up backend-persisted messages and title
queryClient.invalidateQueries({ queryKey: ['chatSessions'] })
setTimeout(() => queryClient.invalidateQueries({ queryKey: ['chatSessions'] }), 3000)
},
onError: (error) => {
console.error('Error sending message:', error)
@ -127,12 +123,24 @@ export default function Chat({
},
})
// Set first model as selected by default
// Set default model: prefer last used model, fall back to first installed if last model not available
useEffect(() => {
if (installedModels.length > 0 && !selectedModel) {
setSelectedModel(installedModels[0].name)
const lastModel = lastModelSetting?.value as string | undefined
if (lastModel && installedModels.some((m) => m.name === lastModel)) {
setSelectedModel(lastModel)
} else {
setSelectedModel(installedModels[0].name)
}
}
}, [installedModels, selectedModel])
}, [installedModels, selectedModel, lastModelSetting])
// Persist model selection
useEffect(() => {
if (selectedModel) {
api.updateSetting('chat.lastModel', selectedModel)
}
}, [selectedModel])
const handleNewChat = useCallback(() => {
// Just clear the active session and messages - don't create a session yet
@ -215,9 +223,6 @@ export default function Chat({
setMessages((prev) => [...prev, userMessage])
// Save user message to backend
await api.addChatMessage(sessionId, 'user', content)
const chatMessages = [
...messages.map((m) => ({ role: m.role, content: m.content })),
{ role: 'user' as const, content },
@ -240,7 +245,7 @@ export default function Chat({
try {
await api.streamChatMessage(
{ model: selectedModel || 'llama3.2', messages: chatMessages, stream: true },
{ model: selectedModel || 'llama3.2', messages: chatMessages, stream: true, sessionId: sessionId ? Number(sessionId) : undefined },
(chunkContent, chunkThinking, done) => {
if (chunkThinking.length > 0 && thinkingStartTime === null) {
thinkingStartTime = Date.now()
@ -321,24 +326,20 @@ export default function Chat({
)
)
await api.addChatMessage(sessionId, 'assistant', fullContent)
const currentSession = sessions.find((s) => s.id === sessionId)
if (currentSession && currentSession.title === 'New Chat') {
const newTitle = content.slice(0, 50) + (content.length > 50 ? '...' : '')
await api.updateChatSession(sessionId, { title: newTitle })
queryClient.invalidateQueries({ queryKey: ['chatSessions'] })
}
// Refresh sessions to pick up backend-persisted messages and title
queryClient.invalidateQueries({ queryKey: ['chatSessions'] })
setTimeout(() => queryClient.invalidateQueries({ queryKey: ['chatSessions'] }), 3000)
}
} else {
// Non-streaming (legacy) path
chatMutation.mutate({
model: selectedModel || 'llama3.2',
messages: chatMessages,
sessionId: sessionId ? Number(sessionId) : undefined,
})
}
},
[activeSessionId, messages, selectedModel, chatMutation, queryClient, streamingEnabled, sessions]
[activeSessionId, messages, selectedModel, chatMutation, queryClient, streamingEnabled]
)
return (

View File

@ -13,12 +13,15 @@ import {
import { usePage } from '@inertiajs/react'
import StyledSidebar from '~/components/StyledSidebar'
import { getServiceLink } from '~/lib/navigation'
import useServiceInstalledStatus from '~/hooks/useServiceInstalledStatus'
import { SERVICE_NAMES } from '../../constants/service_names'
export default function SettingsLayout({ children }: { children: React.ReactNode }) {
const { aiAssistantName } = usePage<{ aiAssistantName: string }>().props
const aiAssistantInstallStatus = useServiceInstalledStatus(SERVICE_NAMES.OLLAMA)
const navigation = [
{ name: aiAssistantName, href: '/settings/models', icon: IconWand, current: false },
...(aiAssistantInstallStatus.isInstalled ? [{ name: aiAssistantName, href: '/settings/models', icon: IconWand, current: false }] : []),
{ name: 'Apps', href: '/settings/apps', icon: IconTerminal2, current: false },
{ name: 'Benchmark', href: '/settings/benchmark', icon: IconChartBar, current: false },
{ name: 'Content Explorer', href: '/settings/zim/remote-explorer', icon: IconZoom, current: false },

View File

@ -163,6 +163,34 @@ class API {
})()
}
async checkServiceUpdates() {
return catchInternal(async () => {
const response = await this.client.post<{ success: boolean; message: string }>(
'/system/services/check-updates'
)
return response.data
})()
}
async getAvailableVersions(serviceName: string) {
return catchInternal(async () => {
const response = await this.client.get<{
versions: Array<{ tag: string; isLatest: boolean; releaseUrl?: string }>
}>(`/system/services/${serviceName}/available-versions`)
return response.data
})()
}
async updateService(serviceName: string, targetVersion: string) {
return catchInternal(async () => {
const response = await this.client.post<{ success: boolean; message: string }>(
'/system/services/update',
{ service_name: serviceName, target_version: targetVersion }
)
return response.data
})()
}
async forceReinstallService(service_name: string) {
return catchInternal(async () => {
const response = await this.client.post<{ success: boolean; message: string }>(
@ -197,7 +225,7 @@ class API {
})()
}
async getAvailableModels(params: { query?: string; recommendedOnly?: boolean; limit?: number }) {
async getAvailableModels(params: { query?: string; recommendedOnly?: boolean; limit?: number; force?: boolean }) {
return catchInternal(async () => {
const response = await this.client.get<{
models: NomadOllamaModel[]

View File

@ -43,7 +43,7 @@ function buildCoreCapabilities(aiAssistantName: string): Capability[] {
features: [
'Complete Wikipedia offline',
'Medical references and first aid guides',
'WikiHow articles and tutorials',
'DIY repair guides and how-to content',
'Project Gutenberg books and literature',
],
services: [SERVICE_NAMES.KIWIX],

View File

@ -13,27 +13,68 @@ import LoadingSpinner from '~/components/LoadingSpinner'
import useErrorNotification from '~/hooks/useErrorNotification'
import useInternetStatus from '~/hooks/useInternetStatus'
import useServiceInstallationActivity from '~/hooks/useServiceInstallationActivity'
import { IconCheck, IconDownload } from '@tabler/icons-react'
import { useTransmit } from 'react-adonis-transmit'
import { BROADCAST_CHANNELS } from '../../../constants/broadcast'
import { IconArrowUp, IconCheck, IconDownload } from '@tabler/icons-react'
import UpdateServiceModal from '~/components/UpdateServiceModal'
function extractTag(containerImage: string): string {
if (!containerImage) return ''
const parts = containerImage.split(':')
return parts.length > 1 ? parts[parts.length - 1] : 'latest'
}
export default function SettingsPage(props: { system: { services: ServiceSlim[] } }) {
const { openModal, closeAllModals } = useModals()
const { showError } = useErrorNotification()
const { isOnline } = useInternetStatus()
const { subscribe } = useTransmit()
const installActivity = useServiceInstallationActivity()
const [isInstalling, setIsInstalling] = useState(false)
const [loading, setLoading] = useState(false)
const [checkingUpdates, setCheckingUpdates] = useState(false)
useEffect(() => {
if (installActivity.length === 0) return
if (installActivity.some((activity) => activity.type === 'completed')) {
// If any activity is completed, we can clear the installActivity state
if (
installActivity.some(
(activity) => activity.type === 'completed' || activity.type === 'update-complete'
)
) {
setTimeout(() => {
window.location.reload() // Reload the page to reflect changes
}, 3000) // Clear after 3 seconds
window.location.reload()
}, 3000)
}
}, [installActivity])
// Listen for service update check completion
useEffect(() => {
const unsubscribe = subscribe(BROADCAST_CHANNELS.SERVICE_UPDATES, () => {
setCheckingUpdates(false)
window.location.reload()
})
return () => { unsubscribe() }
}, [])
async function handleCheckUpdates() {
try {
if (!isOnline) {
showError('You must have an internet connection to check for updates.')
return
}
setCheckingUpdates(true)
const response = await api.checkServiceUpdates()
if (!response?.success) {
throw new Error('Failed to dispatch update check')
}
} catch (error) {
console.error('Error checking for updates:', error)
showError(`Failed to check for updates: ${error.message || 'Unknown error'}`)
setCheckingUpdates(false)
}
}
const handleInstallService = (service: ServiceSlim) => {
openModal(
<StyledModal
@ -97,8 +138,8 @@ export default function SettingsPage(props: { system: { services: ServiceSlim[]
setTimeout(() => {
setLoading(false)
window.location.reload() // Reload the page to reflect changes
}, 3000) // Add small delay to allow for the action to complete
window.location.reload()
}, 3000)
} catch (error) {
console.error(`Error affecting service ${record.service_name}:`, error)
showError(`Failed to ${action} service: ${error.message || 'Unknown error'}`)
@ -120,14 +161,44 @@ export default function SettingsPage(props: { system: { services: ServiceSlim[]
setTimeout(() => {
setLoading(false)
window.location.reload() // Reload the page to reflect changes
}, 3000) // Add small delay to allow for the action to complete
window.location.reload()
}, 3000)
} catch (error) {
console.error(`Error force reinstalling service ${record.service_name}:`, error)
showError(`Failed to force reinstall service: ${error.message || 'Unknown error'}`)
}
}
function handleUpdateService(record: ServiceSlim) {
const currentTag = extractTag(record.container_image)
const latestVersion = record.available_update_version!
openModal(
<UpdateServiceModal
record={record}
currentTag={currentTag}
latestVersion={latestVersion}
onCancel={closeAllModals}
onUpdate={async (targetVersion: string) => {
closeAllModals()
try {
setLoading(true)
const response = await api.updateService(record.service_name, targetVersion)
if (!response?.success) {
throw new Error(response?.message || 'Update failed')
}
} catch (error) {
console.error(`Error updating service ${record.service_name}:`, error)
showError(`Failed to update service: ${error.message || 'Unknown error'}`)
setLoading(false)
}
}}
showError={showError}
/>,
`${record.service_name}-update-modal`
)
}
const AppActions = ({ record }: { record: ServiceSlim }) => {
const ForceReinstallButton = () => (
<StyledButton
@ -162,7 +233,7 @@ export default function SettingsPage(props: { system: { services: ServiceSlim[]
if (!record) return null
if (!record.installed) {
return (
<div className="flex space-x-2">
<div className="flex flex-wrap gap-2">
<StyledButton
icon={'IconDownload'}
variant="primary"
@ -178,7 +249,7 @@ export default function SettingsPage(props: { system: { services: ServiceSlim[]
}
return (
<div className="flex space-x-2">
<div className="flex flex-wrap gap-2">
<StyledButton
icon={'IconExternalLink'}
onClick={() => {
@ -187,6 +258,16 @@ export default function SettingsPage(props: { system: { services: ServiceSlim[]
>
Open
</StyledButton>
{record.available_update_version && (
<StyledButton
icon="IconArrowUp"
variant="primary"
onClick={() => handleUpdateService(record)}
disabled={isInstalling || !isOnline}
>
Update
</StyledButton>
)}
{record.status && record.status !== 'unknown' && (
<>
<StyledButton
@ -254,14 +335,26 @@ export default function SettingsPage(props: { system: { services: ServiceSlim[]
<Head title="App Settings" />
<div className="xl:pl-72 w-full">
<main className="px-12 py-6">
<h1 className="text-4xl font-semibold mb-4">Apps</h1>
<p className="text-gray-500 mb-4">
Manage the applications that are available in your Project N.O.M.A.D. instance.
</p>
<div className="flex items-center justify-between mb-4">
<div>
<h1 className="text-4xl font-semibold">Apps</h1>
<p className="text-gray-500 mt-1">
Manage the applications that are available in your Project N.O.M.A.D. instance. Nightly update checks will automatically detect when new versions of these apps are available.
</p>
</div>
<StyledButton
icon="IconRefreshAlert"
onClick={handleCheckUpdates}
disabled={checkingUpdates || !isOnline}
loading={checkingUpdates}
>
Check for Updates
</StyledButton>
</div>
{loading && <LoadingSpinner fullscreen />}
{!loading && (
<StyledTable<ServiceSlim & { actions?: any }>
className="font-semibold"
className="font-semibold !overflow-x-auto"
rowLines={true}
columns={[
{
@ -296,9 +389,30 @@ export default function SettingsPage(props: { system: { services: ServiceSlim[]
render: (record) =>
record.installed ? <IconCheck className="h-6 w-6 text-desert-green" /> : '',
},
{
accessor: 'container_image',
title: 'Version',
render: (record) => {
if (!record.installed) return null
const currentTag = extractTag(record.container_image)
if (record.available_update_version) {
return (
<div className="flex items-center gap-1.5">
<span className="text-gray-500">{currentTag}</span>
<IconArrowUp className="h-4 w-4 text-desert-green" />
<span className="text-desert-green font-semibold">
{record.available_update_version}
</span>
</div>
)
}
return <span className="text-gray-600">{currentTag}</span>
},
},
{
accessor: 'actions',
title: 'Actions',
className: '!whitespace-normal',
render: (record) => <AppActions record={record} />,
},
]}
@ -313,3 +427,4 @@ export default function SettingsPage(props: { system: { services: ServiceSlim[]
</SettingsLayout>
)
}

View File

@ -1,5 +1,5 @@
import { Head, router, usePage } from '@inertiajs/react'
import { useState } from 'react'
import { useRef, useState } from 'react'
import StyledTable from '~/components/StyledTable'
import SettingsLayout from '~/layouts/SettingsLayout'
import { NomadOllamaModel } from '../../../types/ollama'
@ -16,9 +16,10 @@ import Switch from '~/components/inputs/Switch'
import StyledSectionHeader from '~/components/StyledSectionHeader'
import { useMutation, useQuery } from '@tanstack/react-query'
import Input from '~/components/inputs/Input'
import { IconSearch } from '@tabler/icons-react'
import { IconSearch, IconRefresh } from '@tabler/icons-react'
import useDebounce from '~/hooks/useDebounce'
import ActiveModelDownloads from '~/components/ActiveModelDownloads'
import { useSystemInfo } from '~/hooks/useSystemInfo'
export default function ModelsPage(props: {
models: {
@ -32,6 +33,64 @@ export default function ModelsPage(props: {
const { addNotification } = useNotifications()
const { openModal, closeAllModals } = useModals()
const { debounce } = useDebounce()
const { data: systemInfo } = useSystemInfo({})
const [gpuBannerDismissed, setGpuBannerDismissed] = useState(() => {
try {
return localStorage.getItem('nomad:gpu-banner-dismissed') === 'true'
} catch {
return false
}
})
const [reinstalling, setReinstalling] = useState(false)
const handleDismissGpuBanner = () => {
setGpuBannerDismissed(true)
try {
localStorage.setItem('nomad:gpu-banner-dismissed', 'true')
} catch {}
}
const handleForceReinstallOllama = () => {
openModal(
<StyledModal
title="Reinstall AI Assistant?"
onConfirm={async () => {
closeAllModals()
setReinstalling(true)
try {
const response = await api.forceReinstallService('nomad_ollama')
if (!response || !response.success) {
throw new Error(response?.message || 'Force reinstall failed')
}
addNotification({
message: `${aiAssistantName} is being reinstalled with GPU support. This page will reload shortly.`,
type: 'success',
})
try { localStorage.removeItem('nomad:gpu-banner-dismissed') } catch {}
setTimeout(() => window.location.reload(), 5000)
} catch (error) {
addNotification({
message: `Failed to reinstall: ${error instanceof Error ? error.message : 'Unknown error'}`,
type: 'error',
})
setReinstalling(false)
}
}}
onCancel={closeAllModals}
open={true}
confirmText="Reinstall"
cancelText="Cancel"
>
<p className="text-gray-700">
This will recreate the {aiAssistantName} container with GPU support enabled.
Your downloaded models will be preserved. The service will be briefly
unavailable during reinstall.
</p>
</StyledModal>,
'gpu-health-force-reinstall-modal'
)
}
const [chatSuggestionsEnabled, setChatSuggestionsEnabled] = useState(
props.models.settings.chatSuggestionsEnabled
)
@ -47,13 +106,19 @@ export default function ModelsPage(props: {
setQuery(val)
}, 300)
const { data: availableModelData, isFetching } = useQuery({
const forceRefreshRef = useRef(false)
const [isForceRefreshing, setIsForceRefreshing] = useState(false)
const { data: availableModelData, isFetching, refetch } = useQuery({
queryKey: ['ollama', 'availableModels', query, limit],
queryFn: async () => {
const force = forceRefreshRef.current
forceRefreshRef.current = false
const res = await api.getAvailableModels({
query,
recommendedOnly: false,
limit,
force: force || undefined,
})
if (!res) {
return {
@ -66,6 +131,14 @@ export default function ModelsPage(props: {
initialData: { models: props.models.availableModels, hasMore: false },
})
async function handleForceRefresh() {
forceRefreshRef.current = true
setIsForceRefreshing(true)
await refetch()
setIsForceRefreshing(false)
addNotification({ message: 'Model list refreshed from remote.', type: 'success' })
}
async function handleInstallModel(modelName: string) {
try {
const res = await api.downloadModel(modelName)
@ -164,6 +237,26 @@ export default function ModelsPage(props: {
className="!mt-6"
/>
)}
{isInstalled && systemInfo?.gpuHealth?.status === 'passthrough_failed' && !gpuBannerDismissed && (
<Alert
type="warning"
variant="bordered"
title="GPU Not Accessible"
message={`Your system has an NVIDIA GPU, but ${aiAssistantName} can't access it. AI is running on CPU only, which is significantly slower.`}
className="!mt-6"
dismissible={true}
onDismiss={handleDismissGpuBanner}
buttonProps={{
children: `Fix: Reinstall ${aiAssistantName}`,
icon: 'IconRefresh',
variant: 'action',
size: 'sm',
onClick: handleForceReinstallOllama,
loading: reinstalling,
disabled: reinstalling,
}}
/>
)}
<StyledSectionHeader title="Settings" className="mt-8 mb-4" />
<div className="bg-white rounded-lg border-2 border-gray-200 p-6">
@ -196,7 +289,7 @@ export default function ModelsPage(props: {
<ActiveModelDownloads withHeader />
<StyledSectionHeader title="Models" className="mt-12 mb-4" />
<div className="flex justify-start mt-4">
<div className="flex justify-start items-center gap-3 mt-4">
<Input
name="search"
label=""
@ -209,6 +302,15 @@ export default function ModelsPage(props: {
className="w-1/3"
leftIcon={<IconSearch className="w-5 h-5 text-gray-400" />}
/>
<StyledButton
variant="secondary"
onClick={handleForceRefresh}
icon="IconRefresh"
loading={isForceRefreshing}
className='mt-1'
>
Refresh Models
</StyledButton>
</div>
<StyledTable<NomadOllamaModel>
className="font-semibold mt-4"

View File

@ -1,3 +1,4 @@
import { useState } from 'react'
import { Head } from '@inertiajs/react'
import SettingsLayout from '~/layouts/SettingsLayout'
import { SystemInformationResponse } from '../../../types/system'
@ -6,7 +7,11 @@ import CircularGauge from '~/components/systeminfo/CircularGauge'
import HorizontalBarChart from '~/components/HorizontalBarChart'
import InfoCard from '~/components/systeminfo/InfoCard'
import Alert from '~/components/Alert'
import StyledModal from '~/components/StyledModal'
import { useSystemInfo } from '~/hooks/useSystemInfo'
import { useNotifications } from '~/context/NotificationContext'
import { useModals } from '~/context/ModalContext'
import api from '~/lib/api'
import StatusCard from '~/components/systeminfo/StatusCard'
import { IconCpu, IconDatabase, IconServer, IconDeviceDesktop, IconComponents } from '@tabler/icons-react'
@ -16,6 +21,65 @@ export default function SettingsPage(props: {
const { data: info } = useSystemInfo({
initialData: props.system.info,
})
const { addNotification } = useNotifications()
const { openModal, closeAllModals } = useModals()
const [gpuBannerDismissed, setGpuBannerDismissed] = useState(() => {
try {
return localStorage.getItem('nomad:gpu-banner-dismissed') === 'true'
} catch {
return false
}
})
const [reinstalling, setReinstalling] = useState(false)
const handleDismissGpuBanner = () => {
setGpuBannerDismissed(true)
try {
localStorage.setItem('nomad:gpu-banner-dismissed', 'true')
} catch {}
}
const handleForceReinstallOllama = () => {
openModal(
<StyledModal
title="Reinstall AI Assistant?"
onConfirm={async () => {
closeAllModals()
setReinstalling(true)
try {
const response = await api.forceReinstallService('nomad_ollama')
if (!response || !response.success) {
throw new Error(response?.message || 'Force reinstall failed')
}
addNotification({
message: 'AI Assistant is being reinstalled with GPU support. This page will reload shortly.',
type: 'success',
})
try { localStorage.removeItem('nomad:gpu-banner-dismissed') } catch {}
setTimeout(() => window.location.reload(), 5000)
} catch (error) {
addNotification({
message: `Failed to reinstall: ${error instanceof Error ? error.message : 'Unknown error'}`,
type: 'error',
})
setReinstalling(false)
}
}}
onCancel={closeAllModals}
open={true}
confirmText="Reinstall"
cancelText="Cancel"
>
<p className="text-gray-700">
This will recreate the AI Assistant container with GPU support enabled.
Your downloaded models will be preserved. The service will be briefly
unavailable during reinstall.
</p>
</StyledModal>,
'gpu-health-force-reinstall-modal'
)
}
// Use (total - available) to reflect actual memory pressure.
// mem.used includes reclaimable buff/cache on Linux, which inflates the number.
@ -173,6 +237,27 @@ export default function SettingsPage(props: {
},
]}
/>
{info?.gpuHealth?.status === 'passthrough_failed' && !gpuBannerDismissed && (
<div className="lg:col-span-2">
<Alert
type="warning"
variant="bordered"
title="GPU Not Accessible to AI Assistant"
message="Your system has an NVIDIA GPU, but the AI Assistant can't access it. AI is running on CPU only, which is significantly slower."
dismissible={true}
onDismiss={handleDismissGpuBanner}
buttonProps={{
children: 'Fix: Reinstall AI Assistant',
icon: 'IconRefresh',
variant: 'action',
size: 'sm',
onClick: handleForceReinstallOllama,
loading: reinstalling,
disabled: reinstalling,
}}
/>
</div>
)}
{info?.graphics?.controllers && info.graphics.controllers.length > 0 && (
<InfoCard
title="Graphics"

View File

@ -66,7 +66,7 @@
"stopword": "^3.1.5",
"systeminformation": "^5.30.8",
"tailwindcss": "^4.1.10",
"tar": "^7.5.9",
"tar": "^7.5.10",
"tesseract.js": "^7.0.0",
"url-join": "^5.0.0",
"yaml": "^2.8.0"
@ -4379,7 +4379,6 @@
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0 AND MIT",
"optional": true,
"os": [
@ -4396,7 +4395,6 @@
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0 AND MIT",
"optional": true,
"os": [
@ -4413,7 +4411,6 @@
"cpu": [
"arm"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
@ -4430,7 +4427,6 @@
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0 AND MIT",
"optional": true,
"os": [
@ -4447,7 +4443,6 @@
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0 AND MIT",
"optional": true,
"os": [
@ -4464,7 +4459,6 @@
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0 AND MIT",
"optional": true,
"os": [
@ -4481,7 +4475,6 @@
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0 AND MIT",
"optional": true,
"os": [
@ -4498,7 +4491,6 @@
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0 AND MIT",
"optional": true,
"os": [
@ -4515,7 +4507,6 @@
"cpu": [
"ia32"
],
"dev": true,
"license": "Apache-2.0 AND MIT",
"optional": true,
"os": [
@ -4532,7 +4523,6 @@
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0 AND MIT",
"optional": true,
"os": [
@ -15275,9 +15265,9 @@
}
},
"node_modules/tar": {
"version": "7.5.9",
"resolved": "https://registry.npmjs.org/tar/-/tar-7.5.9.tgz",
"integrity": "sha512-BTLcK0xsDh2+PUe9F6c2TlRp4zOOBMTkoQHQIWSIzI0R7KG46uEwq4OPk2W7bZcprBMsuaeFsqwYr7pjh6CuHg==",
"version": "7.5.10",
"resolved": "https://registry.npmjs.org/tar/-/tar-7.5.10.tgz",
"integrity": "sha512-8mOPs1//5q/rlkNSPcCegA6hiHJYDmSLEI8aMH/CdSQJNWztHC9WHNam5zdQlfpTwB9Xp7IBEsHfV5LKMJGVAw==",
"license": "BlueOak-1.0.0",
"dependencies": {
"@isaacs/fs-minipass": "^4.0.0",

View File

@ -118,7 +118,7 @@
"stopword": "^3.1.5",
"systeminformation": "^5.30.8",
"tailwindcss": "^4.1.10",
"tar": "^7.5.9",
"tar": "^7.5.10",
"tesseract.js": "^7.0.0",
"url-join": "^5.0.0",
"yaml": "^2.8.0"

View File

@ -141,6 +141,9 @@ router
router.post('/services/affect', [SystemController, 'affectService'])
router.post('/services/install', [SystemController, 'installService'])
router.post('/services/force-reinstall', [SystemController, 'forceReinstallService'])
router.post('/services/check-updates', [SystemController, 'checkServiceUpdates'])
router.get('/services/:name/available-versions', [SystemController, 'getAvailableVersions'])
router.post('/services/update', [SystemController, 'updateService'])
router.post('/subscribe-release-notes', [SystemController, 'subscribeToReleaseNotes'])
router.get('/latest-version', [SystemController, 'checkLatestVersion'])
router.post('/update', [SystemController, 'requestSystemUpdate'])

View File

@ -1,6 +1,7 @@
export const KV_STORE_SCHEMA = {
'chat.suggestionsEnabled': 'boolean',
'chat.lastModel': 'string',
'rag.docsEmbedded': 'boolean',
'system.updateAvailable': 'boolean',
'system.latestVersion': 'string',

View File

@ -32,6 +32,7 @@ export type OllamaChatRequest = {
model: string
messages: OllamaChatMessage[]
stream?: boolean
sessionId?: number
}
export type OllamaChatResponse = {

View File

@ -5,3 +5,32 @@ export type EmbedJobWithProgress = {
progress: number
status: string
}
export type ProcessAndEmbedFileResponse = {
success: boolean
message: string
chunks?: number
hasMoreBatches?: boolean
articlesProcessed?: number
totalArticles?: number
}
export type ProcessZIMFileResponse = ProcessAndEmbedFileResponse
export type RAGResult = {
text: string
score: number
keywords: string
chunk_index: number
created_at: number
article_title?: string
section_title?: string
full_title?: string
hierarchy?: string
document_id?: string
content_type?: string
source?: string
}
export type RerankedRAGResult = Omit<RAGResult, 'keywords'> & {
finalScore: number
}

View File

@ -12,4 +12,6 @@ export type ServiceSlim = Pick<
| 'icon'
| 'powered_by'
| 'display_order'
| 'container_image'
| 'available_update_version'
> & { status?: string }

View File

@ -1,5 +1,11 @@
import { Systeminformation } from 'systeminformation'
export type GpuHealthStatus = {
status: 'ok' | 'passthrough_failed' | 'no_gpu' | 'ollama_not_installed'
hasNvidiaRuntime: boolean
ollamaGpuAccessible: boolean
}
export type SystemInformationResponse = {
cpu: Systeminformation.CpuData
mem: Systeminformation.MemData
@ -9,6 +15,7 @@ export type SystemInformationResponse = {
fsSize: Systeminformation.FsSizeData[]
uptime: Systeminformation.TimeData
graphics: Systeminformation.GraphicsData
gpuHealth?: GpuHealthStatus
}
// Type inferrence is not working properly with usePage and shared props, so we define this type manually

View File

@ -156,7 +156,7 @@
{
"name": "Comprehensive",
"slug": "survival-comprehensive",
"description": "Complete prepper video library with long-term food storage strategies. Includes everything in Standard.",
"description": "Complete prepper library with food storage strategies and classic military strategy. Includes everything in Standard.",
"includesTier": "survival-standard",
"resources": [
{
@ -166,6 +166,14 @@
"description": "Long-term food storage and survival meal preparation",
"url": "https://download.kiwix.org/zim/videos/canadian_prepper_preppingfood_en_2025-09.zim",
"size_mb": 2160
},
{
"id": "gutenberg_en_lcc-u",
"version": "2026-03",
"title": "Project Gutenberg: Military Science",
"description": "Classic military strategy, tactics, and field manuals",
"url": "https://download.kiwix.org/zim/gutenberg/gutenberg_en_lcc-u_2026-03.zim",
"size_mb": 1200
}
]
}
@ -247,14 +255,6 @@
"description": "Biology courses and textbooks",
"url": "https://download.kiwix.org/zim/libretexts/libretexts.org_en_bio_2025-01.zim",
"size_mb": 2240
},
{
"id": "gutenberg_en_education",
"version": "2025-12",
"title": "Project Gutenberg: Education",
"description": "Classic educational texts and resources",
"url": "https://download.kiwix.org/zim/gutenberg/gutenberg_en_education_2025-12.zim",
"size_mb": 606
}
]
},
@ -447,7 +447,7 @@
{
"name": "Comprehensive",
"slug": "agriculture-comprehensive",
"description": "Complete self-sufficiency with homesteading videos and advanced techniques. Includes Standard.",
"description": "Complete self-sufficiency with homesteading videos, classic agricultural texts, and advanced techniques. Includes Standard.",
"includesTier": "agriculture-standard",
"resources": [
{
@ -457,6 +457,14 @@
"description": "Beekeeping, animal husbandry, and sustainable living practices",
"url": "https://download.kiwix.org/zim/videos/lrnselfreliance_en_all_2025-12.zim",
"size_mb": 3970
},
{
"id": "gutenberg_en_lcc-s",
"version": "2026-03",
"title": "Project Gutenberg: Agriculture",
"description": "Classic texts on farming, animal husbandry, plant cultivation, and food preservation",
"url": "https://download.kiwix.org/zim/gutenberg/gutenberg_en_lcc-s_2026-03.zim",
"size_mb": 4300
}
]
}

View File

@ -1,6 +1,6 @@
{
"name": "project-nomad",
"version": "1.28.0",
"version": "1.29.0-rc.5",
"description": "\"",
"main": "index.js",
"scripts": {