docs(release): finalize v1.35.1 release notes [skip ci]

Ollama 0.33.3 gives two answers for gemma3: /api/tags lists it as
['completion'], while /api/show for the same model reports
['completion', 'vision']. getModelCapabilities trusted the list outright
and cached it for the life of the process, so image upload stayed disabled
for gemma3, one of the most common small vision models, even though the
model downloader correctly shows it as text + image. Re-pulling does not
help: the local digest matches the registry.

A list that reports vision is still trusted without probing. A list that
omits it is no longer conclusive: the model is probed through /api/show
(then llama.cpp /props), and the list's answer stands only when the probes
have nothing better, which keeps hybrid routers without /api/show working.
The service's own copy of the short-circuit is removed so the rule lives in
the one tested function.

Verified on NOMAD3 (v1.35.0 + this patch): installed-models moves gemma3:4b
from unsupported to supported, every other model unchanged, and a chat with
an attached image is answered correctly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

.gitattributes

@@ -0,0 +1,5 @@
+# Shell scripts must keep LF endings. These are copied into Linux container
+# images, where a CRLF after the shebang makes the interpreter unresolvable and
+# the container exits immediately. Checkouts on Windows would otherwise convert
+# them, which only shows up when building an image locally rather than in CI.
+*.sh text eol=lf

.github/ISSUE_TEMPLATE/config.yml

@@ -1,5 +1,8 @@
 blank_issues_enabled: false
 contact_links:
+  - name: 🔒 Report a Security Vulnerability
+    url: https://github.com/Crosstalk-Solutions/project-nomad/security/advisories/new
+    about: Report a security issue privately. Please do not open a public issue for a vulnerability.
   - name: 💬 Discord Community
     url: https://discord.com/invite/crosstalksolutions
     about: Join our Discord community for general questions, support, and discussions

.github/workflows/build-translate-proxy.yml

@@ -0,0 +1,52 @@
+name: Build Translate Proxy Image
+
+on:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: 'Semantic version to label the Docker image under (no "v" prefix, e.g. "1.2.3")'
+        required: true
+        type: string
+      tag_latest:
+        description: 'Also tag this image as :latest?'
+        required: false
+        type: boolean
+        default: false
+
+jobs:
+  check_authorization:
+    name: Check authorization to publish new Docker image
+    runs-on: ubuntu-latest
+    outputs:
+      isAuthorized: ${{ steps.check-auth.outputs.is_authorized }}
+    steps:
+      - name: check-auth
+        id: check-auth
+        run: echo "is_authorized=${{ contains(secrets.DEPLOYMENT_AUTHORIZED_USERS, github.triggering_actor) }}" >> $GITHUB_OUTPUT
+  build:
+    name: Build translate-proxy image
+    needs: check_authorization
+    if: needs.check_authorization.outputs.isAuthorized == 'true'
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v6
+      - name: Log in to GitHub Container Registry
+        uses: docker/login-action@v4
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - name: Build and push
+        uses: docker/build-push-action@v7
+        with:
+          context: install/nomad-translate
+          platforms: linux/amd64
+          push: true
+          tags: |
+            ghcr.io/crosstalk-solutions/project-nomad-translate:${{ inputs.version }}
+            ghcr.io/crosstalk-solutions/project-nomad-translate:v${{ inputs.version }}
+            ${{ inputs.tag_latest && 'ghcr.io/crosstalk-solutions/project-nomad-translate:latest' || '' }}

.github/workflows/validate-collection-urls.yml

@@ -7,6 +7,8 @@ on:
   pull_request:
     paths:
       - 'collections/**.json'
+  schedule:
+    - cron: '17 6 * * 1'
 
 jobs:
   validate-urls:

.gitignore

@@ -16,6 +16,31 @@ node_modules/
 
 # dotenv environment variables file
 .env
+.env.*
+!.env.example
+
+# Credentials and keys. Never commit these — the installer generates every
+# password and app key locally at install time, so nothing real belongs in the
+# repo. See SECURITY.md.
+*.pem
+*.key
+*.p12
+*.pfx
+*.ppk
+id_rsa
+id_ed25519
+.npmrc
+.netrc
+.htpasswd
+
+# Local copies of the deployed compose file, which contain the real generated
+# DB passwords and APP_KEY. The tracked install/management_compose.yaml template
+# is the only compose file that belongs in git.
+compose.yml
+compose.yaml
+docker-compose.yml
+docker-compose.yaml
+management_compose.local.yaml
 
 # Build / Dist
 dist
@@ -36,8 +61,18 @@ server/temp
 .vscode
 .idea
 
+# Agent Files
+.agents
+.claude
+.codegraph
+.mcp.json
+
 # Frontend assets compiled code
 admin/public/assets
 
 # Admin specific development files
 admin/storage
+
+# Python bytecode (install/nomad-translate helper scripts)
+__pycache__/
+*.pyc

CONTRIBUTING.md

@@ -95,7 +95,21 @@ Because NOMAD relies heavily on Docker, we actually recommend against installing
 
 3. **Make your changes.** Follow existing code style and conventions. Test your changes locally against a running NOMAD instance before submitting.
 
-4. **Add release notes** (see [Release Notes](#release-notes) below).
+4. **If you touched the AI Assistant or RAG, measure it.** Anything affecting
+   chunking, embedding, retrieval, reranking, thresholds, the system prompts, or
+   context assembly should be backed by numbers rather than a spot check —
+   "it seemed better in the chat window" is how a regression ships. From `admin/`:
+
+   ```bash
+   node ace eval:corpus --ingest        # once; safe, never touches your real knowledge base
+   node ace eval:retrieval --ablate     # seconds, deterministic, no chat model
+   node ace eval:generation --model=<your model> --all-modes
+   ```
+
+   Include the before/after numbers in your pull request. See
+   [`admin/tests/eval/README.md`](admin/tests/eval/README.md) for what the
+   metrics mean, how the three generation modes separate a code bug from a model
+   that is simply too small, and the harness's known limitations.
 
 5. **Commit your changes** using [Conventional Commits](#commit-messages).
 

README.md

@@ -1,5 +1,5 @@
 <div align="center">
-<img src="admin/public/project_nomad_logo.webp" width="200" height="200"/>
+<img src="admin/public/nomad-primary.svg" width="200" alt="Project NOMAD"/>
 
 # Project NOMAD
 ### Knowledge That Never Goes Offline
@@ -17,6 +17,8 @@ Project NOMAD is a self-contained, offline-first knowledge and education server
 ## Installation & Quickstart
 Project NOMAD can be installed on any Debian-based operating system (we recommend Ubuntu 26.04 LTS; 24.04 LTS and Debian 12 are also supported). Installation is completely terminal-based, and all tools and resources are designed to be accessed through the browser, so there's no need for a desktop environment if you'd rather setup NOMAD as a "server" and access it through other clients.
 
+**Before you start:** NOMAD itself needs about **5 GB of disk and under 1 GB of RAM**. The offline content you add afterwards is what actually fills a drive, and the AI Assistant is what drives the memory requirement up. See [Device Requirements](#device-requirements) for both, and the [FAQ](https://www.projectnomad.us/docs/faq) for individual content sizes.
+
 *Note: sudo/root privileges are required to run the install script*
 
 ### Quick Install (Debian-based OS Only)
@@ -75,11 +77,13 @@ At its core, however, NOMAD is still very lightweight. For a barebones installat
 
 #### Minimum Specs
 - Processor: 2 GHz dual-core processor or better
-- RAM: 4GB system memory
-- Storage: At least 5 GB free disk space
+- RAM: 4GB system memory (the whole stack without AI runs in under 1 GB)
+- Storage: At least 5 GB free disk space (plan for 10 GB to install comfortably)
 - OS: Debian-based (Ubuntu 26.04 LTS recommended)
 - Stable internet connection (required during install only)
 
+Adding the AI Assistant brings the install to roughly **25 GB**, because Ollama and a general-purpose model are both large, and a model needs about its download size in memory while it answers. That memory comes from VRAM if you have a discrete GPU and from system RAM if you do not, which is the single biggest reason the figures below are so much higher.
+
 To run LLMs and other included AI tools:
 
 #### Optimal Specs

SECURITY.md

@@ -0,0 +1,98 @@
+# Security Policy
+
+## Supported Versions
+
+Only the latest released version of Project NOMAD receives security fixes. If
+you are running an older release, please update before reporting an issue.
+
+## Reporting a Vulnerability
+
+**Please do not open a public issue for a security vulnerability.**
+
+Report it privately using GitHub's built-in reporting form:
+
+1. Go to the [Security tab](https://github.com/Crosstalk-Solutions/project-nomad/security)
+2. Click **Report a vulnerability**
+
+This creates a private advisory that only the maintainers can see. It stays
+private until a fix is available and we choose to publish it.
+
+If you cannot use that form for any reason, email
+**chris@crosstalksolutions.com** instead. Please do not include exploit details
+in a Discord message or a public issue.
+
+### What to include
+
+The more of this you can provide, the faster we can confirm and fix it:
+
+- The version of NOMAD you tested against, and the host OS
+- Which component is affected (Command Center, installer, updater sidecar, a
+  Supply Depot app, the benchmark submission path, and so on)
+- Steps to reproduce, ideally with the exact request or command
+- What an attacker gains, and what access they need to start with
+- Any suggested fix, if you have one
+
+### What to expect
+
+Project NOMAD is maintained by a very small team, so we do not offer a
+guaranteed response time. We read every report. If a report is valid, we will
+work with you on a fix and credit you in the published advisory unless you
+would rather stay anonymous.
+
+We do not run a bug bounty program and cannot offer payment for reports.
+
+## Scope
+
+### In scope
+
+- Remote code execution, container escape, or privilege escalation on the host
+- Any path where a remote party who is **not** on the local network can affect a
+  NOMAD instance, including attacks delivered through a user's browser
+- Unauthenticated access to data outside the NOMAD storage root
+- Path traversal, SSRF that reaches beyond the intended target, or injection in
+  the Command Center API
+- Supply chain problems in our build and release pipeline
+- Credentials or secrets committed to this repository
+
+### Out of scope
+
+Some things that look like vulnerabilities are deliberate design decisions for
+an offline, single-appliance, local-network product. Reports covering the
+following will usually be closed:
+
+- **No authentication on the Command Center.** This is intentional and
+  documented in the [README](README.md#about-security). NOMAD is designed to be
+  open on a trusted local network. If you need access control, use
+  network-level controls. There is an open roadmap item if you want to vote for
+  optional authentication:
+  https://roadmap.projectnomad.us/posts/1/user-authentication-please-build-in-user-auth-with-admin-user-roles
+- **Anything that requires exposing NOMAD directly to the internet.** This is
+  explicitly unsupported and advised against.
+- **Access by someone who is already on the local network.** Local network
+  access is the trust boundary by design.
+- **Requests to internal or private addresses.** NOMAD is expected to reach
+  other hosts on the local network, so RFC1918 destinations are not treated as
+  SSRF.
+- The benchmark submission signing key. It ships inside the image because an
+  offline appliance cannot hold a server-side secret. Forged submissions are
+  handled by moderation on the leaderboard, not by the key.
+- Missing security headers, missing rate limits, or similar findings with no
+  demonstrated impact on an appliance of this design.
+- Vulnerabilities in third-party Supply Depot applications. Please report those
+  to the upstream project. Tell us anyway if the issue is caused by how NOMAD
+  configures or deploys the app.
+- Findings from an automated scanner with no working proof of concept.
+
+If you are not sure whether something is in scope, report it. We would rather
+read an out-of-scope report than miss a real one.
+
+## Secrets in this Repository
+
+Secret scanning and push protection are enabled on this repository. If you
+believe a credential has been committed, report it privately using the process
+above rather than opening an issue, so it can be rotated before it is
+advertised.
+
+Note that the installer generates every database password and application key
+locally at install time. The placeholder values in
+`install/management_compose.yaml` are not real credentials.

admin/adonisrc.ts

@@ -58,6 +58,7 @@ export default defineConfig({
     () => import('#providers/qdrant_restart_policy_provider'),
     () => import('#providers/version_check_provider'),
     () => import('#providers/gpu_passthrough_remediation_provider'),
+    () => import('#providers/drug_install_row_provider'),
   ],
 
   /*

admin/app/controllers/maps_controller.ts

@@ -186,46 +186,68 @@ export default class MapsController {
           longitude: vine.number().min(-180).max(180),
           latitude: vine.number().min(-90).max(90),
           color: vine.string().trim().maxLength(20).optional(),
-          notes: vine.string().trim().nullable().optional(),
+          custom_color: vine.string().trim().maxLength(7).nullable().optional(),
+          icon: vine.string().trim().maxLength(50).nullable().optional(),
+          icon_color: vine.string().trim().maxLength(7).nullable().optional(),
+          visible: vine.boolean().optional(),
+          notes: vine.string().trim().maxLength(500).nullable().optional(),
           marker_type: vine.string().trim().maxLength(20).optional(),
         })
       )
     )
+
     const marker = await MapMarker.create({
       name: payload.name,
       longitude: payload.longitude,
       latitude: payload.latitude,
       color: payload.color ?? 'orange',
+      custom_color: payload.custom_color ?? null,
+      icon: payload.icon ?? null,
+      icon_color: payload.icon_color ?? null,
+      visible: payload.visible ?? true,
       notes: payload.notes ?? null,
       marker_type: payload.marker_type ?? 'pin',
     })
+
     return marker
   }
 
   async updateMarker({ request, response }: HttpContext) {
     const { id } = request.params()
     const marker = await MapMarker.find(id)
+
     if (!marker) {
       return response.status(404).send({ message: 'Marker not found' })
     }
+
     const payload = await request.validateUsing(
       vine.compile(
         vine.object({
           name: vine.string().trim().minLength(1).maxLength(255).optional(),
           color: vine.string().trim().maxLength(20).optional(),
+          custom_color: vine.string().trim().maxLength(7).nullable().optional(),
+          icon: vine.string().trim().maxLength(50).nullable().optional(),
+          icon_color: vine.string().trim().maxLength(7).nullable().optional(),
+          visible: vine.boolean().optional(),
           longitude: vine.number().min(-180).max(180).optional(),
           latitude: vine.number().min(-90).max(90).optional(),
-          notes: vine.string().trim().nullable().optional(),
+          notes: vine.string().trim().maxLength(500).nullable().optional(),
           marker_type: vine.string().trim().maxLength(20).optional(),
         })
       )
     )
+
     if (payload.name !== undefined) marker.name = payload.name
     if (payload.color !== undefined) marker.color = payload.color
+    if (payload.custom_color !== undefined) marker.custom_color = payload.custom_color
+    if (payload.icon !== undefined) marker.icon = payload.icon
+    if (payload.icon_color !== undefined) marker.icon_color = payload.icon_color
+    if (payload.visible !== undefined) marker.visible = payload.visible
     if (payload.longitude !== undefined) marker.longitude = payload.longitude
     if (payload.latitude !== undefined) marker.latitude = payload.latitude
     if (payload.notes !== undefined) marker.notes = payload.notes
     if (payload.marker_type !== undefined) marker.marker_type = payload.marker_type
+
     await marker.save()
     return marker
   }

admin/app/controllers/ollama_controller.ts

@@ -1,7 +1,8 @@
 import { ChatService } from '#services/chat_service'
 import { DockerService } from '#services/docker_service'
-import { NomadMdService } from '#services/nomad_md_service'
-import { OllamaService } from '#services/ollama_service'
+import { OllamaService, type NomadChatUsage } from '#services/ollama_service'
+import { TokenCalibrationService } from '#services/token_calibration_service'
+import { RagPipelineService } from '#services/rag_pipeline_service'
 import { RagService } from '#services/rag_service'
 import Service from '#models/service'
 import KVStore from '#models/kv_store'
@@ -10,10 +11,23 @@ import { chatSchema, getAvailableModelsSchema, unloadChatModelsSchema } from '#v
 import { assertNotCloudMetadataUrl } from '#validators/common'
 import { inject } from '@adonisjs/core'
 import type { HttpContext } from '@adonisjs/core/http'
-import { RAG_CONTEXT_LIMITS, SYSTEM_PROMPTS } from '../../constants/ollama.js'
 import { SERVICE_NAMES } from '../../constants/service_names.js'
+import { DEFAULT_KEEP_ALIVE } from '../../constants/ollama.js'
+import { resolveResponseStyle } from '../utils/response_style.js'
+import { resolveSamplerProfile } from '../utils/sampler.js'
+import type { PipelineTrace } from '../../types/rag.js'
+import { buildCitations } from '../utils/rag_prompt.js'
 import logger from '@adonisjs/core/services/logger'
-type Message = { role: 'system' | 'user' | 'assistant'; content: string }
+import { rm } from 'node:fs/promises'
+import {
+  attachImagesToLatestUserMessage,
+  ChatImageError,
+  normalizeChatImages,
+  type NormalizedChatImage,
+} from '../utils/chat_images.js'
+
+const unknownVisionCompatibilityMessage = (model: string) =>
+  `NOMAD cannot confirm that "${model}" accepts images, and this image request failed. Choose a model marked "Supports images", or remove the image and try again.`
 
 @inject()
 export default class OllamaController {
@@ -21,8 +35,9 @@ export default class OllamaController {
     private chatService: ChatService,
     private dockerService: DockerService,
     private ollamaService: OllamaService,
+    private ragPipelineService: RagPipelineService,
     private ragService: RagService,
-    private nomadMdService: NomadMdService
+    private tokenCalibration: TokenCalibrationService
   ) { }
 
   async availableModels({ request }: HttpContext) {
@@ -50,7 +65,78 @@ export default class OllamaController {
   }
 
   async chat({ request, response }: HttpContext) {
-    const reqData = await request.validateUsing(chatSchema)
+    const uploadedImages = request.files('images', {
+      size: '8mb',
+      extnames: ['jpg', 'jpeg', 'png', 'webp'],
+    })
+    const cleanupUploadedImages = () =>
+      Promise.all(
+        uploadedImages
+          .filter((file) => file.tmpPath)
+          .map((file) => rm(file.tmpPath!, { force: true }).catch(() => undefined))
+      )
+    let multipartPayload: unknown
+    if (uploadedImages.length > 0) {
+      const rawPayload = request.input('payload')
+      if (typeof rawPayload !== 'string') {
+        await cleanupUploadedImages()
+        return response.status(422).send({ message: 'Multipart chat requests require a JSON payload.' })
+      }
+      try {
+        multipartPayload = JSON.parse(rawPayload)
+      } catch {
+        await cleanupUploadedImages()
+        return response.status(422).send({ message: 'The multipart chat payload is not valid JSON.' })
+      }
+    }
+    // A multipart chat carries the collection inside the JSON payload rather than
+    // as a form field, so request.input() alone would silently drop it and every
+    // image request would search the whole knowledge base.
+    const rawCollection =
+      uploadedImages.length > 0 &&
+      multipartPayload &&
+      typeof multipartPayload === 'object' &&
+      'collection' in multipartPayload
+        ? (multipartPayload as { collection?: unknown }).collection
+        : request.input('collection', null)
+    const collectionFilter: string | null =
+      typeof rawCollection === 'string' && rawCollection ? rawCollection : null
+
+    let reqData: Awaited<ReturnType<typeof chatSchema.validate>>
+    try {
+      reqData =
+        uploadedImages.length > 0
+          ? await chatSchema.validate(multipartPayload)
+          : await request.validateUsing(chatSchema)
+    } catch (error) {
+      await cleanupUploadedImages()
+      throw error
+    }
+
+    let normalizedImages: NormalizedChatImage[]
+    try {
+      normalizedImages = await normalizeChatImages(uploadedImages)
+    } catch (error) {
+      if (error instanceof ChatImageError) {
+        return response.status(error.status).send({ message: error.message })
+      }
+      throw error
+    } finally {
+      await cleanupUploadedImages()
+    }
+
+    const modelCapabilities = await this.ollamaService.getModelCapabilities(reqData.model)
+    if (
+      normalizedImages.length > 0 &&
+      !reqData.messages.some((message) => message.role === 'user')
+    ) {
+      return response.status(422).send({ message: 'Images require a user message.' })
+    }
+    if (normalizedImages.length > 0 && modelCapabilities.vision === 'unsupported') {
+      return response.status(422).send({
+        message: `The selected model "${reqData.model}" does not support image input.`,
+      })
+    }
 
     // Flush SSE headers immediately so the client connection is open while
     // pre-processing (query rewriting, RAG lookup) runs in the background.
@@ -61,108 +147,40 @@ export default class OllamaController {
       response.response.flushHeaders()
     }
 
+    let unknownVisionUpstreamRejected = false
     try {
-      // If there are no system messages in the chat inject system prompts
-      const hasSystemMessage = reqData.messages.some((msg) => msg.role === 'system')
-      if (!hasSystemMessage) {
-        const systemPrompt = {
-          role: 'system' as const,
-          content: SYSTEM_PROMPTS.default,
-        }
-        logger.debug('[OllamaController] Injecting system prompt')
-        reqData.messages.unshift(systemPrompt)
-      }
-
-      // Inject the user-managed NOMAD.md as its own leading system message so the
-      // user's persistent instructions take precedence, while the default
-      // formatting prompt and any RAG context below remain intact. A missing or
-      // blank file yields null and changes nothing.
-      const nomadPrompt = await this.nomadMdService.getSystemPrompt()
-      if (nomadPrompt) {
-        logger.debug('[OllamaController] Injecting NOMAD.md system prompt')
-        reqData.messages.unshift({ role: 'system' as const, content: nomadPrompt })
-      }
-
-      // Query rewriting for better RAG retrieval with manageable context
-      // Will return user's latest message if no rewriting is needed
-      const rewrittenQuery = await this.rewriteQueryWithContext(reqData.messages, reqData.model)
-
-      logger.debug(`[OllamaController] Rewritten query for RAG: "${rewrittenQuery}"`)
-      if (rewrittenQuery) {
-        const collectionFilter: string | null = request.input('collection', null)
-        const relevantDocs = await this.ragService.searchSimilarDocuments(
-          rewrittenQuery,
-          5, // Top 5 most relevant chunks
-          0.3, // Minimum similarity score of 0.3
-          collectionFilter ?? undefined
-        )
-
-        logger.debug(`[RAG] Retrieved ${relevantDocs.length} relevant documents for query: "${rewrittenQuery}"`)
-
-        // If relevant context is found, inject as a system message with adaptive limits
-        if (relevantDocs.length > 0) {
-          // Determine context budget based on model size
-          const { maxResults, maxTokens } = this.getContextLimitsForModel(reqData.model)
-          let trimmedDocs = relevantDocs.slice(0, maxResults)
-
-          // Apply token cap if set (estimate ~3.5 chars per token)
-          // Always include the first (most relevant) result — the cap only gates subsequent results
-          if (maxTokens > 0) {
-            const charCap = maxTokens * 3.5
-            let totalChars = 0
-            trimmedDocs = trimmedDocs.filter((doc, idx) => {
-              totalChars += doc.text.length
-              return idx === 0 || totalChars <= charCap
-            })
-          }
-
-          logger.debug(
-            `[RAG] Injecting ${trimmedDocs.length}/${relevantDocs.length} results (model: ${reqData.model}, maxResults: ${maxResults}, maxTokens: ${maxTokens || 'unlimited'})`
-          )
-
-          // Label each context block with its source title when available (a neutral,
-          // honest provenance signal) but never the raw relevance score — nomic cosine
-          // scores for genuinely relevant passages sit ~0.4-0.6, and surfacing e.g.
-          // "42%" primes the model to distrust correct context. Scores stay in the logs
-          // above for debugging.
-          const contextText = trimmedDocs
-            .map((doc, idx) => {
-              const title = doc.metadata?.full_title || doc.metadata?.article_title
-              const label = title ? `[Context ${idx + 1} — ${title}]` : `[Context ${idx + 1}]`
-              return `${label}\n${doc.text}`
-            })
-            .join('\n\n')
-
-          const systemMessage = {
-            role: 'system' as const,
-            content: SYSTEM_PROMPTS.rag_context(contextText),
-          }
-
-          // Insert system message at the beginning (after any existing system messages)
-          const firstNonSystemIndex = reqData.messages.findIndex((msg) => msg.role !== 'system')
-          const insertIndex = firstNonSystemIndex === -1 ? 0 : firstNonSystemIndex
-          reqData.messages.splice(insertIndex, 0, systemMessage)
-        }
-      }
-
-      // If system messages are large (e.g. due to RAG context), request a context window big
-      // enough to fit them. Ollama respects num_ctx per-request; LM Studio ignores it gracefully.
-      const systemChars = reqData.messages
-        .filter((m) => m.role === 'system')
-        .reduce((sum, m) => sum + m.content.length, 0)
-      const estimatedSystemTokens = Math.ceil(systemChars / 3.5)
-      let numCtx: number | undefined
-      if (estimatedSystemTokens > 3000) {
-        const needed = estimatedSystemTokens + 2048 // leave room for conversation + response
-        numCtx = [8192, 16384, 32768, 65536].find((n) => n >= needed) ?? 65536
-        logger.debug(`[OllamaController] Large system prompt (~${estimatedSystemTokens} tokens), requesting num_ctx: ${numCtx}`)
-      }
+      // Everything from system-prompt assembly through query rewriting,
+      // retrieval, context trimming and the num_ctx decision lives in
+      // RagPipelineService so the eval harness exercises this exact code path.
+      // Knowledge base retrieval is user-toggleable (chat header + AI Assistant
+      // settings, both writing rag.enabled). Unset means on, preserving the
+      // behaviour from before the toggle existed.
+      const ragEnabled = (await KVStore.getValue('rag.enabled')) ?? true
+      const trace = await this.ragPipelineService.buildPrompt(reqData.messages, reqData.model, {
+        collection: collectionFilter ?? undefined,
+        skipRetrieval: !ragEnabled,
+      })
+      reqData.messages = trace.messages
+      const numCtx = trace.numCtx
+      const numPredict = trace.numPredict
+      // Provenance for the answer about to be generated (#1179). Built from
+      // trace.injected -- what the model actually read -- and surfaced under the
+      // answer as "Sources". Empty whenever retrieval was skipped or declined,
+      // which is the honest result: no context, no citations.
+      const sources = buildCitations(trace.injected)
+      // Keeping the model resident is what makes the KV cache worth building:
+      // Ollama's default evicts after 5 minutes, which is well inside the time a
+      // user spends reading an answer and typing the next question.
+      const keepAlive = (await KVStore.getValue('ai.keepAlive')) || DEFAULT_KEEP_ALIVE
 
       // Check if the model supports "thinking" capability for enhanced response generation.
       // Thinking is only enabled when the model supports it AND the user wants it: the explicit
       // per-request preference wins, otherwise the global default (ai.autoThinking, default OFF).
       // If gpt-oss model, it requires a text param for "think" https://docs.ollama.com/api/chat
-      const thinkingCapability = await this.ollamaService.checkModelHasThinking(reqData.model)
+      // One /api/show read, memoized per model, for two decisions: whether the
+      // model can think, and what sampler values its author baked in.
+      const modelInfo = await this.ollamaService.getModelInfo(reqData.model)
+      const thinkingCapability = modelInfo.hasThinking
       let thinkingEnabled = false
       if (thinkingCapability) {
         thinkingEnabled = reqData.think ?? ((await KVStore.getValue('ai.autoThinking')) ?? false)
@@ -170,9 +188,23 @@ export default class OllamaController {
       const think: boolean | 'medium' =
         thinkingEnabled ? (reqData.model.startsWith('gpt-oss') ? 'medium' : true) : false
 
+      // Sampler settings for this turn. Chat used to send none of these and
+      // inherit the backend's defaults, which on a heavily quantized small model
+      // is the wandering, repetitive output people read as "the local model is
+      // bad". Resolved here rather than in OllamaService so the ancillary
+      // structured calls, which want temperature 0 and a grammar, stay untouched.
+      const sampler = resolveSamplerProfile(
+        await resolveResponseStyle(),
+        modelInfo.modelfileSamplers
+      )
+
       // Separate sessionId and the resolved thinking preference from the Ollama request payload —
       // Ollama rejects unknown fields, and `think` is re-derived above (not forwarded raw).
       const { sessionId, think: _thinkPref, ...ollamaRequest } = reqData
+      const upstreamMessages = attachImagesToLatestUserMessage(
+        ollamaRequest.messages,
+        normalizedImages
+      )
 
       // Save user message to DB before streaming if sessionId provided
       let userContent: string | null = null
@@ -192,19 +224,30 @@ export default class OllamaController {
         // blocks every later chat/RAG request until the model is manually stopped (#1065).
         const abortController = new AbortController()
         response.response.on('close', () => abortController.abort())
-        const stream = await this.ollamaService.chatStream({
-          ...ollamaRequest,
-          think,
-          thinkingCapable: thinkingCapability,
-          numCtx,
-          signal: abortController.signal,
-        })
         let fullContent = ''
+        let doneReason: string | undefined
+        let completionTokens: number | undefined
         try {
+          const stream = await this.ollamaService.chatStream({
+            ...ollamaRequest,
+            messages: upstreamMessages,
+            think,
+            thinkingCapable: thinkingCapability,
+            numCtx,
+            numPredict,
+            keepAlive,
+          sampler,
+            signal: abortController.signal,
+          })
           for await (const chunk of stream) {
             if (chunk.message?.content) {
               fullContent += chunk.message.content
             }
+            if (chunk.usage) {
+              completionTokens = chunk.usage.completionTokens
+              this._recordUsage(reqData.model, trace, chunk.usage)
+            }
+            if (chunk.done_reason) doneReason = chunk.done_reason
             response.response.write(`data: ${JSON.stringify(chunk)}\n\n`)
           }
         } catch (err) {
@@ -212,13 +255,25 @@ export default class OllamaController {
             logger.debug('[OllamaController] Client disconnected; aborted upstream Ollama generation')
             return
           }
+          unknownVisionUpstreamRejected =
+            normalizedImages.length > 0 && modelCapabilities.vision === 'unknown'
           throw err
         }
+        if (doneReason === 'length') {
+          this._logLengthStop(reqData.model, numCtx, numPredict, completionTokens)
+        }
+        // Trailing citation event, written before end(). It carries no `message`
+        // key, which is how the client tells it apart from Ollama's own chunks.
+        if (sources.length > 0) {
+          response.response.write(`data: ${JSON.stringify({ sources })}
+
+`)
+        }
         response.response.end()
 
         // Save assistant message and optionally generate title
         if (sessionId && fullContent) {
-          await this.chatService.addMessage(sessionId, 'assistant', fullContent)
+          await this.chatService.addMessage(sessionId, 'assistant', fullContent, sources)
           const messageCount = await this.chatService.getMessageCount(sessionId)
           if (messageCount <= 2 && userContent) {
             this.chatService.generateTitle(sessionId, userContent, fullContent, reqData.model).catch((err) => {
@@ -230,10 +285,35 @@ export default class OllamaController {
       }
 
       // Non-streaming (legacy) path
-      const result = await this.ollamaService.chat({ ...ollamaRequest, think, thinkingCapable: thinkingCapability, numCtx })
+      let result
+      try {
+        result = await this.ollamaService.chat({
+          ...ollamaRequest,
+          messages: upstreamMessages,
+          think,
+          thinkingCapable: thinkingCapability,
+          numCtx,
+          numPredict,
+          keepAlive,
+        sampler,
+        })
+      } catch (err) {
+        // A backend that never advertised vision support rejecting a request that
+        // carried images is the signal that it cannot see them. Recorded here and
+        // translated into the compatibility message by the outer catch.
+        unknownVisionUpstreamRejected =
+          normalizedImages.length > 0 && modelCapabilities.vision === 'unknown'
+        throw err
+      }
+      if (result?.usage) {
+        this._recordUsage(reqData.model, trace, result.usage)
+      }
+      if (result?.done_reason === 'length') {
+        this._logLengthStop(reqData.model, numCtx, numPredict, result.usage?.completionTokens)
+      }
 
       if (sessionId && result?.message?.content) {
-        await this.chatService.addMessage(sessionId, 'assistant', result.message.content)
+        await this.chatService.addMessage(sessionId, 'assistant', result.message.content, sources)
         const messageCount = await this.chatService.getMessageCount(sessionId)
         if (messageCount <= 2 && userContent) {
           this.chatService.generateTitle(sessionId, userContent, result.message.content, reqData.model).catch((err) => {
@@ -242,17 +322,75 @@ export default class OllamaController {
         }
       }
 
-      return result
+      return { ...result, sources }
     } catch (error) {
       if (reqData.stream) {
-        response.response.write(`data: ${JSON.stringify({ error: true })}\n\n`)
+        const streamError =
+          unknownVisionUpstreamRejected
+            ? {
+                error: true,
+                message: unknownVisionCompatibilityMessage(reqData.model),
+              }
+            : { error: true }
+        response.response.write(`data: ${JSON.stringify(streamError)}\n\n`)
         response.response.end()
         return
       }
+      if (unknownVisionUpstreamRejected) {
+        return response.status(422).send({
+          message: unknownVisionCompatibilityMessage(reqData.model),
+        })
+      }
       throw error
     }
   }
 
+  /**
+   * A reply that hit the generation cap reaches the user cut off. The client
+   * shows that from done_reason; this puts it in the admin log with the numbers
+   * needed to tell a small window from a cap that is set too low (#1342).
+   */
+  private _logLengthStop(
+    model: string,
+    numCtx: number | undefined,
+    numPredict: number | undefined,
+    completionTokens: number | undefined
+  ): void {
+    logger.info(
+      `[OllamaController] ${model} stopped at the length limit: ` +
+        `${completionTokens ?? '?'} tokens generated, num_predict=${numPredict ?? 'unset'}, num_ctx=${numCtx ?? 'unset'}`
+    )
+  }
+
+  /**
+   * Close the loop on token accounting.
+   *
+   * The backend just told us exactly how many tokens the prompt really was.
+   * Comparing that against what we estimated is what lets the estimator correct
+   * itself per model — free ground truth that used to be discarded. Also logs
+   * the prefill rate, which is the practical prefix-cache-hit signal: a reused
+   * KV prefix means many prompt tokens for very little prefill time.
+   *
+   * Best-effort throughout; a calibration failure must never affect the reply.
+   */
+  private _recordUsage(model: string, trace: PipelineTrace, usage: NomadChatUsage): void {
+    if (trace.uncalibratedPromptTokens && usage.promptTokens) {
+      this.tokenCalibration
+        .record(model, trace.uncalibratedPromptTokens, usage.promptTokens)
+        .catch((err) => {
+          logger.debug(`[OllamaController] Token calibration failed: ${err?.message ?? err}`)
+        })
+
+      if (usage.promptEvalMs !== undefined && usage.promptEvalMs > 0) {
+        const tokensPerMs = usage.promptTokens / usage.promptEvalMs
+        logger.debug(
+          `[OllamaController] Prefill: ${usage.promptTokens} tokens in ${usage.promptEvalMs.toFixed(0)}ms ` +
+            `(${tokensPerMs.toFixed(1)} tok/ms; a high rate means the KV prefix was reused)`
+        )
+      }
+    }
+  }
+
   async remoteStatus() {
     const remoteUrl = await KVStore.getValue('ai.remoteOllamaUrl')
     if (!remoteUrl) {
@@ -409,99 +547,13 @@ export default class OllamaController {
     }
   }
 
-  async installedModels({ }: HttpContext) {
+  async installedModels({}: HttpContext) {
     const models = await this.ollamaService.getModels()
-    // Enrich each model with its thinking capability so the chat picker knows which models
-    // to show the per-model thinking toggle for. checkModelHasThinking memoizes /api/show
-    // results, so this stays cheap on repeat loads. Best-effort per model.
-    const thinking = await Promise.all(
-      models.map((m) => this.ollamaService.checkModelHasThinking(m.name))
+    // Enrich from backend-reported capabilities; never guess from model names.
+    const capabilities = await Promise.all(
+      models.map((m) => this.ollamaService.getModelCapabilities(m.name, m))
     )
-    return models.map((m, i) => ({ ...m, thinking: thinking[i] }))
+    return models.map((m, i) => ({ ...m, ...capabilities[i] }))
   }
 
-  /**
-   * Determines RAG context limits based on model size extracted from the model name.
-   * Parses size indicators like "1b", "3b", "8b", "70b" from model names/tags.
-   */
-  private getContextLimitsForModel(modelName: string): { maxResults: number; maxTokens: number } {
-    // Extract parameter count from model name (e.g., "llama3.2:3b", "qwen2.5:1.5b", "gemma:7b")
-    const sizeMatch = modelName.match(/(\d+\.?\d*)[bB]/)
-    const paramBillions = sizeMatch ? parseFloat(sizeMatch[1]) : 8 // default to 8B if unknown
-
-    for (const tier of RAG_CONTEXT_LIMITS) {
-      if (paramBillions <= tier.maxParams) {
-        return { maxResults: tier.maxResults, maxTokens: tier.maxTokens }
-      }
-    }
-
-    // Fallback: no limits
-    return { maxResults: 5, maxTokens: 0 }
-  }
-
-  private async rewriteQueryWithContext(
-    messages: Message[],
-    model: string
-  ): Promise<string | null> {
-    const lastUserMessage = [...messages].reverse().find(msg => msg.role === 'user')
-
-    try {
-      // Skip the entire RAG pipeline if there are no documents to search
-      const hasDocuments = await this.ragService.hasDocuments()
-      if (!hasDocuments) {
-        return null
-      }
-
-      // Get recent conversation history (last 6 messages for 3 turns)
-      const recentMessages = messages.slice(-6)
-
-      // Skip rewriting on the very first turn — with only one user message
-      // there is no prior context to fold in, so the rewrite would just echo
-      // the message back at the cost of an extra LLM round-trip. From the
-      // first follow-up onward we need the rewrite so the RAG query carries
-      // entities and topics from earlier turns ("the bars" → "Hershey's bars
-      // chocolate poisoning dog"); without it, embeddings match nothing and
-      // the assistant loses the thread.
-      const userMessages = recentMessages.filter(msg => msg.role === 'user')
-      if (userMessages.length < 2) {
-        return lastUserMessage?.content || null
-      }
-
-      const conversationContext = recentMessages
-        .map(msg => {
-          const role = msg.role === 'user' ? 'User' : 'Assistant'
-          // Truncate assistant messages to first 200 chars to keep context manageable
-          const content = msg.role === 'assistant'
-            ? msg.content.slice(0, 200) + (msg.content.length > 200 ? '...' : '')
-            : msg.content
-          return `${role}: "${content}"`
-        })
-        .join('\n')
-
-      const response = await this.ollamaService.chat({
-        model,
-        messages: [
-          {
-            role: 'system',
-            content: SYSTEM_PROMPTS.query_rewrite,
-          },
-          {
-            role: 'user',
-            content: `Conversation:\n${conversationContext}\n\nRewritten Query:`,
-          },
-        ],
-      })
-
-      const rewrittenQuery = response.message.content.trim()
-      logger.info(`[RAG] Query rewritten: "${rewrittenQuery}"`)
-      return rewrittenQuery
-    } catch (error) {
-      logger.error(
-        `[RAG] Query rewriting failed: ${error instanceof Error ? error.message : error}`
-      )
-      // Fallback to last user message if rewriting fails
-      return lastUserMessage?.content || null
-    }
-  }
 }
-

admin/app/controllers/rag_controller.ts

@@ -94,6 +94,57 @@ export default class RagController {
     return response.status(200).json({ message: result.message })
   }
 
+  public async setFileActive({ request, response }: HttpContext) {
+    const source: string | null = request.input('source', null)
+    const active: boolean | null = request.input('active', null)
+
+    if (!source || typeof active !== 'boolean') {
+      return response.status(400).json({ error: 'source and active are required.' })
+    }
+
+    const result = await this.ragService.setFileActive(source, active)
+    if (!result.success) {
+      return response.status(500).json({ error: result.message })
+    }
+    return response.status(200).json({ message: result.message })
+  }
+
+  /** Bulk active/inactive toggle for every file tagged with a KB collection
+   * (the personal-upload `collection` field) -- distinct from the unrelated
+   * curated ZIM-pack "collections" feature. `collection: null` targets the
+   * "Uncategorized" bucket. */
+  public async setKnowledgeCollectionActive({ request, response }: HttpContext) {
+    const rawCollection = request.input('collection', undefined)
+    const active: boolean | null = request.input('active', null)
+
+    // Validate before sanitising. Form-encoded input can yield an array
+    // (`collection[]=a&collection[]=b`), and sanitizeCollectionName would then
+    // throw on `.trim()` and surface as an uncaught 500.
+    //
+    // Requiring `active` to be a real boolean is also load-bearing beyond input
+    // hygiene: a cross-site HTML form can only send the string "true", so this
+    // check is what keeps a bulk Qdrant write across every point in the index
+    // out of reach of a form POST from another origin. Do not "helpfully"
+    // coerce it.
+    if (rawCollection === undefined || typeof active !== 'boolean') {
+      return response.status(400).json({ error: 'collection and active are required.' })
+    }
+
+    if (rawCollection !== null && typeof rawCollection !== 'string') {
+      return response.status(400).json({ error: 'collection must be a string or null.' })
+    }
+
+    const collection = rawCollection === null ? null : sanitizeCollectionName(rawCollection)
+
+    const result = await this.ragService.setKnowledgeCollectionActive(collection, active)
+    if (!result.success) {
+      return response.status(500).json({ error: result.message })
+    }
+    return response
+      .status(200)
+      .json({ message: result.message, affectedCount: result.affectedCount })
+  }
+
   public async renameKnowledgeCollection({ request, response }: HttpContext) {
     const oldName = sanitizeCollectionName(request.input('oldName', null))
     const newName = sanitizeCollectionName(request.input('newName', null))

admin/app/controllers/settings_controller.ts

@@ -1,5 +1,6 @@
 import KVStore from '#models/kv_store'
 import { BenchmarkService } from '#services/benchmark_service'
+import { ContextWindowService } from '#services/context_window_service'
 import { MapService } from '#services/map_service'
 import { OllamaService } from '#services/ollama_service'
 import { SystemService } from '#services/system_service'
@@ -7,6 +8,10 @@ import { getSettingSchema, updateSettingSchema, validateSettingValue } from '#va
 import { inject } from '@adonisjs/core'
 import type { HttpContext } from '@adonisjs/core/http'
 import env from '#start/env'
+import { parseMinRelevance } from '../utils/misc.js'
+import { isRelevanceCheckEnabled } from '../utils/relevance_judge.js'
+import { parseResponseStyle } from '../utils/sampler.js'
+import { RAG_MIN_FINAL_SCORE } from '../../constants/ollama.js'
 
 @inject()
 export default class SettingsController {
@@ -14,7 +19,8 @@ export default class SettingsController {
     private systemService: SystemService,
     private mapService: MapService,
     private benchmarkService: BenchmarkService,
-    private ollamaService: OllamaService
+    private ollamaService: OllamaService,
+    private contextWindowService: ContextWindowService
   ) {}
 
   async system({ inertia }: HttpContext) {
@@ -71,6 +77,25 @@ export default class SettingsController {
     const remoteOllamaUrl = await KVStore.getValue('ai.remoteOllamaUrl')
     const ollamaFlashAttention = await KVStore.getValue('ai.ollamaFlashAttention')
     const autoThinking = await KVStore.getValue('ai.autoThinking')
+    const tasksModel = await KVStore.getValue('ai.tasksModel')
+    const ragEnabled = await KVStore.getValue('rag.enabled')
+    const contextWindow = await KVStore.getValue('ai.contextWindow')
+    const minRelevance = await KVStore.getValue('rag.minRelevance')
+    const relevanceCheck = await KVStore.getValue('rag.relevanceCheck')
+    const responseStyle = await KVStore.getValue('ai.responseStyle')
+    // Resolved window per installed model, so the setting shows what "Auto"
+    // actually produced rather than leaving the user to guess. Best-effort:
+    // a model whose metadata can't be read simply doesn't get a badge.
+    const resolvedContextWindows: Record<string, number> = {}
+    await Promise.all(
+      (installedModels || []).map(async (model) => {
+        try {
+          resolvedContextWindows[model.name] = await this.contextWindowService.windowFor(model.name)
+        } catch {
+          /* leave unset */
+        }
+      })
+    )
     return inertia.render('settings/models', {
       models: {
         availableModels: availableModels?.models || [],
@@ -81,7 +106,18 @@ export default class SettingsController {
           remoteOllamaUrl: remoteOllamaUrl ?? '',
           ollamaFlashAttention: ollamaFlashAttention ?? true,
           autoThinking: autoThinking ?? false,
+          tasksModel: tasksModel ?? '',
+          ragEnabled: ragEnabled ?? true,
+          contextWindow: contextWindow ?? 'auto',
+          // Sent as the resolved number so the select can match an option
+          // without duplicating the "unset means the default" rule in the UI.
+          minRelevance: parseMinRelevance(minRelevance, RAG_MIN_FINAL_SCORE),
+          relevanceCheck: isRelevanceCheckEnabled(relevanceCheck),
+          // Resolved rather than raw, for the same reason: unset means 'auto',
+          // and the select shouldn't have to know that.
+          responseStyle: parseResponseStyle(responseStyle),
         },
+        resolvedContextWindows,
       },
     })
   }

admin/app/controllers/system_controller.ts

@@ -23,6 +23,9 @@ import {
   updateServiceValidator,
   setServiceAutoUpdateValidator,
   setServiceCustomUrlValidator,
+  createLinkTileValidator,
+  updateLinkTileValidator,
+  deleteLinkTileValidator,
   normalizeCustomUrl,
 } from '#validators/system'
 import {
@@ -30,10 +33,12 @@ import {
   DEFAULT_MEMORY_MB,
   evaluateCustomApp,
 } from '#services/custom_app_guard'
+import { DEFAULT_LINK_TILE_ICON, isLinkTileIcon } from '../../constants/link_tile_icons.js'
 import { inject } from '@adonisjs/core'
 import type { HttpContext } from '@adonisjs/core/http'
 import logger from '@adonisjs/core/services/logger'
 import Service from '#models/service'
+import { DEFAULT_LINK_TILE_COLOR } from '../../constants/link_tile_colors.js'
 
 @inject()
 export default class SystemController {
@@ -529,6 +534,125 @@ export default class SystemController {
         return response.send({ success: true, custom_url: service.custom_url })
     }
 
+    /**
+     * Create a dashboard link tile: a shortcut to something the user already runs.
+     *
+     * Stored as a service row with no container behind it. `installed` is true so
+     * the dashboard renders it, `custom_url` carries the destination (getServiceLink
+     * already prefers custom_url over a computed link), and `is_link_tile` marks it
+     * so the UI never offers Start/Stop/Update/Uninstall for something with no
+     * lifecycle. `is_custom` stays false: that flag means a container NOMAD installs.
+     */
+    async createLinkTile({ request, response }: HttpContext) {
+        const payload = await request.validateUsing(createLinkTileValidator)
+
+        const normalized = normalizeCustomUrl(payload.url)
+        if (!normalized) {
+            return response.status(422).send({
+                success: false,
+                message: 'Enter a valid URL, for example 192.168.1.50:8080 or https://nas.local.',
+            })
+        }
+
+        if (payload.icon && !isLinkTileIcon(payload.icon)) {
+            return response.status(422).send({ success: false, message: 'Unknown icon.' })
+        }
+
+        // Namespaced so a tile can never collide with a curated catalog entry. The
+        // seeder only ever touches names in its own DEFAULT_SERVICES list, so a
+        // `nomad_link_` row is invisible to a reseed.
+        const slug = payload.friendly_name
+            .toLowerCase()
+            .replace(/[^a-z0-9]+/g, '_')
+            .replace(/^_+|_+$/g, '')
+        if (!slug) {
+            return response.status(422).send({ success: false, message: 'Enter a name using letters or numbers.' })
+        }
+        const serviceName = `nomad_link_${slug}`
+
+        const existing = await Service.query().where('service_name', serviceName).first()
+        if (existing) {
+            return response.status(409).send({
+                success: false,
+                message: `A link named "${payload.friendly_name}" already exists. Choose a different name.`,
+            })
+        }
+
+        const service = await Service.create({
+            service_name: serviceName,
+            container_image: '',
+            container_config: null,
+            friendly_name: payload.friendly_name,
+            description: payload.description ?? null,
+            icon: payload.icon || DEFAULT_LINK_TILE_ICON,
+            display_order: payload.display_order ?? 90,
+            installed: true,
+            installation_status: 'idle',
+            is_dependency_service: false,
+            ui_location: null,
+            custom_url: normalized,
+            is_custom: false,
+            is_user_modified: true,
+            is_link_tile: true,
+            link_color: payload.link_color ?? DEFAULT_LINK_TILE_COLOR,
+            category: 'Links',
+        })
+
+        return response.status(201).send({ success: true, service_name: service.service_name })
+    }
+
+    /** Reconfigure an existing link tile. service_name is immutable so the tile keeps its identity. */
+    async updateLinkTile({ request, response }: HttpContext) {
+        const payload = await request.validateUsing(updateLinkTileValidator)
+
+        const service = await Service.query().where('service_name', payload.service_name).first()
+        if (!service) {
+            return response.status(404).send({ success: false, message: 'Link not found' })
+        }
+        if (!service.is_link_tile) {
+            return response.status(403).send({ success: false, message: 'That app is not a link.' })
+        }
+
+        const normalized = normalizeCustomUrl(payload.url)
+        if (!normalized) {
+            return response.status(422).send({
+                success: false,
+                message: 'Enter a valid URL, for example 192.168.1.50:8080 or https://nas.local.',
+            })
+        }
+
+        if (payload.icon && !isLinkTileIcon(payload.icon)) {
+            return response.status(422).send({ success: false, message: 'Unknown icon.' })
+        }
+
+        service.friendly_name = payload.friendly_name
+        service.description = payload.description ?? null
+        service.icon = payload.icon || DEFAULT_LINK_TILE_ICON
+        service.display_order = payload.display_order ?? service.display_order ?? 90
+        service.custom_url = normalized
+        service.link_color = payload.link_color ?? service.link_color ?? DEFAULT_LINK_TILE_COLOR
+        await service.save()
+
+        return response.send({ success: true })
+    }
+
+    /** Remove a link tile. Nothing to uninstall: there is no container, image or volume. */
+    async deleteLinkTile({ request, response }: HttpContext) {
+        const payload = await request.validateUsing(deleteLinkTileValidator)
+
+        const service = await Service.query().where('service_name', payload.service_name).first()
+        if (!service) {
+            return response.status(404).send({ success: false, message: 'Link not found' })
+        }
+        if (!service.is_link_tile) {
+            return response.status(403).send({ success: false, message: 'That app is not a link.' })
+        }
+
+        await service.delete()
+
+        return response.send({ success: true })
+    }
+
     /** Re-pull a custom app's image and recreate its container in place (preserving volumes). */
     async updateCustomApp_pullLatest({ request, response }: HttpContext) {
         const payload = await request.validateUsing(installServiceValidator)

admin/app/controllers/zim_controller.ts

@@ -25,8 +25,12 @@ export default class ZimController {
 
   async listRemote({ request }: HttpContext) {
     const payload = await request.validateUsing(listRemoteZimValidator)
-    const { start = 0, count = 12, query } = payload
-    return await this.zimService.listRemote({ start, count, query })
+    const { start = 0, count = 12, query, language } = payload
+    return await this.zimService.listRemote({ start, count, query, language })
+  }
+
+  async listCatalogLanguages({}: HttpContext) {
+    return { languages: await this.zimService.listCatalogLanguages() }
   }
 
   async downloadRemote({ request }: HttpContext) {

admin/app/jobs/download_model_job.ts

@@ -160,17 +160,49 @@ export class DownloadModelJob {
     return await queue.getJob(jobId)
   }
 
+  /**
+   * Returns the job for this model only when it is genuinely in flight.
+   *
+   * A terminal record (completed or failed) is removed instead, because the
+   * jobId is a hash of the model name and `queue.add` with an existing custom
+   * jobId returns the existing job rather than throwing. Leaving a terminal
+   * record in place therefore makes every later dispatch for that model a
+   * silent no-op. Mirrors RunDownloadJob.getActiveByUrl.
+   */
+  static async getActiveByModelName(modelName: string): Promise<Job | undefined> {
+    const job = await this.getByModelName(modelName)
+    if (!job) return undefined
+
+    const state = await job.getState()
+    if (state === 'active' || state === 'waiting' || state === 'delayed') {
+      return job
+    }
+
+    // Terminal state -- clean up so it doesn't block a re-download
+    try {
+      await job.remove()
+    } catch {
+      // May already be gone
+    }
+    return undefined
+  }
+
   static async dispatch(params: DownloadModelJobParams) {
     const queueService = QueueService.getInstance()
     const queue = queueService.getQueue(this.queue)
     const jobId = this.getJobId(params.modelName)
 
-    // Clear any previous failed job so a fresh attempt can be dispatched
-    const existing = await queue.getJob(jobId)
-    if (existing) {
-      const state = await existing.getState()
-      if (state === 'failed') {
-        await existing.remove()
+    // Return an in-flight download as-is, and clear any terminal record so a
+    // fresh attempt can be dispatched. Previously this only cleared `failed`,
+    // so once a model had been downloaded successfully its retained completed
+    // job deduped every later request: the API still reported success and no
+    // worker ran, leaving the model uninstallable until Redis was flushed.
+    const inFlight = await this.getActiveByModelName(params.modelName)
+    if (inFlight) {
+      return {
+        job: inFlight,
+        created: false,
+        message: `Download already in progress for model ${params.modelName}`,
       }
     }
 

admin/app/jobs/embed_file_job.ts

@@ -389,6 +389,33 @@ export class EmbedFileJob {
       jobOptions.jobId = initialJobId
     }
 
+    // Deterministic-jobId dispatches must clear a terminal record before adding.
+    // `queue.add` with an existing custom jobId returns that job instead of
+    // throwing, so a retained failed entry (held by `removeOnFail: { count: 20 }`)
+    // made re-indexing that file a silent no-op: the caller got 202 "Indexing
+    // queued" and nothing was enqueued. In-flight jobs are still returned as-is
+    // so a re-click during an active embed stays idempotent.
+    if (!isContinuation && !force) {
+      const existing = await queue.getJob(initialJobId)
+      if (existing) {
+        const state = await existing.getState()
+        if (state === 'active' || state === 'waiting' || state === 'delayed') {
+          logger.info(`[EmbedFileJob] Job already in progress for file: ${params.fileName}`)
+          return {
+            job: existing,
+            created: false,
+            jobId: initialJobId,
+            message: `Embedding job already exists for: ${params.fileName}`,
+          }
+        }
+        try {
+          await existing.remove()
+        } catch {
+          // May already be gone
+        }
+      }
+    }
+
     try {
       const job = await queue.add(this.key, params, jobOptions)
 

admin/app/jobs/ingest_drug_data_job.ts

@@ -749,37 +749,25 @@ export class IngestDrugDataJob {
     // Install-state write-back: when this ingest was kicked off by a curated-tier
     // install, record an `installed_resources` row so the tier-status math (which
     // is row-driven for ZIM/map) recognizes the dataset uniformly — the home-tile
-    // gate and the tier "installed" badge both read these rows. resource_type
-    // 'dataset' was widened onto the model in the foundational slice. The row's
+    // gate and the tier "installed" badge both read these rows. The row's
     // `version` is the openFDA export_date (the real freshness key), not the
-    // manifest placeholder. Manual downloads pass no resourceMeta → no row, which
-    // is correct: install-state belongs to the curated-tier path only.
+    // manifest placeholder. Manual ingests pass no resourceMeta and write no row
+    // here; DrugInstallRowProvider backfills that case on the next admin boot.
     if (!resourceMeta) return
     try {
-      const { default: InstalledResource } = await import('#models/installed_resource')
-      const { DateTime } = await import('luxon')
-      const totalBytes = await this.totalDownloadedBytes()
-      await InstalledResource.updateOrCreate(
-        { resource_id: resourceMeta.resourceId, resource_type: 'dataset' },
-        {
-          version: exportDate,
-          collection_ref: resourceMeta.collectionRef,
-          url: 'https://api.fda.gov/download.json',
-          // No single on-disk file — the parts are deleted after ingest; the
-          // installed artifact is the DB table. Record the staging dir for
-          // provenance; uninstall keys off resource_id, never this path.
-          file_path: STORAGE_BASE,
-          file_size_bytes: totalBytes,
-          installed_at: DateTime.now(),
-        }
-      )
+      const { DrugReferenceService } = await import('#services/drug_reference_service')
+      await new DrugReferenceService().recordInstalledRow({
+        version: exportDate,
+        collectionRef: resourceMeta.collectionRef,
+        fileSizeBytes: await this.totalDownloadedBytes(),
+      })
       logger.info(
         `[IngestDrugDataJob] Wrote installed_resources row for ${resourceMeta.resourceId} (export_date=${exportDate})`
       )
     } catch (err) {
       // A failed row write must NOT abort a completed ingest — the data is
       // already searchable. Log loud; the tier badge will simply read
-      // not-installed until the next install reconcile.
+      // not-installed until DrugInstallRowProvider backfills it on the next boot.
       logger.error(
         `[IngestDrugDataJob] Failed to write installed_resources row for ${resourceMeta.resourceId}: ${
           err instanceof Error ? err.message : String(err)

admin/app/models/chat_message.ts

@@ -18,6 +18,9 @@ export default class ChatMessage extends BaseModel {
   @column()
   declare content: string
 
+  @column()
+  declare sources: string | null
+
   @belongsTo(() => ChatSession, { foreignKey: 'session_id', localKey: 'id' })
   declare session: BelongsTo<typeof ChatSession>
 

admin/app/models/kb_ingest_state.ts

@@ -31,6 +31,9 @@ export default class KbIngestState extends BaseModel {
   @column()
   declare collection: string | null
 
+  @column()
+  declare active: boolean
+
   @column()
   declare last_error: string | null
 
@@ -43,7 +46,7 @@ export default class KbIngestState extends BaseModel {
   static async getOrCreate(filePath: string, collection?: string): Promise<KbIngestState> {
     return this.firstOrCreate(
       { file_path: filePath },
-      { file_path: filePath, state: 'pending_decision', chunks_embedded: 0, collection: collection ?? null }
+      { file_path: filePath, state: 'pending_decision', chunks_embedded: 0, collection: collection ?? null, active: true }
     )
   }
 

admin/app/models/map_marker.ts

@@ -35,6 +35,22 @@ export default class MapMarker extends BaseModel {
   @column()
   declare notes: string | null
 
+  @column()
+  declare custom_color: string | null
+
+  @column()
+  declare icon: string | null
+
+  @column()
+  declare icon_color: string | null
+
+  @column({
+    // MySQL stores this as tinyint(1) and mysql2 hands back 1/0, not true/false.
+    // Without this, strict comparisons (visible === false) silently never match.
+    consume: (value: number | boolean) => Boolean(value),
+  })
+  declare visible: boolean
+
   @column.dateTime({ autoCreate: true })
   declare created_at: DateTime
 

admin/app/models/service.ts

@@ -82,6 +82,20 @@ export default class Service extends BaseModel {
   })
   declare is_user_modified: boolean
 
+  // True when this row is only a dashboard shortcut: a name, icon and URL with no
+  // container behind it. Distinct from is_custom, which is still a real container
+  // NOMAD installs and manages. A link tile has no lifecycle, so no Start/Stop/
+  // Update/Uninstall may be offered for one.
+  @column({
+    serialize(value) {
+      return Boolean(value)
+    },
+  })
+  declare is_link_tile: boolean
+
+  @column()
+  declare link_color: string | null
+
   @column()
   declare category: string | null
 

admin/app/services/benchmark_service.ts

@@ -30,6 +30,7 @@ import type {
 } from '../../types/benchmark.js'
 import KVStore from '#models/kv_store'
 import { normalizeArchitecture, deriveOsName } from '../utils/platform_metadata.js'
+import { isUnresolvedGpuModel } from '../utils/gpu_model.js'
 import { getFreeBytes } from '../utils/image_disk_preflight.js'
 import { readFile } from 'node:fs/promises'
 import { randomUUID, createHmac } from 'node:crypto'
@@ -186,25 +187,8 @@ function isSelfHostedOllamaUrl(rawUrl: string): boolean {
   return false
 }
 
-/**
- * Is this GPU "model" a placeholder rather than a real name?
- *
- * systeminformation resolves PCI ids against the container's pci.ids database.
- * When a card is newer than that file it reports the raw id verbatim — an RTX
- * 5060 comes back as "Device 2d05". Vendor detection still succeeds, so these
- * strings otherwise pass as legitimate model names and reach the leaderboard.
- *
- * Matches the "Device <hex>" shape plus the empty/unknown cases. Deliberately
- * narrow: it must never reject a real product name, and no shipping GPU is
- * called "Device" followed by four hex digits.
- */
-function isUnresolvedGpuModel(model: string): boolean {
-  const s = model.trim()
-  if (s === '') return true
-  if (/^device\s+[0-9a-f]{4}$/i.test(s)) return true
-  if (/^unknown$/i.test(s)) return true
-  return false
-}
+// Moved to app/utils/gpu_model.ts so the Settings > System display path can
+// share the same definition — see the note there (#1196).
 
 // Minimum free disk required before pulling the AI model on first run. llama3.1:8b
 // (Q4) is ~4.9 GB; this covers the model plus headroom for the transient sysbench

admin/app/services/chat_service.ts

@@ -7,11 +7,44 @@ import { inject } from '@adonisjs/core'
 import { OllamaService } from './ollama_service.js'
 import { SYSTEM_PROMPTS } from '../../constants/ollama.js'
 import { toTitleCase } from '../utils/misc.js'
+import { resolveTasksModel } from '../utils/tasks_model.js'
+import {
+  SUGGESTIONS_SCHEMA,
+  TITLE_SCHEMA,
+  pickSuggestions,
+  pickTitle,
+  resolveStructured,
+} from '../utils/structured_output.js'
+import type { ChatSource } from '../../types/chat.js'
+
+/** Sidebar width, near enough. Applied once, to whichever candidate title won. */
+const TITLE_MAX_LENGTH = 57
+
+function truncateTitle(value: string): string {
+  return value.length > TITLE_MAX_LENGTH ? value.slice(0, TITLE_MAX_LENGTH) + '...' : value
+}
 
 @inject()
 export class ChatService {
   constructor(private ollamaService: OllamaService) {}
 
+  /**
+   * The model to use for ancillary work — chat titles and chat suggestions.
+   *
+   * Prefers the user's `ai.tasksModel` setting so a 30B reasoning model isn't
+   * spending seconds "thinking" to produce a three-word sidebar title. Falls
+   * back to `fallback` when the setting is unset (the default, which preserves
+   * the previous behaviour) or when the configured model is no longer
+   * installed. `installed` is passed in by callers that already listed models,
+   * to avoid a second round-trip.
+   */
+  private async resolveTasksModel(
+    fallback: string | null,
+    installed?: { name: string }[]
+  ): Promise<string | null> {
+    return resolveTasksModel(this.ollamaService, fallback, installed, '[ChatService]')
+  }
+
   async getAllSessions() {
     try {
       const sessions = await ChatSession.query().orderBy('updated_at', 'desc')
@@ -37,11 +70,12 @@ export class ChatService {
         return [] // If no models are available, return empty suggestions
       }
 
-      // Prefer the user's selected chat model. Fall back to the smallest
+      // The user's dedicated tasks model wins when set — suggestions are short
+      // aesthetic prompts that don't benefit from a flagship model. Otherwise
+      // prefer the user's selected chat model, and fall back to the smallest
       // installed model — picking the largest by file size is unsafe: if any
       // installed model exceeds available VRAM (e.g. llama3.1:405b on a 96 GB
       // GPU), Ollama spends minutes trying to load it and the request 500s.
-      // Suggestions are short prompts that don't benefit from a flagship model.
       const lastModel = await KVStore.getValue('chat.lastModel')
       const preferred = lastModel ? models.find((m) => m.name === lastModel) : undefined
       const chosen =
@@ -52,8 +86,16 @@ export class ChatService {
         return []
       }
 
+      const model = (await this.resolveTasksModel(chosen.name, models)) ?? chosen.name
+
+      // Suggestions are a formatting task, not a reasoning one. A reasoning tasks model
+      // would spend the whole response thinking and leave nothing to parse, so suppress
+      // it at the source; `thinkingCapable` is what lets the compat transport pick
+      // reasoning_effort:'none' rather than sending nothing. Memoized per model name.
+      const thinkingCapable = await this.ollamaService.checkModelHasThinking(model)
+
       const response = await this.ollamaService.chat({
-        model: chosen.name,
+        model,
         messages: [
           {
             role: 'user',
@@ -61,11 +103,37 @@ export class ChatService {
           }
         ],
         stream: false,
+        think: false,
+        thinkingCapable,
+        // Grammar-constrained on the native transport, so the response is three
+        // strings in an array rather than whatever prose the model felt like.
+        format: SUGGESTIONS_SCHEMA,
+        // The default of 0.8 is actively hostile to format stability, and there is
+        // nothing creative about picking three canned opening questions.
+        temperature: 0,
       })
 
       if (response && response.message && response.message.content) {
         const content = response.message.content.trim()
-        
+
+        const structured = resolveStructured(content, pickSuggestions, response.structured === true)
+        if (structured.ok) {
+          return structured.value.map((s) => toTitleCase(s))
+        }
+        if (structured.reason === 'constrained-parse-failed') {
+          // The grammar was applied and the model broke it anyway, so what came back
+          // is a broken JSON object rather than prose. Splitting that on commas would
+          // surface `{"suggestions": ["How Do I` as a chip. No chips is the better
+          // failure — they are decorative, and the empty state is already designed.
+          logger.warn(
+            `[ChatService] Model "${model}" broke the suggestion grammar; returning no suggestions`
+          )
+          return []
+        }
+        logger.warn(
+          `[ChatService] Model "${model}" returned no suggestion JSON; falling back to text parsing`
+        )
+
         // Handle both comma-separated and newline-separated formats
         let suggestions: string[] = []
         
@@ -91,6 +159,9 @@ export class ChatService {
 
         return filtered.map((s) => toTitleCase(s))
       } else {
+        // Empty content after the <think> split means the model produced reasoning and
+        // nothing else. Log it rather than silently returning no chips.
+        logger.warn(`[ChatService] Model "${model}" returned no usable suggestion text`)
         return []
       }
     } catch (error) {
@@ -121,6 +192,7 @@ export class ChatService {
           role: msg.role,
           content: msg.content,
           timestamp: msg.created_at.toJSDate(),
+          sources: msg.sources ? JSON.parse(msg.sources) : undefined,
         })),
       }
     } catch (error) {
@@ -183,12 +255,18 @@ export class ChatService {
     }
   }
 
-  async addMessage(sessionId: number, role: 'system' | 'user' | 'assistant', content: string) {
+  async addMessage(
+    sessionId: number,
+    role: 'system' | 'user' | 'assistant',
+    content: string,
+    sources?: ChatSource[]
+  ) {
     try {
       const message = await ChatMessage.create({
         session_id: sessionId,
         role,
         content,
+        sources: sources && sources.length > 0 ? JSON.stringify(sources) : null,
       })
 
       // Update session's updated_at timestamp
@@ -201,6 +279,7 @@ export class ChatService {
         role: message.role,
         content: message.content,
         timestamp: message.created_at.toJSDate(),
+        sources: sources && sources.length > 0 ? sources : undefined,
       }
     } catch (error) {
       logger.error(
@@ -241,22 +320,66 @@ export class ChatService {
 
   async generateTitle(sessionId: number, userMessage: string, assistantMessage: string, model: string) {
     try {
-      let title: string
+      // Titles are aesthetic work; route them to the tasks model when one is
+      // configured rather than the chat model that just answered.
+      const titleModel = (await this.resolveTasksModel(model)) ?? model
+
+      // Naming a chat needs no reasoning; see the note in getChatSuggestions.
+      const thinkingCapable = await this.ollamaService.checkModelHasThinking(titleModel)
 
       const response = await this.ollamaService.chat({
-        model,
+        model: titleModel,
         messages: [
           { role: 'system', content: SYSTEM_PROMPTS.title_generation },
           { role: 'user', content: userMessage },
           { role: 'assistant', content: assistantMessage },
         ],
+        think: false,
+        thinkingCapable,
+        format: TITLE_SCHEMA,
+        // See the note on suggestions: naming a chat is not a creative task, and
+        // the backend default of 0.8 makes the format wobble.
+        temperature: 0,
       })
 
-      title = response?.message?.content?.trim()
-      if (!title) {
-        title = userMessage.slice(0, 57) + (userMessage.length > 57 ? '...' : '')
+      const content = response?.message?.content?.trim() ?? ''
+      const structured = resolveStructured(content, pickTitle, response?.structured === true)
+
+      let title: string
+      if (!content) {
+        // Nothing left once reasoning was split out. Checked before the grammar branch
+        // so the log says what actually happened rather than blaming the schema.
+        logger.warn(
+          `[ChatService] Model "${titleModel}" returned no usable title text; using the user message`
+        )
+        title = userMessage
+      } else if (structured.ok) {
+        title = structured.value
+      } else if (structured.reason === 'constrained-parse-failed') {
+        // A truncated object would otherwise be stored verbatim, leaving `{"title": ...`
+        // in the sidebar. The user's own words are a worse title than the model's but a
+        // far better one than a JSON fragment.
+        logger.warn(
+          `[ChatService] Model "${titleModel}" broke the title grammar; using the user message`
+        )
+        title = userMessage
+      } else {
+        // Unconstrained backend: the response is meant to be the bare title.
+        title = content
       }
 
+      if (!title) {
+        // An unconstrained response that was pure punctuation or whitespace.
+        logger.warn(
+          `[ChatService] Model "${titleModel}" returned no usable title text; using the user message`
+        )
+        title = userMessage
+      }
+
+      // Applied once, to whichever string won: "under 50 characters" is a request the
+      // model can ignore on the schema path and the text path alike.
+      title = truncateTitle(title)
+
       await this.updateSession(sessionId, { title })
       logger.info(`[ChatService] Generated title for session ${sessionId}: "${title}"`)
     } catch (error) {
@@ -265,8 +388,7 @@ export class ChatService {
       )
       // Fall back to truncated user message
       try {
-        const fallbackTitle = userMessage.slice(0, 57) + (userMessage.length > 57 ? '...' : '')
-        await this.updateSession(sessionId, { title: fallbackTitle })
+        await this.updateSession(sessionId, { title: truncateTitle(userMessage) })
       } catch {
         // Silently fail - session keeps "New Chat" title
       }

admin/app/services/collection_update_service.ts

@@ -99,15 +99,19 @@ export class CollectionUpdateService {
     update: ResourceUpdateInfo,
     options?: { auto?: boolean }
   ): Promise<{ success: boolean; jobId?: string; error?: string }> {
-    // Check if a download is already in progress for this URL
-    const existingJob = await RunDownloadJob.getByUrl(update.download_url)
+    // Only block when a download is genuinely in progress. getActiveByUrl
+    // removes any terminal (failed/completed) job for this URL first, so a
+    // previous failed attempt doesn't leave a stale job that blocks the
+    // re-dispatch below. Using the raw getByUrl here (unlike every other
+    // dispatch site) meant a failed download stuck the resource forever: the
+    // failed job survived under the deterministic jobId, dispatch hit "job
+    // already exists" and returned it, and applyUpdate reported success while
+    // nothing was re-downloaded.
+    const existingJob = await RunDownloadJob.getActiveByUrl(update.download_url)
     if (existingJob) {
-      const state = await existingJob.getState()
-      if (state === 'active' || state === 'waiting' || state === 'delayed') {
-        return {
-          success: false,
-          error: `A download is already in progress for ${update.resource_id}`,
-        }
+      return {
+        success: false,
+        error: `A download is already in progress for ${update.resource_id}`,
       }
     }
 

admin/app/services/context_window_service.ts

@@ -0,0 +1,160 @@
+import { inject } from '@adonisjs/core'
+import logger from '@adonisjs/core/services/logger'
+import os from 'node:os'
+import KVStore from '#models/kv_store'
+import { OllamaService } from '#services/ollama_service'
+import { SystemService } from '#services/system_service'
+import {
+  UNKNOWN_BACKEND_CONTEXT,
+  computeKvBytesPerToken,
+  estimateKvBytesPerToken,
+  parseParameterBillions,
+  parseUserContextCap,
+  resolveContextWindow,
+  type ContextWindowDecision,
+} from '../utils/context_window.js'
+
+/**
+ * Decides how large a context window each chat model gets, and then holds that
+ * answer still.
+ *
+ * The "holds it still" part is the whole point. Ollama unloads and reloads a
+ * model whenever a request asks for a different `num_ctx` than the loaded
+ * instance — so a window that grew with the conversation would stall a turn and
+ * throw away the KV cache exactly when the conversation got interesting. The
+ * decision is therefore made once per model and memoized for the process
+ * lifetime; it is a property of (model, hardware), not of the current prompt.
+ *
+ * Inputs, in decreasing order of authority:
+ *   - the user's cap (`ai.contextWindow`), which can only lower the result
+ *   - the model's trained context length, from /api/show `model_info`
+ *   - what the hardware can actually afford, from the exact per-token KV cost in
+ *     the GGUF metadata against detected VRAM (or a share of system RAM)
+ */
+@inject()
+export class ContextWindowService {
+  private decisions: Map<string, Promise<ContextWindowDecision>> = new Map()
+  private availableBytes: Promise<number | undefined> | null = null
+
+  constructor(
+    private ollamaService: OllamaService,
+    private systemService: SystemService
+  ) {}
+
+  /**
+   * The context window to request for `model`. Memoized — repeat calls within a
+   * process return the identical number, which is what keeps the model loaded.
+   */
+  async windowFor(model: string): Promise<number> {
+    return (await this.decisionFor(model)).contextWindow
+  }
+
+  async decisionFor(model: string): Promise<ContextWindowDecision> {
+    const cached = this.decisions.get(model)
+    if (cached) return cached
+
+    const pending = this._resolve(model).catch((error) => {
+      logger.warn(
+        `[ContextWindow] Falling back to ${UNKNOWN_BACKEND_CONTEXT} for ${model}: ${error instanceof Error ? error.message : error}`
+      )
+      return { contextWindow: UNKNOWN_BACKEND_CONTEXT, limitedBy: 'default' as const }
+    })
+    this.decisions.set(model, pending)
+    return pending
+  }
+
+  private async _resolve(model: string): Promise<ContextWindowDecision> {
+    const [info, userCapRaw] = await Promise.all([
+      this.ollamaService.getModelInfo(model),
+      KVStore.getValue('ai.contextWindow'),
+    ])
+    const userCap = parseUserContextCap(userCapRaw)
+
+    // No /api/show means a non-Ollama backend, which sets its context at server
+    // start and ignores anything we send. Budget against a conservative value so
+    // the prompt planner still bounds itself rather than trusting the backend to
+    // truncate gracefully — it won't; it will truncate silently.
+    if (!info.contextLength && !info.rawModelInfo) {
+      const decision: ContextWindowDecision = {
+        contextWindow: userCap ?? UNKNOWN_BACKEND_CONTEXT,
+        limitedBy: userCap ? 'user' : 'default',
+      }
+      logger.info(
+        `[ContextWindow] ${model}: no model metadata available, budgeting at ${decision.contextWindow}`
+      )
+      return decision
+    }
+
+    const kvBytesPerToken =
+      computeKvBytesPerToken(info.rawModelInfo) ??
+      estimateKvBytesPerToken(parseParameterBillions(info.parameterSize, model))
+
+    const decision = resolveContextWindow({
+      modelMaxCtx: info.contextLength,
+      modelfileNumCtx: info.modelfileNumCtx,
+      kvBytesPerToken,
+      availableBytes: await this._availableBytes(),
+      modelBytes: await this._modelBytes(model),
+      userCap,
+    })
+
+    logger.info(
+      `[ContextWindow] ${model}: ${decision.contextWindow} tokens (limited by ${decision.limitedBy}; ` +
+        `${(kvBytesPerToken / 1024).toFixed(0)} KiB/token KV` +
+        `${decision.affordableTokens !== undefined ? `, ~${decision.affordableTokens} affordable` : ''})`
+    )
+    return decision
+  }
+
+  /**
+   * Memory available for inference, in bytes.
+   *
+   * Prefers the VRAM figure Ollama itself reports at startup — it is the number
+   * that actually governs whether a window fits, and it already accounts for
+   * whichever device Ollama chose. Falls back to a conservative share of system
+   * RAM for CPU-only hosts, which is the low end of the hardware NOMAD targets
+   * and precisely where getting this wrong hurts most.
+   */
+  private async _availableBytes(): Promise<number | undefined> {
+    if (!this.availableBytes) {
+      this.availableBytes = (async () => {
+        try {
+          const compute = await this.systemService.getOllamaInferenceComputeFromLogs()
+          if (compute && compute.vramMiB > 0) {
+            return compute.vramMiB * 1024 * 1024
+          }
+        } catch (error) {
+          logger.debug(
+            `[ContextWindow] GPU probe failed, assuming CPU inference: ${error instanceof Error ? error.message : error}`
+          )
+        }
+        // CPU inference shares host RAM with MySQL, Redis, Qdrant and every other
+        // container on the box, so claim only a modest slice of it.
+        return os.totalmem() * CPU_RAM_SHARE
+      })()
+    }
+    return this.availableBytes
+  }
+
+  private async _modelBytes(model: string): Promise<number | undefined> {
+    try {
+      const models = await this.ollamaService.getModels(true)
+      return models.find((m) => m.name === model)?.size || undefined
+    } catch {
+      return undefined
+    }
+  }
+
+  /** Drop memoized decisions — call when the backend or the user's cap changes. */
+  invalidate(): void {
+    this.decisions.clear()
+    this.availableBytes = null
+  }
+}
+
+/**
+ * Share of system RAM assumed available for CPU inference. The rest of the NOMAD
+ * stack (MySQL, Redis, Qdrant, Kiwix, the admin app itself) lives in the same
+ * pool, and a machine that starts swapping is worse than one with a small window.
+ */
+const CPU_RAM_SHARE = 0.5

admin/app/services/docker_service.ts

@@ -4,7 +4,7 @@ import logger from '@adonisjs/core/services/logger'
 import { inject } from '@adonisjs/core'
 import transmit from '@adonisjs/transmit/services/main'
 import { doResumableDownloadWithRetry } from '../utils/downloads.js'
-import { mapGfxToHsaOverride } from '../utils/amd_hsa_override.js'
+import { pickAmdHsaOverride } from '../utils/amd_hsa_override.js'
 import { join } from 'path'
 import os from 'node:os'
 import env from '#start/env'
@@ -26,6 +26,10 @@ import { randomBytes } from 'node:crypto'
 import KVStore from '#models/kv_store'
 import { BROADCAST_CHANNELS } from '../../constants/broadcast.js'
 import { KIWIX_LIBRARY_CMD } from '../../constants/kiwix.js'
+import { DEFAULT_OLLAMA_CONTEXT_LENGTH } from '../../constants/ollama.js'
+
+// Written by install_nomad.sh with the host AMD GPU's gfx target.
+const AMD_GFX_MARKER_PATH = '/app/storage/.nomad-amd-gfx'
 
 @inject()
 export class DockerService {
@@ -387,6 +391,17 @@ export class DockerService {
         if (container) {
           const dockerContainer = this.docker.getContainer(container.Id)
 
+          if (serviceName === SERVICE_NAMES.OLLAMA) {
+            // The recreate builds env from scratch, so capture a working HSA override
+            // before this container, the only place it lives, is removed (#1377).
+            try {
+              const inspect = await dockerContainer.inspect()
+              await this._adoptContainerHsaOverride(inspect.Config?.Env)
+            } catch (error: any) {
+              logger.warn(`[DockerService] Could not preserve the Ollama HSA override: ${error.message}`)
+            }
+          }
+
           // Only try to stop if it's running
           if (container.State === 'running') {
             this._broadcast(serviceName, 'stopping', `Stopping container...`)
@@ -810,7 +825,18 @@ export class DockerService {
         const flashAttentionEnabled = await KVStore.getValue('ai.ollamaFlashAttention')
         if (flashAttentionEnabled !== false) {
           ollamaEnv.push('OLLAMA_FLASH_ATTENTION=1')
+          // KV cache quantization requires flash attention. q8_0 roughly halves the
+          // memory a context window costs for a perplexity delta in the noise, which is
+          // what makes an 8-16k window affordable on modest hardware. Ollama silently
+          // falls back to f16 on architectures that don't support it, so this is treated
+          // as headroom — the context-window resolver still budgets against f16.
+          ollamaEnv.push('OLLAMA_KV_CACHE_TYPE=q8_0')
         }
+        // Floor for any request that doesn't carry its own num_ctx (the OpenAI-compatible
+        // path can't set one at all). Ollama's own default is 4096 on machines under 24GB
+        // VRAM, and anything past the window is silently truncated with no error — so a
+        // conversation that outgrows it just quietly loses its early turns.
+        ollamaEnv.push(`OLLAMA_CONTEXT_LENGTH=${DEFAULT_OLLAMA_CONTEXT_LENGTH}`)
         if (amdGpuConfigured) {
           // gfx-aware HSA override — only set for cards that actually need it. See
           // _resolveAmdHsaOverride() for the resolution order and gfx → version mapping.
@@ -838,8 +864,10 @@ export class DockerService {
         'creating',
         `Creating Docker container for service ${service.service_name}...`
       )
-      const container = await this.docker.createContainer({
-        Image: finalImage,
+      // Built once and reused, so the AMD fallback below cannot drift from the
+      // real payload as this config grows.
+      const buildCreateOptions = (image: string, hostConfig: any, env: string[]) => ({
+        Image: image,
         name: service.service_name,
         Labels: {
           ...(containerConfig?.Labels ?? {}),
@@ -847,10 +875,10 @@ export class DockerService {
           'io.project-nomad.managed': 'true',
         },
         ...(containerConfig?.User && { User: containerConfig.User }),
-        HostConfig: gpuHostConfig,
+        HostConfig: hostConfig,
         ...(containerConfig?.WorkingDir && { WorkingDir: containerConfig.WorkingDir }),
         ...(containerConfig?.ExposedPorts && { ExposedPorts: containerConfig.ExposedPorts }),
-        Env: [...(containerConfig?.Env ?? []), ...ollamaEnv, ...appEnv],
+        Env: env,
         ...(service.container_command ? { Cmd: service.container_command.split(' ') } : {}),
         // Ensure container is attached to the Nomad docker network in production
         ...(process.env.NODE_ENV === 'production' && {
@@ -862,12 +890,76 @@ export class DockerService {
         }),
       })
 
+      let container = await this.docker.createContainer(
+        buildCreateOptions(finalImage, gpuHostConfig, [
+          ...(containerConfig?.Env ?? []),
+          ...ollamaEnv,
+          ...appEnv,
+        ])
+      )
+
       this._broadcast(
         service.service_name,
         'starting',
         `Starting Docker container for service ${service.service_name}...`
       )
-      await container.start()
+      try {
+        await container.start()
+      } catch (error: any) {
+        // An AMD GPU whose ROCm device nodes are absent must not block the install
+        // outright (#1232). GPU detection reads PCI data, so "an AMD GPU is present"
+        // and "ROCm is usable" are different questions, and only the daemon can
+        // answer the second. CPU-only Ollama works fine on these machines.
+        //
+        // This has to sit on start(), not createContainer(): the daemon accepts a
+        // container whose --device path does not exist (create returns 201) and only
+        // resolves devices when the container starts. Measured on Docker 29.6.2.
+        //
+        // Retrying rather than pre-checking is also deliberate. The admin runs in its
+        // own container, so stat()ing /dev/kfd here describes the admin's namespace,
+        // not the host's. Verified on NOMAD6, a working ROCm box: /dev/kfd is present
+        // on the host and absent inside the admin container, so a pre-check would
+        // strip GPU passthrough from a machine where it works.
+        if (!amdGpuConfigured || !this._isMissingDeviceError(error)) throw error
+
+        logger.warn(
+          `[DockerService] AMD device passthrough rejected by the daemon (${error?.message}); ` +
+            `recreating ${service.service_name} CPU-only on ${service.container_image}`
+        )
+        this._broadcast(
+          service.service_name,
+          'gpu-config',
+          `AMD GPU detected, but this system has no usable ROCm device (/dev/kfd is missing, ` +
+            `which means the amdgpu/ROCm kernel driver is not loaded or does not support this GPU). ` +
+            `Installing CPU-only instead so the AI Assistant still works.`
+        )
+
+        // The created container carries the rejected device config, so it cannot be
+        // started as-is and has to go before the name can be reused.
+        await container.remove({ force: true }).catch((removeError: any) => {
+          logger.warn(`[DockerService] Could not remove the failed container: ${removeError?.message}`)
+        })
+
+        const { Devices, ...cpuHostConfig } = gpuHostConfig as any
+        // Drop the ROCm-only env along with the devices. HSA_OVERRIDE_GFX_VERSION and
+        // OLLAMA_IGPU_ENABLE mean nothing to the CPU build, and leaving them behind is
+        // misleading to anyone who later reads `docker inspect`.
+        const cpuOllamaEnv = ollamaEnv.filter(
+          (e) => !e.startsWith('HSA_OVERRIDE_GFX_VERSION=') && !e.startsWith('OLLAMA_IGPU_ENABLE=')
+        )
+        amdGpuConfigured = false
+        // service.container_image is the CPU tag, already pulled earlier in this
+        // function before the AMD branch overrode finalImage to :rocm.
+        finalImage = service.container_image
+        container = await this.docker.createContainer(
+          buildCreateOptions(finalImage, cpuHostConfig, [
+            ...(containerConfig?.Env ?? []),
+            ...cpuOllamaEnv,
+            ...appEnv,
+          ])
+        )
+        await container.start()
+      }
 
       this._broadcast(
         service.service_name,
@@ -1472,6 +1564,17 @@ export class DockerService {
     }
   }
 
+  /**
+   * The HSA_OVERRIDE_GFX_VERSION an AMD install or reinstall would apply right now.
+   * Exposed so GpuPassthroughRemediationProvider can tell whether a reinstall would
+   * change the running container.
+   */
+  async getAmdHsaOverride(
+    options: { quiet?: boolean; containerEnv?: string[] | null } = {}
+  ): Promise<string | null> {
+    return this._resolveAmdHsaOverride(options)
+  }
+
   /**
    * Resolve the HSA_OVERRIDE_GFX_VERSION value for the host's AMD GPU.
    *
@@ -1485,9 +1588,11 @@ export class DockerService {
    * Resolution order:
    *   1. KV `ai.amdHsaOverride` — manual user override; accepts 'none' (disable) or a semver-style value.
    *   2. Marker file `/app/storage/.nomad-amd-gfx` written by install_nomad.sh.
-   *   3. Default: none — let ROCm discover the GPU natively. Users on hardware that still
+   *   3. The override the existing container runs with, when the caller passes its env (#1377).
+   *   4. Default: none — let ROCm discover the GPU natively. Users on hardware that still
    *      needs coercion can force a value via the KV. A hardcoded default gets more wrong
    *      as ROCm adds native targets, so null is the safer forward-looking default.
+   * The pure ordering lives in pickAmdHsaOverride (../utils/amd_hsa_override.ts).
    *
    * Returns null when no override should be applied.
    */
@@ -1511,43 +1616,76 @@ export class DockerService {
     return pepper
   }
 
-  private async _resolveAmdHsaOverride(): Promise<string | null> {
-    const manualRaw = await KVStore.getValue('ai.amdHsaOverride')
-    if (manualRaw !== null && manualRaw !== undefined && String(manualRaw).trim() !== '') {
-      const manual = String(manualRaw).trim().toLowerCase()
-      if (manual === 'none' || manual === 'off' || manual === 'false') {
+  private async _resolveAmdHsaOverride({
+    quiet = false,
+    containerEnv,
+  }: { quiet?: boolean; containerEnv?: string[] | null } = {}): Promise<string | null> {
+    const manual = await KVStore.getValue('ai.amdHsaOverride')
+    const markerGfx = await this._readAmdGfxMarker()
+    const resolved = pickAmdHsaOverride({ manual, markerGfx, containerEnv })
+    if (quiet) return resolved.value
+
+    if (resolved.invalidManual !== undefined) {
+      logger.warn(`[DockerService] Ignoring invalid ai.amdHsaOverride value: ${resolved.invalidManual}`)
+    }
+    switch (resolved.source) {
+      case 'kv-disabled':
         logger.info('[DockerService] HSA override disabled via ai.amdHsaOverride')
-        return null
-      }
-      if (/^\d+\.\d+\.\d+$/.test(manual)) {
-        logger.info(`[DockerService] HSA override forced to ${manual} via ai.amdHsaOverride`)
-        return manual
-      }
-      logger.warn(`[DockerService] Ignoring invalid ai.amdHsaOverride value: ${manualRaw}`)
+        break
+      case 'kv':
+        logger.info(`[DockerService] HSA override forced to ${resolved.value} via ai.amdHsaOverride`)
+        break
+      case 'marker':
+        logger.info(
+          `[DockerService] AMD gfx marker '${markerGfx}' → HSA override ${resolved.value ?? 'none'}`
+        )
+        break
+      case 'container':
+        logger.info(
+          `[DockerService] Keeping HSA override ${resolved.value} from the existing container`
+        )
+        break
+      case 'default':
+        logger.warn(
+          `[DockerService] AMD GPU configured but no gfx marker (${AMD_GFX_MARKER_PATH}) and no ` +
+            'ai.amdHsaOverride KV; relying on native ROCm discovery. iGPUs not on the bundled rocblas ' +
+            'allowlist (e.g. 780M/gfx1103, 680M/gfx1035) will silently fall back to CPU. Set the ' +
+            'ai.amdHsaOverride KV (e.g. 11.0.0 for a 780M) and force-reinstall the AI service if so.'
+        )
+        break
     }
-
-    try {
-      const gfx = (await readFile('/app/storage/.nomad-amd-gfx', 'utf8')).trim()
-      const mapped = this._mapGfxToHsaOverride(gfx)
-      logger.info(`[DockerService] AMD gfx marker '${gfx}' → HSA override ${mapped ?? 'none'}`)
-      return mapped
-    } catch {
-      // Marker absent — most likely an existing install upgraded without re-running
-      // install_nomad.sh. Fall through to the default.
-    }
-
-    logger.warn(
-      '[DockerService] AMD GPU configured but no gfx marker (/app/storage/.nomad-amd-gfx) and no ' +
-        'ai.amdHsaOverride KV; relying on native ROCm discovery. iGPUs not on the bundled rocblas ' +
-        'allowlist (e.g. 780M/gfx1103, 680M/gfx1035) will silently fall back to CPU. Set the ' +
-        'ai.amdHsaOverride KV (e.g. 11.0.0 for a 780M) and force-reinstall the AI service if so.'
-    )
-    return null
+    return resolved.value
   }
 
-  private _mapGfxToHsaOverride(gfx: string): string | null {
-    // Pure mapping lives in ../utils/amd_hsa_override.ts so it stays unit-testable.
-    return mapGfxToHsaOverride(gfx)
+  /**
+   * Before a force reinstall removes nomad_ollama, persist the HSA override it runs with
+   * when nothing else would supply one (#1377). The recreate builds env from scratch, so
+   * without this an upgraded box with a hand-set override loses it. Persisting rather than
+   * passing it along in memory keeps it through a reinstall that fails after removal, and
+   * shows it in the Set GFX Override modal.
+   */
+  private async _adoptContainerHsaOverride(containerEnv: string[] | null | undefined): Promise<void> {
+    const resolved = pickAmdHsaOverride({
+      manual: await KVStore.getValue('ai.amdHsaOverride'),
+      markerGfx: await this._readAmdGfxMarker(),
+      containerEnv,
+    })
+    if (resolved.source !== 'container' || !resolved.value) return
+
+    await KVStore.setValue('ai.amdHsaOverride', resolved.value)
+    logger.info(
+      `[DockerService] Persisted HSA override ${resolved.value} from the existing container to ai.amdHsaOverride`
+    )
+  }
+
+  private async _readAmdGfxMarker(): Promise<string | null> {
+    try {
+      return (await readFile(AMD_GFX_MARKER_PATH, 'utf8')).trim()
+    } catch {
+      // Marker absent: most likely an existing install upgraded without re-running
+      // install_nomad.sh.
+      return null
+    }
   }
 
   /**
@@ -1569,6 +1707,28 @@ export class DockerService {
     ]
   }
 
+  /**
+   * Whether a container-create failure was the daemon refusing a `--device` path
+   * that does not exist on the host, e.g.
+   *
+   *   (HTTP code 500) server error - error gathering device information while
+   *   adding custom device "/dev/kfd": no such file or directory
+   *
+   * GPU *detection* reads PCI data, which says nothing about whether the ROCm
+   * kernel interface was ever loaded. An AMD card with no `/dev/kfd` is a
+   * perfectly normal machine — old pre-ROCm silicon, or amdgpu not loaded — and
+   * on those the create fails outright and the whole install dies (#1232).
+   *
+   * Deliberately keyed on the device-gathering phrase rather than a bare "no
+   * such file or directory", which Docker also emits for missing bind-mount
+   * sources and image layers. Falling back to CPU on one of those would hide a
+   * real failure behind a degraded install.
+   */
+  private _isMissingDeviceError(error: any): boolean {
+    const raw: string = error?.message ?? String(error)
+    return /error gathering device information while adding custom device/i.test(raw)
+  }
+
   /**
    * Update a service container to a new image version while preserving volumes and data.
    * Includes automatic rollback if the new container fails health checks.
@@ -1727,7 +1887,7 @@ export class DockerService {
       const baseEnv = inspectData.Config?.Env || []
       let finalEnv = baseEnv
       if (updatedAmdGpuConfigured) {
-        const hsaOverride = await this._resolveAmdHsaOverride()
+        const hsaOverride = await this._resolveAmdHsaOverride({ containerEnv: baseEnv })
         finalEnv = baseEnv.filter(
           (e: string) =>
             !e.startsWith('HSA_OVERRIDE_GFX_VERSION=') && !e.startsWith('OLLAMA_IGPU_ENABLE=')

admin/app/services/download_service.ts

@@ -56,11 +56,10 @@ export class DownloadService {
   }
 
   async listDownloadJobs(filetype?: string): Promise<DownloadJobWithProgress[]> {
-    const modelQueue = this.queueService.getQueue(DownloadModelJob.queue)
-    const [fileTagged, extractTagged, modelJobs, drugTagged] = await Promise.all([
+    const [fileTagged, extractTagged, modelTagged, drugTagged] = await Promise.all([
       this.fetchJobsWithStates(RunDownloadJob.queue),
       this.fetchJobsWithStates(RunExtractPmtilesJob.queue),
-      modelQueue.getJobs(['waiting', 'active', 'delayed', 'failed']),
+      this.fetchJobsWithStates(DownloadModelJob.queue),
       this.fetchJobsWithStates(DownloadDrugDataJob.queue),
     ])
 
@@ -98,7 +97,7 @@ export class DownloadService {
       }
     })
 
-    const modelDownloads = modelJobs.map((job) => ({
+    const modelDownloads = modelTagged.map(({ job }) => ({
       jobId: job.id!.toString(),
       url: job.data.modelName || 'Unknown Model',
       progress: parseInt(job.progress.toString(), 10),
@@ -191,8 +190,13 @@ export class DownloadService {
           if (!modelName) {
             return { success: false, message: 'Cannot retry: model name not found in job data' }
           }
-          await DownloadModelJob.dispatch({ modelName })
+          // Remove the old failed job first, then dispatch a fresh one. The
+          // model jobId is a hash of the model name, so dispatching first meant
+          // this remove() targeted the job we had just enqueued under that same
+          // id rather than the old record, deleting the retry we were creating.
+          // Matches the file-download branch below.
           await job.remove().catch(() => {})
+          await DownloadModelJob.dispatch({ modelName })
           return { success: true, message: `Retrying download for model ${modelName}` }
         }
 

admin/app/services/drug_reference_service.ts

@@ -566,6 +566,60 @@ export class DrugReferenceService {
     return { ...result, nothingDownloaded: false }
   }
 
+  /**
+   * Write (or refresh) the `installed_resources` 'dataset' row that the tier-
+   * status math and the curated-install filter read. Shared by the ingest job's
+   * final pass, the ZimService tier-select path, and DrugInstallRowProvider's
+   * boot backfill so all three write the same shape. Throws; callers log.
+   */
+  async recordInstalledRow(opts: {
+    version: string
+    collectionRef: string | null
+    fileSizeBytes: number | null
+  }): Promise<void> {
+    const { default: InstalledResource } = await import('#models/installed_resource')
+    const { DateTime } = await import('luxon')
+    await InstalledResource.updateOrCreate(
+      { resource_id: DRUG_DATASET_RESOURCE_ID, resource_type: 'dataset' },
+      {
+        version: opts.version,
+        collection_ref: opts.collectionRef,
+        url: 'https://api.fda.gov/download.json',
+        // No single on-disk file — the parts are deleted after ingest; the
+        // installed artifact is the DB table. Record the staging dir for
+        // provenance; uninstall keys off resource_id, never this path.
+        file_path: STORAGE_BASE,
+        file_size_bytes: opts.fileSizeBytes,
+        installed_at: DateTime.now(),
+      }
+    )
+  }
+
+  /** Whether the `installed_resources` 'dataset' row exists. */
+  async hasInstalledRow(): Promise<boolean> {
+    const { default: InstalledResource } = await import('#models/installed_resource')
+    const row = await InstalledResource.query()
+      .where('resource_type', 'dataset')
+      .where('resource_id', DRUG_DATASET_RESOURCE_ID)
+      .first()
+    return row !== null
+  }
+
+  /**
+   * Whether either drug queue holds an active/waiting/delayed job. Counts the
+   * whole queue rather than the deterministic jobIds, since ingest continuations
+   * run without one. Throws when Redis is unreachable.
+   */
+  async isJobInFlight(): Promise<boolean> {
+    for (const queueName of [DownloadDrugDataJob.queue, IngestDrugDataJob.queue]) {
+      const counts = await QueueService.getInstance()
+        .getQueue(queueName)
+        .getJobCounts('active', 'waiting', 'delayed')
+      if ((counts.active ?? 0) + (counts.waiting ?? 0) + (counts.delayed ?? 0) > 0) return true
+    }
+    return false
+  }
+
   /**
    * Uninstall the offline FDA drug dataset — the curated-tier "remove" path.
    *

admin/app/services/eval_corpus_service.ts

@@ -0,0 +1,202 @@
+import { RagService } from '#services/rag_service'
+import { inject } from '@adonisjs/core'
+import logger from '@adonisjs/core/services/logger'
+import { createHash } from 'node:crypto'
+import { readdir, readFile } from 'node:fs/promises'
+import { basename, join, resolve } from 'node:path'
+import { KB_EVAL_COLLECTION } from '../../constants/kb_collections.js'
+import { EMBEDDING_MODEL_NAME } from '../../constants/ollama.js'
+import {
+  assertGoldensMatchCorpus,
+  computeCorpusFingerprint,
+  parseGoldens,
+  type Golden,
+} from '../utils/eval/golden_set.js'
+import { EVAL_CORPUS_DIR } from '../utils/eval/corpus_source.js'
+
+/** Where the golden question set lives, relative to the app root. */
+export const EVAL_GOLDENS_DIR = 'tests/eval/goldens'
+
+/**
+ * The suite that runs when none is named — i.e. `core.jsonl`. Every other
+ * `.jsonl` in the goldens directory is opt-in via `--suite`, so adding one
+ * cannot move the numbers an existing baseline was recorded against.
+ */
+export const DEFAULT_GOLDEN_SUITE = 'core'
+
+// EVAL_CORPUS_DIR and docIdFromSource live in app/utils/eval/corpus_source.ts:
+// they are pure path logic, and keeping them out of this service is what lets
+// the leak guard be unit-tested without booting AdonisJS.
+export { docIdFromSource, EVAL_CORPUS_DIR } from '../utils/eval/corpus_source.js'
+
+export type CorpusDocument = {
+  /** Filename without extension. This is the id goldens refer to. */
+  docId: string
+  /** Absolute path on disk; becomes the Qdrant `source` payload. */
+  path: string
+  text: string
+}
+
+export type IngestSummary = {
+  fingerprint: string
+  documents: number
+  chunks: number
+  removedBeforeIngest: number
+  failures: Array<{ docId: string; reason: string }>
+}
+
+/**
+ * Owns the frozen evaluation corpus: reading it off disk, fingerprinting it,
+ * and pushing it through NOMAD's real ingest path into the reserved
+ * `__nomad_eval__` collection tag.
+ *
+ * Ingesting through `RagService.embedAndStoreText` rather than writing vectors
+ * directly is the whole point — chunk size, the token-estimate ratio, the
+ * search_document prefix, and the embedding model are all in scope of the
+ * measurement, so a change to any of them shows up as a retrieval score
+ * movement instead of hiding.
+ */
+@inject()
+export class EvalCorpusService {
+  constructor(private ragService: RagService) {}
+
+  private corpusPath(): string {
+    return resolve(join(process.cwd(), EVAL_CORPUS_DIR))
+  }
+
+  /** Read every markdown document in the corpus, sorted for determinism. */
+  async loadCorpus(): Promise<CorpusDocument[]> {
+    const dir = this.corpusPath()
+    const entries = (await readdir(dir)).filter((f) => f.endsWith('.md')).sort()
+    if (entries.length === 0) {
+      throw new Error(`No corpus documents found in ${dir}`)
+    }
+    return Promise.all(
+      entries.map(async (file) => {
+        const path = join(dir, file)
+        return { docId: basename(file, '.md'), path, text: await readFile(path, 'utf8') }
+      })
+    )
+  }
+
+  /**
+   * Load and validate golden files, cross-checked against the corpus.
+   *
+   * Each `.jsonl` in the goldens directory is a *suite*, named after the file.
+   * Only `core` loads by default. That matters for comparability: aggregate
+   * metrics are means over whatever ran, so silently adding goldens to the
+   * default set would move recall@k and correctness for reasons unrelated to any
+   * change under test, and quietly invalidate every committed baseline. Opting
+   * in keeps a suite's numbers separate until someone asks for them.
+   */
+  async loadGoldens(suites: string[] = [DEFAULT_GOLDEN_SUITE]): Promise<Golden[]> {
+    const dir = resolve(join(process.cwd(), EVAL_GOLDENS_DIR))
+    const available = (await readdir(dir)).filter((f) => f.endsWith('.jsonl')).sort()
+    if (available.length === 0) throw new Error(`No golden files found in ${dir}`)
+
+    // '*' loads every suite. Used by `eval:corpus`, which validates the fixtures
+    // rather than measuring anything — a malformed golden should fail validation
+    // whether or not its suite is part of the default run.
+    const wanted = new Set(suites)
+    const files = wanted.has('*')
+      ? available
+      : available.filter((f) => wanted.has(basename(f, '.jsonl')))
+    if (files.length === 0) {
+      throw new Error(
+        `No golden suites matched [${suites.join(', ')}]. Available: ` +
+          available.map((f) => basename(f, '.jsonl')).join(', ')
+      )
+    }
+
+    const goldens: Golden[] = []
+    const seen = new Set<string>()
+    for (const file of files) {
+      const parsed = parseGoldens(await readFile(join(dir, file), 'utf8'), file)
+      for (const g of parsed) {
+        // parseGoldens dedupes within a file; this catches collisions across files.
+        if (seen.has(g.id)) throw new Error(`Duplicate golden id "${g.id}" in ${file}`)
+        seen.add(g.id)
+        goldens.push(g)
+      }
+    }
+
+    const corpus = await this.loadCorpus()
+    assertGoldensMatchCorpus(
+      goldens,
+      corpus.map((d) => d.docId)
+    )
+    return goldens
+  }
+
+  /**
+   * Hash the corpus together with the ingest parameters that shaped it.
+   * Reports carry this; two reports with different fingerprints are not
+   * comparable and `eval:compare` refuses to pretend otherwise.
+   */
+  async fingerprint(): Promise<string> {
+    const corpus = await this.loadCorpus()
+    return computeCorpusFingerprint(
+      {
+        documents: new Map(corpus.map((d) => [d.docId, d.text])),
+        chunkTokens: RagService.TARGET_TOKENS_PER_CHUNK,
+        chunkOverlapTokens: RagService.CHUNK_OVERLAP_TOKENS,
+        charToTokenRatio: RagService.CHAR_TO_TOKEN_RATIO,
+        embeddingModel: EMBEDDING_MODEL_NAME,
+        embeddingDimension: RagService.EMBEDDING_DIMENSION,
+      },
+      (input) => createHash('sha256').update(input).digest('hex')
+    )
+  }
+
+  /** Remove every eval point. Never touches user content. */
+  async reset(): Promise<number> {
+    const removed = await this.ragService.deleteCollectionPoints(KB_EVAL_COLLECTION)
+    logger.info(`[Eval] Removed ${removed} eval corpus chunks`)
+    return removed
+  }
+
+  /** How many eval chunks are currently in the vector store. */
+  async count(): Promise<number> {
+    return this.ragService.countChunksInCollection(KB_EVAL_COLLECTION)
+  }
+
+  /**
+   * Wipe and rebuild the eval corpus.
+   *
+   * Always a full rebuild: a partial re-ingest would leave the vector store in
+   * a state no fingerprint describes, and a fingerprint that does not describe
+   * the store is worse than no fingerprint at all.
+   */
+  async ingest(onProgress?: (docId: string, index: number, total: number) => void): Promise<IngestSummary> {
+    const corpus = await this.loadCorpus()
+    const removedBeforeIngest = await this.reset()
+
+    let chunks = 0
+    const failures: IngestSummary['failures'] = []
+
+    for (const [index, doc] of corpus.entries()) {
+      onProgress?.(doc.docId, index + 1, corpus.length)
+      try {
+        const result = await this.ragService.embedAndStoreText(doc.text, {
+          source: doc.path,
+          collection: KB_EVAL_COLLECTION,
+        })
+        if (!result) {
+          failures.push({ docId: doc.docId, reason: 'embedAndStoreText returned null' })
+          continue
+        }
+        chunks += result.chunks
+      } catch (error) {
+        failures.push({ docId: doc.docId, reason: error instanceof Error ? error.message : String(error) })
+      }
+    }
+
+    return {
+      fingerprint: await this.fingerprint(),
+      documents: corpus.length,
+      chunks,
+      removedBeforeIngest,
+      failures,
+    }
+  }
+}

admin/app/services/eval_generation_service.ts

@@ -0,0 +1,489 @@
+import { EvalCorpusService } from '#services/eval_corpus_service'
+import { OllamaService, type NomadChatUsage } from '#services/ollama_service'
+import { RagPipelineService } from '#services/rag_pipeline_service'
+import { inject } from '@adonisjs/core'
+import logger from '@adonisjs/core/services/logger'
+import { KB_EVAL_COLLECTION } from '../../constants/kb_collections.js'
+import { RAG_MIN_FINAL_SCORE } from '../../constants/ollama.js'
+import { resolveSamplerProfile } from '../utils/sampler.js'
+import type { OllamaChatMessage, ResponseStyle, SamplerProfile } from '../../types/ollama.js'
+import type { PipelineOptions, RetrievedChunk } from '../../types/rag.js'
+import { docIdFromSource } from '../utils/eval/corpus_source.js'
+import type { Golden } from '../utils/eval/golden_set.js'
+import {
+  scoreAnswer,
+  summarizeNumeric,
+  summarizeRepeats,
+  type GenerationScores,
+  type NumericSummary,
+  type RepeatStats,
+} from '../utils/eval/generation_metrics.js'
+
+/**
+ * The three ways to run a question, and what each one isolates.
+ *
+ * This is the part of the harness that answers "code bug or weak model?".
+ * Running the same question all three ways turns one ambiguous score into a
+ * decomposition:
+ *
+ * - `oracle` gives the model perfect context by construction. A low score here
+ *   is the model (or the prompt) failing to use good context — retrieval is
+ *   provably not at fault.
+ * - `e2e` is the real product. `oracle - e2e` is the cost of imperfect retrieval.
+ * - `noretrieval` is the model's parametric baseline. `e2e - noretrieval` is
+ *   what RAG is actually buying, and on the fictional questions it should be
+ *   nearly everything, since no model can know them.
+ */
+export type GenerationMode = 'oracle' | 'e2e' | 'noretrieval'
+
+/**
+ * Sentinel model name for the extractive reference run.
+ *
+ * `--model=mock` needs no Ollama at all: it answers by echoing whatever context
+ * the pipeline injected, and refuses when nothing was injected. That makes it
+ * two useful things at once — a way to exercise scoring and reporting with zero
+ * models installed (so the harness itself is CI-testable), and a genuine
+ * *ceiling* line: the best a perfectly extractive model could score given this
+ * retrieval. A real model below the mock line is the bottleneck; a mock line
+ * that is itself low means retrieval is.
+ */
+export const MOCK_MODEL = 'mock'
+
+/**
+ * Fixed sampling for eval runs. Never used by production chat.
+ *
+ * Temperature 0 is the default because a repeatable number is worth more than a
+ * realistic one for most of what this harness gates. But greedy decoding makes
+ * top_p, top_k and min_p inert — the highest-probability token wins whatever
+ * they say — so a run left at 0 cannot see a Response Style change at all. That
+ * is what `--style` is for: it sets `samplerStyle`, which replaces this
+ * temperature with the style's and sends the real truncation settings, so the
+ * chat path can actually be measured. The seed stays fixed either way, which
+ * makes a sampled run approximately rather than exactly reproducible; `--repeats`
+ * and the `unstable` counter are how that gets accounted for.
+ */
+export const EVAL_TEMPERATURE = 0
+export const EVAL_SEED = 42
+
+/**
+ * What a run sends to the decoder: a fixed temperature, a sampler profile, or
+ * neither. Exactly one of the three, which is why it is a shape rather than two
+ * loose optional arguments threaded through runCase.
+ */
+type EvalSampling = { temperature?: number; sampler?: SamplerProfile }
+
+function resolveEvalSampling(style: ResponseStyle | undefined): EvalSampling {
+  if (!style) return { temperature: EVAL_TEMPERATURE }
+  const sampler = resolveSamplerProfile(style)
+  // `off` resolves to no profile, and sends no temperature either — that is the
+  // pre-setting behaviour this harness needs to be able to reproduce.
+  return sampler ? { sampler } : {}
+}
+
+export type GenerationRunOptions = {
+  mode: GenerationMode
+  model: string
+  repeats?: number
+  topK?: number
+  scoreThreshold?: number
+  /** Post-rerank relevance floor. Defaults to the constant, never to the setting. */
+  minFinalScore?: number
+  /** Skip the history-aware rewrite even on multi-turn goldens. */
+  skipQueryRewrite?: boolean
+  /**
+   * Sample the way real chat does, under this Response Style, instead of at
+   * temperature 0.
+   *
+   * Three states, not two. Unset is the deterministic default and the right
+   * choice for anything being gated on. A style is the chat path as a user
+   * experiences it. `off` sends no sampler settings *and* no temperature, which
+   * is the pre-setting behaviour and therefore the honest control to compare a
+   * style against: `--style=auto` against `--style=off` measures the change,
+   * where `--style=auto` against a temperature-0 run measures the change plus
+   * the difference between greedy and sampled decoding.
+   *
+   * Model-declared samplers are deliberately not merged in even under `auto`:
+   * the harness scores a style, and a number that moved because one machine's
+   * model shipped a different modelfile is not a number worth committing.
+   */
+  samplerStyle?: ResponseStyle
+  onProgress?: (id: string, index: number, total: number) => void
+}
+
+export type GenerationCaseResult = {
+  id: string
+  tags: string[]
+  expectRefusal: boolean
+  /** One entry per repeat. */
+  answers: string[]
+  scores: GenerationScores[]
+  correctness: RepeatStats
+  refusalCorrectness: RepeatStats
+  leakageFree: RepeatStats
+  groundedness: NumericSummary | null
+  retrievedDocIds: string[]
+  injectedChunks: number
+  /**
+   * What the budget planner did with the window on this case.
+   *
+   * Without this a low score is ambiguous: it could mean the model answered
+   * badly, or it could mean the model never saw the material because the
+   * relevant turns were evicted. These fields tell the two apart.
+   */
+  budget?: {
+    contextWindow: number
+    estimatedPromptTokens: number
+    promptBudget: number
+    turnsDropped: number
+    chunksDropped: number
+    historyElided: boolean
+    queryTruncated: boolean
+  }
+  /**
+   * Estimated vs. the backend's real prompt token count. The estimator is the
+   * foundation every budget decision rests on, so its error is worth gating on.
+   */
+  promptTokenError?: number
+  /** Non-null only when something went wrong talking to the model. */
+  error?: string
+}
+
+export type GenerationAggregate = {
+  cases: number
+  /** Cases whose pass/fail flipped across repeats; excluded from gating. */
+  unstable: number
+  errors: number
+  correctness: number | null
+  refusalCorrectness: number | null
+  leakageRate: number | null
+  thinkTagLeakRate: number | null
+  markdownRate: number | null
+  groundedness: NumericSummary | null
+  meanAnswerLength: number | null
+  /** Share of cases where history had to be trimmed to fit the window. */
+  historyElidedRate: number | null
+  /** Share of cases where a retrieved chunk didn't fit the context budget. */
+  chunksDroppedRate: number | null
+  /** Mean absolute error of the token estimator against real prompt counts. */
+  promptTokenError: NumericSummary | null
+  meanPromptTokens: number | null
+}
+
+export type GenerationRunResult = {
+  params: {
+    mode: GenerationMode
+    model: string
+    repeats: number
+    /** Null when the run sent no temperature at all, under `--style=off`. */
+    temperature: number | null
+    seed: number
+    /** The Response Style sampled under; null for the deterministic default. */
+    samplerStyle: ResponseStyle | null
+    topK?: number
+    scoreThreshold?: number
+    minFinalScore: number
+  }
+  overall: GenerationAggregate
+  byTag: Record<string, GenerationAggregate>
+  cases: GenerationCaseResult[]
+  elapsedMs: number
+}
+
+@inject()
+export class EvalGenerationService {
+  constructor(
+    private ollamaService: OllamaService,
+    private pipeline: RagPipelineService,
+    private corpusService: EvalCorpusService
+  ) {}
+
+  async run(goldens: Golden[], options: GenerationRunOptions): Promise<GenerationRunResult> {
+    const repeats = Math.max(1, options.repeats ?? 3)
+    const started = Date.now()
+    const isMock = options.model === MOCK_MODEL
+    const sampling = resolveEvalSampling(options.samplerStyle)
+
+    // Oracle mode needs the corpus text on hand to synthesize perfect context.
+    const corpusText = options.mode === 'oracle' ? await this.loadCorpusText() : null
+
+    if (!isMock) await this.prepareModel(options.model)
+
+    const cases: GenerationCaseResult[] = []
+    for (const [index, golden] of goldens.entries()) {
+      options.onProgress?.(golden.id, index + 1, goldens.length)
+      cases.push(await this.runCase(golden, options, repeats, corpusText, isMock, sampling))
+    }
+
+    return {
+      params: {
+        mode: options.mode,
+        model: options.model,
+        repeats,
+        temperature: sampling.sampler?.temperature ?? sampling.temperature ?? null,
+        seed: EVAL_SEED,
+        samplerStyle: options.samplerStyle ?? null,
+        topK: options.topK,
+        scoreThreshold: options.scoreThreshold,
+        minFinalScore: options.minFinalScore ?? RAG_MIN_FINAL_SCORE,
+      },
+      overall: aggregateGeneration(cases),
+      byTag: aggregateGenerationByTag(cases),
+      cases,
+      elapsedMs: Date.now() - started,
+    }
+  }
+
+  /**
+   * Evict other resident models and burn one throwaway generation.
+   *
+   * Both borrowed from BenchmarkService, and for the same reasons it added
+   * them: a cold first run is dramatically slower and behaves differently, and
+   * leftover models in VRAM change how the one under test is scheduled. Quality
+   * runs are less timing-sensitive than throughput runs, but a first-token
+   * timeout or an OOM-driven CPU fallback absolutely does change the answer.
+   */
+  private async prepareModel(model: string): Promise<void> {
+    try {
+      await this.ollamaService.unloadAllChatModelsExcept(model)
+    } catch (error) {
+      logger.warn(`[Eval] Could not evict resident models: ${errorText(error)}`)
+    }
+    try {
+      await this.ollamaService.chat({
+        model,
+        messages: [{ role: 'user', content: 'Reply with the single word: ready' }],
+        temperature: EVAL_TEMPERATURE,
+        seed: EVAL_SEED,
+      })
+    } catch (error) {
+      // Best-effort: if warm-up fails the scored runs will surface the real
+      // problem with a better message than we could produce here.
+      logger.warn(`[Eval] Warm-up generation failed: ${errorText(error)}`)
+    }
+  }
+
+  private async loadCorpusText(): Promise<Map<string, { text: string; path: string }>> {
+    const docs = await this.corpusService.loadCorpus()
+    return new Map(docs.map((d) => [d.docId, { text: d.text, path: d.path }]))
+  }
+
+  private async runCase(
+    golden: Golden,
+    options: GenerationRunOptions,
+    repeats: number,
+    corpusText: Map<string, { text: string; path: string }> | null,
+    isMock: boolean,
+    sampling: EvalSampling
+  ): Promise<GenerationCaseResult> {
+    const messages: OllamaChatMessage[] = [
+      ...golden.turns.map((t) => ({ role: t.role, content: t.content })),
+      { role: 'user' as const, content: golden.query },
+    ]
+
+    const pipelineOptions: PipelineOptions = {
+      collection: KB_EVAL_COLLECTION,
+      topK: options.topK,
+      scoreThreshold: options.scoreThreshold,
+      // Pinned to the constant, not left to fall through to resolveMinFinalScore()
+      // — otherwise a `rag.minRelevance` slider set on one developer's machine
+      // would silently move every generation score. Same reasoning as the
+      // retrieval tier; see EvalRetrievalService.run.
+      minFinalScore: options.minFinalScore ?? RAG_MIN_FINAL_SCORE,
+      // The mock run must not touch Ollama at all — that is what makes it
+      // usable with no models installed. The rewrite is a chat-model call, so
+      // it is always skipped there (it would 404 and silently fall back, which
+      // works but quietly makes the "no model needed" claim untrue).
+      skipQueryRewrite: options.skipQueryRewrite || isMock,
+      // A developer's personal NOMAD.md would silently skew every score.
+      skipNomadMd: true,
+      // Pinned off rather than read from `rag.relevanceCheck`, for the same
+      // reason as minFinalScore. eval:retrieval --judge-model measures it.
+      relevanceCheckModel: null,
+    }
+
+    if (options.mode === 'oracle') {
+      pipelineOptions.oracleContext = golden.relevantDocIds.map((docId) => {
+        const doc = corpusText?.get(docId)
+        return {
+          text: doc?.text ?? '',
+          score: 1,
+          metadata: { source: doc?.path },
+        } satisfies RetrievedChunk
+      })
+    } else if (options.mode === 'noretrieval') {
+      // An empty oracle context short-circuits retrieval without injecting
+      // anything — the model answers from parametric memory alone.
+      pipelineOptions.oracleContext = []
+    }
+
+    const answers: string[] = []
+    const scores: GenerationScores[] = []
+    let retrievedDocIds: string[] = []
+    let injectedChunks = 0
+    let budget: GenerationCaseResult['budget']
+    let promptTokenError: number | undefined
+    let error: string | undefined
+
+    for (let attempt = 0; attempt < repeats; attempt++) {
+      try {
+        const trace = await this.pipeline.buildPrompt(messages, options.model, pipelineOptions)
+        retrievedDocIds = uniqueDocIds(trace.retrieved)
+        injectedChunks = trace.injected.length
+        const context = trace.injected.map((c) => c.text).join('\n\n')
+        if (trace.budget) {
+          budget = {
+            contextWindow: trace.budget.contextWindow,
+            estimatedPromptTokens: trace.budget.estimatedPromptTokens,
+            promptBudget: trace.budget.promptBudget,
+            turnsDropped: trace.budget.turnsDropped,
+            chunksDropped: trace.budget.chunksDropped,
+            historyElided: trace.budget.historyElided,
+            queryTruncated: trace.budget.queryTruncated,
+          }
+        }
+
+        const generated = isMock
+          ? { answer: mockAnswer(context), usage: undefined }
+          : await this.generate(options.model, trace.messages, trace.numCtx, trace.numPredict, sampling)
+        const answer = generated.answer
+
+        // The backend just reported the prompt's real token count. Comparing it
+        // to the estimate is the only way to know whether the budget above was
+        // built on a number worth trusting.
+        if (generated.usage?.promptTokens && trace.uncalibratedPromptTokens) {
+          promptTokenError =
+            Math.abs(generated.usage.promptTokens - trace.uncalibratedPromptTokens) /
+            generated.usage.promptTokens
+        }
+
+        answers.push(answer)
+        scores.push(
+          scoreAnswer({
+            answer,
+            context,
+            mustInclude: golden.mustInclude,
+            mustNotInclude: golden.mustNotInclude,
+            expectRefusal: golden.expectRefusal,
+          })
+        )
+      } catch (err) {
+        error = errorText(err)
+        break
+      }
+    }
+
+    return {
+      id: golden.id,
+      tags: golden.tags,
+      expectRefusal: golden.expectRefusal,
+      answers,
+      scores,
+      correctness: summarizeRepeats(scores.map((s) => s.correct)),
+      refusalCorrectness: summarizeRepeats(scores.map((s) => s.refusalCorrect)),
+      leakageFree: summarizeRepeats(scores.map((s) => s.leakage.length === 0)),
+      groundedness: summarizeNumeric(scores.map((s) => s.numericGroundedness)),
+      retrievedDocIds,
+      injectedChunks,
+      budget,
+      promptTokenError,
+      error,
+    }
+  }
+
+  private async generate(
+    model: string,
+    messages: OllamaChatMessage[],
+    numCtx?: number,
+    numPredict?: number,
+    sampling: EvalSampling = { temperature: EVAL_TEMPERATURE }
+  ): Promise<{ answer: string; usage?: NomadChatUsage }> {
+    const response = await this.ollamaService.chat({
+      model,
+      messages,
+      numCtx,
+      numPredict,
+      // Spread rather than passed as fields: an explicit temperature outranks a
+      // profile's inside OllamaService, so under a style it has to be absent
+      // entirely or the run would carry the style's truncation settings at
+      // temperature 0 and measure a decoder nobody uses.
+      ...sampling,
+      seed: EVAL_SEED,
+    })
+    return { answer: response.message.content.trim(), usage: response.usage }
+  }
+}
+
+/**
+ * The extractive reference answer: echo the context, or decline when there is
+ * none. Deterministic, model-free, and an honest ceiling for the current
+ * retrieval.
+ */
+export function mockAnswer(context: string): string {
+  if (!context.trim()) return "I don't have information about that."
+  return context
+}
+
+function uniqueDocIds(chunks: RetrievedChunk[]): string[] {
+  const ids: string[] = []
+  const seen = new Set<string>()
+  for (const chunk of chunks) {
+    const docId = docIdFromSource(chunk.metadata?.source)
+    if (docId && !seen.has(docId)) {
+      seen.add(docId)
+      ids.push(docId)
+    }
+  }
+  return ids
+}
+
+function errorText(error: unknown): string {
+  return error instanceof Error ? error.message : String(error)
+}
+
+const rate = (values: boolean[]): number | null =>
+  values.length === 0 ? null : values.filter(Boolean).length / values.length
+
+export function aggregateGeneration(cases: GenerationCaseResult[]): GenerationAggregate {
+  const scored = cases.filter((c) => c.scores.length > 0)
+  const allScores = scored.flatMap((c) => c.scores)
+
+  return {
+    cases: cases.length,
+    unstable: scored.filter((c) => c.correctness.unstable).length,
+    errors: cases.filter((c) => c.error).length,
+    // Mean pass-rate rather than all-or-nothing, so a case that passes 2 of 3
+    // is reported as 0.67 instead of being silently rounded either way.
+    correctness: scored.length === 0 ? null : mean(scored.map((c) => c.correctness.passRate)),
+    refusalCorrectness:
+      scored.length === 0 ? null : mean(scored.map((c) => c.refusalCorrectness.passRate)),
+    leakageRate: rate(allScores.map((s) => s.leakage.length > 0)),
+    thinkTagLeakRate: rate(allScores.map((s) => s.thinkTagLeak)),
+    markdownRate: rate(allScores.map((s) => s.markdownFormatted)),
+    groundedness: summarizeNumeric(allScores.map((s) => s.numericGroundedness)),
+    meanAnswerLength: allScores.length === 0 ? null : mean(allScores.map((s) => s.length)),
+    historyElidedRate: rate(cases.filter((c) => c.budget).map((c) => c.budget!.historyElided)),
+    chunksDroppedRate: rate(cases.filter((c) => c.budget).map((c) => c.budget!.chunksDropped > 0)),
+    promptTokenError: summarizeNumeric(
+      cases.map((c) => c.promptTokenError).filter((e): e is number => e !== undefined)
+    ),
+    meanPromptTokens: (() => {
+      const values = cases
+        .map((c) => c.budget?.estimatedPromptTokens)
+        .filter((v): v is number => v !== undefined)
+      return values.length === 0 ? null : mean(values)
+    })(),
+  }
+}
+
+export function aggregateGenerationByTag(
+  cases: GenerationCaseResult[]
+): Record<string, GenerationAggregate> {
+  const tags = new Set(cases.flatMap((c) => c.tags))
+  const out: Record<string, GenerationAggregate> = {}
+  for (const tag of [...tags].sort()) {
+    out[tag] = aggregateGeneration(cases.filter((c) => c.tags.includes(tag)))
+  }
+  return out
+}
+
+const mean = (values: number[]) => values.reduce((a, b) => a + b, 0) / values.length

admin/app/services/eval_report_service.ts

@@ -0,0 +1,158 @@
+import { inject } from '@adonisjs/core'
+import { execFile } from 'node:child_process'
+import { mkdir, readFile, writeFile } from 'node:fs/promises'
+import { dirname, join, resolve } from 'node:path'
+import { promisify } from 'node:util'
+import type { GenerationRunResult } from './eval_generation_service.js'
+import type { RetrievalRunResult } from './eval_retrieval_service.js'
+import { flattenByK, type EvalReport, type ReportMeta } from '../utils/eval/report.js'
+
+const run = promisify(execFile)
+
+export const EVAL_REPORTS_DIR = 'tests/eval/reports'
+export const EVAL_BASELINES_DIR = 'tests/eval/baselines'
+
+/**
+ * Turns run results into durable, comparable report files.
+ *
+ * Two formats, on purpose: JSON is what `eval:compare` diffs, and Markdown is
+ * what a human actually reads when a number moved and they need to know which
+ * question broke and what the model said.
+ */
+@inject()
+export class EvalReportService {
+  async buildMeta(
+    kind: 'retrieval' | 'generation',
+    corpusFingerprint: string,
+    params: Record<string, unknown>
+  ): Promise<ReportMeta> {
+    const [git, nomadVersion, platform] = await Promise.all([
+      readGitState(),
+      readNomadVersion(),
+      readPlatform(),
+    ])
+    return {
+      kind,
+      createdAt: new Date().toISOString(),
+      corpusFingerprint,
+      gitSha: git.sha,
+      gitBranch: git.branch,
+      gitDirty: git.dirty,
+      nomadVersion,
+      platform,
+      params,
+    }
+  }
+
+  fromRetrieval(meta: ReportMeta, result: RetrievalRunResult): EvalReport {
+    const flatten = (agg: RetrievalRunResult['overall']) => ({
+      ...flattenByK('recall', agg.recall),
+      ...flattenByK('hitRate', agg.hitRate),
+      ...flattenByK('precision', agg.precision),
+      ...flattenByK('ndcg', agg.ndcg),
+      mrr: agg.mrr,
+      emptyRateOnAnswerable: agg.emptyRateOnAnswerable,
+      nonEmptyRateOnRefusal: agg.nonEmptyRateOnRefusal,
+    })
+    return {
+      meta,
+      metrics: flatten(result.overall),
+      byTag: Object.fromEntries(Object.entries(result.byTag).map(([tag, agg]) => [tag, flatten(agg)])),
+      cases: result.cases,
+    }
+  }
+
+  fromGeneration(meta: ReportMeta, result: GenerationRunResult): EvalReport {
+    const flatten = (agg: GenerationRunResult['overall']) => ({
+      correctness: agg.correctness,
+      refusalCorrectness: agg.refusalCorrectness,
+      leakageRate: agg.leakageRate,
+      thinkTagLeakRate: agg.thinkTagLeakRate,
+      markdownRate: agg.markdownRate,
+      groundedness: agg.groundedness?.mean ?? null,
+      historyElidedRate: agg.historyElidedRate,
+      chunksDroppedRate: agg.chunksDroppedRate,
+      promptTokenError: agg.promptTokenError?.mean ?? null,
+      meanPromptTokens: agg.meanPromptTokens,
+    })
+    return {
+      meta,
+      metrics: flatten(result.overall),
+      byTag: Object.fromEntries(Object.entries(result.byTag).map(([tag, agg]) => [tag, flatten(agg)])),
+      // Answers are kept in full. When a score moves, the only question anyone
+      // actually asks is "what did it say?", and a truncated answer cannot
+      // answer it.
+      cases: result.cases,
+    }
+  }
+
+  /** Write `<slug>.json` and `<slug>.md`; returns the JSON path. */
+  async write(report: EvalReport, slug: string, markdown: string): Promise<string> {
+    const dir = resolve(join(process.cwd(), EVAL_REPORTS_DIR))
+    await mkdir(dir, { recursive: true })
+    const jsonPath = join(dir, `${slug}.json`)
+    await writeFile(jsonPath, JSON.stringify(report, null, 2))
+    await writeFile(join(dir, `${slug}.md`), markdown)
+    return jsonPath
+  }
+
+  async read(path: string): Promise<EvalReport> {
+    const parsed = JSON.parse(await readFile(resolve(path), 'utf8'))
+    if (!parsed?.meta?.kind || !parsed?.metrics) {
+      throw new Error(`${path} is not an eval report`)
+    }
+    return parsed as EvalReport
+  }
+
+  /**
+   * Baselines are filed under their corpus fingerprint. That is not cosmetic:
+   * it makes it structurally impossible to overwrite the baseline for one
+   * corpus with a run against another.
+   */
+  baselinePath(report: EvalReport, name: string): string {
+    return resolve(
+      join(process.cwd(), EVAL_BASELINES_DIR, report.meta.corpusFingerprint, `${name}.json`)
+    )
+  }
+
+  async promoteToBaseline(report: EvalReport, name: string): Promise<string> {
+    const path = this.baselinePath(report, name)
+    await mkdir(dirname(path), { recursive: true })
+    await writeFile(path, JSON.stringify(report, null, 2))
+    return path
+  }
+}
+
+async function readGitState() {
+  const git = async (args: string[]) => (await run('git', args)).stdout.trim()
+  try {
+    const [sha, branch, status] = await Promise.all([
+      git(['rev-parse', 'HEAD']),
+      git(['rev-parse', '--abbrev-ref', 'HEAD']),
+      git(['status', '--porcelain']),
+    ])
+    // A dirty tree matters: it means the report does not correspond to any
+    // commit, so a later "this regressed at sha X" claim would be unfounded.
+    return { sha, branch, dirty: status.length > 0 }
+  } catch {
+    return { sha: null, branch: null, dirty: false }
+  }
+}
+
+async function readNomadVersion(): Promise<string | null> {
+  try {
+    const pkg = JSON.parse(await readFile(resolve(join(process.cwd(), '..', 'package.json')), 'utf8'))
+    return pkg.version ?? null
+  } catch {
+    return null
+  }
+}
+
+async function readPlatform() {
+  const os = await import('node:os')
+  return {
+    cpuArchitecture: os.arch(),
+    osName: `${os.type()} ${os.release()}`,
+    nodeVersion: process.version,
+  }
+}

admin/app/services/eval_retrieval_service.ts

@@ -0,0 +1,236 @@
+import { EvalCorpusService } from '#services/eval_corpus_service'
+import { RagService } from '#services/rag_service'
+import { RelevanceJudgeService } from '#services/relevance_judge_service'
+import { inject } from '@adonisjs/core'
+import { KB_EVAL_COLLECTION } from '../../constants/kb_collections.js'
+import {
+  RAG_DEFAULT_SCORE_THRESHOLD,
+  RAG_DEFAULT_TOP_K,
+  RAG_MIN_FINAL_SCORE,
+} from '../../constants/ollama.js'
+import type { RetrievalStages } from '../../types/rag.js'
+import { docIdFromSource } from '../utils/eval/corpus_source.js'
+import type { Golden } from '../utils/eval/golden_set.js'
+import {
+  aggregate,
+  aggregateByTag,
+  DEFAULT_K_VALUES,
+  scoreCase,
+  type RetrievalAggregate,
+  type RetrievalCase,
+  type RetrievalCaseResult,
+  type ScoredChunk,
+} from '../utils/eval/retrieval_metrics.js'
+
+export type RetrievalRunOptions = {
+  topK?: number
+  scoreThreshold?: number
+  /** Post-rerank relevance floor. Defaults to the constant, never to the setting. */
+  minFinalScore?: number
+  kValues?: number[]
+  /** Score the raw dense / reranked / diversified orderings separately. */
+  ablate?: boolean
+  /** Also return every golden's full pre-floor candidate pool (see RetrievalStages.candidates). */
+  dump?: boolean
+  /**
+   * Run the relevance check on this model after the floor, as chat does when it
+   * is enabled. Unset keeps this tier model-free and deterministic.
+   */
+  judgeModel?: string
+}
+
+/** What the relevance check cost and did, when `judgeModel` was set. */
+export type JudgeStats = {
+  model: string
+  calls: number
+  /** Calls that failed open (timeout, error, unusable verdict). */
+  failures: number
+  rejected: number
+  meanMs: number
+  maxMs: number
+}
+
+/** One golden's full candidate pool, for simulating a relevance gate offline. */
+export type CandidatePool = {
+  id: string
+  tags: string[]
+  expectRefusal: boolean
+  relevantDocIds: string[]
+  candidates: Array<{ docId: string | null; score: number; semanticScore: number | null }>
+}
+
+export type StageAblation = {
+  dense: RetrievalAggregate
+  reranked: RetrievalAggregate
+  diversified: RetrievalAggregate
+}
+
+export type RetrievalRunResult = {
+  params: { topK: number; scoreThreshold: number; minFinalScore: number; kValues: number[] }
+  overall: RetrievalAggregate
+  byTag: Record<string, RetrievalAggregate>
+  cases: RetrievalCaseResult[]
+  ablation: StageAblation | null
+  /**
+   * Chunks that came back without a resolvable eval doc id. Non-zero means the
+   * collection filter leaked and the run is measuring the wrong corpus.
+   */
+  unresolvedChunks: number
+  /** Every golden's candidate pool, when `dump` was requested. */
+  pools: CandidatePool[] | null
+  judge: JudgeStats | null
+}
+
+/**
+ * Runs the golden question set through NOMAD's real retrieval path and scores
+ * the result.
+ *
+ * Deliberately does not touch the chat model. The only model call is the
+ * embedding of each query, whose output is stable, so two runs over the same
+ * corpus produce identical numbers on any hardware. That is what makes this the
+ * fast inner loop and the only tier worth gating CI on: a movement here is a
+ * code change, full stop.
+ *
+ * Multi-turn goldens are scored on their raw final message. Resolving the
+ * coreference would need the chat model, which would make this tier
+ * non-deterministic — so the multi-turn bucket here reports the honest floor,
+ * and the rewrite's contribution is measured in the generation tier instead.
+ */
+@inject()
+export class EvalRetrievalService {
+  constructor(
+    private ragService: RagService,
+    private corpusService: EvalCorpusService,
+    private judgeService: RelevanceJudgeService
+  ) {}
+
+  async run(goldens: Golden[], options: RetrievalRunOptions = {}): Promise<RetrievalRunResult> {
+    const topK = options.topK ?? RAG_DEFAULT_TOP_K
+    const scoreThreshold = options.scoreThreshold ?? RAG_DEFAULT_SCORE_THRESHOLD
+    // RAG_MIN_FINAL_SCORE, deliberately — NOT resolveMinFinalScore(). The chat
+    // path reads the user's `rag.minRelevance` setting; this tier must not, or a
+    // slider position on one developer's machine would silently move the numbers
+    // and the committed baseline would stop being reproducible anywhere else.
+    // Same reasoning as the harness omitting `skipRetrieval`.
+    const minFinalScore = options.minFinalScore ?? RAG_MIN_FINAL_SCORE
+    const kValues = options.kValues ?? DEFAULT_K_VALUES
+
+    const cases: RetrievalCase[] = []
+    const denseCases: RetrievalCase[] = []
+    const rerankedCases: RetrievalCase[] = []
+    const diversifiedCases: RetrievalCase[] = []
+    const pools: CandidatePool[] = []
+    const recordStages = options.ablate || options.dump
+    const judgeMs: number[] = []
+    let judgeFailures = 0
+    let judgeRejected = 0
+    let unresolvedChunks = 0
+
+    for (const golden of goldens) {
+      const stages: RetrievalStages = {}
+      let docs = await this.ragService.searchSimilarDocuments(
+        golden.query,
+        topK,
+        scoreThreshold,
+        KB_EVAL_COLLECTION,
+        recordStages ? stages : undefined,
+        minFinalScore
+      )
+
+      if (options.judgeModel && docs.length > 0) {
+        const verdict = await this.judgeService.judge(golden.query, docs, options.judgeModel)
+        judgeMs.push(verdict.ms)
+        if (!verdict.judged) judgeFailures++
+        judgeRejected += verdict.rejected
+        docs = verdict.kept
+      }
+
+      const retrieved: ScoredChunk[] = docs.map((d) => {
+        const docId = docIdFromSource(d.metadata?.source)
+        if (!docId) unresolvedChunks++
+        return { docId, score: d.score, semanticScore: d.metadata?.semantic_score }
+      })
+
+      cases.push(toCase(golden, retrieved))
+
+      if (options.dump) {
+        pools.push({
+          id: golden.id,
+          tags: golden.tags,
+          expectRefusal: golden.expectRefusal,
+          relevantDocIds: golden.relevantDocIds,
+          candidates: (stages.candidates ?? []).map((c) => ({
+            docId: docIdFromSource(c.source),
+            score: c.score,
+            semanticScore: c.semanticScore ?? null,
+          })),
+        })
+      }
+
+      if (options.ablate) {
+        denseCases.push(toCase(golden, stageToChunks(stages.dense)))
+        rerankedCases.push(toCase(golden, stageToChunks(stages.reranked)))
+        diversifiedCases.push(toCase(golden, stageToChunks(stages.diversified)))
+      }
+    }
+
+    const results = cases.map((c) => scoreCase(c, kValues))
+
+    const aggregateStage = (stageCases: RetrievalCase[]) =>
+      aggregate(stageCases, stageCases.map((c) => scoreCase(c, kValues)), kValues)
+
+    return {
+      params: { topK, scoreThreshold, minFinalScore, kValues },
+      overall: aggregate(cases, results, kValues),
+      byTag: aggregateByTag(cases, results, kValues),
+      cases: results,
+      ablation: options.ablate
+        ? {
+            dense: aggregateStage(denseCases),
+            reranked: aggregateStage(rerankedCases),
+            diversified: aggregateStage(diversifiedCases),
+          }
+        : null,
+      unresolvedChunks,
+      pools: options.dump ? pools : null,
+      judge: options.judgeModel
+        ? {
+            model: options.judgeModel,
+            calls: judgeMs.length,
+            failures: judgeFailures,
+            rejected: judgeRejected,
+            meanMs: judgeMs.length ? judgeMs.reduce((a, b) => a + b, 0) / judgeMs.length : 0,
+            maxMs: judgeMs.length ? Math.max(...judgeMs) : 0,
+          }
+        : null,
+    }
+  }
+
+  /** Confirm the corpus is actually ingested before reporting a score of zero. */
+  async assertCorpusReady(): Promise<number> {
+    const chunks = await this.corpusService.count()
+    if (chunks === 0) {
+      throw new Error(
+        'The eval corpus is not ingested — every score would be zero. Run: node ace eval:corpus --ingest'
+      )
+    }
+    return chunks
+  }
+}
+
+function toCase(golden: Golden, retrieved: ScoredChunk[]): RetrievalCase {
+  return {
+    id: golden.id,
+    tags: golden.tags,
+    retrieved,
+    relevantDocIds: golden.relevantDocIds,
+    expectRefusal: golden.expectRefusal,
+  }
+}
+
+function stageToChunks(stage: Array<{ source?: string; score: number }> | undefined): ScoredChunk[] {
+  return (stage ?? []).map((entry) => ({
+    docId: docIdFromSource(entry.source),
+    score: entry.score,
+  }))
+}

admin/app/services/kiwix_catalog_service.ts

@@ -29,9 +29,6 @@ const GITHUB_PMTILES_URL =
   'https://api.github.com/repos/Crosstalk-Solutions/project-nomad-maps/contents/pmtiles'
 
 const CATALOG_TIMEOUT_MS = 15000
-/** Bounded paginated fallback scan when the exact `name=` lookup comes up empty. */
-const KIWIX_PAGE_SIZE = 60
-const MAX_KIWIX_FETCHES = 5
 /** Concurrent ZIM catalog lookups — keep small to avoid hammering the mirror. */
 const ZIM_CHECK_CONCURRENCY = 4
 
@@ -113,14 +110,46 @@ export class KiwixCatalogService {
   async getLatestZim(resourceId: string): Promise<CatalogResult | null> {
     const pattern = new RegExp(`^${escapeRegex(resourceId)}_(\\d{4}-\\d{2})\\.zim$`)
 
-    // 1. Exact-name lookup (the robust path).
+    // 1. Exact-name lookup (the robust path). Resolves most books, whose OPDS `<name>`
+    //    is the full filename base: devdocs_en_python, zimgit-water_en,
+    //    freecodecamp_en_all, nhs.uk_en_medicines ...
     const named = await this.fetchZimEntries({ name: resourceId, count: 50, start: 0 })
     const exact = this.pickNewestZim(named, pattern)
     if (exact) return exact
 
-    // 2. Fallback: bounded keyword scan in case the catalog ignored `name=` or
-    //    indexes the book under a slightly different name.
-    return this.scanZimByQuery(resourceId, pattern)
+    // 2. Flavoured books index under the base WITHOUT the flavour, which the catalog
+    //    carries in a separate `<flavour>` element: `wikibooks_en_all_nopic_2026-04.zim`
+    //    is `<name>wikibooks_en_all</name><flavour>nopic</flavour>`. Our resource ids are
+    //    filename bases, so `name=wikibooks_en_all_nopic` matched nothing and every
+    //    maxi/mini/nopic book -- including every Wikipedia -- was unresolvable.
+    //    Retry against the base; `pattern` still decides, so a broader query is safe.
+    const base = this.stripFlavourSuffix(resourceId)
+    if (base) {
+      const byBase = await this.fetchZimEntries({ name: base, count: 50, start: 0 })
+      const flavoured = this.pickNewestZim(byBase, pattern)
+      if (flavoured) return flavoured
+    }
+
+    // Nothing in the catalog matches this id. There used to be a third step here: a
+    // bounded `q=` keyword scan, up to 5 paged requests per unresolved book on every
+    // check. It was removed because `q=` searches an entry's title and summary, never
+    // its filename, so it cannot match a resource id -- across every id tested it
+    // resolved nothing while the two `name=` lookups above resolved everything that
+    // was resolvable. It was pure repeat traffic aimed at a mirror that has just had
+    // to deploy anti-crawler measures.
+    return null
+  }
+
+  /**
+   * Drop the trailing `_<flavour>` segment from a resource id, or null when there is
+   * nothing to drop. Deliberately not matched against a flavour allowlist: openZIM adds
+   * flavours over time, and a wrong guess costs one extra catalog request because the
+   * authoritative filename check still gates every result.
+   */
+  private stripFlavourSuffix(resourceId: string): string | null {
+    const cut = resourceId.lastIndexOf('_')
+    if (cut <= 0) return null
+    return resourceId.slice(0, cut)
   }
 
   /** Newest catalog version of a single PMTiles map, or null if none/older. */
@@ -144,36 +173,8 @@ export class KiwixCatalogService {
     return latest
   }
 
-  private async scanZimByQuery(
-    resourceId: string,
-    pattern: RegExp
-  ): Promise<CatalogResult | null> {
-    let start = 0
-    let total = 0
-    let latest: CatalogResult | null = null
-
-    for (let i = 0; i < MAX_KIWIX_FETCHES; i++) {
-      const { entries, totalResults } = await this.fetchZimEntriesPage({
-        q: resourceId,
-        count: KIWIX_PAGE_SIZE,
-        start,
-      })
-      total = totalResults
-      if (entries.length === 0) break
-      start += entries.length
-
-      const candidate = this.pickNewestZim(entries, pattern)
-      if (candidate && (!latest || candidate.version > latest.version)) {
-        latest = candidate
-      }
-      if (start >= total) break
-    }
-    return latest
-  }
-
   private async fetchZimEntries(params: {
     name?: string
-    q?: string
     count: number
     start: number
   }): Promise<CatalogZimEntry[]> {
@@ -183,7 +184,6 @@ export class KiwixCatalogService {
 
   private async fetchZimEntriesPage(params: {
     name?: string
-    q?: string
     count: number
     start: number
   }): Promise<{ entries: CatalogZimEntry[]; totalResults: number }> {
@@ -191,9 +191,11 @@ export class KiwixCatalogService {
       params: {
         start: params.start,
         count: params.count,
-        lang: 'eng',
+        // No `lang` filter. This is a freshness check for an ALREADY-INSTALLED book, so
+        // the language is whatever the user installed; `lang: 'eng'` made every
+        // non-English ZIM permanently unresolvable here. The authoritative
+        // `^<id>_YYYY-MM.zim$` filename check is what resolves the right book.
         ...(params.name ? { name: params.name } : {}),
-        ...(params.q ? { q: params.q } : {}),
       },
       responseType: 'text',
       timeout: CATALOG_TIMEOUT_MS,

admin/app/services/map_service.ts

@@ -389,11 +389,7 @@ export class MapService implements IMapService {
   }
 
   async ensureBaseAssets(): Promise<boolean> {
-    const exists = await this.checkBaseAssetsExist()
-    if (!exists) {
-      const downloaded = await this.downloadBaseAssets()
-      if (!downloaded) return false
-    }
+    if (!(await this.ensureBaseAssetFiles())) return false
 
     try {
       await this.ensureWorldBasemap()
@@ -426,8 +422,9 @@ export class MapService implements IMapService {
    * base map" action so the one network dependency can be satisfied deliberately while online (#1030).
    */
   async provisionWorldBasemap(): Promise<boolean> {
-    const baseAssetsExist = await this.ensureBaseAssets()
-    if (!baseAssetsExist) return false
+    // Skip ensureBaseAssets(): it runs its own swallowed ensureWorldBasemap(), so a failure there
+    // would clear the in-flight promise and the call below would launch a second extract.
+    if (!(await this.ensureBaseAssetFiles())) return false
     await this.ensureWorldBasemap()
     return this.checkWorldBasemapExists()
   }
@@ -450,6 +447,13 @@ export class MapService implements IMapService {
     return this.worldBasemapInFlight
   }
 
+  /** Download the style/sprite/font base assets if missing. Does not touch the world basemap. */
+  private async ensureBaseAssetFiles(): Promise<boolean> {
+    const exists = await this.checkBaseAssetsExist()
+    if (exists) return true
+    return this.downloadBaseAssets()
+  }
+
   private async _setupWorldBasemap(): Promise<void> {
     const basePath = resolve(join(this.baseDirPath, 'pmtiles'))
     const filepath = resolve(join(basePath, WORLD_BASEMAP_FILENAME))

admin/app/services/ollama_compute.ts

@@ -0,0 +1,245 @@
+import type Docker from 'dockerode'
+import { mapGfxToHsaOverride, readHsaOverrideFromEnv } from '../utils/amd_hsa_override.js'
+
+/**
+ * Pure helpers for classifying which compute backend Ollama actually loaded,
+ * parsed from its container logs. Extracted so both SystemService (GPU health
+ * display) and GpuPassthroughRemediationProvider (auto-reinstall decision) share
+ * one source of truth, and so the logic is unit-testable without Docker.
+ * readOllamaStartupLogs is the one Docker-facing helper, kept here so both
+ * callers read the same log window.
+ *
+ * Ollama writes one `inference compute` log line per detected device a few seconds
+ * after startup, e.g.:
+ *   GPU:  ... msg="inference compute" id=0 library=CUDA compute=12.1 name=CUDA0 description="NVIDIA GB10" ...
+ *   CPU:  ... msg="inference compute" id=cpu library=cpu compute="" name=cpu ...
+ */
+
+export type OllamaComputeBackend = 'gpu' | 'cpu' | 'unknown'
+
+const INFERENCE_LINE_MARKER = 'msg="inference compute"'
+const GPU_LIBRARY_RE = /library=(CUDA|ROCm|Vulkan)\b/
+
+/**
+ * Classify Ollama's compute backend from raw container log text.
+ *
+ * - 'gpu'     if any `inference compute` line reports a GPU backend
+ *             (CUDA/ROCm/Vulkan). Ollama may emit a CPU line alongside a GPU
+ *             line (CPU is always listed as a fallback device), so a GPU line
+ *             anywhere means the GPU is usable.
+ * - 'cpu'     if `inference compute` lines exist but none report a GPU backend —
+ *             i.e. Ollama fell back to / was created with CPU only.
+ * - 'unknown' if no `inference compute` line is present (container too fresh,
+ *             logs rotated, or Ollama not started).
+ */
+export function classifyOllamaComputeBackend(logText: string): OllamaComputeBackend {
+  const lines = logText.split('\n').filter((line) => line.includes(INFERENCE_LINE_MARKER))
+  if (lines.length === 0) return 'unknown'
+
+  if (lines.some((line) => GPU_LIBRARY_RE.test(line))) return 'gpu'
+
+  // Lines exist but none matched a GPU library. A `library=cpu` line is a
+  // definitive CPU fallback; anything else (unrecognized library) stays unknown.
+  if (lines.some((line) => /library=cpu\b/.test(line))) return 'cpu'
+
+  return 'unknown'
+}
+
+export type OllamaGpuLibrary = 'CUDA' | 'ROCm' | 'Vulkan'
+
+export type OllamaGpuInfo = {
+  library: OllamaGpuLibrary
+  name: string
+  vramMiB: number
+}
+
+/**
+ * Parse the GPU device Ollama reported at startup, or null when no GPU line is
+ * present. Takes the first GPU line rather than the last `inference compute`
+ * line: Ollama also lists a CPU fallback device, and when that line lands last
+ * a working GPU would otherwise read as no GPU at all.
+ */
+export function parseOllamaGpuFromLogs(logText: string): OllamaGpuInfo | null {
+  const line = logText
+    .split('\n')
+    .find((l) => l.includes(INFERENCE_LINE_MARKER) && GPU_LIBRARY_RE.test(l))
+  if (!line) return null
+
+  const library = line.match(GPU_LIBRARY_RE)![1] as OllamaGpuLibrary
+  const descMatch = line.match(/description="([^"]+)"/)
+  const totalMatch = line.match(/total="([0-9.]+)\s*GiB"/)
+  const fallbackName = library === 'CUDA' ? 'NVIDIA GPU' : library === 'ROCm' ? 'AMD GPU' : 'GPU'
+
+  return {
+    library,
+    name: descMatch?.[1] || fallbackName,
+    vramMiB: totalMatch ? Math.round(Number.parseFloat(totalMatch[1]) * 1024) : 0,
+  }
+}
+
+export type GpuVendor = 'nvidia' | 'amd'
+
+/**
+ * The subset of `docker inspect nomad_ollama` the GPU checks read. Structural so
+ * dockerode's ContainerInspectInfo satisfies it and tests can pass plain objects.
+ */
+export type OllamaContainerSnapshot = {
+  Config?: { Image?: string; Env?: string[] | null }
+  HostConfig?: {
+    Devices?: Array<{ PathOnHost?: string }> | null
+    DeviceRequests?: Array<{ Driver?: string }> | null
+  }
+}
+
+function hasDevice(container: OllamaContainerSnapshot, path: string): boolean {
+  return (container.HostConfig?.Devices ?? []).some((d) => d.PathOnHost === path)
+}
+
+// Matches rocm in the tag only (ollama/ollama:rocm, :0.12.0-rocm), not the repo path.
+function isRocmImage(image: string | undefined): boolean {
+  return /:[^/:]*rocm[^/:]*$/i.test(image ?? '')
+}
+
+/**
+ * Which GPU vendor, if any, Ollama is expected to run on.
+ *
+ * The container's own config comes first because it records what the install
+ * actually set up, independent of how the host is detected today:
+ *   - an nvidia DeviceRequest means NVIDIA
+ *   - a ROCm image tag or a /dev/kfd device means AMD (/dev/dri alone is not
+ *     AMD-specific, Intel and Vulkan setups pass it too)
+ * A registered nvidia runtime comes next, which covers a container created
+ * CPU-only before the runtime was added.
+ *
+ * The persisted AMD GPU type counts only when no container exists. An AMD
+ * container without ROCm config is deliberate: either acceleration was turned
+ * off, or the daemon rejected /dev/kfd and the install fell back to CPU (#1232).
+ * Flagging it would offer a reinstall that rebuilds the same CPU container. AMD
+ * never registers a Docker runtime, so gating on the runtime alone skips every
+ * AMD box (#1344).
+ */
+export function resolveExpectedGpuVendor(input: {
+  hasNvidiaRuntime: boolean
+  amdConfigured: boolean
+  container: OllamaContainerSnapshot | null
+}): GpuVendor | null {
+  const { container } = input
+  if (container) {
+    if ((container.HostConfig?.DeviceRequests ?? []).some((r) => r.Driver === 'nvidia')) {
+      return 'nvidia'
+    }
+    if (isRocmImage(container.Config?.Image) || hasDevice(container, '/dev/kfd')) {
+      return 'amd'
+    }
+  }
+  if (input.hasNvidiaRuntime) return 'nvidia'
+  if (!container && input.amdConfigured) return 'amd'
+  return null
+}
+
+/**
+ * Where a running AMD nomad_ollama container differs from what a reinstall would
+ * create. An empty list means a reinstall would rebuild the same container, so it
+ * cannot move Ollama off the CPU and the fix lies elsewhere (usually the HSA
+ * override for an iGPU missing from ROCm's allowlist).
+ */
+export function diffAmdOllamaConfig(
+  container: OllamaContainerSnapshot,
+  desiredHsaOverride: string | null
+): string[] {
+  const drift: string[] = []
+  const env = container.Config?.Env ?? []
+
+  if (!isRocmImage(container.Config?.Image)) {
+    drift.push(`image is ${container.Config?.Image ?? 'unknown'}, not the ROCm build`)
+  }
+  if (!hasDevice(container, '/dev/kfd')) drift.push('/dev/kfd is not passed through')
+  if (!hasDevice(container, '/dev/dri')) drift.push('/dev/dri is not passed through')
+
+  const currentOverride = readHsaOverrideFromEnv(env)
+  if (currentOverride !== desiredHsaOverride) {
+    drift.push(
+      `HSA_OVERRIDE_GFX_VERSION is ${currentOverride ?? 'unset'}, expected ${desiredHsaOverride ?? 'unset'}`
+    )
+  }
+  if (!env.includes('OLLAMA_IGPU_ENABLE=1')) drift.push('OLLAMA_IGPU_ENABLE=1 is not set')
+
+  return drift
+}
+
+/**
+ * The gfx target Ollama named when it rejected an AMD GPU, e.g. 'gfx1103' from
+ *   msg="amdgpu is not supported (supported types:[gfx1030 ...])" gpu_type=gfx1103 ...
+ * Best-effort: the exact wording varies across Ollama releases, so callers treat
+ * null as "unknown" and fall back to asking the user.
+ */
+export function parseAmdGfxTargetFromLogs(logText: string): string | null {
+  return logText.match(/gpu_type=(gfx[0-9a-f]+)\b/i)?.[1]?.toLowerCase() ?? null
+}
+
+/**
+ * Why an AMD container expected on the GPU is running on CPU, in the shape
+ * GpuHealthStatus reports to the UI. desiredHsaOverride must come from
+ * DockerService.getAmdHsaOverride with this container's env, so the diff matches
+ * what a reinstall would build.
+ */
+export function diagnoseAmdCpuFallback(
+  container: OllamaContainerSnapshot,
+  logText: string,
+  desiredHsaOverride: string | null
+) {
+  const gfx = parseAmdGfxTargetFromLogs(logText)
+  const current = readHsaOverrideFromEnv(container.Config?.Env)
+  // With an override applied, Ollama logs the coerced target (gfx1100 for a 780M on
+  // 11.0.0), which maps to "no override" and would suggest removing a working value.
+  // The override the container carries is the better suggestion then (#1377).
+  const suggested = current ?? (gfx ? mapGfxToHsaOverride(gfx) : null)
+  return {
+    amdReinstallWouldChange: diffAmdOllamaConfig(container, desiredHsaOverride).length > 0,
+    ...(gfx && { amdGfxTarget: gfx }),
+    ...(current && { currentHsaOverride: current }),
+    ...(suggested && { suggestedHsaOverride: suggested }),
+  }
+}
+
+/**
+ * Read nomad_ollama's logs from its startup window (the five minutes after it
+ * booted), where the "inference compute" line is emitted. A tail:N read is
+ * fragile here: under embedding load the container writes >1000 lines/min,
+ * which pushes the line out of any reasonable tail within minutes, while the
+ * startup window is bounded and never ages out. Falls back to tail:500 when
+ * StartedAt is missing. Returns '' on any error so callers classify the backend
+ * as 'unknown' rather than throwing.
+ */
+export async function readOllamaStartupLogs(
+  container: Docker.Container,
+  inspect?: { State?: { StartedAt?: string } }
+): Promise<string> {
+  try {
+    const state = inspect ?? (await container.inspect())
+    const startedAtRaw = state?.State?.StartedAt
+    const startedAtMs = startedAtRaw ? new Date(startedAtRaw).getTime() : NaN
+
+    const logsOpts: {
+      stdout: true
+      stderr: true
+      follow: false
+      since?: number
+      until?: number
+      tail?: number
+    } = { stdout: true, stderr: true, follow: false }
+
+    if (Number.isFinite(startedAtMs) && startedAtMs > 0) {
+      const startedAtSec = Math.floor(startedAtMs / 1000)
+      logsOpts.since = startedAtSec
+      logsOpts.until = startedAtSec + 300
+    } else {
+      logsOpts.tail = 500
+    }
+
+    const buf = (await container.logs(logsOpts)) as unknown as Buffer
+    return buf.toString('utf8')
+  } catch {
+    return ''
+  }
+}

admin/app/services/rag_pipeline_service.ts

@@ -0,0 +1,363 @@
+import { ContextWindowService } from '#services/context_window_service'
+import { NomadMdService } from '#services/nomad_md_service'
+import { OllamaService } from '#services/ollama_service'
+import { RagService } from '#services/rag_service'
+import { RelevanceJudgeService } from '#services/relevance_judge_service'
+import { TokenCalibrationService } from '#services/token_calibration_service'
+import { inject } from '@adonisjs/core'
+import logger from '@adonisjs/core/services/logger'
+import {
+  RAG_CONTEXT_LIMITS,
+  RAG_DEFAULT_SCORE_THRESHOLD,
+  RAG_DEFAULT_TOP_K,
+  RAG_PLACEMENT,
+  QUERY_REWRITE_MAX_TOKENS,
+  SYSTEM_PROMPTS,
+} from '../../constants/ollama.js'
+import type { OllamaChatMessage } from '../../types/ollama.js'
+import type { PipelineOptions, PipelineTrace, RetrievalFloorStats, RetrievedChunk } from '../../types/rag.js'
+import { planPrompt } from '../utils/context_budget.js'
+import { estimateMessagesTokens } from '../utils/token_estimate.js'
+import { resolveMinFinalScore } from '../utils/rag_relevance.js'
+import { resolveTasksModel } from '../utils/tasks_model.js'
+import { buildContextBlock, getContextLimitsForModel } from '../utils/rag_prompt.js'
+import { QUERIES_SCHEMA, pickQueries, resolveStructured } from '../utils/structured_output.js'
+
+/**
+ * Everything that happens between "a user sent a message" and "a payload goes
+ * to Ollama": system-prompt assembly, history-aware query rewriting, retrieval,
+ * model-size-aware context trimming, and the num_ctx decision.
+ *
+ * This used to live inline in OllamaController.chat. It was moved here so there
+ * is exactly one implementation of the prompt-building pipeline — the chat
+ * endpoint and the eval harness both call `buildPrompt`, so a measurement of
+ * the harness is a measurement of production, not of a copy that drifts.
+ *
+ * The behaviour is a verbatim port. Every quirk preserved below is marked; the
+ * quirks are worth fixing but each one changes output, and the point of the
+ * harness is to stop changing output without measuring it.
+ */
+@inject()
+export class RagPipelineService {
+  constructor(
+    private ollamaService: OllamaService,
+    private ragService: RagService,
+    private nomadMdService: NomadMdService,
+    private contextWindowService: ContextWindowService,
+    private tokenCalibration: TokenCalibrationService,
+    private relevanceJudge: RelevanceJudgeService
+  ) {}
+
+  /**
+   * Build the exact message array to send to Ollama, plus a trace of every
+   * decision made along the way.
+   *
+   * The caller passes the conversation as received; this never mutates it.
+   */
+  async buildPrompt(
+    messages: OllamaChatMessage[],
+    model: string,
+    opts: PipelineOptions = {}
+  ): Promise<PipelineTrace> {
+    // Split the caller's conversation into the pieces the budget planner needs.
+    // Caller-supplied system messages are treated as stable prefix material and
+    // stay in front of everything, as before.
+    const callerSystem = messages.filter((msg) => msg.role === 'system')
+    const conversation = messages.filter((msg) => msg.role !== 'system')
+
+    const systemBlocks: OllamaChatMessage[] = [...callerSystem]
+
+    // Default formatting prompt, only when the caller supplied no system message.
+    if (callerSystem.length === 0) {
+      logger.debug('[RagPipeline] Injecting system prompt')
+      systemBlocks.push({ role: 'system', content: SYSTEM_PROMPTS.default })
+    }
+
+    // The user-managed NOMAD.md goes in front of the formatting prompt so the
+    // user's persistent instructions take precedence. Skipped in evals, where a
+    // developer's personal NOMAD.md would silently skew every score.
+    if (!opts.skipNomadMd) {
+      const nomadPrompt = await this.nomadMdService.getSystemPrompt()
+      if (nomadPrompt) {
+        logger.debug('[RagPipeline] Injecting NOMAD.md system prompt')
+        systemBlocks.unshift({ role: 'system', content: nomadPrompt })
+      }
+    }
+
+    // The current question is the last user message; everything before it is
+    // history. A conversation with no user message at all still has to produce a
+    // valid payload, so fall back to an empty question.
+    const lastUserIndex = conversation.map((m) => m.role).lastIndexOf('user')
+    const query: OllamaChatMessage =
+      lastUserIndex >= 0 ? conversation[lastUserIndex] : { role: 'user', content: '' }
+    const history = lastUserIndex >= 0 ? conversation.slice(0, lastUserIndex) : conversation
+
+    // Retrieval reads the conversation as the user wrote it — system prompts are
+    // deliberately excluded. They used to be inside the rewrite window, where
+    // they were transcribed as "Assistant" turns and polluted the rewritten query.
+    const working: OllamaChatMessage[] = [...systemBlocks, ...conversation]
+
+    const trace: PipelineTrace = {
+      rewrittenQuery: null,
+      didRewrite: false,
+      retrieved: [],
+      injected: [],
+      messages: working,
+      numCtx: undefined,
+      numPredict: undefined,
+      contextLimits: { maxResults: RAG_DEFAULT_TOP_K, maxTokens: 0 },
+      timings: { rewriteMs: 0, retrievalMs: 0, relevanceCheckMs: 0 },
+      minFinalScore: 0,
+      chunksBelowFloor: 0,
+      relevanceCheckModel: null,
+      chunksRejectedByCheck: 0,
+    }
+
+    // --- Retrieval -------------------------------------------------------
+    // oracleContext bypasses retrieval entirely (eval `oracle` mode).
+    let relevantDocs: RetrievedChunk[] = []
+    if (opts.oracleContext) {
+      relevantDocs = opts.oracleContext
+      trace.retrieved = relevantDocs
+    } else if (opts.skipRetrieval) {
+      // Retrieval turned off by the user (rag.enabled). Bail before the
+      // hasDocuments check, the rewrite LLM call and the vector search — the
+      // whole point is to spend nothing here. relevantDocs stays empty, so no
+      // context block is injected below.
+      logger.debug('[RagPipeline] Retrieval disabled by setting, skipping')
+    } else {
+      const rewriteStart = Date.now()
+      const { query: retrievalQuery, didRewrite } = await this.resolveRetrievalQuery(
+        conversation,
+        model,
+        opts
+      )
+      trace.timings.rewriteMs = Date.now() - rewriteStart
+      trace.rewrittenQuery = retrievalQuery
+      trace.didRewrite = didRewrite
+
+      if (retrievalQuery) {
+        const retrievalStart = Date.now()
+        // An explicit option always wins, so the eval harness never inherits
+        // whatever this machine's `rag.minRelevance` slider happens to be set to.
+        const minFinalScore = opts.minFinalScore ?? (await resolveMinFinalScore())
+        const floor: RetrievalFloorStats = { candidates: 0, belowFloor: 0 }
+        relevantDocs = await this.ragService.searchSimilarDocuments(
+          retrievalQuery,
+          opts.topK ?? RAG_DEFAULT_TOP_K,
+          opts.scoreThreshold ?? RAG_DEFAULT_SCORE_THRESHOLD,
+          opts.collection,
+          undefined,
+          minFinalScore,
+          floor
+        )
+        trace.timings.retrievalMs = Date.now() - retrievalStart
+        trace.retrieved = relevantDocs
+        trace.minFinalScore = minFinalScore
+        trace.chunksBelowFloor = floor.belowFloor
+        logger.debug(
+          `[RAG] Retrieved ${relevantDocs.length} relevant documents for query: "${retrievalQuery}"` +
+            (floor.belowFloor > 0
+              ? ` (${floor.belowFloor} of ${floor.candidates} dropped below the ${minFinalScore} relevance floor)`
+              : '')
+        )
+
+        // Opt-in relevance check (#1341). The floor cannot tell a coherent
+        // off-topic question from a real one on a large general-prose library;
+        // a model can, at ~8B. Judged against the rewritten query, which is the
+        // standalone form of what the user is asking this turn.
+        if (relevantDocs.length > 0) {
+          const checkModel =
+            opts.relevanceCheckModel !== undefined
+              ? opts.relevanceCheckModel
+              : await this.relevanceJudge.resolveModel(model)
+          if (checkModel) {
+            const verdict = await this.relevanceJudge.judge(retrievalQuery, relevantDocs, checkModel)
+            relevantDocs = verdict.kept
+            trace.retrieved = relevantDocs
+            trace.relevanceCheckModel = checkModel
+            trace.chunksRejectedByCheck = verdict.rejected
+            trace.timings.relevanceCheckMs = verdict.ms
+          }
+        }
+      }
+    }
+
+    // --- Budgeted assembly -------------------------------------------------
+    //
+    // Everything above decided *what* is available; this decides what fits. The
+    // model-size tier still caps how many chunks a small model is asked to read
+    // (a 1B model does not get better answers from five chunks), and the budget
+    // planner then enforces the hard token limit on top of that.
+    const modelInfo = await this.ollamaService.getModelInfo(model).catch(() => undefined)
+    const limits = getContextLimitsForModel(model, RAG_CONTEXT_LIMITS, modelInfo?.parameterSize)
+    trace.contextLimits = limits
+    const candidateDocs = relevantDocs.slice(0, limits.maxResults)
+
+    const contextWindow = await this.contextWindowService.windowFor(model)
+    const ratio = await this.tokenCalibration.ratioFor(model)
+
+    const plan = planPrompt({
+      systemBlocks,
+      history,
+      query,
+      ragChunks: candidateDocs,
+      renderRagBlock: (chunks) =>
+        SYSTEM_PROMPTS.rag_context(buildContextBlock(chunks as RetrievedChunk[])),
+      contextWindow,
+      ratio,
+      ragPlacement: opts.ragPlacement ?? RAG_PLACEMENT,
+    })
+
+    trace.messages = plan.messages as OllamaChatMessage[]
+    trace.injected = candidateDocs.slice(0, plan.trace.chunksKept)
+    trace.budget = plan.trace
+    trace.numCtx = contextWindow
+    trace.numPredict = plan.numPredict
+    // Recorded uncalibrated so TokenCalibrationService can compare like with like;
+    // applying the ratio here and again there would make the EWMA chase itself.
+    trace.uncalibratedPromptTokens = estimateMessagesTokens(plan.messages, 1)
+
+    if (plan.trace.turnsDropped > 0 || plan.trace.chunksDropped > 0 || plan.trace.queryTruncated) {
+      logger.info(
+        `[RagPipeline] Budget for ${model}: ${plan.trace.estimatedPromptTokens}/${plan.trace.promptBudget} tokens ` +
+          `(window ${contextWindow}); dropped ${plan.trace.turnsDropped} turn(s), ` +
+          `${plan.trace.chunksDropped} chunk(s)${plan.trace.queryTruncated ? ', truncated the question' : ''}`
+      )
+    }
+
+    return trace
+  }
+
+  /**
+   * Decide what string to hand to retrieval.
+   *
+   * Returns null when the RAG pipeline should be skipped entirely — an empty
+   * knowledge base, or a conversation with no user message at all.
+   */
+  private async resolveRetrievalQuery(
+    messages: OllamaChatMessage[],
+    model: string,
+    opts: PipelineOptions
+  ): Promise<{ query: string | null; didRewrite: boolean }> {
+    const lastUserMessage = [...messages].reverse().find((msg) => msg.role === 'user')
+
+    try {
+      // Skip the entire RAG pipeline if there are no documents to search.
+      const hasDocuments = await this.ragService.hasDocuments()
+      if (!hasDocuments) {
+        return { query: null, didRewrite: false }
+      }
+
+      if (opts.skipQueryRewrite) {
+        return { query: lastUserMessage?.content ?? null, didRewrite: false }
+      }
+
+      // Last 6 messages ≈ 3 turns.
+      //
+      // The caller now passes the conversation *without* system messages, so the
+      // window can no longer scoop up system prompts and transcribe them as
+      // "Assistant" turns — which is what the previous slice did on short
+      // conversations, feeding the rewriter the formatting rules as if the model
+      // had said them.
+      const recentMessages = messages.slice(-6)
+
+      // Skip rewriting on the very first turn — with only one user message there
+      // is no prior context to fold in, so the rewrite would just echo the
+      // message back at the cost of an extra LLM round-trip. From the first
+      // follow-up onward the rewrite carries entities from earlier turns
+      // ("the bars" -> "Hershey's bars chocolate poisoning dog"); without it,
+      // embeddings match nothing and the assistant loses the thread.
+      const userMessages = recentMessages.filter((msg) => msg.role === 'user')
+      if (userMessages.length < 2) {
+        return { query: lastUserMessage?.content ?? null, didRewrite: false }
+      }
+
+      const conversationContext = recentMessages
+        .map((msg) => {
+          const role = msg.role === 'user' ? 'User' : 'Assistant'
+          // Truncate assistant messages to keep the rewrite prompt manageable.
+          const content =
+            msg.role === 'assistant'
+              ? msg.content.slice(0, 200) + (msg.content.length > 200 ? '...' : '')
+              : msg.content
+          return `${role}: "${content}"`
+        })
+        .join('\n')
+
+      // Route to the tasks model when one is configured. This is the ancillary
+      // call that actually costs something: it runs every turn, before the
+      // answer, and it sends a prompt that shares no prefix with the chat. On a
+      // single-slot Ollama (OLLAMA_NUM_PARALLEL=1, the default) running it on the
+      // chat model evicts that model's cached prefix, so the conversation is
+      // re-prefilled from scratch on every turn no matter how stable we keep the
+      // ordering. A small dedicated model keeps the chat model's cache intact.
+      const rewriteModel = (await resolveTasksModel(this.ollamaService, model, undefined, '[RAG]')) ?? model
+
+      // A rewrite is a mechanical transformation, and QUERY_REWRITE_MAX_TOKENS is far
+      // too small a budget for a reasoning model to think and then answer — it would
+      // spend the whole cap thinking and get truncated with no query at all. Suppress
+      // reasoning at the source; `thinkingCapable` is what lets the compat transport
+      // send reasoning_effort:'none' instead of nothing. Memoized per model name.
+      const thinkingCapable = await this.ollamaService.checkModelHasThinking(rewriteModel)
+
+      const response = await this.ollamaService.chat({
+        model: rewriteModel,
+        messages: [
+          { role: 'system', content: SYSTEM_PROMPTS.query_rewrite },
+          {
+            role: 'user',
+            content: `Conversation:\n${conversationContext}\n\nRewritten Query:`,
+          },
+        ],
+        // A rewrite is a short, mechanical transformation. Cap it so a chatty
+        // small model can't spend a thousand tokens restating the question, and
+        // pin the sampler so the same conversation rewrites the same way.
+        numPredict: QUERY_REWRITE_MAX_TOKENS,
+        temperature: 0,
+        think: false,
+        thinkingCapable,
+        // Grammar-constrained on the native transport. An array rather than a bare
+        // string because multi-query fusion then costs a prompt change and nothing
+        // else; only the first entry is used today.
+        format: QUERIES_SCHEMA,
+      })
+
+      const raw = response.message.content.trim()
+      const structured = resolveStructured(raw, pickQueries, response.structured === true)
+      if (!structured.ok && structured.reason === 'constrained-parse-failed') {
+        // The grammar was applied and the output still didn't parse, which here means a
+        // rewrite truncated mid-object by QUERY_REWRITE_MAX_TOKENS. `raw` is a JSON
+        // fragment: embedding it would send the brace and the schema key to Qdrant as
+        // if they were the question. Losing the rewrite for this turn is the cheaper
+        // failure, so take the same path as a rewrite that produced nothing at all.
+        logger.warn(
+          `[RAG] Model "${rewriteModel}" broke the query grammar; falling back to the user message`
+        )
+        return { query: lastUserMessage?.content ?? null, didRewrite: false }
+      }
+      // Unconstrained backends never had the grammar applied, so the raw text is the
+      // rewrite and the old bare-string behaviour is still correct.
+      const rewrittenQuery = structured.ok ? structured.value[0] : raw
+      // Empty means the response was reasoning and nothing else (or was truncated
+      // mid-thought). Embedding an empty string would search the corpus for nothing
+      // and quietly poison retrieval for the rest of the conversation, so fall back
+      // to the raw message — the same shape as the first-turn skip above.
+      if (!rewrittenQuery) {
+        logger.warn(
+          `[RAG] Model "${rewriteModel}" produced no query text; falling back to the user message`
+        )
+        return { query: lastUserMessage?.content ?? null, didRewrite: false }
+      }
+      logger.info(`[RAG] Query rewritten: "${rewrittenQuery}"`)
+      return { query: rewrittenQuery, didRewrite: true }
+    } catch (error) {
+      logger.error(
+        `[RAG] Query rewriting failed: ${error instanceof Error ? error.message : error}`
+      )
+      // Fall back to the last user message rather than losing retrieval entirely.
+      return { query: lastUserMessage?.content ?? null, didRewrite: false }
+    }
+  }
+
+}

admin/app/services/relevance_judge_service.ts

@@ -0,0 +1,121 @@
+import { OllamaService } from '#services/ollama_service'
+import { inject } from '@adonisjs/core'
+import logger from '@adonisjs/core/services/logger'
+import KVStore from '#models/kv_store'
+import { RELEVANCE_CHECK_MAX_TOKENS, SYSTEM_PROMPTS } from '../../constants/ollama.js'
+import type { RetrievedChunk } from '../../types/rag.js'
+import {
+  buildJudgeInput,
+  isRelevanceCheckEnabled,
+  ON_TOPIC_SCHEMA,
+  pickOnTopic,
+} from '../utils/relevance_judge.js'
+import { resolveStructured } from '../utils/structured_output.js'
+import { resolveTasksModel } from '../utils/tasks_model.js'
+
+export type JudgeResult = {
+  /** Every chunk when the model said they fit the question, none when it said not. */
+  kept: RetrievedChunk[]
+  /** How many were dropped. 0 when the check failed open. */
+  rejected: number
+  /** False when the call failed and every chunk was kept unjudged. */
+  judged: boolean
+  ms: number
+}
+
+/**
+ * Asks a model whether the retrieved chunks are about what the user asked, and
+ * drops them all when it says no (#1341). Runs after the relevance floor, so it
+ * only ever sees the few chunks that were about to be injected and cited.
+ *
+ * Fails open by design. The floor has already removed the noise; this removes
+ * the coherent-but-off-topic remainder. A timeout or a garbled verdict costing
+ * one turn that precision is a much smaller failure than an answerable question
+ * losing its context because a model misbehaved.
+ */
+@inject()
+export class RelevanceJudgeService {
+  constructor(private ollamaService: OllamaService) {}
+
+  /**
+   * The model to run the check on this turn, or null to skip it. Opt-in via
+   * `rag.relevanceCheck`; runs on the tasks model when one is configured and on
+   * the chat model otherwise.
+   */
+  async resolveModel(chatModel: string): Promise<string | null> {
+    try {
+      if (!isRelevanceCheckEnabled(await KVStore.getValue('rag.relevanceCheck'))) return null
+    } catch (error) {
+      logger.warn(
+        `[RAG] Failed to read rag.relevanceCheck, skipping the check: ${error instanceof Error ? error.message : error}`
+      )
+      return null
+    }
+    return resolveTasksModel(this.ollamaService, chatModel, undefined, '[RAG]')
+  }
+
+  async judge(question: string, chunks: RetrievedChunk[], model: string): Promise<JudgeResult> {
+    const started = Date.now()
+    const unjudged = (): JudgeResult => ({
+      kept: chunks,
+      rejected: 0,
+      judged: false,
+      ms: Date.now() - started,
+    })
+    if (chunks.length === 0 || !question.trim()) return unjudged()
+
+    try {
+      const thinkingCapable = await this.ollamaService.checkModelHasThinking(model)
+      const response = await this.ollamaService.chat({
+        model,
+        messages: [
+          { role: 'system', content: SYSTEM_PROMPTS.relevance_check },
+          {
+            role: 'user',
+            content: buildJudgeInput(
+              question,
+              chunks.map((c) => ({
+                text: c.text,
+                // Titles only from content metadata. A filename is not a claim
+                // about what a passage says, and in the eval corpus it would
+                // hand the judge the answer ("distractor-…").
+                title:
+                  c.metadata?.full_title || c.metadata?.article_title || c.metadata?.archive_title,
+              }))
+            ),
+          },
+        ],
+        numPredict: RELEVANCE_CHECK_MAX_TOKENS,
+        temperature: 0,
+        think: false,
+        thinkingCapable,
+        format: ON_TOPIC_SCHEMA,
+      })
+
+      const verdict = resolveStructured(
+        response.message.content.trim(),
+        pickOnTopic,
+        response.structured === true
+      )
+      if (!verdict.ok) {
+        logger.warn(
+          `[RAG] Relevance check on "${model}" returned no usable verdict (${verdict.reason}); keeping all chunks`
+        )
+        return unjudged()
+      }
+
+      const ms = Date.now() - started
+      logger.debug(
+        `[RAG] Relevance check on "${model}": ${verdict.value ? 'on topic' : 'off topic'} (${chunks.length} chunk(s), ${ms}ms)`
+      )
+      return verdict.value
+        ? { kept: chunks, rejected: 0, judged: true, ms }
+        : { kept: [], rejected: chunks.length, judged: true, ms }
+    } catch (error) {
+      logger.warn(
+        `[RAG] Relevance check on "${model}" failed, keeping all chunks: ${error instanceof Error ? error.message : error}`
+      )
+      return unjudged()
+    }
+  }
+}

admin/app/services/system_service.ts

@@ -21,7 +21,18 @@ import env from '#start/env'
 import KVStore from '#models/kv_store'
 import { KV_STORE_SCHEMA, KVStoreKey } from '../../types/kv_store.js'
 import { isNewerVersion } from '../utils/version.js'
+import { isUnresolvedGpuModel } from '../utils/gpu_model.js'
+import {
+  classifyOllamaComputeBackend,
+  diagnoseAmdCpuFallback,
+  parseOllamaGpuFromLogs,
+  readOllamaStartupLogs,
+  resolveExpectedGpuVendor,
+  type OllamaGpuInfo,
+} from '#services/ollama_compute'
 import { invalidateAssistantNameCache } from '../../config/inertia.js'
+import { invalidateMinRelevanceCache } from '../utils/rag_relevance.js'
+import { invalidateResponseStyleCache } from '../utils/response_style.js'
 import { KiwixLibraryService } from '#services/kiwix_library_service'
 
 @inject()
@@ -100,6 +111,20 @@ export class SystemService {
     return false
   }
 
+  /**
+   * The running nomad_ollama container and its inspect data, or null when it is
+   * not running. The inspect data records the GPU config the install actually
+   * applied (DeviceRequests, /dev/kfd, the ROCm image), which is what the GPU
+   * health check keys on.
+   */
+  private async _findRunningOllamaContainer() {
+    const containers = await this.dockerService.docker.listContainers({ all: false })
+    const summary = containers.find((c) => c.Names.includes(`/${SERVICE_NAMES.OLLAMA}`))
+    if (!summary) return null
+    const container = this.dockerService.docker.getContainer(summary.Id)
+    return { container, inspect: await container.inspect() }
+  }
+
   /**
    * Probe Ollama startup logs for the canonical "inference compute" line that records
    * which compute backend was selected. This catches silent CPU fallback (e.g. when
@@ -111,70 +136,11 @@ export class SystemService {
    *   - the line has not been emitted (Ollama still starting up)
    *   - logs show CPU-only operation (no GPU detected)
    */
-  async getOllamaInferenceComputeFromLogs(): Promise<{
-    library: 'CUDA' | 'ROCm'
-    name: string
-    vramMiB: number
-  } | null> {
+  async getOllamaInferenceComputeFromLogs(): Promise<OllamaGpuInfo | null> {
     try {
-      const containers = await this.dockerService.docker.listContainers({ all: false })
-      const ollamaContainer = containers.find((c) => c.Names.includes(`/${SERVICE_NAMES.OLLAMA}`))
-      if (!ollamaContainer) return null
-
-      const container = this.dockerService.docker.getContainer(ollamaContainer.Id)
-
-      // Read logs only from the first 5 minutes after container start. The
-      // "inference compute" line is written once during Ollama's GPU discovery
-      // phase, within seconds of startup. Using tail:N here is fragile: under
-      // active embedding workloads we've seen >1000 lines/min, which pushes the
-      // line past any reasonable tail in minutes. Pinning to the startup window
-      // is bounded (~5 min of logs regardless of container uptime) and never
-      // ages out.
-      //
-      // Fall back to the previous tail:500 strategy if StartedAt is missing or
-      // unparseable — we can't construct a since/until window without it, but
-      // tail:500 is still useful when the container just started and the line
-      // is still recent.
-      const inspect = await container.inspect()
-      const startedAtRaw = inspect?.State?.StartedAt
-      const startedAtMs = startedAtRaw ? new Date(startedAtRaw).getTime() : NaN
-      const hasValidStartedAt = Number.isFinite(startedAtMs) && startedAtMs > 0
-
-      const logsOpts: { stdout: true; stderr: true; follow: false; since?: number; until?: number; tail?: number } = {
-        stdout: true,
-        stderr: true,
-        follow: false,
-      }
-      if (hasValidStartedAt) {
-        const startedAtSec = Math.floor(startedAtMs / 1000)
-        logsOpts.since = startedAtSec
-        logsOpts.until = startedAtSec + 300 // 5-minute window
-      } else {
-        logger.warn(
-          `[SystemService] nomad_ollama State.StartedAt missing or invalid (${startedAtRaw ?? 'undefined'}); falling back to tail:500 for inference-compute probe`
-        )
-        logsOpts.tail = 500
-      }
-      const buf = (await container.logs(logsOpts)) as unknown as Buffer
-      const logs = buf.toString('utf8')
-
-      const lines = logs.split('\n').filter((l) => l.includes('msg="inference compute"'))
-      if (lines.length === 0) return null
-
-      const lastLine = lines[lines.length - 1]
-      const libraryMatch = lastLine.match(/library=(CUDA|ROCm)/)
-      if (!libraryMatch) return null
-
-      const descMatch = lastLine.match(/description="([^"]+)"/)
-      const totalMatch = lastLine.match(/total="([0-9.]+)\s*GiB"/)
-
-      return {
-        library: libraryMatch[1] as 'CUDA' | 'ROCm',
-        name:
-          descMatch?.[1] ||
-          (libraryMatch[1] === 'CUDA' ? 'NVIDIA GPU' : 'AMD GPU'),
-        vramMiB: totalMatch ? Math.round(Number.parseFloat(totalMatch[1]) * 1024) : 0,
-      }
+      const ollama = await this._findRunningOllamaContainer()
+      if (!ollama) return null
+      return parseOllamaGpuFromLogs(await readOllamaStartupLogs(ollama.container, ollama.inspect))
     } catch (error) {
       logger.warn(
         `[SystemService] Failed to probe Ollama logs for inference compute line: ${error instanceof Error ? error.message : error}`
@@ -341,6 +307,8 @@ export class SystemService {
         'is_custom',
         'is_user_modified',
         'is_deprecated',
+        'is_link_tile',
+        'link_color',
         'category'
       )
       .where('is_dependency_service', false)
@@ -381,6 +349,8 @@ export class SystemService {
         is_custom: service.is_custom,
         is_user_modified: service.is_user_modified,
         is_deprecated: service.is_deprecated,
+        is_link_tile: service.is_link_tile,
+        link_color: service.link_color,
         category: service.category,
       })
     }
@@ -500,91 +470,125 @@ export class SystemService {
           }
         }
 
-        // Run the probes when controllers are empty (common inside Docker) or
-        // when lspci gave us bogus discrete-GPU BAR0 values that need replacing.
-        if (
+        // The same pci.ids staleness that produces bogus VRAM also produces a
+        // placeholder model name — a card newer than the container's pci.ids is
+        // reported as its raw id, e.g. "Device 2d05" for an RTX 5060 (#1165).
+        //
+        // These are independent symptoms, not one. lspci can hand back a
+        // perfectly plausible BAR0 reading alongside an unresolved name, and in
+        // that case nothing above fires, no probe runs, and Settings > System
+        // renders the raw PCI id as the GPU model (#1196). NVIDIA usually hides
+        // this because the nvidia-smi path tends to be reached for other
+        // reasons; AMD has no equivalent, so it shows the raw id.
+        //
+        // The probes below resolve a real name from Ollama's own startup log,
+        // so trigger them on an unresolved name too rather than only on VRAM.
+        const hasUnresolvedGpuName = (graphics.controllers || []).some((c) =>
+          isUnresolvedGpuModel(c.model || '')
+        )
+
+        // Run the probes when controllers are empty (common inside Docker),
+        // when lspci gave us bogus discrete-GPU BAR0 values that need replacing,
+        // or when it named a card it couldn't resolve. Only in those cases do
+        // probe results replace the lspci controllers; a clean lspci reading
+        // stays as the display source.
+        const lspciNeedsProbe =
           !graphics.controllers ||
           graphics.controllers.length === 0 ||
-          hasLspciBogusDgpuVram
-        ) {
-          const runtimes = dockerInfo.Runtimes || {}
-          gpuHealth.hasNvidiaRuntime = 'nvidia' in runtimes
+          hasLspciBogusDgpuVram ||
+          hasUnresolvedGpuName
+        const useProbeControllers = (controllers: typeof graphics.controllers) => {
+          if (lspciNeedsProbe) graphics.controllers = controllers
+        }
+        const toControllers = (gpus: Array<{ vendor: string; model: string; vram: number }>) =>
+          gpus.map((gpu) => ({
+            model: gpu.model,
+            vendor: gpu.vendor,
+            bus: '',
+            vram: gpu.vram,
+            vramDynamic: false,
+          }))
 
-          // AMD doesn't register a Docker runtime. Detection sources, in priority order:
-          //   1. KV 'gpu.type' (set by DockerService._detectGPUType after first Ollama install)
-          //   2. Marker file at /app/storage/.nomad-gpu-type (written by install_nomad.sh)
-          // The marker file matters because the System page should reflect AMD presence
-          // even before AI Assistant has been installed for the first time.
-          let savedGpuType: string | null | undefined = await KVStore.getValue('gpu.type') as string | undefined
-          if (!savedGpuType) {
-            try {
-              savedGpuType = (await readFile('/app/storage/.nomad-gpu-type', 'utf8')).trim()
-            } catch {}
-          }
-          const amdEnabledRaw = await KVStore.getValue('ai.amdGpuAcceleration')
-          const amdAccelerationEnabled = String(amdEnabledRaw) !== 'false'
-          gpuHealth.hasRocmRuntime = savedGpuType === 'amd' && amdAccelerationEnabled
+        // Runtime flags are reported whichever path runs below. They used to be set
+        // only inside the probe branch, so a box whose lspci reading looked clean
+        // reported hasRocmRuntime: false even with an AMD GPU configured (#1344).
+        const runtimes = dockerInfo.Runtimes || {}
+        gpuHealth.hasNvidiaRuntime = 'nvidia' in runtimes
 
-          if (gpuHealth.hasNvidiaRuntime || gpuHealth.hasRocmRuntime) {
-            gpuHealth.gpuVendor = gpuHealth.hasNvidiaRuntime ? 'nvidia' : 'amd'
+        // AMD doesn't register a Docker runtime. Detection sources, in priority order:
+        //   1. KV 'gpu.type' (set by DockerService._detectGPUType after first Ollama install)
+        //   2. Marker file at /app/storage/.nomad-gpu-type (written by install_nomad.sh)
+        // The marker file matters because the System page should reflect AMD presence
+        // even before AI Assistant has been installed for the first time.
+        let savedGpuType: string | null | undefined = await KVStore.getValue('gpu.type') as string | undefined
+        if (!savedGpuType) {
+          try {
+            savedGpuType = (await readFile('/app/storage/.nomad-gpu-type', 'utf8')).trim()
+          } catch {}
+        }
+        const amdEnabledRaw = await KVStore.getValue('ai.amdGpuAcceleration')
+        const amdAccelerationEnabled = String(amdEnabledRaw) !== 'false'
+        gpuHealth.hasRocmRuntime = savedGpuType === 'amd' && amdAccelerationEnabled
+
+        // When Ollama is expected on a GPU, its startup log decides health whatever
+        // lspci returned. A clean lspci reading says a GPU exists, not that Ollama
+        // uses it: an AMD iGPU silently on CPU used to land in the else branch below
+        // and report 'ok' (#1344).
+        const ollama = await this._findRunningOllamaContainer().catch(() => null)
+        const expectedVendor = resolveExpectedGpuVendor({
+          hasNvidiaRuntime: gpuHealth.hasNvidiaRuntime,
+          amdConfigured: gpuHealth.hasRocmRuntime,
+          container: ollama?.inspect ?? null,
+        })
+
+        if (expectedVendor || lspciNeedsProbe) {
+          if (expectedVendor) {
+            gpuHealth.gpuVendor = expectedVendor
 
             // Primary probe: Ollama log parsing — works for both vendors and catches silent fallback
-            const logInfo = await this.getOllamaInferenceComputeFromLogs()
+            const ollamaLogs = ollama
+              ? await readOllamaStartupLogs(ollama.container, ollama.inspect)
+              : ''
+            const logInfo = parseOllamaGpuFromLogs(ollamaLogs)
             if (logInfo) {
-              graphics.controllers = [
+              useProbeControllers([
                 {
                   model: logInfo.name,
-                  vendor: logInfo.library === 'CUDA' ? 'NVIDIA' : 'AMD',
+                  vendor:
+                    logInfo.library === 'CUDA' ? 'NVIDIA' : logInfo.library === 'ROCm' ? 'AMD' : '',
                   bus: '',
                   vram: logInfo.vramMiB,
                   vramDynamic: false,
                 },
-              ]
+              ])
               gpuHealth.status = 'ok'
               gpuHealth.ollamaGpuAccessible = true
-            } else if (gpuHealth.hasNvidiaRuntime) {
+            } else if (classifyOllamaComputeBackend(ollamaLogs) === 'cpu') {
+              // Ollama reported its devices and none was a GPU. Definitive for both
+              // vendors, so skip the nvidia-smi exec, which cannot run in a CPU-only
+              // container anyway.
+              gpuHealth.status = 'passthrough_failed'
+              logger.warn(
+                `${expectedVendor === 'amd' ? 'AMD' : 'NVIDIA'} GPU expected but Ollama reported a CPU-only backend at startup`
+              )
+            } else if (expectedVendor === 'nvidia') {
               // NVIDIA secondary path: nvidia-smi exec preserves prior behavior when
               // the log parser hasn't seen a startup line yet (e.g. log rotation,
               // very fresh container). Distinguishes "no Ollama container" from
               // "container exists but GPU broken".
               const nvidiaInfo = await this.getNvidiaSmiInfo()
               if (Array.isArray(nvidiaInfo)) {
-                graphics.controllers = nvidiaInfo.map((gpu) => ({
-                  model: gpu.model,
-                  vendor: gpu.vendor,
-                  bus: '',
-                  vram: gpu.vram,
-                  vramDynamic: false,
-                }))
+                useProbeControllers(toControllers(nvidiaInfo))
                 gpuHealth.status = 'ok'
                 gpuHealth.ollamaGpuAccessible = true
-              } else if (nvidiaInfo === 'OLLAMA_NOT_FOUND') {
-                const externalOllamaGpu = await this.getExternalOllamaGpuInfo()
-                if (externalOllamaGpu) {
-                  graphics.controllers = externalOllamaGpu.map((gpu) => ({
-                    model: gpu.model,
-                    vendor: gpu.vendor,
-                    bus: '',
-                    vram: gpu.vram,
-                    vramDynamic: false,
-                  }))
-                  gpuHealth.status = 'ok'
-                  gpuHealth.ollamaGpuAccessible = true
-                } else {
-                  gpuHealth.status = 'ollama_not_installed'
-                }
               } else {
                 const externalOllamaGpu = await this.getExternalOllamaGpuInfo()
                 if (externalOllamaGpu) {
-                  graphics.controllers = externalOllamaGpu.map((gpu) => ({
-                    model: gpu.model,
-                    vendor: gpu.vendor,
-                    bus: '',
-                    vram: gpu.vram,
-                    vramDynamic: false,
-                  }))
+                  useProbeControllers(toControllers(externalOllamaGpu))
                   gpuHealth.status = 'ok'
                   gpuHealth.ollamaGpuAccessible = true
+                } else if (nvidiaInfo === 'OLLAMA_NOT_FOUND') {
+                  gpuHealth.status = 'ollama_not_installed'
                 } else {
                   gpuHealth.status = 'passthrough_failed'
                   logger.warn(
@@ -592,38 +596,46 @@ export class SystemService {
                   )
                 }
               }
-            } else {
+            } else if (!ollama) {
               // AMD path: no nvidia-smi equivalent worth running — log parser is authoritative.
               // Distinguish "Ollama not running" from "Ollama running but no GPU log line".
-              const containers = await this.dockerService.docker.listContainers({ all: false })
-              const ollamaRunning = containers.some((c) =>
-                c.Names.includes(`/${SERVICE_NAMES.OLLAMA}`)
-              )
-              if (!ollamaRunning) {
-                const externalOllamaGpu = await this.getExternalOllamaGpuInfo()
-                if (externalOllamaGpu) {
-                  graphics.controllers = externalOllamaGpu.map((gpu) => ({
-                    model: gpu.model,
-                    vendor: gpu.vendor,
-                    bus: '',
-                    vram: gpu.vram,
-                    vramDynamic: false,
-                  }))
-                  gpuHealth.status = 'ok'
-                  gpuHealth.ollamaGpuAccessible = true
-                } else {
-                  gpuHealth.status = 'ollama_not_installed'
-                }
+              const externalOllamaGpu = await this.getExternalOllamaGpuInfo()
+              if (externalOllamaGpu) {
+                useProbeControllers(toControllers(externalOllamaGpu))
+                gpuHealth.status = 'ok'
+                gpuHealth.ollamaGpuAccessible = true
               } else {
-                gpuHealth.status = 'passthrough_failed'
-                logger.warn(
-                  'AMD GPU detected but Ollama logs show no ROCm initialization — passthrough or HSA override may have failed'
-                )
+                gpuHealth.status = 'ollama_not_installed'
               }
+            } else {
+              gpuHealth.status = 'passthrough_failed'
+              logger.warn(
+                'AMD GPU detected but Ollama logs show no ROCm initialization — passthrough or HSA override may have failed'
+              )
+            }
+
+            // Tell the UI whether a reinstall would help or an HSA override is needed.
+            // Reinstalling an AMD container whose config already matches rebuilds the
+            // same CPU-bound container, so the banner must not offer that as the fix.
+            if (gpuHealth.status === 'passthrough_failed' && expectedVendor === 'amd' && ollama) {
+              Object.assign(
+                gpuHealth,
+                diagnoseAmdCpuFallback(
+                  ollama.inspect,
+                  ollamaLogs,
+                  // Pass the container env so the diff matches what a reinstall would
+                  // really build, including an override it keeps from the container (#1377).
+                  await this.dockerService.getAmdHsaOverride({
+                    quiet: true,
+                    containerEnv: ollama.inspect.Config?.Env,
+                  })
+                )
+              )
             }
           }
         } else {
-          // si.graphics() returned controllers (host install, not Docker) — GPU is working
+          // si.graphics() returned usable controllers and no GPU runtime is expected
+          // (host install, not Docker). Still unprobed, see #1325.
           gpuHealth.status = 'ok'
           gpuHealth.ollamaGpuAccessible = true
         }
@@ -920,6 +932,12 @@ export class SystemService {
     if (key === 'ai.assistantCustomName') {
       invalidateAssistantNameCache()
     }
+    if (key === 'rag.minRelevance') {
+      invalidateMinRelevanceCache()
+    }
+    if (key === 'ai.responseStyle') {
+      invalidateResponseStyleCache()
+    }
     // Re-enabling auto-update after a backoff-driven auto-disable clears the
     // failure state so it gets a fresh start instead of immediately re-tripping.
     if (key === 'autoUpdate.enabled' && (value === true || value === 'true')) {
@@ -960,6 +978,12 @@ export class SystemService {
       const serviceStatusList = await this.dockerService.getServicesStatus()
 
       for (const service of allServices) {
+        // Link tiles are shortcuts with no container behind them, so container
+        // reconciliation does not apply: they are always "installed" in the only
+        // sense that matters, which is that the dashboard should show them.
+        // Without this they are marked not-installed on the next sync and vanish.
+        if (service.is_link_tile) continue
+
         const containerExists = serviceStatusList.find(
           (s) => s.service_name === service.service_name
         )

admin/app/services/token_calibration_service.ts

@@ -0,0 +1,123 @@
+import { inject } from '@adonisjs/core'
+import logger from '@adonisjs/core/services/logger'
+import KVStore from '#models/kv_store'
+import { clampRatio, updateEwma } from '../utils/token_estimate.js'
+
+/**
+ * Learns, per model, how far the token estimator is off — and corrects it.
+ *
+ * The estimator in `token_estimate.ts` cannot tokenize exactly (no vocabulary,
+ * no tokenize endpoint, offline appliance). But every chat response already
+ * carries the exact answer: Ollama reports `prompt_eval_count`, the true token
+ * count of the prompt we just sent. That is free ground truth on every single
+ * turn, and before this it was thrown away.
+ *
+ * So: record what we estimated, compare it to what the backend actually counted,
+ * and fold `actual / estimated` into a per-model EWMA. The budget planner
+ * multiplies by that factor from then on. Measured convergence is a couple of
+ * turns, and it matters — Qwen tokenizes ~26% finer than Llama on the same text,
+ * which no single global constant can absorb.
+ *
+ * Persisted as one JSON blob (`ai.tokenRatios`) rather than a key per model,
+ * because KVStoreKey is a closed union. The data is pure cache: deleting the row
+ * costs nothing but a short re-learn.
+ */
+@inject()
+export class TokenCalibrationService {
+  /** Model name -> learned correction factor. Authoritative once loaded. */
+  private ratios: Map<string, number> = new Map()
+  private loaded: Promise<void> | null = null
+  /** Last persisted value per model, so we only write when the ratio really moved. */
+  private persisted: Map<string, number> = new Map()
+
+  /**
+   * A ratio has to move by more than this before it earns a database write.
+   * Once converged the factor barely twitches, and a row update per chat turn
+   * would be pure noise on a machine that may be running from an SD card.
+   */
+  private static PERSIST_THRESHOLD = 0.02
+
+  private async _load(): Promise<void> {
+    if (!this.loaded) {
+      this.loaded = (async () => {
+        try {
+          const raw = await KVStore.getValue('ai.tokenRatios')
+          if (!raw) return
+          const parsed = JSON.parse(raw)
+          if (parsed && typeof parsed === 'object') {
+            for (const [model, value] of Object.entries(parsed)) {
+              if (typeof value === 'number' && Number.isFinite(value)) {
+                const clamped = clampRatio(value)
+                this.ratios.set(model, clamped)
+                this.persisted.set(model, clamped)
+              }
+            }
+          }
+        } catch (error) {
+          // Corrupt or hand-edited JSON. This is a cache; start over rather than
+          // failing a chat request over it.
+          logger.warn(
+            `[TokenCalibration] Could not read stored ratios, starting fresh: ${error instanceof Error ? error.message : error}`
+          )
+        }
+      })()
+    }
+    return this.loaded
+  }
+
+  /**
+   * The correction factor to apply when estimating for `model`.
+   * Returns 1 for a model we've never seen — an uncalibrated estimate, which is
+   * still materially better than a fixed chars-per-token divisor.
+   */
+  async ratioFor(model: string): Promise<number> {
+    await this._load()
+    return this.ratios.get(model) ?? 1
+  }
+
+  /**
+   * Record one observation.
+   *
+   * `estimatedTokens` must be the *uncalibrated* estimate for exactly the
+   * messages that were sent — otherwise the factor gets applied twice and the
+   * EWMA chases its own tail.
+   */
+  async record(model: string, estimatedTokens: number, actualTokens: number | undefined): Promise<void> {
+    if (!actualTokens || !estimatedTokens || estimatedTokens <= 0 || actualTokens <= 0) return
+    // Tiny prompts are dominated by chat-template overhead and teach us little
+    // about how this model tokenizes real content.
+    if (actualTokens < 32) return
+
+    await this._load()
+
+    const observed = clampRatio(actualTokens / estimatedTokens)
+    const previous = this.ratios.get(model) ?? null
+    const next = clampRatio(updateEwma(previous, observed))
+    this.ratios.set(model, next)
+
+    logger.debug(
+      `[TokenCalibration] ${model}: estimated ${estimatedTokens}, actual ${actualTokens} ` +
+        `(observed ${observed.toFixed(3)}, ratio ${next.toFixed(3)})`
+    )
+
+    const lastWritten = this.persisted.get(model)
+    if (lastWritten === undefined || Math.abs(next - lastWritten) > TokenCalibrationService.PERSIST_THRESHOLD) {
+      await this._persist(model, next)
+    }
+  }
+
+  private async _persist(model: string, ratio: number): Promise<void> {
+    try {
+      const blob: Record<string, number> = {}
+      for (const [name, value] of this.ratios) blob[name] = Number(value.toFixed(4))
+      await KVStore.setValue('ai.tokenRatios', JSON.stringify(blob))
+      this.persisted.set(model, ratio)
+    } catch (error) {
+      // Calibration is an optimization. Losing a write means a slightly worse
+      // estimate next boot, never a failed chat.
+      logger.warn(
+        `[TokenCalibration] Failed to persist ratios: ${error instanceof Error ? error.message : error}`
+      )
+    }
+  }
+}

admin/app/services/zim_extraction_service.ts

@@ -44,7 +44,7 @@ export class ZIMExtractionService {
     async extractZIMContent(
         filePath: string,
         opts: ExtractZIMContentOptions = {}
-    ): Promise<{ chunks: ZIMContentChunk[]; totalArticles: number }> {
+    ): Promise<{ chunks: ZIMContentChunk[]; totalArticles: number; articlesProcessed: number }> {
         try {
             logger.info(`[ZIMExtractionService]: Processing ZIM file at path: ${filePath}`)
             
@@ -165,7 +165,7 @@ export class ZIMExtractionService {
                 textPreview: c.text.substring(0, 100)
             })))
             logger.debug("Total structured sections extracted:", toReturn.length)
-            return { chunks: toReturn, totalArticles: archive.articleCount }
+            return { chunks: toReturn, totalArticles: archive.articleCount, articlesProcessed }
         } catch (error) {
             logger.error('Error processing ZIM file:', error)
             throw error

admin/app/services/zim_service.ts

@@ -1,4 +1,5 @@
 import {
+  CatalogLanguage,
   ListRemoteZimFilesResponse,
   RawRemoteZimFileEntry,
   RemoteZimFileEntry,
@@ -33,6 +34,8 @@ import { SERVICE_NAMES } from '../../constants/service_names.js'
 import { CollectionManifestService } from './collection_manifest_service.js'
 import { KiwixCatalogService } from './kiwix_catalog_service.js'
 import { KiwixLibraryService } from './kiwix_library_service.js'
+import { RagService } from './rag_service.js'
+import { OllamaService } from './ollama_service.js'
 import type { CategoryWithStatus } from '../../types/collections.js'
 import CustomLibrarySource from '#models/custom_library_source'
 import { assertNotPrivateUrl } from '#validators/common'
@@ -77,10 +80,18 @@ export class ZimService {
     start,
     count,
     query,
+    language = 'eng',
   }: {
     start: number
     count: number
     query?: string
+    /**
+     * ISO-639-3 code to filter the catalog by, or `all` for no filter. Defaults to
+     * English, which is what this browser has always shown -- but it is now a default
+     * the user can change, not a hardcode. English is ~1,300 of the catalog's ~10,900
+     * books, so the filter was hiding the large majority of the library.
+     */
+    language?: string
   }): Promise<ListRemoteZimFilesResponse> {
     // Kiwix returns pages of content unaware of what the user has installed locally. When
     // the installed set is large, a single 12-item Kiwix page can come back with everything
@@ -111,7 +122,8 @@ export class ZimService {
         params: {
           start: currentStart,
           count: KIWIX_PAGE_SIZE,
-          lang: 'eng',
+          // `all` means "no filter", which the catalog expresses by omitting `lang`.
+          ...(language && language !== 'all' ? { lang: language } : {}),
           ...(query ? { q: query } : {}),
         },
         responseType: 'text',
@@ -191,6 +203,62 @@ export class ZimService {
     }
   }
 
+  /**
+   * The languages the Kiwix catalog actually holds books in, newest count first.
+   *
+   * Sourced live from the catalog rather than a bundled ISO list, so the options are
+   * always exactly what can be browsed, and each carries a real book count. Titles are
+   * the catalog's own endonyms ("français", "中文"), which is what a reader scanning for
+   * their own language recognises.
+   *
+   * Requires internet, like the rest of this browser. Failures return an empty list
+   * rather than throwing: a missing language filter should degrade to "English only",
+   * never take down the page that lists the books.
+   */
+  async listCatalogLanguages(): Promise<CatalogLanguage[]> {
+    const LANGUAGES_URL = 'https://opds.library.kiwix.org/catalog/v2/languages'
+    try {
+      const res = await axios.get(LANGUAGES_URL, { responseType: 'text', timeout: 15000 })
+      const parser = new XMLParser({
+        ignoreAttributes: false,
+        attributeNamePrefix: '',
+        textNodeName: '#text',
+      })
+      const parsed = parser.parse(res.data)
+      const rawEntries = parsed?.feed?.entry
+        ? Array.isArray(parsed.feed.entry)
+          ? parsed.feed.entry
+          : [parsed.feed.entry]
+        : []
+
+      const languages: CatalogLanguage[] = []
+      for (const raw of rawEntries) {
+        if (!raw || typeof raw !== 'object') continue
+        // `dc:language` is the ISO-639-3 code; `thr:count` is how many books carry it.
+        const code = raw['dc:language']
+        const label = raw.title
+        const bookCount = Number(raw['thr:count'])
+        if (typeof code !== 'string' || !code.trim()) continue
+        if (!Number.isFinite(bookCount) || bookCount <= 0) continue
+        languages.push({
+          code: code.trim(),
+          label: typeof label === 'string' && label.trim() ? label.trim() : code.trim(),
+          book_count: bookCount,
+        })
+      }
+
+      languages.sort((a, b) => b.book_count - a.book_count)
+      return languages
+    } catch (error) {
+      logger.warn(
+        `[ZimService] Catalog language list unavailable: ${
+          error instanceof Error ? error.message : error
+        }`
+      )
+      return []
+    }
+  }
+
   async downloadRemote(url: string, metadata?: { title?: string; summary?: string; author?: string; size_bytes?: number }): Promise<{ filename: string; jobId?: string }> {
     const parsed = new URL(url)
     if (!parsed.pathname.endsWith('.zim')) {
@@ -293,6 +361,26 @@ export class ZimService {
         const status = await drugReferenceService.getIngestStatus()
         if (status.phase === 'ready' || status.rowCount > 0) {
           logger.info('[ZimService] Drug dataset already ingested, skipping dispatch.')
+          // Reaching here means no 'dataset' row exists (the installed filter
+          // above would have dropped the resource). A finished ingest that lost
+          // its row write (1.34.0's narrow enum, or a manual ingest) gets one now,
+          // so re-selecting the tier resolves it without waiting for a reboot.
+          if (status.phase === 'ready' && status.lastUpdated) {
+            try {
+              await drugReferenceService.recordInstalledRow({
+                version: status.lastUpdated,
+                collectionRef: categorySlug,
+                fileSizeBytes: null,
+              })
+              logger.info('[ZimService] Backfilled installed_resources row for the drug dataset.')
+            } catch (err) {
+              logger.error(
+                `[ZimService] Failed to backfill drug dataset row: ${
+                  err instanceof Error ? err.message : String(err)
+                }`
+              )
+            }
+          }
           continue
         }
         // DownloadDrugDataJob.dispatch() is idempotent on its deterministic
@@ -659,6 +747,25 @@ export class ZimService {
 
     await deleteFileIfExists(fullPath)
 
+    // Purge this file's Qdrant points and KbIngestState row directly so a
+    // deleted ZIM stops surfacing as a stale search citation immediately,
+    // rather than waiting for the next scanAndSyncStorage reverse sweep to
+    // catch it (#1170). Never touched Qdrant here before — the file's points
+    // and state row would otherwise linger indefinitely.
+    //
+    // Guarded on the AI Assistant being installed at all — otherwise every
+    // delete on an install without it hits _initializeQdrantClient()'s
+    // "offline" throw and logs a misleading error for the common no-AI case.
+    const qdrantInstalled = !!(await this.dockerService.getServiceURL(SERVICE_NAMES.QDRANT))
+    if (qdrantInstalled) {
+      try {
+        const ragService = new RagService(this.dockerService, new OllamaService())
+        await ragService.purgeIndexedSource(fullPath)
+      } catch (err) {
+        logger.error(`[ZimService] Failed to purge knowledge-base entries for ${fullPath}:`, err)
+      }
+    }
+
     // Remove from kiwix library XML so --monitorLibrary stops serving the deleted file
     const kiwixLibraryService = new KiwixLibraryService()
     await kiwixLibraryService.removeBook(fileName).catch((err) => {

admin/app/utils/amd_hsa_override.ts

@@ -50,3 +50,57 @@ export function mapGfxToHsaOverride(gfx: string): string | null {
   // Unknown/newer target: prefer native discovery over a coercion that's likely wrong.
   return null
 }
+
+const HSA_OVERRIDE_ENV_PREFIX = 'HSA_OVERRIDE_GFX_VERSION='
+const HSA_OVERRIDE_VALUE_RE = /^\d+\.\d+\.\d+$/
+
+/** The raw HSA_OVERRIDE_GFX_VERSION in a container env, or null when it is not set. */
+export function readHsaOverrideFromEnv(env: string[] | null | undefined): string | null {
+  const entry = (env ?? []).find((e) => e.startsWith(HSA_OVERRIDE_ENV_PREFIX))
+  return entry ? entry.slice(HSA_OVERRIDE_ENV_PREFIX.length) : null
+}
+
+export type AmdHsaOverrideResolution = {
+  value: string | null
+  source: 'kv' | 'kv-disabled' | 'marker' | 'container' | 'default'
+  /** Set when the KV held something unusable and was skipped. */
+  invalidManual?: string
+}
+
+/**
+ * Pick the HSA_OVERRIDE_GFX_VERSION an AMD install should apply. Resolution order:
+ *   1. KV `ai.amdHsaOverride`: a version forces it, 'none'/'off'/'false' disables it.
+ *   2. The gfx marker install_nomad.sh writes, mapped through mapGfxToHsaOverride. A
+ *      marker is authoritative even when it maps to none.
+ *   3. The override the existing container already runs with (#1377). Upgraded installs
+ *      that got it by hand or from an older install path have neither 1 nor 2, and
+ *      dropping a working override would push the GPU back to CPU.
+ *   4. None, letting ROCm discover the GPU natively.
+ */
+export function pickAmdHsaOverride(input: {
+  manual: unknown
+  markerGfx: string | null
+  containerEnv?: string[] | null
+}): AmdHsaOverrideResolution {
+  let invalidManual: string | undefined
+  if (input.manual !== null && input.manual !== undefined && String(input.manual).trim() !== '') {
+    const manual = String(input.manual).trim().toLowerCase()
+    if (manual === 'none' || manual === 'off' || manual === 'false') {
+      return { value: null, source: 'kv-disabled' }
+    }
+    if (HSA_OVERRIDE_VALUE_RE.test(manual)) return { value: manual, source: 'kv' }
+    invalidManual = String(input.manual)
+  }
+  const extra = invalidManual !== undefined ? { invalidManual } : {}
+
+  if (input.markerGfx !== null) {
+    return { value: mapGfxToHsaOverride(input.markerGfx), source: 'marker', ...extra }
+  }
+
+  const existing = readHsaOverrideFromEnv(input.containerEnv)?.trim()
+  if (existing && HSA_OVERRIDE_VALUE_RE.test(existing)) {
+    return { value: existing, source: 'container', ...extra }
+  }
+
+  return { value: null, source: 'default', ...extra }
+}

admin/app/utils/chat_images.ts

@@ -0,0 +1,125 @@
+import type { MultipartFile } from '@adonisjs/bodyparser/types'
+import type { ChatCompletionMessageParam } from 'openai/resources/chat/completions.js'
+import sharp from 'sharp'
+
+export const MAX_CHAT_IMAGES = 4
+export const MAX_CHAT_IMAGE_BYTES = 8 * 1024 * 1024
+export const MAX_CHAT_IMAGE_PIXELS = 40_000_000
+export const MAX_NORMALIZED_IMAGE_BYTES = 4 * 1024 * 1024
+export const MAX_NORMALIZED_IMAGE_DIMENSION = 2048
+
+const SUPPORTED_IMAGE_FORMATS = new Set(['jpeg', 'png', 'webp'])
+
+export class ChatImageError extends Error {
+  constructor(
+    message: string,
+    public readonly status: 413 | 415 | 422
+  ) {
+    super(message)
+    this.name = 'ChatImageError'
+  }
+}
+
+export type NormalizedChatImage = {
+  name: string
+  dataUrl: string
+}
+
+export async function normalizeChatImages(files: MultipartFile[]): Promise<NormalizedChatImage[]> {
+  if (files.length > MAX_CHAT_IMAGES) {
+    throw new ChatImageError(`Attach no more than ${MAX_CHAT_IMAGES} images.`, 422)
+  }
+
+  const normalized: NormalizedChatImage[] = []
+  for (const file of files) {
+    normalized.push(await normalizeChatImage(file))
+  }
+  return normalized
+}
+
+async function normalizeChatImage(file: MultipartFile): Promise<NormalizedChatImage> {
+  if (!file.tmpPath) {
+    throw new ChatImageError(`Could not process "${file.clientName}".`, 422)
+  }
+  if (file.size > MAX_CHAT_IMAGE_BYTES) {
+    throw new ChatImageError(`"${file.clientName}" exceeds the 8 MB per-image limit.`, 413)
+  }
+  if (!file.isValid) {
+    throw new ChatImageError(`"${file.clientName}" is not a valid image upload.`, 422)
+  }
+
+  try {
+    const pipeline = sharp(file.tmpPath, {
+      animated: false,
+      failOn: 'warning',
+      limitInputPixels: MAX_CHAT_IMAGE_PIXELS,
+    })
+    const metadata = await pipeline.metadata()
+
+    if (!metadata.format || !SUPPORTED_IMAGE_FORMATS.has(metadata.format)) {
+      throw new ChatImageError(
+        `"${file.clientName}" is not supported. Use JPEG, PNG, or WebP.`,
+        415
+      )
+    }
+    if ((metadata.pages ?? 1) > 1) {
+      throw new ChatImageError(`Animated images are not supported.`, 415)
+    }
+
+    const normalized = await pipeline
+      .rotate()
+      .resize({
+        width: MAX_NORMALIZED_IMAGE_DIMENSION,
+        height: MAX_NORMALIZED_IMAGE_DIMENSION,
+        fit: 'inside',
+        withoutEnlargement: true,
+      })
+      .flatten({ background: '#ffffff' })
+      .jpeg({ quality: 85 })
+      .toBuffer()
+
+    if (normalized.byteLength > MAX_NORMALIZED_IMAGE_BYTES) {
+      throw new ChatImageError(
+        `"${file.clientName}" remains too large after image processing.`,
+        413
+      )
+    }
+
+    return {
+      name: file.clientName,
+      dataUrl: `data:image/jpeg;base64,${normalized.toString('base64')}`,
+    }
+  } catch (error) {
+    if (error instanceof ChatImageError) throw error
+    throw new ChatImageError(`"${file.clientName}" could not be decoded as an image.`, 422)
+  }
+}
+
+export function attachImagesToLatestUserMessage(
+  messages: Array<{ role: 'system' | 'user' | 'assistant'; content: string }>,
+  images: NormalizedChatImage[]
+): ChatCompletionMessageParam[] {
+  if (images.length === 0) return messages
+
+  let latestUserIndex = -1
+  messages.forEach((message, index) => {
+    if (message.role === 'user') latestUserIndex = index
+  })
+  if (latestUserIndex < 0) {
+    throw new ChatImageError('Images require a user message.', 422)
+  }
+
+  return messages.map((message, index) => {
+    if (index !== latestUserIndex) return message
+    return {
+      role: 'user',
+      content: [
+        { type: 'text', text: message.content },
+        ...images.map((image) => ({
+          type: 'image_url' as const,
+          image_url: { url: image.dataUrl },
+        })),
+      ],
+    }
+  })
+}

admin/app/utils/context_budget.ts

@@ -0,0 +1,321 @@
+/**
+ * Decides what actually goes into the prompt, and what gets left out.
+ *
+ * Before this existed, nothing trimmed anything: the browser posted the whole
+ * session, the pipeline copied it verbatim, and overflow was left to the
+ * backend. That is the worst possible arrangement, because backend truncation is
+ * *silent* and *positional* — llama.cpp drops from the middle of the window, so
+ * what gets discarded is conversation history and retrieved knowledge-base
+ * context, while the model gives no indication anything is missing. Measured on
+ * a live Ollama, a 7,000-token prompt came back having processed 2,060 tokens,
+ * and the model confabulated an answer from the fragment that survived.
+ *
+ * So the rule here is: never hand the backend more than it can hold. Decide
+ * explicitly what to drop, drop it in a way the model can reason about, and
+ * record what happened so the eval harness can see it.
+ *
+ * Allocation order, most protected first:
+ *   1. Response reserve  — a floor held back for the answer itself. The actual
+ *                          generation cap is whatever the window has left once
+ *                          the prompt is assembled, which is usually far more.
+ *   2. Fixed blocks      — system prompts and the current question. Never dropped.
+ *   3. Retrieved context — whole chunks, best-first, until its share is spent.
+ *   4. History           — whole turns, newest-first, into whatever remains.
+ *
+ * Pure and dependency-free so it runs under bare `node --test`, the same
+ * convention as `rag_prompt.ts`.
+ */
+
+import { estimateMessagesTokens, estimateTokens } from './token_estimate.js'
+
+export type BudgetRole = 'system' | 'user' | 'assistant'
+export type BudgetMessage = { role: BudgetRole; content: string }
+
+/** A retrieved chunk, already ranked — index 0 is the best match. */
+export type BudgetChunk = { text: string; metadata?: Record<string, any> }
+
+/**
+ * Where the per-turn retrieved-context block goes.
+ *
+ * `tail` places it immediately before the current question, leaving
+ * [system][history] as a byte-identical prefix across turns so the backend can
+ * reuse its KV cache. `front` is the historical behaviour: a system message
+ * ahead of all history, which changes content every turn and therefore
+ * invalidates the cached prefix for the entire conversation behind it.
+ *
+ * The common advice to "inject RAG into the system prompt" assumes the context
+ * is fixed for the session (an attached document), where a fixed position does
+ * preserve the prefix. When context is re-retrieved every turn, the opposite is
+ * true. Both are kept so `eval:generation` can measure the difference rather
+ * than the choice resting on argument alone.
+ */
+export type RagPlacement = 'tail' | 'front'
+
+export type BudgetInputs = {
+  /** Stable system blocks (NOMAD.md, formatting rules), in final order. */
+  systemBlocks: BudgetMessage[]
+  /** Prior conversation, oldest first, excluding the current question. */
+  history: BudgetMessage[]
+  /** The current user message. Never dropped. */
+  query: BudgetMessage
+  /** Retrieved chunks, best first. May be empty. */
+  ragChunks: BudgetChunk[]
+  /** Renders surviving chunks into the final context message. */
+  renderRagBlock: (chunks: BudgetChunk[]) => string
+  /** The resolved context window, in tokens. */
+  contextWindow: number
+  /** Learned per-model token-estimator correction. */
+  ratio?: number
+  ragPlacement?: RagPlacement
+  /** Overrides for the response reserve and RAG share; mostly for tests. */
+  responseReserve?: number
+  ragShare?: number
+}
+
+export type BudgetTrace = {
+  contextWindow: number
+  responseReserve: number
+  promptBudget: number
+  estimatedPromptTokens: number
+  systemTokens: number
+  queryTokens: number
+  ragTokens: number
+  historyTokens: number
+  chunksKept: number
+  chunksDropped: number
+  turnsKept: number
+  turnsDropped: number
+  historyElided: boolean
+  queryTruncated: boolean
+  ragPlacement: RagPlacement
+  numPredict: number
+}
+
+export type BudgetResult = {
+  messages: BudgetMessage[]
+  trace: BudgetTrace
+  /**
+   * What to send as num_predict: the space the window actually has left after
+   * the prompt, less a margin for estimator error. Never below the reserve.
+   */
+  numPredict: number
+}
+
+/**
+ * Ceiling on tokens reserved for the answer.
+ *
+ * This only bounds how much prompt space is held back; it is not the
+ * generation cap. It used to double as one, which pinned every answer at 1024
+ * tokens regardless of window size and cut long answers off mid-sentence
+ * (#1342).
+ */
+export const MAX_RESPONSE_RESERVE = 8192
+/** Share of the window reserved for the answer when that is smaller. */
+export const RESPONSE_RESERVE_FRACTION = 0.25
+/**
+ * Headroom left between the estimated prompt and the end of the window when
+ * sizing num_predict. The estimator is calibrated per model but still only an
+ * estimate; undershooting here means llama.cpp context-shifts mid-answer.
+ */
+export const NUM_PREDICT_SAFETY_FRACTION = 0.05
+export const NUM_PREDICT_SAFETY_MIN = 64
+/** Share of the remaining budget the retrieved context may claim. */
+export const DEFAULT_RAG_SHARE = 0.35
+
+/**
+ * Marker inserted where turns were dropped.
+ *
+ * Worth the handful of tokens: without it the model sees a conversation that
+ * appears to begin mid-thought and has no way to tell that it is missing
+ * context, which is exactly when small models start confidently inventing what
+ * "we discussed earlier".
+ */
+export const ELISION_MARKER =
+  '[Earlier messages in this conversation have been omitted to fit the context window.]'
+
+/**
+ * Turns are dropped in blocks rather than one at a time.
+ *
+ * Dropping a single turn per request would shift the prompt prefix on *every*
+ * subsequent turn, invalidating the backend's KV cache each time — trading a
+ * small saving in tokens for a full re-prefill. Dropping several at once means
+ * the prefix then stays stable for several turns.
+ */
+export const HISTORY_EVICTION_BLOCK = 4
+
+/**
+ * Group a flat message list into whole turns.
+ *
+ * A turn is a user message plus everything that answers it. Truncating on a
+ * message boundary instead can leave the transcript starting with an assistant
+ * reply to a question the model can no longer see — which reads, to the model,
+ * as though it said something unprompted.
+ */
+export function groupIntoTurns(history: BudgetMessage[]): BudgetMessage[][] {
+  const turns: BudgetMessage[][] = []
+  for (const message of history) {
+    if (message.role === 'user' || turns.length === 0) {
+      turns.push([message])
+    } else {
+      turns[turns.length - 1].push(message)
+    }
+  }
+  return turns
+}
+
+/**
+ * Plan the prompt.
+ *
+ * Never returns a message array estimated to exceed `contextWindow -
+ * responseReserve`, except in the degenerate case where the fixed blocks alone
+ * are too large — and in that case the query is truncated visibly rather than
+ * being handed to the backend to cut silently.
+ */
+export function planPrompt(inputs: BudgetInputs): BudgetResult {
+  const ratio = inputs.ratio ?? 1
+  const placement = inputs.ragPlacement ?? 'tail'
+  const contextWindow = inputs.contextWindow
+
+  const responseReserve =
+    inputs.responseReserve ??
+    Math.max(256, Math.min(MAX_RESPONSE_RESERVE, Math.floor(contextWindow * RESPONSE_RESERVE_FRACTION)))
+  const promptBudget = Math.max(0, contextWindow - responseReserve)
+
+  const cost = (messages: BudgetMessage[]) => estimateMessagesTokens(messages, ratio)
+
+  // --- Fixed blocks: system prompts and the question --------------------
+  const systemTokens = cost(inputs.systemBlocks)
+  let query = inputs.query
+  let queryTokens = cost([query])
+  let queryTruncated = false
+
+  // Degenerate case: the system prompts plus the question alone overflow the
+  // window. Truncating here is a bad outcome, but it is a *visible* one — the
+  // alternative is the backend silently cutting the question in half.
+  if (systemTokens + queryTokens > promptBudget) {
+    const available = Math.max(0, promptBudget - systemTokens)
+    if (available > 0) {
+      const perToken = query.content.length / Math.max(1, estimateTokens(query.content, ratio))
+      const keepChars = Math.max(0, Math.floor(available * perToken) - ELISION_MARKER.length)
+      query = {
+        ...query,
+        content:
+          query.content.slice(0, keepChars) +
+          '\n\n[This message was truncated because it exceeds the context window.]',
+      }
+      queryTokens = cost([query])
+      queryTruncated = true
+    }
+  }
+
+  let remaining = promptBudget - systemTokens - queryTokens
+
+  // --- Retrieved context: whole chunks, best first ----------------------
+  //
+  // Whole chunks, deliberately. The previous behaviour capped on a running
+  // character count, which could hand the model a chunk cut off mid-sentence —
+  // the worst of both worlds, since it costs tokens without carrying a complete
+  // fact. A chunk that doesn't fit is dropped, not shortened.
+  const ragBudget = Math.max(0, Math.floor(Math.max(0, remaining) * (inputs.ragShare ?? DEFAULT_RAG_SHARE)))
+  const keptChunks: BudgetChunk[] = []
+  let ragTokens = 0
+
+  if (inputs.ragChunks.length > 0 && ragBudget > 0) {
+    for (const chunk of inputs.ragChunks) {
+      const candidate = [...keptChunks, chunk]
+      const rendered = inputs.renderRagBlock(candidate)
+      const candidateTokens = cost([{ role: 'system', content: rendered }])
+      if (candidateTokens <= ragBudget) {
+        keptChunks.push(chunk)
+        ragTokens = candidateTokens
+      }
+      // Keep scanning: a later, smaller chunk may still fit where this one didn't.
+    }
+  }
+
+  const ragMessage: BudgetMessage | null =
+    keptChunks.length > 0
+      ? { role: 'system', content: inputs.renderRagBlock(keptChunks) }
+      : null
+
+  remaining -= ragTokens
+
+  // --- History: whole turns, newest first -------------------------------
+  const turns = groupIntoTurns(inputs.history)
+  const keptTurns: BudgetMessage[][] = []
+  let historyTokens = 0
+
+  for (let i = turns.length - 1; i >= 0; i--) {
+    const turnTokens = cost(turns[i])
+    if (historyTokens + turnTokens > remaining) break
+    keptTurns.unshift(turns[i])
+    historyTokens += turnTokens
+  }
+
+  const turnsDropped = turns.length - keptTurns.length
+
+  // Chunky eviction: once we're dropping anything, drop a whole block, so the
+  // surviving prefix stays put for the next several turns instead of shifting
+  // on every request and forcing a re-prefill each time.
+  if (turnsDropped > 0 && keptTurns.length > 0) {
+    const overshoot = turnsDropped % HISTORY_EVICTION_BLOCK
+    const extra = overshoot === 0 ? 0 : HISTORY_EVICTION_BLOCK - overshoot
+    for (let i = 0; i < extra && keptTurns.length > 1; i++) {
+      const removed = keptTurns.shift()!
+      historyTokens -= cost(removed)
+    }
+  }
+
+  const historyElided = keptTurns.length < turns.length
+  const historyMessages = keptTurns.flat()
+
+  // --- Assemble, stability-descending -----------------------------------
+  const messages: BudgetMessage[] = []
+  messages.push(...inputs.systemBlocks)
+  if (historyElided) {
+    messages.push({ role: 'system', content: ELISION_MARKER })
+  }
+
+  if (placement === 'front' && ragMessage) {
+    // Historical ordering, kept only so the two can be compared under eval.
+    messages.splice(inputs.systemBlocks.length, 0, ragMessage)
+    messages.push(...historyMessages)
+  } else {
+    messages.push(...historyMessages)
+    if (ragMessage) messages.push(ragMessage)
+  }
+  messages.push(query)
+
+  const estimatedPromptTokens = cost(messages)
+
+  // The planner already kept the prompt within `contextWindow - responseReserve`,
+  // so the reserve floor can never push generation past the window.
+  const safetyMargin = Math.max(
+    NUM_PREDICT_SAFETY_MIN,
+    Math.ceil(estimatedPromptTokens * NUM_PREDICT_SAFETY_FRACTION)
+  )
+  const numPredict = Math.max(responseReserve, contextWindow - estimatedPromptTokens - safetyMargin)
+
+  return {
+    messages,
+    numPredict,
+    trace: {
+      contextWindow,
+      responseReserve,
+      promptBudget,
+      estimatedPromptTokens,
+      systemTokens,
+      queryTokens,
+      ragTokens,
+      historyTokens,
+      chunksKept: keptChunks.length,
+      chunksDropped: inputs.ragChunks.length - keptChunks.length,
+      turnsKept: keptTurns.length,
+      turnsDropped: turns.length - keptTurns.length,
+      historyElided,
+      queryTruncated,
+      ragPlacement: placement,
+      numPredict,
+    },
+  }
+}

admin/app/utils/context_window.ts

@@ -0,0 +1,282 @@
+/**
+ * Pure helpers for deciding how large a model's context window should be.
+ *
+ * Kept out of OllamaService so they can run under bare `node --test` with no
+ * Ollama, Docker, or MySQL — the same convention as `rag_prompt.ts`. The service
+ * supplies the facts (what /api/show said, how much memory is free); everything
+ * here is a function of its arguments.
+ */
+
+/**
+ * Ladder of context sizes we're willing to request.
+ *
+ * Snapping to a ladder is not cosmetic. Ollama unloads and reloads a model
+ * whenever a request asks for a different `num_ctx` than the loaded instance, so
+ * a continuously-varying window would stall a turn and throw away the KV cache
+ * every time the conversation grew. Snapping (plus per-model memoization in the
+ * resolver) keeps the value constant for a session.
+ */
+export const CONTEXT_LADDER = [4096, 8192, 16384, 32768, 65536, 131072] as const
+
+/** Never go below this: it is roughly a system prompt plus a few real turns. */
+export const MIN_CONTEXT = 4096
+
+/**
+ * Fallback window when the backend can't tell us anything — a non-Ollama
+ * OpenAI-compatible server, or /api/show unavailable. Deliberately modest:
+ * budgeting too small wastes capacity, budgeting too large silently truncates.
+ */
+export const UNKNOWN_BACKEND_CONTEXT = 8192
+
+/**
+ * Read the trained context length out of an /api/show `model_info` blob.
+ *
+ * Keys are architecture-prefixed (`llama.context_length`, `qwen3.context_length`,
+ * `gemma3.context_length`, ...), so prefer the prefix named by
+ * `general.architecture` and fall back to any `*.context_length` key. Ollama's
+ * client types this as a Map but the wire format is a plain object; handle both.
+ */
+export function readContextLength(modelInfo: unknown): number | undefined {
+  if (!modelInfo) return undefined
+  const entries: [string, any][] =
+    modelInfo instanceof Map
+      ? [...modelInfo.entries()]
+      : typeof modelInfo === 'object'
+        ? Object.entries(modelInfo as Record<string, any>)
+        : []
+  if (entries.length === 0) return undefined
+
+  const lookup = new Map(entries)
+  const arch = lookup.get('general.architecture')
+  if (typeof arch === 'string') {
+    const exact = lookup.get(`${arch}.context_length`)
+    if (typeof exact === 'number' && exact > 0) return exact
+  }
+
+  for (const [key, value] of entries) {
+    if (key.endsWith('.context_length') && typeof value === 'number' && value > 0) {
+      return value
+    }
+  }
+  return undefined
+}
+
+/**
+ * Pull `num_ctx` out of the `parameters` string /api/show returns (the modelfile
+ * PARAMETER lines, e.g. `num_ctx                        8192`). When a model
+ * author baked in a window, that is a deliberate signal worth respecting as a
+ * ceiling hint.
+ */
+export function readModelfileNumCtx(parameters: unknown): number | undefined {
+  if (typeof parameters !== 'string') return undefined
+  const match = parameters.match(/^\s*num_ctx\s+(\d+)\s*$/m)
+  if (!match) return undefined
+  const value = Number.parseInt(match[1], 10)
+  return Number.isFinite(value) && value > 0 ? value : undefined
+}
+
+/**
+ * Parse Ollama's `details.parameter_size` ("8.0B", "1.5B", "70B") into billions.
+ * Falls back to the tag-name regex — which is what the codebase used to rely on
+ * exclusively, and which guesses wrong on names like "phi3" that carry no size.
+ */
+export function parseParameterBillions(
+  parameterSize: string | undefined,
+  modelName?: string
+): number | undefined {
+  const fromDetails = parameterSize?.match(/([\d.]+)\s*B/i)
+  if (fromDetails) {
+    const value = Number.parseFloat(fromDetails[1])
+    if (Number.isFinite(value)) return value
+  }
+  const fromName = modelName?.match(/(\d+\.?\d*)\s*[bB](?![a-zA-Z])/)
+  if (fromName) {
+    const value = Number.parseFloat(fromName[1])
+    if (Number.isFinite(value)) return value
+  }
+  return undefined
+}
+
+/** Round a token count down to the largest ladder rung that fits. */
+export function snapToLadder(tokens: number): number {
+  let chosen = MIN_CONTEXT
+  for (const rung of CONTEXT_LADDER) {
+    if (rung <= tokens) chosen = rung
+  }
+  return chosen
+}
+
+// ---------------------------------------------------------------------------
+// How much memory a context window actually costs
+// ---------------------------------------------------------------------------
+
+/**
+ * Bytes per element in the KV cache.
+ *
+ * Budgeted at f16 (2 bytes) even though the Ollama container is started with
+ * `OLLAMA_KV_CACHE_TYPE=q8_0`. Ollama silently falls back to f16 on
+ * architectures that don't support cache quantization, so treating q8_0 as
+ * guaranteed would double the real cost of every window we hand out. Quantized
+ * cache is headroom, not a budgeting assumption.
+ */
+const KV_BYTES_PER_ELEMENT = 2
+
+/**
+ * Exact KV-cache cost per token, from the model's own GGUF metadata.
+ *
+ *     2 (K and V) x layers x kv_heads x head_dim x bytes_per_element
+ *
+ * For llama3:8b that is 2 x 32 x 8 x 128 x 2 = 128 KiB per token, so an 8k
+ * window costs 1 GiB — which is why "just set num_ctx high" is not free advice
+ * on the hardware NOMAD targets.
+ *
+ * Returns undefined when the metadata is missing (non-Ollama backend, or an
+ * architecture that names its keys differently), leaving the caller to fall back
+ * to `estimateKvBytesPerToken`.
+ */
+export function computeKvBytesPerToken(modelInfo: unknown): number | undefined {
+  if (!modelInfo) return undefined
+  const entries: [string, any][] =
+    modelInfo instanceof Map
+      ? [...modelInfo.entries()]
+      : typeof modelInfo === 'object'
+        ? Object.entries(modelInfo as Record<string, any>)
+        : []
+  if (entries.length === 0) return undefined
+  const info = new Map(entries)
+
+  const arch = info.get('general.architecture')
+  if (typeof arch !== 'string') return undefined
+  const num = (suffix: string): number | undefined => {
+    const value = info.get(`${arch}.${suffix}`)
+    return typeof value === 'number' && value > 0 ? value : undefined
+  }
+
+  const layers = num('block_count')
+  // Grouped-query attention means KV heads are usually far fewer than attention
+  // heads; using head_count here would overestimate cost several-fold.
+  const kvHeads = num('attention.head_count_kv') ?? num('attention.head_count')
+  if (!layers || !kvHeads) return undefined
+
+  // Prefer explicit key/value lengths; otherwise derive head_dim from the
+  // embedding width and the attention head count.
+  let headDim = num('attention.key_length')
+  if (!headDim) {
+    const embedding = num('embedding_length')
+    const heads = num('attention.head_count')
+    if (embedding && heads) headDim = embedding / heads
+  }
+  if (!headDim) return undefined
+
+  return 2 * layers * kvHeads * headDim * KV_BYTES_PER_ELEMENT
+}
+
+/**
+ * Fallback KV cost per token when GGUF metadata is unavailable, interpolated
+ * from the common architectures at each size. Deliberately pessimistic — an
+ * overestimate costs a smaller window, an underestimate costs an OOM at load.
+ */
+export function estimateKvBytesPerToken(paramBillions: number | undefined): number {
+  const b = paramBillions ?? 8
+  if (b <= 1.5) return 32 * 1024
+  if (b <= 4) return 64 * 1024
+  if (b <= 9) return 128 * 1024
+  if (b <= 15) return 192 * 1024
+  if (b <= 35) return 256 * 1024
+  return 512 * 1024
+}
+
+/**
+ * Fraction of memory left free for weights-adjacent overhead: compute buffers,
+ * activations, the graph, and whatever else shares the device. Without this the
+ * arithmetic says a window fits right up to the byte and the model fails to load.
+ */
+const MEMORY_SAFETY_FRACTION = 0.8
+
+export type ContextWindowInputs = {
+  /** Trained context length from /api/show. The hard ceiling. */
+  modelMaxCtx?: number
+  /** num_ctx baked into the modelfile, treated as an author's ceiling hint. */
+  modelfileNumCtx?: number
+  /** Cost of one token of KV cache, exact or estimated. */
+  kvBytesPerToken: number
+  /** Memory available for the model, in bytes (VRAM, or a share of system RAM). */
+  availableBytes?: number
+  /** On-disk size of the weights, in bytes — they occupy the same pool. */
+  modelBytes?: number
+  /** User cap from `ai.contextWindow`. Only ever lowers the result. */
+  userCap?: number
+}
+
+export type ContextWindowDecision = {
+  contextWindow: number
+  /** Which constraint actually bound the result — surfaced in logs and the UI. */
+  limitedBy: 'user' | 'model' | 'memory' | 'floor' | 'default'
+  affordableTokens?: number
+}
+
+/**
+ * Decide the context window for a model.
+ *
+ * The result must be *stable for the life of the process*: Ollama unloads and
+ * reloads a model whenever a request asks for a different num_ctx than the
+ * loaded instance, which stalls the turn and discards the KV cache. Callers
+ * memoize per model; this function is pure so that memoization is safe.
+ */
+export function resolveContextWindow(inputs: ContextWindowInputs): ContextWindowDecision {
+  const ceilings: Array<{ value: number; source: ContextWindowDecision['limitedBy'] }> = []
+
+  if (inputs.modelMaxCtx && inputs.modelMaxCtx > 0) {
+    ceilings.push({ value: inputs.modelMaxCtx, source: 'model' })
+  }
+  if (inputs.modelfileNumCtx && inputs.modelfileNumCtx > 0) {
+    ceilings.push({ value: inputs.modelfileNumCtx, source: 'model' })
+  }
+  if (inputs.userCap && inputs.userCap > 0) {
+    ceilings.push({ value: inputs.userCap, source: 'user' })
+  }
+
+  let affordableTokens: number | undefined
+  if (inputs.availableBytes && inputs.availableBytes > 0 && inputs.kvBytesPerToken > 0) {
+    const forKv = inputs.availableBytes * MEMORY_SAFETY_FRACTION - (inputs.modelBytes ?? 0)
+    affordableTokens = Math.max(0, Math.floor(forKv / inputs.kvBytesPerToken))
+    ceilings.push({ value: affordableTokens, source: 'memory' })
+  }
+
+  if (ceilings.length === 0) {
+    return { contextWindow: UNKNOWN_BACKEND_CONTEXT, limitedBy: 'default' }
+  }
+
+  const binding = ceilings.reduce((lowest, c) => (c.value < lowest.value ? c : lowest))
+  const snapped = snapToLadder(binding.value)
+
+  // The floor wins over a *memory* shortfall: a window under 4k is not worth
+  // having, and Ollama will offload to host RAM rather than refuse. It must
+  // never win over the model itself, though — asking for more than the trained
+  // context pushes RoPE past what the weights ever saw and degrades output.
+  // tinyllama (trained 2048) is the case that catches this.
+  const hardCeiling = Math.min(
+    ...[inputs.modelMaxCtx, inputs.modelfileNumCtx, inputs.userCap].filter(
+      (v): v is number => typeof v === 'number' && v > 0
+    ),
+    Number.POSITIVE_INFINITY
+  )
+
+  if (snapped <= MIN_CONTEXT) {
+    const floored = Math.min(MIN_CONTEXT, hardCeiling)
+    return {
+      contextWindow: floored,
+      limitedBy: floored < MIN_CONTEXT ? 'model' : 'floor',
+      affordableTokens,
+    }
+  }
+
+  return { contextWindow: snapped, limitedBy: binding.source, affordableTokens }
+}
+
+/** Parse the `ai.contextWindow` setting. "auto"/unset/garbage all mean "no cap". */
+export function parseUserContextCap(raw: string | null | undefined): number | undefined {
+  if (!raw || raw === 'auto') return undefined
+  const value = Number.parseInt(raw, 10)
+  return Number.isFinite(value) && value >= MIN_CONTEXT ? value : undefined
+}

admin/app/utils/drug_install_row_reconcile.ts

@@ -0,0 +1,66 @@
+import type { ZimCategoriesSpec } from '../../types/collections.js'
+
+/**
+ * Observed state of the FDA drug dataset, gathered at admin boot by
+ * DrugInstallRowProvider.
+ */
+export interface DrugRowReconcileInput {
+  /** Rows in `drug_labels`. */
+  rowCount: number
+  /** Whether the `installed_resources` 'dataset' row already exists. */
+  hasInstallRow: boolean
+  /** KV `drugReference.lastUpdatedExportDate`; only the final ingest pass sets it. */
+  lastUpdatedExportDate: string | null
+  /** Whether KV `drugReference.downloadState` still holds a marker. */
+  hasDownloadMarker: boolean
+  /** Whether either drug queue has an active/waiting/delayed job. */
+  jobInFlight: boolean
+}
+
+export type DrugRowReconcileDecision =
+  | { action: 'backfill'; version: string }
+  | { action: 'skip'; reason: string }
+
+/**
+ * Decide whether a completed drug ingest is missing its `installed_resources`
+ * row and should get one.
+ *
+ * On 1.34.0 the column was still enum('zim','map'), so every ingest finished
+ * `ready` with 261k labels and a failed row write. The tier-status math reads
+ * that row, so Medicine never resolved its Standard tier. The same state also
+ * follows a manual ingest (no resourceMeta), including a reset-and-reingest.
+ *
+ * Backfill only when the ingest provably finished: the export-date marker is
+ * written by the final pass, and the download marker is cleared right after it
+ * (the same "completed" inference getIngestStatus() makes). A running job is
+ * left alone, since it writes its own row on `ready`.
+ */
+export function decideDrugRowReconcile(input: DrugRowReconcileInput): DrugRowReconcileDecision {
+  if (input.hasInstallRow) return { action: 'skip', reason: 'install row present' }
+  if (input.rowCount <= 0) return { action: 'skip', reason: 'no drug labels ingested' }
+  if (input.jobInFlight) return { action: 'skip', reason: 'drug download/ingest in flight' }
+  if (input.hasDownloadMarker) {
+    return { action: 'skip', reason: 'download marker present (ingest not finished)' }
+  }
+  const version = input.lastUpdatedExportDate?.trim()
+  if (!version) return { action: 'skip', reason: 'no export date (ingest never completed)' }
+  return { action: 'backfill', version }
+}
+
+/**
+ * Slug of the curated category whose tiers carry the given dataset resource,
+ * for the row's `collection_ref`. Null when the spec is absent or no tier lists it.
+ */
+export function findDatasetCategorySlug(
+  spec: ZimCategoriesSpec | null,
+  resourceId: string
+): string | null {
+  for (const category of spec?.categories ?? []) {
+    for (const tier of category.tiers ?? []) {
+      if ((tier.resources ?? []).some((r) => r.type === 'dataset' && r.id === resourceId)) {
+        return category.slug
+      }
+    }
+  }
+  return null
+}

admin/app/utils/eval/corpus_source.ts

@@ -0,0 +1,38 @@
+import { basename, join, resolve, sep } from 'node:path'
+
+/** Where the frozen corpus lives, relative to the app root. */
+export const EVAL_CORPUS_DIR = 'tests/eval/corpus'
+
+/**
+ * Map a retrieved chunk's `source` payload back to its corpus document id, or
+ * null if the chunk did not come from the eval corpus.
+ *
+ * The eval corpus shares the `nomad_knowledge_base` Qdrant collection with the
+ * developer's real documents — NOMAD "collections" are a payload tag, not
+ * separate Qdrant collections — and is isolated by a
+ * `collection: __nomad_eval__` filter that Qdrant applies during search. That
+ * filter does hold, but a harness whose every number depends on a filter should
+ * be able to *prove* the filter held rather than assume it. This function is
+ * that proof: the caller counts every unresolved chunk and fails the run if the
+ * count is non-zero.
+ *
+ * The check is on the resolved **path**, not the file extension. An earlier
+ * version accepted any `*.md`, which was worse than useless here: NOMAD embeds
+ * its own `admin/docs/*.md` into the knowledge base on first run, so a leaked
+ * `faq.md` would have resolved to the plausible-looking document id "faq", been
+ * counted as a merely-irrelevant chunk, and quietly depressed precision with no
+ * indication that anything had gone wrong.
+ *
+ * Mirrors the resolve-then-prefix-check guard in `RagService.resolveUploadPath`.
+ * `corpusDir` is injectable so this stays testable without a real corpus on disk.
+ */
+export function docIdFromSource(source: unknown, corpusDir?: string): string | null {
+  if (typeof source !== 'string') return null
+  const dir = resolve(corpusDir ?? join(process.cwd(), EVAL_CORPUS_DIR))
+  const abs = resolve(source)
+  // The trailing separator matters: without it, a sibling directory such as
+  // "…/corpus-backup" shares a string prefix with "…/corpus" and would pass.
+  if (!abs.startsWith(dir + sep)) return null
+  if (!abs.endsWith('.md')) return null
+  return basename(abs, '.md')
+}

admin/app/utils/eval/generation_metrics.ts

@@ -0,0 +1,254 @@
+/**
+ * Deterministic scorers for generated answers.
+ *
+ * Every metric here is computable offline with no judge model, which is what
+ * makes the generation tier gateable. An LLM judge can be layered on later for
+ * the things regex genuinely cannot see (nuanced faithfulness, completeness),
+ * but nothing in this file needs one — and a number you can compute without a
+ * model is a number that cannot drift because the judge changed.
+ *
+ * These are intentionally *conservative*. Each one is a proxy, and each proxy's
+ * blind spot is documented where it lives. A proxy you understand the limits of
+ * beats a score you cannot explain.
+ */
+
+/** Answer-level scoring of one generated response against one golden. */
+export type GenerationScores = {
+  /** Every mustInclude pattern matched and no mustNotInclude pattern did. */
+  correct: boolean
+  matchedRequired: string[]
+  missedRequired: string[]
+  /** Forbidden patterns that appeared. Non-empty means a specific wrong claim. */
+  hitForbidden: string[]
+  /** The model declined to answer. */
+  refused: boolean
+  /**
+   * Refusal behaviour was right: declined an out-of-corpus question, or
+   * answered an answerable one.
+   */
+  refusalCorrect: boolean
+  /** The answer narrated its own retrieval ("according to Context 1"). */
+  leakage: string[]
+  /** Reasoning tags survived into the user-visible answer. */
+  thinkTagLeak: boolean
+  /**
+   * Fraction of the answer's numeric claims that also appear in the injected
+   * context. null when the answer makes no numeric claims.
+   */
+  numericGroundedness: number | null
+  /** Numbers asserted by the answer that the context does not support. */
+  ungroundedNumbers: string[]
+  /** The answer used at least some markdown structure. */
+  markdownFormatted: boolean
+  /** Characters. Useful for spotting a model that answers by writing an essay. */
+  length: number
+}
+
+/**
+ * Phrases that mean "I am not answering this".
+ *
+ * Tuned against NOMAD's actual rag_context prompt, which instructs the model to
+ * answer from general knowledge rather than hedge — so a hedge here is either a
+ * correct refusal on an out-of-corpus question or a prompt regression on an
+ * answerable one. Which of those it is, is exactly what `refusalCorrect` says.
+ */
+const REFUSAL_PATTERNS: RegExp[] = [
+  /\bi (?:don'?t|do not) (?:know|have)\b/i,
+  /\bi(?:'m| am) (?:not able|unable) to\b/i,
+  /\bi (?:couldn'?t|could not|can'?t|cannot) find\b/i,
+  /\b(?:no|not enough) (?:information|details|data)\b/i,
+  /\b(?:does|do) not (?:contain|include|mention|specify|provide|cover)\b/i,
+  /\b(?:isn'?t|is not|aren'?t|are not) (?:mentioned|specified|covered|available|provided)\b/i,
+  /\bnot (?:mentioned|specified|covered|stated|documented)\b/i,
+  /\bunable to (?:answer|determine|find)\b/i,
+  /\bthere (?:is|'s) no (?:information|mention|record)\b/i,
+  // "I'd" has no space before the contraction, so this cannot reuse the
+  // `i (?:would|'d)` shape used above.
+  /\bi(?:'d| would) need more\b/i,
+]
+
+/**
+ * Phrases that narrate the retrieval machinery.
+ *
+ * The rag_context prompt explicitly forbids these ("Never narrate your
+ * retrieval or reasoning process"), so any hit is a measurable prompt
+ * regression. This is the cheapest, sharpest signal in the whole harness: pure
+ * regex, zero ambiguity, and it catches a bad prompt edit on the first run.
+ */
+const LEAKAGE_PATTERNS: RegExp[] = [
+  /\b(?:according to|based on|per|from) the (?:provided |retrieved |given |supplied )?context\b/i,
+  /\bcontext \d+\b/i,
+  /\bthe knowledge ?base\b/i,
+  /\bthe (?:provided|retrieved|supplied|given) (?:documents?|passages?|excerpts?|text)\b/i,
+  /\bthe context (?:does not|doesn'?t|is|was|seems|appears)\b/i,
+  /\b(?:in|from) the (?:documents?|passages?) (?:provided|above|below)\b/i,
+  // "I wasn't able to find specific context regarding X, but here's a general
+  // answer" is the verbatim symptom that started NOMAD's RAG work. It is both a
+  // refusal and a leak, and it is the single most important string this
+  // detector has to catch — a prompt change that brings it back must fail
+  // loudly on the very next run.
+  /\b(?:no|any|specific|relevant) context\b/i,
+  /\bsearch results?\b/i,
+]
+
+/** Reasoning-model tags that must never reach the user. */
+const THINK_TAG = /<\/?(?:think|thought|thinking|reasoning)\b[^>]*>/i
+
+export function detectRefusal(answer: string): boolean {
+  return REFUSAL_PATTERNS.some((re) => re.test(answer))
+}
+
+export function detectLeakage(answer: string): string[] {
+  return LEAKAGE_PATTERNS.filter((re) => re.test(answer)).map((re) => re.source)
+}
+
+export function hasThinkTagLeak(answer: string): boolean {
+  return THINK_TAG.test(answer)
+}
+
+/**
+ * Loose markdown check: a header, a list, emphasis, a table, or a code fence.
+ *
+ * Deliberately loose. SYSTEM_PROMPTS.default asks for markdown "for
+ * readability", not for a specific structure, so requiring headers would fail
+ * perfectly good one-sentence answers. This only catches a model that has
+ * stopped formatting entirely.
+ */
+export function isMarkdownFormatted(answer: string): boolean {
+  return /(^|\n)\s{0,3}#{1,6}\s|(^|\n)\s*[-*+]\s|(^|\n)\s*\d+\.\s|\*\*[^*]+\*\*|`[^`]+`|(^|\n)\s*\|/.test(
+    answer
+  )
+}
+
+/**
+ * Numeric tokens the answer asserts, normalized for comparison.
+ *
+ * Numbers are the highest-value fabrication signal in this domain: a wrong
+ * boiling time, bleach dose, or canner pressure is a wrong answer with real
+ * consequences, and it is exactly the kind of specific the rag_context prompt
+ * forbids inventing.
+ *
+ * Thousands separators are stripped so "2,000" and "2000" compare equal.
+ */
+export function extractNumbers(text: string): string[] {
+  const matches = text.match(/\d[\d,]*(?:\.\d+)?/g) ?? []
+  const normalized = matches.map((m) => m.replace(/,/g, '').replace(/\.0+$/, ''))
+  return [...new Set(normalized)]
+}
+
+/**
+ * Fraction of the answer's numeric claims that the injected context supports.
+ *
+ * Returns null when the answer contains no numbers — a qualitative answer is
+ * not ungrounded, it is just not measurable this way, and folding it in as 1.0
+ * would quietly inflate the score.
+ *
+ * **Known limitation, stated plainly:** this only sees numbers. An answer that
+ * fabricates a procedure or a proper noun scores a perfect 1.0 here. It is a
+ * fabrication *detector*, not a faithfulness *guarantee* — the LLM-judge tier
+ * exists for the rest. Small integers (0-10) are excluded because they appear
+ * incidentally in almost any prose ("3 layers", "step 2") and would swamp the
+ * signal with false grounding.
+ */
+export function numericGroundedness(
+  answer: string,
+  context: string
+): { score: number | null; ungrounded: string[] } {
+  const contextNumbers = new Set(extractNumbers(context))
+  const claimed = extractNumbers(answer).filter((n) => {
+    const value = Number.parseFloat(n)
+    return !Number.isNaN(value) && value > 10
+  })
+  if (claimed.length === 0) return { score: null, ungrounded: [] }
+  const ungrounded = claimed.filter((n) => !contextNumbers.has(n))
+  return { score: (claimed.length - ungrounded.length) / claimed.length, ungrounded }
+}
+
+/** Which of a golden's patterns matched, using case-insensitive regex semantics. */
+export function matchPatterns(answer: string, patterns: string[]): { matched: string[]; missed: string[] } {
+  const matched: string[] = []
+  const missed: string[] = []
+  for (const pattern of patterns) {
+    if (new RegExp(pattern, 'i').test(answer)) matched.push(pattern)
+    else missed.push(pattern)
+  }
+  return { matched, missed }
+}
+
+export type ScoreAnswerInput = {
+  answer: string
+  /** The context text actually injected into the prompt; '' when none was. */
+  context: string
+  mustInclude: string[]
+  mustNotInclude: string[]
+  expectRefusal: boolean
+}
+
+export function scoreAnswer(input: ScoreAnswerInput): GenerationScores {
+  const { answer, context, mustInclude, mustNotInclude, expectRefusal } = input
+
+  const required = matchPatterns(answer, mustInclude)
+  const forbidden = matchPatterns(answer, mustNotInclude)
+  const refused = detectRefusal(answer)
+  const grounding = numericGroundedness(answer, context)
+
+  return {
+    // An out-of-corpus question has no mustInclude patterns, so `correct` there
+    // is carried entirely by refusalCorrect below rather than by assertions.
+    correct: required.missed.length === 0 && forbidden.matched.length === 0,
+    matchedRequired: required.matched,
+    missedRequired: required.missed,
+    hitForbidden: forbidden.matched,
+    refused,
+    refusalCorrect: expectRefusal ? refused : !refused,
+    leakage: detectLeakage(answer),
+    thinkTagLeak: hasThinkTagLeak(answer),
+    numericGroundedness: grounding.score,
+    ungroundedNumbers: grounding.ungrounded,
+    markdownFormatted: isMarkdownFormatted(answer),
+    length: answer.length,
+  }
+}
+
+// --- aggregation over repeats -------------------------------------------------
+
+export type RepeatStats = {
+  /** How many of N repeats passed. */
+  passes: number
+  repeats: number
+  passRate: number
+  /**
+   * True when the outcome was neither always-pass nor always-fail. Unstable
+   * cases are excluded from gating — treating a coin flip as a regression is
+   * how a harness loses the team's trust in one afternoon.
+   */
+  unstable: boolean
+}
+
+export function summarizeRepeats(outcomes: boolean[]): RepeatStats {
+  const repeats = outcomes.length
+  const passes = outcomes.filter(Boolean).length
+  return {
+    passes,
+    repeats,
+    passRate: repeats === 0 ? 0 : passes / repeats,
+    unstable: repeats > 1 && passes > 0 && passes < repeats,
+  }
+}
+
+export type NumericSummary = { mean: number; stddev: number; n: number }
+
+/**
+ * Mean and population standard deviation, ignoring nulls.
+ *
+ * The stddev is not decoration: with temperature 0 it should be near zero, and
+ * a non-trivial value is the harness telling you the run is noisier than the
+ * difference you are about to interpret.
+ */
+export function summarizeNumeric(values: Array<number | null>): NumericSummary | null {
+  const defined = values.filter((v): v is number => v !== null && Number.isFinite(v))
+  if (defined.length === 0) return null
+  const mean = defined.reduce((a, b) => a + b, 0) / defined.length
+  const variance = defined.reduce((acc, v) => acc + (v - mean) ** 2, 0) / defined.length
+  return { mean, stddev: Math.sqrt(variance), n: defined.length }
+}

admin/app/utils/eval/golden_set.ts

@@ -0,0 +1,190 @@
+/**
+ * Loading and validating the golden question set, plus the corpus fingerprint.
+ *
+ * Pure and dependency-free so it runs under bare node in unit tests. Callers
+ * supply file contents; nothing here touches the filesystem except through the
+ * explicit `readFile`-shaped arguments the service passes in.
+ */
+
+/** A prior conversation turn, for multi-turn / coreference cases. */
+export type GoldenTurn = { role: 'user' | 'assistant'; content: string }
+
+export type Golden = {
+  /** Stable identifier. Used as the join key across reports and baselines. */
+  id: string
+  /** The user's question, as they would actually type it. */
+  query: string
+  /** Conversation history preceding `query`. Empty for single-turn cases. */
+  turns: GoldenTurn[]
+  /**
+   * Documents that genuinely answer the question, by corpus doc id (the
+   * markdown filename without its extension). Empty for out-of-corpus cases.
+   */
+  relevantDocIds: string[]
+  /**
+   * Patterns the answer must contain. Each entry is a case-insensitive
+   * **regular expression** — plain text is a valid regex, and alternation lets
+   * one entry accept "3 minutes" or "three minutes" without inflating the list.
+   */
+  mustInclude: string[]
+  /** Patterns the answer must NOT contain. Same regex semantics. */
+  mustNotInclude: string[]
+  /**
+   * True when the corpus genuinely cannot answer the question and the correct
+   * behaviour is to decline rather than invent. Scored as refusal-correctness.
+   */
+  expectRefusal: boolean
+  /** Free-form buckets for per-slice reporting (single-hop, acronym, ...). */
+  tags: string[]
+}
+
+export class GoldenSetError extends Error {}
+
+/**
+ * Parse a JSONL golden file, validating hard enough that a typo fails loudly at
+ * load rather than silently scoring zero for the rest of the project's life.
+ */
+export function parseGoldens(jsonl: string, sourceName = 'goldens'): Golden[] {
+  const goldens: Golden[] = []
+  const seen = new Set<string>()
+
+  jsonl.split('\n').forEach((rawLine, idx) => {
+    const line = rawLine.trim()
+    if (!line || line.startsWith('//')) return
+
+    const where = `${sourceName}:${idx + 1}`
+    let parsed: any
+    try {
+      parsed = JSON.parse(line)
+    } catch (err) {
+      throw new GoldenSetError(`${where}: not valid JSON — ${(err as Error).message}`)
+    }
+
+    const req = (field: string) => {
+      if (parsed[field] === undefined) throw new GoldenSetError(`${where}: missing "${field}"`)
+      return parsed[field]
+    }
+
+    const id = req('id')
+    if (typeof id !== 'string' || !id) throw new GoldenSetError(`${where}: "id" must be a non-empty string`)
+    if (seen.has(id)) throw new GoldenSetError(`${where}: duplicate id "${id}"`)
+    seen.add(id)
+
+    const query = req('query')
+    if (typeof query !== 'string' || !query.trim()) {
+      throw new GoldenSetError(`${where}: "query" must be a non-empty string`)
+    }
+
+    const strArray = (field: string): string[] => {
+      const v = parsed[field] ?? []
+      if (!Array.isArray(v) || v.some((x) => typeof x !== 'string')) {
+        throw new GoldenSetError(`${where}: "${field}" must be an array of strings`)
+      }
+      return v
+    }
+
+    const mustInclude = strArray('mustInclude')
+    const mustNotInclude = strArray('mustNotInclude')
+    // Compile every pattern now. A bad regex that only blows up on the one run
+    // where it finally matches is far worse than one that fails at load.
+    for (const pattern of [...mustInclude, ...mustNotInclude]) {
+      try {
+        new RegExp(pattern, 'i')
+      } catch (err) {
+        throw new GoldenSetError(`${where}: invalid regex ${JSON.stringify(pattern)} — ${(err as Error).message}`)
+      }
+    }
+
+    const turns = (parsed.turns ?? []) as GoldenTurn[]
+    if (!Array.isArray(turns) || turns.some((t) => t?.role !== 'user' && t?.role !== 'assistant')) {
+      throw new GoldenSetError(`${where}: "turns" must be an array of {role: user|assistant, content}`)
+    }
+
+    const expectRefusal = Boolean(parsed.expectRefusal)
+    const relevantDocIds = strArray('relevantDocIds')
+    if (expectRefusal && relevantDocIds.length > 0) {
+      throw new GoldenSetError(
+        `${where}: "${id}" expects a refusal but also lists relevant documents — one of those is wrong`
+      )
+    }
+    if (!expectRefusal && relevantDocIds.length === 0) {
+      throw new GoldenSetError(
+        `${where}: "${id}" lists no relevant documents and does not expect a refusal — it can never be scored`
+      )
+    }
+
+    goldens.push({
+      id,
+      query,
+      turns,
+      relevantDocIds,
+      mustInclude,
+      mustNotInclude,
+      expectRefusal,
+      tags: strArray('tags'),
+    })
+  })
+
+  if (goldens.length === 0) throw new GoldenSetError(`${sourceName}: no goldens found`)
+  return goldens
+}
+
+/**
+ * Every doc id a golden refers to must exist in the corpus, or recall is being
+ * computed against a target that can never be hit.
+ */
+export function assertGoldensMatchCorpus(goldens: Golden[], corpusDocIds: Iterable<string>): void {
+  const corpus = new Set(corpusDocIds)
+  const missing = new Set<string>()
+  for (const g of goldens) {
+    for (const docId of g.relevantDocIds) if (!corpus.has(docId)) missing.add(docId)
+  }
+  if (missing.size > 0) {
+    throw new GoldenSetError(
+      `goldens reference documents that are not in the corpus: ${[...missing].sort().join(', ')}`
+    )
+  }
+}
+
+/**
+ * Inputs to the corpus fingerprint. Anything that changes what ends up in the
+ * vector store belongs here — if it changes, prior reports are not comparable.
+ */
+export type FingerprintInputs = {
+  /** docId -> raw file contents. */
+  documents: Map<string, string>
+  chunkTokens: number
+  chunkOverlapTokens: number
+  charToTokenRatio: number
+  embeddingModel: string
+  embeddingDimension: number
+}
+
+/**
+ * A stable hash over the corpus and the ingest parameters that shaped it.
+ *
+ * Reports carry this. Comparing two reports with different fingerprints is
+ * meaningless, and the compare command refuses to do it — which is the whole
+ * point: it makes "I changed the chunk size and the score moved" impossible to
+ * confuse with "I changed the prompt and the score moved".
+ *
+ * `hash` is injected so this stays pure and testable; the service passes a
+ * node:crypto sha256.
+ */
+export function computeCorpusFingerprint(
+  inputs: FingerprintInputs,
+  hash: (input: string) => string
+): string {
+  const docIds = [...inputs.documents.keys()].sort()
+  const parts: string[] = [
+    `chunkTokens=${inputs.chunkTokens}`,
+    `chunkOverlapTokens=${inputs.chunkOverlapTokens}`,
+    `charToTokenRatio=${inputs.charToTokenRatio}`,
+    `embeddingModel=${inputs.embeddingModel}`,
+    `embeddingDimension=${inputs.embeddingDimension}`,
+  ]
+  for (const id of docIds) {
+    parts.push(`doc=${id}\n${inputs.documents.get(id)}`)
+  }
+  return hash(parts.join('\n---\n')).slice(0, 16)
+}

admin/app/utils/eval/quiet.ts

@@ -0,0 +1,22 @@
+import logger from '@adonisjs/core/services/logger'
+
+/**
+ * Turn down the application logger for the duration of an eval run.
+ *
+ * RagService logs every keyword extraction, every batch, and the text of every
+ * chunk at debug level. That is the right default for diagnosing an ingest, and
+ * completely wrong for a tool whose entire output is a score table — the
+ * numbers scroll off the top of the terminal before you can read them.
+ *
+ * Returns a restore function so the level is put back even if the caller throws.
+ * Pass `debug: true` (from `--debug`) to leave the logger alone when you are
+ * actually trying to see the pipeline internals.
+ */
+export function quietLogging(debug = false): () => void {
+  if (debug) return () => {}
+  const previous = logger.level
+  logger.level = 'warn'
+  return () => {
+    logger.level = previous
+  }
+}

admin/app/utils/eval/report.ts

@@ -0,0 +1,264 @@
+/**
+ * Report shape, baseline diffing, and the gate decision.
+ *
+ * Pure functions over plain objects — the service layer does the file I/O. The
+ * comparison logic in particular has to be testable without a filesystem,
+ * because it is the piece that decides whether a pull request is blocked.
+ */
+
+export type ReportKind = 'retrieval' | 'generation'
+
+/**
+ * Provenance. Every field here exists to answer "is this report comparable to
+ * that one?", and `corpusFingerprint` is the one that can veto the comparison
+ * outright.
+ */
+export type ReportMeta = {
+  kind: ReportKind
+  createdAt: string
+  corpusFingerprint: string
+  gitSha: string | null
+  gitBranch: string | null
+  gitDirty: boolean
+  nomadVersion: string | null
+  /** Host platform. Recorded for context; never used to justify a score. */
+  platform: {
+    cpuArchitecture: string | null
+    osName: string | null
+    nodeVersion: string
+  }
+  params: Record<string, unknown>
+}
+
+/**
+ * A report is metadata plus a flat bag of named metrics plus the per-case
+ * detail. Flattening the metrics is deliberate: it means the diff logic does
+ * not need to know anything about retrieval versus generation, so adding a
+ * metric later requires no change to the comparison or the gate.
+ */
+export type EvalReport = {
+  meta: ReportMeta
+  /** metric name -> value. null means "not measurable in this run". */
+  metrics: Record<string, number | null>
+  /** tag -> metric name -> value. */
+  byTag: Record<string, Record<string, number | null>>
+  /** Per-question detail, for the "what did it actually say" question. */
+  cases: unknown[]
+}
+
+/**
+ * Whether a metric going up is good.
+ *
+ * Without this the diff cannot tell an improvement from a regression, and a
+ * leakage rate falling to zero would be reported as a failure.
+ */
+export type MetricDirection = 'higher-is-better' | 'lower-is-better'
+
+export const METRIC_DIRECTIONS: Record<string, MetricDirection> = {
+  // retrieval
+  'recall@1': 'higher-is-better',
+  'recall@3': 'higher-is-better',
+  'recall@5': 'higher-is-better',
+  'recall@10': 'higher-is-better',
+  'hitRate@1': 'higher-is-better',
+  'hitRate@3': 'higher-is-better',
+  'hitRate@5': 'higher-is-better',
+  'hitRate@10': 'higher-is-better',
+  'precision@1': 'higher-is-better',
+  'precision@3': 'higher-is-better',
+  'precision@5': 'higher-is-better',
+  'precision@10': 'higher-is-better',
+  'ndcg@1': 'higher-is-better',
+  'ndcg@3': 'higher-is-better',
+  'ndcg@5': 'higher-is-better',
+  'ndcg@10': 'higher-is-better',
+  mrr: 'higher-is-better',
+  emptyRateOnAnswerable: 'lower-is-better',
+  nonEmptyRateOnRefusal: 'lower-is-better',
+  // generation
+  correctness: 'higher-is-better',
+  refusalCorrectness: 'higher-is-better',
+  leakageRate: 'lower-is-better',
+  thinkTagLeakRate: 'lower-is-better',
+  markdownRate: 'higher-is-better',
+  groundedness: 'higher-is-better',
+  // context budgeting
+  //
+  // These are diagnostics as much as gates. A rise in historyElidedRate means
+  // conversations are being trimmed more often, which explains a correctness
+  // drop that would otherwise look like the model getting worse. promptTokenError
+  // gates the token estimator itself: every budget decision rests on it, so a
+  // regression there quietly degrades everything downstream.
+  historyElidedRate: 'lower-is-better',
+  chunksDroppedRate: 'lower-is-better',
+  promptTokenError: 'lower-is-better',
+}
+
+export type MetricDelta = {
+  metric: string
+  baseline: number | null
+  current: number | null
+  delta: number | null
+  direction: MetricDirection
+  /** Signed so that positive always means "better", whatever the direction. */
+  improvement: number | null
+  regressed: boolean
+  /** Present in one report but not the other. */
+  onlyIn?: 'baseline' | 'current'
+}
+
+export type ComparisonResult = {
+  comparable: boolean
+  /** Why the comparison was refused, when it was. */
+  incomparableReason?: string
+  tolerance: number
+  deltas: MetricDelta[]
+  regressions: MetricDelta[]
+  improvements: MetricDelta[]
+}
+
+/**
+ * Default tolerance band.
+ *
+ * Not zero, on purpose. Even at temperature 0 the generation tier moves a
+ * little between runs, and a gate that fires on 0.001 gets switched off within
+ * a week. 0.02 is roughly "one question in fifty" on a 99-question set — small
+ * enough to catch a real regression, large enough to ignore a coin flip.
+ */
+export const DEFAULT_TOLERANCE = 0.02
+
+/**
+ * Slack for floating-point representation error at the tolerance boundary.
+ *
+ * Without it, `0.9 - 0.02` evaluates to a delta of -0.020000000000000018, which
+ * is "greater than the tolerance" by 1.8e-17 and would block a pull request. A
+ * gate that fires on the last bit of a double is a gate people learn to ignore.
+ */
+const BOUNDARY_EPSILON = 1e-9
+
+export function compareReports(
+  baseline: EvalReport,
+  current: EvalReport,
+  tolerance: number = DEFAULT_TOLERANCE
+): ComparisonResult {
+  if (baseline.meta.kind !== current.meta.kind) {
+    return incomparable(
+      `cannot compare a ${baseline.meta.kind} report against a ${current.meta.kind} report`,
+      tolerance
+    )
+  }
+
+  // The hard veto. Different corpora, chunk sizes, or embedding models mean the
+  // two numbers were produced by different experiments, and diffing them would
+  // manufacture a regression (or hide one) out of nothing.
+  if (baseline.meta.corpusFingerprint !== current.meta.corpusFingerprint) {
+    return incomparable(
+      `corpus fingerprint changed (${baseline.meta.corpusFingerprint} -> ${current.meta.corpusFingerprint}). ` +
+        'The corpus, chunk size, or embedding model differs, so these runs are not measuring the same thing. Re-baseline instead.',
+      tolerance
+    )
+  }
+
+  const names = [...new Set([...Object.keys(baseline.metrics), ...Object.keys(current.metrics)])].sort()
+  const deltas: MetricDelta[] = names.map((metric) => {
+    const b = baseline.metrics[metric] ?? null
+    const c = current.metrics[metric] ?? null
+    const direction = METRIC_DIRECTIONS[metric] ?? 'higher-is-better'
+    const onlyIn =
+      !(metric in baseline.metrics) ? ('current' as const)
+      : !(metric in current.metrics) ? ('baseline' as const)
+      : undefined
+
+    if (b === null || c === null) {
+      return { metric, baseline: b, current: c, delta: null, direction, improvement: null, regressed: false, onlyIn }
+    }
+
+    const delta = c - b
+    const improvement = direction === 'higher-is-better' ? delta : -delta
+    return {
+      metric,
+      baseline: b,
+      current: c,
+      delta,
+      direction,
+      improvement,
+      regressed: improvement < -(tolerance + BOUNDARY_EPSILON),
+      onlyIn,
+    }
+  })
+
+  return {
+    comparable: true,
+    tolerance,
+    deltas,
+    regressions: deltas.filter((d) => d.regressed),
+    improvements: deltas.filter(
+      (d) => d.improvement !== null && d.improvement > tolerance + BOUNDARY_EPSILON
+    ),
+  }
+}
+
+function incomparable(reason: string, tolerance: number): ComparisonResult {
+  return {
+    comparable: false,
+    incomparableReason: reason,
+    tolerance,
+    deltas: [],
+    regressions: [],
+    improvements: [],
+  }
+}
+
+/** Flatten a `Record<number, number|null>` into `name@k` keys. */
+export function flattenByK(
+  prefix: string,
+  values: Record<number, number | null>
+): Record<string, number | null> {
+  return Object.fromEntries(Object.entries(values).map(([k, v]) => [`${prefix}@${k}`, v]))
+}
+
+const pct = (v: number | null) => (v === null ? 'n/a' : v.toFixed(4))
+
+/** Human-readable summary, worst regressions first. */
+export function renderComparisonMarkdown(
+  baselinePath: string,
+  currentPath: string,
+  result: ComparisonResult
+): string {
+  const lines: string[] = ['# Eval comparison', '']
+  lines.push(`- baseline: \`${baselinePath}\``)
+  lines.push(`- current:  \`${currentPath}\``)
+  lines.push(`- tolerance: ${result.tolerance}`)
+  lines.push('')
+
+  if (!result.comparable) {
+    lines.push('## Not comparable', '', result.incomparableReason ?? 'unknown reason', '')
+    return lines.join('\n')
+  }
+
+  lines.push(
+    result.regressions.length > 0
+      ? `## ${result.regressions.length} regression(s)`
+      : '## No regressions'
+  )
+  lines.push('')
+  lines.push('| metric | baseline | current | delta | verdict |')
+  lines.push('|---|---:|---:|---:|---|')
+
+  // Worst first — the thing a developer needs is at the top, not sorted
+  // alphabetically halfway down a wall of unchanged rows.
+  const ordered = [...result.deltas].sort((a, b) => (a.improvement ?? 0) - (b.improvement ?? 0))
+  for (const d of ordered) {
+    const verdict =
+      d.onlyIn === 'current' ? 'new'
+      : d.onlyIn === 'baseline' ? 'removed'
+      : d.delta === null ? '—'
+      : d.regressed ? '**REGRESSED**'
+      : (d.improvement ?? 0) > result.tolerance ? 'improved'
+      : 'within tolerance'
+    const arrow = d.delta === null ? 'n/a' : `${d.delta >= 0 ? '+' : ''}${d.delta.toFixed(4)}`
+    lines.push(`| ${d.metric} | ${pct(d.baseline)} | ${pct(d.current)} | ${arrow} | ${verdict} |`)
+  }
+  lines.push('')
+  return lines.join('\n')
+}

admin/app/utils/eval/retrieval_metrics.ts

@@ -0,0 +1,356 @@
+/**
+ * Information-retrieval metrics for the RAG eval harness.
+ *
+ * Pure functions over ranked results — no I/O, no models, no clock. Everything
+ * here is deterministic, which is the point: a change in these numbers is
+ * unambiguously a change in the code, never in the weather.
+ *
+ * ## Two levels of measurement, and why both
+ *
+ * Retrieval returns *chunks*, but a golden answer lives in a *document*. The
+ * two questions we care about are different, so they are measured differently:
+ *
+ * - **Document level** (recall, MRR, nDCG): "did the answer's document make it
+ *   into the context at all, and how near the top?" Chunks are collapsed to
+ *   their document, keeping each document's best rank.
+ * - **Chunk level** (precision): "how much of what we injected is noise?" This
+ *   is deliberately *not* deduped — five chunks from one irrelevant document
+ *   cost five slots of a small model's context and should be counted five
+ *   times.
+ *
+ * Reporting only one of these hides a real failure mode in the other.
+ */
+
+/** One retrieved chunk, reduced to what scoring needs. */
+export type ScoredChunk = {
+  /** Corpus document this chunk came from; null if it could not be resolved. */
+  docId: string | null
+  /** The pipeline's final (post-rerank) score. */
+  score: number
+  /** The raw pre-rerank cosine score, when available. */
+  semanticScore?: number
+}
+
+export type RetrievalCase = {
+  id: string
+  tags: string[]
+  /** Ranked chunks, best first, exactly as the pipeline returned them. */
+  retrieved: ScoredChunk[]
+  /** Documents that genuinely answer the question. */
+  relevantDocIds: string[]
+  /** True when the right behaviour is to retrieve nothing useful. */
+  expectRefusal: boolean
+}
+
+/**
+ * Collapse a ranked chunk list to a ranked document list, keeping each
+ * document's best (earliest) position. Chunks with no resolvable document are
+ * dropped — they still occupy a rank in the chunk-level metrics, but they
+ * cannot be credited to any document.
+ */
+export function toDocumentRanking(retrieved: ScoredChunk[]): string[] {
+  const seen = new Set<string>()
+  const ranking: string[] = []
+  for (const chunk of retrieved) {
+    if (!chunk.docId || seen.has(chunk.docId)) continue
+    seen.add(chunk.docId)
+    ranking.push(chunk.docId)
+  }
+  return ranking
+}
+
+/**
+ * Fraction of the relevant documents that appear within the top `k` chunks.
+ *
+ * Returns null when there are no relevant documents (an out-of-corpus case) —
+ * recall is undefined there, and returning 0 would drag the mean down for
+ * questions that are *supposed* to retrieve nothing.
+ */
+export function recallAtK(retrieved: ScoredChunk[], relevantDocIds: string[], k: number): number | null {
+  if (relevantDocIds.length === 0) return null
+  const relevant = new Set(relevantDocIds)
+  const found = new Set<string>()
+  for (const chunk of retrieved.slice(0, k)) {
+    if (chunk.docId && relevant.has(chunk.docId)) found.add(chunk.docId)
+  }
+  return found.size / relevant.size
+}
+
+/**
+ * 1 if *any* relevant document appears in the top `k`, else 0.
+ *
+ * Distinct from recall on multi-hop questions: finding one of two required
+ * documents is a hit but only 0.5 recall. Reporting both is what separates
+ * "found something" from "found enough to answer".
+ */
+export function hitRateAtK(retrieved: ScoredChunk[], relevantDocIds: string[], k: number): number | null {
+  if (relevantDocIds.length === 0) return null
+  const relevant = new Set(relevantDocIds)
+  return retrieved.slice(0, k).some((c) => c.docId && relevant.has(c.docId)) ? 1 : 0
+}
+
+/**
+ * Fraction of the top `k` *chunks* that come from a relevant document.
+ *
+ * Not deduped, on purpose — this measures context pollution, and a small model
+ * drowning in four irrelevant chunks does not care that they share a source.
+ * Denominator is min(k, retrieved.length) so a pipeline returning 2 good chunks
+ * is not punished for the 3 it correctly declined to return.
+ */
+export function precisionAtK(retrieved: ScoredChunk[], relevantDocIds: string[], k: number): number | null {
+  if (relevantDocIds.length === 0) return null
+  const window = retrieved.slice(0, k)
+  if (window.length === 0) return 0
+  const relevant = new Set(relevantDocIds)
+  const hits = window.filter((c) => c.docId && relevant.has(c.docId)).length
+  return hits / window.length
+}
+
+/**
+ * Reciprocal of the rank of the first relevant chunk (1-indexed); 0 if none.
+ * Averaged over cases this is MRR.
+ */
+export function reciprocalRank(retrieved: ScoredChunk[], relevantDocIds: string[]): number | null {
+  if (relevantDocIds.length === 0) return null
+  const relevant = new Set(relevantDocIds)
+  const idx = retrieved.findIndex((c) => c.docId && relevant.has(c.docId))
+  return idx === -1 ? 0 : 1 / (idx + 1)
+}
+
+/**
+ * Normalized discounted cumulative gain at `k`, over the **document** ranking,
+ * with binary gain.
+ *
+ *   DCG@k  = Σ_{i=1..k}  rel_i / log2(i + 1)
+ *   IDCG@k = Σ_{i=1..min(R, k)}  1 / log2(i + 1)        where R = |relevant|
+ *   nDCG@k = DCG@k / IDCG@k
+ *
+ * The ideal ranking is defined against the *known* number of relevant documents
+ * rather than against however many happened to be retrieved. That distinction
+ * matters: normalizing against the retrieved set would score a run that found
+ * one of three required documents, and ranked it first, as a perfect 1.0.
+ *
+ * This is the metric that catches "right documents, wrong order" — a reranking
+ * regression that leaves recall untouched while pushing the answer to position
+ * five, where a 1B model with a 2-result budget will never see it.
+ */
+export function ndcgAtK(retrieved: ScoredChunk[], relevantDocIds: string[], k: number): number | null {
+  if (relevantDocIds.length === 0) return null
+  const relevant = new Set(relevantDocIds)
+  const docRanking = toDocumentRanking(retrieved).slice(0, k)
+
+  let dcg = 0
+  docRanking.forEach((docId, i) => {
+    if (relevant.has(docId)) dcg += 1 / Math.log2(i + 2) // i is 0-indexed, so rank = i + 1
+  })
+
+  let idcg = 0
+  for (let i = 0; i < Math.min(relevant.size, k); i++) {
+    idcg += 1 / Math.log2(i + 2)
+  }
+
+  return idcg === 0 ? 0 : dcg / idcg
+}
+
+/**
+ * Score separation: the gap between what relevant and irrelevant chunks score.
+ *
+ * This is how the similarity threshold gets calibrated with evidence instead of
+ * intuition. If the relevant p10 sits below the irrelevant p90, no threshold
+ * can cleanly separate them and the honest conclusion is that the *retriever*
+ * needs work, not the cutoff.
+ *
+ * ## Two axes, because there are two cutoffs
+ *
+ * A chunk carries two scores and they are not interchangeable:
+ *
+ * - the **semantic** score is raw cosine, and it is what Qdrant's
+ *   `score_threshold` filters on — before reranking ever runs;
+ * - the **final** score is what reranking and source diversity produced, and it
+ *   is what `RAG_MIN_FINAL_SCORE` filters on.
+ *
+ * Reranking's boosts are score-scaled and additive (up to ~1.275x), so the two
+ * distributions sit at different heights. Calibrating one cutoff against the
+ * other axis' percentiles is off by exactly that factor, which is why both are
+ * reported separately rather than collapsed into one number.
+ */
+export type ScoreDistribution = {
+  count: number
+  min: number
+  p10: number
+  median: number
+  p90: number
+  max: number
+  mean: number
+}
+
+export function describeScores(scores: number[]): ScoreDistribution | null {
+  if (scores.length === 0) return null
+  const sorted = [...scores].sort((a, b) => a - b)
+  const pct = (p: number) => {
+    // Nearest-rank percentile: no interpolation, so a reported value is always
+    // a value that actually occurred.
+    const idx = Math.min(sorted.length - 1, Math.max(0, Math.ceil((p / 100) * sorted.length) - 1))
+    return sorted[idx]
+  }
+  return {
+    count: sorted.length,
+    min: sorted[0],
+    p10: pct(10),
+    median: pct(50),
+    p90: pct(90),
+    max: sorted[sorted.length - 1],
+    mean: sorted.reduce((a, b) => a + b, 0) / sorted.length,
+  }
+}
+
+/** Mean over the entries that are actually defined; null if none are. */
+export function meanOf(values: Array<number | null>): number | null {
+  const defined = values.filter((v): v is number => v !== null)
+  if (defined.length === 0) return null
+  return defined.reduce((a, b) => a + b, 0) / defined.length
+}
+
+export type RetrievalCaseResult = {
+  id: string
+  tags: string[]
+  retrievedCount: number
+  relevantDocIds: string[]
+  retrievedDocIds: string[]
+  expectRefusal: boolean
+  recall: Record<number, number | null>
+  hitRate: Record<number, number | null>
+  precision: Record<number, number | null>
+  ndcg: Record<number, number | null>
+  reciprocalRank: number | null
+  /** True when this case retrieved nothing at all. */
+  empty: boolean
+  /**
+   * The highest-ranked chunk's scores, or null when nothing was retrieved. Shown
+   * for out-of-corpus leaks: the gap between the two says how much of a false
+   * positive the reranker's lexical boosts supplied.
+   */
+  top: { score: number; semanticScore: number | null } | null
+}
+
+export type RetrievalAggregate = {
+  cases: number
+  /** Cases with at least one relevant document (i.e. excluding refusal cases). */
+  answerable: number
+  recall: Record<number, number | null>
+  hitRate: Record<number, number | null>
+  precision: Record<number, number | null>
+  ndcg: Record<number, number | null>
+  mrr: number | null
+  /**
+   * Fraction of *answerable* questions that retrieved nothing. A non-zero value
+   * means the score threshold is filtering out real answers.
+   */
+  emptyRateOnAnswerable: number | null
+  /**
+   * Fraction of *out-of-corpus* questions that retrieved something anyway. This
+   * is the other half of the threshold trade-off: context handed to the model
+   * for a question the corpus cannot answer is exactly what produces a
+   * confident, wrong reply.
+   */
+  nonEmptyRateOnRefusal: number | null
+  /** Semantic (raw cosine) scores — the axis Qdrant's score_threshold filters on. */
+  relevantScores: ScoreDistribution | null
+  irrelevantScores: ScoreDistribution | null
+  /** Post-rerank scores — the axis RAG_MIN_FINAL_SCORE filters on. */
+  relevantFinalScores: ScoreDistribution | null
+  irrelevantFinalScores: ScoreDistribution | null
+}
+
+export const DEFAULT_K_VALUES = [1, 3, 5, 10]
+
+export function scoreCase(c: RetrievalCase, kValues: number[] = DEFAULT_K_VALUES): RetrievalCaseResult {
+  const byK = <T>(fn: (k: number) => T): Record<number, T> =>
+    Object.fromEntries(kValues.map((k) => [k, fn(k)]))
+
+  return {
+    id: c.id,
+    tags: c.tags,
+    retrievedCount: c.retrieved.length,
+    relevantDocIds: c.relevantDocIds,
+    retrievedDocIds: toDocumentRanking(c.retrieved),
+    expectRefusal: c.expectRefusal,
+    recall: byK((k) => recallAtK(c.retrieved, c.relevantDocIds, k)),
+    hitRate: byK((k) => hitRateAtK(c.retrieved, c.relevantDocIds, k)),
+    precision: byK((k) => precisionAtK(c.retrieved, c.relevantDocIds, k)),
+    ndcg: byK((k) => ndcgAtK(c.retrieved, c.relevantDocIds, k)),
+    reciprocalRank: reciprocalRank(c.retrieved, c.relevantDocIds),
+    empty: c.retrieved.length === 0,
+    top:
+      c.retrieved.length === 0
+        ? null
+        : { score: c.retrieved[0].score, semanticScore: c.retrieved[0].semanticScore ?? null },
+  }
+}
+
+export function aggregate(
+  cases: RetrievalCase[],
+  results: RetrievalCaseResult[],
+  kValues: number[] = DEFAULT_K_VALUES
+): RetrievalAggregate {
+  const answerable = results.filter((r) => !r.expectRefusal)
+  const refusals = results.filter((r) => r.expectRefusal)
+
+  const byK = (pick: (r: RetrievalCaseResult, k: number) => number | null): Record<number, number | null> =>
+    Object.fromEntries(kValues.map((k) => [k, meanOf(results.map((r) => pick(r, k)))]))
+
+  // Split every retrieved chunk's score by whether its document was relevant.
+  // Refusal cases contribute only to the irrelevant side — by definition
+  // nothing they retrieve is relevant, and that is precisely the population a
+  // threshold needs to exclude.
+  const relevantScores: number[] = []
+  const irrelevantScores: number[] = []
+  const relevantFinalScores: number[] = []
+  const irrelevantFinalScores: number[] = []
+  for (const c of cases) {
+    const relevant = new Set(c.relevantDocIds)
+    for (const chunk of c.retrieved) {
+      const isRelevant = Boolean(chunk.docId && relevant.has(chunk.docId))
+      ;(isRelevant ? relevantScores : irrelevantScores).push(chunk.semanticScore ?? chunk.score)
+      ;(isRelevant ? relevantFinalScores : irrelevantFinalScores).push(chunk.score)
+    }
+  }
+
+  return {
+    cases: results.length,
+    answerable: answerable.length,
+    recall: byK((r, k) => r.recall[k]),
+    hitRate: byK((r, k) => r.hitRate[k]),
+    precision: byK((r, k) => r.precision[k]),
+    ndcg: byK((r, k) => r.ndcg[k]),
+    mrr: meanOf(results.map((r) => r.reciprocalRank)),
+    emptyRateOnAnswerable:
+      answerable.length === 0 ? null : answerable.filter((r) => r.empty).length / answerable.length,
+    nonEmptyRateOnRefusal:
+      refusals.length === 0 ? null : refusals.filter((r) => !r.empty).length / refusals.length,
+    relevantScores: describeScores(relevantScores),
+    irrelevantScores: describeScores(irrelevantScores),
+    relevantFinalScores: describeScores(relevantFinalScores),
+    irrelevantFinalScores: describeScores(irrelevantFinalScores),
+  }
+}
+
+/** Aggregate restricted to cases carrying a given tag, for per-slice reporting. */
+export function aggregateByTag(
+  cases: RetrievalCase[],
+  results: RetrievalCaseResult[],
+  kValues: number[] = DEFAULT_K_VALUES
+): Record<string, RetrievalAggregate> {
+  const tags = new Set(results.flatMap((r) => r.tags))
+  const byId = new Map(cases.map((c) => [c.id, c]))
+  const out: Record<string, RetrievalAggregate> = {}
+  for (const tag of [...tags].sort()) {
+    const tagged = results.filter((r) => r.tags.includes(tag))
+    out[tag] = aggregate(
+      tagged.map((r) => byId.get(r.id)!).filter(Boolean),
+      tagged,
+      kValues
+    )
+  }
+  return out
+}

admin/app/utils/gpu_model.ts

@@ -0,0 +1,36 @@
+/**
+ * Is this GPU "model" a placeholder rather than a real name?
+ *
+ * systeminformation resolves PCI ids against the container's pci.ids database.
+ * When a card is newer than that file it reports the raw id verbatim — an RTX
+ * 5060 comes back as "Device 2d05". Vendor detection still succeeds, so these
+ * strings otherwise pass as legitimate model names.
+ *
+ * Deliberately narrow: it must never reject a real product name. No shipping
+ * GPU is called "Device" followed by four hex digits, and the Microsoft entries
+ * below are placeholder adapter names that appear nowhere outside a Windows or
+ * WSL graphics stack.
+ *
+ * Lives in its own module rather than beside its first caller because both the
+ * leaderboard submission path and the Settings > System display path need the
+ * same answer, and they diverged once already: #1165 fixed the submission and
+ * left the System page reporting the raw id (#1196). One definition, two
+ * callers, so the next placeholder shape only has to be added once.
+ */
+export function isUnresolvedGpuModel(model: string): boolean {
+  const s = model.trim()
+  if (s === '') return true
+  if (/^device\s+[0-9a-f]{4}$/i.test(s)) return true
+  if (/^unknown$/i.test(s)) return true
+  // WSL2 exposes the GPU through /dev/dxg rather than the real adapter, so
+  // si.graphics() reports Microsoft's generic placeholder even while CUDA work
+  // is running on a physical card. Left unhandled, an RTX 3090 reaches the
+  // public leaderboard labelled "Microsoft Basic Render Driver", which is worse
+  // than no label: it is wrong, and it fragments per-hardware grouping (#1218).
+  //
+  // These are Microsoft's own placeholder adapter names and appear nowhere
+  // outside a Windows/WSL graphics stack, so this cannot reject a real product
+  // name on a native Linux host.
+  if (/^microsoft basic (render driver|display adapter)$/i.test(s)) return true
+  return false
+}

admin/app/utils/kb_orphan_decision.ts

@@ -0,0 +1,140 @@
+import { sep } from 'node:path'
+
+/**
+ * Past this share of a root's indexed sources, a single sync refuses to purge
+ * that root at all. Losing most of a root at once is far likelier to mean the
+ * root points somewhere wrong (a different disk, a stale copy, a half-finished
+ * data-path move per #1050) than that the user deleted most of their library
+ * by hand. Real deletions through the UI purge their own vectors immediately
+ * (ZimService.delete, deleteFileBySource), so the sweep only ever mops up
+ * stragglers and has no business removing the bulk of a root.
+ */
+export const ORPHAN_PURGE_MAX_FRACTION = 0.5
+
+/**
+ * The mass-removal guard only engages once this many sources would go. Below
+ * it, the fraction is too noisy to mean anything (1 of 2 is 50%) and the cost
+ * of a wrong purge is a few minutes of re-embedding, not hours.
+ */
+export const ORPHAN_PURGE_MIN_GUARDED = 5
+
+export type WithheldOrphans = {
+  /** Scan root whose orphans were left in place. */
+  root: string
+  /** How many indexed sources under it would have been purged. */
+  count: number
+  /**
+   * `empty_root`: the root was walked but held no embeddable files.
+   * `mass_removal`: the purge would remove more than ORPHAN_PURGE_MAX_FRACTION
+   * of the root's indexed sources.
+   */
+  reason: 'empty_root' | 'mass_removal'
+}
+
+export type OrphanDecision = {
+  /** Sources safe to purge. */
+  orphans: string[]
+  /** Roots whose missing sources were held back, for the operator to see. */
+  withheld: WithheldOrphans[]
+}
+
+/**
+ * Decision for the reverse sweep in `RagService.scanAndSyncStorage`.
+ *
+ * This is the pure, I/O-free core of the orphan check described in issue
+ * #1170: `scanAndSyncStorage` already builds `sourcesInQdrant` (from a facet
+ * query) and `embeddableFiles` (from a disk scan) in the same pass, but only
+ * ever asked "is this on-disk file already embedded?" — never the reverse
+ * "does this Qdrant source still have a file on disk?" Sources left behind by
+ * `ZimService.delete()` (which never touched Qdrant) or by
+ * `reconcileReplacedContentFile`'s `qdrant_not_running` no-op therefore never
+ * got reaped.
+ *
+ * Decided per scanned root, because the roots fail independently. Two guards
+ * apply to each, and either one withholds the whole root for this cycle:
+ *
+ * 1. A root with zero embeddable files keeps its sources (#1378). Walking a
+ *    directory successfully does not prove it holds the content: boot runs
+ *    ensureDirectoryExists() on the zim root, so a separate volume that
+ *    failed to mount leaves a real, empty mountpoint behind (often holding a
+ *    freshly regenerated kiwix-library.xml, which is why this counts
+ *    embeddable files rather than directory entries). An empty root is
+ *    indistinguishable from an unmounted one, and a single upload in
+ *    kb_uploads is enough to make the overall scan non-empty.
+ *
+ * 2. A purge that would remove more than ORPHAN_PURGE_MAX_FRACTION of a
+ *    root's indexed sources (and at least ORPHAN_PURGE_MIN_GUARDED of them) is
+ *    withheld. This catches the root pointing at the wrong place while still
+ *    holding a few files, which guard 1 can't see.
+ *
+ * Both guards trade a missed cleanup for safety: re-embedding a wrongly purged
+ * library takes hours to days, while a stale source costs nothing until the
+ * next sync or an explicit reset.
+ */
+export function decideOrphans(
+  sourcesInQdrant: string[],
+  embeddableFiles: string[],
+  scannedRoots: string[]
+): OrphanDecision {
+  const onDisk = new Set(embeddableFiles)
+  const orphans: string[] = []
+  const withheld: WithheldOrphans[] = []
+
+  for (const root of scannedRoots) {
+    const indexed = filterOrphanCandidates(sourcesInQdrant, [root])
+    const missing = indexed.filter((source) => !onDisk.has(source))
+    if (missing.length === 0) continue
+
+    if (filterOrphanCandidates(embeddableFiles, [root]).length === 0) {
+      withheld.push({ root, count: missing.length, reason: 'empty_root' })
+      continue
+    }
+
+    if (
+      missing.length >= ORPHAN_PURGE_MIN_GUARDED &&
+      missing.length / indexed.length > ORPHAN_PURGE_MAX_FRACTION
+    ) {
+      withheld.push({ root, count: missing.length, reason: 'mass_removal' })
+      continue
+    }
+
+    orphans.push(...missing)
+  }
+
+  return { orphans, withheld }
+}
+
+/**
+ * Narrows paths down to the ones under a root the disk scan genuinely walked.
+ * decideOrphans() applies it per root, both to Qdrant sources and to the files
+ * found on disk.
+ *
+ * `scannedRoots` is deliberately the roots that were *walked*, not the roots
+ * that were *configured*. Two different failure modes collapse into that one
+ * rule:
+ *
+ * 1. A source root outside the scan entirely — Nomad's own bundled docs
+ *    (README.md + docs/), or any root added in future. Allowlisting rather
+ *    than denylisting those by name means a new root doesn't get treated as
+ *    orphaned the first time it appears (see #1170's docs-collision near-miss).
+ *
+ * 2. A configured root that wasn't there at scan time. `_discoverKbFiles()`
+ *    skips a missing root rather than failing, so a relocated or not-yet-
+ *    mounted zim directory (#1050) still leaves the scan non-empty via
+ *    kb_uploads. A root we couldn't read tells us nothing about what belongs
+ *    under it. (A root that exists but is empty is the same problem one step
+ *    removed; decideOrphans' per-root guard handles that, see #1378.)
+ *
+ * Passing an empty `scannedRoots` therefore yields no candidates, which is the
+ * correct reading of "we couldn't see any of the storage."
+ */
+export function filterOrphanCandidates(
+  sourcesInQdrant: string[],
+  scannedRoots: string[]
+): string[] {
+  // The trailing separator is what makes this a subpath test rather than a
+  // naive string prefix match, so `<root>-backup/x.zim` doesn't qualify.
+  const prefixes = scannedRoots.map((root) => root + sep)
+  if (prefixes.length === 0) return []
+  return sourcesInQdrant.filter((source) => prefixes.some((prefix) => source.startsWith(prefix)))
+}

admin/app/utils/misc.ts

@@ -12,6 +12,30 @@ export function toTitleCase(str: string): string {
     .join(' ')
 }
 
+/**
+ * Decide which model runs an ancillary AI task (chat titles, chat suggestions).
+ *
+ * `configured` is the user's `ai.tasksModel` setting. It only wins when the
+ * model is still installed — a model can be deleted from /settings/models long
+ * after it was picked here, and a request for a missing model 404s. In every
+ * other case the caller's existing `fallback` applies, so an unset setting
+ * leaves prior behaviour untouched.
+ */
+export function pickTasksModel(
+  configured: string | null | undefined,
+  installedNames: string[],
+  fallback: string | null
+): { model: string | null; staleConfigured: string | null } {
+  const trimmed = configured?.trim()
+  if (!trimmed) {
+    return { model: fallback, staleConfigured: null }
+  }
+  if (installedNames.includes(trimmed)) {
+    return { model: trimmed, staleConfigured: null }
+  }
+  return { model: fallback, staleConfigured: trimmed }
+}
+
 export function parseBoolean(value: any): boolean {
   if (typeof value === 'boolean') return value
   if (typeof value === 'string') {
@@ -22,4 +46,48 @@ export function parseBoolean(value: any): boolean {
     return value === 1
   }
   return false
-}
+}
+
+/**
+ * Interpret a stored `rag.minRelevance` value as a retrieval relevance floor.
+ *
+ * Pure, and separate from the cached KV read in rag_relevance.ts, for the same
+ * reason pickTasksModel is separate from resolveTasksModel: every interesting
+ * case here is a bad or absent value, and none of them need a database.
+ *
+ * Unset, empty, 'auto' and unparseable all mean "use the recommended default" —
+ * a corrupt row must not silently disable filtering. An explicit 0 is different:
+ * it is a real choice, and it turns the floor off.
+ */
+export function parseMinRelevance(raw: string | null | undefined, fallback: number): number {
+  if (raw === null || raw === undefined) return fallback
+  const trimmed = String(raw).trim()
+  if (trimmed === '' || trimmed === 'auto') return fallback
+  const parsed = Number(trimmed)
+  if (!Number.isFinite(parsed)) return fallback
+  return Math.min(1, Math.max(0, parsed))
+}
+
+/**
+ * Apply the retrieval relevance floor to a reranked candidate list.
+ *
+ * Pure, and called from RagService.searchSimilarDocuments between reranking and
+ * the source-diversity penalty. The ordering is the interesting part: the floor
+ * is a judgement about *relevance*, and the reranked score is where that
+ * judgement is best informed. Diversity is about *redundancy* — it multiplies by
+ * 0.85^n per repeated source — so flooring afterwards would drop the fourth
+ * chunk of the one document that actually answers the question and blame a knob
+ * labelled "relevance" for it.
+ *
+ * An empty result is a real answer, not a failure: it means nothing retrieved
+ * was relevant enough, and the caller should inject no context block at all
+ * rather than hand the model passages it has to be talked out of using.
+ */
+export function applyRelevanceFloor<T extends { finalScore: number }>(
+  results: T[],
+  minFinalScore: number
+): { survivors: T[]; belowFloor: number } {
+  if (!(minFinalScore > 0)) return { survivors: results, belowFloor: 0 }
+  const survivors = results.filter((r) => r.finalScore >= minFinalScore)
+  return { survivors, belowFloor: results.length - survivors.length }
+}

admin/app/utils/model_capabilities.ts

@@ -0,0 +1,90 @@
+import type { ModelVisionCapability } from '../../types/ollama.js'
+
+export type ModelCapabilities = {
+  thinking: boolean
+  vision: ModelVisionCapability
+}
+
+export type ModelCapabilityProbes = {
+  ollamaShow: () => Promise<unknown>
+  llamaProps: () => Promise<unknown>
+}
+
+export type InstalledModelMetadata = {
+  name: string
+  size: number
+  capabilities?: string[]
+}
+
+export function capabilitiesFromOllamaShow(value: unknown): ModelCapabilities | null {
+  if (!value || typeof value !== 'object') return null
+  const capabilities = (value as { capabilities?: unknown }).capabilities
+  if (!Array.isArray(capabilities) || !capabilities.every((item) => typeof item === 'string')) {
+    return null
+  }
+
+  return {
+    thinking: capabilities.includes('thinking'),
+    vision: capabilities.includes('vision') ? 'supported' : 'unsupported',
+  }
+}
+
+export function visionCapabilityFromLlamaProps(value: unknown): ModelVisionCapability | null {
+  if (!value || typeof value !== 'object') return null
+  const modalities = (value as { modalities?: unknown }).modalities
+  if (modalities && typeof modalities === 'object' && !Array.isArray(modalities)) {
+    const vision = (modalities as { vision?: unknown }).vision
+    return typeof vision === 'boolean' ? (vision ? 'supported' : 'unsupported') : null
+  }
+  if (Array.isArray(modalities) && modalities.every((item) => typeof item === 'string')) {
+    return modalities.includes('image') || modalities.includes('vision')
+      ? 'supported'
+      : 'unsupported'
+  }
+  return null
+}
+
+export async function resolveModelCapabilities(
+  advertisedMetadata: unknown,
+  probes: ModelCapabilityProbes
+): Promise<ModelCapabilities | null> {
+  const advertised = capabilitiesFromOllamaShow(advertisedMetadata)
+  // A model list that reports vision is trusted outright. One that does not is
+  // not conclusive: Ollama's /api/tags lists gemma3 as ['completion'] while its
+  // /api/show for the same model reports ['completion', 'vision'], which left
+  // image upload disabled for one of the most common small vision models.
+  if (advertised?.vision === 'supported') return advertised
+
+  try {
+    const ollama = capabilitiesFromOllamaShow(await probes.ollamaShow())
+    if (ollama) return ollama
+  } catch {}
+
+  try {
+    const vision = visionCapabilityFromLlamaProps(await probes.llamaProps())
+    if (vision) return { thinking: advertised?.thinking ?? false, vision }
+  } catch {}
+
+  return advertised
+}
+
+export function installedModelsFromOpenAIResponse(value: unknown): InstalledModelMetadata[] {
+  if (!value || typeof value !== 'object') return []
+  const data = (value as { data?: unknown }).data
+  if (!Array.isArray(data)) return []
+
+  return data.flatMap((entry) => {
+    if (!entry || typeof entry !== 'object') return []
+    const { id, capabilities } = entry as { id?: unknown; capabilities?: unknown }
+    if (typeof id !== 'string') return []
+
+    const model: InstalledModelMetadata = { name: id, size: 0 }
+    if (
+      Array.isArray(capabilities) &&
+      capabilities.every((capability) => typeof capability === 'string')
+    ) {
+      model.capabilities = capabilities
+    }
+    return [model]
+  })
+}

admin/app/utils/queue_stall_options.ts

@@ -0,0 +1,68 @@
+/**
+ * Per-queue BullMQ stall-recovery options.
+ *
+ * The pure, dependency-free core of `getStallOptionsForQueue` in
+ * `commands/queue/work.ts` (mirrors `decideScanAction` in
+ * `kb_ingest_decision.ts`).
+ *
+ * Two questions decide a queue's entry, and they pull in opposite directions:
+ *
+ * 1. **How long does one job legitimately hold the lock?** If that can exceed
+ *    `lockDuration`, BullMQ calls a perfectly healthy job stalled.
+ * 2. **Is re-running a job safe?** On a stall BullMQ increments the job's `stc`
+ *    counter and re-queues it, failing it only once the counter passes
+ *    `maxStalledCount` (`moveStalledJobsToWait`). It does **not** check whether
+ *    the original is still running. For an idempotent job that recovery is what
+ *    we want; for a job that writes non-idempotent data it is worse than the
+ *    failure it is preventing.
+ *
+ * Note that `maxStalledCount: 0` is meaningful and is not the same as leaving
+ * it unset: the comparison is `stalledCount > maxStalledCount` after an
+ * increment, so 0 fails the job on the FIRST stall rather than re-queueing it,
+ * while the BullMQ default of 1 re-queues once.
+ */
+
+export const DEFAULT_LOCK_DURATION = 300_000
+export const LONG_LOCK_DURATION = 1_800_000
+
+export interface StallOptions {
+  lockDuration: number
+  /** Left undefined to keep BullMQ's default of 1. */
+  maxStalledCount?: number
+}
+
+export interface StallQueueNames {
+  /** ZIM/document embedding — a self-continuing batch chain. */
+  embedFile: string
+  /** Content downloads — resumable. */
+  download: string
+  drugDownload: string
+  drugIngest: string
+}
+
+export function stallOptionsForQueue(queueName: string, queues: StallQueueNames): StallOptions {
+  // Long single streams (a ~150 MB resumable HTTP pull, then an unzip +
+  // JSON-stream ingest at concurrency 1). Re-running a part is safe: the
+  // download resumes and the ingest upserts by key.
+  if (queueName === queues.drugDownload || queueName === queues.drugIngest) {
+    return { lockDuration: LONG_LOCK_DURATION, maxStalledCount: 3 }
+  }
+
+  // NEVER auto-recover a stalled job on this queue (#1269). Embedding is a
+  // self-continuing chain, so re-queueing while the original is alive produces
+  // two chains walking the same file, both writing to Qdrant with freshly
+  // minted point ids. A failed chain is visible and re-indexable; a forked one
+  // is silent and corrupts the index.
+  if (queueName === queues.embedFile) {
+    return { lockDuration: LONG_LOCK_DURATION, maxStalledCount: 0 }
+  }
+
+  // The opposite call to embeddings (#1203). A download is resumable and
+  // re-running one is harmless, but the default maxStalledCount of 1 fails the
+  // job on a second stall and bypasses its own `attempts: 10` retry policy.
+  if (queueName === queues.download) {
+    return { lockDuration: LONG_LOCK_DURATION, maxStalledCount: 3 }
+  }
+
+  return { lockDuration: DEFAULT_LOCK_DURATION }
+}

admin/app/utils/rag_prompt.ts

@@ -0,0 +1,111 @@
+/**
+ * Pure helpers for shaping the retrieved-context block.
+ *
+ * These live here rather than in RagPipelineService so they can be exercised
+ * under bare `node --experimental-strip-types` with no MySQL, Redis, Qdrant, or
+ * Ollama — the same reason `kb_ratio_lookup.ts` is shaped this way. The service
+ * supplies the config; everything below is a function of its arguments.
+ *
+ * Token budgeting used to live here too, as a chars-per-token estimate feeding a
+ * `num_ctx` ladder. Both are gone: the estimate now comes from
+ * `token_estimate.ts` (calibrated against real token counts) and the allocation
+ * from `context_budget.ts`. The ladder in particular was dead code — it fed a
+ * `num_ctx` that the OpenAI-compatible endpoint silently discarded, so no chat
+ * ever ran at the window it computed.
+ */
+
+import { parseParameterBillions } from './context_window.js'
+import type { ChatSource } from '../../types/chat.js'
+
+export type ContextLimits = { maxResults: number; maxTokens: number }
+export type ContextLimitTier = { maxParams: number; maxResults: number; maxTokens: number }
+
+/** The minimum shape these helpers need; the real chunks carry more. */
+export type BudgetableChunk = { text: string; metadata?: Record<string, any> }
+export type BudgetableMessage = { role: 'system' | 'user' | 'assistant'; content: string }
+
+/**
+ * Determine RAG context limits from the model's parameter count.
+ *
+ * `parameterSize` is Ollama's own `details.parameter_size` ("8.0B") and is the
+ * honest source. The model name is only a fallback: it used to be the *only*
+ * source, which meant a tag carrying no size — "phi3", "mistral-nemo", anything
+ * custom — was silently assumed to be 8B and handed far more context than a
+ * small model can actually use. When neither source knows, 8B remains the
+ * assumption, but now it is one made explicitly rather than by a failed regex.
+ */
+export function getContextLimitsForModel(
+  modelName: string,
+  tiers: readonly ContextLimitTier[],
+  parameterSize?: string
+): ContextLimits {
+  const paramBillions = parseParameterBillions(parameterSize, modelName) ?? 8
+
+  for (const tier of tiers) {
+    if (paramBillions <= tier.maxParams) {
+      return { maxResults: tier.maxResults, maxTokens: tier.maxTokens }
+    }
+  }
+
+  return { maxResults: 5, maxTokens: 0 }
+}
+
+/**
+ * Render retrieved chunks into the block the rag_context prompt wraps.
+ *
+ * Each block is labelled with its source title when one is available — a
+ * neutral, honest provenance signal — but never with the raw relevance score.
+ * nomic cosine scores for genuinely relevant passages sit around 0.4-0.6, and
+ * surfacing e.g. "42%" primes the model to distrust correct context. Scores
+ * stay in the logs and in the eval report.
+ */
+export function buildContextBlock(docs: BudgetableChunk[]): string {
+  return docs
+    .map((doc, idx) => {
+      const title =
+        doc.metadata?.archive_title || doc.metadata?.full_title || doc.metadata?.article_title
+      const date = doc.metadata?.archive_date
+      const label = title
+        ? `[Context ${idx + 1} — ${title}${date ? ` (${date})` : ''}]`
+        : `[Context ${idx + 1}]`
+      return `${label}\n${doc.text}`
+    })
+    .join('\n\n')
+}
+
+/**
+ * Build the deduplicated "Sources" list shown under an assistant answer.
+ *
+ * Deliberately fed from what was *injected* into the prompt rather than from
+ * everything retrieval returned: a chunk dropped by the budget planner never
+ * reached the model, so citing it would credit the answer to a document it was
+ * not based on -- the exact failure the citation list exists to prevent.
+ *
+ * Dedupes on the originating path so a dozen chunks out of one archive collapse
+ * to one entry, falling back to the title when a point carries no path. Points
+ * with neither are skipped rather than shown as "Unknown source": an entry the
+ * user cannot act on is worse than one fewer entry.
+ */
+export function buildCitations(docs: BudgetableChunk[]): ChatSource[] {
+  const seen = new Set<string>()
+  const sources: ChatSource[] = []
+
+  for (const doc of docs) {
+    const title =
+      doc.metadata?.archive_title || doc.metadata?.full_title || doc.metadata?.article_title
+    const path = doc.metadata?.source as string | undefined
+    const key = path || title
+    if (!key || seen.has(key)) continue
+    seen.add(key)
+
+    sources.push({
+      // A user-uploaded PDF carries no embedded title, so fall back to its
+      // filename -- which is what the user named it and will recognise.
+      title: title || path!.split('/').pop() || path!,
+      date: doc.metadata?.archive_date,
+      source: path,
+    })
+  }
+
+  return sources
+}

admin/app/utils/rag_relevance.ts

@@ -0,0 +1,49 @@
+import logger from '@adonisjs/core/services/logger'
+import KVStore from '#models/kv_store'
+import { RAG_MIN_FINAL_SCORE } from '../../constants/ollama.js'
+import { parseMinRelevance } from './misc.js'
+
+/**
+ * Resolves the user's retrieval relevance floor (`rag.minRelevance`), cached.
+ *
+ * Read on every chat turn, changed roughly never, so a KV round-trip per turn
+ * would be pure overhead on the critical path. Cached at module scope rather
+ * than on a service instance: NOMAD's services are `@inject()` but are not
+ * registered as singletons, so instance state lives and dies with a single
+ * request — which is why ContextWindowService's "memoized for the process
+ * lifetime" memo isn't, and why its invalidate() never had a caller to matter to.
+ *
+ * Belt and braces, mirroring the assistant-name cache in config/inertia.ts: the
+ * TTL bounds how stale this can get if an invalidation is ever missed, and
+ * SystemService.updateSetting calls invalidateMinRelevanceCache() so the common
+ * case takes effect immediately.
+ */
+const CACHE_TTL_MS = 60_000
+
+let cache: { value: number; expiresAt: number } | null = null
+
+export function invalidateMinRelevanceCache(): void {
+  cache = null
+}
+
+/** The relevance floor to apply to this turn's retrieval. */
+export async function resolveMinFinalScore(): Promise<number> {
+  const now = Date.now()
+  if (cache && now < cache.expiresAt) return cache.value
+
+  let raw: string | null = null
+  try {
+    raw = await KVStore.getValue('rag.minRelevance')
+  } catch (error) {
+    // A KV read failure must not take retrieval out, and must not be cached —
+    // fall back to the default and let the next turn try again.
+    logger.warn(
+      `[RAG] Failed to read rag.minRelevance, using ${RAG_MIN_FINAL_SCORE}: ${error instanceof Error ? error.message : error}`
+    )
+    return RAG_MIN_FINAL_SCORE
+  }
+
+  const value = parseMinRelevance(raw, RAG_MIN_FINAL_SCORE)
+  cache = { value, expiresAt: now + CACHE_TTL_MS }
+  return value
+}

admin/app/utils/relevance_judge.ts

@@ -0,0 +1,107 @@
+/**
+ * The pure half of the retrieval relevance check (#1341): the grammar, the
+ * setting, and what the judge gets to read.
+ *
+ * Why a model and not another threshold: on a library holding general prose
+ * (Gutenberg, Wikipedia), the nearest chunk to *any* question scores as high as
+ * a genuinely relevant one. Measured on the eval corpus with its distractors,
+ * the weakest correct top hit scored 0.666 post-rerank while "write me a haiku
+ * about autumn leaves" pulled Walden at 0.705. No cutoff on that axis separates
+ * the two, so the decision has to be a judgement about the question.
+ *
+ * Why one verdict for the whole set rather than one per passage: asked
+ * passage-by-passage, every model tried (0.5B to 8B) rejected passages that
+ * were on-topic but did not spell the answer out, and cost 9-17 points of
+ * recall@5. "Are these about what was asked?" is the easier question, and on an
+ * 8B model it took off-topic leaks from 67% to 6% for ~5 points of recall. On a
+ * 3B model it rejects most real questions, which is why the check is opt-in.
+ *
+ * Pure so it unit-tests under bare node; RelevanceJudgeService owns the call.
+ */
+
+/**
+ * Longest excerpt of each passage the judge reads. At 700 characters the window
+ * routinely stopped a sentence short of the answer (wound packing, counterpoise
+ * length) and the judge rightly said the passage didn't cover it.
+ */
+export const JUDGE_EXCERPT_CHARS = 1500
+
+/** Anything shorter carries no topical signal; don't let it steer the excerpt. */
+const MIN_TERM_LENGTH = 4
+
+export type JudgeablePassage = { text: string; title?: string }
+
+export const ON_TOPIC_SCHEMA = {
+  type: 'object',
+  properties: { on_topic: { type: 'boolean' } },
+  required: ['on_topic'],
+  additionalProperties: false,
+} as const
+
+export function pickOnTopic(value: any): boolean | null {
+  return typeof value?.on_topic === 'boolean' ? value.on_topic : null
+}
+
+/**
+ * Interpret the stored `rag.relevanceCheck` setting. Off unless explicitly on:
+ * the check costs a model call per turn and hurts recall on small models, so
+ * nothing short of a deliberate choice should turn it on.
+ */
+export function isRelevanceCheckEnabled(raw: unknown): boolean {
+  return raw === true || raw === 'true'
+}
+
+/**
+ * The stretch of `text` most about the question, at most `maxChars` long.
+ *
+ * Chunks run ~3000 characters and the judge sees half of each, so which half
+ * matters. Picks the window holding the most occurrences of the question's
+ * longer terms, falling back to the start when none appear.
+ */
+export function excerptFor(text: string, question: string, maxChars = JUDGE_EXCERPT_CHARS): string {
+  const clean = text.replace(/\s+/g, ' ').trim()
+  if (clean.length <= maxChars) return clean
+
+  const terms = [
+    ...new Set(
+      question
+        .toLowerCase()
+        .split(/[^\p{L}\p{N}]+/u)
+        .filter((t) => t.length >= MIN_TERM_LENGTH)
+    ),
+  ]
+  const lower = clean.toLowerCase()
+  const hits: number[] = []
+  for (const term of terms) {
+    for (let at = lower.indexOf(term); at !== -1; at = lower.indexOf(term, at + term.length)) {
+      hits.push(at)
+    }
+  }
+  if (hits.length === 0) return clean.slice(0, maxChars) + '…'
+  hits.sort((a, b) => a - b)
+
+  // Densest window: two pointers over the sorted hit positions.
+  let bestStart = hits[0]
+  let bestCount = 0
+  for (let lo = 0, hi = 0; hi < hits.length; hi++) {
+    while (hits[hi] - hits[lo] > maxChars) lo++
+    if (hi - lo + 1 > bestCount) {
+      bestCount = hi - lo + 1
+      bestStart = hits[lo]
+    }
+  }
+
+  // Open a little before the first hit so the sentence it sits in reads whole.
+  const start = Math.max(0, Math.min(bestStart - Math.floor(maxChars / 5), clean.length - maxChars))
+  const end = start + maxChars
+  return (start > 0 ? '…' : '') + clean.slice(start, end) + (end < clean.length ? '…' : '')
+}
+
+/** The user turn of the judge call: the question, then each passage. */
+export function buildJudgeInput(question: string, passages: JudgeablePassage[]): string {
+  const blocks = passages.map((p, i) => {
+    const heading = p.title ? `p${i + 1} (from "${p.title}")` : `p${i + 1}`
+    return `${heading}:\n${excerptFor(p.text, question)}`
+  })
+  return `Question: ${question.trim()}\n\n${blocks.join('\n\n')}`
+}

admin/app/utils/response_style.ts

@@ -0,0 +1,47 @@
+import logger from '@adonisjs/core/services/logger'
+import KVStore from '#models/kv_store'
+import { DEFAULT_RESPONSE_STYLE } from '../../constants/ollama.js'
+import type { ResponseStyle } from '../../types/ollama.js'
+import { parseResponseStyle } from './sampler.js'
+
+/**
+ * Resolves the user's Response Style (`ai.responseStyle`), cached.
+ *
+ * Same shape and same reasoning as the relevance-floor cache next door: read on
+ * every chat turn, changed roughly never, and NOMAD's services are `@inject()`
+ * but not registered as singletons, so caching on a service instance would give
+ * a cache that lives and dies inside one request.
+ *
+ * The TTL bounds how stale this gets if an invalidation is ever missed;
+ * SystemService.updateSetting calls invalidateResponseStyleCache() so the common
+ * case takes effect on the next turn rather than a minute later.
+ */
+const CACHE_TTL_MS = 60_000
+
+let cache: { value: ResponseStyle; expiresAt: number } | null = null
+
+export function invalidateResponseStyleCache(): void {
+  cache = null
+}
+
+/** The response style to sample this turn under. */
+export async function resolveResponseStyle(): Promise<ResponseStyle> {
+  const now = Date.now()
+  if (cache && now < cache.expiresAt) return cache.value
+
+  let raw: string | null = null
+  try {
+    raw = await KVStore.getValue('ai.responseStyle')
+  } catch (error) {
+    // A KV read failure must not take chat out, and must not be cached — fall
+    // back to the default and let the next turn try again.
+    logger.warn(
+      `[Ollama] Failed to read ai.responseStyle, using ${DEFAULT_RESPONSE_STYLE}: ${error instanceof Error ? error.message : error}`
+    )
+    return DEFAULT_RESPONSE_STYLE
+  }
+
+  const value = parseResponseStyle(raw)
+  cache = { value, expiresAt: now + CACHE_TTL_MS }
+  return value
+}

admin/app/utils/sampler.ts

@@ -0,0 +1,133 @@
+/**
+ * Pure helpers for deciding which sampler settings a chat request carries.
+ *
+ * Kept out of OllamaService for the same reason `context_window.ts` is: every
+ * interesting case here is a bad stored value or a model that declares its own
+ * parameters, and none of them need Ollama, Docker or MySQL to exercise. The
+ * service supplies the facts (what the stored setting says, what /api/show
+ * returned); everything here is a function of its arguments.
+ */
+import { DEFAULT_RESPONSE_STYLE, SAMPLER_PRESETS } from '../../constants/ollama.js'
+import type { ResponseStyle, SamplerProfile } from '../../types/ollama.js'
+
+const RESPONSE_STYLES: ResponseStyle[] = ['auto', 'focused', 'creative', 'off']
+
+/**
+ * Interpret a stored `ai.responseStyle` value.
+ *
+ * Unset, empty and unrecognised all mean the default. A corrupt row must not
+ * silently drop chat back to unsampled backend defaults, because that is the
+ * failure this setting exists to fix and it would be invisible from the UI.
+ * `off` is only ever reached by someone choosing it.
+ */
+export function parseResponseStyle(raw: string | null | undefined): ResponseStyle {
+  if (raw === null || raw === undefined) return DEFAULT_RESPONSE_STYLE
+  const trimmed = String(raw).trim().toLowerCase()
+  if (!trimmed) return DEFAULT_RESPONSE_STYLE
+  return (RESPONSE_STYLES as string[]).includes(trimmed)
+    ? (trimmed as ResponseStyle)
+    : DEFAULT_RESPONSE_STYLE
+}
+
+/**
+ * Pull sampler values out of the `parameters` string /api/show returns (the
+ * modelfile PARAMETER lines, e.g. `temperature                    0.6`).
+ *
+ * Sibling of `readModelfileNumCtx`, and there for a sharper reason than it: a
+ * good number of Ollama library models ship author-tuned samplers (qwen3 sets
+ * 0.6/0.95/20, deepseek-r1 sets 0.6), and stamping a generic house profile over
+ * those would make exactly the models someone tuned carefully worse. Under the
+ * `auto` style these values win, key by key.
+ *
+ * Repeated PARAMETER lines are legal in a modelfile and the last one wins in
+ * Ollama, so the scan does not stop at the first match.
+ */
+export function readModelfileSamplers(parameters: unknown): SamplerProfile {
+  const profile: SamplerProfile = {}
+  if (typeof parameters !== 'string') return profile
+
+  const keys: [RegExp, keyof SamplerProfile][] = [
+    [/^\s*temperature\s+(-?[\d.]+)\s*$/gm, 'temperature'],
+    [/^\s*top_p\s+(-?[\d.]+)\s*$/gm, 'topP'],
+    [/^\s*top_k\s+(-?[\d.]+)\s*$/gm, 'topK'],
+    [/^\s*min_p\s+(-?[\d.]+)\s*$/gm, 'minP'],
+    [/^\s*repeat_penalty\s+(-?[\d.]+)\s*$/gm, 'repeatPenalty'],
+  ]
+
+  for (const [pattern, field] of keys) {
+    let match: RegExpExecArray | null
+    while ((match = pattern.exec(parameters)) !== null) {
+      const value = Number.parseFloat(match[1])
+      if (Number.isFinite(value) && value >= 0) profile[field] = value
+    }
+  }
+  return profile
+}
+
+/**
+ * The sampler settings to send for one request.
+ *
+ * `off` sends nothing. `auto` is NOMAD's baseline with the model author's own
+ * values layered over it, which in practice means the model keeps its tuned
+ * temperature and top_p and gains the min_p it was missing. `focused` and
+ * `creative` are explicit user choices and replace the model's values outright,
+ * because a style the user picked by name that quietly did nothing on half
+ * their models would be worse than not offering it.
+ *
+ * A model that declares its own `top_p` under `auto` also gets it on the compat
+ * transport: an author-set nucleus is a better substitute for the missing min_p
+ * than the preset's generic fallback.
+ */
+export function resolveSamplerProfile(
+  style: ResponseStyle,
+  modelfileSamplers?: SamplerProfile
+): SamplerProfile | undefined {
+  if (style === 'off') return undefined
+
+  const preset = SAMPLER_PRESETS[style]
+  if (style !== 'auto') return { ...preset }
+
+  const declared = modelfileSamplers ?? {}
+  const merged: SamplerProfile = { ...preset, ...declared }
+  if (declared.topP !== undefined) merged.compatTopP = declared.topP
+  return merged
+}
+
+/**
+ * Map a profile onto Ollama's native `options` keys.
+ *
+ * `temperature` is deliberately absent: it is carried on ChatInput itself, so an
+ * explicit caller value (eval runs and the grammar-constrained task calls, both
+ * of which pin it to 0) keeps beating whatever style the user has chosen.
+ */
+export function nativeSamplerOptions(profile: SamplerProfile | undefined): Record<string, number> {
+  if (!profile) return {}
+  const options: Record<string, number> = {}
+  if (profile.topP !== undefined) options.top_p = profile.topP
+  if (profile.topK !== undefined) options.top_k = profile.topK
+  if (profile.minP !== undefined) options.min_p = profile.minP
+  if (profile.repeatPenalty !== undefined) options.repeat_penalty = profile.repeatPenalty
+  return options
+}
+
+/**
+ * Map a profile onto the OpenAI-compatible endpoint's params.
+ *
+ * Only `top_p` survives the trip. min_p, top_k and repeat_penalty have no
+ * equivalent there, and presence/frequency_penalty are not stand-ins for
+ * repeat_penalty — they accumulate over the whole completion rather than a
+ * 64-token window, so borrowing one to fake the other would punish a grounded
+ * answer for citing the same identifier twice.
+ *
+ * `compatTopP` rather than `topP` is the value sent, for the reason documented
+ * on the type: the presets set `topP: 1.0` and truncate with min_p, and passing
+ * that through unchanged to a backend that drops min_p would leave the raw
+ * distribution untruncated, which is worse than the default it replaced.
+ */
+export function compatSamplerParams(profile: SamplerProfile | undefined): Record<string, number> {
+  if (!profile) return {}
+  const params: Record<string, number> = {}
+  const topP = profile.compatTopP ?? profile.topP
+  if (topP !== undefined) params.top_p = topP
+  return params
+}

admin/app/utils/service_catalog_merge.ts

@@ -0,0 +1,152 @@
+/**
+ * Merging catalog config onto an already-installed curated service.
+ *
+ * `ServiceSeeder.run()` keeps curated services in sync with the catalog by
+ * overwriting `container_config` and `ui_location` on every boot. That is correct
+ * for anything not yet installed, and wrong for anything that is: an install whose
+ * published host port differs from the catalog default gets its port reset, which
+ * desyncs the row from the container that is actually running. The Open link then
+ * points at a port with nothing behind it, and the next recreate collides with
+ * whatever process took the catalog port in the meantime.
+ *
+ * A host port diverges for an ordinary reason: the default was already taken on
+ * that machine, so the user moved it. That is a property of the machine, not a
+ * preference the catalog should get a vote on.
+ *
+ * These helpers keep the running install's host ports while still applying every
+ * other catalog change, so an app on an alternate port continues to receive image,
+ * env, metadata and scheme updates. The alternative considered was flagging such
+ * rows `is_user_modified` so the sync skips them entirely, which also protects the
+ * port but permanently opts that install out of all future catalog updates. This
+ * is the narrower fix: preserve the one field the machine owns, sync the rest.
+ */
+
+/** `{ "8080/tcp": [{ "HostPort": "8090" }] }` keyed by container port. */
+type PortBindings = Record<string, Array<{ HostPort?: string }> | undefined>
+
+/**
+ * `services.container_config` is a MySQL **json** column, so the driver hands back
+ * an already-parsed object at runtime even though the model declares it
+ * `string | null`. The catalog side of the comparison is a `JSON.stringify` string.
+ * Both shapes reach these helpers, so accept either.
+ *
+ * This is not a hypothetical: a string-only version of this function silently
+ * no-opped against a real database while passing every string-fixture unit test.
+ */
+type SerializedConfig = string | Record<string, any> | null | undefined
+
+function parseConfig(value: SerializedConfig): Record<string, any> | null {
+  if (!value) return null
+  if (typeof value === 'object') return value
+  try {
+    const parsed = JSON.parse(value)
+    return parsed && typeof parsed === 'object' ? parsed : null
+  } catch {
+    // A row we cannot parse gets the catalog value verbatim, which is the
+    // pre-existing behaviour and no worse than what it has now.
+    return null
+  }
+}
+
+function readPortBindings(containerConfig: SerializedConfig): PortBindings | null {
+  const bindings = parseConfig(containerConfig)?.HostConfig?.PortBindings
+  if (!bindings || typeof bindings !== 'object') return null
+  return bindings as PortBindings
+}
+
+function firstHostPort(bindings: PortBindings | null): string | null {
+  if (!bindings) return null
+  for (const list of Object.values(bindings)) {
+    const port = Array.isArray(list) ? list[0]?.HostPort : undefined
+    if (port) return String(port)
+  }
+  return null
+}
+
+/**
+ * Catalog `container_config` with the live install's published host ports kept.
+ *
+ * Matching is per container-side port, so if the catalog changes which port the
+ * app listens on *inside* the container, that is a genuine catalog change and the
+ * catalog value wins. Only the host side of a binding the install already has is
+ * preserved.
+ */
+export function mergeContainerConfigPreservingHostPorts<T extends string | null>(
+  catalogConfig: T,
+  liveConfig: SerializedConfig
+): T {
+  if (!catalogConfig) return catalogConfig
+  const live = readPortBindings(liveConfig)
+  if (!live) return catalogConfig
+
+  const parsed = parseConfig(catalogConfig)
+  if (!parsed) return catalogConfig
+
+  const catalogBindings: PortBindings | undefined = parsed?.HostConfig?.PortBindings
+  if (!catalogBindings || typeof catalogBindings !== 'object') return catalogConfig
+
+  let changed = false
+  for (const [containerPort, catalogList] of Object.entries(catalogBindings)) {
+    const livePort = Array.isArray(live[containerPort]) ? live[containerPort]![0]?.HostPort : undefined
+    if (!livePort) continue
+
+    const catalogPort = Array.isArray(catalogList) ? catalogList[0]?.HostPort : undefined
+    if (catalogPort === undefined || String(catalogPort) === String(livePort)) continue
+
+    catalogBindings[containerPort] = [{ ...(Array.isArray(catalogList) ? catalogList[0] : {}), HostPort: String(livePort) }]
+    changed = true
+  }
+
+  return (changed ? JSON.stringify(parsed) : catalogConfig) as T
+}
+
+/**
+ * Catalog `ui_location` with the live install's port kept, when it is a port at all.
+ *
+ * `ui_location` is one of `"8090"`, `"https:8480"`, or a path like `"/chat"`. Only
+ * the first two forms carry a port, and only those are rewritten. The catalog's
+ * scheme is always taken, so a catalog change like Vaultwarden moving to `https:`
+ * still reaches an install running on an alternate port.
+ *
+ * `mergedConfig` must be the container_config this sync is about to WRITE, meaning
+ * the output of `mergeContainerConfigPreservingHostPorts`, not the live row. The
+ * live port is only honoured when that config actually publishes it, which does
+ * two things: it keeps a stale or hand-edited `ui_location` from pinning the link
+ * to a port nothing is listening on, and it keeps the link in step with the config
+ * in the one case the catalog wins outright. When the catalog moves the
+ * container-side port, no live host port is preserved, so the link takes the
+ * catalog port rather than pointing at a port the recreate will not bind.
+ */
+export function mergeUiLocationPreservingHostPort(
+  catalogUiLocation: string | null,
+  liveUiLocation: string | null | undefined,
+  mergedConfig: SerializedConfig
+): string | null {
+  if (!catalogUiLocation || !liveUiLocation) return catalogUiLocation
+
+  const catalogMatch = catalogUiLocation.match(/^(?:(https?):)?(\d+)$/)
+  const liveMatch = liveUiLocation.match(/^(?:(https?):)?(\d+)$/)
+  if (!catalogMatch || !liveMatch) return catalogUiLocation
+
+  const livePort = liveMatch[2]
+  if (livePort === catalogMatch[2]) return catalogUiLocation
+
+  const bindings = readPortBindings(mergedConfig)
+  const publishedPorts = new Set(
+    bindings
+      ? Object.values(bindings)
+          .flatMap((list) => (Array.isArray(list) ? list : []))
+          .map((b) => (b?.HostPort ? String(b.HostPort) : null))
+          .filter((p): p is string => p !== null)
+      : []
+  )
+  if (!publishedPorts.has(livePort)) return catalogUiLocation
+
+  const scheme = catalogMatch[1]
+  return scheme ? `${scheme}:${livePort}` : livePort
+}
+
+/** Exported for tests: the first published host port in a serialized config. */
+export function firstPublishedHostPort(containerConfig: SerializedConfig): string | null {
+  return firstHostPort(readPortBindings(containerConfig))
+}

admin/app/utils/structured_output.ts

@@ -0,0 +1,158 @@
+/**
+ * JSON Schemas and a tolerant parser for the structured ancillary calls —
+ * chat titles, suggestion chips, and the RAG query rewrite.
+ *
+ * The prompts used to *ask* for a format and the callers recovered the answer by
+ * string surgery: split on comma, else on newline, strip `1.`, strip bullets,
+ * strip quotes. That is a request, not a constraint, and the failure mode was
+ * silent — an empty chip list, or a paragraph as the sidebar title.
+ *
+ * Ollama's `format` parameter compiles a JSON Schema to a GBNF grammar, so the
+ * decoder cannot emit anything else. It only exists on the native /api/chat
+ * transport, so every caller keeps its old parser as the fallback for other
+ * backends; `parseStructured` returning null is what selects that path.
+ *
+ * Pure functions, no framework imports — see `think_stream.ts` / `rag_prompt.ts`.
+ */
+
+/** Sidebar chat title. */
+export const TITLE_SCHEMA = {
+  type: 'object',
+  properties: {
+    title: { type: 'string' },
+  },
+  required: ['title'],
+  additionalProperties: false,
+} as const
+
+/** The three conversation-starter chips on an empty chat. */
+export const SUGGESTIONS_SCHEMA = {
+  type: 'object',
+  properties: {
+    suggestions: {
+      type: 'array',
+      items: { type: 'string' },
+      minItems: 3,
+      maxItems: 3,
+    },
+  },
+  required: ['suggestions'],
+  additionalProperties: false,
+} as const
+
+/**
+ * History-aware query rewrite. An array rather than a single string even though
+ * only the first entry is used today: multi-query / RAG-Fusion (gap analysis 3.2)
+ * then needs a prompt change and nothing else.
+ */
+export const QUERIES_SCHEMA = {
+  type: 'object',
+  properties: {
+    queries: {
+      type: 'array',
+      items: { type: 'string' },
+      minItems: 1,
+    },
+  },
+  required: ['queries'],
+  additionalProperties: false,
+} as const
+
+/**
+ * Pull a JSON object out of a model response and narrow it with `pick`.
+ *
+ * Total by construction — every failure returns null so the caller falls through
+ * to its string parser. This runs on the chat critical path via the rewrite, so
+ * it must never throw.
+ *
+ * Tolerates the things an unconstrained model does anyway: a ```json fence, a
+ * sentence of preamble, trailing commentary. Grammar-constrained output needs
+ * none of that, but the same helper is used on backends where the grammar was
+ * never applied.
+ */
+export function parseStructured<T>(raw: string, pick: (value: any) => T | null): T | null {
+  if (!raw) return null
+
+  // Outermost braces: the first `{` to the last `}`. Preamble and trailing prose
+  // both fall outside it, and a nested object stays inside.
+  const start = raw.indexOf('{')
+  const end = raw.lastIndexOf('}')
+  if (start === -1 || end <= start) return null
+
+  let parsed: any
+  try {
+    parsed = JSON.parse(raw.slice(start, end + 1))
+  } catch {
+    return null
+  }
+  if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) return null
+
+  try {
+    return pick(parsed)
+  } catch {
+    return null
+  }
+}
+
+/**
+ * The two ways a structured call can fail to yield a value.
+ *
+ * `unconstrained` — no grammar reached the model (a non-native backend, or a call
+ * that never asked for one). The response is prose by design and the caller's
+ * legacy string parser is the correct recovery.
+ *
+ * `constrained-parse-failed` — the grammar *was* applied and the output still did
+ * not yield a value. In practice that means a truncated object: the model hit its
+ * token cap mid-JSON. Feeding that to a parser written for prose is what produces
+ * `{"title": ...` in the sidebar or a JSON blob in the embedded query, so the
+ * caller must take its safe path instead.
+ */
+export type StructuredResult<T> =
+  | { ok: true; value: T }
+  | { ok: false; reason: 'constrained-parse-failed' | 'unconstrained' }
+
+/**
+ * `parseStructured` plus the reason it failed, so callers branch on what actually
+ * happened rather than sniffing the raw text for braces.
+ *
+ * `constrained` is `NomadChatResponse.structured` — whether the decoder was really
+ * grammar-constrained for the request, which only the transport knows.
+ *
+ * Total, like `parseStructured`: a picker that throws is a failed parse, not an
+ * exception on the chat critical path.
+ */
+export function resolveStructured<T>(
+  raw: string,
+  pick: (value: any) => T | null,
+  constrained: boolean
+): StructuredResult<T> {
+  const value = parseStructured(raw, pick)
+  if (value !== null) return { ok: true, value }
+  return { ok: false, reason: constrained ? 'constrained-parse-failed' : 'unconstrained' }
+}
+
+/** Non-empty trimmed string, or null. */
+function cleanString(value: unknown): string | null {
+  if (typeof value !== 'string') return null
+  const trimmed = value.trim()
+  return trimmed.length > 0 ? trimmed : null
+}
+
+/** Trimmed, non-empty entries of a string array. Null when nothing survives. */
+function cleanStringArray(value: unknown): string[] | null {
+  if (!Array.isArray(value)) return null
+  const cleaned = value.map(cleanString).filter((s): s is string => s !== null)
+  return cleaned.length > 0 ? cleaned : null
+}
+
+/** `{title: string}` -> the title. */
+export const pickTitle = (value: any): string | null => cleanString(value?.title)
+
+/** `{suggestions: string[]}` -> up to three non-empty suggestions. */
+export const pickSuggestions = (value: any): string[] | null => {
+  const suggestions = cleanStringArray(value?.suggestions)
+  return suggestions ? suggestions.slice(0, 3) : null
+}
+
+/** `{queries: string[]}` -> every non-empty query, best first. */
+export const pickQueries = (value: any): string[] | null => cleanStringArray(value?.queries)

admin/app/utils/tasks_model.ts

@@ -0,0 +1,64 @@
+import logger from '@adonisjs/core/services/logger'
+import KVStore from '#models/kv_store'
+import type { OllamaService } from '#services/ollama_service'
+import { pickTasksModel } from './misc.js'
+
+/**
+ * Resolve the model to use for ancillary work — chat titles, suggestions, and
+ * the per-turn retrieval query rewrite.
+ *
+ * Lifted out of ChatService so the RAG pipeline can use it too. The rewrite is
+ * the ancillary call that matters most and was the one still running on the main
+ * chat model: it happens on *every* turn, on the critical path, and it sends a
+ * completely different prompt. With Ollama's default OLLAMA_NUM_PARALLEL=1 there
+ * is a single slot, so that call evicted the chat model's cached prefix every
+ * turn and guaranteed a full re-prefill of the conversation. Routing it to a
+ * small dedicated model keeps the chat model's KV cache intact.
+ *
+ * Falls back to `fallback` when the setting is unset (preserving the previous
+ * behaviour) or when the configured model is no longer installed. `installed` is
+ * passed by callers that already listed models, to avoid a second round-trip.
+ */
+export async function resolveTasksModel(
+  ollamaService: OllamaService,
+  fallback: string | null,
+  installed?: { name: string }[],
+  logPrefix = '[TasksModel]'
+): Promise<string | null> {
+  let configured: string | null = null
+  try {
+    configured = await KVStore.getValue('ai.tasksModel')
+  } catch (error) {
+    logger.error(
+      `${logPrefix} Failed to read ai.tasksModel: ${error instanceof Error ? error.message : error}`
+    )
+    return fallback
+  }
+  if (!configured?.trim()) {
+    return fallback
+  }
+
+  let models = installed
+  if (!models) {
+    try {
+      models = await ollamaService.getModels()
+    } catch (error) {
+      logger.error(
+        `${logPrefix} Failed to list models while resolving the tasks model: ${error instanceof Error ? error.message : error}`
+      )
+      return fallback
+    }
+  }
+
+  const { model, staleConfigured } = pickTasksModel(
+    configured,
+    (models ?? []).map((m) => m.name),
+    fallback
+  )
+  if (staleConfigured) {
+    logger.warn(
+      `${logPrefix} Configured tasks model "${staleConfigured}" is not installed; falling back to "${fallback ?? 'none'}"`
+    )
+  }
+  return model
+}

admin/app/utils/think_stream.ts

@@ -0,0 +1,122 @@
+/**
+ * Stateful splitter for inline `<think>...</think>` reasoning tags.
+ *
+ * Ollama's native `/api/chat` and its `/v1` shim both surface reasoning as a
+ * dedicated field (`message.thinking` / `delta.reasoning`), but other
+ * OpenAI-compatible backends (LM Studio, llama.cpp, plain GGUF servers) emit
+ * the tags inline in the content stream. This splits them back out so the UI
+ * gets a clean `content` channel and a separate `thinking` channel regardless
+ * of which backend answered.
+ *
+ * Lives here rather than inside OllamaService because both transports need it
+ * and because it is the only real logic in the streaming path — as a pure
+ * class it can be exercised under bare `node --test` with no Ollama, the same
+ * reason `rag_prompt.ts` is shaped this way.
+ *
+ * The hard part is that a tag can be split across chunk boundaries: a chunk may
+ * end with `"<thi"`. Any trailing text that could still turn out to be the start
+ * of a tag is held back in the buffer and re-examined when the next chunk lands.
+ */
+
+const OPEN_TAG = '<think>'
+const CLOSE_TAG = '</think>'
+
+/**
+ * How many trailing characters of `text` could be the beginning of `tag`.
+ * Returns 0 when no suffix of `text` is a prefix of `tag`.
+ */
+export function partialTagSuffix(tag: string, text: string): number {
+  for (let len = Math.min(tag.length - 1, text.length); len >= 1; len--) {
+    if (text.endsWith(tag.slice(0, len))) return len
+  }
+  return 0
+}
+
+export type ThinkSplit = { content: string; thinking: string }
+
+export class ThinkTagSplitter {
+  private buffer = ''
+  private inThink = false
+
+  /**
+   * Feed the next slice of raw model output. Returns only what could be
+   * resolved from this call; text that might be a partial tag stays buffered.
+   */
+  push(raw: string): ThinkSplit {
+    this.buffer += raw
+    let content = ''
+    let thinking = ''
+
+    while (this.buffer.length > 0) {
+      if (this.inThink) {
+        const closeIdx = this.buffer.indexOf(CLOSE_TAG)
+        if (closeIdx !== -1) {
+          thinking += this.buffer.slice(0, closeIdx)
+          this.buffer = this.buffer.slice(closeIdx + CLOSE_TAG.length)
+          this.inThink = false
+        } else {
+          const hold = partialTagSuffix(CLOSE_TAG, this.buffer)
+          thinking += this.buffer.slice(0, this.buffer.length - hold)
+          this.buffer = this.buffer.slice(this.buffer.length - hold)
+          break
+        }
+      } else {
+        const openIdx = this.buffer.indexOf(OPEN_TAG)
+        if (openIdx !== -1) {
+          content += this.buffer.slice(0, openIdx)
+          this.buffer = this.buffer.slice(openIdx + OPEN_TAG.length)
+          this.inThink = true
+        } else {
+          const hold = partialTagSuffix(OPEN_TAG, this.buffer)
+          content += this.buffer.slice(0, this.buffer.length - hold)
+          this.buffer = this.buffer.slice(this.buffer.length - hold)
+          break
+        }
+      }
+    }
+
+    return { content, thinking }
+  }
+
+  /**
+   * Flush whatever is still buffered at end of stream. A held-back partial tag
+   * that never completed was never a tag, so it belongs to whichever channel
+   * we were in when the stream ended — losing it would silently truncate the
+   * answer.
+   */
+  flush(): ThinkSplit {
+    const pending = this.buffer
+    this.buffer = ''
+    return this.inThink ? { content: '', thinking: pending } : { content: pending, thinking: '' }
+  }
+}
+
+/**
+ * One-shot convenience for non-streaming responses.
+ */
+export function splitThinkTags(raw: string): ThinkSplit {
+  const splitter = new ThinkTagSplitter()
+  const streamed = splitter.push(raw)
+  const tail = splitter.flush()
+  return {
+    content: streamed.content + tail.content,
+    thinking: streamed.thinking + tail.thinking,
+  }
+}
+
+/**
+ * Non-streaming counterpart to the merge the streaming normalizers do inline.
+ *
+ * A backend may report reasoning structurally (native `message.thinking`, or
+ * `reasoning` on the /v1 shim), inline as `<think>` tags, or both at once —
+ * and the ancillary calls (title, suggestions, query rewrite) all go through
+ * the non-streaming path. Without this the tags reached the sidebar title and,
+ * worse, the string that gets embedded and sent to Qdrant.
+ */
+export function normalizeNonStreamed(rawContent: string, nativeThinking?: string): ThinkSplit {
+  const split = splitThinkTags(rawContent ?? '')
+  return {
+    content: split.content,
+    thinking: (nativeThinking ?? '') + split.thinking,
+  }
+}

admin/app/utils/token_estimate.ts

@@ -0,0 +1,191 @@
+/**
+ * Token estimation for prompt budgeting.
+ *
+ * NOMAD cannot tokenize exactly. The model's real vocabulary lives inside a GGUF
+ * on the Ollama side, Ollama exposes no tokenize endpoint (ollama#12030 is still
+ * open), and bundling a BPE vocabulary would mean shipping the *wrong* vocabulary
+ * for whatever Llama/Qwen/Gemma/Granite model the user actually pulled. Fetching
+ * the right one per model needs network access, which an offline-first appliance
+ * does not have.
+ *
+ * So this estimates — but it estimates in a way that corrects itself. Two parts:
+ *
+ * 1. A structural segmenter (below). BPE vocabularies are built from natural text,
+ *    so in practice a token is roughly "a word, or a punctuation mark, or a chunk
+ *    of a long/rare word". Counting those directly tracks a real tokenizer far
+ *    better than dividing character count by a constant, because the constant is
+ *    only right for the kind of text it was tuned on. Prose runs ~4 chars/token;
+ *    dense JSON, markdown tables and code run closer to 2.5. Those are exactly the
+ *    payloads that overflow a window, so that is exactly where a fixed ratio is
+ *    most wrong and most dangerous.
+ *
+ * 2. A per-model correction factor learned from ground truth. Every Ollama
+ *    response reports `prompt_eval_count` — the real token count of the prompt we
+ *    just sent. TokenCalibrationService folds the observed error into a per-model
+ *    EWMA and feeds it back here as `ratio`. Costs nothing: no extra inference, no
+ *    extra dependency, and it converges on the actual tokenizer within a couple of
+ *    turns.
+ *
+ * Measured against real `prompt_eval_count` over a six-fixture set (prose,
+ * markdown table, TypeScript, JSON, bullet list, mixed long-form), mean absolute
+ * error after per-model calibration:
+ *
+ *     llama3:8b       segmenter  5.7%   chars/3.5  18.0%
+ *     qwen2.5:0.5b    segmenter  6.2%   chars/3.5  20.0%
+ *     granite4.1:8b   segmenter  6.7%   chars/3.5  18.4%
+ *
+ * The learned factors were k=1.02, k=1.26 and k=1.00 respectively — which is the
+ * argument for making it per-model rather than a global constant. Qwen tokenizes
+ * noticeably finer than Llama, and no single tuned divisor can serve both.
+ *
+ * Kept pure and dependency-free so it runs under bare `node --test`, the same
+ * convention as `rag_prompt.ts`.
+ */
+
+/**
+ * Chars-per-token used by the *ingestion* path when sizing chunks for the
+ * embedding model.
+ *
+ * Deliberately unchanged from RagService's historical value. Chunk size is baked
+ * into the eval corpus fingerprint, so touching it re-chunks the corpus and
+ * invalidates every committed retrieval baseline. Re-tuning it is a legitimate
+ * change, but it is a *retrieval* change and belongs to its own measured PR — not
+ * folded silently into a context-management refactor. Re-exported here only so
+ * there is one obvious place to find every token constant.
+ */
+export const INGEST_CHARS_PER_TOKEN = 2
+
+/**
+ * Per-message overhead from the chat template: role marker, turn delimiters, and
+ * the trailing newline the template adds. Real templates cost roughly 3-5 tokens
+ * a message. The old char-based math ignored this entirely, which quietly
+ * under-counted a 40-turn conversation by well over a hundred tokens.
+ */
+export const PER_MESSAGE_OVERHEAD_TOKENS = 4
+
+/** Tokens the template adds once, for the generation prompt that follows the messages. */
+export const CONVERSATION_OVERHEAD_TOKENS = 3
+
+/**
+ * Safety margin applied when a budget decision must not overflow.
+ *
+ * Under-estimating is strictly worse than over-estimating: an over-estimate wastes
+ * a little window, an under-estimate overflows it and hands the backend a silent
+ * truncation — the exact failure this whole subsystem exists to prevent.
+ */
+export const ESTIMATE_SAFETY_MARGIN = 1.1
+
+/**
+ * Character-class fragments, kept as strings so the CJK ranges appear exactly
+ * once and the regexes below stay readable.
+ */
+const CJK = '\\u3040-\\u30ff\\u3400-\\u4dbf\\u4e00-\\u9fff\\uf900-\\ufaff\\uac00-\\ud7af'
+const WORD = 'A-Za-z0-9\\u00c0-\\u024f'
+
+/**
+ * Segments that a BPE tokenizer tends to split on. Order matters: the first
+ * alternative that matches wins.
+ *
+ * - CJK/Hangul codepoints are usually one token each (often more), so they are
+ *   matched singly rather than as words.
+ * - Runs of letters/digits are word-ish; long ones get sub-split below.
+ * - Runs of punctuation/symbols are matched as a *run*, not per character. BPE
+ *   vocabularies contain merged punctuation pieces (`":`, `",`, `);`, `],`), so
+ *   charging one token per punctuation character over-counts structured text
+ *   badly -- measured at +48% on a JSON payload before this was split out.
+ */
+const SEGMENT_RE = new RegExp(`[${CJK}]|[${WORD}]+|[^\\s${WORD}${CJK}]+`, 'gu')
+
+/** Identifies the punctuation-run alternative when walking matches. */
+const PUNCT_RUN_RE = new RegExp(`^[^\\s${WORD}${CJK}]+$`, 'u')
+
+/**
+ * Above this length, a run of letters/digits is a rare word, an identifier, or a
+ * hash, and a real tokenizer will break it into several pieces.
+ */
+const LONG_WORD_CHARS = 7
+
+/** Roughly how many characters of a punctuation run one merged token covers. */
+const PUNCT_CHARS_PER_TOKEN = 2
+
+/**
+ * Estimate the token count of a string.
+ *
+ * `ratio` is the learned per-model correction (1.0 = uncalibrated). Pass the value
+ * from TokenCalibrationService.ratioFor(model).
+ */
+export function estimateTokens(text: string, ratio = 1): number {
+  if (!text) return 0
+
+  let tokens = 0
+  const matches = text.match(SEGMENT_RE)
+  if (matches) {
+    for (const segment of matches) {
+      if (PUNCT_RUN_RE.test(segment)) {
+        // Punctuation merges: `":` and `},` are usually single tokens.
+        tokens += Math.ceil(segment.length / PUNCT_CHARS_PER_TOKEN)
+      } else if (segment.length > LONG_WORD_CHARS) {
+        // Long words split into roughly one piece per LONG_WORD_CHARS characters.
+        tokens += Math.ceil(segment.length / LONG_WORD_CHARS)
+      } else {
+        tokens += 1
+      }
+    }
+  }
+
+  // Leading whitespace is folded into the following token by most BPE
+  // vocabularies, but newlines are usually tokens in their own right and matter
+  // for the markdown-heavy content NOMAD deals in.
+  const newlines = (text.match(/\n/g) || []).length
+  tokens += newlines
+
+  return Math.max(1, Math.ceil(tokens * ratio))
+}
+
+export type EstimatableMessage = { role: string; content: string }
+
+/**
+ * Estimate the token cost of a full message array as the backend will see it,
+ * including chat-template overhead.
+ */
+export function estimateMessagesTokens(messages: EstimatableMessage[], ratio = 1): number {
+  if (messages.length === 0) return 0
+  let total = CONVERSATION_OVERHEAD_TOKENS
+  for (const message of messages) {
+    total += estimateTokens(message.content, ratio) + PER_MESSAGE_OVERHEAD_TOKENS
+  }
+  return total
+}
+
+/**
+ * Estimate with the safety margin applied — use this wherever exceeding the
+ * budget causes silent truncation rather than merely wasting space.
+ */
+export function estimateTokensConservative(text: string, ratio = 1): number {
+  return Math.ceil(estimateTokens(text, ratio) * ESTIMATE_SAFETY_MARGIN)
+}
+
+/**
+ * Fold a fresh observation into an exponentially-weighted moving average.
+ *
+ * `alpha` is intentionally low: a single odd turn (a huge code paste, a burst of
+ * CJK) should nudge the ratio, not redefine it.
+ */
+export function updateEwma(previous: number | null, observed: number, alpha = 0.25): number {
+  if (previous === null || !Number.isFinite(previous) || previous <= 0) return observed
+  return previous * (1 - alpha) + observed * alpha
+}
+
+/**
+ * Guard rail on the learned ratio. A calibration ratio far outside this range
+ * means something other than tokenization differs — a backend injecting a hidden
+ * system prompt, a tool schema we didn't count, a reasoning preamble. Clamping
+ * keeps one bad reading from destroying the budget for every later turn.
+ */
+export const MIN_CALIBRATION_RATIO = 0.5
+export const MAX_CALIBRATION_RATIO = 2.5
+
+export function clampRatio(ratio: number): number {
+  if (!Number.isFinite(ratio) || ratio <= 0) return 1
+  return Math.min(MAX_CALIBRATION_RATIO, Math.max(MIN_CALIBRATION_RATIO, ratio))
+}

admin/app/utils/zim_batch_decision.ts

@@ -0,0 +1,41 @@
+/**
+ * Decision for whether a batched ZIM ingestion should dispatch a continuation.
+ *
+ * This is the pure, I/O-free core of the batch loop in
+ * `RagService.processZIMFile` (mirrors `decideScanAction` in
+ * `kb_ingest_decision.ts`).
+ *
+ * The signal MUST be the number of articles the extractor *consumed*, never the
+ * number that produced chunks. An article yields zero chunks whenever its text
+ * is empty after HTML cleaning — redirect stubs, category listings, image/video
+ * wrappers, PDF containers. Those are perfectly normal ZIM entries, but they are
+ * invisible to any count derived from the returned chunks.
+ *
+ * Gating on chunk-derived counts silently truncates ingestion the first time a
+ * window of `batchSize` articles happens to contain mostly empty ones: the
+ * continuation is never dispatched and every remaining article in the archive is
+ * skipped. That is not a rare edge case — it is the norm for scraped-site and
+ * media-heavy archives, where the opening entries are usually navigation pages.
+ *
+ * `articlesProcessed < batchSize` means the extractor's iterator ran dry, which
+ * is the only reliable end-of-archive signal available: `archive.articleCount`
+ * cannot serve as an upper bound, because `iterByPath()` legitimately yields
+ * more article entries than that figure for some archives.
+ *
+ * A full final batch costs one extra dispatch that extracts nothing and stops.
+ * That is deliberate: over-running by a single empty batch is cheap, while
+ * stopping one batch early loses the remainder of the archive.
+ */
+export interface ZimBatchProgress {
+  /** Articles the extractor consumed in this batch (not articles that produced chunks). */
+  articlesProcessed: number
+  /** The article ceiling requested for this batch. */
+  batchSize: number
+}
+
+export function hasMoreArticleBatches({
+  articlesProcessed,
+  batchSize,
+}: ZimBatchProgress): boolean {
+  return articlesProcessed >= batchSize
+}

admin/app/validators/settings.ts

@@ -1,6 +1,8 @@
 import vine from "@vinejs/vine";
 import { SETTINGS_KEYS } from "../../constants/kv_store.js";
 import type { KVStoreKey } from "../../types/kv_store.js";
+import { CONTEXT_LADDER } from "../utils/context_window.js";
+import { RESPONSE_STYLE_PRESETS } from "../../constants/ollama.js";
 
 export const getSettingSchema = vine.compile(vine.object({
     key: vine.enum(SETTINGS_KEYS),
@@ -62,6 +64,73 @@ export function validateSettingValue(key: KVStoreKey, value: unknown): string |
             }
             return null
         }
+        case 'ai.contextWindow': {
+            // "auto" (or empty) hands sizing to ContextWindowService. An explicit
+            // value is a *cap*, so it only ever lowers the resolved window — but it
+            // still has to be a real ladder rung, since a value the backend can't
+            // honour would silently fall back to the default and confuse the user.
+            if (value === '' || value === 'auto' || value === undefined || value === null) {
+                return null
+            }
+            const num = Number(value)
+            if (!CONTEXT_LADDER.includes(num as (typeof CONTEXT_LADDER)[number])) {
+                return `Context window must be "auto" or one of: ${CONTEXT_LADDER.join(', ')}.`
+            }
+            return null
+        }
+        case 'rag.minRelevance': {
+            // Empty/'auto' clears the setting and reverts to RAG_MIN_FINAL_SCORE.
+            // An explicit 0 is a real choice — it turns the floor off — so it is
+            // deliberately not treated as "unset".
+            // Trimmed first so the accepted set matches parseMinRelevance's:
+            // Number('  ') is 0, which would otherwise validate as "floor off"
+            // for a value the parser reads as unset.
+            const raw = typeof value === 'string' ? value.trim() : value
+            if (raw === '' || raw === 'auto' || raw === undefined || raw === null) {
+                return null
+            }
+            const num = Number(raw)
+            if (!Number.isFinite(num) || num < 0 || num > 1) {
+                return 'Relevance threshold must be a number between 0 and 1, or empty to use the recommended default.'
+            }
+            return null
+        }
+        case 'ai.responseStyle': {
+            // Empty/unset means 'auto'. Anything else has to be a style we know,
+            // since an unrecognised value silently resolves back to the default
+            // and would leave the select showing a choice nothing honours.
+            if (value === '' || value === undefined || value === null) {
+                return null
+            }
+            const styles = RESPONSE_STYLE_PRESETS.map((preset) => preset.value)
+            if (typeof value !== 'string' || !styles.includes(value as (typeof styles)[number])) {
+                return `Response style must be one of: ${styles.join(', ')}.`
+            }
+            return null
+        }
+        case 'ai.keepAlive': {
+            // Ollama duration format: "10m", "1h", "30s", "-1" (forever), "0" (evict).
+            if (value === '' || value === undefined || value === null) {
+                return null
+            }
+            if (typeof value !== 'string' || !/^(-?\d+|\d+(\.\d+)?(ms|s|m|h))$/.test(value)) {
+                return 'Keep-alive must be a duration like "10m", "1h", or "-1" to keep the model loaded indefinitely.'
+            }
+            return null
+        }
+        case 'ai.amdHsaOverride': {
+            // Lands in the Ollama container env as HSA_OVERRIDE_GFX_VERSION, so only
+            // the forms DockerService._resolveAmdHsaOverride honours are accepted.
+            // Empty clears it and 'none' disables the gfx-marker default.
+            const raw = typeof value === 'string' ? value.trim().toLowerCase() : value
+            if (raw === '' || raw === undefined || raw === null || raw === 'none') {
+                return null
+            }
+            if (typeof raw !== 'string' || !/^\d{1,2}\.\d{1,2}\.\d{1,2}$/.test(raw)) {
+                return 'GFX override must be a version like "11.0.0", "none", or empty to use the detected default.'
+            }
+            return null
+        }
         default:
             return null
     }

admin/app/validators/system.ts

@@ -1,4 +1,5 @@
 import vine from '@vinejs/vine'
+import { LINK_TILE_COLOR_IDS } from '../../constants/link_tile_colors.js'
 
 export const installServiceValidator = vine.compile(
   vine.object({
@@ -100,7 +101,7 @@ export const customAppValidator = vine.compile(
 export const setServiceCustomUrlValidator = vine.compile(
   vine.object({
     service_name: vine.string().trim(),
-    custom_url: vine.string().trim().nullable(),
+    custom_url: vine.string().trim().maxLength(255).nullable(),
   })
 )
 
@@ -117,12 +118,52 @@ export function normalizeCustomUrl(input: string | null | undefined): string | n
   try {
     const url = new URL(withScheme)
     if (url.protocol !== 'http:' && url.protocol !== 'https:') return null
-    return url.href
+    // services.custom_url uses the database driver's default VARCHAR(255) size.
+    // Check the normalized value because prepending a scheme or URL canonicalization
+    // can make a valid-looking input longer than the stored column.
+    return url.href.length <= 255 ? url.href : null
   } catch {
     return null
   }
 }
 
+/**
+ * Dashboard link tile: a shortcut to something the user already runs, with no
+ * container behind it. One `url` field rather than separate host/port/path,
+ * because people paste URLs and it gets https and sub-paths for free. The URL is
+ * normalized and http(s)-restricted by normalizeCustomUrl in the controller,
+ * which is what keeps javascript:/data: out of an href.
+ */
+export const createLinkTileValidator = vine.compile(
+  vine.object({
+    friendly_name: vine.string().trim().minLength(1).maxLength(60),
+    url: vine.string().trim().minLength(1).maxLength(2048),
+    description: vine.string().trim().maxLength(200).nullable().optional(),
+    icon: vine.string().trim().maxLength(60).nullable().optional(),
+    display_order: vine.number().min(0).max(999).optional(),
+    link_color: vine.enum(LINK_TILE_COLOR_IDS).optional(),
+  })
+)
+
+/** Reconfigure an existing link tile. Identified by service_name, which is immutable. */
+export const updateLinkTileValidator = vine.compile(
+  vine.object({
+    service_name: vine.string().trim(),
+    friendly_name: vine.string().trim().minLength(1).maxLength(60),
+    url: vine.string().trim().minLength(1).maxLength(2048),
+    description: vine.string().trim().maxLength(200).nullable().optional(),
+    icon: vine.string().trim().maxLength(60).nullable().optional(),
+    display_order: vine.number().min(0).max(999).optional(),
+    link_color: vine.enum(LINK_TILE_COLOR_IDS).optional(),
+  })
+)
+
+export const deleteLinkTileValidator = vine.compile(
+  vine.object({
+    service_name: vine.string().trim(),
+  })
+)
+
 export const deleteCustomAppValidator = vine.compile(
   vine.object({
     service_name: vine.string().trim(),

admin/app/validators/zim.ts

@@ -5,6 +5,14 @@ export const listRemoteZimValidator = vine.compile(
     start: vine.number().min(0).optional(),
     count: vine.number().min(1).max(100).optional(),
     query: vine.string().optional(),
+    // An ISO-639-3 code (`eng`, `fra`, `zho`) or the literal `all`. Constrained to a
+    // short alphabetic token so this can never smuggle anything else into the upstream
+    // catalog query string.
+    language: vine
+      .string()
+      .trim()
+      .regex(/^(all|[a-z]{2,8})$/)
+      .optional(),
   })
 )
 

admin/commands/eval/compare.ts

@@ -0,0 +1,127 @@
+import { BaseCommand, args, flags } from '@adonisjs/core/ace'
+import type { CommandOptions } from '@adonisjs/core/types/ace'
+
+/**
+ * Diff an eval report against a baseline and fail on regression.
+ *
+ * This is the "did my change help?" command, and the one CI would call.
+ *
+ *   node ace eval:compare tests/eval/baselines/<fp>/retrieval.json tests/eval/reports/latest.json
+ *   node ace eval:compare base.json current.json --tolerance=0.05
+ *   node ace eval:compare --promote=retrieval tests/eval/reports/latest.json
+ */
+export default class EvalCompare extends BaseCommand {
+  static commandName = 'eval:compare'
+  static description = 'Diff an eval report against a baseline; exits non-zero on regression'
+
+  @args.string({ description: 'Baseline report JSON (or the report to promote, with --promote)' })
+  declare baseline: string
+
+  @args.string({ description: 'Current report JSON', required: false })
+  declare current: string
+
+  @flags.string({ description: 'Regression tolerance per metric (default: 0.02)' })
+  declare tolerance: string
+
+  @flags.string({
+    description: 'Promote the given report to a committed baseline under this name, then exit',
+  })
+  declare promote: string
+
+  @flags.boolean({ description: 'Print the full metric table, not just the regressions' })
+  declare verbose: boolean
+
+  /**
+   * Deliberately does not boot the application.
+   *
+   * Comparing two report files is pure I/O — it needs no MySQL, no Redis, no
+   * Qdrant, and no Ollama. Keeping it that way is what lets CI run the gate on
+   * a committed baseline without standing up the whole stack, and it avoids the
+   * app's noisy Redis teardown on exit.
+   */
+  static options: CommandOptions = {
+    startApp: false,
+  }
+
+  async run() {
+    const { EvalReportService } = await import('#services/eval_report_service')
+    const { compareReports, DEFAULT_TOLERANCE } = await import('../../app/utils/eval/report.js')
+    const service = new EvalReportService()
+
+    try {
+      if (this.promote) {
+        const report = await service.read(this.baseline)
+        const path = await service.promoteToBaseline(report, this.promote)
+        this.logger.success(`Promoted to baseline: ${path}`)
+        this.logger.info(`Corpus fingerprint: ${report.meta.corpusFingerprint}`)
+        this.logger.info('Commit this file so the whole team gates against the same numbers.')
+        return
+      }
+
+      if (!this.current) {
+        this.logger.error('Two report paths are required (or use --promote=<name> with one).')
+        this.exitCode = 1
+        return
+      }
+
+      const [baseline, current] = await Promise.all([
+        service.read(this.baseline),
+        service.read(this.current),
+      ])
+      const tolerance = this.tolerance ? Number.parseFloat(this.tolerance) : DEFAULT_TOLERANCE
+      const result = compareReports(baseline, current, tolerance)
+
+      if (!result.comparable) {
+        // Refusing is the correct outcome, not a failure of the tool — diffing
+        // runs from different corpora would manufacture a regression.
+        this.logger.error('Reports are not comparable:')
+        this.logger.error(`  ${result.incomparableReason}`)
+        this.exitCode = 1
+        return
+      }
+
+      this.logger.info(
+        `baseline ${short(baseline.meta.gitSha)} (${baseline.meta.createdAt}) -> current ${short(current.meta.gitSha)} (${current.meta.createdAt})`
+      )
+      this.logger.info(`corpus ${current.meta.corpusFingerprint} · tolerance ${tolerance}`)
+      if (current.meta.gitDirty) {
+        this.logger.warning('Current report was produced from a dirty working tree.')
+      }
+      this.logger.info('')
+
+      const rows = this.verbose
+        ? [...result.deltas].sort((a, b) => (a.improvement ?? 0) - (b.improvement ?? 0))
+        : [...result.regressions, ...result.improvements]
+
+      if (rows.length === 0) {
+        this.logger.success('No metric moved outside the tolerance band.')
+      } else {
+        for (const d of rows) {
+          const line = `  ${d.metric.padEnd(24)} ${fmt(d.baseline)} -> ${fmt(d.current)}  ${signed(d.delta)}`
+          if (d.regressed) this.logger.error(`${line}   REGRESSED`)
+          else if ((d.improvement ?? 0) > tolerance) this.logger.success(`${line}   improved`)
+          else this.logger.info(line)
+        }
+      }
+
+      this.logger.info('')
+      if (result.regressions.length > 0) {
+        this.logger.error(
+          `${result.regressions.length} metric(s) regressed beyond the ${tolerance} tolerance.`
+        )
+        this.exitCode = 1
+      } else {
+        this.logger.success(
+          `No regressions. ${result.improvements.length} metric(s) improved beyond tolerance.`
+        )
+      }
+    } catch (error) {
+      this.logger.error(error instanceof Error ? error.message : String(error))
+      this.exitCode = 1
+    }
+  }
+}
+
+const fmt = (v: number | null) => (v === null ? '   n/a' : v.toFixed(4).padStart(6))
+const signed = (v: number | null) => (v === null ? '     ' : `${v >= 0 ? '+' : ''}${v.toFixed(4)}`)
+const short = (sha: string | null) => (sha ? sha.slice(0, 7) : 'unknown')

admin/commands/eval/corpus.ts

@@ -0,0 +1,155 @@
+import { BaseCommand, flags } from '@adonisjs/core/ace'
+import type { CommandOptions } from '@adonisjs/core/types/ace'
+
+/**
+ * Manage the frozen evaluation corpus.
+ *
+ *   # Validate the corpus and goldens without touching the vector store
+ *   node ace eval:corpus --check
+ *
+ *   # Show what is currently ingested
+ *   node ace eval:corpus --status
+ *
+ *   # Wipe and rebuild (the normal path; safe — never touches user content)
+ *   node ace eval:corpus --ingest
+ *
+ *   # Remove the eval corpus entirely
+ *   node ace eval:corpus --reset
+ */
+export default class EvalCorpus extends BaseCommand {
+  static commandName = 'eval:corpus'
+  static description = 'Manage the frozen RAG evaluation corpus (ingest, reset, status, check)'
+
+  @flags.boolean({ description: 'Wipe and re-ingest the corpus into the reserved eval collection' })
+  declare ingest: boolean
+
+  @flags.boolean({ description: 'Remove every eval corpus chunk from the vector store' })
+  declare reset: boolean
+
+  @flags.boolean({ description: 'Show the fingerprint and current chunk count' })
+  declare status: boolean
+
+  @flags.boolean({
+    description: 'Validate the corpus and goldens only — no Qdrant, no Ollama, no writes',
+  })
+  declare check: boolean
+
+  @flags.boolean({ description: 'Leave application debug logging on (very noisy)' })
+  declare debug: boolean
+
+  static options: CommandOptions = {
+    startApp: true,
+  }
+
+  async run() {
+    const { EvalCorpusService } = await import('#services/eval_corpus_service')
+    const { quietLogging } = await import('../../app/utils/eval/quiet.js')
+    const service = await this.app.container.make(EvalCorpusService)
+    const restoreLogging = quietLogging(this.debug)
+    try {
+      await this.dispatch(service)
+    } finally {
+      restoreLogging()
+    }
+  }
+
+  private async dispatch(service: any) {
+
+    // Default to --status so a bare invocation is always safe.
+    if (!this.ingest && !this.reset && !this.check) this.status = true
+
+    if (this.check) {
+      return this.runCheck(service)
+    }
+
+    if (this.reset) {
+      const removed = await service.reset()
+      this.logger.success(`Removed ${removed} eval corpus chunks`)
+      return
+    }
+
+    if (this.ingest) {
+      return this.runIngest(service)
+    }
+
+    const [fingerprint, chunks, goldens] = await Promise.all([
+      service.fingerprint(),
+      service.count(),
+      service.loadGoldens(['*']),
+    ])
+    this.logger.info(`Fingerprint:   ${fingerprint}`)
+    this.logger.info(`Chunks in KB:  ${chunks}`)
+    this.logger.info(`Goldens:       ${goldens.length}`)
+    if (chunks === 0) {
+      this.logger.warning('Corpus is not ingested. Run: node ace eval:corpus --ingest')
+    }
+  }
+
+  /**
+   * Validation-only path. Deliberately avoids the container-resolved services'
+   * network dependencies so a contributor can check their golden edits without
+   * a running stack.
+   */
+  private async runCheck(service: any) {
+    try {
+      const corpus = await service.loadCorpus()
+      const goldens = await service.loadGoldens(['*'])
+      const fingerprint = await service.fingerprint()
+
+      const tagCounts = new Map<string, number>()
+      for (const g of goldens) {
+        for (const tag of g.tags) tagCounts.set(tag, (tagCounts.get(tag) ?? 0) + 1)
+      }
+
+      // A document no golden asks about is dead weight in the corpus: it costs
+      // ingest time and adds distractor noise nobody chose deliberately.
+      const referenced = new Set<string>(goldens.flatMap((g: any) => g.relevantDocIds))
+      const unreferenced = corpus
+        .map((d: any) => d.docId)
+        .filter((id: string) => !referenced.has(id))
+
+      this.logger.success(`Corpus OK: ${corpus.length} documents, ${goldens.length} goldens`)
+      this.logger.info(`Fingerprint: ${fingerprint}`)
+      this.logger.info(
+        `Tags: ${[...tagCounts.entries()]
+          .sort((a, b) => b[1] - a[1])
+          .map(([t, n]) => `${t}=${n}`)
+          .join(' ')}`
+      )
+      const refusals = goldens.filter((g: any) => g.expectRefusal).length
+      this.logger.info(`Refusal cases: ${refusals} / ${goldens.length}`)
+      if (unreferenced.length > 0) {
+        this.logger.warning(`Documents no golden references: ${unreferenced.join(', ')}`)
+      }
+    } catch (error) {
+      this.logger.error(error instanceof Error ? error.message : String(error))
+      this.exitCode = 1
+    }
+  }
+
+  private async runIngest(service: any) {
+    this.logger.info('Rebuilding the eval corpus (user content is never touched)...')
+    // Validate before destroying anything — a typo in a golden should not cost
+    // you the ingest you were about to run.
+    await service.loadGoldens(['*'])
+
+    const summary = await service.ingest((docId: string, index: number, total: number) => {
+      this.logger.info(`  [${index}/${total}] ${docId}`)
+    })
+
+    this.logger.info('')
+    if (summary.removedBeforeIngest > 0) {
+      this.logger.info(`Removed ${summary.removedBeforeIngest} chunks from the previous ingest`)
+    }
+    this.logger.success(`Ingested ${summary.documents} documents into ${summary.chunks} chunks`)
+    this.logger.info(`Fingerprint: ${summary.fingerprint}`)
+
+    if (summary.failures.length > 0) {
+      this.logger.error(`${summary.failures.length} document(s) failed to ingest:`)
+      for (const f of summary.failures) this.logger.error(`  ${f.docId}: ${f.reason}`)
+      // A partial corpus produces scores that look real and are not.
+      this.logger.error('The corpus is incomplete — results from this state are not comparable.')
+      this.exitCode = 1
+    }
+  }
+}

admin/commands/eval/generation.ts

@@ -0,0 +1,294 @@
+import { BaseCommand, flags } from '@adonisjs/core/ace'
+import type { CommandOptions } from '@adonisjs/core/types/ace'
+import type {
+  GenerationAggregate,
+  GenerationMode,
+} from '../../app/services/eval_generation_service.js'
+import type { ResponseStyle } from '../../types/ollama.js'
+
+const MODES: GenerationMode[] = ['oracle', 'e2e', 'noretrieval']
+const STYLES: ResponseStyle[] = ['auto', 'focused', 'creative', 'off']
+
+/**
+ * Score generated answers against the golden set.
+ *
+ * The `--mode` flag is the whole point of this command:
+ *
+ *   oracle       perfect context injected by construction -> isolates the MODEL
+ *   e2e          real retrieval                           -> the real product
+ *   noretrieval  no context at all                        -> parametric baseline
+ *
+ * Run all three on the same model and the ambiguous complaint "the AI gave a
+ * bad answer" decomposes into a number that can be acted on.
+ *
+ *   node ace eval:generation --model=llama3.2:latest --mode=oracle
+ *   node ace eval:generation --model=llama3:8b --all-modes
+ *   node ace eval:generation --model=mock          # no Ollama needed
+ */
+export default class EvalGeneration extends BaseCommand {
+  static commandName = 'eval:generation'
+  static description = 'Score generated answers, with oracle/e2e/noretrieval ablation'
+
+  @flags.string({ description: 'Ollama model to evaluate, or "mock" for the extractive ceiling' })
+  declare model: string
+
+  @flags.string({ description: `One of: ${MODES.join(', ')} (default: e2e)` })
+  declare mode: string
+
+  @flags.boolean({ description: 'Run all three modes and print the decomposition' })
+  declare allModes: boolean
+
+  @flags.string({ description: 'Repeats per question, for stability (default: 3)' })
+  declare repeats: string
+
+  @flags.string({
+    description:
+      'Comma-separated golden suites to run, by filename without .jsonl (default: core). ' +
+      'Extra suites are opt-in so they cannot move the numbers a baseline was recorded against.',
+  })
+  declare suite: string
+
+  @flags.string({
+    description:
+      `Sample the way real chat does, under one of: ${STYLES.join(', ')}. ` +
+      'Omit for the deterministic temperature-0 default. Greedy decoding makes ' +
+      'min_p, top_p and top_k inert, so a Response Style is only visible with ' +
+      'this flag set, and "off" (no samplers, no temperature) is the control to ' +
+      'compare a style against.',
+  })
+  declare style: string
+
+  @flags.string({ description: 'Only run goldens carrying this tag' })
+  declare tag: string
+
+  @flags.string({ description: 'Limit to the first N goldens (for a quick smoke run)' })
+  declare limit: string
+
+  @flags.boolean({ description: 'Print every failing question with the answer the model gave' })
+  declare verbose: boolean
+
+  @flags.boolean({ description: 'Leave application debug logging on (very noisy)' })
+  declare debug: boolean
+
+  static options: CommandOptions = {
+    startApp: true,
+  }
+
+  async run() {
+    const { EvalCorpusService } = await import('#services/eval_corpus_service')
+    const { EvalGenerationService, MOCK_MODEL } = await import('#services/eval_generation_service')
+    const { quietLogging } = await import('../../app/utils/eval/quiet.js')
+
+    const corpusService = await this.app.container.make(EvalCorpusService)
+    const generationService = await this.app.container.make(EvalGenerationService)
+    const restoreLogging = quietLogging(this.debug)
+
+    try {
+      if (!this.model) {
+        this.logger.error('--model is required (use --model=mock to run without Ollama)')
+        this.exitCode = 1
+        return
+      }
+
+      const modes: GenerationMode[] = this.allModes
+        ? MODES
+        : [(this.mode as GenerationMode) || 'e2e']
+      for (const mode of modes) {
+        if (!MODES.includes(mode)) {
+          this.logger.error(`Unknown mode "${mode}". Expected one of: ${MODES.join(', ')}`)
+          this.exitCode = 1
+          return
+        }
+      }
+
+      if (this.style && !STYLES.includes(this.style as ResponseStyle)) {
+        this.logger.error(`Unknown style "${this.style}". Expected one of: ${STYLES.join(', ')}`)
+        this.exitCode = 1
+        return
+      }
+      const samplerStyle = this.style ? (this.style as ResponseStyle) : undefined
+
+      let goldens = await corpusService.loadGoldens(this.suite ? this.suite.split(',').map((s) => s.trim()) : undefined)
+      if (this.tag) goldens = goldens.filter((g) => g.tags.includes(this.tag))
+      if (this.limit) goldens = goldens.slice(0, Number.parseInt(this.limit, 10))
+      if (goldens.length === 0) {
+        this.logger.error('No goldens matched the given filters')
+        this.exitCode = 1
+        return
+      }
+
+      // e2e and oracle both need the corpus present; noretrieval does not, but
+      // requiring it uniformly keeps the three modes comparable.
+      const chunks = await corpusService.count()
+      if (chunks === 0 && modes.some((m) => m !== 'noretrieval')) {
+        this.logger.error('The eval corpus is not ingested. Run: node ace eval:corpus --ingest')
+        this.exitCode = 1
+        return
+      }
+
+      const fingerprint = await corpusService.fingerprint()
+      const repeats = this.repeats ? Number.parseInt(this.repeats, 10) : 3
+      this.logger.info(
+        `Corpus ${fingerprint} · ${goldens.length} goldens · model=${this.model} · repeats=${repeats}` +
+          (samplerStyle ? ` · style=${samplerStyle}` : ' · style=deterministic (temp 0)')
+      )
+      if (samplerStyle) {
+        this.logger.info(
+          'Sampling is on: the seed is fixed but the run is only approximately reproducible. Watch the unstable count.'
+        )
+      }
+      if (this.model === MOCK_MODEL) {
+        this.logger.info(
+          'Mock model: answers are the injected context verbatim — this is the extractive ceiling for the current retrieval, not a real model.'
+        )
+      }
+      this.logger.info('')
+
+      const summaries: Array<{ mode: GenerationMode; agg: GenerationAggregate }> = []
+
+      for (const mode of modes) {
+        const result = await generationService.run(goldens, {
+          mode,
+          model: this.model,
+          repeats,
+          samplerStyle,
+          onProgress: (id, index, total) => {
+            if (this.verbose) this.logger.info(`  [${index}/${total}] ${mode}: ${id}`)
+          },
+        })
+        summaries.push({ mode, agg: result.overall })
+
+        this.printAggregate(mode, result.overall, result.elapsedMs)
+
+        if (result.overall.errors > 0) {
+          this.logger.error(
+            `${result.overall.errors} question(s) errored talking to the model — the scores above are incomplete.`
+          )
+          const firstError = result.cases.find((c) => c.error)
+          if (firstError) this.logger.error(`  first error (${firstError.id}): ${firstError.error}`)
+          this.exitCode = 1
+        }
+
+        if (this.verbose) this.printFailures(result.cases)
+      }
+
+      if (summaries.length > 1) this.printDecomposition(summaries)
+    } catch (error) {
+      this.logger.error(error instanceof Error ? error.message : String(error))
+      this.exitCode = 1
+    } finally {
+      restoreLogging()
+    }
+  }
+
+  private printAggregate(mode: GenerationMode, agg: GenerationAggregate, elapsedMs: number) {
+    this.logger.info(`=== ${mode.toUpperCase()} (${(elapsedMs / 1000).toFixed(1)}s) ===`)
+    this.logger.info(`  correctness        ${fmt(agg.correctness)}`)
+    this.logger.info(`  refusal correct    ${fmt(agg.refusalCorrectness)}`)
+    this.logger.info(`  leakage rate       ${fmt(agg.leakageRate)}  (lower is better)`)
+    this.logger.info(`  think-tag leak     ${fmt(agg.thinkTagLeakRate)}  (should be 0)`)
+    this.logger.info(`  markdown formatted ${fmt(agg.markdownRate)}`)
+    this.logger.info(
+      `  numeric grounding  ${agg.groundedness ? `${agg.groundedness.mean.toFixed(3)} (n=${agg.groundedness.n})` : '  n/a'}`
+    )
+    this.logger.info(
+      `  mean answer length ${agg.meanAnswerLength === null ? 'n/a' : Math.round(agg.meanAnswerLength)} chars`
+    )
+    // Context budgeting. These explain *why* a correctness number moved: an
+    // answer the model never had the material for is a different failure from
+    // an answer it got wrong, and only these two lines tell them apart.
+    if (agg.meanPromptTokens !== null) {
+      this.logger.info(`  mean prompt tokens ${Math.round(agg.meanPromptTokens)}`)
+    }
+    if (agg.historyElidedRate !== null) {
+      this.logger.info(
+        `  history elided     ${fmt(agg.historyElidedRate)}  (share of cases needing a trim)`
+      )
+    }
+    if (agg.chunksDroppedRate !== null) {
+      this.logger.info(
+        `  context dropped    ${fmt(agg.chunksDroppedRate)}  (share of cases where a chunk didn't fit)`
+      )
+    }
+    if (agg.promptTokenError) {
+      // Every budget decision rests on the estimate; if this drifts, the trims
+      // above are being made against a number that is no longer true.
+      this.logger.info(
+        `  token estimate err ${(agg.promptTokenError.mean * 100).toFixed(1)}%  (vs the backend's real count)`
+      )
+    }
+    if (agg.unstable > 0) {
+      this.logger.warning(
+        `  ${agg.unstable} question(s) flipped between repeats — excluded from gating, do not read them as a regression.`
+      )
+    }
+    this.logger.info('')
+  }
+
+  /**
+   * The reason this command exists: turn three scores into an attribution.
+   */
+  private printDecomposition(summaries: Array<{ mode: GenerationMode; agg: GenerationAggregate }>) {
+    const get = (mode: GenerationMode) => summaries.find((s) => s.mode === mode)?.agg.correctness ?? null
+    const oracle = get('oracle')
+    const e2e = get('e2e')
+    const none = get('noretrieval')
+
+    this.logger.info('=== Decomposition ===')
+    if (oracle !== null && e2e !== null) {
+      const cost = oracle - e2e
+      this.logger.info(`  retrieval cost   ${cost.toFixed(3)}   (oracle ${oracle.toFixed(3)} - e2e ${e2e.toFixed(3)})`)
+      this.logger.info(`  model ceiling    ${oracle.toFixed(3)}   (what this model manages with perfect context)`)
+    }
+    if (e2e !== null && none !== null) {
+      this.logger.info(`  RAG contribution ${(e2e - none).toFixed(3)}   (e2e ${e2e.toFixed(3)} - noretrieval ${none.toFixed(3)})`)
+    }
+
+    if (oracle !== null && e2e !== null) {
+      // The actual triage rule, stated so nobody has to re-derive it.
+      const cost = oracle - e2e
+      if (oracle < 0.6) {
+        this.logger.warning(
+          '  Oracle is low: this model struggles even with perfect context. Retrieval work will not fix it — try a larger model.'
+        )
+      } else if (cost > 0.15) {
+        this.logger.warning(
+          '  Large retrieval cost: the model can use good context but is not being given it. Work on retrieval.'
+        )
+      } else {
+        this.logger.success('  Model and retrieval are both holding up on this corpus.')
+      }
+    }
+    if (none !== null && e2e !== null && e2e <= none) {
+      this.logger.warning(
+        '  RAG is not adding anything over the bare model on this set — check that retrieval is actually reaching the prompt.'
+      )
+    }
+  }
+
+  private printFailures(cases: Array<any>) {
+    const failures = cases.filter((c) => c.correctness.passRate < 1 || c.refusalCorrectness.passRate < 1)
+    if (failures.length === 0) return
+    this.logger.info('--- failures ---')
+    for (const f of failures) {
+      this.logger.info(`  ${f.id}  correct=${f.correctness.passes}/${f.correctness.repeats} refusal=${f.refusalCorrectness.passes}/${f.refusalCorrectness.repeats}`)
+      const score = f.scores[0]
+      if (score?.missedRequired?.length) {
+        this.logger.info(`    missed: ${score.missedRequired.join(' | ')}`)
+      }
+      if (score?.hitForbidden?.length) {
+        this.logger.info(`    said forbidden: ${score.hitForbidden.join(' | ')}`)
+      }
+      if (score?.leakage?.length) {
+        this.logger.info(`    narrated retrieval`)
+      }
+      if (f.answers[0]) {
+        const preview = f.answers[0].replace(/\s+/g, ' ').slice(0, 220)
+        this.logger.info(`    said: ${preview}${f.answers[0].length > 220 ? '…' : ''}`)
+      }
+    }
+    this.logger.info('')
+  }
+}
+
+const fmt = (v: number | null) => (v === null ? '  n/a' : v.toFixed(3))

admin/commands/eval/matrix.ts

@@ -0,0 +1,178 @@
+import { BaseCommand, flags } from '@adonisjs/core/ace'
+import type { CommandOptions } from '@adonisjs/core/types/ace'
+import type { GenerationAggregate } from '../../app/services/eval_generation_service.js'
+
+type Row = {
+  model: string
+  oracle: GenerationAggregate | null
+  e2e: GenerationAggregate | null
+  error?: string
+}
+
+/**
+ * Build the model capability table.
+ *
+ * This is the artifact that answers a GitHub issue. When a user reports "the
+ * AI gave me a bad answer", look up their model in this table:
+ *
+ *   - scoring at or near its row  -> the model is at its ceiling. The honest
+ *                                    answer is "run a larger model", not a bug.
+ *   - scoring well below its row  -> something is wrong with their config or
+ *                                    our code, and it is worth investigating.
+ *
+ * Without it, every quality concern is unfalsifiable.
+ *
+ *   node ace eval:matrix --models=llama3.2:latest,llama3:8b --limit=30
+ *   node ace eval:matrix --models=... --promote   # commit as the reference table
+ */
+export default class EvalMatrix extends BaseCommand {
+  static commandName = 'eval:matrix'
+  static description = 'Score several models side by side to build the capability reference table'
+
+  @flags.string({ description: 'Comma-separated Ollama model names' })
+  declare models: string
+
+  @flags.string({ description: 'Repeats per question (default: 1 — the matrix is already long)' })
+  declare repeats: string
+
+  @flags.string({ description: 'Limit to the first N goldens' })
+  declare limit: string
+
+  @flags.string({ description: 'Only run goldens carrying this tag' })
+  declare tag: string
+
+  @flags.boolean({ description: 'Write the table to tests/eval/baselines/<fingerprint>/matrix.json' })
+  declare promote: boolean
+
+  @flags.boolean({ description: 'Leave application debug logging on (very noisy)' })
+  declare debug: boolean
+
+  static options: CommandOptions = {
+    startApp: true,
+  }
+
+  async run() {
+    const { EvalCorpusService } = await import('#services/eval_corpus_service')
+    const { EvalGenerationService } = await import('#services/eval_generation_service')
+    const { quietLogging } = await import('../../app/utils/eval/quiet.js')
+    const { mkdir, writeFile } = await import('node:fs/promises')
+    const { dirname, join, resolve } = await import('node:path')
+
+    const corpusService = await this.app.container.make(EvalCorpusService)
+    const generationService = await this.app.container.make(EvalGenerationService)
+    const restoreLogging = quietLogging(this.debug)
+
+    try {
+      if (!this.models) {
+        this.logger.error('--models is required, e.g. --models=llama3.2:latest,llama3:8b')
+        this.exitCode = 1
+        return
+      }
+      const models = this.models.split(',').map((m) => m.trim()).filter(Boolean)
+
+      let goldens = await corpusService.loadGoldens()
+      if (this.tag) goldens = goldens.filter((g) => g.tags.includes(this.tag))
+      if (this.limit) goldens = goldens.slice(0, Number.parseInt(this.limit, 10))
+
+      const fingerprint = await corpusService.fingerprint()
+      const repeats = this.repeats ? Number.parseInt(this.repeats, 10) : 1
+      this.logger.info(
+        `Corpus ${fingerprint} · ${goldens.length} goldens · ${models.length} models · repeats=${repeats}`
+      )
+      this.logger.info('Running oracle and e2e for each model. This takes a while.')
+      this.logger.info('')
+
+      const rows: Row[] = []
+      for (const model of models) {
+        this.logger.info(`--- ${model} ---`)
+        const row: Row = { model, oracle: null, e2e: null }
+        try {
+          // Only oracle and e2e: those two are what the triage rule needs, and
+          // adding noretrieval would half again the runtime of an already long
+          // command for a number that does not change the verdict.
+          for (const mode of ['oracle', 'e2e'] as const) {
+            const started = Date.now()
+            const result = await generationService.run(goldens, { mode, model, repeats })
+            row[mode] = result.overall
+            this.logger.info(
+              `  ${mode.padEnd(6)} correctness=${fmt(result.overall.correctness)} refusal=${fmt(result.overall.refusalCorrectness)} (${((Date.now() - started) / 1000).toFixed(0)}s)`
+            )
+            if (result.overall.errors > 0) {
+              this.logger.warning(`  ${result.overall.errors} question(s) errored in ${mode}`)
+            }
+          }
+        } catch (error) {
+          row.error = error instanceof Error ? error.message : String(error)
+          this.logger.error(`  failed: ${row.error}`)
+        }
+        rows.push(row)
+      }
+
+      this.logger.info('')
+      this.printTable(rows)
+
+      if (this.promote) {
+        const path = resolve(join(process.cwd(), 'tests/eval/baselines', fingerprint, 'matrix.json'))
+        await mkdir(dirname(path), { recursive: true })
+        await writeFile(
+          path,
+          JSON.stringify(
+            {
+              corpusFingerprint: fingerprint,
+              createdAt: new Date().toISOString(),
+              goldens: goldens.length,
+              repeats,
+              tag: this.tag ?? null,
+              rows,
+            },
+            null,
+            2
+          )
+        )
+        this.logger.success(`Capability table written: ${path}`)
+        this.logger.info('Commit it — this is the reference a support triage compares against.')
+      }
+    } catch (error) {
+      this.logger.error(error instanceof Error ? error.message : String(error))
+      this.exitCode = 1
+    } finally {
+      restoreLogging()
+    }
+  }
+
+  private printTable(rows: Row[]) {
+    this.logger.info('=== Capability matrix ===')
+    this.logger.info('')
+    const width = Math.max(20, ...rows.map((r) => r.model.length + 2))
+    this.logger.info(
+      `${'model'.padEnd(width)}${'oracle'.padStart(8)}${'e2e'.padStart(8)}${'ret.cost'.padStart(10)}${'refusal'.padStart(9)}${'leakage'.padStart(9)}${'ground'.padStart(8)}`
+    )
+    for (const row of rows) {
+      if (row.error) {
+        this.logger.info(`${row.model.padEnd(width)}  ERROR: ${row.error}`)
+        continue
+      }
+      const cost =
+        row.oracle?.correctness !== null && row.oracle?.correctness !== undefined &&
+        row.e2e?.correctness !== null && row.e2e?.correctness !== undefined
+          ? row.oracle.correctness - row.e2e.correctness
+          : null
+      this.logger.info(
+        row.model.padEnd(width) +
+          fmt(row.oracle?.correctness ?? null).padStart(8) +
+          fmt(row.e2e?.correctness ?? null).padStart(8) +
+          (cost === null ? '   n/a' : `${cost >= 0 ? '+' : ''}${cost.toFixed(3)}`).padStart(10) +
+          fmt(row.e2e?.refusalCorrectness ?? null).padStart(9) +
+          fmt(row.e2e?.leakageRate ?? null).padStart(9) +
+          fmt(row.e2e?.groundedness?.mean ?? null).padStart(8)
+      )
+    }
+    this.logger.info('')
+    this.logger.info('oracle   = correctness with perfect context (the model\'s ceiling)')
+    this.logger.info('e2e      = correctness with real retrieval (what a user gets)')
+    this.logger.info('ret.cost = oracle - e2e (what imperfect retrieval costs this model)')
+    this.logger.info('leakage  = rate of narrating retrieval; lower is better')
+  }
+}
+
+const fmt = (v: number | null) => (v === null ? ' n/a' : v.toFixed(3))

admin/commands/eval/retrieval.ts

@@ -0,0 +1,364 @@
+import { BaseCommand, flags } from '@adonisjs/core/ace'
+import type { CommandOptions } from '@adonisjs/core/types/ace'
+import type { RetrievalAggregate } from '../../app/utils/eval/retrieval_metrics.js'
+
+/**
+ * Score NOMAD's retrieval against the frozen golden set.
+ *
+ * No chat model is involved, so this is deterministic and hardware-independent:
+ * a movement in these numbers is a code change, not a slow machine or an unlucky
+ * sample. It is the fast inner loop for anything touching chunking, embedding,
+ * thresholds, or reranking.
+ *
+ *   node ace eval:retrieval
+ *   node ace eval:retrieval --ablate                 # is the reranker helping?
+ *   node ace eval:retrieval --threshold=0.5          # sweep the Qdrant cutoff
+ *   node ace eval:retrieval --min-final-score=0.6    # sweep the post-rerank floor
+ *   node ace eval:retrieval --tag=multi-hop          # one slice only
+ */
+export default class EvalRetrieval extends BaseCommand {
+  static commandName = 'eval:retrieval'
+  static description = 'Score RAG retrieval against the golden set (deterministic, no chat model)'
+
+  @flags.string({ description: 'Chunks to retrieve per query (default: the production value)' })
+  declare topK: string
+
+  @flags.string({ description: 'Minimum similarity score (default: the production value)' })
+  declare threshold: string
+
+  @flags.string({
+    description:
+      'Post-rerank relevance floor; chunks below it are dropped (default: the production value). ' +
+      'Reads RAG_MIN_FINAL_SCORE, never the rag.minRelevance setting — see EvalRetrievalService.',
+  })
+  declare minFinalScore: string
+
+  @flags.string({
+    description:
+      'Run the retrieval relevance check on this model after the floor (default: off, keeping this tier model-free)',
+  })
+  declare judgeModel: string
+
+  @flags.boolean({ description: 'Also score the raw dense, reranked, and diversified orderings' })
+  declare ablate: boolean
+
+  @flags.string({
+    description:
+      'Comma-separated golden suites to run, by filename without .jsonl (default: core). ' +
+      'Extra suites are opt-in so they cannot move the numbers a baseline was recorded against.',
+  })
+  declare suite: string
+
+  @flags.string({ description: 'Only run goldens carrying this tag' })
+  declare tag: string
+
+  @flags.boolean({ description: 'Print each failing question and what it retrieved' })
+  declare verbose: boolean
+
+  @flags.boolean({ description: 'Leave application debug logging on (very noisy)' })
+  declare debug: boolean
+
+  @flags.boolean({ description: 'Write a JSON + Markdown report to tests/eval/reports/' })
+  declare report: boolean
+
+  @flags.string({
+    description:
+      "Write every golden's full pre-floor candidate pool (boosted + cosine scores) to this JSON path, " +
+      'for simulating a relevance gate offline',
+  })
+  declare dump: string
+
+  static options: CommandOptions = {
+    startApp: true,
+  }
+
+  async run() {
+    const { EvalCorpusService } = await import('#services/eval_corpus_service')
+    const { EvalRetrievalService } = await import('#services/eval_retrieval_service')
+    const { EvalReportService } = await import('#services/eval_report_service')
+    const { quietLogging } = await import('../../app/utils/eval/quiet.js')
+
+    const corpusService = await this.app.container.make(EvalCorpusService)
+    const retrievalService = await this.app.container.make(EvalRetrievalService)
+    const reportService = await this.app.container.make(EvalReportService)
+    const restoreLogging = quietLogging(this.debug)
+
+    try {
+      const chunks = await retrievalService.assertCorpusReady()
+      const fingerprint = await corpusService.fingerprint()
+      let goldens = await corpusService.loadGoldens(this.suite ? this.suite.split(',').map((s) => s.trim()) : undefined)
+
+      if (this.tag) {
+        goldens = goldens.filter((g) => g.tags.includes(this.tag))
+        if (goldens.length === 0) {
+          this.logger.error(`No goldens carry the tag "${this.tag}"`)
+          this.exitCode = 1
+          return
+        }
+      }
+
+      this.logger.info(`Corpus ${fingerprint} · ${chunks} chunks · ${goldens.length} goldens`)
+      this.logger.info('')
+
+      const started = Date.now()
+      const result = await retrievalService.run(goldens, {
+        topK: this.topK ? Number.parseInt(this.topK, 10) : undefined,
+        scoreThreshold: this.threshold ? Number.parseFloat(this.threshold) : undefined,
+        minFinalScore: this.minFinalScore ? Number.parseFloat(this.minFinalScore) : undefined,
+        ablate: this.ablate,
+        dump: Boolean(this.dump),
+        judgeModel: this.judgeModel || undefined,
+      })
+      const elapsed = ((Date.now() - started) / 1000).toFixed(1)
+
+      this.logger.info(
+        `Params: topK=${result.params.topK} threshold=${result.params.scoreThreshold} ` +
+          `minFinalScore=${result.params.minFinalScore}  (${elapsed}s)`
+      )
+      if (result.judge) {
+        const j = result.judge
+        this.logger.info(
+          `Relevance check: ${j.model} · ${j.calls} calls · rejected ${j.rejected} chunk(s) · ` +
+            `${j.failures} failed open · mean ${Math.round(j.meanMs)}ms max ${Math.round(j.maxMs)}ms`
+        )
+      }
+      this.logger.info('')
+      this.printAggregate('OVERALL', result.overall, result.params.kValues)
+
+      if (result.unresolvedChunks > 0) {
+        // Every retrieved chunk should belong to the eval corpus. Anything else
+        // means the collection filter leaked and the numbers above describe a
+        // corpus nobody chose.
+        this.logger.error(
+          `${result.unresolvedChunks} retrieved chunk(s) did not belong to the eval corpus — the collection filter leaked.`
+        )
+        this.exitCode = 1
+      }
+
+      this.printThresholdGuidance(result.overall)
+
+      if (result.ablation) {
+        this.logger.info('')
+        this.logger.info('=== Stage ablation (does each heuristic earn its place?) ===')
+        const k = result.params.kValues.includes(5) ? 5 : result.params.kValues[0]
+        const row = (name: string, agg: RetrievalAggregate) =>
+          this.logger.info(
+            `  ${name.padEnd(14)} recall@${k}=${fmt(agg.recall[k])}  ndcg@${k}=${fmt(agg.ndcg[k])}  mrr=${fmt(agg.mrr)}  prec@${k}=${fmt(agg.precision[k])}`
+          )
+        row('dense only', result.ablation.dense)
+        row('+ rerank', result.ablation.reranked)
+        row('+ diversity', result.ablation.diversified)
+        this.explainAblation(result.ablation, k)
+      }
+
+      this.logger.info('')
+      this.logger.info('=== By tag ===')
+      for (const [tag, agg] of Object.entries(result.byTag).sort()) {
+        const k = result.params.kValues.includes(5) ? 5 : result.params.kValues[0]
+        // A refusal-only tag has no recall to report; what it has is the leak rate,
+        // which is the number #1341 is about.
+        const leak =
+          agg.nonEmptyRateOnRefusal === null ? '' : `  leaked=${pct(agg.nonEmptyRateOnRefusal)}`
+        this.logger.info(
+          `  ${tag.padEnd(20)} n=${String(agg.cases).padStart(3)}  recall@${k}=${fmt(agg.recall[k])}  ndcg@${k}=${fmt(agg.ndcg[k])}${leak}`
+        )
+      }
+
+      const k = result.params.kValues.includes(5) ? 5 : result.params.kValues[0]
+      const misses = result.cases.filter((c) => !c.expectRefusal && (c.recall[k] ?? 1) < 1)
+      this.logger.info('')
+      this.logger.info(`${misses.length} of ${result.overall.answerable} answerable questions missed at k=${k}`)
+
+      if (this.verbose && misses.length > 0) {
+        this.logger.info('')
+        this.logger.info('=== Misses ===')
+        for (const miss of misses) {
+          this.logger.info(`  ${miss.id}`)
+          this.logger.info(`    wanted:   ${miss.relevantDocIds.join(', ') || '(none)'}`)
+          this.logger.info(`    retrieved: ${miss.retrievedDocIds.join(', ') || '(nothing)'}`)
+        }
+      } else if (misses.length > 0) {
+        this.logger.info('Re-run with --verbose to see which questions and what they retrieved.')
+      }
+
+      // The other failure: a question the corpus cannot answer that retrieved
+      // something anyway. Each of these becomes a citation the user sees.
+      const leaks = result.cases.filter((c) => c.expectRefusal && !c.empty)
+      if (leaks.length > 0) {
+        this.logger.info('')
+        this.logger.info(`${leaks.length} out-of-corpus question(s) retrieved something anyway`)
+        if (this.verbose) {
+          this.logger.info('')
+          this.logger.info('=== Leaks ===')
+          for (const leak of leaks) {
+            this.logger.info(`  ${leak.id}`)
+            const top = leak.top
+              ? `  (top final=${leak.top.score.toFixed(3)} semantic=${leak.top.semanticScore?.toFixed(3) ?? 'n/a'})`
+              : ''
+            this.logger.info(`    retrieved: ${leak.retrievedDocIds.join(', ')}${top}`)
+          }
+        }
+      }
+
+      if (this.dump && result.pools) {
+        const { writeFile } = await import('node:fs/promises')
+        await writeFile(this.dump, JSON.stringify({ fingerprint, params: result.params, pools: result.pools }, null, 2))
+        this.logger.info('')
+        this.logger.success(`Candidate pools written: ${this.dump}`)
+      }
+
+      if (this.report) {
+        const meta = await reportService.buildMeta('retrieval', fingerprint, {
+          ...result.params,
+          tag: this.tag ?? null,
+        })
+        const doc = reportService.fromRetrieval(meta, result)
+        const slug = `retrieval-${meta.createdAt.replace(/[:.]/g, '-')}`
+        const path = await reportService.write(doc, slug, renderRetrievalMarkdown(doc, result, misses))
+        this.logger.info('')
+        this.logger.success(`Report written: ${path}`)
+      }
+    } catch (error) {
+      this.logger.error(error instanceof Error ? error.message : String(error))
+      this.exitCode = 1
+    } finally {
+      restoreLogging()
+    }
+  }
+
+  private printAggregate(label: string, agg: RetrievalAggregate, kValues: number[]) {
+    this.logger.info(`=== ${label} (${agg.answerable} answerable of ${agg.cases}) ===`)
+    const row = (name: string, values: Record<number, number | null>) =>
+      this.logger.info(
+        `  ${name.padEnd(10)} ${kValues.map((k) => `@${k}=${fmt(values[k])}`).join('  ')}`
+      )
+    row('recall', agg.recall)
+    row('hit rate', agg.hitRate)
+    row('precision', agg.precision)
+    row('ndcg', agg.ndcg)
+    this.logger.info(`  mrr        ${fmt(agg.mrr)}`)
+  }
+
+  /**
+   * Turn the score distributions into an actual recommendation. The raw
+   * percentiles are the evidence; this is the reading of them, which is what
+   * the threshold constants have never had.
+   *
+   * Printed per axis, because the two cutoffs filter different numbers:
+   * `--threshold` is applied by Qdrant to the raw cosine score, and
+   * `--min-final-score` is applied after reranking. Reading one off the other's
+   * percentiles is wrong by the reranker's boost factor.
+   */
+  private printThresholdGuidance(agg: RetrievalAggregate) {
+    this.logger.info('')
+    this.logger.info('=== Score separation (how to calibrate the cutoffs) ===')
+    this.printAxis(
+      'semantic, pre-rerank  — calibrates --threshold',
+      agg.relevantScores,
+      agg.irrelevantScores
+    )
+    this.printAxis(
+      'final, post-rerank    — calibrates --min-final-score',
+      agg.relevantFinalScores,
+      agg.irrelevantFinalScores
+    )
+    if (agg.emptyRateOnAnswerable !== null && agg.emptyRateOnAnswerable > 0) {
+      this.logger.warning(
+        `  ${pct(agg.emptyRateOnAnswerable)} of answerable questions retrieved nothing — threshold may be too high.`
+      )
+    }
+    if (agg.nonEmptyRateOnRefusal !== null && agg.nonEmptyRateOnRefusal > 0) {
+      this.logger.warning(
+        `  ${pct(agg.nonEmptyRateOnRefusal)} of out-of-corpus questions retrieved something anyway — that context is what produces confident wrong answers.`
+      )
+    }
+  }
+
+  /** One axis of the separation table, plus the reading of it. */
+  private printAxis(
+    label: string,
+    rel: RetrievalAggregate['relevantScores'],
+    irr: RetrievalAggregate['irrelevantScores']
+  ) {
+    if (!rel && !irr) return
+    this.logger.info(`  ${label}`)
+    const row = (name: string, d: NonNullable<typeof rel>) =>
+      this.logger.info(
+        `    ${name.padEnd(18)} n=${d.count} min=${d.min.toFixed(3)} p10=${d.p10.toFixed(3)} median=${d.median.toFixed(3)} p90=${d.p90.toFixed(3)}`
+      )
+    if (rel) row('relevant chunks', rel)
+    if (irr) row('irrelevant chunks', irr)
+    if (rel && irr) {
+      if (rel.p10 > irr.p90) {
+        this.logger.success(
+          `    Clean separation: a cutoff between ${irr.p90.toFixed(3)} and ${rel.p10.toFixed(3)} splits them.`
+        )
+      } else {
+        this.logger.warning(
+          `    Overlapping: relevant p10 (${rel.p10.toFixed(3)}) sits below irrelevant p90 (${irr.p90.toFixed(3)}); ` +
+            `a cutoff near ${rel.min.toFixed(3)} keeps every relevant chunk seen here.`
+        )
+      }
+    }
+  }
+
+  private explainAblation(ablation: { dense: RetrievalAggregate; reranked: RetrievalAggregate; diversified: RetrievalAggregate }, k: number) {
+    const verdict = (name: string, before: number | null, after: number | null) => {
+      if (before === null || after === null) return
+      const delta = after - before
+      if (Math.abs(delta) < 1e-6) {
+        this.logger.warning(`  ${name} changed nothing at k=${k} — it is complexity with no measured benefit.`)
+      } else if (delta < 0) {
+        this.logger.warning(`  ${name} made ndcg@${k} worse by ${Math.abs(delta).toFixed(4)}.`)
+      } else {
+        this.logger.success(`  ${name} improved ndcg@${k} by ${delta.toFixed(4)}.`)
+      }
+    }
+    verdict('Reranking', ablation.dense.ndcg[k], ablation.reranked.ndcg[k])
+    verdict('Source diversity', ablation.reranked.ndcg[k], ablation.diversified.ndcg[k])
+  }
+}
+
+const fmt = (v: number | null) => (v === null ? '  n/a' : v.toFixed(3))
+const pct = (v: number) => `${(v * 100).toFixed(0)}%`
+
+/**
+ * The human-readable half of a report. Leads with the misses, because when a
+ * number moves the next question is always "which questions?" and a table of
+ * aggregates cannot answer it.
+ */
+function renderRetrievalMarkdown(doc: any, result: any, misses: any[]): string {
+  const lines: string[] = ['# Retrieval eval', '']
+  lines.push(`- corpus: \`${doc.meta.corpusFingerprint}\``)
+  lines.push(`- commit: \`${doc.meta.gitSha ?? 'unknown'}\`${doc.meta.gitDirty ? ' (dirty tree)' : ''}`)
+  lines.push(`- when: ${doc.meta.createdAt}`)
+  lines.push(
+    `- params: topK=${result.params.topK} threshold=${result.params.scoreThreshold} minFinalScore=${result.params.minFinalScore}`
+  )
+  lines.push('')
+  lines.push('## Metrics', '')
+  lines.push('| metric | value |', '|---|---:|')
+  for (const [name, value] of Object.entries(doc.metrics)) {
+    lines.push(`| ${name} | ${value === null ? 'n/a' : (value as number).toFixed(4)} |`)
+  }
+  lines.push('')
+  lines.push(`## Misses (${misses.length})`, '')
+  if (misses.length === 0) {
+    lines.push('None.')
+  } else {
+    for (const m of misses) {
+      lines.push(`### \`${m.id}\``)
+      lines.push(`- wanted: ${m.relevantDocIds.join(', ') || '_none_'}`)
+      lines.push(`- retrieved: ${m.retrievedDocIds.join(', ') || '_nothing_'}`)
+      lines.push('')
+    }
+  }
+  lines.push('## By tag', '')
+  lines.push('| tag | recall@5 | ndcg@5 |', '|---|---:|---:|')
+  for (const [tag, metrics] of Object.entries(doc.byTag).sort()) {
+    const m = metrics as Record<string, number | null>
+    const cell = (v: number | null | undefined) => (v === null || v === undefined ? 'n/a' : v.toFixed(4))
+    lines.push(`| ${tag} | ${cell(m['recall@5'])} | ${cell(m['ndcg@5'])} |`)
+  }
+  return lines.join('\n') + '\n'
+}

admin/commands/queue/work.ts

@@ -14,6 +14,7 @@ import { AppAutoUpdateJob } from '#jobs/app_auto_update_job'
 import { ContentAutoUpdateJob } from '#jobs/content_auto_update_job'
 import { DownloadDrugDataJob } from '#jobs/download_drug_data_job'
 import { IngestDrugDataJob } from '#jobs/ingest_drug_data_job'
+import { stallOptionsForQueue, type StallOptions } from '../../app/utils/queue_stall_options.js'
 
 export default class QueueWork extends BaseCommand {
   static commandName = 'queue:work'
@@ -68,11 +69,10 @@ export default class QueueWork extends BaseCommand {
         {
           connection: queueConfig.connection,
           concurrency: this.getConcurrencyForQueue(queueName),
-          // lockDuration/maxStalledCount are per-queue. Non-drug queues keep
-          // the existing default (300000, BullMQ's default maxStalledCount).
-          // The drug download/ingest queues are NEW per-queue overrides
-          // (1_800_000 / 3) — see getStallOptionsForQueue — not a change to
-          // the default applied to every other queue.
+          // lockDuration/maxStalledCount are per-queue — see
+          // getStallOptionsForQueue for which queues override the default
+          // (300000, BullMQ's default maxStalledCount) and why. Queues without
+          // an entry there are unaffected.
           lockDuration: stall.lockDuration,
           ...(stall.maxStalledCount !== undefined
             ? { maxStalledCount: stall.maxStalledCount }
@@ -203,26 +203,17 @@ export default class QueueWork extends BaseCommand {
   }
 
   /**
-   * Per-queue BullMQ stall-recovery options.
-   *
-   * Every queue except the two drug queues keeps the branch default
-   * (lockDuration 300000, and BullMQ's default maxStalledCount of 1 — left
-   * unset). The drug download/ingest queues are the ONLY per-queue override:
-   * each part is a long single stream (a ~150 MB resumable HTTP pull, then an
-   * unzip + JSON-stream ingest at concurrency 1), so a longer lock plus a
-   * higher stalled tolerance keeps a transient lock-renewal miss from killing
-   * the continuation chain ("job stalled more than allowable limit").
+   * Per-queue BullMQ stall-recovery options. See `queue_stall_options.ts` for
+   * the reasoning behind each queue's entry; the decision itself is kept pure
+   * and unit-tested there.
    */
-  private getStallOptionsForQueue(
-    queueName: string
-  ): { lockDuration: number; maxStalledCount?: number } {
-    if (
-      queueName === DownloadDrugDataJob.queue ||
-      queueName === IngestDrugDataJob.queue
-    ) {
-      return { lockDuration: 1_800_000, maxStalledCount: 3 }
-    }
-    return { lockDuration: 300000 }
+  private getStallOptionsForQueue(queueName: string): StallOptions {
+    return stallOptionsForQueue(queueName, {
+      embedFile: EmbedFileJob.queue,
+      download: RunDownloadJob.queue,
+      drugDownload: DownloadDrugDataJob.queue,
+      drugIngest: IngestDrugDataJob.queue,
+    })
   }
 
   /**

admin/constants/kb_collections.ts

@@ -21,6 +21,21 @@ export const KB_COLLECTIONS = [
 
 export type KbCollection = (typeof KB_COLLECTIONS)[number]
 
+/**
+ * Reserved collection tag for the developer evaluation corpus (`ace eval:*`).
+ *
+ * The eval fixtures live in the same Qdrant collection as everything else —
+ * NOMAD "collections" are a payload tag, not separate Qdrant collections — so
+ * this tag is what keeps them out of the user's Knowledge Base UI and out of
+ * ordinary chat retrieval. Every read path that enumerates user content filters
+ * it out, and `sanitizeCollectionName` refuses to mint it, so a user cannot
+ * create a colliding tag by accident.
+ *
+ * The leading/trailing underscores are deliberate: they make the tag obviously
+ * internal if it ever does surface in a log or a raw Qdrant query.
+ */
+export const KB_EVAL_COLLECTION = '__nomad_eval__'
+
 /** Hard cap on a user-created tag's length, enforced client- and server-side. */
 export const KB_COLLECTION_NAME_MAX_LENGTH = 40
 
@@ -36,5 +51,9 @@ export function sanitizeCollectionName(raw: string | null | undefined): string |
   if (!raw) return null
   const trimmed = raw.trim().toLowerCase()
   if (!trimmed) return null
+  // The eval tag is reserved. Treating a collision as "uncategorized" rather
+  // than throwing keeps this a pure normalizer, and the user's documents stay
+  // visible in the KB instead of vanishing into a hidden internal collection.
+  if (trimmed === KB_EVAL_COLLECTION) return null
   return trimmed.slice(0, KB_COLLECTION_NAME_MAX_LENGTH)
 }

admin/constants/kv_store.ts

@@ -11,7 +11,15 @@ export const SETTINGS_KEYS: KVStoreKey[] = [
     'ai.remoteOllamaUrl',
     'ai.ollamaFlashAttention',
     'ai.autoThinking',
+    'ai.tasksModel',
+    'ai.contextWindow',
+    'ai.keepAlive',
+    'ai.responseStyle',
+    'ai.amdHsaOverride',
     'rag.defaultIngestPolicy',
+    'rag.enabled',
+    'rag.minRelevance',
+    'rag.relevanceCheck',
     'autoUpdate.enabled',
     'autoUpdate.windowStart',
     'autoUpdate.windowEnd',

admin/constants/link_tile_colors.ts

@@ -0,0 +1,81 @@
+/**
+ * The colors a user can give a dashboard link tile.
+ *
+ * Drawn from the brand palette rather than arbitrary hex, so a tile always sits
+ * on-theme next to the managed app cards. The default white-on-green contrast
+ * read as jarring on the dashboard, which is why this exists.
+ *
+ * Tiles stay visually distinct from managed apps regardless of color: the
+ * dashed border and the external-link marker are what carry that, not the hue.
+ * The tint is deliberately light so the distinction survives every option.
+ *
+ * `bg` is the tile tint and `swatch` is the same color at full strength. They
+ * differ on purpose: a 10% tint reads well across a 190px card but makes six
+ * 28px picker swatches indistinguishable from each other, so the picker shows
+ * the color solid while the tile stays subtle.
+ *
+ * Class names are written out in full and never interpolated. Tailwind scans
+ * source text for literal class strings, so a computed name like
+ * `bg-desert-${id}/10` would be silently dropped from the build.
+ */
+export const LINK_TILE_COLORS = [
+  {
+    id: 'green',
+    swatch: 'bg-desert-green',
+    label: 'Green',
+    border: 'border-desert-green/70',
+    bg: 'bg-desert-green/10',
+    marker: 'text-desert-green',
+  },
+  {
+    id: 'olive',
+    swatch: 'bg-desert-olive',
+    label: 'Olive',
+    border: 'border-desert-olive/70',
+    bg: 'bg-desert-olive/10',
+    marker: 'text-desert-olive',
+  },
+  {
+    id: 'orange',
+    swatch: 'bg-desert-orange',
+    label: 'Orange',
+    border: 'border-desert-orange/70',
+    bg: 'bg-desert-orange/10',
+    marker: 'text-desert-orange',
+  },
+  {
+    id: 'red',
+    swatch: 'bg-desert-red',
+    label: 'Red',
+    border: 'border-desert-red/70',
+    bg: 'bg-desert-red/10',
+    marker: 'text-desert-red',
+  },
+  {
+    id: 'sand',
+    swatch: 'bg-desert-sand',
+    label: 'Sand',
+    border: 'border-desert-sand/70',
+    bg: 'bg-desert-sand/20',
+    marker: 'text-desert-stone-dark',
+  },
+  {
+    id: 'stone',
+    swatch: 'bg-desert-stone',
+    label: 'Stone',
+    border: 'border-desert-stone/70',
+    bg: 'bg-desert-stone/10',
+    marker: 'text-desert-stone-dark',
+  },
+] as const
+
+export type LinkTileColorId = (typeof LINK_TILE_COLORS)[number]['id']
+
+export const DEFAULT_LINK_TILE_COLOR: LinkTileColorId = 'green'
+
+export const LINK_TILE_COLOR_IDS: readonly string[] = LINK_TILE_COLORS.map((c) => c.id)
+
+/** Resolve a stored id, falling back to the default for anything unknown. */
+export function linkTileColor(id?: string | null) {
+  return LINK_TILE_COLORS.find((c) => c.id === id) ?? LINK_TILE_COLORS[0]
+}

admin/constants/link_tile_icons.ts

@@ -0,0 +1,72 @@
+/**
+ * The icons offered when creating a dashboard link tile: a curated 36, laid out
+ * six across and six down.
+ *
+ * Every name here must exist in the `DynamicIcon` registry (`inertia/lib/icons.ts`).
+ * `DynamicIcon` renders nothing at all for a name it does not know, so an icon
+ * that is not in that registry produces a tile with a silent hole where its icon
+ * should be. Shared by the picker and the server-side validator so the two can
+ * never disagree about what is selectable.
+ *
+ * Chosen for "a thing on my network": storage, media, documents, tooling,
+ * infrastructure. Not a general icon browser. The point of a fixed set is that
+ * the user picks from something that fits on one screen.
+ */
+export const LINK_TILE_ICONS = [
+  // Infrastructure
+  'IconServer',
+  'IconDatabase',
+  'IconWorld',
+  'IconWifi',
+  'IconAntenna',
+  'IconBroadcast',
+
+  // Machines and containers
+  'IconBrandDocker',
+  'IconCpu',
+  'IconHome',
+  'IconBox',
+  'IconFolderOpen',
+  'IconMovie',
+
+  // Reading and reference
+  'IconBooks',
+  'IconBook',
+  'IconLibrary',
+  'IconMap',
+  'IconNotes',
+  'IconFileDescription',
+
+  // Tools and admin
+  'IconCode',
+  'IconTool',
+  'IconSettings',
+  'IconShieldCheck',
+  'IconShieldLock',
+  'IconRobot',
+
+  // Subjects
+  'IconBrain',
+  'IconPlant',
+  'IconChefHat',
+  'IconSchool',
+  'IconStethoscope',
+  'IconSearch',
+
+  // Transfer and misc
+  'IconCloudDownload',
+  'IconCloudUpload',
+  'IconLogs',
+  'IconPlayerPlay',
+  'IconExternalLink',
+  'IconWand',
+] as const
+
+export type LinkTileIconName = (typeof LINK_TILE_ICONS)[number]
+
+/** The icon a tile gets when none was chosen, or when a stored name is no longer offered. */
+export const DEFAULT_LINK_TILE_ICON: LinkTileIconName = 'IconExternalLink'
+
+export function isLinkTileIcon(value: unknown): value is LinkTileIconName {
+  return typeof value === 'string' && (LINK_TILE_ICONS as readonly string[]).includes(value)
+}

admin/constants/ollama.ts

@@ -1,4 +1,4 @@
-import { NomadOllamaModel } from '../types/ollama.js'
+import { NomadOllamaModel, ResponseStyle, SamplerProfile } from '../types/ollama.js'
 
 /**
  * Fallback basic recommended Ollama models in case fetching from the service fails.
@@ -62,10 +62,25 @@ export const FALLBACK_RECOMMENDED_OLLAMA_MODELS: NomadOllamaModel[] = [
   },
 ]
 
-export const DEFAULT_QUERY_REWRITE_MODEL = 'qwen2.5:3b' // default to qwen2.5 for query rewriting with good balance of text task performance and resource usage
-
 export const EMBEDDING_MODEL_NAME = 'nomic-embed-text:v1.5'
 
+/**
+ * Server-side context floor set as `OLLAMA_CONTEXT_LENGTH` on the nomad_ollama
+ * container.
+ *
+ * Ollama defaults to 4096 tokens on machines under 24GB VRAM and silently
+ * truncates anything longer — no error, no warning, the model simply never sees
+ * the start of the conversation. This raises the floor for every request that
+ * can't carry its own `num_ctx`, which notably includes the whole
+ * OpenAI-compatible path (that endpoint has no context-size field at all).
+ *
+ * Deliberately conservative rather than clever: it is allocated at model load on
+ * whatever hardware NOMAD happens to be running, so it has to be affordable on a
+ * small box. Per-model, hardware-aware sizing happens per request via
+ * ContextWindowResolver, which can go well above this.
+ */
+export const DEFAULT_OLLAMA_CONTEXT_LENGTH = 8192
+
 /**
  * Adaptive RAG context limits based on model size.
  * Smaller models get overwhelmed with too much context, so we cap it.
@@ -76,6 +91,228 @@ export const RAG_CONTEXT_LIMITS: { maxParams: number; maxResults: number; maxTok
   { maxParams: Infinity, maxResults: 5, maxTokens: 0 }, // 13B+ (no cap)
 ]
 
+/**
+ * Retrieval defaults for the chat pipeline. These were previously inline
+ * literals at the `searchSimilarDocuments` call site in OllamaController, which
+ * made them impossible to sweep or record in a report. They are named here so
+ * the pipeline and the eval harness read the same numbers.
+ */
+export const RAG_DEFAULT_TOP_K = 5
+export const RAG_DEFAULT_SCORE_THRESHOLD = 0.3
+
+/**
+ * Relevance floor applied to the post-rerank score. Chunks below it are
+ * dropped, and when nothing clears it no context block is injected at all.
+ *
+ * Retrieval used to have exactly one cutoff — RAG_DEFAULT_SCORE_THRESHOLD,
+ * applied by Qdrant to the raw cosine score. nomic scores unrelated text well
+ * above 0.3, so every question retrieved something and the system prompt was
+ * left to compensate in prose: "silently judge whether the context genuinely
+ * addresses the question". That is a relevance classification, handed to a 3B
+ * model, that a number should have decided.
+ *
+ * RAG_DEFAULT_SCORE_THRESHOLD deliberately stays low. It is the candidate net,
+ * and pruning it would starve the reranker (worth +0.0125 ndcg@5 on the golden
+ * set) and, later, the sparse leg of hybrid retrieval. This is the decisive
+ * cutoff instead, applied once at the end where the score means the most.
+ *
+ * Calibrated with `node ace eval:retrieval --min-final-score=<x>` against the
+ * 99-golden set at corpus fingerprint 51e642964facf251:
+ *
+ *     floor   recall@5   precision@5   emptyOnAnswerable   nonEmptyOnRefusal
+ *     0.00      0.991        0.221            0%                 100%
+ *     0.55      0.991        0.289            0%                  80%
+ *     0.62      0.991        0.524            0%                  60%
+ *     0.65      0.991        0.659            0%                  60%
+ *     0.66      0.991        0.695            0%                  60%
+ *     0.67      0.972        0.704            1%                  60%
+ *     0.72      0.888        0.796            6%                  60%
+ *
+ * 0.66 is the last value that costs no recall, and 0.62 is where the refusal
+ * metric reaches its floor. Everything between them buys precision only, so the
+ * default sits at 0.62 rather than at the edge: this corpus is 28 documents and
+ * a real ZIM will not score identically, and the two failure modes are not
+ * symmetric. A dropped relevant chunk means no answer to a question the corpus
+ * *can* answer; an extra irrelevant chunk means noise the model mostly ignores
+ * — measured correctness was fine at 0.221 precision. Buy margin, not precision.
+ *
+ * Note this is a higher cutoff than the same sweep supports on the *semantic*
+ * axis, where recall starts falling at 0.60. That gap is the reranker earning
+ * its place: it separates relevant from irrelevant better than raw cosine does,
+ * so the floor can sit higher without cutting real answers.
+ *
+ * Note the honest ceiling on the refusal metric: three of the five refusal
+ * goldens are tagged `adversarial` and are near-corpus by construction ("what
+ * is the TR-88 pump's warranty period?" against a corpus that documents the
+ * TR-88 but not its warranty). Retrieving that document is *correct*; declining
+ * to answer from it is the generation tier's job, scored by refusalCorrectness.
+ * So nonEmptyRateOnRefusal bottoms out at 0.6 here, and a cutoff tuned to drive
+ * it lower is cutting real documents.
+ *
+ * What this floor cannot do (#1341). With general-prose distractors added to the
+ * corpus (fingerprint b4ebe5dce8699f5d: Gutenberg excerpts, the Python tutorial,
+ * MDN), the `off_topic` suite measures:
+ *
+ *     floor   core recall@5   off-topic questions that retrieve something
+ *     0.62        0.991                  67%   (22% without the distractors)
+ *     0.65        0.991                  33%
+ *     0.69        0.934                  11%
+ *
+ * The weakest correct top hit scores 0.666 while "write me a haiku about autumn
+ * leaves" pulls Walden at 0.705, so no value here separates the two, and a
+ * better-stocked library only makes it worse. 0.65 halves the leaks at no
+ * measured recall cost but sits 0.016 under that weakest hit, which is not the
+ * margin this default was chosen for. The coherent-but-off-topic case is left to
+ * the opt-in relevance check (app/utils/relevance_judge.ts). A cosine floor
+ * alongside this one was also tried and loses recall faster than it cuts leaks:
+ * the reranker's boosts lift relevant chunks more than irrelevant ones.
+ *
+ * Unset `rag.minRelevance` resolves to this value; see app/utils/rag_relevance.ts.
+ */
+export const RAG_MIN_FINAL_SCORE = 0.62
+
+/**
+ * Preset relevance floors offered in Settings > Models, and the labels for them.
+ * The stored setting is the number, not the label, so retuning these presets
+ * later cannot invalidate a value someone has already saved.
+ */
+export const RAG_MIN_RELEVANCE_PRESETS = [
+  { value: 0, label: 'Off — use every passage retrieved' },
+  { value: 0.55, label: 'Lenient' },
+  { value: RAG_MIN_FINAL_SCORE, label: 'Balanced (recommended)' },
+  // The last floor that costs no recall on the golden set. Stricter than this
+  // starts dropping answers, which is a choice to offer, not one to default to.
+  { value: 0.66, label: 'Strict' },
+] as const
+
+/**
+ * Where the per-turn retrieved-context block sits in the prompt.
+ *
+ * `tail` places it immediately before the current question, so
+ * [system][history] stays byte-identical from turn to turn and the backend can
+ * reuse its KV cache for the whole conversation. `front` is the historical
+ * placement — a system message ahead of all history — which changes content
+ * every turn and therefore invalidates the cached prefix behind it, making
+ * follow-ups progressively slower as the conversation grows.
+ *
+ * Measured on llama3:8b over a 14-turn conversation (~400-token retrieved block
+ * per turn), prefill time for the same prompt:
+ *
+ *     turn  1    front  214 ms    tail  214 ms
+ *     turn  7    front  347 ms    tail  225 ms
+ *     turn 14    front  539 ms    tail  231 ms
+ *
+ * front grows about 25 ms per turn; tail grows about 1 ms. By turn 14 that is
+ * 2.3x the time-to-first-token, and the gap keeps widening — the familiar
+ * "first answer fast, follow-ups get slower" complaint.
+ *
+ * Caveat worth knowing before trusting this: the saving is proportional to how
+ * much of the prompt is *stable*. With a very large retrieved block and a short
+ * history the block dominates, must be reprocessed either way, and the two
+ * placements measure the same. Tail placement never loses, but it only clearly
+ * wins once a conversation has accumulated history — which is exactly when
+ * responsiveness starts to matter.
+ *
+ * Both are kept so `eval:generation` can measure the answer-quality difference
+ * rather than the choice resting on the cache argument alone. Flip to 'front'
+ * to reproduce pre-change behaviour.
+ */
+export const RAG_PLACEMENT: 'tail' | 'front' = 'tail'
+
+/**
+ * Token cap on the query-rewrite call. A rewrite is one short sentence; the
+ * prompt already asks for under 150 words. Without a cap a small model can
+ * ramble, and every one of those tokens is latency the user waits through
+ * before retrieval even starts. Sized to leave room for the JSON envelope the
+ * rewrite is now grammar-constrained to emit — a truncated object parses to
+ * nothing and costs the turn its rewrite entirely.
+ */
+export const QUERY_REWRITE_MAX_TOKENS = 160
+
+/**
+ * Token cap on the retrieval relevance check. The answer is `{"on_topic": true}`,
+ * under ten tokens; the rest is headroom, since a truncated verdict is discarded
+ * and the turn keeps its chunks unjudged.
+ */
+export const RELEVANCE_CHECK_MAX_TOKENS = 32
+
+/**
+ * How long Ollama keeps a chat model — and its KV cache — resident after a
+ * request.
+ *
+ * Ollama's default is 5 minutes, which is comfortably shorter than the time a
+ * user spends reading an answer and typing a follow-up. Evicting in that gap
+ * throws away the cached prefix and forces a cold reload plus a full re-prefill
+ * on the next message, which is the single most obvious "why is it slow again?"
+ * moment in a conversation. Overridable via the `ai.keepAlive` setting.
+ */
+export const DEFAULT_KEEP_ALIVE = '15m'
+
+/**
+ * Sampler settings per Response Style.
+ *
+ * Chat used to send none of these, so every turn ran at whatever the backend
+ * picked: temperature 0.8, top_p 0.9, top_k 40 and no min_p on a bare Ollama
+ * model. On a heavily quantized 1-3B model, which is NOMAD's realistic target,
+ * those defaults are what produces the wandering, repetitive output that reads
+ * as "the local model is bad".
+ *
+ * Three things are worth knowing before retuning these numbers.
+ *
+ * min_p is the actual missing lever, not temperature. It is a floor relative to
+ * the top token's probability, so it truncates hard when the model is confident
+ * and barely at all when it isn't. top_p can't do that: a fixed nucleus keeps
+ * the same share of mass either way, which is why a flat distribution at the end
+ * of a long generation is where small models start to wander. With min_p doing
+ * the work, top_p is left at 1.0 rather than stacked on top of it, and a higher
+ * temperature stays safe.
+ *
+ * repeat_penalty moves with the style instead of sitting at Ollama's 1.1. The
+ * penalty applies over the last 64 tokens with no idea what a token means, and
+ * a grounded answer out of this corpus legitimately repeats a part number, a
+ * band designator or a drug name inside that window. So the factual style
+ * relaxes it and the creative style, which is where genuine loops happen,
+ * tightens it.
+ *
+ * `off` is not a preset, it is the absence of one: no options are sent and the
+ * backend decides, exactly as it did before this setting existed.
+ */
+export const SAMPLER_PRESETS: Record<Exclude<ResponseStyle, 'off'>, SamplerProfile> = {
+  auto: { temperature: 0.6, topP: 1.0, topK: 40, minP: 0.05, repeatPenalty: 1.1, compatTopP: 0.9 },
+  focused: {
+    temperature: 0.2,
+    topP: 1.0,
+    topK: 40,
+    minP: 0.1,
+    repeatPenalty: 1.05,
+    compatTopP: 0.85,
+  },
+  creative: {
+    temperature: 1.0,
+    topP: 1.0,
+    topK: 100,
+    minP: 0.03,
+    repeatPenalty: 1.15,
+    compatTopP: 0.95,
+  },
+}
+
+/** Unset `ai.responseStyle` resolves to this; see app/utils/response_style.ts. */
+export const DEFAULT_RESPONSE_STYLE: ResponseStyle = 'auto'
+
+/**
+ * The Response Style options offered in Settings > Models, and their labels.
+ *
+ * Ordered least to most adventurous, with the default first, matching how the
+ * Context Window and Knowledge Base Relevance selects above it are ordered.
+ */
+export const RESPONSE_STYLE_PRESETS: { value: ResponseStyle; label: string }[] = [
+  { value: 'auto', label: 'Auto (recommended)' },
+  { value: 'focused', label: 'Focused (factual, repeatable)' },
+  { value: 'creative', label: 'Creative (varied, exploratory)' },
+  { value: 'off', label: 'Backend defaults (send nothing)' },
+]
+
 export const SYSTEM_PROMPTS = {
   default: `
  Format all responses using markdown for better readability. Vanilla markdown or GitHub-flavored markdown is preferred.
@@ -86,27 +323,24 @@ export const SYSTEM_PROMPTS = {
  - Use tables when presenting structured data.
 `,
   rag_context: (context: string) => `
-Information has been retrieved from the NOMAD knowledge base that MAY be relevant to the
-user's question. It was selected by automated similarity search, which is imperfect — some
-or all of it may be unrelated to what the user actually asked.
+Information has been retrieved from the NOMAD knowledge base for the user's question. It
+cleared a relevance filter before reaching you, so it is likely on topic — but it was
+selected by automated search, so it may still not contain the specific detail asked for.
 
 [Knowledge Base Context]
 ${context}
 
 HOW TO ANSWER:
-1. First, silently judge whether the context genuinely addresses the user's question. Use
-   it ONLY when it really contains relevant information. Do not force a connection that
-   isn't there: poetic, narrative, tangential, or topically-unrelated passages are NOT
-   relevant just because they share a word with the question — ignore them.
-2. When the context is relevant, base your answer on it and answer directly and specifically.
-3. When the context does not actually address the question, ignore it completely and answer
-   from your own general knowledge. Do this silently — do not mention the knowledge base,
-   the context, or the fact that it lacked an answer, and do not apologize.
-4. Never narrate your retrieval or reasoning process. Do not write "according to Context 1",
+1. When the context contains what was asked for, base your answer on it and answer
+   directly and specifically.
+2. When it does not, ignore it and answer from your own general knowledge. Do this
+   silently — do not mention the knowledge base, the context, or the fact that it lacked
+   an answer, and do not apologize.
+3. Never narrate your retrieval or reasoning process. Do not write "according to Context 1",
    "the context is unrelated, but", "I couldn't find specific context", or similar. Just
    give the answer as if you simply knew it.
-5. Do not fabricate specifics (numbers, names, procedures) that are neither supported by
-   genuinely relevant context nor part of your reliable knowledge.
+4. Do not fabricate specifics (numbers, names, procedures) that are neither supported by
+   the context nor part of your reliable knowledge.
 
 Format your response using markdown for readability.
 `,
@@ -128,13 +362,20 @@ Do NOT use:
 - Statements that are not suggestions themselves, such as praise for asking the question
 - Direct questions or commands to the user
 
-Return ONLY the 3 suggestions as a comma-separated list with no additional text, formatting, numbering, or quotation marks.
 The suggestions should be in title case.
-Ensure that your suggestions are comma-separated with no conjunctions like "and" or "or".
-Do not use line breaks, new lines, or extra spacing to separate the suggestions.
-Format: suggestion1, suggestion2, suggestion3
+
+Respond with JSON: {"suggestions": ["...", "...", "..."]}
 `,
-  title_generation: `You are a title generator. Given the start of a conversation, generate a concise, descriptive title under 50 characters. Return ONLY the title text with no quotes, punctuation wrapping, or extra formatting.`,
+  title_generation: `You are a title generator. Given the start of a conversation, generate a concise, descriptive title under 50 characters.
+
+Respond with JSON: {"title": "..."}`,
+  relevance_check: `You check whether search results are about what a user asked.
+
+Answer true if at least one passage is about the same specific subject as the question, even if it does not state the exact answer.
+
+Answer false if the passages only share a word or a loose theme with the question. Also answer false if the user is not asking for information at all: greetings, chit-chat, or asking you to produce something new such as a poem, a joke, a translation, or a letter.
+
+Respond with JSON: {"on_topic": true} or {"on_topic": false}`,
   query_rewrite: `
 You are a query rewriting assistant. Your task is to reformulate the user's latest question to include relevant context from the conversation history.
 
@@ -145,7 +386,7 @@ Rules:
 2. Include key entities, topics, and context from previous messages
 3. Make it a clear, searchable query
 4. Do NOT answer the question - only rewrite the user's query to be more effective for retrieval
-5. Output ONLY the rewritten query, nothing else
+5. Respond with JSON: {"queries": ["..."]} — a single rewritten query in the array
 
 Examples:
 
@@ -154,7 +395,7 @@ User: "How do I install Gentoo?"
 Assistant: [detailed installation guide]
 User: "Is an internet connection required to install?"
 
-Rewritten Query: "Is an internet connection required to install Gentoo Linux?"
+Rewritten Query: {"queries": ["Is an internet connection required to install Gentoo Linux?"]}
 
 ---
 
@@ -163,6 +404,6 @@ User: "What's the best way to preserve meat?"
 Assistant: [preservation methods]
 User: "How long does it last?"
 
-Rewritten Query: "How long does preserved meat last using curing or smoking methods?"
+Rewritten Query: {"queries": ["How long does preserved meat last using curing or smoking methods?"]}
 `,
 }

admin/constants/service_names.ts

@@ -18,4 +18,5 @@ export const SERVICE_NAMES = {
   HOMEBOX: 'nomad_homebox',
   VAULTWARDEN: 'nomad_vaultwarden',
   JELLYFIN: 'nomad_jellyfin',
+  TRANSLATE: 'nomad_translate',
 }

admin/constants/supply_depot_docs.ts

@@ -20,6 +20,7 @@ export const SUPPLY_DEPOT_DOC_ANCHORS: Record<string, string> = {
   [SERVICE_NAMES.KOLIBRI]: 'kolibri',
   [SERVICE_NAMES.KOLIBRI_GEN2]: 'kolibri',
   [SERVICE_NAMES.MESHCORE_WEB]: 'meshcore-web',
+  [SERVICE_NAMES.TRANSLATE]: 'offline-translation',
 }
 
 // Returns the in-app docs link for a service, or null if it has no documentation section.

admin/database/migrations/1771200000001_create_map_markers_table.ts

@@ -9,7 +9,11 @@ export default class extends BaseSchema {
       table.string('name').notNullable()
       table.double('longitude').notNullable()
       table.double('latitude').notNullable()
-      table.string('color', 20).notNullable().defaultTo('orange')
+      table.string('color', 20).notNullable().defaultTo('orange') // retaining this for backward compatibility
+      table.string('custom_color', 7).nullable() // "#aabbcc"
+      table.string('icon', 50).nullable() // "circle", "star", "flag", etc.
+      table.string('icon_color', 7).nullable()
+      table.boolean('visible').notNullable().defaultTo(true)
       table.string('marker_type', 20).notNullable().defaultTo('pin')
       table.string('route_id').nullable()
       table.integer('route_order').nullable()

admin/database/migrations/1771200000002_add_visibility_and_icon_fields_to_map_markers.ts

@@ -0,0 +1,28 @@
+import { BaseSchema } from '@adonisjs/lucid/schema'
+
+export default class AddVisibilityAndIconFieldsToMapMarkers extends BaseSchema {
+  protected tableName = 'map_markers'
+
+  async up() {
+    // create_map_markers_table now includes these columns on fresh installs.
+    if (await this.schema.hasColumn(this.tableName, 'custom_color')) {
+      return
+    }
+
+    this.schema.alterTable(this.tableName, (table) => {
+      table.string('custom_color', 7).nullable()
+      table.string('icon', 50).nullable()
+      table.string('icon_color', 7).nullable()
+      table.boolean('visible').notNullable().defaultTo(true)
+    })
+  }
+
+  async down() {
+    this.schema.alterTable(this.tableName, (table) => {
+      table.dropColumn('custom_color')
+      table.dropColumn('icon')
+      table.dropColumn('icon_color')
+      table.dropColumn('visible')
+    })
+  }
+}

admin/database/migrations/1778800000001_add_dataset_to_installed_resources_type.ts

@@ -0,0 +1,34 @@
+import { BaseSchema } from '@adonisjs/lucid/schema'
+
+/**
+ * Widen `installed_resources.resource_type` to accept 'dataset'.
+ *
+ * The model already declares `'zim' | 'map' | 'dataset'` and
+ * `IngestDrugDataJob` writes a 'dataset' row for the openFDA drug labels, but
+ * the column was still `enum('zim','map')` from the table's original
+ * migration. Every write failed with "Data truncated for column
+ * 'resource_type'", so the drug reference ingested its 261k labels and then
+ * left no install-state row — the tier-status math and the home-tile gate both
+ * read those rows, so the resource stayed invisible to them.
+ */
+export default class extends BaseSchema {
+  protected tableName = 'installed_resources'
+
+  async up() {
+    await this.db.rawQuery(
+      "ALTER TABLE installed_resources MODIFY COLUMN resource_type " +
+        "enum('zim','map','dataset') NOT NULL"
+    )
+  }
+
+  async down() {
+    // Rows that only exist because of this widening have to go before the
+    // column can be narrowed again; MySQL would otherwise truncate them to ''
+    // (or fail outright under STRICT_TRANS_TABLES).
+    await this.db.rawQuery("DELETE FROM installed_resources WHERE resource_type = 'dataset'")
+    await this.db.rawQuery(
+      "ALTER TABLE installed_resources MODIFY COLUMN resource_type " +
+        "enum('zim','map') NOT NULL"
+    )
+  }
+}

admin/database/migrations/1785468975052_create_add_sources_to_chat_messages_table.ts

@@ -0,0 +1,17 @@
+import { BaseSchema } from '@adonisjs/lucid/schema'
+
+export default class extends BaseSchema {
+  protected tableName = 'chat_messages'
+
+  async up() {
+    this.schema.alterTable(this.tableName, (table) => {
+      table.text('sources').nullable()
+    })
+  }
+
+  async down() {
+    this.schema.alterTable(this.tableName, (table) => {
+      table.dropColumn('sources')
+    })
+  }
+}

admin/database/migrations/1785712100000_add_active_to_kb_ingest_state.ts

@@ -0,0 +1,17 @@
+import { BaseSchema } from '@adonisjs/lucid/schema'
+
+export default class extends BaseSchema {
+  protected tableName = 'kb_ingest_state'
+
+  async up() {
+    this.schema.alterTable(this.tableName, (table) => {
+      table.boolean('active').notNullable().defaultTo(true)
+    })
+  }
+
+  async down() {
+    this.schema.alterTable(this.tableName, (table) => {
+      table.dropColumn('active')
+    })
+  }
+}

admin/database/migrations/1786000000001_add_is_link_tile_to_services.ts

@@ -0,0 +1,22 @@
+import { BaseSchema } from '@adonisjs/lucid/schema'
+
+export default class extends BaseSchema {
+  protected tableName = 'services'
+
+  async up() {
+    this.schema.alterTable(this.tableName, (table) => {
+      // Marks a row that is only a dashboard shortcut: a name, an icon and a URL,
+      // with no container behind it. Distinct from is_custom, which still means a
+      // user-defined container NOMAD installs and manages. A link tile has no
+      // image, no ports and no lifecycle, so the UI must not offer Start, Stop,
+      // Update or Uninstall for one.
+      table.boolean('is_link_tile').notNullable().defaultTo(false)
+    })
+  }
+
+  async down() {
+    this.schema.alterTable(this.tableName, (table) => {
+      table.dropColumn('is_link_tile')
+    })
+  }
+}

admin/database/migrations/1786000000002_add_link_color_to_services.ts

@@ -0,0 +1,21 @@
+import { BaseSchema } from '@adonisjs/lucid/schema'
+
+export default class extends BaseSchema {
+  protected tableName = 'services'
+
+  async up() {
+    this.schema.alterTable(this.tableName, (table) => {
+      // Brand-palette colour for a dashboard link tile. Null means the default.
+      // A separate migration from the one that added is_link_tile, because that
+      // one has already run on the test box; editing it in place would leave
+      // fresh installs and upgraded ones with different schemas.
+      table.string('link_color', 20).nullable()
+    })
+  }
+
+  async down() {
+    this.schema.alterTable(this.tableName, (table) => {
+      table.dropColumn('link_color')
+    })
+  }
+}

admin/database/seeders/service_seeder.ts

@@ -4,6 +4,10 @@ import { ModelAttributes } from '@adonisjs/lucid/types/model'
 import env from '#start/env'
 import { SERVICE_NAMES } from '../../constants/service_names.js'
 import { KIWIX_LIBRARY_CMD } from '../../constants/kiwix.js'
+import {
+  mergeContainerConfigPreservingHostPorts,
+  mergeUiLocationPreservingHostPort,
+} from '../../app/utils/service_catalog_merge.js'
 
 type ServiceSeedRecord = Omit<
   ModelAttributes<Service>,
@@ -15,6 +19,8 @@ type ServiceSeedRecord = Omit<
   | 'metadata'
   | 'is_user_modified'
   | 'is_deprecated'
+  | 'is_link_tile'
+  | 'link_color'
   | 'custom_url'
   | 'auto_update_enabled'
   | 'available_update_first_seen_at'
@@ -92,7 +98,7 @@ export default class ServiceSeeder extends BaseSeeder {
       display_order: 3,
       description: 'Local AI chat that runs entirely on your hardware - no internet required',
       icon: 'IconWand',
-      container_image: 'ollama/ollama:0.24.0',
+      container_image: 'ollama/ollama:0.33.3',
       source_repo: 'https://github.com/ollama/ollama',
       container_command: 'serve',
       container_config: JSON.stringify({
@@ -536,6 +542,51 @@ export default class ServiceSeeder extends BaseSeeder {
       depends_on: null,
       metadata: JSON.stringify({ minMemoryMB: 2048, minDiskMB: 20480 }),
     },
+    {
+      service_name: SERVICE_NAMES.TRANSLATE,
+      friendly_name: 'Translated Library',
+      powered_by: 'Bergamot',
+      display_order: 28,
+      description:
+        'Read the Information Library in another language. Machine translation that works offline, on CPU',
+      icon: 'IconWorld',
+      container_image: 'ghcr.io/crosstalk-solutions/project-nomad-translate:0.1.1',
+      source_repo: 'https://github.com/browsermt/bergamot-translator',
+      container_command: null,
+      container_config: JSON.stringify({
+        HostConfig: {
+          RestartPolicy: { Name: 'unless-stopped' },
+          PortBindings: { '8391/tcp': [{ HostPort: '8460' }] },
+          Binds: [`${ServiceSeeder.NOMAD_STORAGE_ABS_PATH}/translate/models:/models`],
+        },
+        ExposedPorts: { '8391/tcp': {} },
+        // TRANSLATE_LANGS is the language set fetched on first start, about
+        // 45-140 MB per language for the pair in both directions. Editable via
+        // Manage > Edit; the container re-checks on restart and only fetches
+        // what is missing.
+        Env: [
+          // Reached by container name on the shared NOMAD network, which
+          // DockerService attaches every managed container to. Using the name
+          // rather than a host port means this survives the library being
+          // remapped.
+          'KIWIX=http://nomad_kiwix_server:8080',
+          'TRANSLATE_LANGS=fr,es,de',
+          'WORKERS=8',
+        ],
+      }),
+      ui_location: '8460',
+      installed: false,
+      installation_status: 'idle',
+      is_dependency_service: false,
+      is_custom: false,
+      category: 'education',
+      // Translating an article it cannot fetch is meaningless, so the library
+      // has to be there first.
+      depends_on: SERVICE_NAMES.KIWIX,
+      // Peak RSS measured at 729 MB with three language pairs resident; models
+      // are about 45-140 MB per language on disk.
+      metadata: JSON.stringify({ minMemoryMB: 1536, minDiskMB: 1024 }),
+    },
   ]
 
   async run() {
@@ -543,6 +594,10 @@ export default class ServiceSeeder extends BaseSeeder {
       'service_name',
       'is_custom',
       'is_user_modified',
+      // Needed to keep an installed app's published host port across catalog sync.
+      'installed',
+      'container_config',
+      'ui_location',
     ])
     const existingServiceMap = new Map(existingServices.map((s) => [s.service_name, s]))
 
@@ -562,12 +617,36 @@ export default class ServiceSeeder extends BaseSeeder {
     for (const service of ServiceSeeder.DEFAULT_SERVICES) {
       const existing = existingServiceMap.get(service.service_name)
       if (existing && !existing.is_custom && !existing.is_user_modified) {
+        // An installed app's published host port belongs to the machine, not the
+        // catalog: it diverges because the default was already taken on that host.
+        // Overwriting it desyncs the row from the running container, which breaks
+        // the Open link now and collides on the next recreate (#1005). Keep the
+        // live host ports and apply every other catalog change. Not-yet-installed
+        // rows take the catalog verbatim, since nothing is running to conflict.
+        const containerConfig = existing.installed
+          ? mergeContainerConfigPreservingHostPorts(
+              service.container_config,
+              existing.container_config
+            )
+          : service.container_config
+        // Derive the link from the config we are about to WRITE, not the live one. When a
+        // catalog change to the container-side port means the live host port was not
+        // preserved above, the link has to follow the catalog too, or the row ships a
+        // container bound to one port and an Open button pointing at another.
+        const uiLocation = existing.installed
+          ? mergeUiLocationPreservingHostPort(
+              service.ui_location,
+              existing.ui_location,
+              containerConfig
+            )
+          : service.ui_location
+
         await Service.query().where('service_name', service.service_name).update({
-          container_config: service.container_config,
+          container_config: containerConfig,
           container_command: service.container_command ?? null,
           metadata: (service as any).metadata ?? null,
           category: service.category,
-          ui_location: service.ui_location,
+          ui_location: uiLocation,
         })
       }
     }

admin/docs/faq.md

@@ -29,8 +29,32 @@ NOMAD is designed for capable hardware, especially if you want to use the AI fea
 
 **For detailed build recommendations at three price points ($150–$1,000+), see the [Hardware Guide](https://www.projectnomad.us/hardware).**
 
+### How much RAM do I need?
+
+**Without the AI Assistant, the whole stack sits under 1GB.** Measured on a running
+install: the Command Center 241MB, MySQL 444MB, Redis 9MB, Kiwix 101MB, and the
+knowledge base index 155MB. That is why 4GB is a real minimum rather than a
+defensive one.
+
+**The AI is the variable, and it is the model rather than NOMAD.** Ollama idles at
+roughly 1.2GB, and a model needs about its download size resident while it answers,
+so an 8B model at ~4.6GB wants about that much on top. **8GB is workable for a small
+model, 16GB is comfortable, and 32GB is the recommendation if you want to run larger
+ones.**
+
+Where that memory comes from depends on your hardware. With a discrete GPU the model
+loads into VRAM and never touches system RAM, so VRAM is the number that limits which
+models you can run. With an integrated GPU or no GPU at all, it comes out of system
+RAM, which is why a CPU-only box needs more of it.
+
 ### How much storage do I need?
-It depends on what you download:
+
+**To install NOMAD itself: about 5GB, so leave 10GB free.** That covers the
+Command Center and its database, before any content. Adding the AI Assistant
+brings the total to roughly 25GB, because Ollama and a general-purpose model are
+both large.
+
+After that it depends entirely on what you download:
 - Full Wikipedia: ~95GB
 - Khan Academy courses: ~50GB
 - Medical references: ~500MB

admin/docs/release-notes.md

@@ -1,5 +1,77 @@
 # Release Notes
 
+## Version 1.35.1 - October 5, 2026
+
+### Features
+
+### Bug Fixes
+- **AI**: probe a model the list reports without vision before disabling image attachments, so `gemma3` accepts images again (#1409). Thanks @chriscrosstalk for the contribution!
+- **Translate**: forward upstream redirects instead of following them, so books open styled and their article links resolve (#1379). Thanks @chriscrosstalk for the contribution!
+- **Translate**: download every released language instead of crashing on pre-release models, and list all supported languages in the docs (#1405). Thanks @chriscrosstalk for the contribution!
+
+### Improvements
+
+## Version 1.35.0 - September 29, 2026
+
+### Features
+- **AI**: support local vision attachments in chat (#1288). Thanks @gabegraves and @chriscrosstalk for the contribution!
+- **AI**: show source document and date as citations under RAG answers (#1287). Thanks @chriscrosstalk for the contribution!
+- **AI**: budget the prompt and actually set the context window on a native Ollama chat path (#1253). Thanks @jakeaturner for the contribution!
+- **AI**: give chat a sampler with four response-style profiles instead of backend defaults (#1282). Thanks @jakeaturner for the contribution!
+- **AI**: constrain the structured task calls (titles, chips, query rewrite) instead of asking nicely (#1259). Thanks @jakeaturner for the contribution!
+- **AI**: let retrieval decline via a post-rerank relevance floor (#1281). Thanks @jakeaturner for the contribution!
+- **AI**: add opt-in relevance check for off-topic retrieval (#1365). Thanks @jakeaturner for the contribution!
+- **AI**: specify a lightweight model for ancillary tasks (#1244). Thanks @jakeaturner for the contribution!
+- **AI**: make RAG retrieval optional per chat (#1247). Thanks @jakeaturner for the contribution!
+- **Dashboard**: user-added link tiles for services NOMAD does not manage (#1293). Thanks @chriscrosstalk for the contribution!
+- **KB**: per-file active toggle plus collection grouping and bulk toggle (#1286). Thanks @just-jbc and @chriscrosstalk for the contribution!
+- **Kiwix Library**: browse the remote Kiwix catalog in any language, not just English (#1299). Thanks @chriscrosstalk for the contribution!
+- **Maps**: full marker management (visibility, search/sort, per-marker icon and colour, coordinate navigation, delete confirmation) (#1291). Thanks @kennethbrewer3 and @chriscrosstalk for the contribution!
+- **Translate**: offline machine translation for the Information Library, shipped as a Supply Depot app (#1292). Thanks @chriscrosstalk for the contribution!
+
+### Bug Fixes
+- **AI**: stop `<think>` output leaking into titles, chips, and the RAG rewrite (#1254). Thanks @jakeaturner for the contribution!
+- **AI**: report a failed Ollama model pull as a failure instead of a success (#1274). Thanks @chriscrosstalk for the contribution!
+- **AI**: don't probe the native embed endpoint on a non-Ollama backend (#1284). Thanks @chriscrosstalk for the contribution!
+- **AI**: don't classify the backend as non-Ollama when it is merely unreachable (#1357). Thanks @chriscrosstalk for the contribution!
+- **AI**: bump the Ollama pin to 0.33.3 to fix the disconnect hang (#1356). Thanks @chriscrosstalk for the contribution!
+- **AI**: remove the long-dead default rewrite model warning (#1249). Thanks @jakeaturner for the contribution!
+- **Benchmark**: show the v2 score in Benchmark History (#1210). Thanks @chriscrosstalk for the contribution!
+- **Benchmark**: stop WSL2 submitting "Microsoft Basic Render Driver" as the GPU model (#1277). Thanks @chriscrosstalk for the contribution!
+- **Chat**: abort abandoned response streams (#1220). Thanks @NgoQuocViet2001 for the contribution!
+- **Chat**: stop capping answers at 1024 tokens and surface truncation (#1360). Thanks @jakeaturner for the contribution!
+- **Chat**: render the user's message before creating the session (#1301). Thanks @chriscrosstalk for the contribution!
+- **Content**: re-download content updates after a failed attempt (#1213). Thanks @Osamaali313 for the contribution!
+- **Database**: allow `dataset` in `installed_resources.resource_type` (#1243). Thanks @rcorvus for the contribution!
+- **Downloads**: guard orphaned model jobs (#1245). Thanks @NgoQuocViet2001 for the contribution!
+- **GPU**: detect Ollama CPU fallback from logs so auto-remediation fires (#1109). Thanks @fpolica91 for the contribution!
+- **GPU**: detect Ollama CPU fallback on AMD and offer the right fix (#1370). Thanks @jakeaturner for the contribution!
+- **GPU**: stop a missing `/dev/kfd` blocking the AI Assistant install entirely (#1276). Thanks @chriscrosstalk for the contribution!
+- **GPU**: keep an existing AMD HSA override across reinstall and update (#1390). Thanks @jakeaturner for the contribution!
+- **Install**: time out Redis readiness probes (#1347). Thanks @dajiaohuang for the contribution!
+- **Install**: match the HawkPoint1 lspci string for gfx1103 AMD GPU detection (#1385). Thanks @Yi-111-a for the contribution!
+- **KB**: stop leaking Qdrant vectors when a ZIM is deleted or replaced (#1227). Thanks @just-jbc for the contribution!
+- **KB**: stop the orphan sweep purging an empty but present scan root (#1393). Thanks @jakeaturner for the contribution!
+- **KB**: persist the source active toggle when no ingest state row exists (#1395). Thanks @jakeaturner for the contribution!
+- **Maps**: stop world basemap provisioning running pmtiles extract twice (#1394). Thanks @jakeaturner for the contribution!
+- **Queue**: clear terminal jobs before re-dispatch so downloads aren't silently dropped (#1225). Thanks @chriscrosstalk for the contribution!
+- **Queue**: stop stalled-job recovery forking the embedding chain (#1270). Thanks @chriscrosstalk for the contribution!
+- **RAG**: don't abandon a ZIM after one sparse batch (#1242). Thanks @rcorvus for the contribution!
+- **Security**: reject oversized custom URLs (#1354). Thanks @dajiaohuang for the contribution!
+- **Seeder**: keep an installed app's host port across catalog sync (#1300). Thanks @johno10661 and @chriscrosstalk for the contribution!
+- **System**: resolve a placeholder GPU name on the System page too (#1285). Thanks @chriscrosstalk for the contribution!
+- **UI**: restore dark mode text contrast (#1358). Thanks @Tiancheng-Xu for the contribution!
+- **ZIM**: resolve flavoured and non-English ZIMs in the update check (#1298). Thanks @chriscrosstalk for the contribution!
+
+### Improvements
+- **Brand**: replace the retired N.O.M.A.D. badge with the new logo (#1375). Thanks @chriscrosstalk for the contribution!
+- **CI**: add a weekly schedule trigger to collection URL validation (#1217). Thanks @cclavin for the contribution!
+- **Dashboard**: tighten the Drug Reference tile description (#1297). Thanks @chriscrosstalk for the contribution!
+- **Docs**: add SECURITY.md and tighten .gitignore for credential files (#1223). Thanks @chriscrosstalk for the contribution!
+- **Docs**: state how much disk NOMAD needs before any content (#1278). Thanks @chriscrosstalk for the contribution!
+- **Eval**: standardized RAG quality evaluation harness with a frozen corpus and golden question set (#1233). Thanks @jakeaturner for the contribution!
+- **Install**: add the Project NOMAD ASCII banner to the installer (#1310). Thanks @chriscrosstalk for the contribution!
+
 ## Version 1.34.0 - August 4, 2026
 
 ### Features

admin/docs/supply-depot-apps.md

@@ -282,3 +282,85 @@ A browser-based client for [MeshCore](https://meshcore.io) radios. MeshCore is a
 **Your data:** There's nothing to set up or store on your NOMAD for this app. Your radio's settings live on the radio itself, and the app's preferences live in your browser. There's no NOMAD folder to manage.
 
 **Works offline:** Fully offline, which is the whole point of MeshCore. The app is served from your NOMAD and talks to your radio directly over USB or Bluetooth, never the internet.
+
+## Translated Library {% #offline-translation %}
+
+Reads the Information Library in another language. Open an article and a **Translate this page** bar appears at the top with a button for each installed language, plus **Original** to switch back. Your choice sticks as you click through to other articles.
+
+**Why this instead of the AI Assistant:** the AI Assistant can translate, but this is roughly 1,600 times faster on the same machine, and it does not need a graphics card, so it works on every NOMAD. It is also more careful with names: asked to translate a page, the AI Assistant will happily translate "Project NOMAD" into another language, and this will not.
+
+**Choosing languages:** French, Spanish and German are set up by default. To add or remove languages:
+
+1. In the **Supply Depot**, open the Translated Library's **Manage** menu and choose **Edit**.
+2. Under **Environment Variables**, find `TRANSLATE_LANGS=fr,es,de`.
+3. Change the list of language codes, separated by commas. For example, `TRANSLATE_LANGS=fr,es,de,sv` adds Swedish.
+4. Click **Save**. The app restarts, downloads any new languages, and adds a button for each one to the **Translate this page** bar.
+
+Adding a language needs an internet connection, and each one takes between about 45 MB and 140 MB of disk. Languages you already have keep working offline. Removing a code from the list does not remove a language you have already downloaded; it stays in `storage/translate/models` and keeps its button.
+
+These 50 languages are available. Chinese is not available yet.
+
+| Language | Code |
+|---|---|
+| Afrikaans | `af` |
+| Arabic | `ar` |
+| Basque | `eu` |
+| Bengali | `bn` |
+| Bosnian | `bs` |
+| Bulgarian | `bg` |
+| Catalan | `ca` |
+| Croatian | `hr` |
+| Czech | `cs` |
+| Danish | `da` |
+| Dutch | `nl` |
+| Estonian | `et` |
+| Finnish | `fi` |
+| French | `fr` |
+| Galician | `gl` |
+| German | `de` |
+| Greek | `el` |
+| Gujarati | `gu` |
+| Hebrew | `he` |
+| Hindi | `hi` |
+| Hungarian | `hu` |
+| Icelandic | `is` |
+| Indonesian | `id` |
+| Italian | `it` |
+| Japanese | `ja` |
+| Kannada | `kn` |
+| Korean | `ko` |
+| Latvian | `lv` |
+| Lithuanian | `lt` |
+| Malay | `ms` |
+| Malayalam | `ml` |
+| Marathi | `mr` |
+| Norwegian | `nb` |
+| Persian | `fa` |
+| Polish | `pl` |
+| Portuguese | `pt` |
+| Romanian | `ro` |
+| Russian | `ru` |
+| Serbian | `sr` |
+| Slovak | `sk` |
+| Slovenian | `sl` |
+| Spanish | `es` |
+| Swedish | `sv` |
+| Tamil | `ta` |
+| Telugu | `te` |
+| Thai | `th` |
+| Turkish | `tr` |
+| Ukrainian | `uk` |
+| Urdu | `ur` |
+| Vietnamese | `vi` |
+
+**First start needs internet.** The language models download when the app first runs, the same as installing any other app. After that it is entirely offline. If you install this while disconnected the app still starts and the library still works, just without translation until it can fetch the models.
+
+**What it does not translate:** tables and infoboxes, the page title in your browser tab, and Kiwix's own search results. Searching also still matches the original language, so look things up in English and translate the article you land on.
+
+**A note on two language buttons:** the library's own toolbar has a globe that changes the *menus* around the page. The bar this app adds changes the *article*. They are different things and sit close together, which is unfortunate but not something we can move.
+
+**Accuracy:** this is machine translation, and it is literal. It is very good for getting the sense of an article. Be careful relying on it for exact medical or safety wording, where the correct term in another language is often not the literal one.
+
+**Your data:** language models live in `storage/translate/models`. Nothing you read is stored or sent anywhere.
+
+**Works offline:** yes, once the models have downloaded.

admin/inertia/components/AppUrlModal.tsx

@@ -79,6 +79,7 @@ export default function AppUrlModal({ open, service, onClose, onSaved, showError
               name="customUrl"
               label="Custom URL"
               placeholder="http://jellyfin.myhomelab.net"
+              maxLength={255}
               value={value}
               onChange={(e) => setValue(e.target.value)}
               error={isInvalid}

admin/inertia/components/BouncingLogo.tsx

@@ -31,9 +31,9 @@ const FadingImage = ({  alt = "Fading image", className = "" }) => {
       isVisible ? 'opacity-100' : 'opacity-0'
     }`}>
       <img
-        src={`/project_nomad_logo.webp`}
+        src="/nomad-primary.svg"
         alt={alt}
-        className={`w-64 h-64 ${className}`}
+        className={`w-64 h-64 object-contain ${className}`}
       />
     </div>
   );