fix(ai): a failed Ollama model pull is reported to the user as a success

Ollama's /api/pull returns HTTP 200 and reports failures IN-BAND, as a JSON
line inside the NDJSON stream body:

    HTTP_STATUS=200
    {"status":"pulling manifest"}
    {"error":"pull model manifest: file does not exist"}

The stream handler only ever read completed/total/digest off each line and
silently dropped everything else, including that error line. The stream then
ended normally, on('end') resolved unconditionally, and we logged the model as
downloaded successfully. on('error') never fires for this, because at the HTTP
layer the request succeeded.

Every failed model pull, for any reason, was reported as a success. On #1071
that surfaced as an ~800 GB fp16 model "downloading" in two seconds and nothing
being installed, which had been triaged twice as disk space or a stale blob.
Those explanations produce workarounds that cannot work, since the pull never
reaches the disk at all.

Three changes:

- Check parsed.error in the data handler and destroy the stream with it, so the
  existing catch reports a real failure.
- Verify the model is actually in getModels() before returning success. This is
  the load-bearing half: it catches silent failures we have not anticipated,
  not only the ones Ollama labels. includeEmbeddings must be true or
  nomic-embed-text fails verification after a good pull, and names are
  normalised because Ollama resolves a bare name to ":latest".
- Treat an in-band "model does not exist" as permanent. It was retryable, so a
  bad model name spun through attempts: 10 on exponential backoff and the user
  watched a delayed job for hours instead of being told the name was wrong.

Verified on NOMAD3 against the shipped v1.34.0 build, which carries the same
defect:

- bogus tag, before: 'Model "..." downloaded successfully.'
- bogus tag, after:  'Failed to download model "...": pull model manifest:
  file does not exist', and the job stops at 1 attempt instead of retrying
- real model (qwen2.5:0.5b): still succeeds, passes verification, and is
  present in `ollama list`

Fixes #1273
Refs #1071

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LtGWCVMPYn6qPWVYiycK74

admin/app/services/ollama_service.ts

@@ -338,6 +338,20 @@ export class OllamaService {
             if (!line.trim()) continue
             try {
               const parsed = JSON.parse(line)
+              // Ollama reports pull failures IN-BAND: the HTTP request returns 200 and the
+              // failure arrives as a line in the NDJSON body, e.g.
+              //   {"error":"pull model manifest: file does not exist"}
+              // The 'error' event below only fires for transport failures (a destroyed
+              // socket), so without this the line is dropped, the stream ends cleanly, and
+              // a pull that transferred nothing is reported to the user as a success.
+              if (parsed.error) {
+                const message =
+                  typeof parsed.error === 'string' ? parsed.error : JSON.stringify(parsed.error)
+                const err: any = new Error(message)
+                err.code = 'ERR_OLLAMA_PULL'
+                pullResponse.data.destroy(err)
+                return
+              }
               if (parsed.completed && parsed.total && parsed.digest) {
                 // Update this digest's progress — take the max seen value so transient
                 // out-of-order updates don't make the aggregate jump backwards.
@@ -391,6 +405,22 @@ export class OllamaService {
         })
       })
 
+      // A clean stream end is not proof the model landed. Ollama can end the stream without
+      // an error line, and historically any such case was reported to the user as a
+      // successful download. Confirm against the installed list before claiming success.
+      //
+      // includeEmbeddings must be true: getModels() filters out anything with "embed" in the
+      // name by default, so nomic-embed-text would otherwise fail verification after a
+      // perfectly good pull. Names are normalised because Ollama resolves a bare name to
+      // ":latest", so a pull of "llama3.1" comes back as "llama3.1:latest".
+      const tag = (n: string) => (n.includes(':') ? n : `${n}:latest`)
+      const installedAfter = await this.getModels(true)
+      if (!installedAfter.some((m) => tag(m.name) === tag(model))) {
+        throw new Error(
+          'Ollama reported no error but the model is not installed. The download did not complete.'
+        )
+      }
+
       logger.info(`[OllamaService] Model "${model}" downloaded successfully.`)
       return { success: true, message: 'Model downloaded successfully.' }
     } catch (error) {
@@ -413,14 +443,28 @@ export class OllamaService {
 
       // Check for version mismatch (Ollama 412 response)
       const isVersionMismatch = errorMessage.includes('newer version of Ollama')
+
+      // An in-band pull error that says the model does not exist is permanent. Retrying it
+      // ten times on an exponential backoff just leaves the user watching a "delayed" job
+      // for hours instead of showing them the real reason, which is a bad model name.
+      const isPermanentPullError =
+        (error as any)?.code === 'ERR_OLLAMA_PULL' &&
+        /manifest|not found|does not exist|unauthorized|no such/i.test(errorMessage)
+
       const userMessage = isVersionMismatch
         ? 'This model requires a newer version of Ollama. Please update AI Assistant from the Apps page.'
-        : `Failed to download model: ${errorMessage}`
+        : isPermanentPullError
+          ? `Ollama could not find this model: ${errorMessage}`
+          : `Failed to download model: ${errorMessage}`
 
       // Broadcast failure to connected clients so UI can show the error
       this.broadcastDownloadError(model, userMessage)
 
-      return { success: false, message: userMessage, retryable: !isVersionMismatch }
+      return {
+        success: false,
+        message: userMessage,
+        retryable: !isVersionMismatch && !isPermanentPullError,
+      }
     }
   }