mirror of
https://github.com/Crosstalk-Solutions/project-nomad.git
synced 2026-10-17 05:14:19 +02:00
ollama/ollama:0.24.0 never cancels in-flight generation when a client disconnects: the runner process is left spinning at 300%+ CPU with no active requests and never recovers. NOMAD triggers this organically via its own 60s RAG embedding timeouts, so slow CPU-only boxes can wedge themselves after a single slow request. Reported with a standalone container repro in #1321; the cancellation fix landed upstream between 0.24 and 0.33. Regression-tested on 0.33.3 (RTX 5060 box): - model listing via /api/tags (NOMAD backend detection intact) - GPU acceleration (100% GPU placement, 82.8 tok/s on llama3.1:8b) - model pull through the NOMAD download pipeline (NDJSON progress) - embeddings via /api/embed with nomic-embed-text (768-dim) - streaming chat end-to-end through the Command Center UI - the disconnect hang itself: container idles at 0.06% CPU 20s after a mid-generation client drop The AMD path is unaffected: it always overrides to ollama/ollama:rocm. The seeder does not sync container_image on existing rows, so this pin reaches fresh installs; existing installs pick it up via the opt-in app auto-update or a manual update. Fixes #1321 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| service_seeder.ts | ||