mirror of
https://github.com/Crosstalk-Solutions/project-nomad.git
synced 2026-10-17 05:14:19 +02:00
c4c2bdc02a
Two defects made chat hang forever with thinking-capable models on the OpenAI-compat (/v1) path, which NOMAD uses for both local and remote Ollama: 1. Field mismatch. chatStream()/chat() read `delta.thinking` / `message.thinking`, but Ollama's /v1 endpoint emits thinking tokens as `reasoning`. All thinking output was silently dropped, so the SSE stream was nothing but empty content+thinking chunks and never reached done. Now read `thinking ?? reasoning` in both paths (the inline <think>-tag parser for other backends is unchanged). 2. No abort on client disconnect. When the user gave up and closed the chat, the upstream generation kept decoding server-side. With Ollama's default OLLAMA_NUM_PARALLEL=1 that abandoned request occupied the only slot, so every later chat/RAG request queued behind it and the whole assistant appeared dead. The controller now wires an AbortController to the response 'close' event and threads the signal into the OpenAI SDK request, so a disconnect aborts the upstream generation. Verified on NOMAD2 (qwen3:0.6b, which reports the `thinking` capability and emits `reasoning` on /v1): before, the stream was endless empty chunks; after, thinking streams visibly and reaches done. On disconnect, Ollama's decode counter freezes and the server logs `cancel task` / `slot release`, freeing the slot for the next request. Note: thinking is still force-on for capable models here; a user-facing per-model thinking toggle (default off) is a planned follow-up. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| app | ||
| bin | ||
| commands | ||
| config | ||
| constants | ||
| database | ||
| docs | ||
| inertia | ||
| providers | ||
| public | ||
| resources | ||
| scripts | ||
| start | ||
| tests | ||
| types | ||
| util | ||
| views | ||
| .editorconfig | ||
| .env.example | ||
| ace.js | ||
| adonisrc.ts | ||
| eslint.config.js | ||
| package-lock.json | ||
| package.json | ||
| tailwind.config.ts | ||
| tsconfig.json | ||
| vite.config.ts | ||