project-nomad/admin/inertia/components/GpuPassthroughAlert.tsx
jakeaturner ac437f8429 fix(gpu): detect Ollama CPU fallback on AMD and offer the right fix
GpuPassthroughRemediationProvider returned early unless the NVIDIA
runtime was registered with Docker. AMD doesn't register one, so the
log-based CPU-fallback detection added in v1.35.0-rc.1 never ran on
AMD boxes. On a Phoenix1 iGPU running ollama:rocm with /dev/kfd and
/dev/dri passed through, Ollama logged library=cpu while gpuHealth
reported status "ok".

The gpuHealth side had its own bug. hasNvidiaRuntime and
hasRocmRuntime were set only inside the lspci probe branch. When lspci
returned a plausible controller, the else branch reported "ok" without
probing and left both flags false. A blind reinstall would not have
helped either: gfx1103 needs HSA_OVERRIDE_GFX_VERSION=11.0.0, and
without the gfx marker or ai.amdHsaOverride a reinstall rebuilds the
same CPU-bound container.

- Add resolveExpectedGpuVendor(). It reads the nomad_ollama container
  config first (nvidia DeviceRequest, ROCm image tag, /dev/kfd), then
  the nvidia runtime, then the AMD GPU type when no container exists.
  /dev/dri alone is not treated as AMD. A CPU container on an AMD box
  is trusted as deliberate (#1232 fallback, acceleration disabled).
- Gate the provider on the expected vendor. NVIDIA auto-reinstall is
  unchanged but now skips when the container requests a GPU with no
  runtime registered. AMD is detect-only: the provider logs whether a
  reinstall would change the container and never reinstalls itself.
- In gpuHealth, set the runtime flags up front and let Ollama's
  startup log decide health whenever a GPU is expected, whatever lspci
  returned. A CPU-only log now reports passthrough_failed and skips
  the nvidia-smi exec. lspci stays the display source when clean.
- Parse the first GPU "inference compute" line instead of the last
  line, which could be the CPU fallback device, and recognize Vulkan.
  Move the startup-log reader into ollama_compute so both callers
  share it.
- Report amdReinstallWouldChange, amdGfxTarget, currentHsaOverride and
  suggestedHsaOverride on AMD passthrough_failed. The gfx target is
  parsed from Ollama's "amdgpu is not supported" line (best-effort).
- Add GpuPassthroughAlert, shared by Settings > System and Settings >
  Models. When a reinstall would rebuild the same AMD container, it
  offers "Fix: Set GFX Override" instead of a reinstall. The dialog
  saves ai.amdHsaOverride and then reinstalls.
- Expose ai.amdHsaOverride through the settings API. The value lands
  in the container env, so only x.y.z, "none" or empty is accepted.
- Silence the HSA override resolver's logging on the gpuHealth poll.

The #1325 case where no GPU runtime is expected and lspci reports a
controller still returns "ok" unprobed.

Fixes #1344
Refs #1325, #1232
2026-09-29 15:54:41 -07:00

218 lines
6.9 KiB
TypeScript

import { useState } from 'react'
import Alert from '~/components/Alert'
import StyledModal from '~/components/StyledModal'
import Input from '~/components/inputs/Input'
import { useNotifications } from '~/context/NotificationContext'
import { useModals } from '~/context/ModalContext'
import api from '~/lib/api'
import type { GpuHealthStatus } from '../../types/system'
const DISMISS_KEY = 'nomad:gpu-banner-dismissed'
const HSA_OVERRIDE_PATTERN = /^\d{1,2}\.\d{1,2}\.\d{1,2}$/
/**
* Banner for gpuHealth.status === 'passthrough_failed'.
*
* NVIDIA, and AMD whose container differs from what a reinstall would create, get
* the one-click reinstall. AMD whose container already matches gets an HSA override
* prompt instead: reinstalling it rebuilds the same CPU-bound container (#1344).
*/
export default function GpuPassthroughAlert({
gpuHealth,
assistantName,
className,
}: {
gpuHealth: GpuHealthStatus | undefined
assistantName: string
className?: string
}) {
const { addNotification } = useNotifications()
const { openModal, closeAllModals } = useModals()
const [reinstalling, setReinstalling] = useState(false)
const [dismissed, setDismissed] = useState(() => {
try {
return localStorage.getItem(DISMISS_KEY) === 'true'
} catch {
return false
}
})
if (gpuHealth?.status !== 'passthrough_failed' || dismissed) return null
const isAmd = gpuHealth.gpuVendor === 'amd'
const needsHsaOverride = isAmd && gpuHealth.amdReinstallWouldChange === false
const handleDismiss = () => {
setDismissed(true)
try {
localStorage.setItem(DISMISS_KEY, 'true')
} catch {}
}
const reinstall = async () => {
setReinstalling(true)
try {
const response = await api.forceReinstallService('nomad_ollama')
if (!response || !response.success) {
throw new Error(response?.message || 'Force reinstall failed')
}
addNotification({
message: `${assistantName} is being reinstalled with GPU support. This page will reload shortly.`,
type: 'success',
})
try {
localStorage.removeItem(DISMISS_KEY)
} catch {}
setTimeout(() => window.location.reload(), 5000)
} catch (error) {
addNotification({
message: `Failed to reinstall: ${error instanceof Error ? error.message : 'Unknown error'}`,
type: 'error',
})
setReinstalling(false)
}
}
const openReinstallModal = () => {
openModal(
<StyledModal
title={`Reinstall ${assistantName}?`}
onConfirm={() => {
closeAllModals()
reinstall()
}}
onCancel={closeAllModals}
open={true}
confirmText="Reinstall"
cancelText="Cancel"
>
<p className="text-text-primary">
This will recreate the {assistantName} container with GPU support enabled. Your
downloaded models will be preserved. The service will be briefly unavailable during
reinstall.
</p>
</StyledModal>,
'gpu-health-force-reinstall-modal'
)
}
const openHsaOverrideModal = () => {
openModal(
<HsaOverrideModal
gpuHealth={gpuHealth}
assistantName={assistantName}
onCancel={closeAllModals}
onSaved={() => {
closeAllModals()
reinstall()
}}
/>,
'gpu-health-hsa-override-modal'
)
}
const vendorName = isAmd ? 'an AMD' : 'an NVIDIA'
const message = needsHsaOverride
? `Your system has an AMD GPU, but ${assistantName} is running on CPU only. Reinstalling won't fix this: the GPU${gpuHealth.amdGfxTarget ? ` (${gpuHealth.amdGfxTarget})` : ''} isn't on ROCm's supported list and needs a GFX version override.`
: `Your system has ${vendorName} GPU, but ${assistantName} can't access it. AI is running on CPU only, which is significantly slower.`
return (
<Alert
type="warning"
variant="bordered"
title={`GPU Not Accessible to ${assistantName}`}
message={message}
className={className}
dismissible={true}
onDismiss={handleDismiss}
buttonProps={{
children: needsHsaOverride ? 'Fix: Set GFX Override' : `Fix: Reinstall ${assistantName}`,
icon: needsHsaOverride ? 'IconTool' : 'IconRefresh',
variant: 'action',
size: 'sm',
onClick: needsHsaOverride ? openHsaOverrideModal : openReinstallModal,
loading: reinstalling,
disabled: reinstalling,
}}
/>
)
}
function HsaOverrideModal({
gpuHealth,
assistantName,
onCancel,
onSaved,
}: {
gpuHealth: GpuHealthStatus
assistantName: string
onCancel: () => void
onSaved: () => void
}) {
const { addNotification } = useNotifications()
const [value, setValue] = useState(gpuHealth.suggestedHsaOverride ?? '')
const [saving, setSaving] = useState(false)
const valid = HSA_OVERRIDE_PATTERN.test(value.trim())
const save = async () => {
setSaving(true)
try {
const response = await api.updateSetting('ai.amdHsaOverride', value.trim())
// catchInternal already notified on a request failure and returned undefined.
if (!response) {
setSaving(false)
return
}
if (!response.success) throw new Error(response.message)
onSaved()
} catch (error) {
addNotification({
message: `Failed to save the GFX override: ${error instanceof Error ? error.message : 'Unknown error'}`,
type: 'error',
})
setSaving(false)
}
}
return (
<StyledModal
title="Set AMD GFX Override"
onConfirm={save}
onCancel={onCancel}
open={true}
confirmText="Save and Reinstall"
cancelText="Cancel"
confirmLoading={saving}
confirmDisabled={!valid || saving}
>
<div className="space-y-4 text-text-primary">
<p>
ROCm ships GPU kernels for a fixed list of AMD chips. Integrated Radeon GPUs outside
that list run on CPU unless ROCm treats them as a supported chip, set through
HSA_OVERRIDE_GFX_VERSION.
</p>
<ul className="list-disc pl-5 space-y-1">
<li>Radeon 780M / 760M (gfx1103): 11.0.0</li>
<li>Radeon 680M / 660M and other RDNA 2 iGPUs (gfx1031 to gfx1036): 10.3.0</li>
</ul>
<p className="text-sm text-text-secondary">
Detected GPU: {gpuHealth.amdGfxTarget ?? 'not reported by Ollama'}. Current override:{' '}
{gpuHealth.currentHsaOverride ?? 'none'}.
{gpuHealth.suggestedHsaOverride
? ` Recommended for this GPU: ${gpuHealth.suggestedHsaOverride}.`
: ''}
</p>
<Input
name="amdHsaOverride"
label="GFX version override"
placeholder="11.0.0"
value={value}
onChange={(e) => setValue(e.target.value)}
error={value.trim() !== '' && !valid}
helpText={`Saving recreates the ${assistantName} container with this override. Downloaded models are preserved.`}
/>
</div>
</StyledModal>
)
}