mirror of
https://github.com/Crosstalk-Solutions/project-nomad.git
synced 2026-10-17 13:19:18 +02:00
GpuPassthroughRemediationProvider returned early unless the NVIDIA runtime was registered with Docker. AMD doesn't register one, so the log-based CPU-fallback detection added in v1.35.0-rc.1 never ran on AMD boxes. On a Phoenix1 iGPU running ollama:rocm with /dev/kfd and /dev/dri passed through, Ollama logged library=cpu while gpuHealth reported status "ok". The gpuHealth side had its own bug. hasNvidiaRuntime and hasRocmRuntime were set only inside the lspci probe branch. When lspci returned a plausible controller, the else branch reported "ok" without probing and left both flags false. A blind reinstall would not have helped either: gfx1103 needs HSA_OVERRIDE_GFX_VERSION=11.0.0, and without the gfx marker or ai.amdHsaOverride a reinstall rebuilds the same CPU-bound container. - Add resolveExpectedGpuVendor(). It reads the nomad_ollama container config first (nvidia DeviceRequest, ROCm image tag, /dev/kfd), then the nvidia runtime, then the AMD GPU type when no container exists. /dev/dri alone is not treated as AMD. A CPU container on an AMD box is trusted as deliberate (#1232 fallback, acceleration disabled). - Gate the provider on the expected vendor. NVIDIA auto-reinstall is unchanged but now skips when the container requests a GPU with no runtime registered. AMD is detect-only: the provider logs whether a reinstall would change the container and never reinstalls itself. - In gpuHealth, set the runtime flags up front and let Ollama's startup log decide health whenever a GPU is expected, whatever lspci returned. A CPU-only log now reports passthrough_failed and skips the nvidia-smi exec. lspci stays the display source when clean. - Parse the first GPU "inference compute" line instead of the last line, which could be the CPU fallback device, and recognize Vulkan. Move the startup-log reader into ollama_compute so both callers share it. - Report amdReinstallWouldChange, amdGfxTarget, currentHsaOverride and suggestedHsaOverride on AMD passthrough_failed. The gfx target is parsed from Ollama's "amdgpu is not supported" line (best-effort). - Add GpuPassthroughAlert, shared by Settings > System and Settings > Models. When a reinstall would rebuild the same AMD container, it offers "Fix: Set GFX Override" instead of a reinstall. The dialog saves ai.amdHsaOverride and then reinstalls. - Expose ai.amdHsaOverride through the settings API. The value lands in the container env, so only x.y.z, "none" or empty is accepted. - Silence the HSA override resolver's logging on the gpuHealth poll. The #1325 case where no GPU runtime is expected and lspci reports a controller still returns "ok" unprobed. Fixes #1344 Refs #1325, #1232
218 lines
6.9 KiB
TypeScript
218 lines
6.9 KiB
TypeScript
import { useState } from 'react'
|
|
import Alert from '~/components/Alert'
|
|
import StyledModal from '~/components/StyledModal'
|
|
import Input from '~/components/inputs/Input'
|
|
import { useNotifications } from '~/context/NotificationContext'
|
|
import { useModals } from '~/context/ModalContext'
|
|
import api from '~/lib/api'
|
|
import type { GpuHealthStatus } from '../../types/system'
|
|
|
|
const DISMISS_KEY = 'nomad:gpu-banner-dismissed'
|
|
const HSA_OVERRIDE_PATTERN = /^\d{1,2}\.\d{1,2}\.\d{1,2}$/
|
|
|
|
/**
|
|
* Banner for gpuHealth.status === 'passthrough_failed'.
|
|
*
|
|
* NVIDIA, and AMD whose container differs from what a reinstall would create, get
|
|
* the one-click reinstall. AMD whose container already matches gets an HSA override
|
|
* prompt instead: reinstalling it rebuilds the same CPU-bound container (#1344).
|
|
*/
|
|
export default function GpuPassthroughAlert({
|
|
gpuHealth,
|
|
assistantName,
|
|
className,
|
|
}: {
|
|
gpuHealth: GpuHealthStatus | undefined
|
|
assistantName: string
|
|
className?: string
|
|
}) {
|
|
const { addNotification } = useNotifications()
|
|
const { openModal, closeAllModals } = useModals()
|
|
const [reinstalling, setReinstalling] = useState(false)
|
|
const [dismissed, setDismissed] = useState(() => {
|
|
try {
|
|
return localStorage.getItem(DISMISS_KEY) === 'true'
|
|
} catch {
|
|
return false
|
|
}
|
|
})
|
|
|
|
if (gpuHealth?.status !== 'passthrough_failed' || dismissed) return null
|
|
|
|
const isAmd = gpuHealth.gpuVendor === 'amd'
|
|
const needsHsaOverride = isAmd && gpuHealth.amdReinstallWouldChange === false
|
|
|
|
const handleDismiss = () => {
|
|
setDismissed(true)
|
|
try {
|
|
localStorage.setItem(DISMISS_KEY, 'true')
|
|
} catch {}
|
|
}
|
|
|
|
const reinstall = async () => {
|
|
setReinstalling(true)
|
|
try {
|
|
const response = await api.forceReinstallService('nomad_ollama')
|
|
if (!response || !response.success) {
|
|
throw new Error(response?.message || 'Force reinstall failed')
|
|
}
|
|
addNotification({
|
|
message: `${assistantName} is being reinstalled with GPU support. This page will reload shortly.`,
|
|
type: 'success',
|
|
})
|
|
try {
|
|
localStorage.removeItem(DISMISS_KEY)
|
|
} catch {}
|
|
setTimeout(() => window.location.reload(), 5000)
|
|
} catch (error) {
|
|
addNotification({
|
|
message: `Failed to reinstall: ${error instanceof Error ? error.message : 'Unknown error'}`,
|
|
type: 'error',
|
|
})
|
|
setReinstalling(false)
|
|
}
|
|
}
|
|
|
|
const openReinstallModal = () => {
|
|
openModal(
|
|
<StyledModal
|
|
title={`Reinstall ${assistantName}?`}
|
|
onConfirm={() => {
|
|
closeAllModals()
|
|
reinstall()
|
|
}}
|
|
onCancel={closeAllModals}
|
|
open={true}
|
|
confirmText="Reinstall"
|
|
cancelText="Cancel"
|
|
>
|
|
<p className="text-text-primary">
|
|
This will recreate the {assistantName} container with GPU support enabled. Your
|
|
downloaded models will be preserved. The service will be briefly unavailable during
|
|
reinstall.
|
|
</p>
|
|
</StyledModal>,
|
|
'gpu-health-force-reinstall-modal'
|
|
)
|
|
}
|
|
|
|
const openHsaOverrideModal = () => {
|
|
openModal(
|
|
<HsaOverrideModal
|
|
gpuHealth={gpuHealth}
|
|
assistantName={assistantName}
|
|
onCancel={closeAllModals}
|
|
onSaved={() => {
|
|
closeAllModals()
|
|
reinstall()
|
|
}}
|
|
/>,
|
|
'gpu-health-hsa-override-modal'
|
|
)
|
|
}
|
|
|
|
const vendorName = isAmd ? 'an AMD' : 'an NVIDIA'
|
|
const message = needsHsaOverride
|
|
? `Your system has an AMD GPU, but ${assistantName} is running on CPU only. Reinstalling won't fix this: the GPU${gpuHealth.amdGfxTarget ? ` (${gpuHealth.amdGfxTarget})` : ''} isn't on ROCm's supported list and needs a GFX version override.`
|
|
: `Your system has ${vendorName} GPU, but ${assistantName} can't access it. AI is running on CPU only, which is significantly slower.`
|
|
|
|
return (
|
|
<Alert
|
|
type="warning"
|
|
variant="bordered"
|
|
title={`GPU Not Accessible to ${assistantName}`}
|
|
message={message}
|
|
className={className}
|
|
dismissible={true}
|
|
onDismiss={handleDismiss}
|
|
buttonProps={{
|
|
children: needsHsaOverride ? 'Fix: Set GFX Override' : `Fix: Reinstall ${assistantName}`,
|
|
icon: needsHsaOverride ? 'IconTool' : 'IconRefresh',
|
|
variant: 'action',
|
|
size: 'sm',
|
|
onClick: needsHsaOverride ? openHsaOverrideModal : openReinstallModal,
|
|
loading: reinstalling,
|
|
disabled: reinstalling,
|
|
}}
|
|
/>
|
|
)
|
|
}
|
|
|
|
function HsaOverrideModal({
|
|
gpuHealth,
|
|
assistantName,
|
|
onCancel,
|
|
onSaved,
|
|
}: {
|
|
gpuHealth: GpuHealthStatus
|
|
assistantName: string
|
|
onCancel: () => void
|
|
onSaved: () => void
|
|
}) {
|
|
const { addNotification } = useNotifications()
|
|
const [value, setValue] = useState(gpuHealth.suggestedHsaOverride ?? '')
|
|
const [saving, setSaving] = useState(false)
|
|
const valid = HSA_OVERRIDE_PATTERN.test(value.trim())
|
|
|
|
const save = async () => {
|
|
setSaving(true)
|
|
try {
|
|
const response = await api.updateSetting('ai.amdHsaOverride', value.trim())
|
|
// catchInternal already notified on a request failure and returned undefined.
|
|
if (!response) {
|
|
setSaving(false)
|
|
return
|
|
}
|
|
if (!response.success) throw new Error(response.message)
|
|
onSaved()
|
|
} catch (error) {
|
|
addNotification({
|
|
message: `Failed to save the GFX override: ${error instanceof Error ? error.message : 'Unknown error'}`,
|
|
type: 'error',
|
|
})
|
|
setSaving(false)
|
|
}
|
|
}
|
|
|
|
return (
|
|
<StyledModal
|
|
title="Set AMD GFX Override"
|
|
onConfirm={save}
|
|
onCancel={onCancel}
|
|
open={true}
|
|
confirmText="Save and Reinstall"
|
|
cancelText="Cancel"
|
|
confirmLoading={saving}
|
|
confirmDisabled={!valid || saving}
|
|
>
|
|
<div className="space-y-4 text-text-primary">
|
|
<p>
|
|
ROCm ships GPU kernels for a fixed list of AMD chips. Integrated Radeon GPUs outside
|
|
that list run on CPU unless ROCm treats them as a supported chip, set through
|
|
HSA_OVERRIDE_GFX_VERSION.
|
|
</p>
|
|
<ul className="list-disc pl-5 space-y-1">
|
|
<li>Radeon 780M / 760M (gfx1103): 11.0.0</li>
|
|
<li>Radeon 680M / 660M and other RDNA 2 iGPUs (gfx1031 to gfx1036): 10.3.0</li>
|
|
</ul>
|
|
<p className="text-sm text-text-secondary">
|
|
Detected GPU: {gpuHealth.amdGfxTarget ?? 'not reported by Ollama'}. Current override:{' '}
|
|
{gpuHealth.currentHsaOverride ?? 'none'}.
|
|
{gpuHealth.suggestedHsaOverride
|
|
? ` Recommended for this GPU: ${gpuHealth.suggestedHsaOverride}.`
|
|
: ''}
|
|
</p>
|
|
<Input
|
|
name="amdHsaOverride"
|
|
label="GFX version override"
|
|
placeholder="11.0.0"
|
|
value={value}
|
|
onChange={(e) => setValue(e.target.value)}
|
|
error={value.trim() !== '' && !valid}
|
|
helpText={`Saving recreates the ${assistantName} container with this override. Downloaded models are preserved.`}
|
|
/>
|
|
</div>
|
|
</StyledModal>
|
|
)
|
|
}
|