drm/amdgpu/vce: fix integer overflow in image size

Fix a security vulnerability where malicious VCE command streams
with oversized dimensions (e.g. 65536×65536) cause 32-bit integer
overflow, wrapping the calculated buffer size to 0. This bypasses
validation and allows GPU firmware to perform out-of-bound memory
access.

The fix uses 64-bit arithmetic to detect overflow and rejects
invalid dimensions before they reach the hardware.

V2: remove redundant check
V3: modify max height value
V4: remove size64

Signed-off-by: Boyuan Zhang <boyuan.zhang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit cbe408dba581755ad1279a487ec786d8927d778d)
Cc: stable@vger.kernel.org
This commit is contained in:
Boyuan Zhang 2026-05-25 11:34:27 -04:00 committed by Alex Deucher
parent 3b4082fabc
commit 186bfdc4e2

View File

@ -877,9 +877,20 @@ int amdgpu_vce_ring_parse_cs(struct amdgpu_cs_parser *p,
goto out;
}
*size = amdgpu_ib_get_value(ib, idx + 8) *
amdgpu_ib_get_value(ib, idx + 10) *
8 * 3 / 2;
uint32_t width, height;
width = amdgpu_ib_get_value(ib, idx + 8);
height = amdgpu_ib_get_value(ib, idx + 10);
if (width == 0 || height == 0 ||
width > 4096 || height > 2304) {
DRM_ERROR("invalid VCE image size: %ux%u\n",
width, height);
r = -EINVAL;
goto out;
}
*size = width * height * 8 * 3 / 2;
break;
case 0x04000001: /* config extension */