drm/sysfb: ofdrm: Fix integer overflow in fb_size calculation

The framebuffer size calculation `fb_size = linebytes * height` can
overflow when both values are large (e.g., 46341 * 46341 > INT_MAX).
Since linebytes and height are both int types, the multiplication is
performed as int * int, which results in undefined behavior on overflow.

Use check_mul_overflow() to detect and prevent this overflow, consistent
with the approach used in simpledrm.c and corebootdrm.c.

Signed-off-by: Shixiong Ou <oushixiong@kylinos.cn>
Reviewed-by: Thomas Zimmermann <tzimmermann@suse.de>
Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Fixes: c8a17756c4 ("drm/ofdrm: Add ofdrm for Open Firmware framebuffers")
Cc: <stable@vger.kernel.org> # v6.2+
Link: https://patch.msgid.link/20260825104134.669676-1-oushixiong1025@163.com
This commit is contained in:
Shixiong Ou 2026-08-25 18:41:34 +08:00 committed by Thomas Zimmermann
parent 5bb489b333
commit c6f48e59ec

View File

@ -2,6 +2,7 @@
#include <linux/aperture.h>
#include <linux/of_address.h>
#include <linux/overflow.h>
#include <linux/pci.h>
#include <linux/platform_device.h>
#include <linux/pm.h>
@ -913,7 +914,10 @@ static struct ofdrm_device *ofdrm_device_create(struct drm_driver *drv,
return ERR_PTR(-EINVAL);
}
fb_size = linebytes * height;
if (check_mul_overflow(linebytes, height, &fb_size)) {
drm_err(dev, "framebuffer size exceeds maximum\n");
return ERR_PTR(-EINVAL);
}
/*
* Try to figure out the address of the framebuffer. Unfortunately, Open