diff options
| author | Colin Ian King <[email protected]> | 2020-09-15 17:20:49 +0100 |
|---|---|---|
| committer | Heiko Stuebner <[email protected]> | 2021-05-28 20:27:44 +0200 |
| commit | ce0cb93a5adb283f577cd4661f511047b5e39028 (patch) | |
| tree | 9c9f907905d7619ce37e2d15a70bf96692561452 /scripts/gdb/linux/tasks.py | |
| parent | 3dfa159f6b0c054eb63673fbf643a5f2cc862e63 (diff) | |
drm/rockchip: cdn-dp: fix sign extension on an int multiply for a u64 result
The variable bit_per_pix is a u8 and is promoted in the multiplication
to an int type and then sign extended to a u64. If the result of the
int multiplication is greater than 0x7fffffff then the upper 32 bits will
be set to 1 as a result of the sign extension. Avoid this by casting
tu_size_reg to u64 to avoid sign extension and also a potential overflow.
Fixes: 1a0f7ed3abe2 ("drm/rockchip: cdn-dp: add cdn DP support for rk3399")
Signed-off-by: Colin Ian King <[email protected]>
Reviewed-by: Guenter Roeck <[email protected]>
Signed-off-by: Heiko Stuebner <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]
Diffstat (limited to 'scripts/gdb/linux/tasks.py')
0 files changed, 0 insertions, 0 deletions