diff options
| author | Christophe Leroy <christophe.leroy@csgroup.eu> | 2025-11-17 17:43:41 +0100 |
|---|---|---|
| committer | Thomas Gleixner <tglx@linutronix.de> | 2025-11-18 15:27:34 +0100 |
| commit | 4db1df7a7217827ee7f8a3414932e250f1ac2204 (patch) | |
| tree | f46d53d628d75760c2b274b137bdb2bf8ac1dab0 | |
| parent | 3ce17e6909944b3f83b54915e36f5957f1327712 (diff) | |
iov_iter: Convert copy_from_user_iter() to masked user access
copy_from_user_iter() lacks a speculation barrier, which will degrade
performance on some architecture like x86, which would be unfortunate as
copy_from_user_iter() is a critical hotpath function.
Convert copy_from_user_iter() to using masked user access on architecture
that support it. This allows to add the speculation barrier without
impacting performance.
This is similar to what was done for copy_from_user() in commit
0fc810ae3ae1 ("x86/uaccess: Avoid barrier_nospec() in 64-bit
copy_from_user()")
[ tglx: Massage change log ]
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://patch.msgid.link/58e4b07d469ca68a2b9477fe2c1ccc8a44cef131.1763396724.git.christophe.leroy@csgroup.eu
| -rw-r--r-- | lib/iov_iter.c | 16 |
1 files changed, 10 insertions, 6 deletions
diff --git a/lib/iov_iter.c b/lib/iov_iter.c index 2fe66a6b8789..a589935bf302 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -49,12 +49,16 @@ size_t copy_from_user_iter(void __user *iter_from, size_t progress, if (should_fail_usercopy()) return len; - if (access_ok(iter_from, len)) { - to += progress; - instrument_copy_from_user_before(to, iter_from, len); - res = raw_copy_from_user(to, iter_from, len); - instrument_copy_from_user_after(to, iter_from, len, res); - } + if (can_do_masked_user_access()) + iter_from = mask_user_address(iter_from); + else if (!access_ok(iter_from, len)) + return res; + + to += progress; + instrument_copy_from_user_before(to, iter_from, len); + res = raw_copy_from_user(to, iter_from, len); + instrument_copy_from_user_after(to, iter_from, len, res); + return res; } |