diff options
| author | Pavel Begunkov <[email protected]> | 2024-03-18 22:00:32 +0000 |
|---|---|---|
| committer | Jens Axboe <[email protected]> | 2024-04-15 08:10:24 -0600 |
| commit | 902ce82c2aa130bea5e3feca2d4ae62781865da7 (patch) | |
| tree | b1dc57d30ccaa7605ad5d675ce8e7a6c9756e7f8 /include/linux | |
| parent | e5c12945be5016d681ff305ea7306fef5902219d (diff) | |
io_uring: get rid of intermediate aux cqe caches
io_post_aux_cqe(), which is used for multishot requests, delays
completions by putting CQEs into a temporary array for the purpose
completion lock/flush batching.
DEFER_TASKRUN doesn't need any locking, so for it we can put completions
directly into the CQ and defer post completion handling with a flag.
That leaves !DEFER_TASKRUN, which is not that interesting / hot for
multishot requests, so have conditional locking with deferred flush
for them.
Signed-off-by: Pavel Begunkov <[email protected]>
Tested-by: Ming Lei <[email protected]>
Link: https://lore.kernel.org/r/b1d05a81fd27aaa2a07f9860af13059e7ad7a890.1710799188.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <[email protected]>
Diffstat (limited to 'include/linux')
| -rw-r--r-- | include/linux/io_uring_types.h | 3 |
1 files changed, 1 insertions, 2 deletions
diff --git a/include/linux/io_uring_types.h b/include/linux/io_uring_types.h index 90e5af401800..75b46119d4c8 100644 --- a/include/linux/io_uring_types.h +++ b/include/linux/io_uring_types.h @@ -205,6 +205,7 @@ struct io_submit_state { bool plug_started; bool need_plug; + bool cq_flush; unsigned short submit_nr; unsigned int cqes_count; struct blk_plug plug; @@ -341,8 +342,6 @@ struct io_ring_ctx { unsigned cq_last_tm_flush; } ____cacheline_aligned_in_smp; - struct io_uring_cqe completion_cqes[16]; - spinlock_t completion_lock; /* IRQ completion list, under ->completion_lock */ |