| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
This is a straightforward conversion.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
This is close to a 1:1 replacement of radix tree APIs with their XArray
equivalents. It would be possible to optimise nilfs_copy_back_pages(),
but that doesn't seem to be in the performance path. Also, I think
it has a pre-existing bug, and I've added a note to that effect in the
source code.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
A couple of short loops.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Mostly comment fixes, but one use of __xa_set_mark.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Signed-off-by: Matthew Wilcox <[email protected]>
Acked-by: David Sterba <[email protected]>
|
|
Remove the last mentions of radix tree from various comments.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Switch to a batch-processing model like memfd_wait_for_pins() and
use the xa_state previously set up by memfd_wait_for_pins().
Signed-off-by: Matthew Wilcox <[email protected]>
Reviewed-by: Mike Kravetz <[email protected]>
|
|
Simplify the locking by taking the spinlock while we walk the tree on
the assumption that many acquires and releases of the lock will be worse
than holding the lock while we process an entire batch of pages.
Signed-off-by: Matthew Wilcox <[email protected]>
Reviewed-by: Mike Kravetz <[email protected]>
|
|
Simpler code because the xarray takes care of things like the limit and
dereferencing the slot.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Since we are conditionally storing NULL in the XArray, we do not need
to allocate memory and the GFP flags will be unused.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
xa_find() is a slightly easier API to use than
radix_tree_gang_lookup_slot() because it contains its own RCU locking.
This commit removes the last user of radix_tree_gang_lookup_slot()
so remove the function too.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
We can use xas_find_conflict() instead of radix_tree_gang_lookup_slot()
to find any conflicting entry and combine the three paths through this
function into one.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
This is a 1:1 conversion. The major part of this patch is converting
the test framework from userspace to kernel space and mirroring the
algorithm now used in find_swap_entry().
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
xa_load has its own RCU locking, so we can eliminate it here.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Rename shmem_radix_tree_replace() to shmem_replace_entry() and
convert it to use the XArray API.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Removes sparse warnings.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
This is just a variable rename and comment change.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Slightly shorter and easier to read code.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
I found another victim of the radix tree being hard to use. Because
there was no call to radix_tree_preload(), khugepaged was allocating
radix_tree_nodes using GFP_ATOMIC.
I also converted a local_irq_save()/restore() pair to
disable()/enable().
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Quite a straightforward conversion.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
This one is trivial.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Both callers of __delete_from_swap_cache have the swp_entry_t already,
so pass that in to make constructing the XA_STATE easier.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Combine __add_to_swap_cache and add_to_swap_cache into one function
since there is no more need to preload.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
This is essentially xa_cmpxchg() with the locking handled above us,
and it doesn't have to handle replacing a NULL entry.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
We construct an XA_STATE and use it to delete the node with
xas_store() rather than adding a special function for this unique
use case. Includes a test that simulates this usage for the
test suite.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Includes moving mapping_tagged() to fs.h as a static inline, and
changing it to return bool.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Instead of calling find_get_pages_range() and putting any reference,
use xas_find() to iterate over any entries in the range, skipping the
shadow/swap entries.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Rename the function from page_cache_tree_delete_batch to just
page_cache_delete_batch.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Now the page cache lookup is using the XArray, let's convert this
regression test from the radix tree API to the XArray so it's testing
roughly the same thing it was testing before.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Slight change of strategy here; if we have trouble getting hold of a
page for whatever reason (eg a compound page is split underneath us),
don't spin to stabilise the page, just continue the iteration, like we
would if we failed to trylock the page. Since this is a speculative
optimisation, it feels like we should allow the process to take an extra
fault if it turns out to need this page instead of spending time to pin
down a page it may not need.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Slightly shorter and simpler code.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
The 'end' parameter of the xas_for_each iterator avoids a useless
iteration at the end of the range.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
There's no direct replacement for radix_tree_for_each_contig()
in the XArray API as it's an unusual thing to do. Instead,
open-code a loop using xas_next(). This removes the only user of
radix_tree_for_each_contig() so delete the iterator from the API and
the test suite code for it.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
The 'end' parameter of the xas_for_each iterator avoids a useless
iteration at the end of the range.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Slightly shorter and simpler code.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Slightly shorter and simpler code.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
The code is slightly shorter and simpler.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Use the XArray APIs to add and replace pages in the page cache. This
removes two uses of the radix tree preload API and is significantly
shorter code. It also removes the last user of __radix_tree_create()
outside radix-tree.c itself, so make it static.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
The page cache offers the ability to search for a miss in the previous or
next N locations. Rather than teach the XArray about the page cache's
definition of a miss, use xas_prev() and xas_next() to search the page
array. This should be more efficient as it does not have to start the
lookup from the top for each index.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Change i_pages from a radix_tree_root to an xarray, convert the
documentation into kernel-doc format and change the order of the elements
to pack them better on 64-bit systems.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Use the XA_TRACK_FREE ability to track which entries have a free bit,
similarly to how it uses the radix tree's IDR_FREE tag. This eliminates
the per-cpu ida_bitmap preload, and fixes the memory consumption
regression I introduced when making the IDR able to store any pointer.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Add the optional ability to track which entries in an XArray are free
and provide xa_alloc() to replace most of the functionality of the IDR.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
Add myself as XArray and IDR maintainer.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
This function reserves a slot in the XArray for users which need
to acquire multiple locks before storing their entry in the tree and
so cannot use a plain xa_store().
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
This hopefully temporary function is useful for users who have not yet
been converted to multi-index entries.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
This iterator iterates over each entry that is stored in the index or
indices specified by the xa_state. This is intended for use for a
conditional store of a multiindex entry, or to allow entries which are
about to be removed from the xarray to be disposed of properly.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
The xas_next and xas_prev functions move the xas index by one position,
and adjust the rest of the iterator state to match it. This is more
efficient than calling xas_set() as it keeps the iterator at the leaves
of the tree instead of walking the iterator from the root each time.
Signed-off-by: Matthew Wilcox <[email protected]>
|
|
This function frees all the internal memory allocated to the xarray
and reinitialises it to be empty.
Signed-off-by: Matthew Wilcox <[email protected]>
|