diff options
| author | Charan Teja Kalla <[email protected]> | 2023-12-14 04:58:41 +0000 |
|---|---|---|
| committer | Andrew Morton <[email protected]> | 2023-12-20 13:46:19 -0800 |
| commit | fc346d0a70a13d52fe1c4bc49516d83a42cd7c4c (patch) | |
| tree | a7185749258900db9678c580aa8f316b64dfa0ad | |
| parent | 4249f13c11be8b8b7bf93204185e150c3bdc968d (diff) | |
mm: migrate high-order folios in swap cache correctly
Large folios occupy N consecutive entries in the swap cache instead of
using multi-index entries like the page cache. However, if a large folio
is re-added to the LRU list, it can be migrated. The migration code was
not aware of the difference between the swap cache and the page cache and
assumed that a single xas_store() would be sufficient.
This leaves potentially many stale pointers to the now-migrated folio in
the swap cache, which can lead to almost arbitrary data corruption in the
future. This can also manifest as infinite loops with the RCU read lock
held.
[[email protected]: modifications to the changelog & tweaked the fix]
Fixes: 3417013e0d18 ("mm/migrate: Add folio_migrate_mapping()")
Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Charan Teja Kalla <[email protected]>
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Reported-by: Charan Teja Kalla <[email protected]>
Closes: https://lkml.kernel.org/r/[email protected]
Cc: David Hildenbrand <[email protected]>
Cc: Johannes Weiner <[email protected]>
Cc: Kirill A. Shutemov <[email protected]>
Cc: Naoya Horiguchi <[email protected]>
Cc: Shakeel Butt <[email protected]>
Cc: <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
| -rw-r--r-- | mm/migrate.c | 9 |
1 files changed, 8 insertions, 1 deletions
diff --git a/mm/migrate.c b/mm/migrate.c index 35a88334bb3c..397f2a6e34cb 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -405,6 +405,7 @@ int folio_migrate_mapping(struct address_space *mapping, int dirty; int expected_count = folio_expected_refs(mapping, folio) + extra_count; long nr = folio_nr_pages(folio); + long entries, i; if (!mapping) { /* Anonymous page without mapping */ @@ -442,8 +443,10 @@ int folio_migrate_mapping(struct address_space *mapping, folio_set_swapcache(newfolio); newfolio->private = folio_get_private(folio); } + entries = nr; } else { VM_BUG_ON_FOLIO(folio_test_swapcache(folio), folio); + entries = 1; } /* Move dirty while page refs frozen and newpage not yet exposed */ @@ -453,7 +456,11 @@ int folio_migrate_mapping(struct address_space *mapping, folio_set_dirty(newfolio); } - xas_store(&xas, newfolio); + /* Swap cache still stores N entries instead of a high-order entry */ + for (i = 0; i < entries; i++) { + xas_store(&xas, newfolio); + xas_next(&xas); + } /* * Drop cache reference from old page by unfreezing |