mm: initialize pages on demand during boot

Deferred page initialization allows the boot cpu to initialize a small
subset of the system's pages early in boot, with other cpus doing the
rest later on.

It is, however, problematic to know how many pages the kernel needs
during boot.  Different modules and kernel parameters may change the
requirement, so the boot cpu either initializes too many pages or runs
out of memory.

To fix that, initialize early pages on demand.  This ensures the kernel
does the minimum amount of work to initialize pages during boot and
leaves the rest to be divided in the multithreaded initialization path
(deferred_init_memmap).

The on-demand code is permanently disabled using static branching once
deferred pages are initialized.  After the static branch is changed to
false, the overhead is up-to two branch-always instructions if the zone
watermark check fails or if rmqueue fails.

Sergey Senozhatsky noticed that while deferred pages currently make
sense only on NUMA machines (we start one thread per latency node),
CONFIG_NUMA is not a requirement for CONFIG_DEFERRED_STRUCT_PAGE_INIT,
so that is also must be addressed in the patch.

[[email protected]: fix typo in comment, make deferred_pages static]
[[email protected]: fix min() type mismatch warning]
  Link: http://lkml.kernel.org/r/[email protected]
[[email protected]: use zone_to_nid() in deferred_grow_zone()]
  Link: http://lkml.kernel.org/r/[email protected]
[[email protected]: might_sleep warning]
  Link: http://lkml.kernel.org/r/[email protected]
[[email protected]: s/spin_lock/spin_lock_irq/ in page_alloc_init_late()]
[[email protected]: v5]
  Link: http://lkml.kernel.org/r/[email protected]
[[email protected]: tweak comments]
[[email protected]: v6]
  Link: http://lkml.kernel.org/r/[email protected]
[[email protected]: coding-style fixes]
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Pavel Tatashin <[email protected]>
Reviewed-by: Daniel Jordan <[email protected]>
Reviewed-by: Steven Sistare <[email protected]>
Reviewed-by: Andrew Morton <[email protected]>
Tested-by: Masayoshi Mizuma <[email protected]>
Acked-by: Mel Gorman <[email protected]>
Cc: Michal Hocko <[email protected]>
Cc: Catalin Marinas <[email protected]>
Cc: AKASHI Takahiro <[email protected]>
Cc: Gioh Kim <[email protected]>
Cc: Heiko Carstens <[email protected]>
Cc: Yaowei Bai <[email protected]>
Cc: Wei Yang <[email protected]>
Cc: Paul Burton <[email protected]>
Cc: Miles Chen <[email protected]>
Cc: Vlastimil Babka <[email protected]>
Cc: Johannes Weiner <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>

0 files changed, 0 insertions, 0 deletions