linux-IllusionX/fs
KAMEZAWA Hiroyuki 34e55232e5 mm: avoid false sharing of mm_counter
Considering the nature of per mm stats, it's the shared object among
threads and can be a cache-miss point in the page fault path.

This patch adds per-thread cache for mm_counter.  RSS value will be
counted into a struct in task_struct and synchronized with mm's one at
events.

Now, in this patch, the event is the number of calls to handle_mm_fault.
Per-thread value is added to mm at each 64 calls.

 rough estimation with small benchmark on parallel thread (2threads) shows
 [before]
     4.5 cache-miss/faults
 [after]
     4.0 cache-miss/faults
 Anyway, the most contended object is mmap_sem if the number of threads grows.

[akpm@linux-foundation.org: coding-style fixes]
Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
Cc: Christoph Lameter <cl@linux-foundation.org>
Cc: Lee Schermerhorn <lee.schermerhorn@hp.com>
Cc: David Rientjes <rientjes@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2010-03-06 11:26:24 -08:00
..
9p fs/9p: Add hardlink support to .u extension 2010-03-05 15:04:42 -06:00
adfs pass writeback_control to ->write_inode 2010-03-05 13:25:52 -05:00
affs pass writeback_control to ->write_inode 2010-03-05 13:25:52 -05:00
afs make sure data is on disk before calling ->write_inode 2010-03-05 13:25:10 -05:00
autofs
autofs4 Use kill_litter_super() in autofs4 ->kill_sb() 2010-03-03 14:07:54 -05:00
befs befs: fix leak 2010-02-07 03:06:21 -05:00
bfs pass writeback_control to ->write_inode 2010-03-05 13:25:52 -05:00
btrfs pass writeback_control to ->write_inode 2010-03-05 13:25:52 -05:00
cachefiles CacheFiles: Fix a race in cachefiles_delete_object() vs rename 2010-02-20 10:06:35 -05:00
cifs Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs-2.6 2010-03-04 08:15:33 -08:00
coda
configfs Fix configfs leak 2010-01-14 09:05:42 -05:00
cramfs
debugfs Lose the new_name argument of fsnotify_move() 2010-02-08 14:38:36 -05:00
devpts
dlm dlm: use bastmode in debugfs output 2010-02-26 12:15:54 -06:00
ecryptfs ecryptfs: use after free 2010-01-19 22:36:06 -06:00
efs
exofs pass writeback_control to ->write_inode 2010-03-05 13:25:52 -05:00
exportfs
ext2 Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs-2.6 2010-03-05 13:20:53 -08:00
ext3 Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs-2.6 2010-03-05 13:20:53 -08:00
ext4 Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs-2.6 2010-03-05 13:20:53 -08:00
fat pass writeback_control to ->write_inode 2010-03-05 13:25:52 -05:00
freevxfs
fscache
fuse Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse 2010-03-03 08:08:21 -08:00
gfs2 Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs-2.6 2010-03-05 13:20:53 -08:00
hfs pass writeback_control to ->write_inode 2010-03-05 13:25:52 -05:00
hfsplus pass writeback_control to ->write_inode 2010-03-05 13:25:52 -05:00
hostfs
hpfs Don't mess with generic_permission() under ->d_lock in hpfs 2010-03-03 14:07:58 -05:00
hppfs hppfs can use existing proc_mnt, no need for do_kern_mount() in there 2010-03-03 14:08:00 -05:00
hugetlbfs
isofs
jbd jbd: Delay discarding buffers in journal_unmap_buffer 2010-03-05 00:20:26 +01:00
jbd2 jbd2: clean up an assertion in jbd2_journal_commit_transaction() 2010-02-24 12:11:20 -05:00
jffs2
jfs Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs-2.6 2010-03-05 13:20:53 -08:00
lockd
minix pass writeback_control to ->write_inode 2010-03-05 13:25:52 -05:00
ncpfs
nfs Merge branch 'writeback-for-2.6.34' into nfs-for-2.6.34 2010-03-05 15:46:18 -05:00
nfs_common
nfsd Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs-2.6 2010-03-05 13:20:53 -08:00
nilfs2 Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs-2.6 2010-03-04 08:15:33 -08:00
nls
notify switch inotify_user to anon_inode 2010-02-19 03:35:12 -05:00
ntfs pass writeback_control to ->write_inode 2010-03-05 13:25:52 -05:00
ocfs2 bitops: rename for_each_bit() to for_each_set_bit() 2010-03-06 11:26:23 -08:00
omfs pass writeback_control to ->write_inode 2010-03-05 13:25:52 -05:00
openpromfs
partitions block: Stop using byte offsets 2010-01-11 14:30:09 +01:00
proc mm: clean up mm_counter 2010-03-06 11:26:23 -08:00
qnx4
quota quota: stop using QUOTA_OK / NO_QUOTA 2010-03-05 00:20:31 +01:00
ramfs nommu: fix shared mmap after truncate shrinkage problems 2010-01-16 12:15:40 -08:00
reiserfs Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs-2.6 2010-03-05 13:20:53 -08:00
romfs fix leak in romfs_fill_super() 2010-01-26 22:22:26 -05:00
smbfs
squashfs Squashfs: get rid of obsolete definition in header file 2010-03-05 15:35:35 +00:00
sysfs sysfs: sysfs_sd_setattr set iattrs unconditionally 2010-02-16 15:42:42 -08:00
sysv pass writeback_control to ->write_inode 2010-03-05 13:25:52 -05:00
ubifs pass writeback_control to ->write_inode 2010-03-05 13:25:52 -05:00
udf Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs-2.6 2010-03-05 13:20:53 -08:00
ufs Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs-2.6 2010-03-05 13:20:53 -08:00
xfs Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs-2.6 2010-03-05 13:20:53 -08:00
aio.c
anon_inodes.c Sanitize f_flags helpers 2009-12-22 12:27:34 -05:00
attr.c dquot: move dquot transfer responsibility into the filesystem 2010-03-05 00:20:28 +01:00
bad_inode.c
binfmt_aout.c Split 'flush_old_exec' into two functions 2010-01-29 08:22:01 -08:00
binfmt_elf.c Split 'flush_old_exec' into two functions 2010-01-29 08:22:01 -08:00
binfmt_elf_fdpic.c Split 'flush_old_exec' into two functions 2010-01-29 08:22:01 -08:00
binfmt_em86.c
binfmt_flat.c Split 'flush_old_exec' into two functions 2010-01-29 08:22:01 -08:00
binfmt_misc.c
binfmt_script.c
binfmt_som.c Split 'flush_old_exec' into two functions 2010-01-29 08:22:01 -08:00
bio-integrity.c block: fix bugs in bio-integrity mempool usage 2010-01-30 20:28:19 +01:00
bio.c Revert "blkdev: fix merge_bvec_fn return value checks" 2010-03-02 19:17:34 +01:00
block_dev.c freeze_bdev: don't deactivate successfully frozen MS_RDONLY sb 2010-02-07 03:06:21 -05:00
buffer.c
char_dev.c
compat.c
compat_binfmt_elf.c
compat_ioctl.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi-rc-fixes-2.6 2010-02-11 14:05:55 -08:00
dcache.c fix race in d_splice_alias() 2010-03-03 14:13:08 -05:00
dcookies.c
direct-io.c dio: fix use-after-free 2009-12-17 04:52:13 -05:00
drop_caches.c
eventfd.c eventfd - allow atomic read and waitqueue remove 2010-01-25 12:26:38 -02:00
eventpoll.c anonfd: Allow making anon files read-only 2009-12-22 12:27:34 -05:00
exec.c mm: avoid false sharing of mm_counter 2010-03-06 11:26:24 -08:00
fcntl.c Fix race in tty_fasync() properly 2010-02-07 10:26:01 -08:00
fifo.c
file.c vfs: Apply lockdep-based checking to rcu_dereference() uses 2010-02-25 10:34:48 +01:00
file_table.c Take ima_file_free() to proper place. 2010-02-07 03:07:29 -05:00
filesystems.c
fs-writeback.c pass writeback_control to ->write_inode 2010-03-05 13:25:52 -05:00
fs_struct.c
generic_acl.c
inode.c dquot: move dquot initialization responsibility into the filesystem 2010-03-05 00:20:30 +01:00
internal.h Take vfsmount_lock to fs/internal.h 2010-03-03 14:07:59 -05:00
ioctl.c
ioprio.c
Kconfig Revert "task_struct: make journal_info conditional" 2009-12-17 13:23:24 -08:00
Kconfig.binfmt
libfs.c libfs: Unexport and kill simple_prepare_write 2010-03-03 13:00:17 -05:00
locks.c Switch may_open() and break_lease() to passing O_... 2010-03-03 13:00:21 -05:00
Makefile
mbcache.c
mpage.c
namei.c Fix a dumb typo - use of & instead of && 2010-03-06 10:54:48 -08:00
namespace.c vfs: add NOFOLLOW flag to umount(2) 2010-03-03 14:08:00 -05:00
nfsctl.c Switch may_open() and break_lease() to passing O_... 2010-03-03 13:00:21 -05:00
no-block.c
open.c Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs-2.6 2010-03-05 13:20:53 -08:00
pipe.c fs: no games with DCACHE_UNHASHED 2009-12-17 10:51:40 -05:00
pnode.c Kill CL_PROPAGATION, sanitize fs/pnode.c:get_source() 2010-03-03 13:00:22 -05:00
pnode.h VFS: Clean up shared mount flag propagation 2010-03-03 14:07:55 -05:00
posix_acl.c
read_write.c
read_write.h
readdir.c
select.c
seq_file.c seq_file: add RCU versions of new hlist/list iterators (v3) 2010-02-22 15:45:54 -08:00
signalfd.c anonfd: Allow making anon files read-only 2009-12-22 12:27:34 -05:00
splice.c
stack.c VFS/fsstack: handle 32-bit smp + preempt + large files in fsstack_copy_inode_size 2009-12-17 10:58:17 -05:00
stat.c Add unlocked version of inode_add_bytes() function 2009-12-23 13:33:54 +01:00
super.c Mirror MS_KERNMOUNT in ->mnt_flags 2010-03-03 14:08:00 -05:00
sync.c quota: move code from sync_quota_sb into vfs_quota_sync 2010-03-05 00:20:24 +01:00
timerfd.c anonfd: Allow making anon files read-only 2009-12-22 12:27:34 -05:00
utimes.c
xattr.c
xattr_acl.c