github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-06-09 23:23:53 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph
42c85f01b8
linux/fs
History
Linus Torvalds d4c96061fd vfs: fix bad hashing of dentries 
commit 99d263d4c5 upstream.

Josef Bacik found a performance regression between 3.2 and 3.10 and
narrowed it down to commit bfcfaa77bd ("vfs: use 'unsigned long'
accesses for dcache name comparison and hashing"). He reports:

 "The test case is essentially

      for (i = 0; i < 1000000; i++)
              mkdir("a$i");

  On xfs on a fio card this goes at about 20k dir/sec with 3.2, and 12k
  dir/sec with 3.10.  This is because we spend waaaaay more time in
  __d_lookup on 3.10 than in 3.2.

  The new hashing function for strings is suboptimal for <
  sizeof(unsigned long) string names (and hell even > sizeof(unsigned
  long) string names that I've tested).  I broke out the old hashing
  function and the new one into a userspace helper to get real numbers
  and this is what I'm getting:

      Old hash table had 1000000 entries, 0 dupes, 0 max dupes
      New hash table had 12628 entries, 987372 dupes, 900 max dupes
      We had 11400 buckets with a p50 of 30 dupes, p90 of 240 dupes, p99 of 567 dupes for the new hash

  My test does the hash, and then does the d_hash into a integer pointer
  array the same size as the dentry hash table on my system, and then
  just increments the value at the address we got to see how many
  entries we overlap with.

  As you can see the old hash function ended up with all 1 million
  entries in their own bucket, whereas the new one they are only
  distributed among ~12.5k buckets, which is why we're using so much
  more CPU in __d_lookup".

The reason for this hash regression is two-fold:

 - On 64-bit architectures the down-mixing of the original 64-bit
   word-at-a-time hash into the final 32-bit hash value is very
   simplistic and suboptimal, and just adds the two 32-bit parts
   together.

   In particular, because there is no bit shuffling and the mixing
   boundary is also a byte boundary, similar character patterns in the
   low and high word easily end up just canceling each other out.

 - the old byte-at-a-time hash mixed each byte into the final hash as it
   hashed the path component name, resulting in the low bits of the hash
   generally being a good source of hash data.  That is not true for the
   word-at-a-time case, and the hash data is distributed among all the
   bits.

The fix is the same in both cases: do a better job of mixing the bits up
and using as much of the hash data as possible.  We already have the
"hash_32|64()" functions to do that.

Reported-by: Josef Bacik <jbacik@fb.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Chris Mason <clm@fb.com>
Cc: linux-fsdevel@vger.kernel.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-17 09:04:02 -07:00
..
9p aio: don't include aio.h in sched.h 2013-05-07 20:16:25 -07:00
adfs fs: Limit sys_mount to only request filesystem modules. 2013-03-03 19:36:31 -08:00
affs fs: Limit sys_mount to only request filesystem modules. 2013-03-03 19:36:31 -08:00
afs aio: don't include aio.h in sched.h 2013-05-07 20:16:25 -07:00
autofs4 autofs - remove autofs dentry mount check 2013-05-06 13:06:59 -07:00
befs befs_readdir(): do not increment ->f_pos if filldir tells us to stop 2013-05-31 15:17:56 -04:00
bfs fs: Limit sys_mount to only request filesystem modules. 2013-03-03 19:36:31 -08:00
btrfs Btrfs: fix csum tree corruption, duplicate and outdated checksums 2014-09-05 16:28:36 -07:00
cachefiles lift sb_start_write() out of ->write() 2013-04-09 14:12:56 -04:00
ceph ceph: allow sync_read/write return partial successed size of read/write. 2014-01-09 12:24:25 -08:00
cifs CIFS: Fix wrong restart readdir for SMB1 2014-09-17 09:04:01 -07:00
coda lift sb_start_write() out of ->write() 2013-04-09 14:12:56 -04:00
configfs configfs: fix race between dentry put and lookup 2013-11-29 11:11:53 -08:00
cramfs fs: Limit sys_mount to only request filesystem modules. 2013-03-03 19:36:31 -08:00
debugfs debugfs: debugfs_remove_recursive() must not rely on list_empty(d_subdirs) 2013-08-14 22:59:10 -07:00
devpts devpts: plug the memory leak in kill_sb 2013-12-04 10:55:49 -08:00
dlm Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next 2013-05-01 14:08:52 -07:00
ecryptfs ecryptfs: Fix memory leakage in keystore.c 2013-11-13 12:05:31 +09:00
efivarfs efivarfs: Never return ENOENT from firmware again 2013-05-13 20:12:10 +01:00
efs fs: Limit sys_mount to only request filesystem modules. 2013-03-03 19:36:31 -08:00
exofs ore: Fix wrong math in allocation of per device BIO 2014-02-13 13:48:00 -08:00
exportfs hlist: drop the node parameter from iterators 2013-02-27 19:10:24 -08:00
ext2 aio: don't include aio.h in sched.h 2013-05-07 20:16:25 -07:00
ext3 ext3,ext4: don't mess with dir_file->f_pos in htree_dirblock_to_tree() 2013-07-21 18:21:23 -07:00
ext4 ext4: fix BUG_ON in mb_free_blocks() 2014-09-05 16:28:36 -07:00
f2fs f2fs updates for v3.10 2013-05-08 15:11:48 -07:00
fat fat: fix possible overflow for fat_clusters 2013-05-24 16:22:50 -07:00
freevxfs fs: Readd the fs module aliases. 2013-03-12 18:55:21 -07:00
fscache fs/fscache/stats.c: fix memory leak 2013-04-29 15:54:27 -07:00
fuse fuse: handle large user and group ID 2014-07-28 08:00:02 -07:00
gfs2 GFS2: Increase i_writecount during gfs2_setattr_chown 2014-01-25 08:27:11 -08:00
hfs hfs: avoid crash in hfs_bnode_create 2013-05-24 16:22:51 -07:00
hfsplus aio: don't include aio.h in sched.h 2013-05-07 20:16:25 -07:00
hostfs hostfs: use kmalloc instead of kzalloc 2013-05-04 15:48:45 -04:00
hpfs hpfs: better test for errors 2013-07-13 11:42:26 -07:00
hppfs hppfs: get rid of ->fsync() 2013-04-29 15:41:42 -04:00
hugetlbfs cope with potentially long ->d_dname() output for shmem/hugetlb 2013-10-18 07:45:45 -07:00
isofs isofs: Fix unbounded recursion when processing relocated directories 2014-09-05 16:28:34 -07:00
jbd Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs 2013-05-03 09:56:25 -07:00
jbd2 jbd2: fix infinite loop when recovering corrupt journal blocks 2014-09-05 16:28:36 -07:00
jffs2 jffs2: remove from wait queue after schedule() 2014-04-26 17:15:36 -07:00
jfs jfs: fix error path in ialloc 2013-11-13 12:05:31 +09:00
lockd lockd: ensure we tear down any live sockets when socket creation fails during lockd_up 2014-05-13 13:59:46 +02:00
logfs block: Remove bi_idx references 2013-03-23 14:15:31 -07:00
minix fs: Limit sys_mount to only request filesystem modules. 2013-03-03 19:36:31 -08:00
ncpfs ncpfs: fix rmdir returns Device or resource busy 2013-06-07 12:15:38 -04:00
nfs NFSv4: Fix problems with close in the presence of a delegation 2014-09-05 16:28:37 -07:00
nfs_common nfs_common: Update the translation between nfsv3 acls linux posix acls 2013-02-13 06:15:14 -08:00
nfsd svcrdma: Select NFSv4.1 backchannel transport based on forward channel 2014-09-05 16:28:37 -07:00
nilfs2 nilfs2: fix segctor bug that causes file system corruption 2014-01-25 08:27:12 -08:00
nls
notify compat: fix sys_fanotify_mark 2014-02-13 13:48:00 -08:00
ntfs aio: don't include aio.h in sched.h 2013-05-07 20:16:25 -07:00
ocfs2 ocfs2: do not put bh when buffer_uptodate failed 2014-05-06 07:55:32 -07:00
omfs fs: Limit sys_mount to only request filesystem modules. 2013-03-03 19:36:31 -08:00
openpromfs fs: Limit sys_mount to only request filesystem modules. 2013-03-03 19:36:31 -08:00
proc CAPABILITIES: remove undefined caps from all processes 2014-09-17 09:03:57 -07:00
pstore Couple of pstore cleanups 2013-05-09 16:42:10 -07:00
qnx4 fs: Limit sys_mount to only request filesystem modules. 2013-03-03 19:36:31 -08:00
qnx6 qnx6: qnx6_readdir() has a braino in pos calculation 2013-05-31 15:17:31 -04:00
quota quota: Fix race between dqput() and dquot_scan_active() 2014-03-06 21:30:12 -08:00
ramfs Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2013-02-26 20:16:07 -08:00
reiserfs reiserfs: call truncate_setsize under tailpack mutex 2014-07-06 18:54:15 -07:00
romfs romfs: fix nommu map length to keep inside filesystem 2013-04-29 09:17:57 +10:00
squashfs fs: Limit sys_mount to only request filesystem modules. (Part 3) 2013-03-11 07:09:48 -07:00
sysfs sysfs: check if one entry has been removed before freeing 2013-04-05 15:35:52 -07:00
sysv sysv: Add forgotten superblock lock init for v7 fs 2013-10-05 07:13:09 -07:00
ubifs UBIFS: Remove incorrect assertion in shrink_tnc() 2014-07-06 18:54:13 -07:00
udf udf: Refuse RW mount of the filesystem instead of making it RO 2013-10-01 09:17:48 -07:00
ufs Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial 2013-04-30 09:36:50 -07:00
xfs xfs: don't zero partial page cache pages during O_DIRECT writes 2014-09-17 09:04:01 -07:00
aio.c aio: fix kernel memory disclosure in io_getevents() introduced in v3.10 2014-06-30 20:09:45 -07:00
anon_inodes.c get_empty_filp()/alloc_file() leave both ->f_pos and ->f_version zero 2013-02-26 02:46:11 -05:00
attr.c fs,userns: Change inode_capable to capable_wrt_inode_uidgid 2014-06-16 13:42:52 -07:00
bad_inode.c lseek: the "whence" argument is called "whence" 2012-12-17 17:15:12 -08:00
binfmt_aout.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2013-05-01 17:51:54 -07:00
binfmt_elf_fdpic.c Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/benh/powerpc 2013-05-02 10:16:16 -07:00
binfmt_elf.c fs/binfmt_elf.c: prevent a coredump with a large vm_map_count from Oopsing 2013-10-13 16:08:31 -07:00
binfmt_em86.c exec: use -ELOOP for max recursion depth 2012-12-17 17:15:23 -08:00
binfmt_flat.c new helper: read_code() 2013-04-29 15:40:23 -04:00
binfmt_misc.c binfmt_misc: reuse string_unescape_inplace() 2013-04-30 17:04:03 -07:00
binfmt_script.c exec: do not leave bprm->interp on stack 2012-12-20 17:40:19 -08:00
binfmt_som.c get rid of pt_regs argument of ->load_binary() 2012-11-28 21:53:38 -05:00
bio-integrity.c bio-integrity: Fix bio_integrity_verify segment start bug 2014-03-23 21:38:21 -07:00
bio.c block: Fix bio_copy_data() 2013-10-05 07:13:09 -07:00
block_dev.c writeback: Fix periodic writeback after fs mount 2013-07-28 16:29:40 -07:00
buffer.c mm: __set_page_dirty uses spin_lock_irqsave instead of spin_lock_irq 2014-02-20 11:06:11 -08:00
char_dev.c char_dev: pin parent kobject 2012-10-22 08:50:37 +03:00
compat_binfmt_elf.c coredump: extend core dump note section to contain file names of mapped files 2012-10-06 03:05:17 +09:00
compat_ioctl.c Removed unused typedef to avoid "unused local typedef" warnings. 2013-05-04 15:03:05 -04:00
compat.c aio: don't include aio.h in sched.h 2013-05-07 20:16:25 -07:00
coredump.c coredump: fix the setting of PF_DUMPCORE 2014-07-31 12:53:50 -07:00
coredump.h coredump: update coredump-related headers 2012-10-06 03:05:15 +09:00
dcache.c vfs: fix bad hashing of dentries 2014-09-17 09:04:02 -07:00
dcookies.c fs/compat: fix lookup_dcookie() parameter handling 2014-02-13 13:48:00 -08:00
direct-io.c Merge branch 'for-3.10/core' of git://git.kernel.dk/linux-block 2013-05-08 10:13:35 -07:00
drop_caches.c
eventfd.c fs, eventfd: add procfs fdinfo helper 2012-12-17 17:15:27 -08:00
eventpoll.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/signal 2013-05-01 07:21:43 -07:00
exec.c metag: Reduce maximum stack size to 256MB 2014-06-07 13:25:38 -07:00
fcntl.c new helper: file_inode(file) 2013-02-22 23:31:31 -05:00
fhandle.c Merge branch 'for-3.8' of git://linux-nfs.org/~bfields/linux 2012-12-20 14:04:11 -08:00
file_table.c don't bother with {get,put}_write_access() on non-regular files 2014-05-30 21:52:12 -07:00
file.c fs/file.c:fdtable: avoid triggering OOMs from alloc_fdmem 2014-02-22 12:41:25 -08:00
filesystems.c fs: Limit sys_mount to only request filesystem modules. 2013-03-03 19:36:31 -08:00
fs_struct.c constify path_get/path_put and fs_struct.c stuff 2013-03-01 23:51:07 -05:00
fs-writeback.c bdi: avoid oops on device removal 2014-04-26 17:15:35 -07:00
generic_acl.c
inode.c fs,userns: Change inode_capable to capable_wrt_inode_uidgid 2014-06-16 13:42:52 -07:00
internal.h splice: don't pass the address of ->f_pos to methods 2013-06-20 19:02:45 +04:00
ioctl.c new helper: file_inode(file) 2013-02-22 23:31:31 -05:00
ioprio.c
Kconfig efivarfs: Move to fs/efivarfs 2013-04-17 13:25:09 +01:00
Kconfig.binfmt fs: make binfmt support for #! scripts modular and removable 2013-04-30 17:04:04 -07:00
libfs.c vfs: drop vmtruncate 2012-12-20 18:46:29 -05:00
locks.c locks: allow __break_lease to sleep even when break_time is 0 2014-05-13 13:59:44 +02:00
Makefile Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2013-05-01 17:51:54 -07:00
mbcache.c
mount.h vfs: Is mounted should be testing mnt_ns for NULL or error. 2014-02-06 11:08:16 -08:00
mpage.c
namei.c vfs: fix bad hashing of dentries 2014-09-17 09:04:02 -07:00
namespace.c mnt: Change the default remount atime from relatime to the existing value 2014-09-17 09:04:00 -07:00
no-block.c
open.c don't bother with {get,put}_write_access() on non-regular files 2014-05-30 21:52:12 -07:00
pipe.c vfs: fix subtle use-after-free of pipe_inode_info 2013-12-11 22:36:26 -08:00
pnode.c vfs: Fix invalid ida_remove() call 2013-05-31 15:16:33 -04:00
pnode.h Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2013-05-01 17:51:54 -07:00
posix_acl.c posix_acl: handle NULL ACL in posix_acl_equiv_mode 2014-06-07 13:25:33 -07:00
proc_namespace.c
read_write.c fs/compat: fix parameter handling for compat readv/writev syscalls 2014-02-13 13:48:00 -08:00
readdir.c new helper: file_inode(file) 2013-02-22 23:31:31 -05:00
select.c sched/rt: Move rt specific bits into new header file 2013-02-07 20:51:08 +01:00
seq_file.c seq_file: always update file->f_pos in seq_lseek() 2013-11-13 12:05:34 +09:00
signalfd.c switch signalfd{,4}() to COMPAT_SYSCALL_DEFINE 2013-03-03 22:58:46 -05:00
splice.c fuse: fix pipe_buf_operations 2014-02-13 13:47:59 -08:00
stack.c
stat.c switch vfs_getattr() to struct path 2013-02-26 02:46:08 -05:00
statfs.c vfs: allow O_PATH file descriptors for fstatfs() 2013-10-18 07:45:44 -07:00
super.c livelock avoidance in sget() 2013-08-04 16:51:15 +08:00
sync.c teach SYSCALL_DEFINE<n> how to deal with long long/unsigned long long 2013-03-03 22:46:22 -05:00
timerfd.c compat: restore timerfd settime and gettime compat syscalls 2013-03-02 09:35:13 -05:00
utimes.c vfs: allow utimensat() calls to retry once on an ESTALE error 2012-12-20 18:50:08 -05:00
xattr_acl.c userns: Fix posix_acl_file_xattr_userns gid conversion 2012-10-12 13:16:48 -07:00
xattr.c vfs: make lremovexattr retry once on ESTALE error 2012-12-20 18:50:11 -05:00