Commit Graph

378370 Commits

Author SHA1 Message Date
Jeff Vander Stoep
c8c3cd48e4 security: lsm_audit: add ioctl specific auditing
Add information about ioctl calls to the LSM audit data. Log the
file path and command number.

Bug: 18087110
Change-Id: Idbbd106db6226683cb30022d9e8f6f3b8fab7f84
Signed-off-by: Jeff Vander Stoep <jeffv@google.com>
2015-04-13 19:55:16 +00:00
jinqian
c2e60d7cef Power: Report suspend times from last_suspend_time
This node epxorts two values separated by space.
From left to right:
1. time spent in suspend/resume process
2. time spent sleep in suspend state

Change-Id: I2cb9a9408a5fd12166aaec11b935a0fd6a408c63
2015-04-09 18:50:24 +00:00
Sami Tolvanen
11a2b3dd2e arm: crypto: Add optimized SHA-256/224
Add Andy Polyakov's optimized assembly and NEON implementations for
SHA-256/224.

The sha256-armv4.pl script for generating the assembly code is from
OpenSSL commit 51f8d095562f36cdaa6893597b5c609e943b0565.

Compared to sha256-generic these implementations have the following
tcrypt speed improvements on Motorola Nexus 6 (Snapdragon 805):

  bs    b/u      sha256-neon  sha256-asm
  16    16       x1.32        x1.19
  64    16       x1.27        x1.15
  64    64       x1.36        x1.20
  256   16       x1.22        x1.11
  256   64       x1.36        x1.19
  256   256      x1.59        x1.23
  1024  16       x1.21        x1.10
  1024  256      x1.65        x1.23
  1024  1024     x1.76        x1.25
  2048  16       x1.21        x1.10
  2048  256      x1.66        x1.23
  2048  1024     x1.78        x1.25
  2048  2048     x1.79        x1.25
  4096  16       x1.20        x1.09
  4096  256      x1.66        x1.23
  4096  1024     x1.79        x1.26
  4096  4096     x1.82        x1.26
  8192  16       x1.20        x1.09
  8192  256      x1.67        x1.23
  8192  1024     x1.80        x1.26
  8192  4096     x1.85        x1.28
  8192  8192     x1.85        x1.27

Where bs refers to block size and b/u to bytes per update.

Change-Id: I83938010007660f7f3f77f2946c8d22557e3a327
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
2015-04-09 17:39:04 +00:00
jinqian
3aaf46626a proc: uid: Adds accounting for the cputimes per uid.
Adds proc files /proc/uid_cputime/show_uid_stat and
/proc/uid_cputime/remove_uid_range.

show_uid_stat lists the total utime and stime for the active as well as
terminated processes for each of the uids.

Writing a range of uids to remove_uid_range will delete the accounting
for all the uids within that range.

Change-Id: I21d9210379da730b33ddc1a0ea663c8c9d2ac15b
2015-04-06 23:37:40 +00:00
Mark Salyzyn
ae2019d83e USB: gadget: android: Integrate f_midi USB MIDI gadget driver part deux
free_ep_req and alloc_ep_req are static open coded and conflicting
in f_midi.c. The exported versions are present in f_sourcesink.c.
Changed names to protect the innocent.

Change-Id: I4aee40054b5715d0532d433d23dea2bccff7ec30
Signed-off-by: Mark Salyzyn <salyzyn@google.com>
2015-03-24 12:42:35 -07:00
Mark Salyzyn
b34ee78acf android: base-cfg: disable ALARM_DEV
Signed-off-by: Mark Salyzyn <salyzyn@google.com>
Change-Id: If8d324ffdb4ebd56e5d68876f8e229547e20eaf5
2015-03-23 14:23:48 -07:00
John Stultz
a290c3e430 staging: Remove logger and alarm-dev from android Makefile
My previous patches deleting logger and alarm-dev from staging
missed the android Makefile.

This patch cleans up the Makefile to remove the now non-existent
files.

Cc: Rom Lemarchand <romlem@google.com>,
Cc: Mark Salyzyn <salyzyn@google.com>,
Cc: Kees Cook <keescook@chromium.org>,
Cc: Android Kernel Team <kernel-team@android.com>,
Cc: Valentin Rothberg <valentinrothberg@gmail.com>,
Cc: Greg Hackmann <ghackmann@google.com>,
Cc: Elliott Hughes <enh@google.com>,
Cc: Todd Poynor <toddpoynor@google.com>,
Reported-by: Paul Bolle <pebolle@tiscali.nl>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Acked-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Bug: 13505761
Change-Id: Id8c027ce51a663ee2e9ea2618612921db7cf7635
2015-03-23 14:19:52 -07:00
John Stultz
013845fd73 staging: Remove the Android alarm-dev driver
The functionality provided by the Android alarm-dev driver
should now be present in the timerfd interface (thanks to
Greg Hackmann and Todd Poynor).

As of Lollipop, AOSP can make use of the timerfd if
alarm-dev is not present (though a fixup for setting the
rtc time if rtc0 isn't the backing for _ALARM clockids has
been applied post-Lollipop).

Thus, we should be able to remove alarm-dev from staging.

Cc: Greg Hackmann <ghackmann@google.com>
Cc: Elliott Hughes <enh@google.com>
Cc: Todd Poynor <toddpoynor@google.com>
Cc: Android Kernel Team <kernel-team@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Acked-by: Mark Salyzyn <salyzyn@google.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Change-Id: Ia905d4b809cc1614ddde01ccb791fc56ac292fa7
2015-03-23 14:19:46 -07:00
John Stultz
4f8c75c8d9 staging: Remove the Android logger driver
With the relase of Lollipop, Android no longer
requires the logger driver.

There are three patches which the android dev's
still need before they drop logger on all their
devices:

[PATCH v4 1/5] pstores: use scnprintf
[PATCH v2 2/5] pstore: remove superfluous memory size check
[PATCH 3/5] pstore: handle zero-sized prz in series
[PATCH v4 4/5] pstore: add pmsg
[PATCH 5/5] pstore: selinux: add security in-core xattr support for pstore and debugfs

But these seem to have been acked and are hopefully
queued for upstream.

So this patch removes the logger driver from staging.

Cc: Rom Lemarchand <romlem@google.com>,
Cc: Mark Salyzyn <salyzyn@google.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Android Kernel Team <kernel-team@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Bug: 13505761
Change-Id: I21b6897f01871851e05b6eb53c7c08a1cb597e3d
2015-03-23 14:19:08 -07:00
Greg Hackmann
ad41e06766 net: ping: fix constness inconsistency in ipv6_chk_addr
The ipv6_chk_addr declarations and implementations aren't consistent
about the dev parameter's constness.  6bc19fb upstream resolved this in
favor of const, so let's do the same here.

This fixes the following warning:

net/ipv6/ping.c: In function 'pingv6_init':
net/ipv6/ping.c:87:27: warning: assignment from incompatible pointer type
  pingv6_ops.ipv6_chk_addr = ipv6_chk_addr;
                           ^

Change-Id: Ibd9efd7d489e661f71ab641c981689bf92258b7c
Signed-off-by: Greg Hackmann <ghackmann@google.com>
2015-03-16 15:23:53 -07:00
Mike Lockwood
3f048135f3 USB: gadget: android: Integrate f_midi USB MIDI gadget driver
Change-Id: I4c4b99f9d54314fc31445cf42b825527ca483af9
Signed-off-by: Mike Lockwood <lockwood@google.com>
2015-03-16 11:05:34 -07:00
Mike Lockwood
d9de6bef15 USB: gadget: f_audio_source: Rename symbols to avoid conflicts with f_midi
Change-Id: I424b942d64c9f818351e49ee2150f466b9cc7f84
Signed-off-by: Mike Lockwood <lockwood@google.com>
2015-03-16 10:51:35 -07:00
Mike Lockwood
261230ce54 USB: gadget: midi: Fix deadlock unbinding driver while it is in use
Using snd_card_free_when_closed rather than snd_card_free in f_midi_unbind
makes it safe to disable the driver while a userspace client has the
ALSA device open.

Change-Id: Ibc40c01e7b1ce90fc61d3ea654b4816fadfc7ffd
Signed-off-by: Mike Lockwood <lockwood@google.com>
2015-03-16 10:51:35 -07:00
Amit Pundir
4be5c6a45a fs: ecryptfs: readdir: constify actor
actor is a constant in dir_context struct and
because of that we run into following build failure:
----------
fs/ecryptfs/file.c: In function ‘ecryptfs_readdir’:
fs/ecryptfs/file.c:130:2: error: assignment of read-only member ‘actor’
make[2]: *** [fs/ecryptfs/file.o] Error 1
make[1]: *** [fs/ecryptfs] Error 2
make: *** [fs] Error 2
----------

This fix is based on commit: b2497fc3057a([readdir] constify ->actor)

Signed-off-by: Amit Pundir <amit.pundir@linaro.org>
2015-03-13 14:25:54 +05:30
Jeff Vander Stoep
e48e8c4592 mm: reorder can_do_mlock to fix audit denial
A userspace call to mmap(MAP_LOCKED) may result in the successful locking
of memory while also producing a confusing audit log denial.  can_do_mlock
checks capable and rlimit.  If either of these return positive
can_do_mlock returns true.  The capable check leads to an LSM hook used by
apparmour and selinux which produce the audit denial.  Reordering so
rlimit is checked first eliminates the denial on success, only recording a
denial when the lock is unsuccessful as a result of the denial.

Signed-off-by: Jeff Vander Stoep <jeffv@google.com>
Acked-by: Nick Kralevich <nnk@google.com>
Cc: Jeff Vander Stoep <jeffv@google.com>
Cc: Sasha Levin <sasha.levin@oracle.com>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Paul Cassella <cassella@cray.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2015-03-11 14:32:24 -07:00
Sami Tolvanen
062a6b5efe dm-verity: Add modes and emit uevent on corrupted blocks
Add a device specific mode to dm-verity for handling corrupted
blocks:

  DM_VERITY_MODE_EIO is the default behavior, where reading a
  corrupted block results in -EIO.

  DM_VERITY_MODE_LOGGING only logs corrupted blocks, but does
  not block the read.

  DM_VERITY_MODE_RESTART calls kernel_restart when a corrupted
  block is discovered.

Each mode sends a uevent to notify userspace of corruption and
allow further recovery actions.

Defaults to previous behavior, other modes can be enabled with
an optional parameter added to the verity table.

Change-Id: Ib72ae6ccb865594d28f3553bdcc5a40b1d7af390
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
2015-03-11 18:17:56 +00:00
Rom Lemarchand
7b4fff3195 proc: make oom adjustment files user read-only
Make oom_adj and oom_score_adj user read-only.

Bug: 19636629
Change-Id: I055bb172d5b4d3d856e25918f3c5de8edf31e4a3
Signed-off-by: Rom Lemarchand <romlem@google.com>
2015-03-09 19:45:15 -07:00
Rom Lemarchand
4f1d293713 Revert "Grants system server access to /proc/<pid>/oom_adj for Android applications."
This reverts commit aa3305f2ba.

Bug: 19636629
Change-Id: I8dcd6a12cc591cb410d6a799c7e875726495504f
2015-03-09 19:38:56 -07:00
Petr Cermak
740006ef28 fs/proc/task_mmu.c: add user-space support for resetting mm->hiwater_rss (peak RSS)
Peak resident size of a process can be reset back to the process's
current rss value by writing "5" to /proc/pid/clear_refs.  The driving
use-case for this would be getting the peak RSS value, which can be
retrieved from the VmHWM field in /proc/pid/status, per benchmark
iteration or test scenario.

Origin:
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=695f055936938c674473ea071ca7359a863551e7

[akpm@linux-foundation.org: clarify behaviour in documentation]
Signed-off-by: Petr Cermak <petrcermak@chromium.org>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Primiano Tucci <primiano@chromium.org>
Cc: Petr Cermak <petrcermak@chromium.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Change-Id: I543a7640639d9916e813af875003fe3ee3a6bfe0
2015-03-06 19:44:26 +00:00
Lorenzo Colitti
428e6d675d net: ping: Return EAFNOSUPPORT when appropriate.
1. For an IPv4 ping socket, ping_check_bind_addr does not check
   the family of the socket address that's passed in. Instead,
   make it behave like inet_bind, which enforces either that the
   address family is AF_INET, or that the family is AF_UNSPEC and
   the address is 0.0.0.0.
2. For an IPv6 ping socket, ping_check_bind_addr returns EINVAL
   if the socket family is not AF_INET6. Return EAFNOSUPPORT
   instead, for consistency with inet6_bind.
3. Make ping_v4_sendmsg and ping_v6_sendmsg return EAFNOSUPPORT
   instead of EINVAL if an incorrect socket address structure is
   passed in.
4. Make IPv6 ping sockets be IPv6-only. The code does not support
   IPv4, and it cannot easily be made to support IPv4 because
   the protocol numbers for ICMP and ICMPv6 are different. This
   makes connect(::ffff:192.0.2.1) fail with EAFNOSUPPORT instead
   of making the socket unusable.

Among other things, this fixes an oops that can be triggered by:

    int s = socket(AF_INET, SOCK_DGRAM, IPPROTO_ICMP);
    struct sockaddr_in6 sin6 = {
        .sin6_family = AF_INET6,
        .sin6_addr = in6addr_any,
    };
    bind(s, (struct sockaddr *) &sin6, sizeof(sin6));

[backport of net 9145736d48]

Change-Id: If06ca86d9f1e4593c0d6df174caca3487c57a241
Signed-off-by: Lorenzo Colitti <lorenzo@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2015-03-05 16:51:54 +09:00
Sergey Senozhatsky
92b78405e5 lz4: fix compression/decompression signedness mismatch
LZ4 compression and decompression functions require different in
signedness input/output parameters: unsigned char for compression and
signed char for decompression.

Change decompression API to require "(const) unsigned char *".

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Cc: Kyungsik Lee <kyungsik.lee@lge.com>
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: Yann Collet <yann.collet.73@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-03-04 11:31:37 -08:00
Chanho Min
7207f784ce lib: add lz4 compressor module
This patchset is for supporting LZ4 compression and the crypto API using
it.

As shown below, the size of data is a little bit bigger but compressing
speed is faster under the enabled unaligned memory access.  We can use
lz4 de/compression through crypto API as well.  Also, It will be useful
for another potential user of lz4 compression.

lz4 Compression Benchmark:
Compiler: ARM gcc 4.6.4
ARMv7, 1 GHz based board
   Kernel: linux 3.4
   Uncompressed data Size: 101 MB
         Compressed Size  compression Speed
   LZO   72.1MB		  32.1MB/s, 33.0MB/s(UA)
   LZ4   75.1MB		  30.4MB/s, 35.9MB/s(UA)
   LZ4HC 59.8MB		   2.4MB/s,  2.5MB/s(UA)
- UA: Unaligned memory Access support
- Latest patch set for LZO applied

This patch:

Add support for LZ4 compression in the Linux Kernel.  LZ4 Compression APIs
for kernel are based on LZ4 implementation by Yann Collet and were changed
for kernel coding style.

LZ4 homepage : http://fastcompression.blogspot.com/p/lz4.html
LZ4 source repository : http://code.google.com/p/lz4/
svn revision : r90

Two APIs are added:

lz4_compress() support basic lz4 compression whereas lz4hc_compress()
support high compression or CPU performance get lower but compression
ratio get higher.  Also, we require the pre-allocated working memory with
the defined size and destination buffer must be allocated with the size of
lz4_compressbound.

[akpm@linux-foundation.org: make lz4_compresshcctx() static]
Signed-off-by: Chanho Min <chanho.min@lge.com>
Cc: "Darrick J. Wong" <djwong@us.ibm.com>
Cc: Bob Pearson <rpearson@systemfabricworks.com>
Cc: Richard Weinberger <richard@nod.at>
Cc: Herbert Xu <herbert@gondor.hengli.com.au>
Cc: Yann Collet <yann.collet.73@gmail.com>
Cc: Kyungsik Lee <kyungsik.lee@lge.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-03-04 11:30:47 -08:00
Kyungsik Lee
d10a716c35 decompressor: add LZ4 decompressor module
Add support for LZ4 decompression in the Linux Kernel.  LZ4 Decompression
APIs for kernel are based on LZ4 implementation by Yann Collet.

Benchmark Results(PATCH v3)
Compiler: Linaro ARM gcc 4.6.2

1. ARMv7, 1.5GHz based board
   Kernel: linux 3.4
   Uncompressed Kernel Size: 14MB
        Compressed Size  Decompression Speed
   LZO  6.7MB            20.1MB/s, 25.2MB/s(UA)
   LZ4  7.3MB            29.1MB/s, 45.6MB/s(UA)

2. ARMv7, 1.7GHz based board
   Kernel: linux 3.7
   Uncompressed Kernel Size: 14MB
        Compressed Size  Decompression Speed
   LZO  6.0MB            34.1MB/s, 52.2MB/s(UA)
   LZ4  6.5MB            86.7MB/s
- UA: Unaligned memory Access support
- Latest patch set for LZO applied

This patch set is for adding support for LZ4-compressed Kernel.  LZ4 is a
very fast lossless compression algorithm and it also features an extremely
fast decoder [1].

But we have five of decompressors already and one question which does
arise, however, is that of where do we stop adding new ones?  This issue
had been discussed and came to the conclusion [2].

Russell King said that we should have:

 - one decompressor which is the fastest
 - one decompressor for the highest compression ratio
 - one popular decompressor (eg conventional gzip)

If we have a replacement one for one of these, then it should do exactly
that: replace it.

The benchmark shows that an 8% increase in image size vs a 66% increase
in decompression speed compared to LZO(which has been known as the
fastest decompressor in the Kernel).  Therefore the "fast but may not be
small" compression title has clearly been taken by LZ4 [3].

[1] http://code.google.com/p/lz4/
[2] http://thread.gmane.org/gmane.linux.kbuild.devel/9157
[3] http://thread.gmane.org/gmane.linux.kbuild.devel/9347

LZ4 homepage: http://fastcompression.blogspot.com/p/lz4.html
LZ4 source repository: http://code.google.com/p/lz4/

Signed-off-by: Kyungsik Lee <kyungsik.lee@lge.com>
Signed-off-by: Yann Collet <yann.collet.73@gmail.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Russell King <rmk@arm.linux.org.uk>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Florian Fainelli <florian@openwrt.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-03-04 11:28:05 -08:00
Phillip Lougher
163b9ffb69 Squashfs: Add LZ4 compression configuration option
Add the glue code, and also update the documentation.

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
2015-03-04 11:27:29 -08:00
Phillip Lougher
5c913a40d2 Squashfs: add LZ4 compression support
Add support for reading file systems compressed with the
LZ4 compression algorithm.

This patch adds the LZ4 decompressor wrapper code.

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
2015-03-04 11:27:24 -08:00
Fabian Frederick
b9281218c2 fs/squashfs/super.c: logging cleanup
- Convert printk to pr_foo()
- Add pr_fmt for future logging entries
- Coalesce formats

Signed-off-by: Fabian Frederick <fabf@skynet.be>
Cc: Phillip Lougher <phillip@squashfs.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-03-04 11:27:19 -08:00
Fabian Frederick
7fcc243f30 fs/squashfs/file_direct.c: replace count*size kmalloc by kmalloc_array
kmalloc_array() manages count*sizeof overflow.

Signed-off-by: Fabian Frederick <fabf@skynet.be>
Cc: Phillip Lougher <phillip@squashfs.org.uk>
Cc: Joe Perches <joe@perches.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-03-04 11:27:13 -08:00
Fabian Frederick
d933d04a83 fs/squashfs/squashfs.h: replace pr_warning by pr_warn
Update the last pr_warning callsite in fs branch

Signed-off-by: Fabian Frederick <fabf@skynet.be>
Cc: Phillip Lougher <phillip@squashfs.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-03-04 11:27:08 -08:00
Theodore Ts'o
e57e5b786f fs: push sync_filesystem() down to the file system's remount_fs()
Previously, the no-op "mount -o mount /dev/xxx" operation when the
file system is already mounted read-write causes an implied,
unconditional syncfs().  This seems pretty stupid, and it's certainly
documented or guaraunteed to do this, nor is it particularly useful,
except in the case where the file system was mounted rw and is getting
remounted read-only.

However, it's possible that there might be some file systems that are
actually depending on this behavior.  In most file systems, it's
probably fine to only call sync_filesystem() when transitioning from
read-write to read-only, and there are some file systems where this is
not needed at all (for example, for a pseudo-filesystem or something
like romfs).

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: linux-fsdevel@vger.kernel.org
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Artem Bityutskiy <dedekind1@gmail.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Evgeniy Dushistov <dushistov@mail.ru>
Cc: Jan Kara <jack@suse.cz>
Cc: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
Cc: Anders Larsen <al@alarsen.net>
Cc: Phillip Lougher <phillip@squashfs.org.uk>
Cc: Kees Cook <keescook@chromium.org>
Cc: Mikulas Patocka <mikulas@artax.karlin.mff.cuni.cz>
Cc: Petr Vandrovec <petr@vandrovec.name>
Cc: xfs@oss.sgi.com
Cc: linux-btrfs@vger.kernel.org
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Cc: codalist@coda.cs.cmu.edu
Cc: linux-ext4@vger.kernel.org
Cc: linux-f2fs-devel@lists.sourceforge.net
Cc: fuse-devel@lists.sourceforge.net
Cc: cluster-devel@redhat.com
Cc: linux-mtd@lists.infradead.org
Cc: jfs-discussion@lists.sourceforge.net
Cc: linux-nfs@vger.kernel.org
Cc: linux-nilfs@vger.kernel.org
Cc: linux-ntfs-dev@lists.sourceforge.net
Cc: ocfs2-devel@oss.oracle.com
Cc: reiserfs-devel@vger.kernel.org
2015-03-04 11:26:46 -08:00
Phillip Lougher
3a2109763d Squashfs: fix failure to unlock pages on decompress error
Direct decompression into the page cache.  If we fall back
to using an intermediate buffer (because we cannot grab all the
page cache pages) and we get a decompress fail, we forgot to
release the pages.

Reported-by: Roman Peniaev <r.peniaev@gmail.com>
Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
2015-03-04 11:23:02 -08:00
Phillip Lougher
6a1e8fd1e4 Squashfs: Check stream is not NULL in decompressor_multi.c
Fix static checker complaint that stream is not checked in
squashfs_decompressor_destroy().

Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
Reviewed-by: Minchan Kim <minchan@kernel.org>
2015-03-04 11:22:58 -08:00
Phillip Lougher
a7b16c939e Squashfs: Directly decompress into the page cache for file data
This introduces an implementation of squashfs_readpage_block()
that directly decompresses into the page cache.

This uses the previously added page handler abstraction to push
down the necessary kmap_atomic/kunmap_atomic operations on the
page cache buffers into the decompressors.  This enables
direct copying into the page cache without using the slow
kmap/kunmap calls.

The code detects when multiple threads are racing in
squashfs_readpage() to decompress the same block, and avoids
this regression by falling back to using an intermediate
buffer.

This patch enhances the performance of Squashfs significantly
when multiple processes are accessing the filesystem simultaneously
because it not only reduces memcopying, but it more importantly
eliminates the lock contention on the intermediate buffer.

Using single-thread decompression.

        dd if=file1 of=/dev/null bs=4096 &
        dd if=file2 of=/dev/null bs=4096 &
        dd if=file3 of=/dev/null bs=4096 &
        dd if=file4 of=/dev/null bs=4096

Before:

629145600 bytes (629 MB) copied, 45.8046 s, 13.7 MB/s

After:

629145600 bytes (629 MB) copied, 9.29414 s, 67.7 MB/s

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
Reviewed-by: Minchan Kim <minchan@kernel.org>
2015-03-04 11:22:54 -08:00
Phillip Lougher
4ff4dc4d15 Squashfs: Restructure squashfs_readpage()
Restructure squashfs_readpage() splitting it into separate
functions for datablocks, fragments and sparse blocks.

Move the memcpying (from squashfs cache entry) implementation of
squashfs_readpage_block into file_cache.c

This allows different implementations to be supported.

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
Reviewed-by: Minchan Kim <minchan@kernel.org>
2015-03-04 11:22:27 -08:00
Phillip Lougher
dad13d6854 Squashfs: Generalise paging handling in the decompressors
Further generalise the decompressors by adding a page handler
abstraction.  This adds helpers to allow the decompressors
to access and process the output buffers in an implementation
independant manner.

This allows different types of output buffer to be passed
to the decompressors, with the implementation specific
aspects handled at decompression time, but without the
knowledge being held in the decompressor wrapper code.

This will allow the decompressors to handle Squashfs
cache buffers, and page cache pages.

This patch adds the abstraction and an implementation for
the caches.

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
Reviewed-by: Minchan Kim <minchan@kernel.org>
2015-03-04 11:22:22 -08:00
Phillip Lougher
887bd836dc Squashfs: add multi-threaded decompression using percpu variable
Add a multi-threaded decompression implementation which uses
percpu variables.

Using percpu variables has advantages and disadvantages over
implementations which do not use percpu variables.

Advantages:
  * the nature of percpu variables ensures decompression is
    load-balanced across the multiple cores.
  * simplicity.

Disadvantages: it limits decompression to one thread per core.

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
2015-03-04 11:22:13 -08:00
Minchan Kim
c0f8b08ad9 squashfs: Enhance parallel I/O
Now squashfs have used for only one stream buffer for decompression
so it hurts parallel read performance so this patch supports
multiple decompressor to enhance performance parallel I/O.

Four 1G file dd read on KVM machine which has 2 CPU and 4G memory.

dd if=test/test1.dat of=/dev/null &
dd if=test/test2.dat of=/dev/null &
dd if=test/test3.dat of=/dev/null &
dd if=test/test4.dat of=/dev/null &

old : 1m39s -> new : 9s

* From v1
  * Change comp_strm with decomp_strm - Phillip
  * Change/add comments - Phillip

Signed-off-by: Minchan Kim <minchan@kernel.org>
Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
2015-03-04 11:22:09 -08:00
Phillip Lougher
06e4ac0b0a Squashfs: Refactor decompressor interface and code
The decompressor interface and code was written from
the point of view of single-threaded operation.  In doing
so it mixed a lot of single-threaded implementation specific
aspects into the decompressor code and elsewhere which makes it
difficult to seamlessly support multiple different decompressor
implementations.

This patch does the following:

1.  It removes compressor_options parsing from the decompressor
    init() function.  This allows the decompressor init() function
    to be dynamically called to instantiate multiple decompressors,
    without the compressor options needing to be read and parsed each
    time.

2.  It moves threading and all sleeping operations out of the
    decompressors.  In doing so, it makes the decompressors
    non-blocking wrappers which only deal with interfacing with
    the decompressor implementation.

3. It splits decompressor.[ch] into decompressor generic functions
   in decompressor.[ch], and moves the single threaded
   decompressor implementation into decompressor_single.c.

The result of this patch is Squashfs should now be able to
support multiple decompressors by adding new decompressor_xxx.c
files with specialised implementations of the functions in
decompressor_single.c

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
Reviewed-by: Minchan Kim <minchan@kernel.org>
2015-03-04 11:22:02 -08:00
Phillip Lougher
bb5919cc20 Squashfs: add corruption check for type in squashfs_readdir()
We read the type field from disk.  This value should be sanity
checked for correctness to avoid an out of bounds access when
reading the squashfs_filetype_table array.

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
2015-03-04 11:21:48 -08:00
Phillip Lougher
4fed250b48 Squashfs: add corruption check in get_dir_index_using_offset()
We read the size (of the name) field from disk.  This value should
be sanity checked for correctness to avoid blindly reading
huge amounts of unnecessary data from disk on corruption.

Note, here we're not actually reading the name into a buffer, but
skipping it, and so corruption doesn't cause buffer overflow, merely
lots of unnecessary amounts of data to be read.

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
2015-03-04 11:21:43 -08:00
Phillip Lougher
4ff8d316da Squashfs: fix corruption checks in squashfs_readdir()
The dir_count and size fields when read from disk are sanity
checked for correctness.  However, the sanity checks only check the
values are not greater than expected.  As dir_count and size were
incorrectly defined as signed ints, this can lead to corrupted values
appearing as negative which are not trapped.

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
2015-03-04 11:21:38 -08:00
Phillip Lougher
98bdbcf71a Squashfs: fix corruption checks in squashfs_lookup()
The dir_count and size fields when read from disk are sanity
checked for correctness.  However, the sanity checks only check the
values are not greater than expected.  As dir_count and size were
incorrectly defined as signed ints, this can lead to corrupted values
appearing as negative which are not trapped.

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
2015-03-04 11:20:48 -08:00
Phillip Lougher
edd2496ccf Squashfs: fix corruption check in get_dir_index_using_name()
Patch "Squashfs: sanity check information from disk" from
Dan Carpenter adds a missing check for corruption in the
"size" field while reading the directory index from disk.

It, however, sets err to -EINVAL, this value is not used later, and
so setting it is completely redundant.  So remove it.

Errors in reading the index are deliberately non-fatal.  If we
get an error in reading the index we just return the part of the
index we have managed to read - the index isn't essential,
just quicker.

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
2015-03-04 11:20:43 -08:00
Manish Sharma
706a7943a8 Squashfs: Optimized uncompressed buffer loop
Merged the two for loops. We might get a little gain by overlapping
wait_on_bh and the memcpy operations.

Signed-off-by: Manish Sharma <manishrma@gmail.com>
Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
2015-03-04 11:20:31 -08:00
Dan Carpenter
639839d89f Squashfs: sanity check information from disk
We read the size of the name from the disk, but a larger name than
expected would cause memory corruption.

Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
2015-03-04 11:20:23 -08:00
Al Viro
b2497fc305 [readdir] constify ->actor
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2015-03-04 11:19:34 -08:00
Al Viro
83fd542759 [readdir] introduce ->iterate(), ctx->pos, dir_emit()
New method - ->iterate(file, ctx).  That's the replacement for ->readdir();
it takes callback from ctx->actor, uses ctx->pos instead of file->f_pos and
calls dir_emit(ctx, ...) instead of filldir(data, ...).  It does *not*
update file->f_pos (or look at it, for that matter); iterate_dir() does the
update.

Note that dir_emit() takes the offset from ctx->pos (and eventually
filldir_t will lose that argument).

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2015-03-04 11:15:29 -08:00
Al Viro
c301a0e047 [readdir] introduce iterate_dir() and dir_context
iterate_dir(): new helper, replacing vfs_readdir().

struct dir_context: contains the readdir callback (and will get more stuff
in it), embedded into whatever data that callback wants to deal with;
eventually, we'll be passing it to ->readdir() replacement instead of
(data,filldir) pair.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2015-03-04 11:15:13 -08:00
Al Viro
78bae7615d [readdir] convert squashfs
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2015-03-04 11:14:14 -08:00
Cong Wang
9c086b4cf2 ipv4, fib: pass LOOPBACK_IFINDEX instead of 0 to flowi4_iif
As suggested by Julian:

	Simply, flowi4_iif must not contain 0, it does not
	look logical to ignore all ip rules with specified iif.

because in fib_rule_match() we do:

        if (rule->iifindex && (rule->iifindex != fl->flowi_iif))
                goto out;

flowi4_iif should be LOOPBACK_IFINDEX by default.

We need to move LOOPBACK_IFINDEX to include/net/flow.h:

1) It is mostly used by flowi_iif

2) Fix the following compile error if we use it in flow.h
by the patches latter:

In file included from include/linux/netfilter.h:277:0,
                 from include/net/netns/netfilter.h:5,
                 from include/net/net_namespace.h:21,
                 from include/linux/netdevice.h:43,
                 from include/linux/icmpv6.h:12,
                 from include/linux/ipv6.h:61,
                 from include/net/ipv6.h:16,
                 from include/linux/sunrpc/clnt.h:27,
                 from include/linux/nfs_fs.h:30,
                 from init/do_mounts.c:32:
include/net/flow.h: In function ‘flowi4_init_output’:
include/net/flow.h:84:32: error: ‘LOOPBACK_IFINDEX’ undeclared (first use in this function)

[Backport of net-next 6a662719c9]

Change-Id: Ib7a0a08d78c03800488afa1b2c170cb70e34cfd9
Cc: Eric Biederman <ebiederm@xmission.com>
Cc: Julian Anastasov <ja@ssi.bg>
Cc: David S. Miller <davem@davemloft.net>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
Signed-off-by: Cong Wang <cwang@twopensource.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Lorenzo Colitti <lorenzo@google.com>
2015-03-03 12:47:11 +09:00
Richard Weinberger
0f708442d1 um: remove dead code
"me" is not used.

[cherry-pick of upstream 9e82d45053]

Change-Id: Ifc3550184931dddf8feebd6c3137e60b97f6a0f1
Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: Lorenzo Colitti <lorenzo@google.com>
2015-03-03 12:47:11 +09:00