github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-06-08 06:25:52 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

Merge branch 'linux-linaro-lsk-v4.4-android' of git://git.linaro.org/kernel/linux-linaro-stable.git

Browse Source 
* linux-linaro-lsk-v4.4-android: (521 commits)
  Linux 4.4.66
  ftrace/x86: Fix triple fault with graph tracing and suspend-to-ram
  ARCv2: save r30 on kernel entry as gcc uses it for code-gen
  nfsd: check for oversized NFSv2/v3 arguments
  Input: i8042 - add Clevo P650RS to the i8042 reset list
  p9_client_readdir() fix
  MIPS: Avoid BUG warning in arch_check_elf
  MIPS: KGDB: Use kernel context for sleeping threads
  ALSA: seq: Don't break snd_use_lock_sync() loop by timeout
  ALSA: firewire-lib: fix inappropriate assignment between signed/unsigned type
  ipv6: check raw payload size correctly in ioctl
  ipv6: check skb->protocol before lookup for nexthop
  macvlan: Fix device ref leak when purging bc_queue
  ip6mr: fix notification device destruction
  netpoll: Check for skb->queue_mapping
  net: ipv6: RTF_PCPU should not be settable from userspace
  dp83640: don't recieve time stamps twice
  tcp: clear saved_syn in tcp_disconnect()
  sctp: listen on the sock only when it's state is listening or closed
  net: ipv4: fix multipath RTM_GETROUTE behavior when iif is given
  ...

Conflicts:
	drivers/usb/dwc3/gadget.c
	include/linux/usb/quirks.h

Change-Id: I490f766b9a530b10da3107e20709538e4536a99d
This commit is contained in:
Huang, Tao 2017-05-06 14:23:00 +08:00
commit 986d4e4637
498 changed files with 10176 additions and 5397 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
Documentation/devicetree/bindings/clock/sunxi.txt
View File

@ -18,6 +18,7 @@ Required properties:
"allwinner,sun4i-a10-cpu-clk" - for the CPU multiplexer clock
"allwinner,sun4i-a10-axi-clk" - for the AXI clock
"allwinner,sun8i-a23-axi-clk" - for the AXI clock on A23
"allwinner,sun4i-a10-gates-clk" - for generic gates on all compatible SoCs
"allwinner,sun4i-a10-axi-gates-clk" - for the AXI gates
"allwinner,sun4i-a10-ahb-clk" - for the AHB clock
"allwinner,sun5i-a13-ahb-clk" - for the AHB clock on A13
@ -43,6 +44,7 @@ Required properties:
"allwinner,sun6i-a31-apb0-gates-clk" - for the APB0 gates on A31
"allwinner,sun7i-a20-apb0-gates-clk" - for the APB0 gates on A20
"allwinner,sun8i-a23-apb0-gates-clk" - for the APB0 gates on A23
"allwinner,sun8i-h3-apb0-gates-clk" - for the APB0 gates on H3
"allwinner,sun9i-a80-apb0-gates-clk" - for the APB0 gates on A80
"allwinner,sun4i-a10-apb1-clk" - for the APB1 clock
"allwinner,sun9i-a80-apb1-clk" - for the APB1 bus clock on A80

13
Documentation/networking/ip-sysctl.txt
View File

@ -1413,11 +1413,20 @@ accept_ra_pinfo - BOOLEAN
Functional default: enabled if accept_ra is enabled.
disabled if accept_ra is disabled.
accept_ra_rt_info_min_plen - INTEGER
Minimum prefix length of Route Information in RA.
Route Information w/ prefix smaller than this variable shall
be ignored.
Functional default: 0 if accept_ra_rtr_pref is enabled.
-1 if accept_ra_rtr_pref is disabled.
accept_ra_rt_info_max_plen - INTEGER
Maximum prefix length of Route Information in RA.
Route Information w/ prefix larger than or equal to this
variable shall be ignored.
Route Information w/ prefix larger than this variable shall
be ignored.
Functional default: 0 if accept_ra_rtr_pref is enabled.
-1 if accept_ra_rtr_pref is disabled.

332
Documentation/networking/netlink_mmap.txt
View File

@ -1,332 +0,0 @@
This file documents how to use memory mapped I/O with netlink.
Author: Patrick McHardy <kaber@trash.net>
Overview
--------
Memory mapped netlink I/O can be used to increase throughput and decrease
overhead of unicast receive and transmit operations. Some netlink subsystems
require high throughput, these are mainly the netfilter subsystems
nfnetlink_queue and nfnetlink_log, but it can also help speed up large
dump operations of f.i. the routing database.
Memory mapped netlink I/O used two circular ring buffers for RX and TX which
are mapped into the processes address space.
The RX ring is used by the kernel to directly construct netlink messages into
user-space memory without copying them as done with regular socket I/O,
additionally as long as the ring contains messages no recvmsg() or poll()
syscalls have to be issued by user-space to get more message.
The TX ring is used to process messages directly from user-space memory, the
kernel processes all messages contained in the ring using a single sendmsg()
call.
Usage overview
--------------
In order to use memory mapped netlink I/O, user-space needs three main changes:
- ring setup
- conversion of the RX path to get messages from the ring instead of recvmsg()
- conversion of the TX path to construct messages into the ring
Ring setup is done using setsockopt() to provide the ring parameters to the
kernel, then a call to mmap() to map the ring into the processes address space:
- setsockopt(fd, SOL_NETLINK, NETLINK_RX_RING, &params, sizeof(params));
- setsockopt(fd, SOL_NETLINK, NETLINK_TX_RING, &params, sizeof(params));
- ring = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0)
Usage of either ring is optional, but even if only the RX ring is used the
mapping still needs to be writable in order to update the frame status after
processing.
Conversion of the reception path involves calling poll() on the file
descriptor, once the socket is readable the frames from the ring are
processed in order until no more messages are available, as indicated by
a status word in the frame header.
On kernel side, in order to make use of memory mapped I/O on receive, the
originating netlink subsystem needs to support memory mapped I/O, otherwise
it will use an allocated socket buffer as usual and the contents will be
copied to the ring on transmission, nullifying most of the performance gains.
Dumps of kernel databases automatically support memory mapped I/O.
Conversion of the transmit path involves changing message construction to
use memory from the TX ring instead of (usually) a buffer declared on the
stack and setting up the frame header appropriately. Optionally poll() can
be used to wait for free frames in the TX ring.
Structured and definitions for using memory mapped I/O are contained in
<linux/netlink.h>.
RX and TX rings
----------------
Each ring contains a number of continuous memory blocks, containing frames of
fixed size dependent on the parameters used for ring setup.
Ring: [ block 0 ]
[ frame 0 ]
[ frame 1 ]
[ block 1 ]
[ frame 2 ]
[ frame 3 ]
...
[ block n ]
[ frame 2 * n ]
[ frame 2 * n + 1 ]
The blocks are only visible to the kernel, from the point of view of user-space
the ring just contains the frames in a continuous memory zone.
The ring parameters used for setting up the ring are defined as follows:
struct nl_mmap_req {
unsigned int nm_block_size;
unsigned int nm_block_nr;
unsigned int nm_frame_size;
unsigned int nm_frame_nr;
};
Frames are grouped into blocks, where each block is a continuous region of memory
and holds nm_block_size / nm_frame_size frames. The total number of frames in
the ring is nm_frame_nr. The following invariants hold:
- frames_per_block = nm_block_size / nm_frame_size
- nm_frame_nr = frames_per_block * nm_block_nr
Some parameters are constrained, specifically:
- nm_block_size must be a multiple of the architectures memory page size.
The getpagesize() function can be used to get the page size.
- nm_frame_size must be equal or larger to NL_MMAP_HDRLEN, IOW a frame must be
able to hold at least the frame header
- nm_frame_size must be smaller or equal to nm_block_size
- nm_frame_size must be a multiple of NL_MMAP_MSG_ALIGNMENT
- nm_frame_nr must equal the actual number of frames as specified above.
When the kernel can't allocate physically continuous memory for a ring block,
it will fall back to use physically discontinuous memory. This might affect
performance negatively, in order to avoid this the nm_frame_size parameter
should be chosen to be as small as possible for the required frame size and
the number of blocks should be increased instead.
Ring frames
------------
Each frames contain a frame header, consisting of a synchronization word and some
meta-data, and the message itself.
Frame: [ header message ]
The frame header is defined as follows:
struct nl_mmap_hdr {
unsigned int nm_status;
unsigned int nm_len;
__u32 nm_group;
/* credentials */
__u32 nm_pid;
__u32 nm_uid;
__u32 nm_gid;
};
- nm_status is used for synchronizing processing between the kernel and user-
space and specifies ownership of the frame as well as the operation to perform
- nm_len contains the length of the message contained in the data area
- nm_group specified the destination multicast group of message
- nm_pid, nm_uid and nm_gid contain the netlink pid, UID and GID of the sending
process. These values correspond to the data available using SOCK_PASSCRED in
the SCM_CREDENTIALS cmsg.
The possible values in the status word are:
- NL_MMAP_STATUS_UNUSED:
RX ring: frame belongs to the kernel and contains no message
for user-space. Approriate action is to invoke poll()
to wait for new messages.
TX ring: frame belongs to user-space and can be used for
message construction.
- NL_MMAP_STATUS_RESERVED:
RX ring only: frame is currently used by the kernel for message
construction and contains no valid message yet.
Appropriate action is to invoke poll() to wait for
new messages.
- NL_MMAP_STATUS_VALID:
RX ring: frame contains a valid message. Approriate action is
to process the message and release the frame back to
the kernel by setting the status to
NL_MMAP_STATUS_UNUSED or queue the frame by setting the
status to NL_MMAP_STATUS_SKIP.
TX ring: the frame contains a valid message from user-space to
be processed by the kernel. After completing processing
the kernel will release the frame back to user-space by
setting the status to NL_MMAP_STATUS_UNUSED.
- NL_MMAP_STATUS_COPY:
RX ring only: a message is ready to be processed but could not be
stored in the ring, either because it exceeded the
frame size or because the originating subsystem does
not support memory mapped I/O. Appropriate action is
to invoke recvmsg() to receive the message and release
the frame back to the kernel by setting the status to
NL_MMAP_STATUS_UNUSED.
- NL_MMAP_STATUS_SKIP:
RX ring only: user-space queued the message for later processing, but
processed some messages following it in the ring. The
kernel should skip this frame when looking for unused
frames.
The data area of a frame begins at a offset of NL_MMAP_HDRLEN relative to the
frame header.
TX limitations
--------------
As of Jan 2015 the message is always copied from the ring frame to an
allocated buffer due to unresolved security concerns.
See commit 4682a0358639b29cf ("netlink: Always copy on mmap TX.").
Example
-------
Ring setup:
unsigned int block_size = 16 * getpagesize();
struct nl_mmap_req req = {
.nm_block_size = block_size,
.nm_block_nr = 64,
.nm_frame_size = 16384,
.nm_frame_nr = 64 * block_size / 16384,
};
unsigned int ring_size;
void *rx_ring, *tx_ring;
/* Configure ring parameters */
if (setsockopt(fd, SOL_NETLINK, NETLINK_RX_RING, &req, sizeof(req)) < 0)
exit(1);
if (setsockopt(fd, SOL_NETLINK, NETLINK_TX_RING, &req, sizeof(req)) < 0)
exit(1)
/* Calculate size of each individual ring */
ring_size = req.nm_block_nr * req.nm_block_size;
/* Map RX/TX rings. The TX ring is located after the RX ring */
rx_ring = mmap(NULL, 2 * ring_size, PROT_READ | PROT_WRITE,
MAP_SHARED, fd, 0);
if ((long)rx_ring == -1L)
exit(1);
tx_ring = rx_ring + ring_size:
Message reception:
This example assumes some ring parameters of the ring setup are available.
unsigned int frame_offset = 0;
struct nl_mmap_hdr *hdr;
struct nlmsghdr *nlh;
unsigned char buf[16384];
ssize_t len;
while (1) {
struct pollfd pfds[1];
pfds[0].fd = fd;
pfds[0].events = POLLIN | POLLERR;
pfds[0].revents = 0;
if (poll(pfds, 1, -1) < 0 && errno != -EINTR)
exit(1);
/* Check for errors. Error handling omitted */
if (pfds[0].revents & POLLERR)
<handle error>
/* If no new messages, poll again */
if (!(pfds[0].revents & POLLIN))
continue;
/* Process all frames */
while (1) {
/* Get next frame header */
hdr = rx_ring + frame_offset;
if (hdr->nm_status == NL_MMAP_STATUS_VALID) {
/* Regular memory mapped frame */
nlh = (void *)hdr + NL_MMAP_HDRLEN;
len = hdr->nm_len;
/* Release empty message immediately. May happen
* on error during message construction.
*/
if (len == 0)
goto release;
} else if (hdr->nm_status == NL_MMAP_STATUS_COPY) {
/* Frame queued to socket receive queue */
len = recv(fd, buf, sizeof(buf), MSG_DONTWAIT);
if (len <= 0)
break;
nlh = buf;
} else
/* No more messages to process, continue polling */
break;
process_msg(nlh);
release:
/* Release frame back to the kernel */
hdr->nm_status = NL_MMAP_STATUS_UNUSED;
/* Advance frame offset to next frame */
frame_offset = (frame_offset + frame_size) % ring_size;
}
}
Message transmission:
This example assumes some ring parameters of the ring setup are available.
A single message is constructed and transmitted, to send multiple messages
at once they would be constructed in consecutive frames before a final call
to sendto().
unsigned int frame_offset = 0;
struct nl_mmap_hdr *hdr;
struct nlmsghdr *nlh;
struct sockaddr_nl addr = {
.nl_family = AF_NETLINK,
};
hdr = tx_ring + frame_offset;
if (hdr->nm_status != NL_MMAP_STATUS_UNUSED)
/* No frame available. Use poll() to avoid. */
exit(1);
nlh = (void *)hdr + NL_MMAP_HDRLEN;
/* Build message */
build_message(nlh);
/* Fill frame header: length and status need to be set */
hdr->nm_len = nlh->nlmsg_len;
hdr->nm_status = NL_MMAP_STATUS_VALID;
if (sendto(fd, NULL, 0, 0, &addr, sizeof(addr)) < 0)
exit(1);
/* Advance frame offset to next frame */
frame_offset = (frame_offset + frame_size) % ring_size;

7
Documentation/sysctl/fs.txt
View File

@ -265,6 +265,13 @@ aio-nr can grow to.
==============================================================
mount-max:
This denotes the maximum number of mounts that may exist
in a mount namespace.
==============================================================
2. /proc/sys/fs/binfmt_misc
----------------------------------------------------------

10
Makefile
View File

@ -1,6 +1,6 @@
VERSION = 4
PATCHLEVEL = 4
SUBLEVEL = 55
SUBLEVEL = 66
EXTRAVERSION =
NAME = Blurry Fish Butt
@ -146,7 +146,7 @@ PHONY += $(MAKECMDGOALS) sub-make
$(filter-out _all sub-make $(CURDIR)/Makefile, $(MAKECMDGOALS)) _all: sub-make
@:
sub-make: FORCE
sub-make:
$(Q)$(MAKE) -C $(KBUILD_OUTPUT) KBUILD_SRC=$(CURDIR) \
-f $(CURDIR)/Makefile $(filter-out _all sub-make,$(MAKECMDGOALS))
@ -1022,7 +1022,7 @@ prepare1: prepare2 $(version_h) include/generated/utsrelease.h \
archprepare: archheaders archscripts prepare1 scripts_basic
prepare0: archprepare FORCE
prepare0: archprepare
$(Q)$(MAKE) $(build)=.
# All the preparing..
@ -1067,7 +1067,7 @@ INSTALL_FW_PATH=$(INSTALL_MOD_PATH)/lib/firmware
export INSTALL_FW_PATH
PHONY += firmware_install
firmware_install: FORCE
firmware_install:
@mkdir -p $(objtree)/firmware
$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.fwinst obj=firmware __fw_install
@ -1087,7 +1087,7 @@ PHONY += archscripts
archscripts:
PHONY += __headers
__headers: $(version_h) scripts_basic asm-generic archheaders archscripts FORCE
__headers: $(version_h) scripts_basic asm-generic archheaders archscripts
$(Q)$(MAKE) $(build)=scripts build_unifdef
PHONY += headers_install_all

29
android/configs/android-base.cfg
View File

@ -1,17 +1,19 @@
# KEEP ALPHABETICALLY SORTED
# CONFIG_DEVKMEM is not set
# CONFIG_DEVMEM is not set
# CONFIG_FHANDLE is not set
# CONFIG_INET_LRO is not set
# CONFIG_MODULES is not set
# CONFIG_OABI_COMPAT is not set
# CONFIG_SYSVIPC is not set
# CONFIG_USELIB is not set
CONFIG_ANDROID=y
CONFIG_ANDROID_BINDER_DEVICES=binder,hwbinder,vndbinder
CONFIG_ANDROID_BINDER_IPC=y
CONFIG_ANDROID_LOW_MEMORY_KILLER=y
CONFIG_ARMV8_DEPRECATED=y
CONFIG_ASHMEM=y
CONFIG_AUDIT=y
CONFIG_BLK_DEV_DM=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_CGROUPS=y
CONFIG_CGROUP_CPUACCT=y
@ -19,14 +21,15 @@ CONFIG_CGROUP_DEBUG=y
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_SCHED=y
CONFIG_CP15_BARRIER_EMULATION=y
CONFIG_DM_CRYPT=y
CONFIG_DM_VERITY=y
CONFIG_DM_VERITY_FEC=y
CONFIG_DEFAULT_SECURITY_SELINUX=y
CONFIG_EMBEDDED=y
CONFIG_FB=y
CONFIG_HARDENED_USERCOPY=y
CONFIG_HIGH_RES_TIMERS=y
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_INET6_AH=y
CONFIG_INET6_DIAG_DESTROY=y
CONFIG_INET6_ESP=y
CONFIG_INET6_IPCOMP=y
CONFIG_INET=y
@ -42,7 +45,6 @@ CONFIG_IPV6=y
CONFIG_IPV6_MIP6=y
CONFIG_IPV6_MULTIPLE_TABLES=y
CONFIG_IPV6_OPTIMISTIC_DAD=y
CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_IPV6_ROUTE_INFO=y
CONFIG_IP_ADVANCED_ROUTER=y
@ -64,6 +66,9 @@ CONFIG_IP_NF_TARGET_MASQUERADE=y
CONFIG_IP_NF_TARGET_NETMAP=y
CONFIG_IP_NF_TARGET_REDIRECT=y
CONFIG_IP_NF_TARGET_REJECT=y
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODVERSIONS=y
CONFIG_NET=y
CONFIG_NETDEVICES=y
CONFIG_NETFILTER=y
@ -141,9 +146,9 @@ CONFIG_PREEMPT=y
CONFIG_PROFILING=y
CONFIG_QFMT_V2=y
CONFIG_QUOTA=y
CONFIG_QUOTACTL=y
CONFIG_QUOTA_NETLINK_INTERFACE=y
CONFIG_QUOTA_TREE=y
CONFIG_QUOTACTL=y
CONFIG_RANDOMIZE_BASE=y
CONFIG_RTC_CLASS=y
CONFIG_RT_GROUP_SCHED=y
@ -157,16 +162,16 @@ CONFIG_STAGING=y
CONFIG_SWP_EMULATION=y
CONFIG_SYNC=y
CONFIG_TUN=y
CONFIG_UID_CPUTIME=y
CONFIG_UID_SYS_STATS=y
CONFIG_UNIX=y
CONFIG_USB_GADGET=y
CONFIG_USB_CONFIGFS=y
CONFIG_USB_CONFIGFS_F_FS=y
CONFIG_USB_CONFIGFS_F_MTP=y
CONFIG_USB_CONFIGFS_F_PTP=y
CONFIG_USB_CONFIGFS_F_ACC=y
CONFIG_USB_CONFIGFS_F_AUDIO_SRC=y
CONFIG_USB_CONFIGFS_UEVENT=y
CONFIG_USB_CONFIGFS_F_FS=y
CONFIG_USB_CONFIGFS_F_MIDI=y
CONFIG_USB_CONFIGFS_F_MTP=y
CONFIG_USB_CONFIGFS_F_PTP=y
CONFIG_USB_CONFIGFS_UEVENT=y
CONFIG_USB_GADGET=y
CONFIG_USB_OTG_WAKELOCK=y
CONFIG_XFRM_USER=y

9
android/configs/android-recommended.cfg
View File

@ -7,16 +7,21 @@
# CONFIG_PM_WAKELOCKS_GC is not set
# CONFIG_VT is not set
CONFIG_ANDROID_TIMED_GPIO=y
CONFIG_ARM_KERNMEM_PERMS=y
CONFIG_ARM64_SW_TTBR0_PAN=y
CONFIG_ARM_KERNMEM_PERMS=y
CONFIG_BACKLIGHT_LCD_SUPPORT=y
CONFIG_BLK_DEV_DM=y
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_SIZE=8192
CONFIG_CC_STACKPROTECTOR_STRONG=y
CONFIG_COMPACTION=y
CONFIG_CPU_SW_DOMAIN_PAN=y
CONFIG_DEBUG_RODATA=y
CONFIG_DM_CRYPT=y
CONFIG_DM_UEVENT=y
CONFIG_DM_VERITY=y
CONFIG_DM_VERITY_FEC=y
CONFIG_DRAGONRISE_FF=y
CONFIG_ENABLE_DEFAULT_TRACERS=y
CONFIG_EXT4_FS=y
@ -92,6 +97,7 @@ CONFIG_LOGIRUMBLEPAD2_FF=y
CONFIG_LOGITECH_FF=y
CONFIG_MD=y
CONFIG_MEDIA_SUPPORT=y
CONFIG_MEMORY_STATE_TIME=y
CONFIG_MSDOS_FS=y
CONFIG_PANIC_TIMEOUT=5
CONFIG_PANTHERLORD_FF=y
@ -121,7 +127,6 @@ CONFIG_TIMER_STATS=y
CONFIG_TMPFS=y
CONFIG_TMPFS_POSIX_ACL=y
CONFIG_UHID=y
CONFIG_MEMORY_STATE_TIME=y
CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
CONFIG_USB_EHCI_HCD=y
CONFIG_USB_HIDDEV=y

2
arch/arc/include/asm/entry-arcv2.h
View File

@ -16,6 +16,7 @@
;
; Now manually save: r12, sp, fp, gp, r25
PUSH r30
PUSH r12
; Saving pt_regs->sp correctly requires some extra work due to the way
@ -72,6 +73,7 @@
POPAX AUX_USER_SP
1:
POP r12
POP r30
.endm

2
arch/arc/include/asm/ptrace.h
View File

@ -84,7 +84,7 @@ struct pt_regs {
unsigned long fp;
unsigned long sp; /* user/kernel sp depending on where we came from */
unsigned long r12;
unsigned long r12, r30;
/*------- Below list auto saved by h/w -----------*/
unsigned long r0, r1, r2, r3, r4, r5, r6, r7, r8, r9, r10, r11;

35
arch/arm/boot/dts/sama5d2.dtsi
View File

@ -856,6 +856,13 @@ uart0: serial@f801c000 {
compatible = "atmel,at91sam9260-usart";
reg = <0xf801c000 0x100>;
interrupts = <24 IRQ_TYPE_LEVEL_HIGH 7>;
dmas = <&dma0
(AT91_XDMAC_DT_MEM_IF(0) | AT91_XDMAC_DT_PER_IF(1) |
AT91_XDMAC_DT_PERID(35))>,
<&dma0
(AT91_XDMAC_DT_MEM_IF(0) | AT91_XDMAC_DT_PER_IF(1) |
AT91_XDMAC_DT_PERID(36))>;
dma-names = "tx", "rx";
clocks = <&uart0_clk>;
clock-names = "usart";
status = "disabled";
@ -865,6 +872,13 @@ uart1: serial@f8020000 {
compatible = "atmel,at91sam9260-usart";
reg = <0xf8020000 0x100>;
interrupts = <25 IRQ_TYPE_LEVEL_HIGH 7>;
dmas = <&dma0
(AT91_XDMAC_DT_MEM_IF(0) | AT91_XDMAC_DT_PER_IF(1) |
AT91_XDMAC_DT_PERID(37))>,
<&dma0
(AT91_XDMAC_DT_MEM_IF(0) | AT91_XDMAC_DT_PER_IF(1) |
AT91_XDMAC_DT_PERID(38))>;
dma-names = "tx", "rx";
clocks = <&uart1_clk>;
clock-names = "usart";
status = "disabled";
@ -874,6 +888,13 @@ uart2: serial@f8024000 {
compatible = "atmel,at91sam9260-usart";
reg = <0xf8024000 0x100>;
interrupts = <26 IRQ_TYPE_LEVEL_HIGH 7>;
dmas = <&dma0
(AT91_XDMAC_DT_MEM_IF(0) | AT91_XDMAC_DT_PER_IF(1) |
AT91_XDMAC_DT_PERID(39))>,
<&dma0
(AT91_XDMAC_DT_MEM_IF(0) | AT91_XDMAC_DT_PER_IF(1) |
AT91_XDMAC_DT_PERID(40))>;
dma-names = "tx", "rx";
clocks = <&uart2_clk>;
clock-names = "usart";
status = "disabled";
@ -985,6 +1006,13 @@ uart3: serial@fc008000 {
compatible = "atmel,at91sam9260-usart";
reg = <0xfc008000 0x100>;
interrupts = <27 IRQ_TYPE_LEVEL_HIGH 7>;
dmas = <&dma0
(AT91_XDMAC_DT_MEM_IF(0) | AT91_XDMAC_DT_PER_IF(1) |
AT91_XDMAC_DT_PERID(41))>,
<&dma0
(AT91_XDMAC_DT_MEM_IF(0) | AT91_XDMAC_DT_PER_IF(1) |
AT91_XDMAC_DT_PERID(42))>;
dma-names = "tx", "rx";
clocks = <&uart3_clk>;
clock-names = "usart";
status = "disabled";
@ -993,6 +1021,13 @@ uart3: serial@fc008000 {
uart4: serial@fc00c000 {
compatible = "atmel,at91sam9260-usart";
reg = <0xfc00c000 0x100>;
dmas = <&dma0
(AT91_XDMAC_DT_MEM_IF(0) | AT91_XDMAC_DT_PER_IF(1) |
AT91_XDMAC_DT_PERID(43))>,
<&dma0
(AT91_XDMAC_DT_MEM_IF(0) | AT91_XDMAC_DT_PER_IF(1) |
AT91_XDMAC_DT_PERID(44))>;
dma-names = "tx", "rx";
interrupts = <28 IRQ_TYPE_LEVEL_HIGH 7>;
clocks = <&uart4_clk>;
clock-names = "usart";

4
arch/arm/crypto/aes-ce-glue.c
View File

@ -369,7 +369,7 @@ static struct crypto_alg aes_algs[] = { {
.cra_blkcipher = {
.min_keysize = AES_MIN_KEY_SIZE,
.max_keysize = AES_MAX_KEY_SIZE,
.ivsize = AES_BLOCK_SIZE,
.ivsize = 0,
.setkey = ce_aes_setkey,
.encrypt = ecb_encrypt,
.decrypt = ecb_decrypt,
@ -446,7 +446,7 @@ static struct crypto_alg aes_algs[] = { {
.cra_ablkcipher = {
.min_keysize = AES_MIN_KEY_SIZE,
.max_keysize = AES_MAX_KEY_SIZE,
.ivsize = AES_BLOCK_SIZE,
.ivsize = 0,
.setkey = ablk_set_key,
.encrypt = ablk_encrypt,
.decrypt = ablk_decrypt,

13
arch/arm/kernel/vdso.c
View File

@ -17,6 +17,7 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include <linux/cache.h>
#include <linux/elf.h>
#include <linux/err.h>
#include <linux/kernel.h>
@ -39,7 +40,7 @@
static struct page **vdso_text_pagelist;
/* Total number of pages needed for the data and text portions of the VDSO. */
unsigned int vdso_total_pages __read_mostly;
unsigned int vdso_total_pages __ro_after_init;
/*
* The VDSO data page.
@ -47,13 +48,13 @@ unsigned int vdso_total_pages __read_mostly;
static union vdso_data_store vdso_data_store __page_aligned_data;
static struct vdso_data *vdso_data = &vdso_data_store.data;
static struct page *vdso_data_page;
static struct vm_special_mapping vdso_data_mapping = {
static struct page *vdso_data_page __ro_after_init;
static const struct vm_special_mapping vdso_data_mapping = {
.name = "[vvar]",
.pages = &vdso_data_page,
};
static struct vm_special_mapping vdso_text_mapping = {
static struct vm_special_mapping vdso_text_mapping __ro_after_init = {
.name = "[vdso]",
};
@ -67,7 +68,7 @@ struct elfinfo {
/* Cached result of boot-time check for whether the arch timer exists,
* and if so, whether the virtual counter is useable.
*/
static bool cntvct_ok __read_mostly;
static bool cntvct_ok __ro_after_init;
static bool __init cntvct_functional(void)
{
@ -224,7 +225,7 @@ static int install_vvar(struct mm_struct *mm, unsigned long addr)
VM_READ | VM_MAYREAD,
&vdso_data_mapping);
return IS_ERR(vma) ? PTR_ERR(vma) : 0;
return PTR_ERR_OR_ZERO(vma);
}
/* assumes mmap_sem is write-locked */

25
arch/arm/kvm/mmu.c
View File

@ -301,6 +301,14 @@ static void unmap_range(struct kvm *kvm, pgd_t *pgdp,
next = kvm_pgd_addr_end(addr, end);
if (!pgd_none(*pgd))
unmap_puds(kvm, pgd, addr, next);
/*
* If we are dealing with a large range in
* stage2 table, release the kvm->mmu_lock
* to prevent starvation and lockup detector
* warnings.
*/
if (kvm && (next != end))
cond_resched_lock(&kvm->mmu_lock);
} while (pgd++, addr = next, addr != end);
}
@ -745,6 +753,7 @@ int kvm_alloc_stage2_pgd(struct kvm *kvm)
*/
static void unmap_stage2_range(struct kvm *kvm, phys_addr_t start, u64 size)
{
assert_spin_locked(&kvm->mmu_lock);
unmap_range(kvm, kvm->arch.pgd, start, size);
}
@ -803,6 +812,7 @@ void stage2_unmap_vm(struct kvm *kvm)
int idx;
idx = srcu_read_lock(&kvm->srcu);
down_read(&current->mm->mmap_sem);
spin_lock(&kvm->mmu_lock);
slots = kvm_memslots(kvm);
@ -810,6 +820,7 @@ void stage2_unmap_vm(struct kvm *kvm)
stage2_unmap_memslot(kvm, memslot);
spin_unlock(&kvm->mmu_lock);
up_read(&current->mm->mmap_sem);
srcu_read_unlock(&kvm->srcu, idx);
}
@ -829,7 +840,10 @@ void kvm_free_stage2_pgd(struct kvm *kvm)
if (kvm->arch.pgd == NULL)
return;
spin_lock(&kvm->mmu_lock);
unmap_stage2_range(kvm, 0, KVM_PHYS_SIZE);
spin_unlock(&kvm->mmu_lock);
kvm_free_hwpgd(kvm_get_hwpgd(kvm));
if (KVM_PREALLOC_LEVEL > 0)
kfree(kvm->arch.pgd);
@ -1771,6 +1785,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
(KVM_PHYS_SIZE >> PAGE_SHIFT))
return -EFAULT;
down_read(&current->mm->mmap_sem);
/*
* A memory region could potentially cover multiple VMAs, and any holes
* between them, so iterate over all of them to find out if we can map
@ -1814,8 +1829,10 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
pa += vm_start - vma->vm_start;
/* IO region dirty page logging not allowed */
if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES)
return -EINVAL;
if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) {
ret = -EINVAL;
goto out;
}
ret = kvm_phys_addr_ioremap(kvm, gpa, pa,
vm_end - vm_start,
@ -1827,7 +1844,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
} while (hva < reg_end);
if (change == KVM_MR_FLAGS_ONLY)
return ret;
goto out;
spin_lock(&kvm->mmu_lock);
if (ret)
@ -1835,6 +1852,8 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
else
stage2_flush_memslot(kvm, memslot);
spin_unlock(&kvm->mmu_lock);
out:
up_read(&current->mm->mmap_sem);
return ret;
}

18
arch/arm/mach-at91/pm.c
View File

@ -286,6 +286,22 @@ static void at91_ddr_standby(void)
at91_ramc_write(1, AT91_DDRSDRC_LPR, saved_lpr1);
}
static void sama5d3_ddr_standby(void)
{
u32 lpr0;
u32 saved_lpr0;
saved_lpr0 = at91_ramc_read(0, AT91_DDRSDRC_LPR);
lpr0 = saved_lpr0 & ~AT91_DDRSDRC_LPCB;
lpr0 |= AT91_DDRSDRC_LPCB_POWER_DOWN;
at91_ramc_write(0, AT91_DDRSDRC_LPR, lpr0);
cpu_do_idle();
at91_ramc_write(0, AT91_DDRSDRC_LPR, saved_lpr0);
}
/* We manage both DDRAM/SDRAM controllers, we need more than one value to
* remember.
*/
@ -320,7 +336,7 @@ static const struct of_device_id const ramc_ids[] __initconst = {
{ .compatible = "atmel,at91rm9200-sdramc", .data = at91rm9200_standby },
{ .compatible = "atmel,at91sam9260-sdramc", .data = at91sam9_sdram_standby },
{ .compatible = "atmel,at91sam9g45-ddramc", .data = at91_ddr_standby },
{ .compatible = "atmel,sama5d3-ddramc", .data = at91_ddr_standby },
{ .compatible = "atmel,sama5d3-ddramc", .data = sama5d3_ddr_standby },
{ /*sentinel*/ }
};

7
arch/arm/mach-omap2/timer.c
View File

@ -496,8 +496,7 @@ void __init omap_init_time(void)
__omap_sync32k_timer_init(1, "timer_32k_ck", "ti,timer-alwon",
2, "timer_sys_ck", NULL, false);
if (of_have_populated_dt())
clocksource_probe();
clocksource_probe();
}
#if defined(CONFIG_ARCH_OMAP3) || defined(CONFIG_SOC_AM43XX)
@ -505,6 +504,8 @@ void __init omap3_secure_sync32k_timer_init(void)
{
__omap_sync32k_timer_init(12, "secure_32k_fck", "ti,timer-secure",
2, "timer_sys_ck", NULL, false);
clocksource_probe();
}
#endif /* CONFIG_ARCH_OMAP3 */
@ -513,6 +514,8 @@ void __init omap3_gptimer_timer_init(void)
{
__omap_sync32k_timer_init(2, "timer_sys_ck", NULL,
1, "timer_sys_ck", "ti,timer-alwon", true);
clocksource_probe();
}
#endif

2
arch/arm/vdso/Makefile
View File

@ -74,5 +74,5 @@ $(MODLIB)/vdso: FORCE
@mkdir -p $(MODLIB)/vdso
PHONY += vdso_install
vdso_install: $(obj)/vdso.so.dbg $(MODLIB)/vdso FORCE
vdso_install: $(obj)/vdso.so.dbg $(MODLIB)/vdso
$(call cmd,vdso_install)

32
arch/arm64/Kconfig
View File

@ -141,6 +141,18 @@ config ARCH_MMAP_RND_COMPAT_BITS_MIN
config ARCH_MMAP_RND_COMPAT_BITS_MAX
default 16
config ARM64_PAGE_SHIFT
int
default 16 if ARM64_64K_PAGES
default 14 if ARM64_16K_PAGES
default 12
config ARM64_CONT_SHIFT
int
default 5 if ARM64_64K_PAGES
default 7 if ARM64_16K_PAGES
default 4
config NO_IOPORT_MAP
def_bool y if !PCI
@ -934,6 +946,26 @@ config BUILD_ARM64_APPENDED_DTB_IMAGE
DTBs to be built by default (instead of a standalone Image.gz.)
The image will built in arch/arm64/boot/Image.gz-dtb
choice
prompt "Appended DTB Kernel Image name"
depends on BUILD_ARM64_APPENDED_DTB_IMAGE
help
Enabling this option will cause a specific kernel image Image or
Image.gz to be used for final image creation.
The image will built in arch/arm64/boot/IMAGE-NAME-dtb
config IMG_GZ_DTB
bool "Image.gz-dtb"
config IMG_DTB
bool "Image-dtb"
endchoice
config BUILD_ARM64_APPENDED_KERNEL_IMAGE_NAME
string
depends on BUILD_ARM64_APPENDED_DTB_IMAGE
default "Image.gz-dtb" if IMG_GZ_DTB
default "Image-dtb" if IMG_DTB
config BUILD_ARM64_APPENDED_DTB_IMAGE_NAMES
string "Default dtb names"
depends on BUILD_ARM64_APPENDED_DTB_IMAGE

16
arch/arm64/Makefile
View File

@ -61,7 +61,9 @@ head-y := arch/arm64/kernel/head.o
# The byte offset of the kernel image in RAM from the start of RAM.
ifeq ($(CONFIG_ARM64_RANDOMIZE_TEXT_OFFSET), y)
TEXT_OFFSET := $(shell awk 'BEGIN {srand(); printf "0x%03x000\n", int(512 * rand())}')
TEXT_OFFSET := $(shell awk "BEGIN {srand(); printf \"0x%06x\n\", \
int(2 * 1024 * 1024 / (2 ^ $(CONFIG_ARM64_PAGE_SHIFT)) * \
rand()) * (2 ^ $(CONFIG_ARM64_PAGE_SHIFT))}")
else
TEXT_OFFSET := 0x00080000
endif
@ -85,7 +87,7 @@ core-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
# Default target when executing plain make
ifeq ($(CONFIG_BUILD_ARM64_APPENDED_DTB_IMAGE),y)
KBUILD_IMAGE := Image.gz-dtb
KBUILD_IMAGE := $(subst $\",,$(CONFIG_BUILD_ARM64_APPENDED_KERNEL_IMAGE_NAME))
else
KBUILD_IMAGE := Image.gz
endif
@ -128,6 +130,16 @@ archclean:
$(Q)$(MAKE) $(clean)=$(boot)
$(Q)$(MAKE) $(clean)=$(boot)/dts
# We need to generate vdso-offsets.h before compiling certain files in kernel/.
# In order to do that, we should use the archprepare target, but we can't since
# asm-offsets.h is included in some files used to generate vdso-offsets.h, and
# asm-offsets.h is built in prepare0, for which archprepare is a dependency.
# Therefore we need to generate the header after prepare0 has been made, hence
# this hack.
prepare: vdso_prepare
vdso_prepare: prepare0
$(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso include/generated/vdso-offsets.h
define archhelp
echo '* Image.gz - Compressed kernel image (arch/$(ARCH)/boot/Image.gz)'
echo ' Image - Uncompressed kernel image (arch/$(ARCH)/boot/Image)'

28
arch/arm64/boot/dts/arm/juno-r1.dts
View File

@ -60,6 +60,28 @@ core3 {
};
};
idle-states {
entry-method = "arm,psci";
CPU_SLEEP_0: cpu-sleep-0 {
compatible = "arm,idle-state";
arm,psci-suspend-param = <0x0010000>;
local-timer-stop;
entry-latency-us = <300>;
exit-latency-us = <1200>;
min-residency-us = <2000>;
};
CLUSTER_SLEEP_0: cluster-sleep-0 {
compatible = "arm,idle-state";
arm,psci-suspend-param = <0x1010000>;
local-timer-stop;
entry-latency-us = <400>;
exit-latency-us = <1200>;
min-residency-us = <2500>;
};
};
A57_0: cpu@0 {
compatible = "arm,cortex-a57","arm,armv8";
reg = <0x0 0x0>;
@ -67,6 +89,7 @@ A57_0: cpu@0 {
enable-method = "psci";
next-level-cache = <&A57_L2>;
clocks = <&scpi_dvfs 0>;
cpu-idle-states = <&CPU_SLEEP_0 &CLUSTER_SLEEP_0>;
};
A57_1: cpu@1 {
@ -76,6 +99,7 @@ A57_1: cpu@1 {
enable-method = "psci";
next-level-cache = <&A57_L2>;
clocks = <&scpi_dvfs 0>;
cpu-idle-states = <&CPU_SLEEP_0 &CLUSTER_SLEEP_0>;
};
A53_0: cpu@100 {
@ -85,6 +109,7 @@ A53_0: cpu@100 {
enable-method = "psci";
next-level-cache = <&A53_L2>;
clocks = <&scpi_dvfs 1>;
cpu-idle-states = <&CPU_SLEEP_0 &CLUSTER_SLEEP_0>;
};
A53_1: cpu@101 {
@ -94,6 +119,7 @@ A53_1: cpu@101 {
enable-method = "psci";
next-level-cache = <&A53_L2>;
clocks = <&scpi_dvfs 1>;
cpu-idle-states = <&CPU_SLEEP_0 &CLUSTER_SLEEP_0>;
};
A53_2: cpu@102 {
@ -103,6 +129,7 @@ A53_2: cpu@102 {
enable-method = "psci";
next-level-cache = <&A53_L2>;
clocks = <&scpi_dvfs 1>;
cpu-idle-states = <&CPU_SLEEP_0 &CLUSTER_SLEEP_0>;
};
A53_3: cpu@103 {
@ -112,6 +139,7 @@ A53_3: cpu@103 {
enable-method = "psci";
next-level-cache = <&A53_L2>;
clocks = <&scpi_dvfs 1>;
cpu-idle-states = <&CPU_SLEEP_0 &CLUSTER_SLEEP_0>;
};
A57_L2: l2-cache0 {

147
arch/arm64/boot/dts/arm/juno-sched-energy.dtsi Normal file
View File

@ -0,0 +1,147 @@
/*
* ARM JUNO specific energy cost model data. There are no unit requirements for
* the data. Data can be normalized to any reference point, but the
* normalization must be consistent. That is, one bogo-joule/watt must be the
* same quantity for all data, but we don't care what it is.
*/
/* static struct idle_state idle_states_cluster_a53[] = { */
/* { .power = 56 }, /\* arch_cpu_idle() (active idle) = WFI *\/ */
/* { .power = 56 }, /\* WFI *\/ */
/* { .power = 56 }, /\* cpu-sleep-0 *\/ */
/* { .power = 17 }, /\* cluster-sleep-0 *\/ */
/* }; */
/* static struct idle_state idle_states_cluster_a57[] = { */
/* { .power = 65 }, /\* arch_cpu_idle() (active idle) = WFI *\/ */
/* { .power = 65 }, /\* WFI *\/ */
/* { .power = 65 }, /\* cpu-sleep-0 *\/ */
/* { .power = 24 }, /\* cluster-sleep-0 *\/ */
/* }; */
/* static struct capacity_state cap_states_cluster_a53[] = { */
/* /\* Power per cluster *\/ */
/* { .cap = 235, .power = 26, }, /\* 450 MHz *\/ */
/* { .cap = 303, .power = 30, }, /\* 575 MHz *\/ */
/* { .cap = 368, .power = 39, }, /\* 700 MHz *\/ */
/* { .cap = 406, .power = 47, }, /\* 775 MHz *\/ */
/* { .cap = 447, .power = 57, }, /\* 850 Mhz *\/ */
/* }; */
/* static struct capacity_state cap_states_cluster_a57[] = { */
/* /\* Power per cluster *\/ */
/* { .cap = 417, .power = 24, }, /\* 450 MHz *\/ */
/* { .cap = 579, .power = 32, }, /\* 625 MHz *\/ */
/* { .cap = 744, .power = 43, }, /\* 800 MHz *\/ */
/* { .cap = 883, .power = 49, }, /\* 950 MHz *\/ */
/* { .cap = 1024, .power = 64, }, /\* 1100 MHz *\/ */
/* }; */
/* static struct sched_group_energy energy_cluster_a53 = { */
/* .nr_idle_states = ARRAY_SIZE(idle_states_cluster_a53), */
/* .idle_states = idle_states_cluster_a53, */
/* .nr_cap_states = ARRAY_SIZE(cap_states_cluster_a53), */
/* .cap_states = cap_states_cluster_a53, */
/* }; */
/* static struct sched_group_energy energy_cluster_a57 = { */
/* .nr_idle_states = ARRAY_SIZE(idle_states_cluster_a57), */
/* .idle_states = idle_states_cluster_a57, */
/* .nr_cap_states = ARRAY_SIZE(cap_states_cluster_a57), */
/* .cap_states = cap_states_cluster_a57, */
/* }; */
/* static struct idle_state idle_states_core_a53[] = { */
/* { .power = 6 }, /\* arch_cpu_idle() (active idle) = WFI *\/ */
/* { .power = 6 }, /\* WFI *\/ */
/* { .power = 0 }, /\* cpu-sleep-0 *\/ */
/* { .power = 0 }, /\* cluster-sleep-0 *\/ */
/* }; */
/* static struct idle_state idle_states_core_a57[] = { */
/* { .power = 15 }, /\* arch_cpu_idle() (active idle) = WFI *\/ */
/* { .power = 15 }, /\* WFI *\/ */
/* { .power = 0 }, /\* cpu-sleep-0 *\/ */
/* { .power = 0 }, /\* cluster-sleep-0 *\/ */
/* }; */
/* static struct capacity_state cap_states_core_a53[] = { */
/* /\* Power per cpu *\/ */
/* { .cap = 235, .power = 33, }, /\* 450 MHz *\/ */
/* { .cap = 302, .power = 46, }, /\* 575 MHz *\/ */
/* { .cap = 368, .power = 61, }, /\* 700 MHz *\/ */
/* { .cap = 406, .power = 76, }, /\* 775 MHz *\/ */
/* { .cap = 447, .power = 93, }, /\* 850 Mhz *\/ */
/* }; */
/* static struct capacity_state cap_states_core_a57[] = { */
/* /\* Power per cpu *\/ */
/* { .cap = 417, .power = 168, }, /\* 450 MHz *\/ */
/* { .cap = 579, .power = 251, }, /\* 625 MHz *\/ */
/* { .cap = 744, .power = 359, }, /\* 800 MHz *\/ */
/* { .cap = 883, .power = 479, }, /\* 950 MHz *\/ */
/* { .cap = 1024, .power = 616, }, /\* 1100 MHz *\/ */
/* }; */
energy-costs {
CPU_COST_A57: core-cost0 {
busy-cost-data = <
417 168
579 251
744 359
883 479
1023 616
>;
idle-cost-data = <
15
15
0
0
>;
};
CPU_COST_A53: core-cost1 {
busy-cost-data = <
235 33
302 46
368 61
406 76
447 93
>;
idle-cost-data = <
6
6
0
0
>;
};
CLUSTER_COST_A57: cluster-cost0 {
busy-cost-data = <
417 24
579 32
744 43
883 49
1024 64
>;
idle-cost-data = <
65
65
65
24
>;
};
CLUSTER_COST_A53: cluster-cost1 {
busy-cost-data = <
235 26
303 30
368 39
406 47
447 57
>;
idle-cost-data = <
56
56
56
17
>;
};
};

36
arch/arm64/boot/dts/arm/juno.dts
View File

@ -60,6 +60,28 @@ core3 {
};
};
idle-states {
entry-method = "arm,psci";
CPU_SLEEP_0: cpu-sleep-0 {
compatible = "arm,idle-state";
arm,psci-suspend-param = <0x0010000>;
local-timer-stop;
entry-latency-us = <300>;
exit-latency-us = <1200>;
min-residency-us = <2000>;
};
CLUSTER_SLEEP_0: cluster-sleep-0 {
compatible = "arm,idle-state";
arm,psci-suspend-param = <0x1010000>;
local-timer-stop;
entry-latency-us = <400>;
exit-latency-us = <1200>;
min-residency-us = <2500>;
};
};
A57_0: cpu@0 {
compatible = "arm,cortex-a57","arm,armv8";
reg = <0x0 0x0>;
@ -67,6 +89,8 @@ A57_0: cpu@0 {
enable-method = "psci";
next-level-cache = <&A57_L2>;
clocks = <&scpi_dvfs 0>;
cpu-idle-states = <&CPU_SLEEP_0 &CLUSTER_SLEEP_0>;
sched-energy-costs = <&CPU_COST_A57 &CLUSTER_COST_A57>;
};
A57_1: cpu@1 {
@ -76,6 +100,8 @@ A57_1: cpu@1 {
enable-method = "psci";
next-level-cache = <&A57_L2>;
clocks = <&scpi_dvfs 0>;
cpu-idle-states = <&CPU_SLEEP_0 &CLUSTER_SLEEP_0>;
sched-energy-costs = <&CPU_COST_A57 &CLUSTER_COST_A57>;
};
A53_0: cpu@100 {
@ -85,6 +111,8 @@ A53_0: cpu@100 {
enable-method = "psci";
next-level-cache = <&A53_L2>;
clocks = <&scpi_dvfs 1>;
cpu-idle-states = <&CPU_SLEEP_0 &CLUSTER_SLEEP_0>;
sched-energy-costs = <&CPU_COST_A53 &CLUSTER_COST_A53>;
};
A53_1: cpu@101 {
@ -94,6 +122,8 @@ A53_1: cpu@101 {
enable-method = "psci";
next-level-cache = <&A53_L2>;
clocks = <&scpi_dvfs 1>;
cpu-idle-states = <&CPU_SLEEP_0 &CLUSTER_SLEEP_0>;
sched-energy-costs = <&CPU_COST_A53 &CLUSTER_COST_A53>;
};
A53_2: cpu@102 {
@ -103,6 +133,8 @@ A53_2: cpu@102 {
enable-method = "psci";
next-level-cache = <&A53_L2>;
clocks = <&scpi_dvfs 1>;
cpu-idle-states = <&CPU_SLEEP_0 &CLUSTER_SLEEP_0>;
sched-energy-costs = <&CPU_COST_A53 &CLUSTER_COST_A53>;
};
A53_3: cpu@103 {
@ -112,6 +144,8 @@ A53_3: cpu@103 {
enable-method = "psci";
next-level-cache = <&A53_L2>;
clocks = <&scpi_dvfs 1>;
cpu-idle-states = <&CPU_SLEEP_0 &CLUSTER_SLEEP_0>;
sched-energy-costs = <&CPU_COST_A53 &CLUSTER_COST_A53>;
};
A57_L2: l2-cache0 {
@ -121,6 +155,8 @@ A57_L2: l2-cache0 {
A53_L2: l2-cache1 {
compatible = "cache";
};
/include/ "juno-sched-energy.dtsi"
};
pmu_a57 {

5
arch/arm64/crypto/Kconfig
View File

@ -23,6 +23,11 @@ config CRYPTO_GHASH_ARM64_CE
depends on ARM64 && KERNEL_MODE_NEON
select CRYPTO_HASH
config CRYPTO_POLY_HASH_ARM64_CE
tristate "poly_hash (for HEH encryption mode) using ARMv8 Crypto Extensions"
depends on ARM64 && KERNEL_MODE_NEON
select CRYPTO_HASH
config CRYPTO_AES_ARM64_CE
tristate "AES core cipher using ARMv8 Crypto Extensions"
depends on ARM64 && KERNEL_MODE_NEON

3
arch/arm64/crypto/Makefile
View File

@ -17,6 +17,9 @@ sha2-ce-y := sha2-ce-glue.o sha2-ce-core.o
obj-$(CONFIG_CRYPTO_GHASH_ARM64_CE) += ghash-ce.o
ghash-ce-y := ghash-ce-glue.o ghash-ce-core.o
obj-$(CONFIG_CRYPTO_POLY_HASH_ARM64_CE) += poly-hash-ce.o
poly-hash-ce-y := poly-hash-ce-glue.o poly-hash-ce-core.o
obj-$(CONFIG_CRYPTO_AES_ARM64_CE) += aes-ce-cipher.o
CFLAGS_aes-ce-cipher.o += -march=armv8-a+crypto

4
arch/arm64/crypto/aes-glue.c
View File

@ -294,7 +294,7 @@ static struct crypto_alg aes_algs[] = { {
.cra_blkcipher = {
.min_keysize = AES_MIN_KEY_SIZE,
.max_keysize = AES_MAX_KEY_SIZE,
.ivsize = AES_BLOCK_SIZE,
.ivsize = 0,
.setkey = aes_setkey,
.encrypt = ecb_encrypt,
.decrypt = ecb_decrypt,
@ -371,7 +371,7 @@ static struct crypto_alg aes_algs[] = { {
.cra_ablkcipher = {
.min_keysize = AES_MIN_KEY_SIZE,
.max_keysize = AES_MAX_KEY_SIZE,
.ivsize = AES_BLOCK_SIZE,
.ivsize = 0,
.setkey = ablk_set_key,
.encrypt = ablk_encrypt,
.decrypt = ablk_decrypt,

163
arch/arm64/crypto/poly-hash-ce-core.S Normal file
View File

@ -0,0 +1,163 @@
/*
* Accelerated poly_hash implementation with ARMv8 PMULL instructions.
*
* Based on ghash-ce-core.S.
*
* Copyright (C) 2014 Linaro Ltd. <ard.biesheuvel@linaro.org>
* Copyright (C) 2017 Google, Inc. <ebiggers@google.com>
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*/
#include <linux/linkage.h>
#include <asm/assembler.h>
KEY .req v0
KEY2 .req v1
T1 .req v2
T2 .req v3
GSTAR .req v4
XL .req v5
XM .req v6
XH .req v7
.text
.arch armv8-a+crypto
/* 16-byte aligned (2**4 = 16); not required, but might as well */
.align 4
.Lgstar:
.quad 0x87, 0x87
/*
* void pmull_poly_hash_update(le128 *digest, const le128 *key,
* const u8 *src, unsigned int blocks,
* unsigned int partial);
*/
ENTRY(pmull_poly_hash_update)
/* Load digest into XL */
ld1 {XL.16b}, [x0]
/* Load key into KEY */
ld1 {KEY.16b}, [x1]
/* Load g*(x) = g(x) + x^128 = x^7 + x^2 + x + 1 into both halves of
* GSTAR */
adr x1, .Lgstar
ld1 {GSTAR.2d}, [x1]
/* Set KEY2 to (KEY[1]+KEY[0]):(KEY[1]+KEY[0]). This is needed for
* Karatsuba multiplication. */
ext KEY2.16b, KEY.16b, KEY.16b, #8
eor KEY2.16b, KEY2.16b, KEY.16b
/* If 'partial' is nonzero, then we're finishing a pending block and
* should go right to the multiplication. */
cbnz w4, 1f
0:
/* Add the next block from 'src' to the digest */
ld1 {T1.16b}, [x2], #16
eor XL.16b, XL.16b, T1.16b
sub w3, w3, #1
1:
/*
* Multiply the current 128-bit digest (a1:a0, in XL) by the 128-bit key
* (b1:b0, in KEY) using Karatsuba multiplication.
*/
/* T1 = (a1+a0):(a1+a0) */
ext T1.16b, XL.16b, XL.16b, #8
eor T1.16b, T1.16b, XL.16b
/* XH = a1 * b1 */
pmull2 XH.1q, XL.2d, KEY.2d
/* XL = a0 * b0 */
pmull XL.1q, XL.1d, KEY.1d
/* XM = (a1+a0) * (b1+b0) */
pmull XM.1q, T1.1d, KEY2.1d
/* XM += (XH[0]:XL[1]) + XL + XH */
ext T1.16b, XL.16b, XH.16b, #8
eor T2.16b, XL.16b, XH.16b
eor XM.16b, XM.16b, T1.16b
eor XM.16b, XM.16b, T2.16b
/*
* Now the 256-bit product is in XH[1]:XM:XL[0]. It represents a
* polynomial over GF(2) with degree as large as 255. We need to
* compute its remainder modulo g(x) = x^128+x^7+x^2+x+1. For this it
* is sufficient to compute the remainder of the high half 'c(x)x^128'
* add it to the low half. To reduce the high half we use the Barrett
* reduction method. The basic idea is that we can express the
* remainder p(x) as g(x)q(x) mod x^128, where q(x) = (c(x)x^128)/g(x).
* As detailed in [1], to avoid having to divide by g(x) at runtime the
* following equivalent expression can be derived:
*
* p(x) = [ g*(x)((c(x)q+(x))/x^128) ] mod x^128
*
* where g*(x) = x^128+g(x) = x^7+x^2+x+1, and q+(x) = x^256/g(x) = g(x)
* in this case. This is also equivalent to:
*
* p(x) = [ g*(x)((c(x)(x^128 + g*(x)))/x^128) ] mod x^128
* = [ g*(x)(c(x) + (c(x)g*(x))/x^128) ] mod x^128
*
* Since deg g*(x) < 64:
*
* p(x) = [ g*(x)(c(x) + ((c(x)/x^64)g*(x))/x^64) ] mod x^128
* = [ g*(x)((c(x)/x^64)x^64 + (c(x) mod x^64) +
* ((c(x)/x^64)g*(x))/x^64) ] mod x^128
*
* Letting t(x) = g*(x)(c(x)/x^64):
*
* p(x) = [ t(x)x^64 + g*(x)((c(x) mod x^64) + t(x)/x^64) ] mod x^128
*
* Therefore, to do the reduction we only need to issue two 64-bit =>
* 128-bit carryless multiplications: g*(x) times c(x)/x^64, and g*(x)
* times ((c(x) mod x^64) + t(x)/x^64). (Multiplication by x^64 doesn't
* count since it is simply a shift or move.)
*
* An alternate reduction method, also based on Barrett reduction and
* described in [1], uses only shifts and XORs --- no multiplications.
* However, the method with multiplications requires fewer instructions
* and is faster on processors with fast carryless multiplication.
*
* [1] "Intel Carry-Less Multiplication Instruction and its Usage for
* Computing the GCM Mode",
* https://software.intel.com/sites/default/files/managed/72/cc/clmul-wp-rev-2.02-2014-04-20.pdf
*/
/* 256-bit product is XH[1]:XM:XL[0], so c(x) is XH[1]:XM[1] */
/* T1 = t(x) = g*(x)(c(x)/x^64) */
pmull2 T1.1q, GSTAR.2d, XH.2d
/* T2 = g*(x)((c(x) mod x^64) + t(x)/x^64) */
eor T2.16b, XM.16b, T1.16b
pmull2 T2.1q, GSTAR.2d, T2.2d
/* Make XL[0] be the low half of the 128-bit result by adding the low 64
* bits of the T2 term to what was already there. The 't(x)x^64' term
* makes no difference, so skip it. */
eor XL.16b, XL.16b, T2.16b
/* Make XL[1] be the high half of the 128-bit result by adding the high
* 64 bits of the 't(x)x^64' and T2 terms to what was already in XM[0],
* then moving XM[0] to XL[1]. */
eor XM.16b, XM.16b, T1.16b
ext T2.16b, T2.16b, T2.16b, #8
eor XM.16b, XM.16b, T2.16b
mov XL.d[1], XM.d[0]
/* If more blocks remain, then loop back to process the next block;
* else, store the digest and return. */
cbnz w3, 0b
st1 {XL.16b}, [x0]
ret
ENDPROC(pmull_poly_hash_update)

166
arch/arm64/crypto/poly-hash-ce-glue.c Normal file
View File

@ -0,0 +1,166 @@
/*
* Accelerated poly_hash implementation with ARMv8 PMULL instructions.
*
* Based on ghash-ce-glue.c.
*
* poly_hash is part of the HEH (Hash-Encrypt-Hash) encryption mode, proposed in
* Internet Draft https://tools.ietf.org/html/draft-cope-heh-01.
*
* poly_hash is very similar to GHASH: both algorithms are keyed hashes which
* interpret their input data as coefficients of a polynomial over GF(2^128),
* then calculate a hash value by evaluating that polynomial at the point given
* by the key, e.g. using Horner's rule. The difference is that poly_hash uses
* the more natural "ble" convention to represent GF(2^128) elements, whereas
* GHASH uses the less natural "lle" convention (see include/crypto/gf128mul.h).
* The ble convention makes it simpler to implement GF(2^128) multiplication.
*
* Copyright (C) 2014 Linaro Ltd. <ard.biesheuvel@linaro.org>
* Copyright (C) 2017 Google Inc. <ebiggers@google.com>
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*/
#include <asm/neon.h>
#include <crypto/b128ops.h>
#include <crypto/internal/hash.h>
#include <linux/cpufeature.h>
#include <linux/crypto.h>
#include <linux/module.h>
/*
* Note: in this algorithm we currently use 'le128' to represent GF(2^128)
* elements, even though poly_hash-generic uses 'be128'. Both types are
* actually "wrong" because the elements are actually in 'ble' format, and there
* should be a ble type to represent this --- as well as lle, bbe, and lbe types
* for the other conventions for representing GF(2^128) elements. But
* practically it doesn't matter which type we choose here, so we just use le128
* since it's arguably more accurate, while poly_hash-generic still has to use
* be128 because the generic GF(2^128) multiplication functions all take be128.
*/
struct poly_hash_desc_ctx {
le128 digest;
unsigned int count;
};
asmlinkage void pmull_poly_hash_update(le128 *digest, const le128 *key,
const u8 *src, unsigned int blocks,
unsigned int partial);
static int poly_hash_setkey(struct crypto_shash *tfm,
const u8 *key, unsigned int keylen)
{
if (keylen != sizeof(le128)) {
crypto_shash_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN);
return -EINVAL;
}
memcpy(crypto_shash_ctx(tfm), key, sizeof(le128));
return 0;
}
static int poly_hash_init(struct shash_desc *desc)
{
struct poly_hash_desc_ctx *ctx = shash_desc_ctx(desc);
ctx->digest = (le128) { 0 };
ctx->count = 0;
return 0;
}
static int poly_hash_update(struct shash_desc *desc, const u8 *src,
unsigned int len)
{
struct poly_hash_desc_ctx *ctx = shash_desc_ctx(desc);
unsigned int partial = ctx->count % sizeof(le128);
u8 *dst = (u8 *)&ctx->digest + partial;
ctx->count += len;
/* Finishing at least one block? */
if (partial + len >= sizeof(le128)) {
const le128 *key = crypto_shash_ctx(desc->tfm);
if (partial) {
/* Finish the pending block. */
unsigned int n = sizeof(le128) - partial;
len -= n;
do {
*dst++ ^= *src++;
} while (--n);
}
/*
* Do the real work. If 'partial' is nonzero, this starts by
* multiplying 'digest' by 'key'. Then for each additional full
* block it adds the block to 'digest' and multiplies by 'key'.
*/
kernel_neon_begin_partial(8);
pmull_poly_hash_update(&ctx->digest, key, src,
len / sizeof(le128), partial);
kernel_neon_end();
src += len - (len % sizeof(le128));
len %= sizeof(le128);
dst = (u8 *)&ctx->digest;
}
/* Continue adding the next block to 'digest'. */
while (len--)
*dst++ ^= *src++;
return 0;
}
static int poly_hash_final(struct shash_desc *desc, u8 *out)
{
struct poly_hash_desc_ctx *ctx = shash_desc_ctx(desc);
unsigned int partial = ctx->count % sizeof(le128);
/* Finish the last block if needed. */
if (partial) {
const le128 *key = crypto_shash_ctx(desc->tfm);
kernel_neon_begin_partial(8);
pmull_poly_hash_update(&ctx->digest, key, NULL, 0, partial);
kernel_neon_end();
}
memcpy(out, &ctx->digest, sizeof(le128));
return 0;
}
static struct shash_alg poly_hash_alg = {
.digestsize = sizeof(le128),
.init = poly_hash_init,
.update = poly_hash_update,
.final = poly_hash_final,
.setkey = poly_hash_setkey,
.descsize = sizeof(struct poly_hash_desc_ctx),
.base = {
.cra_name = "poly_hash",
.cra_driver_name = "poly_hash-ce",
.cra_priority = 300,
.cra_ctxsize = sizeof(le128),
.cra_module = THIS_MODULE,
},
};
static int __init poly_hash_ce_mod_init(void)
{
return crypto_register_shash(&poly_hash_alg);
}
static void __exit poly_hash_ce_mod_exit(void)
{
crypto_unregister_shash(&poly_hash_alg);
}
MODULE_DESCRIPTION("Polynomial evaluation hash using ARMv8 Crypto Extensions");
MODULE_AUTHOR("Eric Biggers <ebiggers@google.com>");
MODULE_LICENSE("GPL v2");
module_cpu_feature_match(PMULL, poly_hash_ce_mod_init);
module_exit(poly_hash_ce_mod_exit);

6
arch/arm64/include/asm/hw_breakpoint.h
View File

@ -68,7 +68,11 @@ static inline void decode_ctrl_reg(u32 reg,
/* Lengths */
#define ARM_BREAKPOINT_LEN_1 0x1
#define ARM_BREAKPOINT_LEN_2 0x3
#define ARM_BREAKPOINT_LEN_3 0x7
#define ARM_BREAKPOINT_LEN_4 0xf
#define ARM_BREAKPOINT_LEN_5 0x1f
#define ARM_BREAKPOINT_LEN_6 0x3f
#define ARM_BREAKPOINT_LEN_7 0x7f
#define ARM_BREAKPOINT_LEN_8 0xff
/* Kernel stepping */
@ -110,7 +114,7 @@ struct perf_event;
struct pmu;
extern int arch_bp_generic_fields(struct arch_hw_breakpoint_ctrl ctrl,
int *gen_len, int *gen_type);
int *gen_len, int *gen_type, int *offset);
extern int arch_check_bp_in_kernelspace(struct perf_event *bp);
extern int arch_validate_hwbkpt_settings(struct perf_event *bp);
extern int hw_breakpoint_exceptions_notify(struct notifier_block *unused,

12
arch/arm64/include/asm/page.h
View File

@ -23,16 +23,8 @@
/* PAGE_SHIFT determines the page size */
/* CONT_SHIFT determines the number of pages which can be tracked together */
#ifdef CONFIG_ARM64_64K_PAGES
#define PAGE_SHIFT 16
#define CONT_SHIFT 5
#elif defined(CONFIG_ARM64_16K_PAGES)
#define PAGE_SHIFT 14
#define CONT_SHIFT 7
#else
#define PAGE_SHIFT 12
#define CONT_SHIFT 4
#endif
#define PAGE_SHIFT CONFIG_ARM64_PAGE_SHIFT
#define CONT_SHIFT CONFIG_ARM64_CONT_SHIFT
#define PAGE_SIZE (_AC(1, UL) << PAGE_SHIFT)
#define PAGE_MASK (~(PAGE_SIZE-1))

7
arch/arm64/include/asm/spinlock.h
View File

@ -31,6 +31,12 @@ static inline void arch_spin_unlock_wait(arch_spinlock_t *lock)
unsigned int tmp;
arch_spinlock_t lockval;
/*
* Ensure prior spin_lock operations to other locks have completed
* on this CPU before we test whether "lock" is locked.
*/
smp_mb();
asm volatile(
" sevl\n"
"1: wfe\n"
@ -152,6 +158,7 @@ static inline int arch_spin_value_unlocked(arch_spinlock_t lock)
static inline int arch_spin_is_locked(arch_spinlock_t *lock)
{
smp_mb(); /* See arch_spin_unlock_wait */
return !arch_spin_value_unlocked(READ_ONCE(*lock));
}

8
arch/arm64/include/asm/vdso_datapage.h
View File

@ -22,6 +22,8 @@
struct vdso_data {
__u64 cs_cycle_last; /* Timebase at clocksource init */
__u64 raw_time_sec; /* Raw time */
__u64 raw_time_nsec;
__u64 xtime_clock_sec; /* Kernel time */
__u64 xtime_clock_nsec;
__u64 xtime_coarse_sec; /* Coarse time */
@ -29,8 +31,10 @@ struct vdso_data {
__u64 wtm_clock_sec; /* Wall to monotonic time */
__u64 wtm_clock_nsec;
__u32 tb_seq_count; /* Timebase sequence counter */
__u32 cs_mult; /* Clocksource multiplier */
__u32 cs_shift; /* Clocksource shift */
/* cs_* members must be adjacent and in this order (ldp accesses) */
__u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
__u32 cs_shift; /* Clocksource shift (mono = raw) */
__u32 cs_raw_mult; /* Raw clocksource multiplier */
__u32 tz_minuteswest; /* Whacky timezone stuff */
__u32 tz_dsttime;
__u32 use_syscall;

4
arch/arm64/kernel/Makefile
View File

@ -50,7 +50,3 @@ obj-y += $(arm64-obj-y) vdso/ probes/
obj-m += $(arm64-obj-m)
head-y := head.o
extra-y += $(head-y) vmlinux.lds
# vDSO - this must be built first to generate the symbol offsets
$(call objectify,$(arm64-obj-y)): $(obj)/vdso/vdso-offsets.h
$(obj)/vdso/vdso-offsets.h: $(obj)/vdso

6
arch/arm64/kernel/asm-offsets.c
View File

@ -92,6 +92,7 @@ int main(void)
BLANK();
DEFINE(CLOCK_REALTIME, CLOCK_REALTIME);
DEFINE(CLOCK_MONOTONIC, CLOCK_MONOTONIC);
DEFINE(CLOCK_MONOTONIC_RAW, CLOCK_MONOTONIC_RAW);
DEFINE(CLOCK_REALTIME_RES, MONOTONIC_RES_NSEC);
DEFINE(CLOCK_REALTIME_COARSE, CLOCK_REALTIME_COARSE);
DEFINE(CLOCK_MONOTONIC_COARSE,CLOCK_MONOTONIC_COARSE);
@ -99,6 +100,8 @@ int main(void)
DEFINE(NSEC_PER_SEC, NSEC_PER_SEC);
BLANK();
DEFINE(VDSO_CS_CYCLE_LAST, offsetof(struct vdso_data, cs_cycle_last));
DEFINE(VDSO_RAW_TIME_SEC, offsetof(struct vdso_data, raw_time_sec));
DEFINE(VDSO_RAW_TIME_NSEC, offsetof(struct vdso_data, raw_time_nsec));
DEFINE(VDSO_XTIME_CLK_SEC, offsetof(struct vdso_data, xtime_clock_sec));
DEFINE(VDSO_XTIME_CLK_NSEC, offsetof(struct vdso_data, xtime_clock_nsec));
DEFINE(VDSO_XTIME_CRS_SEC, offsetof(struct vdso_data, xtime_coarse_sec));
@ -106,7 +109,8 @@ int main(void)
DEFINE(VDSO_WTM_CLK_SEC, offsetof(struct vdso_data, wtm_clock_sec));
DEFINE(VDSO_WTM_CLK_NSEC, offsetof(struct vdso_data, wtm_clock_nsec));
DEFINE(VDSO_TB_SEQ_COUNT, offsetof(struct vdso_data, tb_seq_count));
DEFINE(VDSO_CS_MULT, offsetof(struct vdso_data, cs_mult));
DEFINE(VDSO_CS_MONO_MULT, offsetof(struct vdso_data, cs_mono_mult));
DEFINE(VDSO_CS_RAW_MULT, offsetof(struct vdso_data, cs_raw_mult));
DEFINE(VDSO_CS_SHIFT, offsetof(struct vdso_data, cs_shift));
DEFINE(VDSO_TZ_MINWEST, offsetof(struct vdso_data, tz_minuteswest));
DEFINE(VDSO_TZ_DSTTIME, offsetof(struct vdso_data, tz_dsttime));

82
arch/arm64/kernel/hibernate.c
View File

@ -34,6 +34,7 @@
#include <asm/pgtable-hwdef.h>
#include <asm/sections.h>
#include <asm/suspend.h>
#include <asm/sysreg.h>
#include <asm/virt.h>
/*
@ -216,12 +217,22 @@ static int create_safe_exec_page(void *src_start, size_t length,
set_pte(pte, __pte(virt_to_phys((void *)dst) |
pgprot_val(PAGE_KERNEL_EXEC)));
/* Load our new page tables */
asm volatile("msr ttbr0_el1, %0;"
"isb;"
"tlbi vmalle1is;"
"dsb ish;"
"isb" : : "r"(virt_to_phys(pgd)));
/*
* Load our new page tables. A strict BBM approach requires that we
* ensure that TLBs are free of any entries that may overlap with the
* global mappings we are about to install.
*
* For a real hibernate/resume cycle TTBR0 currently points to a zero
* page, but TLBs may contain stale ASID-tagged entries (e.g. for EFI
* runtime services), while for a userspace-driven test_resume cycle it
* points to userspace page tables (and we must point it at a zero page
* ourselves). Elsewhere we only (un)install the idmap with preemption
* disabled, so T0SZ should be as required regardless.
*/
cpu_set_reserved_ttbr0();
local_flush_tlb_all();
write_sysreg(virt_to_phys(pgd), ttbr0_el1);
isb();
*phys_dst_addr = virt_to_phys((void *)dst);
@ -387,6 +398,38 @@ int swsusp_arch_resume(void)
void __noreturn (*hibernate_exit)(phys_addr_t, phys_addr_t, void *,
void *, phys_addr_t, phys_addr_t);
/*
* Restoring the memory image will overwrite the ttbr1 page tables.
* Create a second copy of just the linear map, and use this when
* restoring.
*/
tmp_pg_dir = (pgd_t *)get_safe_page(GFP_ATOMIC);
if (!tmp_pg_dir) {
pr_err("Failed to allocate memory for temporary page tables.");
rc = -ENOMEM;
goto out;
}
rc = copy_page_tables(tmp_pg_dir, PAGE_OFFSET, 0);
if (rc)
goto out;
/*
* Since we only copied the linear map, we need to find restore_pblist's
* linear map address.
*/
lm_restore_pblist = LMADDR(restore_pblist);
/*
* We need a zero page that is zero before & after resume in order to
* to break before make on the ttbr1 page tables.
*/
zero_page = (void *)get_safe_page(GFP_ATOMIC);
if (!zero_page) {
pr_err("Failed to allocate zero page.");
rc = -ENOMEM;
goto out;
}
/*
* Locate the exit code in the bottom-but-one page, so that *NULL
* still has disastrous affects.
@ -412,27 +455,6 @@ int swsusp_arch_resume(void)
*/
__flush_dcache_area(hibernate_exit, exit_size);
/*
* Restoring the memory image will overwrite the ttbr1 page tables.
* Create a second copy of just the linear map, and use this when
* restoring.
*/
tmp_pg_dir = (pgd_t *)get_safe_page(GFP_ATOMIC);
if (!tmp_pg_dir) {
pr_err("Failed to allocate memory for temporary page tables.");
rc = -ENOMEM;
goto out;
}
rc = copy_page_tables(tmp_pg_dir, PAGE_OFFSET, 0);
if (rc)
goto out;
/*
* Since we only copied the linear map, we need to find restore_pblist's
* linear map address.
*/
lm_restore_pblist = LMADDR(restore_pblist);
/*
* KASLR will cause the el2 vectors to be in a different location in
* the resumed kernel. Load hibernate's temporary copy into el2.
@ -447,12 +469,6 @@ int swsusp_arch_resume(void)
__hyp_set_vectors(el2_vectors);
}
/*
* We need a zero page that is zero before & after resume in order to
* to break before make on the ttbr1 page tables.
*/
zero_page = (void *)get_safe_page(GFP_ATOMIC);
hibernate_exit(virt_to_phys(tmp_pg_dir), resume_hdr.ttbr1_el1,
resume_hdr.reenter_kernel, lm_restore_pblist,
resume_hdr.__hyp_stub_vectors, virt_to_phys(zero_page));

155
arch/arm64/kernel/hw_breakpoint.c
View File

@ -317,9 +317,21 @@ static int get_hbp_len(u8 hbp_len)
case ARM_BREAKPOINT_LEN_2:
len_in_bytes = 2;
break;
case ARM_BREAKPOINT_LEN_3:
len_in_bytes = 3;
break;
case ARM_BREAKPOINT_LEN_4:
len_in_bytes = 4;
break;
case ARM_BREAKPOINT_LEN_5:
len_in_bytes = 5;
break;
case ARM_BREAKPOINT_LEN_6:
len_in_bytes = 6;
break;
case ARM_BREAKPOINT_LEN_7:
len_in_bytes = 7;
break;
case ARM_BREAKPOINT_LEN_8:
len_in_bytes = 8;
break;
@ -349,7 +361,7 @@ int arch_check_bp_in_kernelspace(struct perf_event *bp)
* to generic breakpoint descriptions.
*/
int arch_bp_generic_fields(struct arch_hw_breakpoint_ctrl ctrl,
int *gen_len, int *gen_type)
int *gen_len, int *gen_type, int *offset)
{
/* Type */
switch (ctrl.type) {
@ -369,17 +381,33 @@ int arch_bp_generic_fields(struct arch_hw_breakpoint_ctrl ctrl,
return -EINVAL;
}
if (!ctrl.len)
return -EINVAL;
*offset = __ffs(ctrl.len);
/* Len */
switch (ctrl.len) {
switch (ctrl.len >> *offset) {
case ARM_BREAKPOINT_LEN_1:
*gen_len = HW_BREAKPOINT_LEN_1;
break;
case ARM_BREAKPOINT_LEN_2:
*gen_len = HW_BREAKPOINT_LEN_2;
break;
case ARM_BREAKPOINT_LEN_3:
*gen_len = HW_BREAKPOINT_LEN_3;
break;
case ARM_BREAKPOINT_LEN_4:
*gen_len = HW_BREAKPOINT_LEN_4;
break;
case ARM_BREAKPOINT_LEN_5:
*gen_len = HW_BREAKPOINT_LEN_5;
break;
case ARM_BREAKPOINT_LEN_6:
*gen_len = HW_BREAKPOINT_LEN_6;
break;
case ARM_BREAKPOINT_LEN_7:
*gen_len = HW_BREAKPOINT_LEN_7;
break;
case ARM_BREAKPOINT_LEN_8:
*gen_len = HW_BREAKPOINT_LEN_8;
break;
@ -423,9 +451,21 @@ static int arch_build_bp_info(struct perf_event *bp)
case HW_BREAKPOINT_LEN_2:
info->ctrl.len = ARM_BREAKPOINT_LEN_2;
break;
case HW_BREAKPOINT_LEN_3:
info->ctrl.len = ARM_BREAKPOINT_LEN_3;
break;
case HW_BREAKPOINT_LEN_4:
info->ctrl.len = ARM_BREAKPOINT_LEN_4;
break;
case HW_BREAKPOINT_LEN_5:
info->ctrl.len = ARM_BREAKPOINT_LEN_5;
break;
case HW_BREAKPOINT_LEN_6:
info->ctrl.len = ARM_BREAKPOINT_LEN_6;
break;
case HW_BREAKPOINT_LEN_7:
info->ctrl.len = ARM_BREAKPOINT_LEN_7;
break;
case HW_BREAKPOINT_LEN_8:
info->ctrl.len = ARM_BREAKPOINT_LEN_8;
break;
@ -517,18 +557,17 @@ int arch_validate_hwbkpt_settings(struct perf_event *bp)
default:
return -EINVAL;
}
info->address &= ~alignment_mask;
info->ctrl.len <<= offset;
} else {
if (info->ctrl.type == ARM_BREAKPOINT_EXECUTE)
alignment_mask = 0x3;
else
alignment_mask = 0x7;
if (info->address & alignment_mask)
return -EINVAL;
offset = info->address & alignment_mask;
}
info->address &= ~alignment_mask;
info->ctrl.len <<= offset;
/*
* Disallow per-task kernel breakpoints since these would
* complicate the stepping code.
@ -661,12 +700,47 @@ static int breakpoint_handler(unsigned long unused, unsigned int esr,
}
NOKPROBE_SYMBOL(breakpoint_handler);
/*
* Arm64 hardware does not always report a watchpoint hit address that matches
* one of the watchpoints set. It can also report an address "near" the
* watchpoint if a single instruction access both watched and unwatched
* addresses. There is no straight-forward way, short of disassembling the
* offending instruction, to map that address back to the watchpoint. This
* function computes the distance of the memory access from the watchpoint as a
* heuristic for the likelyhood that a given access triggered the watchpoint.
*
* See Section D2.10.5 "Determining the memory location that caused a Watchpoint
* exception" of ARMv8 Architecture Reference Manual for details.
*
* The function returns the distance of the address from the bytes watched by
* the watchpoint. In case of an exact match, it returns 0.
*/
static u64 get_distance_from_watchpoint(unsigned long addr, u64 val,
struct arch_hw_breakpoint_ctrl *ctrl)
{
u64 wp_low, wp_high;
u32 lens, lene;
lens = __ffs(ctrl->len);
lene = __fls(ctrl->len);
wp_low = val + lens;
wp_high = val + lene;
if (addr < wp_low)
return wp_low - addr;
else if (addr > wp_high)
return addr - wp_high;
else
return 0;
}
static int watchpoint_handler(unsigned long addr, unsigned int esr,
struct pt_regs *regs)
{
int i, step = 0, *kernel_step, access;
int i, step = 0, *kernel_step, access, closest_match = 0;
u64 min_dist = -1, dist;
u32 ctrl_reg;
u64 val, alignment_mask;
u64 val;
struct perf_event *wp, **slots;
struct debug_info *debug_info;
struct arch_hw_breakpoint *info;
@ -675,35 +749,15 @@ static int watchpoint_handler(unsigned long addr, unsigned int esr,
slots = this_cpu_ptr(wp_on_reg);
debug_info = &current->thread.debug;
/*
* Find all watchpoints that match the reported address. If no exact
* match is found. Attribute the hit to the closest watchpoint.
*/
rcu_read_lock();
for (i = 0; i < core_num_wrps; ++i) {
rcu_read_lock();
wp = slots[i];
if (wp == NULL)
goto unlock;
info = counter_arch_bp(wp);
/* AArch32 watchpoints are either 4 or 8 bytes aligned. */
if (is_compat_task()) {
if (info->ctrl.len == ARM_BREAKPOINT_LEN_8)
alignment_mask = 0x7;
else
alignment_mask = 0x3;
} else {
alignment_mask = 0x7;
}
/* Check if the watchpoint value matches. */
val = read_wb_reg(AARCH64_DBG_REG_WVR, i);
if (val != (addr & ~alignment_mask))
goto unlock;
/* Possible match, check the byte address select to confirm. */
ctrl_reg = read_wb_reg(AARCH64_DBG_REG_WCR, i);
decode_ctrl_reg(ctrl_reg, &ctrl);
if (!((1 << (addr & alignment_mask)) & ctrl.len))
goto unlock;
continue;
/*
* Check that the access type matches.
@ -712,18 +766,41 @@ static int watchpoint_handler(unsigned long addr, unsigned int esr,
access = (esr & AARCH64_ESR_ACCESS_MASK) ? HW_BREAKPOINT_W :
HW_BREAKPOINT_R;
if (!(access & hw_breakpoint_type(wp)))
goto unlock;
continue;
/* Check if the watchpoint value and byte select match. */
val = read_wb_reg(AARCH64_DBG_REG_WVR, i);
ctrl_reg = read_wb_reg(AARCH64_DBG_REG_WCR, i);
decode_ctrl_reg(ctrl_reg, &ctrl);
dist = get_distance_from_watchpoint(addr, val, &ctrl);
if (dist < min_dist) {
min_dist = dist;
closest_match = i;
}
/* Is this an exact match? */
if (dist != 0)
continue;
info = counter_arch_bp(wp);
info->trigger = addr;
perf_bp_event(wp, regs);
/* Do we need to handle the stepping? */
if (!wp->overflow_handler)
step = 1;
unlock:
rcu_read_unlock();
}
if (min_dist > 0 && min_dist != -1) {
/* No exact match found. */
wp = slots[closest_match];
info = counter_arch_bp(wp);
info->trigger = addr;
perf_bp_event(wp, regs);
/* Do we need to handle the stepping? */
if (!wp->overflow_handler)
step = 1;
}
rcu_read_unlock();
if (!step)
return 0;

7
arch/arm64/kernel/ptrace.c
View File

@ -327,13 +327,13 @@ static int ptrace_hbp_fill_attr_ctrl(unsigned int note_type,
struct arch_hw_breakpoint_ctrl ctrl,
struct perf_event_attr *attr)
{
int err, len, type, disabled = !ctrl.enabled;
int err, len, type, offset, disabled = !ctrl.enabled;
attr->disabled = disabled;
if (disabled)
return 0;
err = arch_bp_generic_fields(ctrl, &len, &type);
err = arch_bp_generic_fields(ctrl, &len, &type, &offset);
if (err)
return err;
@ -352,6 +352,7 @@ static int ptrace_hbp_fill_attr_ctrl(unsigned int note_type,
attr->bp_len = len;
attr->bp_type = type;
attr->bp_addr += offset;
return 0;
}
@ -404,7 +405,7 @@ static int ptrace_hbp_get_addr(unsigned int note_type,
if (IS_ERR(bp))
return PTR_ERR(bp);
*addr = bp ? bp->attr.bp_addr : 0;
*addr = bp ? counter_arch_bp(bp)->address : 0;
return 0;
}

5
arch/arm64/kernel/stacktrace.c
View File

@ -43,6 +43,9 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame)
unsigned long fp = frame->fp;
unsigned long irq_stack_ptr;
if (!tsk)
tsk = current;
/*
* Switching between stacks is valid when tracing current and in
* non-preemptible context.
@ -67,7 +70,7 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame)
frame->pc = *(unsigned long *)(fp + 8);
#ifdef CONFIG_FUNCTION_GRAPH_TRACER
if (tsk && tsk->ret_stack &&
if (tsk->ret_stack &&
(frame->pc == (unsigned long)return_to_handler)) {
/*
* This is a case where function graph tracer has

10
arch/arm64/kernel/traps.c
View File

@ -149,6 +149,11 @@ static void dump_backtrace(struct pt_regs *regs, struct task_struct *tsk)
unsigned long irq_stack_ptr;
int skip;
pr_debug("%s(regs = %p tsk = %p)\n", __func__, regs, tsk);
if (!tsk)
tsk = current;
/*
* Switching between stacks is valid when tracing current and in
* non-preemptible context.
@ -158,11 +163,6 @@ static void dump_backtrace(struct pt_regs *regs, struct task_struct *tsk)
else
irq_stack_ptr = 0;
pr_debug("%s(regs = %p tsk = %p)\n", __func__, regs, tsk);
if (!tsk)
tsk = current;
if (tsk == current) {
frame.fp = (unsigned long)__builtin_frame_address(0);
frame.sp = current_stack_pointer;

12
arch/arm64/kernel/vdso.c
View File

@ -55,7 +55,7 @@ struct vdso_data *vdso_data = &vdso_data_store.data;
*/
static struct page *vectors_page[1];
static int alloc_vectors_page(void)
static int __init alloc_vectors_page(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
@ -88,7 +88,7 @@ int aarch32_setup_vectors_page(struct linux_binprm *bprm, int uses_interp)
{
struct mm_struct *mm = current->mm;
unsigned long addr = AARCH32_VECTORS_BASE;
static struct vm_special_mapping spec = {
static const struct vm_special_mapping spec = {
.name = "[vectors]",
.pages = vectors_page,
@ -212,10 +212,16 @@ void update_vsyscall(struct timekeeper *tk)
vdso_data->wtm_clock_nsec = tk->wall_to_monotonic.tv_nsec;
if (!use_syscall) {
/* tkr_mono.cycle_last == tkr_raw.cycle_last */
vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last;
vdso_data->raw_time_sec = tk->raw_time.tv_sec;
vdso_data->raw_time_nsec = tk->raw_time.tv_nsec;
vdso_data->xtime_clock_sec = tk->xtime_sec;
vdso_data->xtime_clock_nsec = tk->tkr_mono.xtime_nsec;
vdso_data->cs_mult = tk->tkr_mono.mult;
/* tkr_raw.xtime_nsec == 0 */
vdso_data->cs_mono_mult = tk->tkr_mono.mult;
vdso_data->cs_raw_mult = tk->tkr_raw.mult;
/* tkr_mono.shift == tkr_raw.shift */
vdso_data->cs_shift = tk->tkr_mono.shift;
}

7
arch/arm64/kernel/vdso/Makefile
View File

@ -23,7 +23,7 @@ GCOV_PROFILE := n
ccflags-y += -Wl,-shared
obj-y += vdso.o
extra-y += vdso.lds vdso-offsets.h
extra-y += vdso.lds
CPPFLAGS_vdso.lds += -P -C -U$(ARCH)
# Force dependency (incbin is bad)
@ -42,11 +42,10 @@ $(obj)/%.so: $(obj)/%.so.dbg FORCE
gen-vdsosym := $(srctree)/$(src)/gen_vdso_offsets.sh
quiet_cmd_vdsosym = VDSOSYM $@
define cmd_vdsosym
$(NM) $< | $(gen-vdsosym) | LC_ALL=C sort > $@ && \
cp $@ include/generated/
$(NM) $< | $(gen-vdsosym) | LC_ALL=C sort > $@
endef
$(obj)/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE
include/generated/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE
$(call if_changed,vdsosym)
# Assembly rules for the .S files

331
arch/arm64/kernel/vdso/gettimeofday.S
View File

@ -26,24 +26,109 @@
#define NSEC_PER_SEC_HI16 0x3b9a
vdso_data .req x6
use_syscall .req w7
seqcnt .req w8
seqcnt .req w7
w_tmp .req w8
x_tmp .req x8
/*
* Conventions for macro arguments:
* - An argument is write-only if its name starts with "res".
* - All other arguments are read-only, unless otherwise specified.
*/
.macro seqcnt_acquire
9999: ldr seqcnt, [vdso_data, #VDSO_TB_SEQ_COUNT]
tbnz seqcnt, #0, 9999b
dmb ishld
ldr use_syscall, [vdso_data, #VDSO_USE_SYSCALL]
.endm
.macro seqcnt_read, cnt
.macro seqcnt_check fail
dmb ishld
ldr \cnt, [vdso_data, #VDSO_TB_SEQ_COUNT]
ldr w_tmp, [vdso_data, #VDSO_TB_SEQ_COUNT]
cmp w_tmp, seqcnt
b.ne \fail
.endm
.macro seqcnt_check, cnt, fail
cmp \cnt, seqcnt
b.ne \fail
.macro syscall_check fail
ldr w_tmp, [vdso_data, #VDSO_USE_SYSCALL]
cbnz w_tmp, \fail
.endm
.macro get_nsec_per_sec res
mov \res, #NSEC_PER_SEC_LO16
movk \res, #NSEC_PER_SEC_HI16, lsl #16
.endm
/*
* Returns the clock delta, in nanoseconds left-shifted by the clock
* shift.
*/
.macro get_clock_shifted_nsec res, cycle_last, mult
/* Read the virtual counter. */
isb
mrs x_tmp, cntvct_el0
/* Calculate cycle delta and convert to ns. */
sub \res, x_tmp, \cycle_last
/* We can only guarantee 56 bits of precision. */
movn x_tmp, #0xff00, lsl #48
and \res, x_tmp, \res
mul \res, \res, \mult
.endm
/*
* Returns in res_{sec,nsec} the REALTIME timespec, based on the
* "wall time" (xtime) and the clock_mono delta.
*/
.macro get_ts_realtime res_sec, res_nsec, \
clock_nsec, xtime_sec, xtime_nsec, nsec_to_sec
add \res_nsec, \clock_nsec, \xtime_nsec
udiv x_tmp, \res_nsec, \nsec_to_sec
add \res_sec, \xtime_sec, x_tmp
msub \res_nsec, x_tmp, \nsec_to_sec, \res_nsec
.endm
/*
* Returns in res_{sec,nsec} the timespec based on the clock_raw delta,
* used for CLOCK_MONOTONIC_RAW.
*/
.macro get_ts_clock_raw res_sec, res_nsec, clock_nsec, nsec_to_sec
udiv \res_sec, \clock_nsec, \nsec_to_sec
msub \res_nsec, \res_sec, \nsec_to_sec, \clock_nsec
.endm
/* sec and nsec are modified in place. */
.macro add_ts sec, nsec, ts_sec, ts_nsec, nsec_to_sec
/* Add timespec. */
add \sec, \sec, \ts_sec
add \nsec, \nsec, \ts_nsec
/* Normalise the new timespec. */
cmp \nsec, \nsec_to_sec
b.lt 9999f
sub \nsec, \nsec, \nsec_to_sec
add \sec, \sec, #1
9999:
cmp \nsec, #0
b.ge 9998f
add \nsec, \nsec, \nsec_to_sec
sub \sec, \sec, #1
9998:
.endm
.macro clock_gettime_return, shift=0
.if \shift == 1
lsr x11, x11, x12
.endif
stp x10, x11, [x1, #TSPEC_TV_SEC]
mov x0, xzr
ret
.endm
.macro jump_slot jumptable, index, label
.if (. - \jumptable) != 4 * (\index)
.error "Jump slot index mismatch"
.endif
b \label
.endm
.text
@ -51,18 +136,25 @@ seqcnt .req w8
/* int __kernel_gettimeofday(struct timeval *tv, struct timezone *tz); */
ENTRY(__kernel_gettimeofday)
.cfi_startproc
mov x2, x30
.cfi_register x30, x2
/* Acquire the sequence counter and get the timespec. */
adr vdso_data, _vdso_data
1: seqcnt_acquire
cbnz use_syscall, 4f
/* If tv is NULL, skip to the timezone code. */
cbz x0, 2f
bl __do_get_tspec
seqcnt_check w9, 1b
/* Compute the time of day. */
1: seqcnt_acquire
syscall_check fail=4f
ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
/* w11 = cs_mono_mult, w12 = cs_shift */
ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
seqcnt_check fail=1b
get_nsec_per_sec res=x9
lsl x9, x9, x12
get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
get_ts_realtime res_sec=x10, res_nsec=x11, \
clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
/* Convert ns to us. */
mov x13, #1000
@ -76,95 +168,126 @@ ENTRY(__kernel_gettimeofday)
stp w4, w5, [x1, #TZ_MINWEST]
3:
mov x0, xzr
ret x2
ret
4:
/* Syscall fallback. */
mov x8, #__NR_gettimeofday
svc #0
ret x2
ret
.cfi_endproc
ENDPROC(__kernel_gettimeofday)
#define JUMPSLOT_MAX CLOCK_MONOTONIC_COARSE
/* int __kernel_clock_gettime(clockid_t clock_id, struct timespec *tp); */
ENTRY(__kernel_clock_gettime)
.cfi_startproc
cmp w0, #CLOCK_REALTIME
ccmp w0, #CLOCK_MONOTONIC, #0x4, ne
b.ne 2f
mov x2, x30
.cfi_register x30, x2
/* Get kernel timespec. */
cmp w0, #JUMPSLOT_MAX
b.hi syscall
adr vdso_data, _vdso_data
1: seqcnt_acquire
cbnz use_syscall, 7f
adr x_tmp, jumptable
add x_tmp, x_tmp, w0, uxtw #2
br x_tmp
bl __do_get_tspec
seqcnt_check w9, 1b
ALIGN
jumptable:
jump_slot jumptable, CLOCK_REALTIME, realtime
jump_slot jumptable, CLOCK_MONOTONIC, monotonic
b syscall
b syscall
jump_slot jumptable, CLOCK_MONOTONIC_RAW, monotonic_raw
jump_slot jumptable, CLOCK_REALTIME_COARSE, realtime_coarse
jump_slot jumptable, CLOCK_MONOTONIC_COARSE, monotonic_coarse
mov x30, x2
.if (. - jumptable) != 4 * (JUMPSLOT_MAX + 1)
.error "Wrong jumptable size"
.endif
cmp w0, #CLOCK_MONOTONIC
b.ne 6f
ALIGN
realtime:
seqcnt_acquire
syscall_check fail=syscall
ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
/* w11 = cs_mono_mult, w12 = cs_shift */
ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
seqcnt_check fail=realtime
/* Get wtm timespec. */
ldp x13, x14, [vdso_data, #VDSO_WTM_CLK_SEC]
/* All computations are done with left-shifted nsecs. */
get_nsec_per_sec res=x9
lsl x9, x9, x12
/* Check the sequence counter. */
seqcnt_read w9
seqcnt_check w9, 1b
b 4f
2:
cmp w0, #CLOCK_REALTIME_COARSE
ccmp w0, #CLOCK_MONOTONIC_COARSE, #0x4, ne
b.ne 8f
get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
get_ts_realtime res_sec=x10, res_nsec=x11, \
clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
clock_gettime_return, shift=1
/* xtime_coarse_nsec is already right-shifted */
mov x12, #0
ALIGN
monotonic:
seqcnt_acquire
syscall_check fail=syscall
ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
/* w11 = cs_mono_mult, w12 = cs_shift */
ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
ldp x3, x4, [vdso_data, #VDSO_WTM_CLK_SEC]
seqcnt_check fail=monotonic
/* Get coarse timespec. */
adr vdso_data, _vdso_data
3: seqcnt_acquire
ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
/* All computations are done with left-shifted nsecs. */
lsl x4, x4, x12
get_nsec_per_sec res=x9
lsl x9, x9, x12
/* Get wtm timespec. */
ldp x13, x14, [vdso_data, #VDSO_WTM_CLK_SEC]
get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
get_ts_realtime res_sec=x10, res_nsec=x11, \
clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
/* Check the sequence counter. */
seqcnt_read w9
seqcnt_check w9, 3b
add_ts sec=x10, nsec=x11, ts_sec=x3, ts_nsec=x4, nsec_to_sec=x9
clock_gettime_return, shift=1
cmp w0, #CLOCK_MONOTONIC_COARSE
b.ne 6f
4:
/* Add on wtm timespec. */
add x10, x10, x13
ALIGN
monotonic_raw:
seqcnt_acquire
syscall_check fail=syscall
ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
/* w11 = cs_raw_mult, w12 = cs_shift */
ldp w12, w11, [vdso_data, #VDSO_CS_SHIFT]
ldp x13, x14, [vdso_data, #VDSO_RAW_TIME_SEC]
seqcnt_check fail=monotonic_raw
/* All computations are done with left-shifted nsecs. */
lsl x14, x14, x12
add x11, x11, x14
get_nsec_per_sec res=x9
lsl x9, x9, x12
/* Normalise the new timespec. */
mov x15, #NSEC_PER_SEC_LO16
movk x15, #NSEC_PER_SEC_HI16, lsl #16
lsl x15, x15, x12
cmp x11, x15
b.lt 5f
sub x11, x11, x15
add x10, x10, #1
5:
cmp x11, #0
b.ge 6f
add x11, x11, x15
sub x10, x10, #1
get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
get_ts_clock_raw res_sec=x10, res_nsec=x11, \
clock_nsec=x15, nsec_to_sec=x9
6: /* Store to the user timespec. */
lsr x11, x11, x12
stp x10, x11, [x1, #TSPEC_TV_SEC]
mov x0, xzr
ret
7:
mov x30, x2
8: /* Syscall fallback. */
add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
clock_gettime_return, shift=1
ALIGN
realtime_coarse:
seqcnt_acquire
ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
seqcnt_check fail=realtime_coarse
clock_gettime_return
ALIGN
monotonic_coarse:
seqcnt_acquire
ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
ldp x13, x14, [vdso_data, #VDSO_WTM_CLK_SEC]
seqcnt_check fail=monotonic_coarse
/* Computations are done in (non-shifted) nsecs. */
get_nsec_per_sec res=x9
add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
clock_gettime_return
ALIGN
syscall: /* Syscall fallback. */
mov x8, #__NR_clock_gettime
svc #0
ret
@ -176,6 +299,7 @@ ENTRY(__kernel_clock_getres)
.cfi_startproc
cmp w0, #CLOCK_REALTIME
ccmp w0, #CLOCK_MONOTONIC, #0x4, ne
ccmp w0, #CLOCK_MONOTONIC_RAW, #0x4, ne
b.ne 1f
ldr x2, 5f
@ -203,46 +327,3 @@ ENTRY(__kernel_clock_getres)
.quad CLOCK_COARSE_RES
.cfi_endproc
ENDPROC(__kernel_clock_getres)
/*
* Read the current time from the architected counter.
* Expects vdso_data to be initialised.
* Clobbers the temporary registers (x9 - x15).
* Returns:
* - w9 = vDSO sequence counter
* - (x10, x11) = (ts->tv_sec, shifted ts->tv_nsec)
* - w12 = cs_shift
*/
ENTRY(__do_get_tspec)
.cfi_startproc
/* Read from the vDSO data page. */
ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
ldp w11, w12, [vdso_data, #VDSO_CS_MULT]
seqcnt_read w9
/* Read the virtual counter. */
isb
mrs x15, cntvct_el0
/* Calculate cycle delta and convert to ns. */
sub x10, x15, x10
/* We can only guarantee 56 bits of precision. */
movn x15, #0xff00, lsl #48
and x10, x15, x10
mul x10, x10, x11
/* Use the kernel time to calculate the new timespec. */
mov x11, #NSEC_PER_SEC_LO16
movk x11, #NSEC_PER_SEC_HI16, lsl #16
lsl x11, x11, x12
add x15, x10, x14
udiv x14, x15, x11
add x10, x13, x14
mul x13, x14, x11
sub x11, x15, x13
ret
.cfi_endproc
ENDPROC(__do_get_tspec)

22
arch/arm64/mm/hugetlbpage.c
View File

@ -51,20 +51,8 @@ static int find_num_contig(struct mm_struct *mm, unsigned long addr,
*pgsize = PAGE_SIZE;
if (!pte_cont(pte))
return 1;
if (!pgd_present(*pgd)) {
VM_BUG_ON(!pgd_present(*pgd));
return 1;
}
pud = pud_offset(pgd, addr);
if (!pud_present(*pud)) {
VM_BUG_ON(!pud_present(*pud));
return 1;
}
pmd = pmd_offset(pud, addr);
if (!pmd_present(*pmd)) {
VM_BUG_ON(!pmd_present(*pmd));
return 1;
}
if ((pte_t *)pmd == ptep) {
*pgsize = PMD_SIZE;
return CONT_PMDS;
@ -212,7 +200,7 @@ pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
ncontig = find_num_contig(mm, addr, cpte, *cpte, &pgsize);
/* save the 1st pte to return */
pte = ptep_get_and_clear(mm, addr, cpte);
for (i = 1; i < ncontig; ++i) {
for (i = 1, addr += pgsize; i < ncontig; ++i, addr += pgsize) {
/*
* If HW_AFDBM is enabled, then the HW could
* turn on the dirty bit for any of the page
@ -250,8 +238,8 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma,
pfn = pte_pfn(*cpte);
ncontig = find_num_contig(vma->vm_mm, addr, cpte,
*cpte, &pgsize);
for (i = 0; i < ncontig; ++i, ++cpte) {
changed = ptep_set_access_flags(vma, addr, cpte,
for (i = 0; i < ncontig; ++i, ++cpte, addr += pgsize) {
changed |= ptep_set_access_flags(vma, addr, cpte,
pfn_pte(pfn,
hugeprot),
dirty);
@ -273,7 +261,7 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm,
cpte = huge_pte_offset(mm, addr);
ncontig = find_num_contig(mm, addr, cpte, *cpte, &pgsize);
for (i = 0; i < ncontig; ++i, ++cpte)
for (i = 0; i < ncontig; ++i, ++cpte, addr += pgsize)
ptep_set_wrprotect(mm, addr, cpte);
} else {
ptep_set_wrprotect(mm, addr, ptep);
@ -291,7 +279,7 @@ void huge_ptep_clear_flush(struct vm_area_struct *vma,
cpte = huge_pte_offset(vma->vm_mm, addr);
ncontig = find_num_contig(vma->vm_mm, addr, cpte,
*cpte, &pgsize);
for (i = 0; i < ncontig; ++i, ++cpte)
for (i = 0; i < ncontig; ++i, ++cpte, addr += pgsize)
ptep_clear_flush(vma, addr, cpte);
} else {
ptep_clear_flush(vma, addr, ptep);

9
arch/arm64/mm/proc.S
View File

@ -100,7 +100,16 @@ ENTRY(cpu_do_resume)
msr tcr_el1, x8
msr vbar_el1, x9
/*
* __cpu_setup() cleared MDSCR_EL1.MDE and friends, before unmasking
* debug exceptions. By restoring MDSCR_EL1 here, we may take a debug
* exception. Mask them until local_dbg_restore() in cpu_suspend()
* resets them.
*/
disable_dbg
msr mdscr_el1, x10
msr sctlr_el1, x12
/*
* Restore oslsr_el1 by writing oslar_el1

41
arch/c6x/kernel/ptrace.c
View File

@ -69,46 +69,6 @@ static int gpr_get(struct task_struct *target,
0, sizeof(*regs));
}
static int gpr_set(struct task_struct *target,
const struct user_regset *regset,
unsigned int pos, unsigned int count,
const void *kbuf, const void __user *ubuf)
{
int ret;
struct pt_regs *regs = task_pt_regs(target);
/* Don't copyin TSR or CSR */
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
&regs,
0, PT_TSR * sizeof(long));
if (ret)
return ret;
ret = user_regset_copyin_ignore(&pos, &count, &kbuf, &ubuf,
PT_TSR * sizeof(long),
(PT_TSR + 1) * sizeof(long));
if (ret)
return ret;
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
&regs,
(PT_TSR + 1) * sizeof(long),
PT_CSR * sizeof(long));
if (ret)
return ret;
ret = user_regset_copyin_ignore(&pos, &count, &kbuf, &ubuf,
PT_CSR * sizeof(long),
(PT_CSR + 1) * sizeof(long));
if (ret)
return ret;
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
&regs,
(PT_CSR + 1) * sizeof(long), -1);
return ret;
}
enum c6x_regset {
REGSET_GPR,
};
@ -120,7 +80,6 @@ static const struct user_regset c6x_regsets[] = {
.size = sizeof(u32),
.align = sizeof(u32),
.get = gpr_get,
.set = gpr_set
},
};

8
arch/h8300/kernel/ptrace.c
View File

@ -95,7 +95,8 @@ static int regs_get(struct task_struct *target,
long *reg = (long *)&regs;
/* build user regs in buffer */
for (r = 0; r < ARRAY_SIZE(register_offset); r++)
BUILD_BUG_ON(sizeof(regs) % sizeof(long) != 0);
for (r = 0; r < sizeof(regs) / sizeof(long); r++)
*reg++ = h8300_get_reg(target, r);
return user_regset_copyout(&pos, &count, &kbuf, &ubuf,
@ -113,7 +114,8 @@ static int regs_set(struct task_struct *target,
long *reg;
/* build user regs in buffer */
for (reg = (long *)&regs, r = 0; r < ARRAY_SIZE(register_offset); r++)
BUILD_BUG_ON(sizeof(regs) % sizeof(long) != 0);
for (reg = (long *)&regs, r = 0; r < sizeof(regs) / sizeof(long); r++)
*reg++ = h8300_get_reg(target, r);
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
@ -122,7 +124,7 @@ static int regs_set(struct task_struct *target,
return ret;
/* write back to pt_regs */
for (reg = (long *)&regs, r = 0; r < ARRAY_SIZE(register_offset); r++)
for (reg = (long *)&regs, r = 0; r < sizeof(regs) / sizeof(long); r++)
h8300_put_reg(target, r, *reg++);
return 0;
}

4
arch/ia64/Makefile
View File

@ -95,8 +95,8 @@ define archhelp
echo '* unwcheck - Check vmlinux for invalid unwind info'
endef
archprepare: make_nr_irqs_h FORCE
archprepare: make_nr_irqs_h
PHONY += make_nr_irqs_h FORCE
make_nr_irqs_h: FORCE
make_nr_irqs_h:
$(Q)$(MAKE) $(build)=arch/ia64/kernel include/generated/nr-irqs.h

15
arch/metag/include/asm/uaccess.h
View File

@ -197,20 +197,21 @@ extern long __must_check strnlen_user(const char __user *src, long count);
#define strlen_user(str) strnlen_user(str, 32767)
extern unsigned long __must_check __copy_user_zeroing(void *to,
const void __user *from,
unsigned long n);
extern unsigned long raw_copy_from_user(void *to, const void __user *from,
unsigned long n);
static inline unsigned long
copy_from_user(void *to, const void __user *from, unsigned long n)
{
unsigned long res = n;
if (likely(access_ok(VERIFY_READ, from, n)))
return __copy_user_zeroing(to, from, n);
memset(to, 0, n);
return n;
res = raw_copy_from_user(to, from, n);
if (unlikely(res))
memset(to + (n - res), 0, res);
return res;
}
#define __copy_from_user(to, from, n) __copy_user_zeroing(to, from, n)
#define __copy_from_user(to, from, n) raw_copy_from_user(to, from, n)
#define __copy_from_user_inatomic __copy_from_user
extern unsigned long __must_check __copy_user(void __user *to,

19
arch/metag/kernel/ptrace.c
View File

@ -24,6 +24,16 @@
* user_regset definitions.
*/
static unsigned long user_txstatus(const struct pt_regs *regs)
{
unsigned long data = (unsigned long)regs->ctx.Flags;
if (regs->ctx.SaveMask & TBICTX_CBUF_BIT)
data |= USER_GP_REGS_STATUS_CATCH_BIT;
return data;
}
int metag_gp_regs_copyout(const struct pt_regs *regs,
unsigned int pos, unsigned int count,
void *kbuf, void __user *ubuf)
@ -62,9 +72,7 @@ int metag_gp_regs_copyout(const struct pt_regs *regs,
if (ret)
goto out;
/* TXSTATUS */
data = (unsigned long)regs->ctx.Flags;
if (regs->ctx.SaveMask & TBICTX_CBUF_BIT)
data |= USER_GP_REGS_STATUS_CATCH_BIT;
data = user_txstatus(regs);
ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf,
&data, 4*25, 4*26);
if (ret)
@ -119,6 +127,7 @@ int metag_gp_regs_copyin(struct pt_regs *regs,
if (ret)
goto out;
/* TXSTATUS */
data = user_txstatus(regs);
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
&data, 4*25, 4*26);
if (ret)
@ -244,6 +253,8 @@ int metag_rp_state_copyin(struct pt_regs *regs,
unsigned long long *ptr;
int ret, i;
if (count < 4*13)
return -EINVAL;
/* Read the entire pipeline before making any changes */
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
&rp, 0, 4*13);
@ -303,7 +314,7 @@ static int metag_tls_set(struct task_struct *target,
const void *kbuf, const void __user *ubuf)
{
int ret;
void __user *tls;
void __user *tls = target->thread.tls_ptr;
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &tls, 0, -1);
if (ret)

324
arch/metag/lib/usercopy.c
View File

@ -29,7 +29,6 @@
COPY \
"1:\n" \
" .section .fixup,\"ax\"\n" \
" MOV D1Ar1,#0\n" \
FIXUP \
" MOVT D1Ar1,#HI(1b)\n" \
" JUMP D1Ar1,#LO(1b)\n" \
@ -260,27 +259,31 @@
"MGETL D0FrT, D0.5, D0.6, D0.7, [%1++]\n" \
"22:\n" \
"MSETL [%0++], D0FrT, D0.5, D0.6, D0.7\n" \
"SUB %3, %3, #32\n" \
"23:\n" \
"MGETL D0FrT, D0.5, D0.6, D0.7, [%1++]\n" \
"SUB %3, %3, #32\n" \
"24:\n" \
"MGETL D0FrT, D0.5, D0.6, D0.7, [%1++]\n" \
"25:\n" \
"MSETL [%0++], D0FrT, D0.5, D0.6, D0.7\n" \
"26:\n" \
"SUB %3, %3, #32\n" \
"DCACHE [%1+#-64], D0Ar6\n" \
"BR $Lloop"id"\n" \
\
"MOV RAPF, %1\n" \
"25:\n" \
"MGETL D0FrT, D0.5, D0.6, D0.7, [%1++]\n" \
"26:\n" \
"MSETL [%0++], D0FrT, D0.5, D0.6, D0.7\n" \
"SUB %3, %3, #32\n" \
"27:\n" \
"MGETL D0FrT, D0.5, D0.6, D0.7, [%1++]\n" \
"28:\n" \
"MSETL [%0++], D0FrT, D0.5, D0.6, D0.7\n" \
"SUB %0, %0, #8\n" \
"29:\n" \
"SUB %3, %3, #32\n" \
"30:\n" \
"MGETL D0FrT, D0.5, D0.6, D0.7, [%1++]\n" \
"31:\n" \
"MSETL [%0++], D0FrT, D0.5, D0.6, D0.7\n" \
"32:\n" \
"SUB %0, %0, #8\n" \
"33:\n" \
"SETL [%0++], D0.7, D1.7\n" \
"SUB %3, %3, #32\n" \
"1:" \
@ -312,11 +315,15 @@
" .long 26b,3b\n" \
" .long 27b,3b\n" \
" .long 28b,3b\n" \
" .long 29b,4b\n" \
" .long 29b,3b\n" \
" .long 30b,3b\n" \
" .long 31b,3b\n" \
" .long 32b,3b\n" \
" .long 33b,4b\n" \
" .previous\n" \
: "=r" (to), "=r" (from), "=r" (ret), "=d" (n) \
: "0" (to), "1" (from), "2" (ret), "3" (n) \
: "D1Ar1", "D0Ar2", "memory")
: "D1Ar1", "D0Ar2", "cc", "memory")
/* rewind 'to' and 'from' pointers when a fault occurs
*
@ -342,7 +349,7 @@
#define __asm_copy_to_user_64bit_rapf_loop(to, from, ret, n, id)\
__asm_copy_user_64bit_rapf_loop(to, from, ret, n, id, \
"LSR D0Ar2, D0Ar2, #8\n" \
"AND D0Ar2, D0Ar2, #0x7\n" \
"ANDS D0Ar2, D0Ar2, #0x7\n" \
"ADDZ D0Ar2, D0Ar2, #4\n" \
"SUB D0Ar2, D0Ar2, #1\n" \
"MOV D1Ar1, #4\n" \
@ -403,47 +410,55 @@
"MGETD D0FrT, D0.5, D0.6, D0.7, [%1++]\n" \
"22:\n" \
"MSETD [%0++], D0FrT, D0.5, D0.6, D0.7\n" \
"SUB %3, %3, #16\n" \
"23:\n" \
"MGETD D0FrT, D0.5, D0.6, D0.7, [%1++]\n" \
"24:\n" \
"MSETD [%0++], D0FrT, D0.5, D0.6, D0.7\n" \
"SUB %3, %3, #16\n" \
"25:\n" \
"24:\n" \
"MGETD D0FrT, D0.5, D0.6, D0.7, [%1++]\n" \
"26:\n" \
"25:\n" \
"MSETD [%0++], D0FrT, D0.5, D0.6, D0.7\n" \
"26:\n" \
"SUB %3, %3, #16\n" \
"27:\n" \
"MGETD D0FrT, D0.5, D0.6, D0.7, [%1++]\n" \
"28:\n" \
"MSETD [%0++], D0FrT, D0.5, D0.6, D0.7\n" \
"29:\n" \
"SUB %3, %3, #16\n" \
"30:\n" \
"MGETD D0FrT, D0.5, D0.6, D0.7, [%1++]\n" \
"31:\n" \
"MSETD [%0++], D0FrT, D0.5, D0.6, D0.7\n" \
"32:\n" \
"SUB %3, %3, #16\n" \
"DCACHE [%1+#-64], D0Ar6\n" \
"BR $Lloop"id"\n" \
\
"MOV RAPF, %1\n" \
"29:\n" \
"MGETD D0FrT, D0.5, D0.6, D0.7, [%1++]\n" \
"30:\n" \
"MSETD [%0++], D0FrT, D0.5, D0.6, D0.7\n" \
"SUB %3, %3, #16\n" \
"31:\n" \
"MGETD D0FrT, D0.5, D0.6, D0.7, [%1++]\n" \
"32:\n" \
"MSETD [%0++], D0FrT, D0.5, D0.6, D0.7\n" \
"SUB %3, %3, #16\n" \
"33:\n" \
"MGETD D0FrT, D0.5, D0.6, D0.7, [%1++]\n" \
"34:\n" \
"MSETD [%0++], D0FrT, D0.5, D0.6, D0.7\n" \
"SUB %3, %3, #16\n" \
"35:\n" \
"MGETD D0FrT, D0.5, D0.6, D0.7, [%1++]\n" \
"SUB %3, %3, #16\n" \
"36:\n" \
"MSETD [%0++], D0FrT, D0.5, D0.6, D0.7\n" \
"SUB %0, %0, #4\n" \
"MGETD D0FrT, D0.5, D0.6, D0.7, [%1++]\n" \
"37:\n" \
"MSETD [%0++], D0FrT, D0.5, D0.6, D0.7\n" \
"38:\n" \
"SUB %3, %3, #16\n" \
"39:\n" \
"MGETD D0FrT, D0.5, D0.6, D0.7, [%1++]\n" \
"40:\n" \
"MSETD [%0++], D0FrT, D0.5, D0.6, D0.7\n" \
"41:\n" \
"SUB %3, %3, #16\n" \
"42:\n" \
"MGETD D0FrT, D0.5, D0.6, D0.7, [%1++]\n" \
"43:\n" \
"MSETD [%0++], D0FrT, D0.5, D0.6, D0.7\n" \
"44:\n" \
"SUB %0, %0, #4\n" \
"45:\n" \
"SETD [%0++], D0.7\n" \
"SUB %3, %3, #16\n" \
"1:" \
@ -483,11 +498,19 @@
" .long 34b,3b\n" \
" .long 35b,3b\n" \
" .long 36b,3b\n" \
" .long 37b,4b\n" \
" .long 37b,3b\n" \
" .long 38b,3b\n" \
" .long 39b,3b\n" \
" .long 40b,3b\n" \
" .long 41b,3b\n" \
" .long 42b,3b\n" \
" .long 43b,3b\n" \
" .long 44b,3b\n" \
" .long 45b,4b\n" \
" .previous\n" \
: "=r" (to), "=r" (from), "=r" (ret), "=d" (n) \
: "0" (to), "1" (from), "2" (ret), "3" (n) \
: "D1Ar1", "D0Ar2", "memory")
: "D1Ar1", "D0Ar2", "cc", "memory")
/* rewind 'to' and 'from' pointers when a fault occurs
*
@ -513,7 +536,7 @@
#define __asm_copy_to_user_32bit_rapf_loop(to, from, ret, n, id)\
__asm_copy_user_32bit_rapf_loop(to, from, ret, n, id, \
"LSR D0Ar2, D0Ar2, #8\n" \
"AND D0Ar2, D0Ar2, #0x7\n" \
"ANDS D0Ar2, D0Ar2, #0x7\n" \
"ADDZ D0Ar2, D0Ar2, #4\n" \
"SUB D0Ar2, D0Ar2, #1\n" \
"MOV D1Ar1, #4\n" \
@ -538,23 +561,31 @@ unsigned long __copy_user(void __user *pdst, const void *psrc,
if ((unsigned long) src & 1) {
__asm_copy_to_user_1(dst, src, retn);
n--;
if (retn)
return retn + n;
}
if ((unsigned long) dst & 1) {
/* Worst case - byte copy */
while (n > 0) {
__asm_copy_to_user_1(dst, src, retn);
n--;
if (retn)
return retn + n;
}
}
if (((unsigned long) src & 2) && n >= 2) {
__asm_copy_to_user_2(dst, src, retn);
n -= 2;
if (retn)
return retn + n;
}
if ((unsigned long) dst & 2) {
/* Second worst case - word copy */
while (n >= 2) {
__asm_copy_to_user_2(dst, src, retn);
n -= 2;
if (retn)
return retn + n;
}
}
@ -569,6 +600,8 @@ unsigned long __copy_user(void __user *pdst, const void *psrc,
while (n >= 8) {
__asm_copy_to_user_8x64(dst, src, retn);
n -= 8;
if (retn)
return retn + n;
}
}
if (n >= RAPF_MIN_BUF_SIZE) {
@ -581,6 +614,8 @@ unsigned long __copy_user(void __user *pdst, const void *psrc,
while (n >= 8) {
__asm_copy_to_user_8x64(dst, src, retn);
n -= 8;
if (retn)
return retn + n;
}
}
#endif
@ -588,11 +623,15 @@ unsigned long __copy_user(void __user *pdst, const void *psrc,
while (n >= 16) {
__asm_copy_to_user_16(dst, src, retn);
n -= 16;
if (retn)
return retn + n;
}
while (n >= 4) {
__asm_copy_to_user_4(dst, src, retn);
n -= 4;
if (retn)
return retn + n;
}
switch (n) {
@ -609,6 +648,10 @@ unsigned long __copy_user(void __user *pdst, const void *psrc,
break;
}
/*
* If we get here, retn correctly reflects the number of failing
* bytes.
*/
return retn;
}
EXPORT_SYMBOL(__copy_user);
@ -617,16 +660,14 @@ EXPORT_SYMBOL(__copy_user);
__asm_copy_user_cont(to, from, ret, \
" GETB D1Ar1,[%1++]\n" \
"2: SETB [%0++],D1Ar1\n", \
"3: ADD %2,%2,#1\n" \
" SETB [%0++],D1Ar1\n", \
"3: ADD %2,%2,#1\n", \
" .long 2b,3b\n")
#define __asm_copy_from_user_2x_cont(to, from, ret, COPY, FIXUP, TENTRY) \
__asm_copy_user_cont(to, from, ret, \
" GETW D1Ar1,[%1++]\n" \
"2: SETW [%0++],D1Ar1\n" COPY, \
"3: ADD %2,%2,#2\n" \
" SETW [%0++],D1Ar1\n" FIXUP, \
"3: ADD %2,%2,#2\n" FIXUP, \
" .long 2b,3b\n" TENTRY)
#define __asm_copy_from_user_2(to, from, ret) \
@ -636,145 +677,26 @@ EXPORT_SYMBOL(__copy_user);
__asm_copy_from_user_2x_cont(to, from, ret, \
" GETB D1Ar1,[%1++]\n" \
"4: SETB [%0++],D1Ar1\n", \
"5: ADD %2,%2,#1\n" \
" SETB [%0++],D1Ar1\n", \
"5: ADD %2,%2,#1\n", \
" .long 4b,5b\n")
#define __asm_copy_from_user_4x_cont(to, from, ret, COPY, FIXUP, TENTRY) \
__asm_copy_user_cont(to, from, ret, \
" GETD D1Ar1,[%1++]\n" \
"2: SETD [%0++],D1Ar1\n" COPY, \
"3: ADD %2,%2,#4\n" \
" SETD [%0++],D1Ar1\n" FIXUP, \
"3: ADD %2,%2,#4\n" FIXUP, \
" .long 2b,3b\n" TENTRY)
#define __asm_copy_from_user_4(to, from, ret) \
__asm_copy_from_user_4x_cont(to, from, ret, "", "", "")
#define __asm_copy_from_user_5(to, from, ret) \
__asm_copy_from_user_4x_cont(to, from, ret, \
" GETB D1Ar1,[%1++]\n" \
"4: SETB [%0++],D1Ar1\n", \
"5: ADD %2,%2,#1\n" \
" SETB [%0++],D1Ar1\n", \
" .long 4b,5b\n")
#define __asm_copy_from_user_6x_cont(to, from, ret, COPY, FIXUP, TENTRY) \
__asm_copy_from_user_4x_cont(to, from, ret, \
" GETW D1Ar1,[%1++]\n" \
"4: SETW [%0++],D1Ar1\n" COPY, \
"5: ADD %2,%2,#2\n" \
" SETW [%0++],D1Ar1\n" FIXUP, \
" .long 4b,5b\n" TENTRY)
#define __asm_copy_from_user_6(to, from, ret) \
__asm_copy_from_user_6x_cont(to, from, ret, "", "", "")
#define __asm_copy_from_user_7(to, from, ret) \
__asm_copy_from_user_6x_cont(to, from, ret, \
" GETB D1Ar1,[%1++]\n" \
"6: SETB [%0++],D1Ar1\n", \
"7: ADD %2,%2,#1\n" \
" SETB [%0++],D1Ar1\n", \
" .long 6b,7b\n")
#define __asm_copy_from_user_8x_cont(to, from, ret, COPY, FIXUP, TENTRY) \
__asm_copy_from_user_4x_cont(to, from, ret, \
" GETD D1Ar1,[%1++]\n" \
"4: SETD [%0++],D1Ar1\n" COPY, \
"5: ADD %2,%2,#4\n" \
" SETD [%0++],D1Ar1\n" FIXUP, \
" .long 4b,5b\n" TENTRY)
#define __asm_copy_from_user_8(to, from, ret) \
__asm_copy_from_user_8x_cont(to, from, ret, "", "", "")
#define __asm_copy_from_user_9(to, from, ret) \
__asm_copy_from_user_8x_cont(to, from, ret, \
" GETB D1Ar1,[%1++]\n" \
"6: SETB [%0++],D1Ar1\n", \
"7: ADD %2,%2,#1\n" \
" SETB [%0++],D1Ar1\n", \
" .long 6b,7b\n")
#define __asm_copy_from_user_10x_cont(to, from, ret, COPY, FIXUP, TENTRY) \
__asm_copy_from_user_8x_cont(to, from, ret, \
" GETW D1Ar1,[%1++]\n" \
"6: SETW [%0++],D1Ar1\n" COPY, \
"7: ADD %2,%2,#2\n" \
" SETW [%0++],D1Ar1\n" FIXUP, \
" .long 6b,7b\n" TENTRY)
#define __asm_copy_from_user_10(to, from, ret) \
__asm_copy_from_user_10x_cont(to, from, ret, "", "", "")
#define __asm_copy_from_user_11(to, from, ret) \
__asm_copy_from_user_10x_cont(to, from, ret, \
" GETB D1Ar1,[%1++]\n" \
"8: SETB [%0++],D1Ar1\n", \
"9: ADD %2,%2,#1\n" \
" SETB [%0++],D1Ar1\n", \
" .long 8b,9b\n")
#define __asm_copy_from_user_12x_cont(to, from, ret, COPY, FIXUP, TENTRY) \
__asm_copy_from_user_8x_cont(to, from, ret, \
" GETD D1Ar1,[%1++]\n" \
"6: SETD [%0++],D1Ar1\n" COPY, \
"7: ADD %2,%2,#4\n" \
" SETD [%0++],D1Ar1\n" FIXUP, \
" .long 6b,7b\n" TENTRY)
#define __asm_copy_from_user_12(to, from, ret) \
__asm_copy_from_user_12x_cont(to, from, ret, "", "", "")
#define __asm_copy_from_user_13(to, from, ret) \
__asm_copy_from_user_12x_cont(to, from, ret, \
" GETB D1Ar1,[%1++]\n" \
"8: SETB [%0++],D1Ar1\n", \
"9: ADD %2,%2,#1\n" \
" SETB [%0++],D1Ar1\n", \
" .long 8b,9b\n")
#define __asm_copy_from_user_14x_cont(to, from, ret, COPY, FIXUP, TENTRY) \
__asm_copy_from_user_12x_cont(to, from, ret, \
" GETW D1Ar1,[%1++]\n" \
"8: SETW [%0++],D1Ar1\n" COPY, \
"9: ADD %2,%2,#2\n" \
" SETW [%0++],D1Ar1\n" FIXUP, \
" .long 8b,9b\n" TENTRY)
#define __asm_copy_from_user_14(to, from, ret) \
__asm_copy_from_user_14x_cont(to, from, ret, "", "", "")
#define __asm_copy_from_user_15(to, from, ret) \
__asm_copy_from_user_14x_cont(to, from, ret, \
" GETB D1Ar1,[%1++]\n" \
"10: SETB [%0++],D1Ar1\n", \
"11: ADD %2,%2,#1\n" \
" SETB [%0++],D1Ar1\n", \
" .long 10b,11b\n")
#define __asm_copy_from_user_16x_cont(to, from, ret, COPY, FIXUP, TENTRY) \
__asm_copy_from_user_12x_cont(to, from, ret, \
" GETD D1Ar1,[%1++]\n" \
"8: SETD [%0++],D1Ar1\n" COPY, \
"9: ADD %2,%2,#4\n" \
" SETD [%0++],D1Ar1\n" FIXUP, \
" .long 8b,9b\n" TENTRY)
#define __asm_copy_from_user_16(to, from, ret) \
__asm_copy_from_user_16x_cont(to, from, ret, "", "", "")
#define __asm_copy_from_user_8x64(to, from, ret) \
asm volatile ( \
" GETL D0Ar2,D1Ar1,[%1++]\n" \
"2: SETL [%0++],D0Ar2,D1Ar1\n" \
"1:\n" \
" .section .fixup,\"ax\"\n" \
" MOV D1Ar1,#0\n" \
" MOV D0Ar2,#0\n" \
"3: ADD %2,%2,#8\n" \
" SETL [%0++],D0Ar2,D1Ar1\n" \
" MOVT D0Ar2,#HI(1b)\n" \
" JUMP D0Ar2,#LO(1b)\n" \
" .previous\n" \
@ -789,36 +711,57 @@ EXPORT_SYMBOL(__copy_user);
*
* Rationale:
* A fault occurs while reading from user buffer, which is the
* source. Since the fault is at a single address, we only
* need to rewind by 8 bytes.
* source.
* Since we don't write to kernel buffer until we read first,
* the kernel buffer is at the right state and needn't be
* corrected.
* corrected, but the source must be rewound to the beginning of
* the block, which is LSM_STEP*8 bytes.
* LSM_STEP is bits 10:8 in TXSTATUS which is already read
* and stored in D0Ar2
*
* NOTE: If a fault occurs at the last operation in M{G,S}ETL
* LSM_STEP will be 0. ie: we do 4 writes in our case, if
* a fault happens at the 4th write, LSM_STEP will be 0
* instead of 4. The code copes with that.
*/
#define __asm_copy_from_user_64bit_rapf_loop(to, from, ret, n, id) \
__asm_copy_user_64bit_rapf_loop(to, from, ret, n, id, \
"SUB %1, %1, #8\n")
"LSR D0Ar2, D0Ar2, #5\n" \
"ANDS D0Ar2, D0Ar2, #0x38\n" \
"ADDZ D0Ar2, D0Ar2, #32\n" \
"SUB %1, %1, D0Ar2\n")
/* rewind 'from' pointer when a fault occurs
*
* Rationale:
* A fault occurs while reading from user buffer, which is the
* source. Since the fault is at a single address, we only
* need to rewind by 4 bytes.
* source.
* Since we don't write to kernel buffer until we read first,
* the kernel buffer is at the right state and needn't be
* corrected.
* corrected, but the source must be rewound to the beginning of
* the block, which is LSM_STEP*4 bytes.
* LSM_STEP is bits 10:8 in TXSTATUS which is already read
* and stored in D0Ar2
*
* NOTE: If a fault occurs at the last operation in M{G,S}ETL
* LSM_STEP will be 0. ie: we do 4 writes in our case, if
* a fault happens at the 4th write, LSM_STEP will be 0
* instead of 4. The code copes with that.
*/
#define __asm_copy_from_user_32bit_rapf_loop(to, from, ret, n, id) \
__asm_copy_user_32bit_rapf_loop(to, from, ret, n, id, \
"SUB %1, %1, #4\n")
"LSR D0Ar2, D0Ar2, #6\n" \
"ANDS D0Ar2, D0Ar2, #0x1c\n" \
"ADDZ D0Ar2, D0Ar2, #16\n" \
"SUB %1, %1, D0Ar2\n")
/* Copy from user to kernel, zeroing the bytes that were inaccessible in
userland. The return-value is the number of bytes that were
inaccessible. */
unsigned long __copy_user_zeroing(void *pdst, const void __user *psrc,
unsigned long n)
/*
* Copy from user to kernel. The return-value is the number of bytes that were
* inaccessible.
*/
unsigned long raw_copy_from_user(void *pdst, const void __user *psrc,
unsigned long n)
{
register char *dst asm ("A0.2") = pdst;
register const char __user *src asm ("A1.2") = psrc;
@ -830,6 +773,8 @@ unsigned long __copy_user_zeroing(void *pdst, const void __user *psrc,
if ((unsigned long) src & 1) {
__asm_copy_from_user_1(dst, src, retn);
n--;
if (retn)
return retn + n;
}
if ((unsigned long) dst & 1) {
/* Worst case - byte copy */
@ -837,12 +782,14 @@ unsigned long __copy_user_zeroing(void *pdst, const void __user *psrc,
__asm_copy_from_user_1(dst, src, retn);
n--;
if (retn)
goto copy_exception_bytes;
return retn + n;
}
}
if (((unsigned long) src & 2) && n >= 2) {
__asm_copy_from_user_2(dst, src, retn);
n -= 2;
if (retn)
return retn + n;
}
if ((unsigned long) dst & 2) {
/* Second worst case - word copy */
@ -850,16 +797,10 @@ unsigned long __copy_user_zeroing(void *pdst, const void __user *psrc,
__asm_copy_from_user_2(dst, src, retn);
n -= 2;
if (retn)
goto copy_exception_bytes;
return retn + n;
}
}
/* We only need one check after the unalignment-adjustments,
because if both adjustments were done, either both or
neither reference had an exception. */
if (retn != 0)
goto copy_exception_bytes;
#ifdef USE_RAPF
/* 64 bit copy loop */
if (!(((unsigned long) src | (unsigned long) dst) & 7)) {
@ -872,7 +813,7 @@ unsigned long __copy_user_zeroing(void *pdst, const void __user *psrc,
__asm_copy_from_user_8x64(dst, src, retn);
n -= 8;
if (retn)
goto copy_exception_bytes;
return retn + n;
}
}
@ -888,7 +829,7 @@ unsigned long __copy_user_zeroing(void *pdst, const void __user *psrc,
__asm_copy_from_user_8x64(dst, src, retn);
n -= 8;
if (retn)
goto copy_exception_bytes;
return retn + n;
}
}
#endif
@ -898,7 +839,7 @@ unsigned long __copy_user_zeroing(void *pdst, const void __user *psrc,
n -= 4;
if (retn)
goto copy_exception_bytes;
return retn + n;
}
/* If we get here, there were no memory read faults. */
@ -924,21 +865,8 @@ unsigned long __copy_user_zeroing(void *pdst, const void __user *psrc,
/* If we get here, retn correctly reflects the number of failing
bytes. */
return retn;
copy_exception_bytes:
/* We already have "retn" bytes cleared, and need to clear the
remaining "n" bytes. A non-optimized simple byte-for-byte in-line
memset is preferred here, since this isn't speed-critical code and
we'd rather have this a leaf-function than calling memset. */
{
char *endp;
for (endp = dst + n; dst < endp; dst++)
*dst = 0;
}
return retn + n;
}
EXPORT_SYMBOL(__copy_user_zeroing);
EXPORT_SYMBOL(raw_copy_from_user);
#define __asm_clear_8x64(to, ret) \
asm volatile ( \

3
arch/mips/Kconfig
View File

@ -9,6 +9,7 @@ config MIPS
select HAVE_CONTEXT_TRACKING
select HAVE_GENERIC_DMA_COHERENT
select HAVE_IDE
select HAVE_IRQ_EXIT_ON_IRQ_STACK
select HAVE_OPROFILE
select HAVE_PERF_EVENTS
select PERF_USE_VMALLOC
@ -1463,7 +1464,7 @@ config CPU_MIPS64_R6
select CPU_SUPPORTS_HIGHMEM
select CPU_SUPPORTS_MSA
select GENERIC_CSUM
select MIPS_O32_FP64_SUPPORT if MIPS32_O32
select MIPS_O32_FP64_SUPPORT if 32BIT || MIPS32_O32
help
Choose this option to build a kernel for release 6 or later of the
MIPS64 architecture. New MIPS processors, starting with the Warrior

12
arch/mips/include/asm/irq.h
View File

@ -17,6 +17,18 @@
#include <irq.h>
#define IRQ_STACK_SIZE THREAD_SIZE
extern void *irq_stack[NR_CPUS];
static inline bool on_irq_stack(int cpu, unsigned long sp)
{
unsigned long low = (unsigned long)irq_stack[cpu];
unsigned long high = low + IRQ_STACK_SIZE;
return (low <= sp && sp <= high);
}
#ifdef CONFIG_I8259
static inline int irq_canonicalize(int irq)
{

8
arch/mips/include/asm/spinlock.h
View File

@ -112,7 +112,7 @@ static inline void arch_spin_lock(arch_spinlock_t *lock)
" andi %[ticket], %[ticket], 0xffff \n"
" bne %[ticket], %[my_ticket], 4f \n"
" subu %[ticket], %[my_ticket], %[ticket] \n"
"2: \n"
"2: .insn \n"
" .subsection 2 \n"
"4: andi %[ticket], %[ticket], 0xffff \n"
" sll %[ticket], 5 \n"
@ -187,7 +187,7 @@ static inline unsigned int arch_spin_trylock(arch_spinlock_t *lock)
" sc %[ticket], %[ticket_ptr] \n"
" beqz %[ticket], 1b \n"
" li %[ticket], 1 \n"
"2: \n"
"2: .insn \n"
" .subsection 2 \n"
"3: b 2b \n"
" li %[ticket], 0 \n"
@ -367,7 +367,7 @@ static inline int arch_read_trylock(arch_rwlock_t *rw)
" .set reorder \n"
__WEAK_LLSC_MB
" li %2, 1 \n"
"2: \n"
"2: .insn \n"
: "=" GCC_OFF_SMALL_ASM() (rw->lock), "=&r" (tmp), "=&r" (ret)
: GCC_OFF_SMALL_ASM() (rw->lock)
: "memory");
@ -407,7 +407,7 @@ static inline int arch_write_trylock(arch_rwlock_t *rw)
" lui %1, 0x8000 \n"
" sc %1, %0 \n"
" li %2, 1 \n"
"2: \n"
"2: .insn \n"
: "=" GCC_OFF_SMALL_ASM() (rw->lock), "=&r" (tmp),
"=&r" (ret)
: GCC_OFF_SMALL_ASM() (rw->lock)

7
arch/mips/include/asm/stackframe.h
View File

@ -216,12 +216,19 @@
LONG_S $25, PT_R25(sp)
LONG_S $28, PT_R28(sp)
LONG_S $31, PT_R31(sp)
/* Set thread_info if we're coming from user mode */
mfc0 k0, CP0_STATUS
sll k0, 3 /* extract cu0 bit */
bltz k0, 9f
ori $28, sp, _THREAD_MASK
xori $28, _THREAD_MASK
#ifdef CONFIG_CPU_CAVIUM_OCTEON
.set mips64
pref 0, 0($28) /* Prefetch the current pointer */
#endif
9:
.set pop
.endm

1
arch/mips/kernel/asm-offsets.c
View File

@ -101,6 +101,7 @@ void output_thread_info_defines(void)
OFFSET(TI_REGS, thread_info, regs);
DEFINE(_THREAD_SIZE, THREAD_SIZE);
DEFINE(_THREAD_MASK, THREAD_MASK);
DEFINE(_IRQ_STACK_SIZE, IRQ_STACK_SIZE);
BLANK();
}

16
arch/mips/kernel/crash.c
View File

@ -14,12 +14,22 @@ static int crashing_cpu = -1;
static cpumask_t cpus_in_crash = CPU_MASK_NONE;
#ifdef CONFIG_SMP
static void crash_shutdown_secondary(void *ignore)
static void crash_shutdown_secondary(void *passed_regs)
{
struct pt_regs *regs;
struct pt_regs *regs = passed_regs;
int cpu = smp_processor_id();
regs = task_pt_regs(current);
/*
* If we are passed registers, use those. Otherwise get the
* regs from the last interrupt, which should be correct, as
* we are in an interrupt. But if the regs are not there,
* pull them from the top of the stack. They are probably
* wrong, but we need something to keep from crashing again.
*/
if (!regs)
regs = get_irq_regs();
if (!regs)
regs = task_pt_regs(current);
if (!cpu_online(cpu))
return;

2
arch/mips/kernel/elf.c
View File

@ -206,7 +206,7 @@ int arch_check_elf(void *_ehdr, bool has_interpreter,
else if ((prog_req.fr1 && prog_req.frdefault) ||
(prog_req.single && !prog_req.frdefault))
/* Make sure 64-bit MIPS III/IV/64R1 will not pick FR1 */
state->overall_fp_mode = ((current_cpu_data.fpu_id & MIPS_FPIR_F64) &&
state->overall_fp_mode = ((raw_current_cpu_data.fpu_id & MIPS_FPIR_F64) &&
cpu_has_mips_r2_r6) ?
FP_FR1 : FP_FR0;
else if (prog_req.fr1)

81
arch/mips/kernel/genex.S
View File

@ -188,9 +188,44 @@ NESTED(handle_int, PT_SIZE, sp)
LONG_L s0, TI_REGS($28)
LONG_S sp, TI_REGS($28)
PTR_LA ra, ret_from_irq
PTR_LA v0, plat_irq_dispatch
jr v0
/*
* SAVE_ALL ensures we are using a valid kernel stack for the thread.
* Check if we are already using the IRQ stack.
*/
move s1, sp # Preserve the sp
/* Get IRQ stack for this CPU */
ASM_CPUID_MFC0 k0, ASM_SMP_CPUID_REG
#if defined(CONFIG_32BIT) || defined(KBUILD_64BIT_SYM32)
lui k1, %hi(irq_stack)
#else
lui k1, %highest(irq_stack)
daddiu k1, %higher(irq_stack)
dsll k1, 16
daddiu k1, %hi(irq_stack)
dsll k1, 16
#endif
LONG_SRL k0, SMP_CPUID_PTRSHIFT
LONG_ADDU k1, k0
LONG_L t0, %lo(irq_stack)(k1)
# Check if already on IRQ stack
PTR_LI t1, ~(_THREAD_SIZE-1)
and t1, t1, sp
beq t0, t1, 2f
/* Switch to IRQ stack */
li t1, _IRQ_STACK_SIZE
PTR_ADD sp, t0, t1
2:
jal plat_irq_dispatch
/* Restore sp */
move sp, s1
j ret_from_irq
#ifdef CONFIG_CPU_MICROMIPS
nop
#endif
@ -263,8 +298,44 @@ NESTED(except_vec_vi_handler, 0, sp)
LONG_L s0, TI_REGS($28)
LONG_S sp, TI_REGS($28)
PTR_LA ra, ret_from_irq
jr v0
/*
* SAVE_ALL ensures we are using a valid kernel stack for the thread.
* Check if we are already using the IRQ stack.
*/
move s1, sp # Preserve the sp
/* Get IRQ stack for this CPU */
ASM_CPUID_MFC0 k0, ASM_SMP_CPUID_REG
#if defined(CONFIG_32BIT) || defined(KBUILD_64BIT_SYM32)
lui k1, %hi(irq_stack)
#else
lui k1, %highest(irq_stack)
daddiu k1, %higher(irq_stack)
dsll k1, 16
daddiu k1, %hi(irq_stack)
dsll k1, 16
#endif
LONG_SRL k0, SMP_CPUID_PTRSHIFT
LONG_ADDU k1, k0
LONG_L t0, %lo(irq_stack)(k1)
# Check if already on IRQ stack
PTR_LI t1, ~(_THREAD_SIZE-1)
and t1, t1, sp
beq t0, t1, 2f
/* Switch to IRQ stack */
li t1, _IRQ_STACK_SIZE
PTR_ADD sp, t0, t1
2:
jalr v0
/* Restore sp */
move sp, s1
j ret_from_irq
END(except_vec_vi_handler)
/*

11
arch/mips/kernel/irq.c
View File

@ -25,6 +25,8 @@
#include <linux/atomic.h>
#include <asm/uaccess.h>
void *irq_stack[NR_CPUS];
/*
* 'what should we do if we get a hw irq event on an illegal vector'.
* each architecture has to answer this themselves.
@ -55,6 +57,15 @@ void __init init_IRQ(void)
irq_set_noprobe(i);
arch_init_irq();
for_each_possible_cpu(i) {
int irq_pages = IRQ_STACK_SIZE / PAGE_SIZE;
void *s = (void *)__get_free_pages(GFP_KERNEL, irq_pages);
irq_stack[i] = s;
pr_debug("CPU%d IRQ stack at 0x%p - 0x%p\n", i,
irq_stack[i], irq_stack[i] + IRQ_STACK_SIZE);
}
}
#ifdef CONFIG_DEBUG_STACKOVERFLOW

46
arch/mips/kernel/kgdb.c
View File

@ -244,9 +244,6 @@ static int compute_signal(int tt)
void sleeping_thread_to_gdb_regs(unsigned long *gdb_regs, struct task_struct *p)
{
int reg;
struct thread_info *ti = task_thread_info(p);
unsigned long ksp = (unsigned long)ti + THREAD_SIZE - 32;
struct pt_regs *regs = (struct pt_regs *)ksp - 1;
#if (KGDB_GDB_REG_SIZE == 32)
u32 *ptr = (u32 *)gdb_regs;
#else
@ -254,25 +251,46 @@ void sleeping_thread_to_gdb_regs(unsigned long *gdb_regs, struct task_struct *p)
#endif
for (reg = 0; reg < 16; reg++)
*(ptr++) = regs->regs[reg];
*(ptr++) = 0;
/* S0 - S7 */
for (reg = 16; reg < 24; reg++)
*(ptr++) = regs->regs[reg];
*(ptr++) = p->thread.reg16;
*(ptr++) = p->thread.reg17;
*(ptr++) = p->thread.reg18;
*(ptr++) = p->thread.reg19;
*(ptr++) = p->thread.reg20;
*(ptr++) = p->thread.reg21;
*(ptr++) = p->thread.reg22;
*(ptr++) = p->thread.reg23;
for (reg = 24; reg < 28; reg++)
*(ptr++) = 0;
/* GP, SP, FP, RA */
for (reg = 28; reg < 32; reg++)
*(ptr++) = regs->regs[reg];
*(ptr++) = (long)p;
*(ptr++) = p->thread.reg29;
*(ptr++) = p->thread.reg30;
*(ptr++) = p->thread.reg31;
*(ptr++) = regs->cp0_status;
*(ptr++) = regs->lo;
*(ptr++) = regs->hi;
*(ptr++) = regs->cp0_badvaddr;
*(ptr++) = regs->cp0_cause;
*(ptr++) = regs->cp0_epc;
*(ptr++) = p->thread.cp0_status;
/* lo, hi */
*(ptr++) = 0;
*(ptr++) = 0;
/*
* BadVAddr, Cause
* Ideally these would come from the last exception frame up the stack
* but that requires unwinding, otherwise we can't know much for sure.
*/
*(ptr++) = 0;
*(ptr++) = 0;
/*
* PC
* use return address (RA), i.e. the moment after return from resume()
*/
*(ptr++) = p->thread.reg31;
}
void kgdb_arch_set_pc(struct pt_regs *regs, unsigned long pc)

15
arch/mips/kernel/process.c
View File

@ -32,6 +32,7 @@
#include <asm/cpu.h>
#include <asm/dsp.h>
#include <asm/fpu.h>
#include <asm/irq.h>
#include <asm/msa.h>
#include <asm/pgtable.h>
#include <asm/mipsregs.h>
@ -552,7 +553,19 @@ EXPORT_SYMBOL(unwind_stack_by_address);
unsigned long unwind_stack(struct task_struct *task, unsigned long *sp,
unsigned long pc, unsigned long *ra)
{
unsigned long stack_page = (unsigned long)task_stack_page(task);
unsigned long stack_page = 0;
int cpu;
for_each_possible_cpu(cpu) {
if (on_irq_stack(cpu, *sp)) {
stack_page = (unsigned long)irq_stack[cpu];
break;
}
}
if (!stack_page)
stack_page = (unsigned long)task_stack_page(task);
return unwind_stack_by_address(stack_page, sp, pc, ra);
}
#endif

3
arch/mips/kernel/ptrace.c
View File

@ -485,7 +485,8 @@ static int fpr_set(struct task_struct *target,
&target->thread.fpu,
0, sizeof(elf_fpregset_t));
for (i = 0; i < NUM_FPU_REGS; i++) {
BUILD_BUG_ON(sizeof(fpr_val) != sizeof(elf_fpreg_t));
for (i = 0; i < NUM_FPU_REGS && count >= sizeof(elf_fpreg_t); i++) {
err = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
&fpr_val, i * sizeof(elf_fpreg_t),
(i + 1) * sizeof(elf_fpreg_t));

2
arch/mips/lantiq/xway/sysctrl.c
View File

@ -467,7 +467,7 @@ void __init ltq_soc_init(void)
if (!np_xbar)
panic("Failed to load xbar nodes from devicetree");
if (of_address_to_resource(np_pmu, 0, &res_xbar))
if (of_address_to_resource(np_xbar, 0, &res_xbar))
panic("Failed to get xbar resources");
if (request_mem_region(res_xbar.start, resource_size(&res_xbar),
res_xbar.name) < 0)

25
arch/mips/mm/tlbex.c
View File

@ -757,7 +757,8 @@ static void build_huge_update_entries(u32 **p, unsigned int pte,
static void build_huge_handler_tail(u32 **p, struct uasm_reloc **r,
struct uasm_label **l,
unsigned int pte,
unsigned int ptr)
unsigned int ptr,
unsigned int flush)
{
#ifdef CONFIG_SMP
UASM_i_SC(p, pte, 0, ptr);
@ -766,6 +767,22 @@ static void build_huge_handler_tail(u32 **p, struct uasm_reloc **r,
#else
UASM_i_SW(p, pte, 0, ptr);
#endif
if (cpu_has_ftlb && flush) {
BUG_ON(!cpu_has_tlbinv);
UASM_i_MFC0(p, ptr, C0_ENTRYHI);
uasm_i_ori(p, ptr, ptr, MIPS_ENTRYHI_EHINV);
UASM_i_MTC0(p, ptr, C0_ENTRYHI);
build_tlb_write_entry(p, l, r, tlb_indexed);
uasm_i_xori(p, ptr, ptr, MIPS_ENTRYHI_EHINV);
UASM_i_MTC0(p, ptr, C0_ENTRYHI);
build_huge_update_entries(p, pte, ptr);
build_huge_tlb_write_entry(p, l, r, pte, tlb_random, 0);
return;
}
build_huge_update_entries(p, pte, ptr);
build_huge_tlb_write_entry(p, l, r, pte, tlb_indexed, 0);
}
@ -2082,7 +2099,7 @@ static void build_r4000_tlb_load_handler(void)
uasm_l_tlbl_goaround2(&l, p);
}
uasm_i_ori(&p, wr.r1, wr.r1, (_PAGE_ACCESSED | _PAGE_VALID));
build_huge_handler_tail(&p, &r, &l, wr.r1, wr.r2);
build_huge_handler_tail(&p, &r, &l, wr.r1, wr.r2, 1);
#endif
uasm_l_nopage_tlbl(&l, p);
@ -2137,7 +2154,7 @@ static void build_r4000_tlb_store_handler(void)
build_tlb_probe_entry(&p);
uasm_i_ori(&p, wr.r1, wr.r1,
_PAGE_ACCESSED | _PAGE_MODIFIED | _PAGE_VALID | _PAGE_DIRTY);
build_huge_handler_tail(&p, &r, &l, wr.r1, wr.r2);
build_huge_handler_tail(&p, &r, &l, wr.r1, wr.r2, 1);
#endif
uasm_l_nopage_tlbs(&l, p);
@ -2193,7 +2210,7 @@ static void build_r4000_tlb_modify_handler(void)
build_tlb_probe_entry(&p);
uasm_i_ori(&p, wr.r1, wr.r1,
_PAGE_ACCESSED | _PAGE_MODIFIED | _PAGE_VALID | _PAGE_DIRTY);
build_huge_handler_tail(&p, &r, &l, wr.r1, wr.r2);
build_huge_handler_tail(&p, &r, &l, wr.r1, wr.r2, 0);
#endif
uasm_l_nopage_tlbm(&l, p);

4
arch/mips/ralink/rt3883.c
View File

@ -36,7 +36,7 @@ static struct rt2880_pmx_func uartlite_func[] = { FUNC("uartlite", 0, 15, 2) };
static struct rt2880_pmx_func jtag_func[] = { FUNC("jtag", 0, 17, 5) };
static struct rt2880_pmx_func mdio_func[] = { FUNC("mdio", 0, 22, 2) };
static struct rt2880_pmx_func lna_a_func[] = { FUNC("lna a", 0, 32, 3) };
static struct rt2880_pmx_func lna_g_func[] = { FUNC("lna a", 0, 35, 3) };
static struct rt2880_pmx_func lna_g_func[] = { FUNC("lna g", 0, 35, 3) };
static struct rt2880_pmx_func pci_func[] = {
FUNC("pci-dev", 0, 40, 32),
FUNC("pci-host2", 1, 40, 32),
@ -44,7 +44,7 @@ static struct rt2880_pmx_func pci_func[] = {
FUNC("pci-fnc", 3, 40, 32)
};
static struct rt2880_pmx_func ge1_func[] = { FUNC("ge1", 0, 72, 12) };
static struct rt2880_pmx_func ge2_func[] = { FUNC("ge1", 0, 84, 12) };
static struct rt2880_pmx_func ge2_func[] = { FUNC("ge2", 0, 84, 12) };
static struct rt2880_pmx_group rt3883_pinmux_data[] = {
GRP("i2c", i2c_func, 1, RT3883_GPIO_MODE_I2C),

7
arch/nios2/kernel/prom.c
View File

@ -48,6 +48,13 @@ void * __init early_init_dt_alloc_memory_arch(u64 size, u64 align)
return alloc_bootmem_align(size, align);
}
int __init early_init_dt_reserve_memory_arch(phys_addr_t base, phys_addr_t size,
bool nomap)
{
reserve_bootmem(base, size, BOOTMEM_DEFAULT);
return 0;
}
void __init early_init_devtree(void *params)
{
__be32 *dtb = (u32 *)__dtb_start;

3
arch/nios2/kernel/setup.c
View File

@ -195,6 +195,9 @@ void __init setup_arch(char **cmdline_p)
}
#endif /* CONFIG_BLK_DEV_INITRD */
early_init_fdt_reserve_self();
early_init_fdt_scan_reserved_mem();
unflatten_and_copy_device_tree();
setup_cpuinfo();

1
arch/powerpc/boot/zImage.lds.S
View File

@ -68,6 +68,7 @@ SECTIONS
}
#ifdef CONFIG_PPC64_BOOT_WRAPPER
. = ALIGN(256);
.got :
{
__toc_start = .;

27
arch/powerpc/kernel/align.c
View File

@ -808,14 +808,25 @@ int fix_alignment(struct pt_regs *regs)
nb = aligninfo[instr].len;
flags = aligninfo[instr].flags;
/* ldbrx/stdbrx overlap lfs/stfs in the DSISR unfortunately */
if (IS_XFORM(instruction) && ((instruction >> 1) & 0x3ff) == 532) {
nb = 8;
flags = LD+SW;
} else if (IS_XFORM(instruction) &&
((instruction >> 1) & 0x3ff) == 660) {
nb = 8;
flags = ST+SW;
/*
* Handle some cases which give overlaps in the DSISR values.
*/
if (IS_XFORM(instruction)) {
switch (get_xop(instruction)) {
case 532: /* ldbrx */
nb = 8;
flags = LD+SW;
break;
case 660: /* stdbrx */
nb = 8;
flags = ST+SW;
break;
case 20: /* lwarx */
case 84: /* ldarx */
case 116: /* lharx */
case 276: /* lqarx */
return 0; /* not emulated ever */
}
}
/* Byteswap little endian loads and stores */

6
arch/powerpc/kernel/entry_64.S
View File

@ -716,7 +716,7 @@ resume_kernel:
addi r8,r1,INT_FRAME_SIZE /* Get the kprobed function entry */
lwz r3,GPR1(r1)
ld r3,GPR1(r1)
subi r3,r3,INT_FRAME_SIZE /* dst: Allocate a trampoline exception frame */
mr r4,r1 /* src: current exception frame */
mr r1,r3 /* Reroute the trampoline frame to r1 */
@ -730,8 +730,8 @@ resume_kernel:
addi r6,r6,8
bdnz 2b
/* Do real store operation to complete stwu */
lwz r5,GPR1(r1)
/* Do real store operation to complete stdu */
ld r5,GPR1(r1)
std r8,0(r5)
/* Clear _TIF_EMULATE_STACK_STORE flag */

9
arch/powerpc/kernel/setup_64.c
View File

@ -220,6 +220,15 @@ static void cpu_ready_for_interrupts(void)
unsigned long lpcr = mfspr(SPRN_LPCR);
mtspr(SPRN_LPCR, lpcr | LPCR_AIL_3);
}
/*
* Fixup HFSCR:TM based on CPU features. The bit is set by our
* early asm init because at that point we haven't updated our
* CPU features from firmware and device-tree. Here we have,
* so let's do it.
*/
if (cpu_has_feature(CPU_FTR_HVMODE) && !cpu_has_feature(CPU_FTR_TM_COMP))
mtspr(SPRN_HFSCR, mfspr(SPRN_HFSCR) & ~HFSCR_TM);
}
/*

1
arch/powerpc/kvm/emulate.c
View File

@ -302,7 +302,6 @@ int kvmppc_emulate_instruction(struct kvm_run *run, struct kvm_vcpu *vcpu)
advance = 0;
printk(KERN_ERR "Couldn't emulate instruction 0x%08x "
"(op %d xop %d)\n", inst, get_op(inst), get_xop(inst));
kvmppc_core_queue_program(vcpu, 0);
}
}

7
arch/powerpc/mm/hash_native_64.c
View File

@ -645,6 +645,10 @@ static void native_flush_hash_range(unsigned long number, int local)
unsigned long psize = batch->psize;
int ssize = batch->ssize;
int i;
unsigned int use_local;
use_local = local && mmu_has_feature(MMU_FTR_TLBIEL) &&
mmu_psize_defs[psize].tlbiel && !cxl_ctx_in_use();
local_irq_save(flags);
@ -671,8 +675,7 @@ static void native_flush_hash_range(unsigned long number, int local)
} pte_iterate_hashed_end();
}
if (mmu_has_feature(MMU_FTR_TLBIEL) &&
mmu_psize_defs[psize].tlbiel && local) {
if (use_local) {
asm volatile("ptesync":::"memory");
for (i = 0; i < number; i++) {
vpn = batch->vpn[i];

3
arch/s390/Kconfig
View File

@ -62,6 +62,9 @@ config PCI_QUIRKS
config ARCH_SUPPORTS_UPROBES
def_bool y
config DEBUG_RODATA
def_bool y
config S390
def_bool y
select ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE

35
arch/s390/boot/compressed/misc.c
View File

@ -141,31 +141,34 @@ static void check_ipl_parmblock(void *start, unsigned long size)
unsigned long decompress_kernel(void)
{
unsigned long output_addr;
unsigned char *output;
void *output, *kernel_end;
output_addr = ((unsigned long) &_end + HEAP_SIZE + 4095UL) & -4096UL;
check_ipl_parmblock((void *) 0, output_addr + SZ__bss_start);
memset(&_bss, 0, &_ebss - &_bss);
free_mem_ptr = (unsigned long)&_end;
free_mem_end_ptr = free_mem_ptr + HEAP_SIZE;
output = (unsigned char *) output_addr;
output = (void *) ALIGN((unsigned long) &_end + HEAP_SIZE, PAGE_SIZE);
kernel_end = output + SZ__bss_start;
check_ipl_parmblock((void *) 0, (unsigned long) kernel_end);
#ifdef CONFIG_BLK_DEV_INITRD
/*
* Move the initrd right behind the end of the decompressed
* kernel image.
* kernel image. This also prevents initrd corruption caused by
* bss clearing since kernel_end will always be located behind the
* current bss section..
*/
if (INITRD_START && INITRD_SIZE &&
INITRD_START < (unsigned long) output + SZ__bss_start) {
check_ipl_parmblock(output + SZ__bss_start,
INITRD_START + INITRD_SIZE);
memmove(output + SZ__bss_start,
(void *) INITRD_START, INITRD_SIZE);
INITRD_START = (unsigned long) output + SZ__bss_start;
if (INITRD_START && INITRD_SIZE && kernel_end > (void *) INITRD_START) {
check_ipl_parmblock(kernel_end, INITRD_SIZE);
memmove(kernel_end, (void *) INITRD_START, INITRD_SIZE);
INITRD_START = (unsigned long) kernel_end;
}
#endif
/*
* Clear bss section. free_mem_ptr and free_mem_end_ptr need to be
* initialized afterwards since they reside in bss.
*/
memset(&_bss, 0, &_ebss - &_bss);
free_mem_ptr = (unsigned long) &_end;
free_mem_end_ptr = free_mem_ptr + HEAP_SIZE;
puts("Uncompressing Linux... ");
__decompress(input_data, input_len, NULL, NULL, output, 0, NULL, error);
puts("Ok, booting the kernel.\n");

2
arch/s390/include/asm/pgtable.h
View File

@ -829,6 +829,8 @@ static inline void set_pte_at(struct mm_struct *mm, unsigned long addr,
{
pgste_t pgste;
if (pte_present(entry))
pte_val(entry) &= ~_PAGE_UNUSED;
if (mm_has_pgste(mm)) {
pgste = pgste_get_lock(ptep);
pgste_val(pgste) &= ~_PGSTE_GPS_ZERO;

2
arch/s390/include/asm/uaccess.h
View File

@ -150,7 +150,7 @@ unsigned long __must_check __copy_to_user(void __user *to, const void *from,
" jg 2b\n" \
".popsection\n" \
EX_TABLE(0b,3b) EX_TABLE(1b,3b) \
: "=d" (__rc), "=Q" (*(to)) \
: "=d" (__rc), "+Q" (*(to)) \
: "d" (size), "Q" (*(from)), \
"d" (__reg0), "K" (-EFAULT) \
: "cc"); \

10
arch/s390/mm/init.c
View File

@ -109,6 +109,13 @@ void __init paging_init(void)
free_area_init_nodes(max_zone_pfns);
}
void mark_rodata_ro(void)
{
/* Text and rodata are already protected. Nothing to do here. */
pr_info("Write protecting the kernel read-only data: %luk\n",
((unsigned long)&_eshared - (unsigned long)&_stext) >> 10);
}
void __init mem_init(void)
{
if (MACHINE_HAS_TLB_LC)
@ -127,9 +134,6 @@ void __init mem_init(void)
setup_zero_pages(); /* Setup zeroed pages. */
mem_init_print_info(NULL);
printk("Write protected kernel read-only data: %#lx - %#lx\n",
(unsigned long)&_stext,
PFN_ALIGN((unsigned long)&_eshared) - 1);
}
void free_initmem(void)

16
arch/s390/pci/pci_dma.c
View File

@ -455,7 +455,7 @@ int zpci_dma_init_device(struct zpci_dev *zdev)
zdev->dma_table = dma_alloc_cpu_table();
if (!zdev->dma_table) {
rc = -ENOMEM;
goto out_clean;
goto out;
}
/*
@ -475,18 +475,22 @@ int zpci_dma_init_device(struct zpci_dev *zdev)
zdev->iommu_bitmap = vzalloc(zdev->iommu_pages / 8);
if (!zdev->iommu_bitmap) {
rc = -ENOMEM;
goto out_reg;
goto free_dma_table;
}
rc = zpci_register_ioat(zdev, 0, zdev->start_dma, zdev->end_dma,
(u64) zdev->dma_table);
if (rc)
goto out_reg;
return 0;
goto free_bitmap;
out_reg:
return 0;
free_bitmap:
vfree(zdev->iommu_bitmap);
zdev->iommu_bitmap = NULL;
free_dma_table:
dma_free_cpu_table(zdev->dma_table);
out_clean:
zdev->dma_table = NULL;
out:
return rc;
}

15
arch/sparc/include/asm/pgtable_64.h
View File

@ -668,6 +668,14 @@ static inline unsigned long pmd_pfn(pmd_t pmd)
return pte_pfn(pte);
}
#define __HAVE_ARCH_PMD_WRITE
static inline unsigned long pmd_write(pmd_t pmd)
{
pte_t pte = __pte(pmd_val(pmd));
return pte_write(pte);
}
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
static inline unsigned long pmd_dirty(pmd_t pmd)
{
@ -683,13 +691,6 @@ static inline unsigned long pmd_young(pmd_t pmd)
return pte_young(pte);
}
static inline unsigned long pmd_write(pmd_t pmd)
{
pte_t pte = __pte(pmd_val(pmd));
return pte_write(pte);
}
static inline unsigned long pmd_trans_huge(pmd_t pmd)
{
pte_t pte = __pte(pmd_val(pmd));

2
arch/sparc/kernel/ptrace_64.c
View File

@ -311,7 +311,7 @@ static int genregs64_set(struct task_struct *target,
}
if (!ret) {
unsigned long y;
unsigned long y = regs->y;
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
&y,

2
arch/sparc/mm/init_64.c
View File

@ -1493,7 +1493,7 @@ bool kern_addr_valid(unsigned long addr)
if ((long)addr < 0L) {
unsigned long pa = __pa(addr);
if ((addr >> max_phys_bits) != 0UL)
if ((pa >> max_phys_bits) != 0UL)
return false;
return pfn_valid(pa >> PAGE_SHIFT);

26
arch/x86/crypto/ghash-clmulni-intel_glue.c
View File

@ -219,6 +219,29 @@ static int ghash_async_final(struct ahash_request *req)
}
}
static int ghash_async_import(struct ahash_request *req, const void *in)
{
struct ahash_request *cryptd_req = ahash_request_ctx(req);
struct shash_desc *desc = cryptd_shash_desc(cryptd_req);
struct ghash_desc_ctx *dctx = shash_desc_ctx(desc);
ghash_async_init(req);
memcpy(dctx, in, sizeof(*dctx));
return 0;
}
static int ghash_async_export(struct ahash_request *req, void *out)
{
struct ahash_request *cryptd_req = ahash_request_ctx(req);
struct shash_desc *desc = cryptd_shash_desc(cryptd_req);
struct ghash_desc_ctx *dctx = shash_desc_ctx(desc);
memcpy(out, dctx, sizeof(*dctx));
return 0;
}
static int ghash_async_digest(struct ahash_request *req)
{
struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
@ -288,8 +311,11 @@ static struct ahash_alg ghash_async_alg = {
.final = ghash_async_final,
.setkey = ghash_async_setkey,
.digest = ghash_async_digest,
.export = ghash_async_export,
.import = ghash_async_import,
.halg = {
.digestsize = GHASH_DIGEST_SIZE,
.statesize = sizeof(struct ghash_desc_ctx),
.base = {
.cra_name = "ghash",
.cra_driver_name = "ghash-clmulni",

4
arch/x86/entry/vdso/Makefile
View File

@ -187,10 +187,10 @@ vdso_img_insttargets := $(vdso_img_sodbg:%.dbg=install_%)
$(MODLIB)/vdso: FORCE
@mkdir -p $(MODLIB)/vdso
$(vdso_img_insttargets): install_%: $(obj)/%.dbg $(MODLIB)/vdso FORCE
$(vdso_img_insttargets): install_%: $(obj)/%.dbg $(MODLIB)/vdso
$(call cmd,vdso_install)
PHONY += vdso_install $(vdso_img_insttargets)
vdso_install: $(vdso_img_insttargets) FORCE
vdso_install: $(vdso_img_insttargets)
clean-files := vdso32.so vdso32.so.dbg vdso64* vdso-image-*.c vdsox32.so*

11
arch/x86/entry/vdso/vdso32-setup.c
View File

@ -31,8 +31,10 @@ static int __init vdso32_setup(char *s)
{
vdso32_enabled = simple_strtoul(s, NULL, 0);
if (vdso32_enabled > 1)
if (vdso32_enabled > 1) {
pr_warn("vdso32 values other than 0 and 1 are no longer allowed; vdso disabled\n");
vdso32_enabled = 0;
}
return 1;
}
@ -63,13 +65,18 @@ subsys_initcall(sysenter_setup);
/* Register vsyscall32 into the ABI table */
#include <linux/sysctl.h>
static const int zero;
static const int one = 1;
static struct ctl_table abi_table2[] = {
{
.procname = "vsyscall32",
.data = &vdso32_enabled,
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec
.proc_handler = proc_dointvec_minmax,
.extra1 = (int *)&zero,
.extra2 = (int *)&one,
},
{}
};

2
arch/x86/include/asm/elf.h
View File

@ -278,7 +278,7 @@ struct task_struct;
#define ARCH_DLINFO_IA32 \
do { \
if (vdso32_enabled) { \
if (VDSO_CURRENT_BASE) { \
NEW_AUX_ENT(AT_SYSINFO, VDSO_ENTRY); \
NEW_AUX_ENT(AT_SYSINFO_EHDR, VDSO_CURRENT_BASE); \
} \

45
arch/x86/include/asm/pmem.h
View File

@ -72,8 +72,8 @@ static inline void arch_wmb_pmem(void)
* @size: number of bytes to write back
*
* Write back a cache range using the CLWB (cache line write back)
* instruction. This function requires explicit ordering with an
* arch_wmb_pmem() call. This API is internal to the x86 PMEM implementation.
* instruction. Note that @size is internally rounded up to be cache
* line size aligned.
*/
static inline void __arch_wb_cache_pmem(void *vaddr, size_t size)
{
@ -87,15 +87,6 @@ static inline void __arch_wb_cache_pmem(void *vaddr, size_t size)
clwb(p);
}
/*
* copy_from_iter_nocache() on x86 only uses non-temporal stores for iovec
* iterators, so for other types (bvec & kvec) we must do a cache write-back.
*/
static inline bool __iter_needs_pmem_wb(struct iov_iter *i)
{
return iter_is_iovec(i) == false;
}
/**
* arch_copy_from_iter_pmem - copy data from an iterator to PMEM
* @addr: PMEM destination address
@ -114,8 +105,36 @@ static inline size_t arch_copy_from_iter_pmem(void __pmem *addr, size_t bytes,
/* TODO: skip the write-back by always using non-temporal stores */
len = copy_from_iter_nocache(vaddr, bytes, i);
if (__iter_needs_pmem_wb(i))
__arch_wb_cache_pmem(vaddr, bytes);
/*
* In the iovec case on x86_64 copy_from_iter_nocache() uses
* non-temporal stores for the bulk of the transfer, but we need
* to manually flush if the transfer is unaligned. A cached
* memory copy is used when destination or size is not naturally
* aligned. That is:
* - Require 8-byte alignment when size is 8 bytes or larger.
* - Require 4-byte alignment when size is 4 bytes.
*
* In the non-iovec case the entire destination needs to be
* flushed.
*/
if (iter_is_iovec(i)) {
unsigned long flushed, dest = (unsigned long) addr;
if (bytes < 8) {
if (!IS_ALIGNED(dest, 4) || (bytes != 4))
__arch_wb_cache_pmem(addr, 1);
} else {
if (!IS_ALIGNED(dest, 8)) {
dest = ALIGN(dest, boot_cpu_data.x86_clflush_size);
__arch_wb_cache_pmem(addr, 1);
}
flushed = dest - (unsigned long) addr;
if (bytes > flushed && !IS_ALIGNED(bytes - flushed, 8))
__arch_wb_cache_pmem(addr + bytes - 1, 1);
}
} else
__arch_wb_cache_pmem(addr, bytes);
return len;
}

2
arch/x86/kernel/cpu/mcheck/mce_amd.c
View File

@ -53,7 +53,7 @@ static const char * const th_names[] = {
"load_store",
"insn_fetch",
"combined_unit",
"",
"decode_unit",
"northbridge",
"execution_unit",
};

24
arch/x86/kernel/cpu/mshyperv.c
View File

@ -30,6 +30,7 @@
#include <asm/apic.h>
#include <asm/timer.h>
#include <asm/reboot.h>
#include <asm/nmi.h>
struct ms_hyperv_info ms_hyperv;
EXPORT_SYMBOL_GPL(ms_hyperv);
@ -157,6 +158,26 @@ static unsigned char hv_get_nmi_reason(void)
return 0;
}
#ifdef CONFIG_X86_LOCAL_APIC
/*
* Prior to WS2016 Debug-VM sends NMIs to all CPUs which makes
* it dificult to process CHANNELMSG_UNLOAD in case of crash. Handle
* unknown NMI on the first CPU which gets it.
*/
static int hv_nmi_unknown(unsigned int val, struct pt_regs *regs)
{
static atomic_t nmi_cpu = ATOMIC_INIT(-1);
if (!unknown_nmi_panic)
return NMI_DONE;
if (atomic_cmpxchg(&nmi_cpu, -1, raw_smp_processor_id()) != -1)
return NMI_HANDLED;
return NMI_DONE;
}
#endif
static void __init ms_hyperv_init_platform(void)
{
/*
@ -182,6 +203,9 @@ static void __init ms_hyperv_init_platform(void)
printk(KERN_INFO "HyperV: LAPIC Timer Frequency: %#x\n",
lapic_timer_frequency);
}
register_nmi_handler(NMI_UNKNOWN, hv_nmi_unknown, NMI_FLAG_FIRST,
"hv_nmi_unknown");
#endif
if (ms_hyperv.features & HV_X64_MSR_TIME_REF_COUNT_AVAILABLE)

4
arch/x86/kernel/cpu/perf_event.c
View File

@ -1996,8 +1996,8 @@ static int x86_pmu_event_init(struct perf_event *event)
static void refresh_pce(void *ignored)
{
if (current->mm)
load_mm_cr4(current->mm);
if (current->active_mm)
load_mm_cr4(current->active_mm);
}
static void x86_pmu_event_mapped(struct perf_event *event)

3
arch/x86/kernel/cpu/perf_event_intel_lbr.c
View File

@ -410,6 +410,9 @@ static void intel_pmu_lbr_read_32(struct cpu_hw_events *cpuc)
cpuc->lbr_entries[i].to = msr_lastbranch.to;
cpuc->lbr_entries[i].mispred = 0;
cpuc->lbr_entries[i].predicted = 0;
cpuc->lbr_entries[i].in_tx = 0;
cpuc->lbr_entries[i].abort = 0;
cpuc->lbr_entries[i].cycles = 0;
cpuc->lbr_entries[i].reserved = 0;
}
cpuc->lbr_stack.nr = i;

12
arch/x86/kernel/ftrace.c
View File

@ -977,6 +977,18 @@ void prepare_ftrace_return(unsigned long self_addr, unsigned long *parent,
unsigned long return_hooker = (unsigned long)
&return_to_handler;
/*
* When resuming from suspend-to-ram, this function can be indirectly
* called from early CPU startup code while the CPU is in real mode,
* which would fail miserably. Make sure the stack pointer is a
* virtual address.
*
* This check isn't as accurate as virt_addr_valid(), but it should be
* good enough for this purpose, and it's fast.
*/
if (unlikely((long)__builtin_frame_address(0) >= 0))
return;
if (unlikely(ftrace_graph_is_dead()))
return;

1
arch/x86/kernel/head64.c
View File

@ -4,6 +4,7 @@
* Copyright (C) 2000 Andrea Arcangeli <andrea@suse.de> SuSE
*/
#define DISABLE_BRANCH_PROFILING
#include <linux/init.h>
#include <linux/linkage.h>
#include <linux/types.h>

10
arch/x86/kvm/vmx.c
View File

@ -6678,14 +6678,20 @@ static int nested_vmx_check_vmptr(struct kvm_vcpu *vcpu, int exit_reason,
}
page = nested_get_page(vcpu, vmptr);
if (page == NULL ||
*(u32 *)kmap(page) != VMCS12_REVISION) {
if (page == NULL) {
nested_vmx_failInvalid(vcpu);
skip_emulated_instruction(vcpu);
return 1;
}
if (*(u32 *)kmap(page) != VMCS12_REVISION) {
kunmap(page);
nested_release_page_clean(page);
nested_vmx_failInvalid(vcpu);
skip_emulated_instruction(vcpu);
return 1;
}
kunmap(page);
nested_release_page_clean(page);
vmx->nested.vmxon_ptr = vmptr;
break;
case EXIT_REASON_VMCLEAR:

41
arch/x86/mm/init.c
View File

@ -628,21 +628,40 @@ void __init init_mem_mapping(void)
* devmem_is_allowed() checks to see if /dev/mem access to a certain address
* is valid. The argument is a physical page number.
*
*
* On x86, access has to be given to the first megabyte of ram because that area
* contains BIOS code and data regions used by X and dosemu and similar apps.
* Access has to be given to non-kernel-ram areas as well, these contain the PCI
* mmio resources as well as potential bios/acpi data regions.
* On x86, access has to be given to the first megabyte of RAM because that
* area traditionally contains BIOS code and data regions used by X, dosemu,
* and similar apps. Since they map the entire memory range, the whole range
* must be allowed (for mapping), but any areas that would otherwise be
* disallowed are flagged as being "zero filled" instead of rejected.
* Access has to be given to non-kernel-ram areas as well, these contain the
* PCI mmio resources as well as potential bios/acpi data regions.
*/
int devmem_is_allowed(unsigned long pagenr)
{
if (pagenr < 256)
return 1;
if (iomem_is_exclusive(pagenr << PAGE_SHIFT))
if (page_is_ram(pagenr)) {
/*
* For disallowed memory regions in the low 1MB range,
* request that the page be shown as all zeros.
*/
if (pagenr < 256)
return 2;
return 0;
if (!page_is_ram(pagenr))
return 1;
return 0;
}
/*
* This must follow RAM test, since System RAM is considered a
* restricted resource under CONFIG_STRICT_IOMEM.
*/
if (iomem_is_exclusive(pagenr << PAGE_SHIFT)) {
/* Low 1MB bypasses iomem restrictions. */
if (pagenr < 256)
return 1;
return 0;
}
return 1;
}
void free_init_pages(char *what, unsigned long begin, unsigned long end)

1
arch/x86/mm/kasan_init_64.c
View File

@ -1,3 +1,4 @@
#define DISABLE_BRANCH_PROFILING
#define pr_fmt(fmt) "kasan: " fmt
#include <linux/bootmem.h>
#include <linux/kasan.h>

23
arch/x86/pci/xen.c
View File

@ -231,23 +231,14 @@ static int xen_hvm_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
return 1;
for_each_pci_msi_entry(msidesc, dev) {
__pci_read_msi_msg(msidesc, &msg);
pirq = MSI_ADDR_EXT_DEST_ID(msg.address_hi) |
((msg.address_lo >> MSI_ADDR_DEST_ID_SHIFT) & 0xff);
if (msg.data != XEN_PIRQ_MSI_DATA ||
xen_irq_from_pirq(pirq) < 0) {
pirq = xen_allocate_pirq_msi(dev, msidesc);
if (pirq < 0) {
irq = -ENODEV;
goto error;
}
xen_msi_compose_msg(dev, pirq, &msg);
__pci_write_msi_msg(msidesc, &msg);
dev_dbg(&dev->dev, "xen: msi bound to pirq=%d\n", pirq);
} else {
dev_dbg(&dev->dev,
"xen: msi already bound to pirq=%d\n", pirq);
pirq = xen_allocate_pirq_msi(dev, msidesc);
if (pirq < 0) {
irq = -ENODEV;
goto error;
}
xen_msi_compose_msg(dev, pirq, &msg);
__pci_write_msi_msg(msidesc, &msg);
dev_dbg(&dev->dev, "xen: msi bound to pirq=%d\n", pirq);
irq = xen_bind_pirq_msi_to_irq(dev, msidesc, pirq,
(type == PCI_CAP_ID_MSI) ? nvec : 1,
(type == PCI_CAP_ID_MSIX) ?

6
arch/x86/xen/setup.c
View File

@ -713,10 +713,9 @@ static void __init xen_reserve_xen_mfnlist(void)
size = PFN_PHYS(xen_start_info->nr_p2m_frames);
}
if (!xen_is_e820_reserved(start, size)) {
memblock_reserve(start, size);
memblock_reserve(start, size);
if (!xen_is_e820_reserved(start, size))
return;
}
#ifdef CONFIG_X86_32
/*
@ -727,6 +726,7 @@ static void __init xen_reserve_xen_mfnlist(void)
BUG();
#else
xen_relocate_p2m();
memblock_free(start, size);
#endif
}

6
arch/x86/xen/spinlock.c
View File

@ -27,6 +27,12 @@ static bool xen_pvspin = true;
static void xen_qlock_kick(int cpu)
{
int irq = per_cpu(lock_kicker_irq, cpu);
/* Don't kick if the target's kicker interrupt is not initialized. */
if (irq == -1)
return;
xen_send_IPI_one(cpu, XEN_SPIN_UNLOCK_VECTOR);
}

6
arch/x86/xen/time.c
View File

@ -343,11 +343,11 @@ static int xen_vcpuop_set_next_event(unsigned long delta,
WARN_ON(!clockevent_state_oneshot(evt));
single.timeout_abs_ns = get_abs_timeout(delta);
single.flags = VCPU_SSHOTTMR_future;
/* Get an event anyway, even if the timeout is already expired */
single.flags = 0;
ret = HYPERVISOR_vcpu_op(VCPUOP_set_singleshot_timer, cpu, &single);
BUG_ON(ret != 0 && ret != -ETIME);
BUG_ON(ret != 0);
return ret;
}

12
block/bio.c
View File

@ -373,10 +373,14 @@ static void punt_bios_to_rescuer(struct bio_set *bs)
bio_list_init(&punt);
bio_list_init(&nopunt);
while ((bio = bio_list_pop(current->bio_list)))
while ((bio = bio_list_pop(&current->bio_list[0])))
bio_list_add(bio->bi_pool == bs ? &punt : &nopunt, bio);
current->bio_list[0] = nopunt;
*current->bio_list = nopunt;
bio_list_init(&nopunt);
while ((bio = bio_list_pop(&current->bio_list[1])))
bio_list_add(bio->bi_pool == bs ? &punt : &nopunt, bio);
current->bio_list[1] = nopunt;
spin_lock(&bs->rescue_lock);
bio_list_merge(&bs->rescue_list, &punt);
@ -464,7 +468,9 @@ struct bio *bio_alloc_bioset(gfp_t gfp_mask, int nr_iovecs, struct bio_set *bs)
* we retry with the original gfp_flags.
*/
if (current->bio_list && !bio_list_empty(current->bio_list))
if (current->bio_list &&
(!bio_list_empty(&current->bio_list[0]) ||
!bio_list_empty(&current->bio_list[1])))
gfp_mask &= ~__GFP_DIRECT_RECLAIM;
p = mempool_alloc(bs->bio_pool, gfp_mask);

40
block/blk-core.c
View File

@ -2023,7 +2023,14 @@ generic_make_request_checks(struct bio *bio)
*/
blk_qc_t generic_make_request(struct bio *bio)
{
struct bio_list bio_list_on_stack;
/*
* bio_list_on_stack[0] contains bios submitted by the current
* make_request_fn.
* bio_list_on_stack[1] contains bios that were submitted before
* the current make_request_fn, but that haven't been processed
* yet.
*/
struct bio_list bio_list_on_stack[2];
blk_qc_t ret = BLK_QC_T_NONE;
if (!generic_make_request_checks(bio))
@ -2040,7 +2047,7 @@ blk_qc_t generic_make_request(struct bio *bio)
* should be added at the tail
*/
if (current->bio_list) {
bio_list_add(current->bio_list, bio);
bio_list_add(&current->bio_list[0], bio);
goto out;
}
@ -2059,24 +2066,39 @@ blk_qc_t generic_make_request(struct bio *bio)
* bio_list, and call into ->make_request() again.
*/
BUG_ON(bio->bi_next);
bio_list_init(&bio_list_on_stack);
current->bio_list = &bio_list_on_stack;
bio_list_init(&bio_list_on_stack[0]);
current->bio_list = bio_list_on_stack;
do {
struct request_queue *q = bdev_get_queue(bio->bi_bdev);
if (likely(blk_queue_enter(q, __GFP_DIRECT_RECLAIM) == 0)) {
struct bio_list lower, same;
/* Create a fresh bio_list for all subordinate requests */
bio_list_on_stack[1] = bio_list_on_stack[0];
bio_list_init(&bio_list_on_stack[0]);
ret = q->make_request_fn(q, bio);
blk_queue_exit(q);
bio = bio_list_pop(current->bio_list);
/* sort new bios into those for a lower level
* and those for the same level
*/
bio_list_init(&lower);
bio_list_init(&same);
while ((bio = bio_list_pop(&bio_list_on_stack[0])) != NULL)
if (q == bdev_get_queue(bio->bi_bdev))
bio_list_add(&same, bio);
else
bio_list_add(&lower, bio);
/* now assemble so we handle the lowest level first */
bio_list_merge(&bio_list_on_stack[0], &lower);
bio_list_merge(&bio_list_on_stack[0], &same);
bio_list_merge(&bio_list_on_stack[0], &bio_list_on_stack[1]);
} else {
struct bio *bio_next = bio_list_pop(current->bio_list);
bio_io_error(bio);
bio = bio_next;
}
bio = bio_list_pop(&bio_list_on_stack[0]);
} while (bio);
current->bio_list = NULL; /* deactivate */

Some files were not shown because too many files have changed in this diff Show More