linux/net
Weston Andros Adamson 42e0de9d0e sunrpc: Fix infinite loop in RPC state machine
commit 6ff33b7dd0 upstream.

When a task enters call_refreshresult with status 0 from call_refresh and
!rpcauth_uptodatecred(task) it enters call_refresh again with no rate-limiting
or max number of retries.

Instead of trying forever, make use of the retry path that other errors use.

This only seems to be possible when the crrefresh callback is gss_refresh_null,
which only happens when destroying the context.

To reproduce:

1) mount with sec=krb5 (or sec=sys with krb5 negotiated for non FSID specific
   operations).

2) reboot - the client will be stuck and will need to be hard rebooted

BUG: soft lockup - CPU#0 stuck for 22s! [kworker/0:2:46]
Modules linked in: rpcsec_gss_krb5 nfsv4 nfs fscache ppdev crc32c_intel aesni_intel aes_x86_64 glue_helper lrw gf128mul ablk_helper cryptd serio_raw i2c_piix4 i2c_core e1000 parport_pc parport shpchp nfsd auth_rpcgss oid_registry exportfs nfs_acl lockd sunrpc autofs4 mptspi scsi_transport_spi mptscsih mptbase ata_generic floppy
irq event stamp: 195724
hardirqs last  enabled at (195723): [<ffffffff814a925c>] restore_args+0x0/0x30
hardirqs last disabled at (195724): [<ffffffff814b0a6a>] apic_timer_interrupt+0x6a/0x80
softirqs last  enabled at (195722): [<ffffffff8103f583>] __do_softirq+0x1df/0x276
softirqs last disabled at (195717): [<ffffffff8103f852>] irq_exit+0x53/0x9a
CPU: 0 PID: 46 Comm: kworker/0:2 Not tainted 3.13.0-rc3-branch-dros_testing+ #4
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/31/2013
Workqueue: rpciod rpc_async_schedule [sunrpc]
task: ffff8800799c4260 ti: ffff880079002000 task.ti: ffff880079002000
RIP: 0010:[<ffffffffa0064fd4>]  [<ffffffffa0064fd4>] __rpc_execute+0x8a/0x362 [sunrpc]
RSP: 0018:ffff880079003d18  EFLAGS: 00000246
RAX: 0000000000000005 RBX: 0000000000000007 RCX: 0000000000000007
RDX: 0000000000000007 RSI: ffff88007aecbae8 RDI: ffff8800783d8900
RBP: ffff880079003d78 R08: ffff88006e30e9f8 R09: ffffffffa005a3d7
R10: ffff88006e30e7b0 R11: ffff8800783d8900 R12: ffffffffa006675e
R13: ffff880079003ce8 R14: ffff88006e30e7b0 R15: ffff8800783d8900
FS:  0000000000000000(0000) GS:ffff88007f200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f3072333000 CR3: 0000000001a0b000 CR4: 00000000001407f0
Stack:
 ffff880079003d98 0000000000000246 0000000000000000 ffff88007a9a4830
 ffff880000000000 ffffffff81073f47 ffff88007f212b00 ffff8800799c4260
 ffff8800783d8988 ffff88007f212b00 ffffe8ffff604800 0000000000000000
Call Trace:
 [<ffffffff81073f47>] ? trace_hardirqs_on_caller+0x145/0x1a1
 [<ffffffffa00652d3>] rpc_async_schedule+0x27/0x32 [sunrpc]
 [<ffffffff81052974>] process_one_work+0x211/0x3a5
 [<ffffffff810528d5>] ? process_one_work+0x172/0x3a5
 [<ffffffff81052eeb>] worker_thread+0x134/0x202
 [<ffffffff81052db7>] ? rescuer_thread+0x280/0x280
 [<ffffffff81052db7>] ? rescuer_thread+0x280/0x280
 [<ffffffff810584a0>] kthread+0xc9/0xd1
 [<ffffffff810583d7>] ? __kthread_parkme+0x61/0x61
 [<ffffffff814afd6c>] ret_from_fork+0x7c/0xb0
 [<ffffffff810583d7>] ? __kthread_parkme+0x61/0x61
Code: e8 87 63 fd e0 c6 05 10 dd 01 00 01 48 8b 43 70 4c 8d 6b 70 45 31 e4 a8 02 0f 85 d5 02 00 00 4c 8b 7b 48 48 c7 43 48 00 00 00 00 <4c> 8b 4b 50 4d 85 ff 75 0c 4d 85 c9 4d 89 cf 0f 84 32 01 00 00

And the output of "rpcdebug -m rpc -s all":

RPC:    61 call_refresh (status 0)
RPC:    61 call_refresh (status 0)
RPC:    61 refreshing RPCSEC_GSS cred ffff88007a413cf0
RPC:    61 refreshing RPCSEC_GSS cred ffff88007a413cf0
RPC:    61 call_refreshresult (status 0)
RPC:    61 refreshing RPCSEC_GSS cred ffff88007a413cf0
RPC:    61 call_refreshresult (status 0)
RPC:    61 refreshing RPCSEC_GSS cred ffff88007a413cf0
RPC:    61 call_refresh (status 0)
RPC:    61 call_refreshresult (status 0)
RPC:    61 call_refresh (status 0)
RPC:    61 call_refresh (status 0)
RPC:    61 refreshing RPCSEC_GSS cred ffff88007a413cf0
RPC:    61 call_refreshresult (status 0)
RPC:    61 call_refresh (status 0)
RPC:    61 refreshing RPCSEC_GSS cred ffff88007a413cf0
RPC:    61 call_refresh (status 0)
RPC:    61 refreshing RPCSEC_GSS cred ffff88007a413cf0
RPC:    61 refreshing RPCSEC_GSS cred ffff88007a413cf0
RPC:    61 call_refreshresult (status 0)
RPC:    61 call_refresh (status 0)
RPC:    61 call_refresh (status 0)
RPC:    61 call_refresh (status 0)
RPC:    61 call_refresh (status 0)
RPC:    61 call_refreshresult (status 0)
RPC:    61 refreshing RPCSEC_GSS cred ffff88007a413cf0

Signed-off-by: Weston Andros Adamson <dros@netapp.com>
Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-02-13 13:48:01 -08:00
..
9p 9p: send uevent after adding/removing mount_tag attribute 2013-12-04 10:57:19 -08:00
802 net/802/mrp: fix lockdep splat 2013-05-14 13:02:30 -07:00
8021q vlan: Fix header ops passthru when doing TX VLAN offload. 2014-01-15 15:28:49 -08:00
appletalk net: rework recvmsg handler msg_name and msg_namelen logic 2013-12-08 07:29:25 -08:00
atm net: rework recvmsg handler msg_name and msg_namelen logic 2013-12-08 07:29:25 -08:00
ax25 net: rework recvmsg handler msg_name and msg_namelen logic 2013-12-08 07:29:25 -08:00
batman-adv batman-adv: set up network coding packet handlers during module init 2013-11-20 12:27:47 -08:00
bluetooth net: rework recvmsg handler msg_name and msg_namelen logic 2013-12-08 07:29:25 -08:00
bridge bridge: use spin_lock_bh() in br_multicast_set_hash_max 2014-01-15 15:28:50 -08:00
caif net: rework recvmsg handler msg_name and msg_namelen logic 2013-12-08 07:29:25 -08:00
can Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2013-05-01 17:51:54 -07:00
ceph libceph: add function to ensure notifies are complete 2014-01-09 12:24:26 -08:00
core fuse: fix pipe_buf_operations 2014-02-13 13:47:59 -08:00
dcb rtnetlink: Remove passing of attributes into rtnl_doit functions 2013-03-22 10:31:16 -04:00
dccp net:dccp: do not report ICMP redirects to user space 2013-10-13 16:08:30 -07:00
decnet decnet: remove duplicated include from dn_table.c 2013-04-07 17:12:01 -04:00
dns_resolver Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security 2012-12-16 15:40:50 -08:00
dsa dsa: fix freeing of sparse port allocation 2013-03-25 12:23:41 -04:00
ethernet net: add ETH_P_802_3_MIN 2013-03-28 01:20:42 -04:00
ieee802154 ieee802154: Fix memory leak in ieee802154_add_iface() 2014-02-06 11:08:16 -08:00
ipv4 net: Fix memory leak if TPROXY used with TCP early demux 2014-02-06 11:08:17 -08:00
ipv6 ip6tnl: fix double free of fb_tnl_dev on exit 2014-02-06 11:08:17 -08:00
ipx net: rework recvmsg handler msg_name and msg_namelen logic 2013-12-08 07:29:25 -08:00
irda net: rework recvmsg handler msg_name and msg_namelen logic 2013-12-08 07:29:25 -08:00
iucv net: rework recvmsg handler msg_name and msg_namelen logic 2013-12-08 07:29:25 -08:00
key net: rework recvmsg handler msg_name and msg_namelen logic 2013-12-08 07:29:25 -08:00
l2tp inet: fix addr_len/msg->msg_namelen assignment in recv_error and rxpmtu functions 2013-12-08 07:29:25 -08:00
lapb net/lapb: remove depends on CONFIG_EXPERIMENTAL 2013-01-11 11:40:01 -08:00
llc net: llc: fix use after free in llc_ui_recvmsg 2014-01-15 15:28:50 -08:00
mac80211 mac80211: move "bufferable MMPDU" check to fix AP mode scan 2014-01-15 15:28:53 -08:00
mac802154 Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2013-04-30 03:55:20 -04:00
netfilter netfilter: nf_nat: fix access to uninitialized buffer in IRC NAT helper 2014-01-15 15:28:53 -08:00
netlabel netlabel: improve domain mapping validation 2013-05-19 14:49:55 -07:00
netlink net: rework recvmsg handler msg_name and msg_namelen logic 2013-12-08 07:29:25 -08:00
netrom net: rework recvmsg handler msg_name and msg_namelen logic 2013-12-08 07:29:25 -08:00
nfc net: rework recvmsg handler msg_name and msg_namelen logic 2013-12-08 07:29:25 -08:00
openvswitch openvswitch: Remove unneeded ovs_netdev_get_ifindex() 2013-04-30 00:19:11 -04:00
packet packet: fix send path when running with proto == 0 2014-01-15 15:28:46 -08:00
phonet inet: prevent leakage of uninitialized memory to user in recv syscalls 2013-12-08 07:29:25 -08:00
rds net: rds: fix per-cpu helper usage 2014-02-06 11:08:16 -08:00
rfkill Merge branch 'for-john' of git://git.kernel.org/pub/scm/linux/kernel/git/jberg/mac80211-next 2013-04-22 14:58:14 -04:00
rose net: rose: restore old recvmsg behavior 2014-01-15 15:28:49 -08:00
rxrpc net: rework recvmsg handler msg_name and msg_namelen logic 2013-12-08 07:29:25 -08:00
sched net_sched: htb: fix a typo in htb_change_class() 2013-10-13 16:08:29 -07:00
sctp sctp: Perform software checksum if packet has to be fragmented. 2013-11-04 04:31:04 -08:00
sunrpc sunrpc: Fix infinite loop in RPC state machine 2014-02-13 13:48:01 -08:00
tipc net: rework recvmsg handler msg_name and msg_namelen logic 2013-12-08 07:29:25 -08:00
unix net: unix: allow bind to fail on mutex lock 2014-01-15 15:28:48 -08:00
vmw_vsock net: rework recvmsg handler msg_name and msg_namelen logic 2013-12-08 07:29:25 -08:00
wimax
wireless radiotap: fix bitmap-end-finding buffer overrun 2014-01-09 12:24:23 -08:00
x25 net: rework recvmsg handler msg_name and msg_namelen logic 2013-12-08 07:29:25 -08:00
xfrm xfrm: force a garbage collection after deleting a policy 2013-05-31 17:30:07 -07:00
compat.c x86, x32: Correct invalid use of user timespec in the kernel 2014-02-06 11:08:12 -08:00
Kconfig netlink: kconfig: move mmap i/o into netlink kconfig 2013-05-01 15:02:42 -04:00
Makefile VSOCK: Introduce VM Sockets 2013-02-10 19:41:08 -05:00
nonet.c
socket.c net: clamp ->msg_namelen instead of returning an error 2013-12-08 07:29:25 -08:00
sysctl_net.c net: Update the sysctl permissions handler to test effective uid/gid 2013-10-13 16:08:34 -07:00