Chuck Lever e28b4fc652 svcrdma: Fix trace point use-after-free race ... I hit this while testing nfsd-5.7 with kernel memory debugging enabled on my server: Mar 30 13:21:45 klimt kernel: BUG: unable to handle page fault for address: ffff8887e6c279a8 Mar 30 13:21:45 klimt kernel: #PF: supervisor read access in kernel mode Mar 30 13:21:45 klimt kernel: #PF: error_code(0x0000) - not-present page Mar 30 13:21:45 klimt kernel: PGD 3601067 P4D 3601067 PUD 87c519067 PMD 87c3e2067 PTE 800ffff8193d8060 Mar 30 13:21:45 klimt kernel: Oops: 0000 [#1] SMP DEBUG_PAGEALLOC PTI Mar 30 13:21:45 klimt kernel: CPU: 2 PID: 1933 Comm: nfsd Not tainted 5.6.0-rc6-00040-g881e87a3c6f9 #1591 Mar 30 13:21:45 klimt kernel: Hardware name: Supermicro Super Server/X10SRL-F, BIOS 1.0c 09/09/2015 Mar 30 13:21:45 klimt kernel: RIP: 0010:svc_rdma_post_chunk_ctxt+0xab/0x284 [rpcrdma] Mar 30 13:21:45 klimt kernel: Code: c1 83 34 02 00 00 29 d0 85 c0 7e 72 48 8b bb a0 02 00 00 48 8d 54 24 08 4c 89 e6 48 8b 07 48 8b 40 20 e8 5a 5c 2b e1 41 89 c6 <8b> 45 20 89 44 24 04 8b 05 02 e9 01 00 85 c0 7e 33 e9 5e 01 00 00 Mar 30 13:21:45 klimt kernel: RSP: 0018:ffffc90000dfbdd8 EFLAGS: 00010286 Mar 30 13:21:45 klimt kernel: RAX: 0000000000000000 RBX: ffff8887db8db400 RCX: 0000000000000030 Mar 30 13:21:45 klimt kernel: RDX: 0000000000000040 RSI: 0000000000000000 RDI: 0000000000000246 Mar 30 13:21:45 klimt kernel: RBP: ffff8887e6c27988 R08: 0000000000000000 R09: 0000000000000004 Mar 30 13:21:45 klimt kernel: R10: ffffc90000dfbdd8 R11: 00c068ef00000000 R12: ffff8887eb4e4a80 Mar 30 13:21:45 klimt kernel: R13: ffff8887db8db634 R14: 0000000000000000 R15: ffff8887fc931000 Mar 30 13:21:45 klimt kernel: FS: 0000000000000000(0000) GS:ffff88885bd00000(0000) knlGS:0000000000000000 Mar 30 13:21:45 klimt kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Mar 30 13:21:45 klimt kernel: CR2: ffff8887e6c279a8 CR3: 000000081b72e002 CR4: 00000000001606e0 Mar 30 13:21:45 klimt kernel: Call Trace: Mar 30 13:21:45 klimt kernel: ? svc_rdma_vec_to_sg+0x7f/0x7f [rpcrdma] Mar 30 13:21:45 klimt kernel: svc_rdma_send_write_chunk+0x59/0xce [rpcrdma] Mar 30 13:21:45 klimt kernel: svc_rdma_sendto+0xf9/0x3ae [rpcrdma] Mar 30 13:21:45 klimt kernel: ? nfsd_destroy+0x51/0x51 [nfsd] Mar 30 13:21:45 klimt kernel: svc_send+0x105/0x1e3 [sunrpc] Mar 30 13:21:45 klimt kernel: nfsd+0xf2/0x149 [nfsd] Mar 30 13:21:45 klimt kernel: kthread+0xf6/0xfb Mar 30 13:21:45 klimt kernel: ? kthread_queue_delayed_work+0x74/0x74 Mar 30 13:21:45 klimt kernel: ret_from_fork+0x3a/0x50 Mar 30 13:21:45 klimt kernel: Modules linked in: ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue ib_umad ib_ipoib mlx4_ib sb_edac x86_pkg_temp_thermal iTCO_wdt iTCO_vendor_support coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel glue_helper crypto_simd cryptd pcspkr rpcrdma i2c_i801 rdma_ucm lpc_ich mfd_core ib_iser rdma_cm iw_cm ib_cm mei_me raid0 libiscsi mei sg scsi_transport_iscsi ioatdma wmi ipmi_si ipmi_devintf ipmi_msghandler acpi_power_meter nfsd nfs_acl lockd auth_rpcgss grace sunrpc ip_tables xfs libcrc32c mlx4_en sd_mod sr_mod cdrom mlx4_core crc32c_intel igb nvme i2c_algo_bit ahci i2c_core libahci nvme_core dca libata t10_pi qedr dm_mirror dm_region_hash dm_log dm_mod dax qede qed crc8 ib_uverbs ib_core Mar 30 13:21:45 klimt kernel: CR2: ffff8887e6c279a8 Mar 30 13:21:45 klimt kernel: ---[ end trace 87971d2ad3429424 ]--- It's absolutely not safe to use resources pointed to by the @send_wr argument of ib_post_send() _after_ that function returns. Those resources are typically freed by the Send completion handler, which can run before ib_post_send() returns. Thus the trace points currently around ib_post_send() in the server's RPC/RDMA transport are a hazard, even when they are disabled. Rearrange them so that they touch the Work Request only _before_ ib_post_send() is invoked. Fixes: bd2abef333 ("svcrdma: Trace key RDMA API events") Fixes: 4201c74647 ("svcrdma: Introduce svc_rdma_send_ctxt") Signed-off-by: Chuck Lever <chuck.lever@oracle.com>		2020-04-17 12:40:38 -04:00
..
acpi	Additional ACPI updates for 5.7-rc1	2020-04-06 10:35:06 -07:00
asm-generic	userfaultfd: wp: add pmd_swp_*uffd_wp() helpers	2020-04-07 10:43:39 -07:00
clocksource	pwm: omap-dmtimer: Drop unused header file	2020-03-30 18:03:06 +02:00
crypto	crypto: curve25519 - do not pollute dispatcher based on assembler	2020-04-09 00:01:59 +09:00
drm	drm/bridge: analogix_dp: Split bind() into probe() and real bind()	2020-04-09 10:29:35 +02:00
dt-bindings	RISC-V Patches for the 5.7 Merge Window, Part 1	2020-04-09 10:51:30 -07:00
keys	KEYS: Don't write out to userspace while holding key semaphore	2020-03-29 12:40:41 +01:00
kunit	kunit: subtests should be indented 4 spaces according to TAP	2020-03-26 14:08:41 -06:00
kvm	KVM: arm64: GICv4.1: Allow SGIs to switch between HW and SW interrupts	2020-03-24 12:15:51 +00:00
linux	Three small fixes/updates for the locking core code:	2020-04-12 09:47:10 -07:00
math-emu
media	media: cec-notifier: make cec_notifier_get_conn() static	2020-03-20 09:02:45 +01:00
misc
net	9p pull request for inclusion in 5.7	2020-04-06 08:46:59 -07:00
pcmcia
ras
rdma	IB/mlx5: Expose UAR object and its alloc/destroy commands	2020-03-27 12:59:04 -03:00
scsi	SCSI misc on 20200402	2020-04-02 17:03:53 -07:00
soc	ARM: driver updates	2020-04-03 15:05:35 -07:00
sound	ASoC: Fixes for v5.7	2020-04-08 18:08:09 +02:00
target	scsi: target: fix hang when multiple threads try to destroy the same iscsi session	2020-03-26 21:47:47 -04:00
trace	svcrdma: Fix trace point use-after-free race	2020-04-17 12:40:38 -04:00
uapi	IOMMU Updates for Linux v5.7	2020-04-08 11:00:00 -07:00
vdso	vdso: Fix clocksource.h macro detection	2020-03-23 18:51:08 +01:00
video
xen	xen: Use evtchn_type_t as a type for event channels	2020-04-07 12:12:54 +02:00