scx_call_op_set_cpumask() builds a per-cpu cmask in the set_cmask scratch,
which lives in BPF-writable arena. A scheduler can corrupt the scratch's
inline header (base, nr_cids, alloc_words) from another cpu, so sizing and
indexing the write from it risks an out-of-bounds write.
Drive the build from kernel-known geometry instead.
scx_cmask_ref_init_kern() imposes base and nr_cids rather than reading them,
and scx_cmask_ref_from_cpumask() fills the scratch from the ref. Neither
reads the header back.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
kfuncs taking struct scx_cmask * from BPF arena memory have two problems.
The pointer can be any value the BPF prog hands in, and the header (@base,
@nr_cids, @alloc_words) can be mutated by the prog concurrently with kernel
access.
Add scx_cmask_ref, a validated handle. _init() normalizes the input pointer
into the arena's kern_vm range via scx_arena_to_kaddr() and snapshots the
header, rejecting a range outside the machine or a nr_cids whose words
exceed the declared @alloc_words. Downstream sizing uses the snapshot, not
the live header. _shard() reads slices while _or() and _copy() write back,
all bounded by the snapshot. No callers yet.
struct scx_cmask's bits[] carried __counted_by(alloc_words), so
UBSAN_BOUNDS and FORTIFY_SOURCE bound accesses to the array. That bound is
read from @alloc_words at the access. For an arena cmask @alloc_words is
BPF-writable. A prog that sets it larger than the real allocation makes the
check pass on a genuine overrun, so the annotation catches nothing, and it
only runs under those debug configs. Drop it - _init() validates
@alloc_words explicitly, and kernel-owned cmasks set it themselves.
v2: Validate @alloc_words in _init(), drop __counted_by. (Andrea, sashiko AI)
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Add struct scx_pshard and sch->pshard[] indexed by shard_idx, each entry
allocated on its shard's NUMA node from scx_shard_node[si]. The struct
starts empty (one dummy field). Follow-up patches will grow it as
shard-local state lands. Only cid-type schedulers with an arena pool get
pshards.
Allocation happens after ops.init_cids() returns so any
scx_bpf_cid_override() it issues has finalized scx_nr_cid_shards and
scx_shard_node[]. sch->nr_pshards records the array size for the async RCU
free path, which may run after a later scheduler's scx_cid_init() has
rewritten the global.
v3: Build and publish pshard[] fully-formed here rather than a later patch.
v2: Free the partially-allocated pshard array on alloc failure. (sashiko AI)
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Split kobject_init_and_add() in scx_alloc_and_add_sched(): only
kobject_init() runs there. A new scx_sched_sysfs_add() helper does
kobject_add() (and creates sub_kset when the scheduler implements
ops.sub_attach), called by both enable workfns once @sch is linked and its
sysfs-visible state is initialized. Prep so a future caps attribute can rely
on @sch being fully built by the time it's sysfs-visible. Add early enough
that a stall later in enable still leaves sysfs inspectable.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
An overridden cid mapping invalidates the auto-generated shard layout, so
the override call has to provide both. Extend scx_bpf_cid_override() with a
shard_start[] array that lists the first cid of each shard (starting at 0,
strictly increasing, last shard implicitly extends to num_possible_cpus()).
A scheduler that wants only custom shards with the auto-generated cid
mapping can read the current mapping and pass it back unchanged.
Overridden shards can span NUMA nodes, so scx_shard_node[] is rebuilt by
majority count: each shard is assigned to the node that owns the most cpus
in it.
v2: Snapshot the caller's cpu_to_cid/shard_start arrays before validating. (sashiko AI)
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Sub-sched operations need a scalable locking / work domain smaller than the
whole cid space. Carve the cid space into topology-respecting shards: each
shard is a contiguous cid range that stays within one LLC, and LLCs larger
than the per-shard cap (default 24 cids, configurable via
ops.cid_shard_size) split into enough shards to fit. A hard cap of
SCX_CID_SHARD_MAX_CPUS prevents pathological sizes under custom
configurations.
No-topo cids pack into their own shards so every cid has a shard assignment.
Also build scx_cid_shard_ranges[] for O(1) shard-to-cid-range lookup and
scx_shard_node[] so callers can size or place work by NUMA without walking
cids. Auto-built shards inherit their LLC's node. No-topo shards carry
NUMA_NO_NODE.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
A cid-form scheduler that calls scx_bpf_cid_override() to install a custom
cid layout can only do so from ops.init(). Enable-path setup that depends on
the cid layout thus has to run after ops.init(), and ops.init() itself can't
use anything derived from the final layout, which turned out to be too
restrictive.
Add an ops.init_cids() callback dedicated to finalizing the cid layout. It
runs before the rest of the enable-path setup, so the final layout is in
effect for everything that follows including ops.init(), which now runs
after the arena pool and cmask scratch allocations.
scx_bpf_cid_override() is restricted to ops.init_cids() at load time. It
sits in a kfunc set gated by SCX_KF_ALLOW_INIT_CIDS, a flag set only on the
init_cids op, so the verifier rejects a call from any other context. The
runtime root-only check is dropped as ops.init_cids() only runs during root
enable.
The qmap demo moves its override into a dedicated qmap_init_cids() and,
while at it, introduces an enum for the cid override modes instead of
hard-coded integers.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
The kick machinery kept its targets in rq->scx shared by every sched on the
cpu. A preempt kick carried no record of which scheduler requested it.
A later patch needs preempt kicks scoped to the requesting scheduler so a
sub-scheduler can preempt only tasks in its own subtree. Move the kick masks
into the per-sched per-cpu scx_sched_pcpu and have scx_kick_cpu() link the
sched onto a per-cpu list (rq->scx.sched_pcpus_to_kick). The cpu's single
kick irq_work walks that list and kicks each sched's targets on its behalf,
so a kick stays attributed to its scheduler.
The SCX_KICK_WAIT sync set (cpus_to_sync, the kick_sync snapshot and the
balance-callback trigger) stays in rq->scx: the waiter is the cpu, not the
scheduler, and its only writers, the kick irq_work and the wait balance
callback, are cpu-local.
On disable, free_kick_syncs() flushes each cpu's pending kick irq_work
before clearing @ksyncs, so a late kick unlinks its to_kick_node instead of
early-returning on a NULL @ksyncs and leaving the node linked at free.
v3: Flush the kick irq_work in free_kick_syncs() before clearing @ksyncs. (sashiko AI)
v2: Warn once per sched on scx_bpf_kick_cpu() from NMI.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
A per-task op must be dispatched on the scheduler that owns the task.
SCX_CALL_OP_TASK() and its _RET twin take @sch explicitly, and a caller that
passes the wrong scheduler would silently run the op on it. Add a
WARN_ON_ONCE() that @sch matches the task's owner so such a mismatch is
caught rather than hidden.
Two sites legitimately target a scheduler other than the task's owner:
cgroup_move() runs on the root sched, and scx_sub_init_cancel_task() fires
exit_task() on a task not yet associated with @sch. Both switch to the inner
__SCX_CALL_OP_TASK(), which dispatches on the explicit @sch without the
assert.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
In fcg_read_stats(), the memset() that zeroes the output @stats array
sits after the calloc() failure check. When calloc() fails, the
function returns without writing @stats.
The caller in main() declares acc_stats uninitialized, passes it as
the @stats argument, and then reads it unconditionally:
__u64 acc_stats[FCG_NR_STATS];
fcg_read_stats(skel, acc_stats);
stats[i] = acc_stats[i] - last_stats[i]; // reads garbage
Because fcg_read_stats() returns void, the caller cannot detect the
failure. Reading the uninitialized array is undefined behavior, and
the garbage is further copied into last_stats via memcpy(), corrupting
the baseline used by the next interval.
This regression was introduced by commit cabd76bbc0 ("tools/sched_ext:
scx_flatcg: fix potential stack overflow from VLA in fcg_read_stats"),
which replaced the VLA with calloc() and inserted the failure check
before the existing memset().
Move the memset() above the calloc() failure check so @stats is always
zeroed regardless of allocation outcome.
Fixes: cabd76bbc0 ("tools/sched_ext: scx_flatcg: fix potential stack overflow from VLA in fcg_read_stats")
Signed-off-by: Liang Luo <luoliang@kylinos.cn>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
sched_ext schedulers have state in BPF programs and kernel. scx_dump
provides kernel state and BPF program state on error, but this is static
in what it can provide.
Add a sched_ext_exit tracepoint in scx_claim_exit() so that BPF programs
can dynamically inspect scheduler specific state at the moment of exit.
Pass the exiting scx_sched so attached programs can read its state, and,
since exits propagate through a hierarchy of sub-schedulers, identify
which scheduler each event belongs to.
Signed-off-by: Pat Somaru <patso@likewhatevs.io>
Signed-off-by: Tejun Heo <tj@kernel.org>
consume_remote_task() drops this_rq before unlinking a remote task from
its DSQ and locking src_rq. When called from ops.dispatch(),
scx_locked_rq() keeps pointing to this_rq across the lock dance. The
subsequent switch_rq_lock(src_rq, this_rq) cannot update the tracking
because its guard does not match, leaving the tracking stale while
src_rq is held.
Keep this_rq locked until the task has been unlinked and the DSQ lock
released, then use switch_rq_lock() to switch directly to src_rq. Use
the same helper to restore this_rq after losing the dequeue race. The
successful path already switches back through
move_remote_task_to_local_dsq(), so scx_locked_rq() now follows the
actually held rq throughout the consume path.
Suggested-by: Tejun Heo <tj@kernel.org>
Link: https://lore.kernel.org/all/455e701bca66bdecde530d225f4dba0a@kernel.org
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
scx_bpf_kick_cid() returned an error code, but the value conveys nothing
actionable and no caller consumes it. The kick is asynchronous, so a
successful return only means it was queued. An invalid @cid is already
reported through scx_error() by scx_cid_to_cpu(), and a missing scheduler
leaves nothing to kick.
Make scx_bpf_kick_cid() return void to match scx_bpf_kick_cpu(). The
cid-form kfuncs are not in practical use yet, so the ABI change is safe.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Direct writes to p->scx.slice and p->scx.dsq_vtime bypass
scx_bpf_task_set_slice/dsq_vtime() and the authority checks they carry.
Those checks exist for sub-schedulers, which attach only through the
cid-form struct_ops, so the direct writes only need to be closed there.
Give sched_ext_ops_cid its own verifier ops that reject the two fields.
cid-form is a new interface with no legacy users, so there is no
compatibility to keep. The cpu-form keeps direct writes, and the deprecation
warning they carried is dropped.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
scx_qmap.c and the shared scx_qmap.h mixed __u64/__u32/__s32 with the bare
typedefs that scx/common.h provides. Convert the remaining __-prefixed
integer types to the bare forms for consistency. The struct fields become
bare u64 (uint64_t), so the stats printfs that fed them to %llu now cast to
unsigned long long. No functional change.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
scx_rq.extra_enq_flags carries scx-specific enqueue flags across the
activate_task() boundary during a cross-rq task move in
move_remote_task_to_local_dsq(). Rename it to remote_activate_enq_flags to
name that role, and fix the stale comment that referenced the old
move_task_to_local_dsq() name.
No functional change.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
The cid-form ops overlay their cpu-form siblings at the same struct slot.
Ops whose signature matches the sibling are invoked through the cpu-form
call sites unchanged, but set_cmask() takes an arena cmask address rather
than a cpumask, so scx_call_op_set_cpumask() calls ops_cid.set_cmask()
directly and hand-rolled the kf_tasks[] and locked_rq bracket that
SCX_CALL_OP_TASK() provides. The hand-rolled bracket reset locked_rq to
NULL on exit instead of restoring the saved value, so a nested call would
clobber the outer op's locked-rq tracking.
Parameterize the dispatch macros by the ops-table member and add
SCX_CALL_CID_OP_TASK(), which routes through sch->ops_cid. Convert
scx_call_op_set_cpumask() to it and drop the hand-rolled bracket. The only
behavioral change is that locked_rq is now saved and restored like every
other op call site.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
The cmask ops can operate on BPF-arena cmasks which BPF programs may read
and write concurrently. The _RACY op variants existed to make such lockless
reads explicit but this turned out to be too restrictive. Mark the word
accesses in all the two-cmask ops with READ_ONCE/WRITE_ONCE instead and drop
the _RACY variants.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Pull to receive:
db4e9defd2 ("sched_ext: Record an error on errno-only sub-enable failure")
49b3378a75 ("sched_ext: Fix premature ops->priv publication in scx_alloc_and_add_sched()")
e6979d05c6 ("tools/sched_ext: scx - Fix cmask_subset(), cmask_equal() and cmask_weight()")
for further sub-sched changes and to resolve the conflicts with the
sub-sched updates on for-7.3.
db4e9defd2 adds scx_error() to the sub-enable err_disable sink which
for-7.3 moved from ext.c into sub.c. Resolved by applying the fix to
scx_sub_enable_workfn() in sub.c.
49b3378a75 drops RCU_INIT_POINTER() from an scx_alloc_and_add_sched()
unwind label whose body changed with for-7.3's stall_cpus addition.
Resolved by dropping the line from the updated unwind.
Signed-off-by: Tejun Heo <tj@kernel.org>
cmask_equal(), cmask_weight() and cmask_subset() bounded their word walks
with CMASK_NR_WORDS(nr_cids), which pads by one word and can't tell the last
word in use without @base. The walks could thus cover a slack word past the
active range, which cmask_reframe() leaves non-zero: a stale bit there gave
cmask_equal() a spurious mismatch, cmask_weight() an inflated count, and
cmask_subset() a spurious violation. cmask_subset() could also read
@b->bits[] one word past its allocation (within the arena's fault-recovered
range, so harmless), and deviated from the kernel scx_cmask_subset() by
failing any @a range that doesn't nest inside @b's even when the overhanging
bits are all clear.
Bound the cmask_equal() and cmask_weight() walks by the words the range
actually spans, with early returns for empty ranges. Rewrite cmask_subset()
to match the kernel semantics: scan @a's overhangs for set bits with
cmask_next_set() and walk the words of the range intersection.
cmask_subset() moves below cmask_next_set(), which it now uses. Padding bits
don't need masking as every cmask helper keeps them clear.
Fixes: a58e6b79b4 ("sched_ext: Add cmask, a base-windowed bitmap over cid space")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
scx_alloc_and_add_sched() publishes @sch through ops->priv before allocating
the cgroup path. If that allocation fails, the unwind path clears ops->priv
and frees @sch immediately. scx_prog_sched() callers can dereference
ops->priv from RCU context the moment it is set, so freeing without a grace
period can use-after-free a concurrent kfunc caller.
Move the publication below the cgroup path allocation so that every failure
path after publication frees @sch through kobject_put(), whose release path
defers the freeing by a grace period.
Fixes: 105dcd005b ("sched_ext: Introduce scx_prog_sched()")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
scx_sub_enable_workfn() has several failure paths that only return an errno
(e.g. -ENOMEM from an allocation) and jump to err_disable without calling
scx_error(). scx_flush_disable_work() runs the disable, and thus ops.exit(),
only when an error has been recorded, so an errno-only failure leaves the
half-initialized sub-scheduler linked.
Record an error at the err_disable sink so every errno-only failure runs the
disable path.
Fixes: ebeca1f930 ("sched_ext: Introduce cgroup sub-sched support")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
In prog_run's run(), the bpf_link is attached early but only destroyed
on the success path. The three SCX_EQ assertions between attach and
destroy expand to a direct 'return SCX_TEST_FAIL', so if any of them
triggers, bpf_link__destroy() is never reached and the BPF scheduler
stays loaded. All subsequent tests then fail to attach because SCX is
not in the DISABLED state.
Convert those assertions to explicit checks that jump to a unified
'out' label which always runs the cleanup, matching the pattern used
in cyclic_kick_wait.c.
Fixes: a5db7817af ("sched_ext: Add selftests")
Signed-off-by: Liang Luo <luoliang@kylinos.cn>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
The comment for scx_bpf_dsq_insert() references "@dsp_id" in the
description body, but the parameter is named "@dsq_id" in both the
parameter list and the function signature.
Signed-off-by: Liang Luo <luoliang@kylinos.cn>
Acked-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
Finite-slice EXT tasks need the periodic scheduler tick to expire their
slices even when nohz_full is enabled.
Add a regression test that selects a nohz_full CPU and exercises both
infinite-to-finite and finite-to-finite slice transitions across an idle
interval. For each finite task, verify that its ops.tick() callback is
invoked.
Skip the test when an allowed nohz_full CPU and a separate housekeeping
CPU are not available.
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
set_next_task_scx() updates the tick dependency before __schedule()
updates rq->curr. When switching from a non-EXT task, such as idle, to
an EXT task with a finite slice, sched_update_tick_dependency() checks
the outgoing task and can allow the tick to remain stopped.
The dependency can also be lost without a slice-type transition. After a
finite-slice task leaves the CPU idle, the enqueue path can clear the
dependency against the idle rq->curr. SCX_RQ_CAN_STOP_TICK still records
a finite slice, so another finite task skips the transition block and
can run without the ticks needed to expire its slice.
The reverse mismatch can also happen when the last finite-slice EXT task
is dequeued: sub_nr_running() updates the dependency before rq->curr
changes, so the outgoing task state can keep the dependency set after
the CPU goes idle.
Fix this by unconditionally enabling the scheduler tick whenever a
finite-slice EXT task is selected on a nohz_full CPU. Moreover, when the
last runnable EXT task leaves, ignore the outgoing EXT slice state so
the generic scheduler can correctly re-evaluate and clear the tick
dependency.
Fixes: 22a920209a ("sched_ext: Implement tickless support")
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
dispatch_to_local_dsq() can run from scx_bpf_dsq_move_to_local() while
ops.dispatch() has recorded the current rq. Moving a task to a local DSQ
may switch to the source or destination rq before synchronously invoking
ops.dequeue() through the following path:
SCX_CALL_OP(dispatch, rq)
ops.dispatch()
scx_bpf_dsq_move_to_local()
scx_flush_dispatch_buf()
finish_dispatch()
dispatch_to_local_dsq()
scx_dispatch_enqueue()
local_dsq_post_enq()
call_task_dequeue()
SCX_CALL_OP_TASK(dequeue, locked_rq, ...)
The nested callback saves the recorded rq and restores it on return. If
the rq tracking does not follow the lock switch, update_locked_rq() can
trigger the following lockdep assertion while restoring an rq which is
no longer held:
WARNING: kernel/sched/sched.h:1641 at call_task_dequeue+0x160/0x170
Call Trace:
scx_dispatch_enqueue+0x2b0/0x460
dispatch_to_local_dsq+0x138/0x230
scx_flush_dispatch_buf+0x1af/0x220
scx_bpf_dsq_move_to_local___v2+0xe2/0x1c0
bpf__sched_ext_ops_dispatch+0x4b/0xa7
do_pick_task_scx+0x3b6/0x910
__pick_next_task+0x105/0x1f0
__schedule+0x3e7/0x1980
Introduce switch_rq_lock() to update the tracking state together with
each rq lock handoff. Use it in dispatch_to_local_dsq(),
move_remote_task_to_local_dsq() and the in-balance paths of
scx_dsq_move(), ensuring that scx_locked_rq() consistently refers to the
rq whose lock is actually held throughout the lock dance.
Fixes: 7fb39e4eb4 ("sched_ext: Save and restore scx_locked_rq across SCX_CALL_OP")
Cc: stable@vger.kernel.org # 7.1+
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
The "Where to Look" and "ABI Instability" sections state that the ops
table is defined in include/linux/sched/ext.h. However, struct
sched_ext_ops is actually defined in kernel/sched/ext/internal.h, along
with the SCX_OPS_* flags; include/linux/sched/ext.h holds the core data
structures (struct sched_ext_entity, struct scx_dispatch_q, ...) and the
DSQ constants. Point the ops table references to the correct header.
Signed-off-by: Liang Luo <luoliang@kylinos.cn>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
The sub-scheduler implementation has grown and will continue to expand. Move
the sub-scheduler functions from ext.c into a new kernel/sched/ext/sub.c.
sub.h holds the prototypes and the !CONFIG_EXT_SUB_SCHED no-op stubs.
scx_dispatch_sched() is shared: balance_one() in ext.c and the
scx_bpf_sub_dispatch() kfunc in sub.c both call it, and the latter re-enters
it as sub-scheduler dispatch nests. It moves into sub.h as a static
__always_inline so both callers keep it inlined and per-level stack stays
bounded across the recursion. The event macros it uses move to internal.h.
No functional change.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
The following trivial helpers in ext.c are called from both ext.c and the
sub-scheduler code. Define them as static inline in internal.h.
- scx_bypass_dsq()
- scx_bypass_dsp_enabled()
- scx_ops_sanitize_err()
- scx_schedule_reenq_local()
No functional change.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
The sub-scheduler implementation is about to move into its own sub.c, from
where it calls a set of ext.c helpers and shares a few ext.c globals. Make
those reachable across the new file boundary ahead of the move.
No functional change.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
A later change moves the sub-scheduler implementation out of ext.c into its
own file, from where it calls a number of file-local ext.c helpers. Give
those helpers the scx_ prefix that cross-file sched_ext symbols carry, ahead
of the move so the mechanical rename stays out of the code-motion patch. No
functional change.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Fix checkpatch "Macros with complex values should be enclosed in
parentheses" error.
Suggested-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Cheng-Yang Chou <yphbchou0911@gmail.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
Three sites enumerate every scx_event_stats field by name:
scx_read_events(), scx_attr_events_show(), and scx_dump_state().
Adding a new SCX_EV_* event requires three separate manual edits.
Missing any one silently drops counts or omits the field from
diagnostic output with no compile-time indication.
SCX_EVENTS_LIST(SCX_EVENT) in internal.h becomes the single
authoritative list of event names, so all three sites stay in sync.
No functional changes.
Signed-off-by: Cheng-Yang Chou <yphbchou0911@gmail.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
__COMPAT_read_enum() and __COMPAT_struct_has_field() open-code the
vlen lookup via the raw BTF_INFO_VLEN(t->info) UAPI macro. libbpf
exposes btf_vlen() for exactly this purpose; use it in the three
call sites, matching the pattern in kernel/bpf/inode.c and
tools/bpf/bpftool.
btf_vlen() returns __u32 (since commit cacd6729c0, "libbpf:
Adjust btf_vlen() to return a __u32", which expanded the BTF vlen
field from 16 to 24 bits). Declare the loop counters as __u32 to
match the return type, keeping the comparison as a plain
'__u32 < __u32' and silencing the -Wsign-compare warnings.
No functional change.
Suggested-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Liang Luo <luoliang@kylinos.cn>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
put_prev_task_scx() warns when a runnable task drops to a lower sched_class
without SCX_OPS_ENQ_LAST, on the assumption that balance_one() would have
kept it running. Core scheduling breaks that: a forced-idle SMT sibling
reschedules through the core_pick fast path in pick_next_task(), which skips
pick_task_scx() and thus balance_one(), so a runnable task can drop to idle
with ENQ_LAST unset.
Gate the warning on sched_cpu_cookie_match(): a cookie mismatch means core
scheduling forced the idle, while a match (or core scheduling off) still
catches a genuine missing-ENQ_LAST drop.
Fixes: 7c65ae81ea ("sched_ext: Don't call put_prev_task_scx() before picking the next task")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
A child sub-scheduler dereferences its parent scx_sched throughout its life,
e.g., in scx_sub_disable() which reparents the child's tasks and calls
parent->ops.sub_detach() after unlinking from the parent. However, the
parent is pinned only through parent->sub_kset, which is dropped during
disable. The parent scx_sched can be RCU-freed while a child is still
disabling.
Take a direct reference on the parent in scx_alloc_and_add_sched(), dropped
in scx_sched_free_rcu_work(), so a parent always outlives its descendants.
Signed-off-by: Tejun Heo <tj@kernel.org>
Four example schedulers (scx_simple, scx_cpu0, scx_sdt, scx_userland)
declare the variable that holds getopt()'s return value as __u32.
getopt() returns int and uses -1 to mark end-of-options; storing that
sentinel in an unsigned variable turns it into 0xffffffff, and the
subsequent 'opt != -1' test only happens to keep working because both
operands of != are promoted to the same 0xffffffff.
Declare the variable as __s32 instead, matching getopt()'s actual
contract and the style already used in scx_qmap.c (int opt) and
scx_pair/central/flatcg.c (__s32 opt). This also silences the
-Wsign-compare warnings emitted for the affected files.
Signed-off-by: Liang Luo <luoliang@kylinos.cn>
Signed-off-by: Tejun Heo <tj@kernel.org>
scx_arena_grow() was left open-coded when 2e05f2fd0d ("sched_ext: Add
scx_arena_to_kaddr() / scx_kaddr_to_arena()") introduced the helper.
Replace the manual bpf_arena_map_kern_vm_start() fetch and #ifdef guard
with scx_arena_to_kaddr(sch, p).
Signed-off-by: Cheng-Yang Chou <yphbchou0911@gmail.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
scx_rcu_cpu_stall() previously recorded the detector CPU rather than the
stalled one, and the expedited grace period path had no stalled CPU to
report at all.
Thread a cpumask through panic_on_rcu_stall() and scx_rcu_cpu_stall()
to capture all stalled CPUs. Report cpumask_first() as exit_cpu and the
full CPU list in the exit message. Task-only stalls yield exit_cpu = -1.
Store the stall mask in scx_sched rather than scx_exit_info, keeping the
BPF-visible struct unchanged. scx_dump_state() reads sch->stall_cpus
directly and dumps all stalled CPUs first to avoid losing them to
truncation.
Signed-off-by: Cheng-Yang Chou <yphbchou0911@gmail.com>
Reviewed-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
handle_lockup() uses raw_smp_processor_id() for exit_cpu, which is wrong
for two paths:
- scx_hardlockup_irq_workfn() has the hung CPU in a local variable but
irq_work may run elsewhere. Pass the local cpu explicitly.
- scx_rcu_cpu_stall() records the detector CPU rather than the stalled
one. Pass -1 for now. The next patch fixes this properly.
Signed-off-by: Cheng-Yang Chou <yphbchou0911@gmail.com>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
1. cgrp_cap_budget() used __sync_fetch_and_sub(&cgc->cvtime_delta,
cgc->cvtime_delta) to atomically read and clear cvtime_delta. However,
this is not a true atomic read-clear operation: the second argument
(cgc->cvtime_delta) is evaluated as a normal read before the atomic
fetch_and_sub executes. If a concurrent __sync_fetch_and_add() happens
between the read and the sub, the added value gets included in the
returned delta AND remains in cvtime_delta, causing double charging.
Example:
CPU 0 runs cgrp_cap_budget(), CPU 1 runs fcg_stopping().
Assume cvtime_delta = 100 initially.
T1 CPU 0: sub_val = cvtime_delta = 100 cvtime_delta = 100
T2 CPU 1: __sync_fetch_and_add(&cvtime_delta, 10) cvtime_delta = 110
T3 CPU 0: __sync_fetch_and_sub(&cvtime_delta, sub_val) cvtime_delta = 10
returns old=110
delta = 110 (includes the 10 from CPU 1), but cvtime_delta = 10
(the 10 also remains). The 10 is charged twice: once in delta
(applied to cgv_node->cvtime) and once in the residual cvtime_delta
(fetched again next time).
Fix by using __sync_fetch_and_and(&cgc->cvtime_delta, 0).
Disassembly comparison:
(1) delta = __sync_fetch_and_sub(&cgc->cvtime_delta, cgc->cvtime_delta);
228: (79) r7 = *(u64 *)(r9 +40)
229: (87) r7 = -r7
230: (db) r7 = atomic64_fetch_add((u64 *)(r9 +40), r7) //r9 may be changed
(2) delta = __sync_fetch_and_and(&cgc->cvtime_delta, 0);
228: (b7) r8 = 0
229: (db) r8 = atomic64_xchg((u64 *)(r9 +40), r8)
2. The bypass charging path in fcg_stopping() charges raw execution time
to cvtime_delta without scaling by the inverse of the cgroup hweight.
Since cvtime_delta is eventually applied to cgv_node->cvtime which is
in vtime space (weight-scaled), the bypass path should also scale by
FCG_HWEIGHT_ONE / hweight to match the units used by the dispatch path.
Fixes: a4103eacc2 ("sched_ext: Add a cgroup scheduler which uses flattened hierarchy")
Signed-off-by: Wanwu Li <liwanwu@kylinos.cn>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
scx_bpf_cpu_rq() exposes rq pointers without requiring the rq lock and
has emitted a deprecation warning since commit
5c48d88fe0 ("sched_ext: deprecation warn for scx_bpf_cpu_rq()").
The supported replacements cover the intended uses:
scx_bpf_locked_rq() for locked rq access and
scx_bpf_cpu_curr() for remote curr lookup.
Remove the kfunc, its BTF registrations, the deprecation warning
state, and the BPF-side prototype and compat fallback.
Signed-off-by: Christian Loehle <christian.loehle@arm.com>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Reviewed-by: Hongyan Xia <hongyan.xia@transsion.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
scx_kick_syncs is a per-CPU __rcu pointer, so per_cpu_ptr() returns struct
scx_kick_syncs __rcu **. alloc_kick_syncs() and free_kick_syncs() stored it
in a plain struct scx_kick_syncs **ksyncs, which sparse flags as an __rcu
address-space mismatch. Annotate ksyncs to match. Its accesses already go
through rcu_*_pointer().
Fixes: 987e00035c ("sched_ext: Rename pnt_seq to kick_sync")
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202606122315.pbnDHP0n-lkp@intel.com/
Signed-off-by: Tejun Heo <tj@kernel.org>
task_can_run_on_remote_rq() operates under the assumption that
p->migration_disabled is stable, i.e. if the kernel observed
is_migration_disabled(p) == true, then the BPF scheduler must have also
been able to see this when dispatching the task, and it's the BPF
scheduler's fault that it tried to dispatch a task with migration
disabled to a CPU other than the task's current CPU.
This assumption does not always hold. It's possible that the BPF
scheduler saw is_migration_disabled(p) == false, while the kernel
observes is_migration_disabled(p) == true in dispatch_to_local_dsq()
-> task_can_run_on_remote_rq().
The crucial thing here is that with CONFIG_PREEMPT_RCU, migration is
disabled while a task is executing a BPF program. So, if there's a
situation where the BPF scheduler checks a task while it's not executing
a BPF program, while the kernel checks it while it is executing one,
the BPF scheduler will be killed through no fault of its own.
Consider the following scenario:
1. SCX task @p is executing on CPU A and CPU A gets preempted by a
higher-priority scheduling class. On entry to __schedule(),
p->migration_disabled == 0.
2. In put_prev_task_scx() @p is enqueued on the BPF scheduler's internal
data structures, making it available for other CPUs to dispatch.
3. CPU B enters ops.dispatch(), pops @p from the BPF scheduler's data
structures, checks is_migration_disabled(p) which returns false,
and dispatches @p to CPU B's local DSQ.
4. On CPU A, @p hasn't been switched out yet. Execution reaches
trace_sched_switch() which enters a BPF program, as the BPF scheduler
hooks into the sched_switch tracepoint to detect idle->fair
transitions. On entry into the BPF program, @p disables migration.
5. CPU B enters finish_dispatch() -> dispatch_to_local_dsq() ->
task_can_run_on_remote_rq() which observes
is_migration_disabled(p) == true, triggering scx_error().
This all happens while holding CPU B's rq lock, so it's not
synchronized with @p switching out.
This patch fixes this by moving the call to task_can_run_on_remote_rq()
after @p's rq lock is acquired in dispatch_to_local_dsq(). This way, we
synchronize with @p switching out, since @p holds its rq lock all
the way until it's switched out. Thus, any BPF programs that are called
between put_prev_task_scx() and the end of the context switch are
guaranteed to have finished and cannot influence p->migration_disabled.
Also add a lockdep assertion in task_can_run_on_remote_rq() which
ensures the task rq lock is held if enforce == true.
Signed-off-by: Kuba Piecuch <jpiecuch@google.com>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
When a task leaves the BPF scheduler's control, p->scx.dsq_vtime and
p->scx.slice keep whatever values they last held. The slice value is
core-managed and is refilled on the next enqueue, but dsq_vtime is owned
by the BPF scheduler and is never cleared by the core, so a task that
leaves SCX and later returns carries a stale dsq_vtime across the
round-trip.
The stale values are also visible to other SCX schedulers that inspect
the scx fields of non-SCX tasks.
Fix this by resetting both dsq_vtime and slice in scx_disable_task(),
after ops.disable(), so the BPF scheduler can still observe the task's
final values and non-SCX tasks do not retain stale SCX state.
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
The deprecation notices for direct p->scx.slice/dsq_vtime writes and for
ops->cpu_acquire/release() use plain pr_warn(), so they repeat on every
scheduler (re)load and can flood the kernel log.
The slice/dsq_vtime notice is the worst offender: it is emitted from the
BPF verifier's btf_struct_access callback, which is re-evaluated as the
verifier explores program paths, so a single scheduler load can print it
many times -- hundreds of lines on some hosts, dozens within the same
second.
Switch both notices to pr_warn_ratelimited() so each deprecation is still
reported but bursts no longer spam the log, and add the missing newline
to the slice/dsq_vtime message.
Signed-off-by: Breno Leitao <leitao@debian.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
Added:
depth limit to indx_find_buffer() to prevent stack overflow
validate split-point offset in indx_insert_into_buffer()
bounds check to run_get_highest_vcn()
fileattr_get() and fileattr_set() support
zero stale pagecache beyond valid data length
handle delayed allocation overlap in run lookup
validate lcns_follow in log_replay() conversion
cap RESTART_TABLE free-chain walker at rt->used
resize log->one_page_buf when adopting on-disk page size
reject direct userspace writes to reserved $LX* xattrs
Fixed:
out-of-bounds read in decompress_lznt()
avoid -Wmaybe-uninitialized warnings
hold ni_lock across readdir metadata walk
preserve non-DOS attribute bits in system.dos_attrib
validate index entry key bounds
syncing wrong inode on DIRSYNC cross-directory rename
validate Dirty Page Table capacity in log_replay() copy_lcns
wrong LCN in run_remove_range() when splitting a run
allocate iomap inline_data using alloc_page
mount failure on 64K page-size kernels
out-of-bounds read in ntfs_dir_emit() and hdr_find_e()
bound attr_off in UpdateResidentValue against data_off
bound DeleteIndexEntryAllocation memmove length
bound copy_lcns dp->page_lcns[] index in analysis pass
bound NTFS_DE view.data_off in UpdateRecordData{Root,Allocation}
prevent potential lcn remains uninitialized
Changed:
bound to_move in indx_insert_into_root() before hdr_insert_head()
call _ntfs_bad_inode() when failing to rename
fold resident writeback into writepages loop
force waiting for direct I/O completion
fold file size handling into ntfs_set_size()
reject SEEK_DATA and SEEK_HOLE past EOF early
format code, add descriptive comments and remove non-useful
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEh0DEKNP0I9IjwfWEqbAzH4MkB7YFAmoqcPYACgkQqbAzH4Mk
B7ZQ3A//ZYsz0s0qIZ0ErRuxQqmliZc1hzVGbFdKi046AKeRfhN1nV/1MP75+F4V
eD3sJ4kiROT4oc1x//uJdCoMrrH7qZs2Rcrzv3azC4F9BEFxcxLtJkyZ5NVU4eCj
vfKaRWZ8ewKeMm37Laz8DOpsz8193KzAVYK/Fm1KYoKMR0Jt+/sdOkIO/NVczZEk
gY4EAqKUTORfN0a/iELaA+NIrViTk2Wjzzu74YNl/1RDii1LGFaTfa3cmB6jylTY
AVjPX/lMMtdhy5k9Thcp4lG4uK6x6fSPYHEqvB8+Q3/JGbJfS92Oz3FR9zNKlKz4
y8depBmT85aZJ2psKetFJCXcVj3EIC1aVY/1CCgJCnLymANUuHlwFDv+3QY4D2bR
Me//cob7zLFNuul22Uveb00+34H+Tqf1QQNFNUtam6aeXC3bK9PEMbYIQ2+rU6jr
8M9MTDqfdc6WjyjXwVTNXqki5ZEZmEbSk0FXJM9JsKAyLeo2aentrQtCj5CuTBaH
sfIbJjT6g/UCfQudNxUDPtlKbhCqYA8SU23iOUlVe7TQDNKnHaQ+NEJ0prqAt303
2uVupQSJJTu+qv4s2s7ZdaA8z44WrrZfFFQm0okUIA/4NkrAKMYLTQsmki9xpW0h
KADepNmqtQ4tva9OjTkdOYiy8FUCGTKdZTJOgihjqGO+Z881xOY=
=OVdT
-----END PGP SIGNATURE-----
Merge tag 'ntfs3_for_7.2' of https://github.com/Paragon-Software-Group/linux-ntfs3
Pull ntfs3 updates from Konstantin Komarov:
"Added:
- depth limit to indx_find_buffer() to prevent stack overflow
- validate split-point offset in indx_insert_into_buffer()
- bounds check to run_get_highest_vcn()
- fileattr_get() and fileattr_set() support
- zero stale pagecache beyond valid data length
- handle delayed allocation overlap in run lookup
- validate lcns_follow in log_replay() conversion
- cap RESTART_TABLE free-chain walker at rt->used
- resize log->one_page_buf when adopting on-disk page size
- reject direct userspace writes to reserved $LX* xattrs
Fixed:
- out-of-bounds read in decompress_lznt()
- avoid -Wmaybe-uninitialized warnings
- hold ni_lock across readdir metadata walk
- preserve non-DOS attribute bits in system.dos_attrib
- validate index entry key bounds
- syncing wrong inode on DIRSYNC cross-directory rename
- validate Dirty Page Table capacity in log_replay() copy_lcns
- wrong LCN in run_remove_range() when splitting a run
- allocate iomap inline_data using alloc_page
- mount failure on 64K page-size kernels
- out-of-bounds read in ntfs_dir_emit() and hdr_find_e()
- bound attr_off in UpdateResidentValue against data_off
- bound DeleteIndexEntryAllocation memmove length
- bound copy_lcns dp->page_lcns[] index in analysis pass
- bound NTFS_DE view.data_off in UpdateRecordData{Root,Allocation}
- prevent potential lcn remains uninitialized
Changed:
- bound to_move in indx_insert_into_root() before hdr_insert_head()
- call _ntfs_bad_inode() when failing to rename
- fold resident writeback into writepages loop
- force waiting for direct I/O completion
- fold file size handling into ntfs_set_size()
- reject SEEK_DATA and SEEK_HOLE past EOF early
- format code, add descriptive comments and remove non-useful"
* tag 'ntfs3_for_7.2' of https://github.com/Paragon-Software-Group/linux-ntfs3: (34 commits)
ntfs3: reject direct userspace writes to reserved $LX* xattrs
fs/ntfs3: resize log->one_page_buf when adopting on-disk page size
fs/ntfs3: prevent potential lcn remains uninitialized
ntfs3: cap RESTART_TABLE free-chain walker at rt->used
fs/ntfs3: bound NTFS_DE view.data_off in UpdateRecordData{Root,Allocation}
fs/ntfs3: validate lcns_follow in log_replay conversion
fs/ntfs3: bound copy_lcns dp->page_lcns[] index in analysis pass
fs/ntfs3: bound DeleteIndexEntryAllocation memmove length
fs/ntfs3: bound attr_off in UpdateResidentValue against data_off
ntfs3: fix out-of-bounds read in ntfs_dir_emit() and hdr_find_e()
fs/ntfs3: fix mount failure on 64K page-size kernels
ntfs3: avoid another -Wmaybe-uninitialized warning
ntfs3: Allocate iomap inline_data using alloc_page
fs/ntfs3: format code, deal with comments
fs/ntfs3: reject SEEK_DATA and SEEK_HOLE past EOF early
fs/ntfs3: fold file size handling into ntfs_set_size()
fs/ntfs3: force waiting for direct I/O completion
fs/ntfs3: fold resident writeback into writepages loop
fs/ntfs3: handle delayed allocation overlap in run lookup
fs/ntfs3: zero stale pagecache beyond valid data length
...
New Features:
* XPRTRDMA: Decouple req recycling from RPC completion
* NFS: Expose FMODE_NOWAIT for read-only files
Bugfixes:
* SUNRPC: Fix sunrpc sysfs error handling
* SUNRPC: Fix uninitialized xprt_create_args structure
* XPRTRDMA: Harden connect and reply handling
* NFS: Fix EOF updates after fallocate/zero-range
* NFS: Keep PG_UPTODATE clear after read errors in page groups
* NFS: Use nfsi->rwsem to protect traversal of the file lock list
* NFS: Prevent resource leak in nfs_alloc_server()
* NFSv4: Clear exception state on successful mkdir retry
* NFSv4: Don't skip revalidate when holding a dir delegation and attrs are stale
* pNFS: Fix use-after-free in pnfs_update_layout()
* pNFS: Defer return_range callbacks until after inode unlock
* pNFS: Fix LAYOUTCOMMIT retry loop on OLD_STATEID
* pNFS: Reject zero-length r_addr in nfs4_decode_mp_ds_addr
* NFS/flexfiles: Reject zero-length filehandle version arrays
* NFS/flexfiles: Fix checking if a layout is striped
* NFS/flexfiles: Fixes for honoring FF_FLAGS_NO_IO_THRU_MDS
Other Cleanups and Improvements:
* Remove the fileid field from struct nfs_inode
* Move long-delayed xprtrdma work onto the system_dfl_long_wq
* Convert xprtrdma send buffer free list to an llist
* Show "<redacted>" for cert_serial and privkey_serial mount options
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEnZ5MQTpR7cLU7KEp18tUv7ClQOsFAmo64NUACgkQ18tUv7Cl
QOvVMRAAnto2SAwqPkUf2V6dET141qKhWLRKLUqbYxkzc1PKqJBfJuJBwNWHNtyb
M9JXpx00WSCjfksP5SyD5YugOzom1/SbMJlZB2FCBW6+LTyP/jwsBmqzWXdiKc/d
x2pD7dkKVdjQUg8siNRLkJR4cyquySUlV39JNKHtPzhHTyWCVYqpvBcsFZwvPPPp
TKC2ubpbu3zFlZUIYUEKMpPq44dOOlLzMzjWxMO8yTy/s/+5LsNLFRiSadr2sINp
EWdPn2rpaQT1KmkHdklwUy8xtS+Zw0LaH0g0bVGJfd2ptiMz2VdFIFzxJkQh8jMT
x0FkUBWDbTdVyiI0OZDo3uh/pJiKzTQI2SecE9to4rNHlNVDeOT9n8UanSYs71rz
emXQIgszv2juiUvbSRcgzQ+SFKcxq332eDRWmpPIQox+/NvMFK+aMLS7aTd319Up
bfVMRp5uAp5r2oVz3ETg7RDqatMJ2S0/J2HB3zVf5ONzaBaA//TUrCiSAt49Ep7a
SsK7VXJCnxw2S23fa3RqlylZ3Gw29QiRjK7INoe8iNjLTqxAvtwcCTum7Ys+IGEl
VVyxzBzgeGLlT4mU9BpMRZ9BZUjqgmflL8t4FwiFZQD1nZmJLwulZ8zSjIJ7OK2g
8G8SWP3K7igEbWGCOwqqZWTtkzQC7OYR27vQuz6aPcgIS/fuMxg=
=hj8e
-----END PGP SIGNATURE-----
Merge tag 'nfs-for-7.2-1' of git://git.linux-nfs.org/projects/anna/linux-nfs
Pull NFS client updates from Anna Schumaker:
"New features:
- XPRTRDMA: Decouple req recycling from RPC completion
- NFS: Expose FMODE_NOWAIT for read-only files
Bugfixes:
- SUNRPC:
- Fix sunrpc sysfs error handling
- Fix uninitialized xprt_create_args structure
- XPRTRDMA:
- Harden connect and reply handling
- NFS:
- Fix EOF updates after fallocate/zero-range
- Keep PG_UPTODATE clear after read errors in page groups
- Use nfsi->rwsem to protect traversal of the file lock list
- Prevent resource leak in nfs_alloc_server()
- NFSv4:
- Clear exception state on successful mkdir retry
- Don't skip revalidate when holding a dir delegation and attrs are stale
- pNFS:
- Fix use-after-free in pnfs_update_layout()
- Defer return_range callbacks until after inode unlock
- Fix LAYOUTCOMMIT retry loop on OLD_STATEID
- Reject zero-length r_addr in nfs4_decode_mp_ds_addr
- NFS/flexfiles:
- Reject zero-length filehandle version arrays
- Fix checking if a layout is striped
- Fixes for honoring FF_FLAGS_NO_IO_THRU_MDS
Other cleanups and improvements:
- Remove the fileid field from struct nfs_inode
- Move long-delayed xprtrdma work onto the system_dfl_long_wq
- Convert xprtrdma send buffer free list to an llist
- Show "<redacted>" for cert_serial and privkey_serial mount options"
* tag 'nfs-for-7.2-1' of git://git.linux-nfs.org/projects/anna/linux-nfs: (42 commits)
NFS: Use common error handling code in nfs_alloc_server()
NFS: Prevent resource leak in nfs_alloc_server()
NFSv4/pNFS: reject zero-length r_addr in nfs4_decode_mp_ds_addr
nfs: don't skip revalidate on directory delegation when attrs flagged stale
xprtrdma: Return sendctx slot after Send preparation failure
xprtrdma: Repost Receive buffers for malformed replies
xprtrdma: Sanitize the reply credit grant after parsing
xprtrdma: Fix bcall rep leak and unbounded peek
xprtrdma: Resize reply buffers before reposting receives
xprtrdma: Check frwr_wp_create() during connect
xprtrdma: Initialize re_id before removal registration
xprtrdma: Fix ep kref imbalance on ADDR_CHANGE
xprtrdma: Convert send buffer free list to llist
NFS: correct CONFIG_NFS_V4 macro name in #endif comment
nfs: use nfsi->rwsem to protect traversal of the file lock list
NFSv4.1/pNFS: fix LAYOUTCOMMIT retry loop on OLD_STATEID
nfs: expose FMODE_NOWAIT for read-only files
nfs: add nowait version of nfs_start_io_direct
NFSv4/flexfiles: honor FF_FLAGS_NO_IO_THRU_MDS in pg_get_mirror_count_write
NFSv4/flexfiles: honor FF_FLAGS_NO_IO_THRU_MDS on fatal DS connect errors
...
