- Simplify option parsing
Auto-generate getopt_long() optstring for short options from long options
array, avoiding the need to specify it manually and reducing the surface for
mistakes.
- Add unit tests
Implement unit tests (make unit-tests) using libcheck, next to existing
runtime tests (make check). Currently, three functions from utils.c are
tested.
- Add --stack-format option
In addition to stopping stack pointer decoding (with -s/--stack option)
on first unresolvable pointer, allow also skipping unresolvable pointers
and displaying everything, configurable with a new option.
- Unify number of CPUs into one global variable
Use one global variable, nr_cpus, to store the number of CPUs instead of
retrieving it and passing it at multiple places.
- Fix behavior in various corner cases
Make RTLA behave correctly in several corner cases: memory allocation
failure, invalid value read from kernel side, thread creation failure,
malformed time value input, and read/write failure or interruption by
signal.
- Improve string handling
Simplify several places in the code that handle strings, including
parsing of action arguments. A few new helper functions and variables
are added for that purpose.
- Get rid of magic numbers
Few places handling paths use a magic number of 1024. Replace it with
MAX_PATH and ARRAY_SIZE() macro.
- Unify threshold handling
Code that handles response to latency threshold is duplicated between
tools, which has led to bugs in the past. Unify it into a new helper
as much as possible.
- Fix segfault on SIGINT during cleanup
The SIGINT handler touches dynamically allocated memory. Detach it
before freeing it during cleanup to prevent segmentation fault and
discarding of output buffers. Also, properly document SIGINT handling
while at it.
-----BEGIN PGP SIGNATURE-----
iIoEABYKADIWIQRRSw7ePDh/lE+zeZMp5XQQmuv6qgUCad4njhQccm9zdGVkdEBn
b29kbWlzLm9yZwAKCRAp5XQQmuv6qoKkAP0bYgkuqvyBsplZf1Mqil7IRM3tTFni
XJdkFFr6E6jS/gEAynaf7oq4gz7isvMvZZTOVcl1VflQAjLuY9ezO5LQzw0=
=mR69
-----END PGP SIGNATURE-----
Merge tag 'trace-rtla-v7.1' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull RTLA updates from Steven Rostedt:
- Simplify option parsing
Auto-generate getopt_long() optstring for short options from long
options array, avoiding the need to specify it manually and reducing
the surface for mistakes.
- Add unit tests
Implement unit tests (make unit-tests) using libcheck, next to
existing runtime tests (make check). Currently, three functions from
utils.c are tested.
- Add --stack-format option
In addition to stopping stack pointer decoding (with -s/--stack
option) on first unresolvable pointer, allow also skipping
unresolvable pointers and displaying everything, configurable with a
new option.
- Unify number of CPUs into one global variable
Use one global variable, nr_cpus, to store the number of CPUs instead
of retrieving it and passing it at multiple places.
- Fix behavior in various corner cases
Make RTLA behave correctly in several corner cases: memory allocation
failure, invalid value read from kernel side, thread creation
failure, malformed time value input, and read/write failure or
interruption by signal.
- Improve string handling
Simplify several places in the code that handle strings, including
parsing of action arguments. A few new helper functions and variables
are added for that purpose.
- Get rid of magic numbers
Few places handling paths use a magic number of 1024. Replace it with
MAX_PATH and ARRAY_SIZE() macro.
- Unify threshold handling
Code that handles response to latency threshold is duplicated between
tools, which has led to bugs in the past. Unify it into a new helper
as much as possible.
- Fix segfault on SIGINT during cleanup
The SIGINT handler touches dynamically allocated memory. Detach it
before freeing it during cleanup to prevent segmentation fault and
discarding of output buffers. Also, properly document SIGINT handling
while at it.
* tag 'trace-rtla-v7.1' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: (28 commits)
Documentation/rtla: Document SIGINT behavior
rtla: Fix segfault on multiple SIGINTs
rtla/utils: Fix loop condition in PID validation
rtla/utils: Fix resource leak in set_comm_sched_attr()
rtla/trace: Fix I/O handling in save_trace_to_file()
rtla/trace: Fix write loop in trace_event_save_hist()
rtla/timerlat: Simplify RTLA_NO_BPF environment variable check
rtla: Use str_has_prefix() for option prefix check
rtla: Enforce exact match for time unit suffixes
rtla: Use str_has_prefix() for prefix checks
rtla: Add str_has_prefix() helper function
rtla: Handle pthread_create() failure properly
rtla/timerlat: Add bounds check for softirq vector
rtla: Simplify code by caching string lengths
rtla: Replace magic number with MAX_PATH
rtla: Introduce common_threshold_handler() helper
rtla/actions: Simplify argument parsing
rtla: Use strdup() to simplify code
rtla: Exit on memory allocation failures during initialization
tools/rtla: Remove unneeded nr_cpus from for_each_monitored_cpu
...
- Refactor da_monitor header to share handlers across monitor types
No functional changes, only less code duplication.
- Add Hybrid Automata model class
Add a new model class that extends deterministic automata by adding
constraints on transitions and states. Those constraints can take into
account wall-clock time and as such allow RV monitor to make
assertions on real time. Add documentation and code generation
scripts.
- Add stall monitor as hybrid automaton example
Add a monitor that triggers a violation when a task is stalling as an
example of automaton working with real time variables.
- Convert the opid monitor to a hybrid automaton
The opid monitor can be heavily simplified if written as a hybrid
automaton: instead of tracking preempt and interrupt enable/disable
events, it can just run constraints on the preemption/interrupt
states when events like wakeup and need_resched verify.
- Add support for per-object monitors in DA/HA
Allow writing deterministic and hybrid automata monitors for generic
objects (e.g. any struct), by exploiting a hash table where objects
are saved. This allows to track more than just tasks in RV. For
instance it will be used to track deadline entities in deadline
monitors.
- Add deadline tracepoints and move some deadline utilities
Prepare the ground for deadline monitors by defining events and
exporting helpers.
- Add nomiss deadline monitor
Add first example of deadline monitor asserting all entities complete
before their deadline.
- Improve rvgen error handling
Introduce AutomataError exception class and better handle expected
exceptions while showing a backtrace for unexpected ones.
- Improve python code quality in rvgen
Refactor the rvgen generation scripts to align with python best
practices: use f-strings instead of %, use len() instead of __len__(),
remove semicolons, use context managers for file operations, fix
whitespace violations, extract magic strings into constants, remove
unused imports and methods.
- Fix small bugs in rvgen
The generator scripts presented some corner case bugs: logical error in
validating what a correct dot file looks like, fix an isinstance()
check, enforce a dot file has an initial state, fix type annotations
and typos in comments.
- rvgen refactoring
Refactor automata.py to use iterator-based parsing and handle required
arguments directly in argparse.
- Allow epoll in rtapp-sleep monitor
The epoll_wait call is now rt-friendly so it should be allowed in the
sleep monitor as a valid sleep method.
-----BEGIN PGP SIGNATURE-----
iIoEABYKADIWIQRRSw7ePDh/lE+zeZMp5XQQmuv6qgUCad4kUxQccm9zdGVkdEBn
b29kbWlzLm9yZwAKCRAp5XQQmuv6qiJgAPsGJ0TCTubdn2x6fpwLDKUcSsNsYrok
m8gLHeK0rmkKNQD/ajUW+RBsOufXAsnSzUoUcl7sw1TSiQJ085U+0+ZeDgM=
=WZqk
-----END PGP SIGNATURE-----
Merge tag 'trace-rv-v7.1' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull runtime verification updates from Steven Rostedt:
- Refactor da_monitor header to share handlers across monitor types
No functional changes, only less code duplication.
- Add Hybrid Automata model class
Add a new model class that extends deterministic automata by adding
constraints on transitions and states. Those constraints can take
into account wall-clock time and as such allow RV monitor to make
assertions on real time. Add documentation and code generation
scripts.
- Add stall monitor as hybrid automaton example
Add a monitor that triggers a violation when a task is stalling as an
example of automaton working with real time variables.
- Convert the opid monitor to a hybrid automaton
The opid monitor can be heavily simplified if written as a hybrid
automaton: instead of tracking preempt and interrupt enable/disable
events, it can just run constraints on the preemption/interrupt
states when events like wakeup and need_resched verify.
- Add support for per-object monitors in DA/HA
Allow writing deterministic and hybrid automata monitors for generic
objects (e.g. any struct), by exploiting a hash table where objects
are saved. This allows to track more than just tasks in RV. For
instance it will be used to track deadline entities in deadline
monitors.
- Add deadline tracepoints and move some deadline utilities
Prepare the ground for deadline monitors by defining events and
exporting helpers.
- Add nomiss deadline monitor
Add first example of deadline monitor asserting all entities complete
before their deadline.
- Improve rvgen error handling
Introduce AutomataError exception class and better handle expected
exceptions while showing a backtrace for unexpected ones.
- Improve python code quality in rvgen
Refactor the rvgen generation scripts to align with python best
practices: use f-strings instead of %, use len() instead of
__len__(), remove semicolons, use context managers for file
operations, fix whitespace violations, extract magic strings into
constants, remove unused imports and methods.
- Fix small bugs in rvgen
The generator scripts presented some corner case bugs: logical error
in validating what a correct dot file looks like, fix an isinstance()
check, enforce a dot file has an initial state, fix type annotations
and typos in comments.
- rvgen refactoring
Refactor automata.py to use iterator-based parsing and handle
required arguments directly in argparse.
- Allow epoll in rtapp-sleep monitor
The epoll_wait call is now rt-friendly so it should be allowed in the
sleep monitor as a valid sleep method.
* tag 'trace-rv-v7.1' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: (32 commits)
rv: Allow epoll in rtapp-sleep monitor
rv/rvgen: fix _fill_states() return type annotation
rv/rvgen: fix unbound loop variable warning
rv/rvgen: enforce presence of initial state
rv/rvgen: extract node marker string to class constant
rv/rvgen: fix isinstance check in Variable.expand()
rv/rvgen: make monitor arguments required in rvgen
rv/rvgen: remove unused __get_main_name method
rv/rvgen: remove unused sys import from dot2c
rv/rvgen: refactor automata.py to use iterator-based parsing
rv/rvgen: use class constant for init marker
rv/rvgen: fix DOT file validation logic error
rv/rvgen: fix PEP 8 whitespace violations
rv/rvgen: fix typos in automata and generator docstring and comments
rv/rvgen: use context managers for file operations
rv/rvgen: remove unnecessary semicolons
rv/rvgen: replace __len__() calls with len()
rv/rvgen: replace % string formatting with f-strings
rv/rvgen: remove bare except clauses in generator
rv/rvgen: introduce AutomataError exception class
...
Web links in the documentation are not properly displayed.
In the man pages web links look like:
Osnoise tracer documentation: < <https://www.kernel.org/doc/html/lat‐
est/trace/osnoise-tracer.html> >
On web pages the URL caption is the URL itself.
Convert tracer documentation links to RST anonymous hyperlink format
for better rendering. Use newer docs.kernel.org instead of
www.kernel.org/doc/html/latest for brevity.
After the change, the links in the man pages look like:
Osnoise tracer <https://docs.kernel.org/trace/osnoise-tracer.html>
On web pages the captions are the titles of the links.
Signed-off-by: Costa Shulyupin <costa.shul@redhat.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260405163847.3337981-1-costa.shul@redhat.com>
Add a sample monitor to showcase hybrid/timed automata.
The stall monitor identifies tasks stalled for longer than a threshold
and reacts when that happens.
Reviewed-by: Nam Cao <namcao@linutronix.de>
Link: https://lore.kernel.org/r/20260330111010.153663-7-gmonaco@redhat.com
Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
The behavior of RTLA on receiving SIGINT is currently undocumented.
Describe it in RTLA's common appendix that appears in man pages for all
RTLA tools to avoid confusion.
Suggested-by: Attila Fazekas <afazekas@redhat.com>
Reviewed-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Link: https://lore.kernel.org/r/20260324123229.152424-1-tglozar@redhat.com
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
Add an autodoc entry for the new kdoc_yaml_file module.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <32b86abe7acee2dd4f73a35836ec94e8690f04cd.1774256269.git.mchehab+huawei@kernel.org>
Do some fixes at groups() description for it to be parsed by
Sphinx and add it to the documentation.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <799178cf30dd4022fdb1d029ba998a458e037b52.1773823995.git.mchehab+huawei@kernel.org>
While python internal libraries have support for unit tests, its
output is not nice. Add a helper module to improve its output.
I wrote this module last year while testing some scripts I used
internally. The initial skeleton was generated with the help of
LLM tools, but it was higly modified to ensure that it will work
as I would expect.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <37999041f616ddef41e84cf2686c0264d1a51dc9.1773074166.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <d81be167b8cdeb003c1f8dcc7ad83a5ed2b520b6.1773770483.git.mchehab+huawei@kernel.org>
Over the time, most of the changes at kernel-doc are related
to maintaining a list of transforms to convert macros into pure
C code.
Place such transforms on a separate module, to cleanup the
parser module.
There is an advantage on that: QEMU also uses our own kernel-doc,
but the xforms list there is different. By placing it on a
separate module, we can minimize the differences and make it
easier to keep QEMU in sync with Kernel upstream.
No functional changes.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <ccd74b7589e1fff340a74bf8ed16a974532cb54f.1772469446.git.mchehab+huawei@kernel.org>
Add hwnoise to the command list and SEE ALSO section. The command list
is ordered from low level to high level.
Signed-off-by: Costa Shulyupin <costa.shul@redhat.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260215131249.33437-1-costa.shul@redhat.com>
- Remove unused function declarations
Some functions were removed in recent code consolidation 6.18, but
their prototypes were not removed from headers. Remove them.
- Set stop threshold after enabling instances
Prefer recording samples without stopping on them on the start of
tracing to stopping on samples that are never recorded. This fixes
flakiness of some RTLA tests and unifies behavior of sample collection
between tracefs mode and BPF mode.
- Consolidate usage help message implementation
RTLA tools (osnoise-top, osnoise-hist, timerlat-top, timerlat-hist)
each implement usage help individually. Move common logic between
them into a new function to reduce code duplication.
- Add BPF actions feature
Add option --bpf-action to attach a BPF program (passed as filename of
its ELF representation) to be executed via BPF tail call at latency
threshold.
- Consolidate command line option parsing
Each RTLA tool implements the parsing of command line options
individually. Now that we have a common structure for parameters,
unify the parsing of those options common among all four tools into
one function.
- De-duplicate cgroup common code
Two functions in utils.c, setting cgroup for comm and setting cgroup
for pid, duplicate code for constructing the cgroup path. Extract it
to a new helper function.
- String and error handling fixes and cleanups
There were several instances of unsafe string and error handling that
could cause invalid memory access; fix them. Also, remove a few unused
headers, update .gitignore, and deduplicate code.
-----BEGIN PGP SIGNATURE-----
iIoEABYKADIWIQRRSw7ePDh/lE+zeZMp5XQQmuv6qgUCaY3l2RQccm9zdGVkdEBn
b29kbWlzLm9yZwAKCRAp5XQQmuv6qrWcAP4lsfIbNhfvmc9htDON/GlCu9FK8PTT
hmAsO7MeHh29OwD+OS7RZzRza+3G+B4aTcqUXQVfAvx9qenS591LcktQSAo=
=VUW2
-----END PGP SIGNATURE-----
Merge tag 'trace-rtla-v7.0' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull RTLA updates from Steven Rostedt:
- Remove unused function declarations
Some functions were removed in recent code consolidation 6.18, but
their prototypes were not removed from headers. Remove them.
- Set stop threshold after enabling instances
Prefer recording samples without stopping on them on the start of
tracing to stopping on samples that are never recorded. This fixes
flakiness of some RTLA tests and unifies behavior of sample
collection between tracefs mode and BPF mode.
- Consolidate usage help message implementation
RTLA tools (osnoise-top, osnoise-hist, timerlat-top, timerlat-hist)
each implement usage help individually. Move common logic between
them into a new function to reduce code duplication.
- Add BPF actions feature
Add option --bpf-action to attach a BPF program (passed as filename
of its ELF representation) to be executed via BPF tail call at
latency threshold.
- Consolidate command line option parsing
Each RTLA tool implements the parsing of command line options
individually. Now that we have a common structure for parameters,
unify the parsing of those options common among all four tools into
one function.
- De-duplicate cgroup common code
Two functions in utils.c, setting cgroup for comm and setting cgroup
for pid, duplicate code for constructing the cgroup path. Extract it
to a new helper function.
- String and error handling fixes and cleanups
There were several instances of unsafe string and error handling that
could cause invalid memory access; fix them. Also, remove a few
unused headers, update .gitignore, and deduplicate code.
* tag 'trace-rtla-v7.0' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: (30 commits)
rtla: Fix parse_cpu_set() bug introduced by strtoi()
rtla: Fix parse_cpu_set() return value documentation
rtla: Ensure null termination after read operations in utils.c
rtla: Make stop_tracing variable volatile
rtla: Add generated output files to gitignore
rtla: Fix NULL pointer dereference in actions_parse
rtla: Remove unused headers
rtla: Remove redundant memset after calloc
rtla: Use standard exit codes for result enum
rtla: Replace atoi() with a robust strtoi()
rtla: Introduce for_each_action() helper
tools/rtla: Deduplicate cgroup path opening code
tools/rtla: Consolidate -H/--house-keeping option parsing
tools/rtla: Consolidate -P/--priority option parsing
tools/rtla: Consolidate -e/--event option parsing
tools/rtla: Consolidate -d/--duration option parsing
tools/rtla: Consolidate -D/--debug option parsing
tools/rtla: Consolidate -C/--cgroup option parsing
tools/rtla: Consolidate -c/--cpus option parsing
tools/rtla: Add common_parse_options()
...
The root document usually has a special :ref:`genindex` link to the
generated index. This is also the case for Documentation/index.rst. The
other index.rst files deeper in the directory hierarchy usually don't.
For SPHINXDIRS builds, the root document isn't Documentation/index.rst,
but some other index.rst in the hierarchy. Currently they have a
".. only::" block to add the index link when doing SPHINXDIRS html
builds.
This is obviously very tedious and repetitive. The link is also added to
all index.rst files in the hierarchy for SPHINXDIRS builds, not just the
root document.
Put the boilerplate in a sphinx-includes/subproject-index.rst file, and
include it at the end of the root document for subproject builds in an
ad-hoc source-read extension defined in conf.py.
For now, keep having the boilerplate in translations, because this
approach currently doesn't cover translated index link headers.
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Mauro Carvalho Chehab <mchehab@kernel.org>
Cc: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Tested-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
[jc: did s/doctree/kern_doc_dir/ ]
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260123143149.2024303-1-jani.nikula@intel.com>
Place parse_features module documentation at Linux Kernel docs.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <7c1e41468f765587f0962222e7f52125a039028f.1768838938.git.mchehab+huawei@kernel.org>
Place jobserver module documentation at Linux Kernel docs.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <688215d6a41d78bd5e37854472a5fc802d58c77a.1768838938.git.mchehab+huawei@kernel.org>
Place kernel abi modules documentation at Linux Kernel docs.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <ddc02f11d64fdfc0d8d3e3e0967e041b5172da6c.1768838938.git.mchehab+huawei@kernel.org>
Place kernel-doc modules documentation at Linux Kernel docs.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <88ac2d82a45718c4e27aefac831586a71204ebf2.1768838938.git.mchehab+huawei@kernel.org>
Add new option --bpf-action into common_timerlat_options.txt, including
the format in which it takes the BPF program, and a reference to an
example.
Link: https://lore.kernel.org/r/20251126144205.331954-8-tglozar@redhat.com
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
The sample/ directory in tools/tracing/rtla was renamed to example/ in
an earlier commit.
Rename it also in the documentation.
Reviewed-by: Wander Lairson Costa <wander@redhat.com>
Link: https://lore.kernel.org/r/20251126144205.331954-7-tglozar@redhat.com
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
- Officially add Tomas Glozar as a maintainer to RTLA tool
- Add for_each_monitored_cpu() helper
In multiple places, RTLA tools iterate over the list of CPUs running
tracer threads.
Use single helper instead of repeating the for/if combination.
- Remove unused variable option_index in argument parsing
RTLA tools use getopt_long() for argument parsing. For its last
argument, an unused variable "option_index" is passed.
Remove the variable and pass NULL to getopt_long() to shorten
the naturally long parsing functions, and make them more readable.
- Fix unassigned nr_cpus after code consolidation
In recent code consolidation, timerlat tool cleanup, previously
implemented separately for each tool, was moved to a common function
timerlat_free().
The cleanup relies on nr_cpus being set. This was not done in the new
function, leaving the variable uninitialized.
Initialize the variable properly, and remove silencing of compiler
warning for uninitialized variables.
- Stop tracing on user latency in BPF mode
Despite the name, rtla-timerlat's -T/--thread option sets timerlat's
stop_tracing_total_us option, which also stops tracing on
return-from-user latency, not only on thread latency.
Implement the same behavior also in BPF sample collection stop tracing
handler to avoid a discrepancy and restore correspondence of behavior
with the equivalent option of cyclictest.
- Fix threshold actions always triggering
A bug in threshold action logic caused the action to execute even
if tracing did not stop because of threshold.
Fix the logic to stop correctly.
- Fix few minor issues in tests
Extend tests that were shown to need it to 5s, fix osnoise test
calling timerlat by mistake, and use new, more reliable output
checking in timerlat's "top stop at failed action" test.
- Do not print usage on argument parsing error
RTLA prints the entire usage message on encountering errors in
argument parsing, like a malformed CPU list.
The usage message has gotten too long. Instead of printing it,
use newly added fatal() helper function to simply exit with
the error message, excluding the usage.
- Fix unintuitive -C/--cgroup interface
"-C cgroup" and "--cgroup cgroup" are invalid syntax, despite that
being a common way to specify an option with argument. Moreover,
using them fails silently and no cgroup is set.
Create new helper function to unify the handling of all such options
and allow all of:
-Xsomething
-X=something
-X something
as well as the equivalent for the long option.
- Fix -a overriding -t argument filename
Fix a bug where -a following -t custom_file.txt overrides the custom
filename with the default timerlat_trace.txt.
- Stop tracing correctly on multiple events at once
In some race scenarios, RTLA BPF sample collection might send multiple
stop tracing events via the BPF ringbuffer at once.
Compare the number of events for != 0 instead of == 1 to cover for
this scenario and stop tracing properly.
-----BEGIN PGP SIGNATURE-----
iIoEABYKADIWIQRRSw7ePDh/lE+zeZMp5XQQmuv6qgUCaS9bxBQccm9zdGVkdEBn
b29kbWlzLm9yZwAKCRAp5XQQmuv6qhrgAP0a/AtsL9+IFXAK5JK8aO1XWApVyK9n
48FRZWu/jrupuAD7BO+EHazmPEourNaUqYPeuymwxT+4O47RH1Q/aasLQwo=
=RvNH
-----END PGP SIGNATURE-----
Merge tag 'trace-tools-v6.19' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull rtla trace tooling updates from Steven Rostedt:
- Officially add Tomas Glozar as a maintainer to RTLA tool
- Add for_each_monitored_cpu() helper
In multiple places, RTLA tools iterate over the list of CPUs running
tracer threads.
Use single helper instead of repeating the for/if combination.
- Remove unused variable option_index in argument parsing
RTLA tools use getopt_long() for argument parsing. For its last
argument, an unused variable "option_index" is passed.
Remove the variable and pass NULL to getopt_long() to shorten the
naturally long parsing functions, and make them more readable.
- Fix unassigned nr_cpus after code consolidation
In recent code consolidation, timerlat tool cleanup, previously
implemented separately for each tool, was moved to a common function
timerlat_free().
The cleanup relies on nr_cpus being set. This was not done in the new
function, leaving the variable uninitialized.
Initialize the variable properly, and remove silencing of compiler
warning for uninitialized variables.
- Stop tracing on user latency in BPF mode
Despite the name, rtla-timerlat's -T/--thread option sets timerlat's
stop_tracing_total_us option, which also stops tracing on
return-from-user latency, not only on thread latency.
Implement the same behavior also in BPF sample collection stop
tracing handler to avoid a discrepancy and restore correspondence of
behavior with the equivalent option of cyclictest.
- Fix threshold actions always triggering
A bug in threshold action logic caused the action to execute even if
tracing did not stop because of threshold.
Fix the logic to stop correctly.
- Fix few minor issues in tests
Extend tests that were shown to need it to 5s, fix osnoise test
calling timerlat by mistake, and use new, more reliable output
checking in timerlat's "top stop at failed action" test.
- Do not print usage on argument parsing error
RTLA prints the entire usage message on encountering errors in
argument parsing, like a malformed CPU list.
The usage message has gotten too long. Instead of printing it, use
newly added fatal() helper function to simply exit with the error
message, excluding the usage.
- Fix unintuitive -C/--cgroup interface
"-C cgroup" and "--cgroup cgroup" are invalid syntax, despite that
being a common way to specify an option with argument. Moreover,
using them fails silently and no cgroup is set.
Create new helper function to unify the handling of all such options
and allow all of:
-Xsomething
-X=something
-X something
as well as the equivalent for the long option.
- Fix -a overriding -t argument filename
Fix a bug where -a following -t custom_file.txt overrides the custom
filename with the default timerlat_trace.txt.
- Stop tracing correctly on multiple events at once
In some race scenarios, RTLA BPF sample collection might send
multiple stop tracing events via the BPF ringbuffer at once.
Compare the number of events for != 0 instead of == 1 to cover for
this scenario and stop tracing properly.
* tag 'trace-tools-v6.19' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
rtla/timerlat: Exit top main loop on any non-zero wait_retval
rtla/tests: Don't rely on matching ^1ALL
rtla: Fix -a overriding -t argument
rtla: Fix -C/--cgroup interface
tools/rtla: Replace osnoise_hist_usage("...") with fatal("...")
tools/rtla: Replace osnoise_top_usage("...") with fatal("...")
tools/rtla: Replace timerlat_hist_usage("...") with fatal("...")
tools/rtla: Replace timerlat_top_usage("...") with fatal("...")
tools/rtla: Add fatal() and replace error handling pattern
rtla/tests: Fix osnoise test calling timerlat
rtla/tests: Extend action tests to 5s
tools/rtla: Fix --on-threshold always triggering
rtla/timerlat_bpf: Stop tracing on user latency
tools/rtla: Fix unassigned nr_cpus
tools/rtla: Remove unused optional option_index
tools/rtla: Add for_each_monitored_cpu() helper
MAINTAINERS: Add Tomas Glozar as a maintainer to RTLA tool
Currently, user can only specify cgroup to the tracer's thread the
following ways:
`-C[cgroup]`
`-C[=cgroup]`
`--cgroup[=cgroup]`
If user tries to specify cgroup as `-C [cgroup]` or `--cgroup [cgroup]`,
the parser silently fails and rtla's cgroup is used for the tracer
threads.
To make interface more user-friendly, allow user to specify cgroup in
the aforementioned way, i.e. `-C [cgroup]` and `--cgroup [cgroup]`.
Refactor identical logic between -t/--trace and -C/--cgroup into a
common function.
Change documentation to reflect this user interface change.
Fixes: a957cbc025 ("rtla: Add -C cgroup support")
Signed-off-by: Ivan Pravdin <ipravdin.official@gmail.com>
Reviewed-by: Tomas Glozar <tglozar@redhat.com>
Link: https://lore.kernel.org/r/16132f1565cf5142b5fbd179975be370b529ced7.1762186418.git.ipravdin.official@gmail.com
[ use capital letter in subject, as required by tracing subsystem ]
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
Commit 0122938a7a ("rtla: Always set all tracer options") changed the
behavior of RTLA to always set all osnoise and timerlat tracer options
to default values taken from the tracers whenever an RTLA measurement
is started. The change was done to make RTLA results consistent on
subsequent runs of the same command.
Include the default values for tracer options also in documentation
where appropriate.
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20251010083338.478961-10-tglozar@redhat.com>
The RTLA option -C/--cgroup is used to set a cgroup for workload
threads. This is either a specific cgroup, if passed an argument, or
rtla's cgroup, if no argument is given.
Expand the documentation of the -C option to also include the
information about the cgroup settings when the option is not specified.
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20251010083338.478961-8-tglozar@redhat.com>
RTLA allows the priority of workload threads to be set using the -P
option. This is covered in docs, but the default state for RTLA's own
user workload (implemented in timerlat_u.c) is not mentioned.
Add mention of the default user workload priority as well as a reference
to osnoise and timerlat tracers for kernel workload priority.
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20251010083338.478961-7-tglozar@redhat.com>
Several options in common_options.rst say "osnoise tracer" for both
osnoise and timerlat.
Use |tool| variable so that the correct tool name is used.
Fixes: b1be48307d ("rtla: Add rtla osnoise top documentation")
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20251010083338.478961-6-tglozar@redhat.com>
Fix spelling error "equilavent" in place of "equivalent".
Fixes: 173a3b0148 ("rtla/timerlat: Add the automatic trace option")
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20251010083338.478961-5-tglozar@redhat.com>
Fix "seem" in place of intended "seen" in rtla-timerlat-top
documentation.
Fixes: df337d014b ("rtla: Add rtla timerlat top documentation")
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20251010083338.478961-4-tglozar@redhat.com>
Fix "awakes" being used in place of "awakened" in --users-threads option
documentation.
Fixes: 6127383217 ("Documentation: Add tools/rtla timerlat -u option documentation")
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20251010083338.478961-3-tglozar@redhat.com>
Sphinx reports htmldocs errors:
Documentation/tools/rtla/common_options.rst:58: ERROR: Undefined substitution referenced: "threshold".
Documentation/tools/rtla/common_options.rst:88: ERROR: Undefined substitution referenced: "tool".
Documentation/tools/rtla/common_options.rst:88: ERROR: Undefined substitution referenced: "thresharg".
Documentation/tools/rtla/common_options.rst:88: ERROR: Undefined substitution referenced: "tracer".
Documentation/tools/rtla/common_options.rst:92: ERROR: Undefined substitution referenced: "tracer".
Documentation/tools/rtla/common_options.rst:98: ERROR: Undefined substitution referenced: "actionsperf".
Documentation/tools/rtla/common_options.rst:113: ERROR: Undefined substitution referenced: "tool".
common_*.rst files are snippets that are intended to be included by rtla
docs (rtla*.rst). common_options.rst in particular contains
substitutions which depend on other common_* includes, so building it
independently as reST source results in above errors.
Rename all common_*.rst files to common_*.txt to prevent Sphinx from
building these snippets as standalone reST source and update all include
references accordingly.
Link: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#substitutions
Suggested-by: Tomas Glozar <tglozar@redhat.com>
Suggested-by: Bagas Sanjaya <bagasdotme@gmail.com>
Signed-off-by: Gopi Krishna Menon <krishnagopi487@gmail.com>
Reviewed-by: Tomas Glozar <tglozar@redhat.com>
Fixes: 05b7e10687 ("tools/rtla: Add remaining support for osnoise actions")
Reviewed-by: Bagas Sanjaya <bagasdotme@gmail.com>
Link: https://lore.kernel.org/r/20251008184522.13201-1-krishnagopi487@gmail.com
[Bagas: massage commit message and apply trailers]
Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com>
Reviewed-by: Randy Dunlap <rdunlap@infradead.org>
Tested-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20251013092719.30780-2-bagasdotme@gmail.com>
The basic functionality came with the consolidation; now hook up the
command line options, and add documentation and tests.
Cc: John Kacur <jkacur@redhat.com>
Cc: Costa Shulyupin <costa.shul@redhat.com>
Link: https://lore.kernel.org/20250907022325.243930-8-crwood@redhat.com
Reviewed-by: Tomas Glozar <tglozar@redhat.com>
Signed-off-by: Crystal Wood <crwood@redhat.com>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
- Introduce enum timerlat_tracing_mode
Now that BPF based sampling has been added to timerlat, add an enum to
represent which mode timerlat is running in.
- Add action on timelat threshold feature
A new option, --on-threshold, is added, taking an argument
that further specifies the action. Actions added in this patch are:
- trace[,file=<filename>]: Saves tracefs buffer, optionally taking a
filename.
- signal,num=<sig>,pid=<pid>: Sends signal to process. "parent" might
be specified instead of number to send signal to parent process.
- shell,command=<command>: Execute shell command.
- Allow resuming tracing in timerlat bpf
rtla-timerlat BPF program uses a global variable stored in a .bss section to
store whether tracing has been stopped. Map it to allow it to resume tracing
after it has been stopped.
- Add continue action to timerlat
Introduce option to resume tracing after a latency threshold overflow. The
option is implemented as an action named "continue".
- Add action on end feature to timerlat
Implement actions on end next to actions on threshold. A new option,
--on-end is added, parallel to --on-threshold. Instead of being executed
whenever a latency threshold is reached, it is executed at the end of the
measurement.
- Have rtla tests check output with grep
Add argument to the check command in the test suite that takes a regular
expression that the output of rtla command is checked against. This allows
testing for specific information in rtla output in addition to checking the
return value.
- Add tests for timerlat actions
- Update the documentation for the new features
-----BEGIN PGP SIGNATURE-----
iIoEABYKADIWIQRRSw7ePDh/lE+zeZMp5XQQmuv6qgUCaIuAtBQccm9zdGVkdEBn
b29kbWlzLm9yZwAKCRAp5XQQmuv6qk1uAQCzkGc7IUqp8038HU0aZoOwSxS04ma3
NvrF0Vuw8wvVqgD+Pogi1M7fr3LyXHzW4k3Vx7Ot3YSy52T6aCohjcKQ0wo=
=P46B
-----END PGP SIGNATURE-----
Merge tag 'trace-tools-v6.17' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull tracing tools updates from Steven Rostedt:
- Introduce enum timerlat_tracing_mode
Now that BPF based sampling has been added to timerlat, add an enum
to represent which mode timerlat is running in
- Add action on timelat threshold feature
A new option, --on-threshold, is added, taking an argument that
further specifies the action. Actions added in this patch are:
- trace[,file=<filename>]: Saves tracefs buffer, optionally taking a
filename
- signal,num=<sig>,pid=<pid>: Sends signal to process. "parent" might
be specified instead of number to send signal to parent process
- shell,command=<command>: Execute shell command
- Allow resuming tracing in timerlat bpf
rtla-timerlat BPF program uses a global variable stored in a .bss
section to store whether tracing has been stopped. Map it to allow it
to resume tracing after it has been stopped
- Add continue action to timerlat
Introduce option to resume tracing after a latency threshold
overflow. The option is implemented as an action named "continue"
- Add action on end feature to timerlat
Implement actions on end next to actions on threshold. A new option,
--on-end is added, parallel to --on-threshold. Instead of being
executed whenever a latency threshold is reached, it is executed at
the end of the measurement
- Have rtla tests check output with grep
Add argument to the check command in the test suite that takes a
regular expression that the output of rtla command is checked
against. This allows testing for specific information in rtla output
in addition to checking the return value
- Add tests for timerlat actions
- Update the documentation for the new features
* tag 'trace-tools-v6.17' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
rtla/tests: Test timerlat -P option using actions
rtla/tests: Add grep checks for base test cases
Documentation/rtla: Add actions feature
rtla/tests: Limit duration to maximum of 10s
rtla/tests: Add tests for actions
rtla/tests: Check rtla output with grep
rtla/timerlat: Add action on end feature
rtla/timerlat: Add continue action
rtla/timerlat_bpf: Allow resuming tracing
rtla/timerlat: Add action on threshold feature
rtla/timerlat: Introduce enum timerlat_tracing_mode
Commit 18682166f6 ("rtla: Set distinctive exit value for failed
tests") expands exit status making it useful.
Add section 'EXIT STATUS' and required SPDX-License-Identifier
to the documentation.
Signed-off-by: Costa Shulyupin <costa.shul@redhat.com>
Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org>
[jc: fixed sphinx error caused by missing blank line]
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250608105531.758809-2-costa.shul@redhat.com
Add include common_appendix.rst into
Documentation/tools/rtla/rtla-timerlat-hist.rst - the only file of
rtla-*.rst still without common_appendix.rst.
Signed-off-by: Costa Shulyupin <costa.shul@redhat.com>
Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250608104437.753708-2-costa.shul@redhat.com
Add dependencies needed to build rtla with BPF sample collection support
to README, and document both ways of sample collection in the manpages.
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Reviewed-by: Luis Claudio R. Goncalves <lgoncalv@redhat.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250311114936.148012-5-tglozar@redhat.com
A passage about how the timerlat tracer outputs information is included
in both common_timerlat_description.rst and rtla-timerlat.rst, leading
it to be displayed twice in the rtla-timerlat page.
Remove the duplicate passage from rtla-timerlat.rst.
Fixes: 29380d4055 ("rtla: Add rtla timerlat documentation")
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
Reviewed-by: Luis Claudio R. Goncalves <lgoncalv@redhat.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250311114936.148012-2-tglozar@redhat.com
The pages Documentation/tools/rv/rv-mon-sched.rst and
Documentation/trace/rv/monitor_sched.rst were introduced but not
included in any index.
Add them to the respective indices.
Cc: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/20250327081240.46422-1-gmonaco@redhat.com
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Fixes: 03abeaa63c ("Documentation/rv: Add docs for the sched monitors")
Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Add man page and kernel documentation for the sched monitors, as sched
is a container of other monitors, document all in the same page.
sched is the first nested monitor, also explain what is a nested monitor
and how enabling containers or children monitors work.
To: Ingo Molnar <mingo@redhat.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: John Kacur <jkacur@redhat.com>
Cc: Clark Williams <williams@redhat.com>
Link: https://lore.kernel.org/20250305140406.350227-9-gmonaco@redhat.com
Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Add --deepest-idle-state to manpage and mention libcpupower dependency
in README.txt.
Link: https://lore.kernel.org/20241017140914.3200454-7-tglozar@redhat.com
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Move -t, --trace from common_options.rst to
common_osnoise_options.rst and
common_timerlat_options.rst
so that it will appear in the man pages
rtla-timerlat-hist.1
rtla-timerlat-top.1
rtla-osnoise-hist.1
rtla-osnoise-top.1
Remove the equals ('=') sign and add a space.
Link: https://lkml.kernel.org/r/20240516143121.12614-1-jkacur@redhat.com
Cc: Daniel Bristot de Oliveria <bristot@kernel.org>
Signed-off-by: John Kacur <jkacur@redhat.com>
Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
Add the option allow the users to set a different buffer size for the
trace. For example, in large systems, the user might be interested on
reducing the trace buffer to avoid large tracing files.
The buffer size is specified in kB, and it is only affecting
the tracing instance.
The function trace_set_buffer_size() appears on libtracefs v1.6,
so increase the minimum required version on Makefile.config.
Link: https://lkml.kernel.org/r/e7c9ca5b3865f28e131a49ec3b984fadf2d056c6.1715860611.git.bristot@kernel.org
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: John Kacur <jkacur@redhat.com>
Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
After ther -u addition, most of the known users are setting it. And
it makes sense, as it adds more information, and inherits the default
setup for the threads - e.g., cgroups configs.
Thus, if the user-space interface is available, enable -u. Otherwise,
use the in-kernel thread.
Add the -k option to allow the user to request kernel-threads.
Link: https://lkml.kernel.org/r/9241d3089de4091b124f780ed832a0e6646cadaa.1713968967.git.bristot@kernel.org
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Juri Lelli <juri.lelli@redhat.com>
Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
On many cases, the results right after the startup are different
from the rest of the execution, biasing the results. For example,
on osnoise, the scheduler might take some time to adapt to the new
busy-loop workload.
Add the --warm-up <seconds> option, adding a warm-up phase (in
seconds) where the workload is set, but the results are discarded.
Link: https://lkml.kernel.org/r/e682d5ce5af90f123bd13220f63d5c3d118a92be.1713968967.git.bristot@kernel.org
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Juri Lelli <juri.lelli@redhat.com>
Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
The timerlat tracer provides an interface for any application to wait
for the timerlat's periodic wakeup. Currently, rtla timerlat uses it
to dispatch its user-space workload (-u option).
But as the tracer interface is generic, rtla timerlat can also be used
to monitor any workload that uses it. For example, a user might
place their own workload to wait on the tracer interface, and
monitor the results with rtla timerlat.
Add the -U option to rtla timerlat top and hist. With this option, rtla
timerlat will not dispatch its workload but only setting up the
system, waiting for a user to dispatch its workload.
The sample code in this patch is an example of python application
that loops in the timerlat tracer fd.
To use it, dispatch:
# rtla timerlat -U
In a terminal, then run the python program on another terminal,
specifying the CPU to run it. For example, setting on CPU 1:
#./timerlat_load.py 1
Then rtla timerlat will start printing the statistics of the
./timerlat_load.py app.
An interesting point is that the "Ret user Timer Latency" value
is the overall response time of the load. The sample load does
a memory copy to exemplify that.
The stop tracing options on rtla timerlat works in this setup
as well, including auto analysis.
Link: https://lkml.kernel.org/r/36e6bcf18fe15c7601048fd4c65aeb193c502cc8.1707229706.git.bristot@kernel.org
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
The following error message is reported when running the example in document.
# timerlat hist -d 10m -c 0-4 -P d:100us:1ms -p 1ms --no-aa
Failed to set timerlat period
Could not apply config
The unit of the period is microsecond, '1ms' cannot be accepted.
usage: [rtla] timerlat hist [-h] [-q] [-d s] [-D] [-n] [-a us] [-p us] [-i us] [-T us] [-s us] ...
...
-p/--period us: timerlat period in us
...
Also fix another minor missleading comment.
Link: https://lore.kernel.org/lkml/20230919133028.697144-1-xiexiuqi@huaweicloud.com
Signed-off-by: Xie XiuQi <xiexiuqi@huawei.com>
Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
