Simplify even further the docs Makefile by moving rust build logic
to the wrapper.
After this change, running make on an environment with rust enabled
works as expected.
With CONFIG_RUST:
$ make O=/tmp/foo LLVM=1 SPHINXDIRS=peci htmldocs
make[1]: Entrando no diretório '/tmp/foo'
Using alabaster theme
Using Python kernel-doc
GEN Makefile
DESCEND objtool
CC arch/x86/kernel/asm-offsets.s
INSTALL libsubcmd_headers
CALL /new_devel/docs/scripts/checksyscalls.sh
RUSTC L rust/core.o
BINDGEN rust/bindings/bindings_generated.rs
BINDGEN rust/bindings/bindings_helpers_generated.rs
...
Without it:
$ make SPHINXDIRS=peci htmldocs
Using alabaster theme
Using Python kernel-doc
Both work as it is it is supposed to do.
After the change, it is also possible to build directly with the script
by passing "--rustodoc".
if CONFIG_RUST, this works fine:
$ ./tools/docs/sphinx-build-wrapper --sphinxdirs peci --rustdoc -- htmldocs
Using alabaster theme
Using Python kernel-doc
SYNC include/config/auto.conf
...
RUSTC L rust/core.o
...
If not, it will produce a warning that RUST may be disabled:
$ ./tools/docs/sphinx-build-wrapper --sphinxdirs peci --rustdoc -- htmldocs
Using alabaster theme
Using Python kernel-doc
***
*** Configuration file ".config" not found!
***
*** Please run some configurator (e.g. "make oldconfig" or
*** "make menuconfig" or "make xconfig").
***
make[1]: *** [/new_devel/docs/Makefile:829: .config] Error 1
make: *** [Makefile:248: __sub-make] Error 2
Ignored errors when building rustdoc: Command '['make', 'LLVM=1', 'rustdoc']' returned non-zero exit status 2.. Is RUST enabled?
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <fa1235ccf859f6ebfeef7ffba0ebde2015a75042.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Generating man files currently requires running a separate
script. The target also doesn't appear at the docs Makefile.
Add support for mandocs at the Makefile, adding the build
logic inside sphinx-build-wrapper, updating documentation
and dropping the ancillary script.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <3d248d724e7f3154f6e3a227e5923d7360201de9.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
When SPHINXDIRS is used, basename may be identical for different
files. If this happens, the summary and error detection won't be
accurate.
Fix it by using relative names from builddir.
While here, don't duplicate names. Report, instead:
- SUCCESS
output PDF file was built
- FAILED
latexmk/xelatex didn't build any PDF output
- FAILED: no .tex files were generated
Sphinx didn't build any tex file for SPHINXDIRS directories
- FAILED ({python exception})
When a concurrent.futures is catched. Usually indicates an
internal error at the build logic.
With that, building multiple dirs with the same name is reported
properly:
$ make V=1 SPHINXDIRS="admin-guide/media driver-api/media userspace-api/media" pdfdocs
Summary
=======
admin-guide/media/pdf/media.pdf : SUCCESS
driver-api/media/pdf/media.pdf : SUCCESS
userspace-api/media/pdf/media.pdf: SUCCESS
And if at least one of them fails, return code will be 1.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <d4a4f16f6c0c423ad38531a490888be3bf01e574.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
On a properly set system, LANG and LC_ALL is always defined.
However, some distros like Debian, Gentoo and their variants
start with those undefioned.
When Sphinx tries to set a locale with:
locale.setlocale(locale.LC_ALL, '')
It raises an exception, making Sphinx fail. This is more likely
to happen with test containers.
Add a logic to detect and workaround such issue by setting
locale to C.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <1d0afad8fe3d83182be3a08eb00dd71322e23e69.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Use POSIX jobserver when available or -j<number> to run PDF
builds in parallel, restoring pdf build performance. Yet,
running it when debugging troubles is a bad idea, so, when
calling directly via command line, except if "-j" is splicitly
requested, it will serialize the build.
With such change, a PDF doc builds now takes around 5 minutes
on a Ryzen 9 machine with 32 cpu threads:
# Explicitly paralelize both Sphinx and LaTeX pdf builds
$ make cleandocs; time scripts/sphinx-build-wrapper pdfdocs -j 33
real 5m17.901s
user 15m1.499s
sys 2m31.482s
# Use POSIX jobserver to paralelize both sphinx-build and LaTeX
$ make cleandocs; time make pdfdocs
real 5m22.369s
user 15m9.076s
sys 2m31.419s
# Serializes PDF build, while keeping Sphinx parallelized.
# it is equivalent of passing -jauto via command line
$ make cleandocs; time scripts/sphinx-build-wrapper pdfdocs
real 11m20.901s
user 13m2.910s
sys 1m44.553s
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <42eef319f9af6f9feb12bcd74ca6392c8119929d.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
By default, we use LaTeX batch mode to build docs. This way, when
an error happens, the build fails. This is good for normal builds,
but when debugging problems with pdf generation, the best is to
use interactive mode.
We already support it via LATEXOPTS, but having a command line
argument makes it easier:
Interactive mode:
./scripts/sphinx-build-wrapper pdfdocs --sphinxdirs peci -v -i
...
Running 'xelatex --no-pdf -no-pdf -recorder ".../Documentation/output/peci/latex/peci.tex"'
...
Default batch mode:
./scripts/sphinx-build-wrapper pdfdocs --sphinxdirs peci -v
...
Running 'xelatex --no-pdf -no-pdf -interaction=batchmode -no-shell-escape -recorder ".../Documentation/output/peci/latex/peci.tex"'
...
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <9e5b9a8becc981b47ca3bf3ddce034f273400738.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
There are too much magic inside docs Makefile to properly run
sphinx-build. Create an ancillary script that contains all
kernel-related sphinx-build call logic currently at Makefile.
Such script is designed to work both as an standalone command
and as part of a Makefile. As such, it properly handles POSIX
jobserver used by GNU make.
On a side note, there was a line number increase due to the
conversion (ignoring comments) is:
Documentation/Makefile | 131 +++----------
tools/docs/sphinx-build-wrapper | 293 +++++++++++++++++++++++++++++++
2 files changed, 323 insertions(+), 101 deletions(-)
Comments and descriptions adds:
tools/docs/sphinx-build-wrapper | 261 +++++++++++++++++++++++++++++++-
So, about half of the script are comments/descriptions.
This is because some things are more verbosed on Python and because
it requires reading env vars from Makefile. Besides it, this script
has some extra features that don't exist at the Makefile:
- It can be called directly from command line;
- It properly return PDF build errors.
When running the script alone, it will only take handle sphinx-build
targets. On other words, it won't runn make rustdoc after building
htmlfiles, nor it will run the extra check scripts.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <80ae57b01fcfb1d338d93b8f8e26e57b69b5f16b.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
The sphinx-pre-install code has some logic to deal with Python
version, which ensures that a minimal version will be enforced
for documentation build logic.
Move it to a separate library to allow re-using its code.
No functional changes.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <d134ace64b55c827565ce68f0527e20c735f0d2e.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
The caller script may not want an automatic execution of the new
version. Add two parameters to allow showing alternatives and to
bail out if version is incompatible.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <19777bc710bf901ffbb0ad0f1bb57b18fc01b163.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
The version print at the lib was added for debugging purposes.
Get rid of it.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <27f76a4df2b80c38d277d58a92c85c614544e013.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
As we're reorganizing the place where doc scripts are located,
move this one to tools/docs.
No functional changes.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <5e2c40d3aebfd67b7ac7817f548bd1fa4ff661a8.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Use lib docstring to output the comments via --help/-h. With
that, update the default instructions to recomment it instead
of asking the user to read the source code.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <577162cf4e07de74c4a783f16e3404f0040e5e0a.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
As we'll be using the actual code inside sphinx-build-wrapper,
split the library from the executable, placing the exec at
the new place we've been using:
tools/docs
No functional changes.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <8adbc22df1d43b1c5a673799d2333cc429ffe9fc.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Add a new script and a new documentation 'make' target,
htmldocs-redirects.
This will generate HTML stub files in the HTML documentation output
directory that redirect the browser to the new path.
Suggested-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Suggested-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Vegard Nossum <vegard.nossum@oracle.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20250905144608.577449-4-vegard.nossum@oracle.com>
Add a new script that wraps git to trawl the repository history for
renames of .rst files in the Documentation/ directory.
Example usage:
tools/docs/gen-renames.py --rev v6.17-rc3 > Documentation/.renames.txt
The output format is simply:
<old path> SPACE <new path> NEWLINE
where neither <old path> nor <new path> contain the Documentation/
prefix or the .rst suffix. The file is sorted alphabetically.
We can suggest rerunning the script for future renames (and squash the
resulting change) or rerun it periodically to keep the file up to date.
Signed-off-by: Vegard Nossum <vegard.nossum@oracle.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20250905144608.577449-2-vegard.nossum@oracle.com>
When running it from command line, we want to write an output
file, but when used as a class, one may just want the output
content returned as a string.
Split write_output() on two methods to allow both usecases.
Also add an extra method to produce a TOC.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/c98bdec3380aad54178baf2751a2f1fcd128576b.1755872208.git.mchehab+huawei@kernel.org
As suggested by Jon, we should start having a tools/docs
directory, instead of placing everything under scripts.
In the specific case of parse-headers.py, the previous
location is where we're placing Sphinx extensions, which is
not the right place for execs.
Move it to tools/docs/parse-headers.py.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/0f5ac2d704cffe9834e589b39549d2393e1237ef.1755872208.git.mchehab+huawei@kernel.org
