| Age | Commit message (Collapse) | Author |
|---|
|
"scripts/lib" was always a bit of an awkward place for Python modules. We
already have tools/lib; create a tools/lib/python, move the libraries
there, and update the users accordingly.
While at it, move the contents of tools/docs/lib. Rather than make another
directory, just put these documentation-oriented modules under "kdoc".
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20251110220430.726665-2-corbet@lwn.net>
|
|
As reported by Randy, currently kdoc_files can go into endless
looks when symlinks are used:
$ ln -s . Documentation/peci/foo
$ ./scripts/kernel-doc Documentation/peci/
...
File "/new_devel/docs/scripts/lib/kdoc/kdoc_files.py", line 52, in _parse_dir
if entry.is_dir():
~~~~~~~~~~~~^^
OSError: [Errno 40] Too many levels of symbolic links: 'Documentation/peci/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo'
Prevent that by not considering symlinks as directories.
Reported-by: Randy Dunlap <rdunlap@infradead.org>
Closes: https://lore.kernel.org/linux-doc/80701524-09fd-4d68-8715-331f47c969f2@infradead.org/
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Reviewed-by: Randy Dunlap <rdunlap@infradead.org>
Tested-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <73c3450f34e2a4b42ef2ef279d7487c47d22e3bd.1763027622.git.mchehab+huawei@kernel.org>
|
|
When kernel-doc parses the sections for the documentation some errors
may occur. In many cases the warning is simply stored to the current
"entry" object. However, in the most of such cases this object gets
discarded and there is no way for the output engine to even know about
that. To avoid that, check if the "entry" is going to be discarded and
if there warnings have been collected, issue them to the current logger
as is and then flush the "entry". This fixes the problem that original
Perl implementation doesn't have.
As of Linux kernel v6.18-rc4 the reproducer can be:
$ scripts/kernel-doc -v -none -Wall include/linux/util_macros.h
...
Info: include/linux/util_macros.h:138 Scanning doc for function to_user_ptr
...
while with the proposed change applied it gives one more line:
$ scripts/kernel-doc -v -none -Wall include/linux/util_macros.h
...
Info: include/linux/util_macros.h:138 Scanning doc for function to_user_ptr
Warning: include/linux/util_macros.h:144 expecting prototype for to_user_ptr(). Prototype was for u64_to_user_ptr() instead
...
And with the original Perl script:
$ scripts/kernel-doc.pl -v -none -Wall include/linux/util_macros.h
...
include/linux/util_macros.h:139: info: Scanning doc for function to_user_ptr
include/linux/util_macros.h:149: warning: expecting prototype for to_user_ptr(). Prototype was for u64_to_user_ptr() instead
...
Fixes: 9cbc2d3b137b ("scripts/kernel-doc.py: postpone warnings to the output plugin")
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Reviewed-by: Randy Dunlap <rdunlap@infradead.org>
Tested-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20251104215502.1049817-1-andriy.shevchenko@linux.intel.com>
|
|
The python version of the kernel-doc parser emits some strange warnings
with just a line number in certain cases:
$ ./scripts/kernel-doc -Wall -none 'include/linux/virtio_config.h'
Warning: 174
Warning: 184
Warning: 190
Warning: include/linux/virtio_config.h:226 No description found for return value of '__virtio_test_bit'
Warning: include/linux/virtio_config.h:259 No description found for return value of 'virtio_has_feature'
Warning: include/linux/virtio_config.h:283 No description found for return value of 'virtio_has_dma_quirk'
Warning: include/linux/virtio_config.h:392 No description found for return value of 'virtqueue_set_affinity'
I eventually tracked this down to the lone call of emit_msg() in the
KernelEntry class, which looks like:
self.emit_msg(self.new_start_line, f"duplicate section name '{name}'\n")
This looks like all the other emit_msg calls. Unfortunately, the definition
within the KernelEntry class takes only a message parameter and not a line
number. The intended message is passed as the warning!
Pass the filename to the KernelEntry class, and use this to build the log
message in the same way as the KernelDoc class does.
To avoid future errors, mark the warning parameter for both emit_msg
definitions as a keyword-only argument. This will prevent accidentally
passing a string as the warning parameter in the future.
Also fix the call in dump_section to avoid an unnecessary additional
newline.
Fixes: e3b42e94cf10 ("scripts/lib/kdoc/kdoc_parser.py: move kernel entry to a class")
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20251030-jk-fix-kernel-doc-duplicate-return-warning-v2-1-ec4b5c662881@intel.com>
|
|
for man pages, it is helpful to know from where the man page
were generated.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <ac25496a27a0c90494a634d342207ef1ff6216e9.1759327966.git.mchehab+huawei@kernel.org>
|
|
When running kernel-doc over multiple documents, it emits
one error message per file with is not what we want:
$ python3.6 scripts/kernel-doc.py . --none
...
Warning: ./include/trace/events/swiotlb.h:0 Python 3.7 or later is required for correct results
Warning: ./include/trace/events/iommu.h:0 Python 3.7 or later is required for correct results
Warning: ./include/trace/events/sock.h:0 Python 3.7 or later is required for correct results
...
Change the logic to warn it only once at the library:
$ python3.6 scripts/kernel-doc.py . --none
Warning: Python 3.7 or later is required for correct results
Warning: ./include/cxl/features.h:0 Python 3.7 or later is required for correct results
When running from command line, it warns twice, but that sounds
ok.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <68e54cf8b1201d1f683aad9bc710a99421910356.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
While cross-references are complex, as related ones can be on
different files, we can at least correlate the ones that belong
to the same file, adding a SEE ALSO section for them.
The result is not bad. See for instance:
$ tools/docs/sphinx-build-wrapper --sphinxdirs driver-api/media -- mandocs
$ man Documentation/output/driver-api/man/edac_pci_add_device.9
edac_pci_add_device(9) Kernel Hacker's Manual edac_pci_add_device(9)
NAME
edac_pci_add_device - Insert the 'edac_dev' structure into the
edac_pci global list and create sysfs entries associated with
edac_pci structure.
SYNOPSIS
int edac_pci_add_device (struct edac_pci_ctl_info *pci , int
edac_idx );
ARGUMENTS
pci pointer to the edac_device structure to be added to
the list
edac_idx A unique numeric identifier to be assigned to the
RETURN
0 on Success, or an error code on failure
SEE ALSO
edac_pci_alloc_ctl_info(9), edac_pci_free_ctl_info(9),
edac_pci_alloc_index(9), edac_pci_del_device(9), edac_pci_cre‐
ate_generic_ctl(9), edac_pci_release_generic_ctl(9),
edac_pci_create_sysfs(9), edac_pci_remove_sysfs(9)
August 2025 edac_pci_add_device edac_pci_add_device(9)
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <fba25efb41eadad17a54e6275a6191173d702f00.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
Merge "typedef" into the typedef_type pattern rather than repeating it
later, and add some comments.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
By the time we get here, comments have long since been stripped out; there
is no need to do it again.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
The regex in this block of code makes no sense, and a quick test shows that
it never matches anything; simply delete the code.
No output changes.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
Add some more comments to dump_function(), add some comments, and trim out
an unneeded duplicate output_declaration() call.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
The logic to handle macros is split in dump_function(); bring it all
together into a single place and add a comment saying what's going on.
Remove the unneeded is_define_proto variable, and tighten up the code
a bit.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
The regexes for the parsing of function prototypes were more complicated
than they needed to be and difficult to understand -- at least, I spent a
fair amount of time bashing my head against them. Simplify them, and add
some documentation comments as well.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
The is_define_proto case in dump_function() uses a regex with an empty
capture group - () - that has no use; just take it out.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
The "name" regex in dump_function() includes both the tilde and colon
characters, but neither has any place in function prototypes. Remove the
characters, after which the regex simplifies to "\w+"
No output changes.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
Both functions and structs are passed through a set of regex-based
transforms, but the two were structured differently, despite being the same
thing. Create a utility function to apply transformations and use it in
both cases.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
Move these definitions to file level, where they are executed once, and
don't clutter the function itself.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
struct_attribute is only used once, so just put its value there directly
and drop the name.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
The handling of untyped parameters involved a number of redundant tests;
restructure the code to remove them and be more compact.
No output changes.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
The special case for __cacheline_group_begin/end() can be handled by just
adding another pattern to the struct_prefixes, eliminating the need for a
special case in push_parameter().
One change is that these annotations no longer appear in the rendered
output, just like all the other annotations that we clean out.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
By the time stuff gets to create_parameter_list(), comments have long since
been stripped out, so we do not need to do it again here.
Acked-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250814154035.328769-8-corbet@lwn.net
|
|
Tighten up the code and remove an unneeded regex operation.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250814154035.328769-7-corbet@lwn.net
|
|
Simplify one gnarly regex and remove another altogether; add a comment
describing what is going on. There will be no #-substituted commas in this
case, so don't bother trying to put them back.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250814154035.328769-6-corbet@lwn.net
|
|
Make what the final code is doing a bit more clear to slow readers like me.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250814154035.328769-5-corbet@lwn.net
|
|
The logic for finding the name of the first in a series of variable names
is somewhat convoluted and, in the use of .extend(), actively buggy.
Document what is happening and simplify the logic.
Acked-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250814154035.328769-4-corbet@lwn.net
|
|
Remove a redundant test and add a comment describing what the space removal
is doing.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250814154035.328769-3-corbet@lwn.net
|
|
create_parameter_list() tests an argument against the same regex twice, in
two different locations; remove the pointless extra tests and the
never-executed error cases that go with them.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250814154035.328769-2-corbet@lwn.net
|
|
In my ongoing effort to truly understand our new kernel-doc, I continue to
make changes to improve the code, and to try to make the understanding task
easier for the next person. These patches focus on dump_struct() in
particular, which starts out at nearly 300 lines long - to much to fit into
my little brain anyway. Hopefully the result is easier to manage.
There are no changes in the rendered docs.
|
|
Add a couple more comments so that each phase of the process is
now clearly marked.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250807211639.47286-13-corbet@lwn.net
|
|
The last thing done in dump_struct() is to format the structure for
printing. That, too, is a separate activity; split it out into its own
function.
dump_struct() now fits in a single, full-hight editor screen.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250807211639.47286-12-corbet@lwn.net
|
|
Get rid of some redundant checks, and generally tighten up the code; no
logical change.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250807211639.47286-11-corbet@lwn.net
|
|
Add comments to rewrite_struct_members() describing what it is actually
doing, and reformat/comment the main struct_members regex so that it is
(more) comprehensible to humans.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250807211639.47286-10-corbet@lwn.net
|
|
...the variable in question was already strip()ed at the top of the loop.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250807211639.47286-9-corbet@lwn.net
|
|
Adopt a more Pythonic form for the main loop of this function, getting rid
of the "while True:" construction and making the actual loop invariant
explicit.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250807211639.47286-8-corbet@lwn.net
|
|
The massive loop that massages struct members shares no data with the rest
of dump_struct(); split it out into its own function. Code movement only,
no other changes.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250807211639.47286-7-corbet@lwn.net
|
|
Move the initial split of the prototype into its own function in the
ongoing effort to cut dump_struct() down to size.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250807211639.47286-6-corbet@lwn.net
|
|
dump_struct is one of the longest functions in the kdoc_parser class,
making it hard to read and reason about. Move the definition of the prefix
transformations out of the function, join them with the definition of
"attribute" (which was defined at the top of the file but only used here),
and reformat the code slightly for shorter line widths.
Just code movement in the end.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250807211639.47286-5-corbet@lwn.net
|
|
A lot of the regular expressions in this file have extraneous backslashes
that may have been needed in Perl, but aren't helpful here. Take them out
to reduce slightly the visual noise.
Escaping of (){}[] has been left in place, even when unnecessary, for
visual clarity.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250807211639.47286-4-corbet@lwn.net
|
|
The complex struct_members regex was defined far from its use; bring the
two together. Remove some extraneous backslashes while making the move.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250807211639.47286-3-corbet@lwn.net
|
|
There were two locations duplicating the logic of stripping private members
and associated comments; coalesce them into one, and add some comments
describing what's going on.
Output change: we now no longer add extraneous white space around macro
definitions.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250807211639.47286-2-corbet@lwn.net
|
|
Pull documentation updates from Jonathan Corbet:
"It has been a relatively busy cycle for docs, especially the build
system:
- The Perl kernel-doc script was added to 2.3.52pre1 just after the
turn of the millennium. Over the following 25 years, it accumulated
a vast amount of cruft, all in a language few people want to deal
with anymore. Mauro's Python replacement in 6.16 faithfully
reproduced all of the cruft in the hope of avoiding regressions.
Now that we have a more reasonable code base, though, we can work
on cleaning it up; many of the changes this time around are toward
that end.
- A reorganization of the ext4 docs into the usual TOC format.
- Various Chinese translations and updates.
- A new script from Mauro to help with docs-build testing.
- A new document for linked lists
- A sweep through MAINTAINERS fixing broken GitHub git:// repository
links.
...and lots of fixes and updates"
* tag 'docs-6.17' of git://git.lwn.net/linux: (147 commits)
scripts: add origin commit identification based on specific patterns
sphinx: kernel_abi: fix performance regression with O=<dir>
Documentation: core-api: entry: Replace deprecated KVM entry/exit functions
docs: fault-injection: drop reference to md-faulty
docs: document linked lists
scripts: kdoc: make it backward-compatible with Python 3.7
docs: kernel-doc: emit warnings for ancient versions of Python
Documentation/rtla: Describe exit status
Documentation/rtla: Add include common_appendix.rst
docs: kernel: Clarify printk_ratelimit_burst reset behavior
Documentation: ioctl-number: Don't repeat macro names
Documentation: ioctl-number: Shorten macros table
Documentation: ioctl-number: Correct full path to papr-physical-attestation.h
Documentation: ioctl-number: Extend "Include File" column width
Documentation: ioctl-number: Fix linuxppc-dev mailto link
overlayfs.rst: fix typos
docs: kdoc: emit a warning for ancient versions of Python
docs: kdoc: clean up check_sections()
docs: kdoc: directly access the always-there KdocItem fields
docs: kdoc: straighten up dump_declaration()
...
|
|
There was a change at kdoc that ended breaking compatibility
with Python 3.7: str.removesuffix() was introduced on version
3.9.
Restore backward compatibility.
Reported-by: Akira Yokosawa <akiyks@gmail.com>
Closes: https://lore.kernel.org/linux-doc/57be9f77-9a94-4cde-aacb-184cae111506@gmail.com/
Fixes: 27ad33b6b349 ("kernel-doc: Fix symbol matching for dropped suffixes")
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/d13058d285838ac2bc04c492e60531c013a8a919.1752218291.git.mchehab+huawei@kernel.org
|
|
The kerneldoc parsing phase gathers all of the information about the
declarations of interest, then passes it through to the output phase as a
dict that is an unstructured blob of information; this organization has its
origins in the Perl version of the program. It results in an interface
that is difficult to reason about, dozen-parameter function calls, and
other ills.
Introduce a new class (KdocItem) to carry this information between the
parser and the output modules, and, step by step, modify the system to use
this class in a more structured way. This could be taken further by
creating a subclass of KdocItem for each declaration type (function,
struct, ...), but that is probably more structure than we need.
The result is (I hope) clearer code, the removal of a bunch of boilerplate,
and no changes to the generated output.
|
|
Versions of Python prior to 3.7 do not guarantee to remember the insertion
order of dicts; since kernel-doc depends on that guarantee, running with
such older versions could result in output with reordered sections.
Python 3.9 is the minimum for the kernel as a whole, so this should not be
a problem, but put in a warning just in case somebody tries to use
something older.
Suggested-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
entry.sectcheck is just a duplicate of our list of sections that is only
passed to check_sections(); its main purpose seems to be to avoid checking
the special named sections. Rework check_sections() to not use that field
(which is then deleted), tocheck for the known sections directly, and
tighten up the logic in general.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
They are part of the interface, so use them directly. This allows the
removal of the transitional __dict__ hack in KdocItem.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
Get rid of the excess "return" statements in dump_declaration(), along with
a line of never-executed dead code.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
Each declaration type passes through the name in a unique field of the
"args" blob - even though we have always just passed the name separately.
Get rid of all the weird names and just use the common version.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
Callers to output_declaration() always pass the parameter information from
self.entry; remove all of the boilerplate arguments and just get at that
information directly. Formalize its placement in the KdocItem class.
It would be nice to get rid of parameterlist as well, but that has the
effect of reordering the output of function parameters and struct fields to
match the order in the kerneldoc comment rather than in the declaration.
One could argue about which is more correct, but the ordering has been left
unchanged for now.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
Callers of check_sections() join parameterlist into a single string, which
is then immediately split back into the original list. Rather than do all
that, just use parameterlist directly in check_sections().
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
