github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-07-27 09:36:22 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

Rust changes for v7.2

Browse Source 
Toolchain and infrastructure:
 
  - Introduce support for the 'zerocopy' library [1][2]:
 
        Fast, safe, compile error. Pick two.
 
        Zerocopy makes zero-cost memory manipulation effortless. We write
        `unsafe` so you don't have to.
 
    It essentially provides derivable traits (e.g. 'FromBytes') and
    macros (e.g. 'transmute!') for safely converting between byte
    sequences and other types. Having such support allows us to remove
    some 'unsafe' code.
 
    It is among the most downloaded Rust crates and it is also used by
    the Rust compiler itself.
 
    It is licensed under "BSD-2-Clause OR Apache-2.0 OR MIT".
 
    The crates are imported essentially as-is (only +2/-3 lines needed
    to be adapted), plus SPDX identifiers. Upstream has since added the
    SPDX identifiers as well as one of the tweaks at my request, thus
    reducing our future diffs on updates -- I keep the details in one of
    our usual live lists [3].
 
    In total, it is about ~39k lines added, ~32k without counting
    'benches/' which are just for documentation purposes.
 
    The series includes a few Kbuild and rust-analyzer improvements and
    an example patch using it in Nova, removing one 'unsafe impl'.
 
    I checked that the codegen of an isolated example function (similar
    to the Nova patch on top) is essentially identical. It also turns out
    that (for that particular case) the 'zerocopy' version, even with
    'debug-assertions' enabled, has no remaining panics, unlike a few in
    the current code (since the compiler can prove the remaining
    'ub_checks' statically).
 
    So their "fast, safe" does indeed check out -- at least in that case.
 
    Link: https://github.com/google/zerocopy [1]
    Link: https://docs.rs/zerocopy [2]
    Link: https://github.com/Rust-for-Linux/linux/issues/1239 [3]
 
  - Support AutoFDO. This allows Rust code to be profiled and optimized
    based on the profile. Tested with Rust Binder: ~13% slower without
    AutoFDO in the binderAddInts benchmark (using an app-launch benchmark
    for the profile).
 
  - Support Software Tag-Based KASAN.
 
    In addition, fix KASAN Kconfig by requiring Clang.
 
  - Add Kconfig options for each existing Rust KUnit test suite, such as
    'CONFIG_RUST_BITMAP_KUNIT_TEST'. They are placed within a new menu,
    'CONFIG_RUST_KUNIT_TESTS', in the new 'rust/kernel/Kconfig.test'
    file.
 
  - Support the upcoming Rust 1.98.0 release (expected 2026-08-20): lint
    cleanups and an unstable flag rename.
 
  - Disable 'rustdoc' documentation inlining for all prelude items, which
    bloats the generated documentation.
 
  - Ignore (in Git) and clean (in Kbuild) the (rarely) 'rustc'-generated
    '*.long-type-*.txt' files.
 
 'kernel' crate:
 
  - Add new 'bitfield' module with the 'bitfield!' macro (extracted from
    the existing 'register!' one), which declares integer types that are
    split into distinct bit fields of arbitrary length.
 
    Each field is a 'Bounded' of the appropriate bit width (ensuring
    values are properly validated and avoiding implicit data loss) and
    gets several generated getters and setters (infallible, 'const' and
    fallible) as well as associated constants ('_MASK', '_SHIFT' and
    '_RANGE'). It also supports fields that can be converted from/to
    custom types, either fallibly ('?=>') or infallibly ('=>').
 
    For instance:
 
        bitfield! {
            struct Rgb(u16) {
                15:11 blue;
                10:5 green;
                4:0 red;
            }
        }
 
        // Compile-time checks.
        let color = Rgb::zeroed().with_const_green::<0x1f>();
 
        assert_eq!(color.green(), 0x1f);
        assert_eq!(color.into_raw(), 0x1f << Rgb::GREEN_SHIFT);
 
    Add as well documentation and a test suite for it, as usual; and
    update the 'register!' macro to use it.
 
    It will be maintained by Alexandre Courbot (with Yury Norov as
    reviewer) under a new 'MAINTAINERS' entry: 'RUST [BITFIELD]'.
 
  - 'ptr' module: rework index projection syntax into keyworded syntax
    and introduce panicking variant.
 
    The keyword syntax ('build:', 'try:', 'panic:') is more explicit and
    paves the way of perhaps adding more flavors in the future, e.g. an
    'unsafe' index projection.
 
    For instance, projections now look like this:
 
        fn f(p: *const [u8; 32]) -> Result {
            // Ok, within bounds, checked at build time.
            project!(p, [build: 1]);
 
            // Build error.
            project!(p, [build: 128]);
 
            // `OutOfBound` runtime error (convertible to `ERANGE`).
            project!(p, [try: 128]);
 
            // Runtime panic.
            project!(p, [panic: 128]);
 
            Ok(())
        }
 
    Update as well the users, which now look like e.g.
 
        // Pointer to the first entry of the GSP message queue.
        let data = project!(self.0.as_ptr(), .gspq.msgq.data[build: 0]);
 
  - 'build_assert' module: make the module the home of its macros instead
    of rendering them twice.
 
  - 'sync' module: add 'UniqueArc::as_ptr()' associated function.
 
  - 'alloc' module:
 
     + Fix the 'Vec::reserve()' doctest to properly account for the
       existing vector length in the capacity assertion.
 
     + Fix an incorrect operator in the 'Vec::extend_with()' 'SAFETY'
       comment; add a doc test demonstrating basic usage and the
       zero-length case.
 
  - Clean imports across several modules to follow the "kernel vertical"
    import style in order to minimize conflicts.
 
 'pin-init' crate:
 
  - User visible changes:
 
     + Do not generate 'non_snake_case' warnings for identifiers that are
       syntactically just users of a field name. This would allow all
       '#[allow(non_snake_case)]' in nova-core to be removed, which Gary
       will send to the nova tree next cycle.
 
     + Filter non-cfg attributes out properly in derived structs. This
       improves pin-init compatibility with other derive macros.
 
     + Insert projection types' where clause properly.
 
  - Other changes:
 
     + Bump MSRV to 1.82, plus associated cleanups.
 
     + Overhaul how init slots are projected. The new approach is easier
       to justify with safety comments.
 
     + Mark more functions as inline, which should help mitigate the
       super-long symbol name issue due to lack of inlining.
 
 rust-analyzer:
 
  - Support '--envs' for passing env vars for crates like 'zerocopy'.
 
 'MAINTAINERS':
 
  - Add the following reviewers to the 'RUST' entry:
 
     + Daniel Almeida
     + Tamir Duberstein
     + Alexandre Courbot
     + Onur Özkan
 
    They have been involved in the Rust for Linux project for about
    7 collective years and bring expertise across several domains, which
    will be very useful to have around in the future.
 
    Thanks everyone for stepping up!
 
 And some other fixes, cleanups and improvements.
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEPjU5OPd5QIZ9jqqOGXyLc2htIW0FAmovBQsACgkQGXyLc2ht
 IW0BGA//WT/3qZTOE0yNtjG2/VAgbi6onNQeUf/NWkgo0HmPx0AUsKaedbWLOkTK
 FYTj0XOb7XlTV6ryuDsYfbQUm8vNGI8HEGAxWJmSThrY0dqBgnKTWWeYUCaG1OEp
 OBjf3i1gGS9A7WUWrBijjyeOui+dvm3wXBdKFREqVn7cRDjYUYUw75ZZsUxTigZf
 pA/tW5GEmrQw5NmuNB8bLeQacwwIwDSfnnkxE6d+FDAOngwyM7IM9ENLvy5cl2Ui
 yVUkEpXvA8nnvy4MXQ5toIUbDGMFKJBpIL1GeDgMc7DQtDOxNFeBnBe9hrpfQr2L
 XgeyWDs3+QD5WdVkjCPJEVS2gGpVDYIRUJRRHitGp+g+WDZsTA8FKvjjJjPnvHE8
 WSdmhB3/EP40vkLoKKHTb1/vizeJ3io+ku52fZLemTJESe1vqzc9sTlZFQ4zpp09
 /KCAwF+43XEPA1ETbLZQ0Wx7hTz0wjHIbF45hDGOGuvcjFepdFsFVKsHxDDXqkiB
 AqsdIR5IGPeVOLWDvWlRRrZvPQNGkxhf5zc+Ah0TfYfN4kyBuoUkdOpS0mdYVb1y
 nAULtyDkw3Ty8ZDVXgpl+o99kX7ajbgmIhOW6SrvKt43k9YQJ7A3NnaLCuoM3zOf
 wYzy/HNNMkal+8NZ67kT20BceuHlGAY3awIM7NbRAGGt3taMtwo=
 =Setk
 -----END PGP SIGNATURE-----

Merge tag 'rust-7.2' of gitolite.kernel.org:pub/scm/linux/kernel/git/ojeda/linux

Pull Rust updates from Miguel Ojeda:
 "This one is big due to the vendoring of the `zerocopy` library, which
  allows us to replace a bunch of `unsafe` code dealing with conversions
  between byte sequences and other types with safe alternatives. More
  details on that below (and in its merge commit).

  Toolchain and infrastructure:

   - Introduce support for the 'zerocopy' library [1][2]:

         Fast, safe, compile error. Pick two.

         Zerocopy makes zero-cost memory manipulation effortless. We write
         `unsafe` so you don't have to.

     It essentially provides derivable traits (e.g. 'FromBytes') and
     macros (e.g. 'transmute!') for safely converting between byte
     sequences and other types. Having such support allows us to remove
     some 'unsafe' code.

     It is among the most downloaded Rust crates and it is also used by
     the Rust compiler itself.

     It is licensed under "BSD-2-Clause OR Apache-2.0 OR MIT".

     The crates are imported essentially as-is (only +2/-3 lines needed
     to be adapted), plus SPDX identifiers. Upstream has since added the
     SPDX identifiers as well as one of the tweaks at my request, thus
     reducing our future diffs on updates -- I keep the details in one
     of our usual live lists [3].

     In total, it is about ~39k lines added, ~32k without counting
     'benches/' which are just for documentation purposes.

     The series includes a few Kbuild and rust-analyzer improvements and
     an example patch using it in Nova, removing one 'unsafe impl'.

     I checked that the codegen of an isolated example function (similar
     to the Nova patch on top) is essentially identical. It also turns
     out that (for that particular case) the 'zerocopy' version, even
     with 'debug-assertions' enabled, has no remaining panics, unlike a
     few in the current code (since the compiler can prove the remaining
     'ub_checks' statically).

     So their "fast, safe" does indeed check out -- at least in that
     case.

   - Support AutoFDO. This allows Rust code to be profiled and optimized
     based on the profile. Tested with Rust Binder: ~13% slower without
     AutoFDO in the binderAddInts benchmark (using an app-launch
     benchmark for the profile).

   - Support Software Tag-Based KASAN.

     In addition, fix KASAN Kconfig by requiring Clang.

   - Add Kconfig options for each existing Rust KUnit test suite, such
     as 'CONFIG_RUST_BITMAP_KUNIT_TEST'.

     They are placed within a new menu, 'CONFIG_RUST_KUNIT_TESTS', in
     the new 'rust/kernel/Kconfig.test' file.

   - Support the upcoming Rust 1.98.0 release (expected 2026-08-20):
     lint cleanups and an unstable flag rename.

   - Disable 'rustdoc' documentation inlining for all prelude items,
     which bloats the generated documentation.

   - Ignore (in Git) and clean (in Kbuild) the (rarely) 'rustc'-generated
     '*.long-type-*.txt' files.

  'kernel' crate:

   - Add new 'bitfield' module with the 'bitfield!' macro (extracted
     from the existing 'register!' one), which declares integer types
     that are split into distinct bit fields of arbitrary length.

     Each field is a 'Bounded' of the appropriate bit width (ensuring
     values are properly validated and avoiding implicit data loss) and
     gets several generated getters and setters (infallible, 'const' and
     fallible) as well as associated constants ('_MASK', '_SHIFT' and
     '_RANGE'). It also supports fields that can be converted from/to
     custom types, either fallibly ('?=>') or infallibly ('=>').

     For instance:

         bitfield! {
             struct Rgb(u16) {
                 15:11 blue;
                 10:5 green;
                 4:0 red;
             }
         }

         // Compile-time checks.
         let color = Rgb::zeroed().with_const_green::<0x1f>();

         assert_eq!(color.green(), 0x1f);
         assert_eq!(color.into_raw(), 0x1f << Rgb::GREEN_SHIFT);

     Add as well documentation and a test suite for it, as usual; and
     update the 'register!' macro to use it.

     It will be maintained by Alexandre Courbot (with Yury Norov as
     reviewer) under a new 'MAINTAINERS' entry: 'RUST [BITFIELD]'.

   - 'ptr' module: rework index projection syntax into keyworded syntax
     and introduce panicking variant.

     The keyword syntax ('build:', 'try:', 'panic:') is more explicit
     and paves the way of perhaps adding more flavors in the future,
     e.g. an 'unsafe' index projection.

     For instance, projections now look like this:

         fn f(p: *const [u8; 32]) -> Result {
             // Ok, within bounds, checked at build time.
             project!(p, [build: 1]);

             // Build error.
             project!(p, [build: 128]);

             // `OutOfBound` runtime error (convertible to `ERANGE`).
             project!(p, [try: 128]);

             // Runtime panic.
             project!(p, [panic: 128]);

             Ok(())
         }

     Update as well the users, which now look like e.g.

         // Pointer to the first entry of the GSP message queue.
         let data = project!(self.0.as_ptr(), .gspq.msgq.data[build: 0]);

   - 'build_assert' module: make the module the home of its macros
     instead of rendering them twice.

   - 'sync' module: add 'UniqueArc::as_ptr()' associated function.

   - 'alloc' module:

       - Fix the 'Vec::reserve()' doctest to properly account for the
         existing vector length in the capacity assertion.

       - Fix an incorrect operator in the 'Vec::extend_with()' 'SAFETY'
         comment; add a doc test demonstrating basic usage and the
         zero-length case.

   - Clean imports across several modules to follow the "kernel
     vertical" import style in order to minimize conflicts.

  'pin-init' crate:

   - User visible changes:

       - Do not generate 'non_snake_case' warnings for identifiers that
         are syntactically just users of a field name. This would allow
         all '#[allow(non_snake_case)]' in nova-core to be removed,
         which Gary will send to the nova tree next cycle.

       - Filter non-cfg attributes out properly in derived structs. This
         improves pin-init compatibility with other derive macros.

       - Insert projection types' where clause properly.

   - Other changes:

       - Bump MSRV to 1.82, plus associated cleanups.

       - Overhaul how init slots are projected. The new approach is
         easier to justify with safety comments.

       - Mark more functions as inline, which should help mitigate the
         super-long symbol name issue due to lack of inlining.

  rust-analyzer:

   - Support '--envs' for passing env vars for crates like 'zerocopy'.

  'MAINTAINERS':

   - Add the following reviewers to the 'RUST' entry:
       - Daniel Almeida
       - Tamir Duberstein
       - Alexandre Courbot
       - Onur Özkan

     They have been involved in the Rust for Linux project for about 7
     collective years and bring expertise across several domains, which
     will be very useful to have around in the future.

     Thanks everyone for stepping up!

  And some other fixes, cleanups and improvements"

Link: https://github.com/google/zerocopy [1]
Link: https://docs.rs/zerocopy [2]
Link: https://github.com/Rust-for-Linux/linux/issues/1239 [3]

* tag 'rust-7.2' of gitolite.kernel.org:pub/scm/linux/kernel/git/ojeda/linux: (86 commits)
  MAINTAINERS: add Onur Özkan as Rust reviewer
  MAINTAINERS: add Alexandre Courbot as Rust reviewer
  MAINTAINERS: add Tamir Duberstein as Rust reviewer
  MAINTAINERS: add Daniel Almeida as Rust reviewer
  kbuild: rust: clean `zerocopy-derive` in `mrproper`
  rust: make `build_assert` module the home of related macros
  rust: str: clean unused import for Rust >= 1.98
  rust: str: use the "kernel vertical" imports style
  rust: aref: use the "kernel vertical" imports style
  rust: page: use the "kernel vertical" imports style
  gpu: nova-core: firmware: parse `FalconUCodeDescV2` via `zerocopy`
  rust: prelude: add `zerocopy{,_derive}::FromBytes`
  rust: zerocopy-derive: enable support in kbuild
  rust: zerocopy-derive: add `README.md`
  rust: zerocopy-derive: avoid generating non-ASCII identifiers
  rust: zerocopy-derive: add SPDX License Identifiers
  rust: zerocopy-derive: import crate
  rust: zerocopy: enable support in kbuild
  rust: zerocopy: add `README.md`
  rust: zerocopy: remove float `Display` support
  ...
This commit is contained in:
Linus Torvalds 2026-06-15 09:25:48 +05:30
commit b079329b86
310 changed files with 40556 additions and 844 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.gitignore vendored
View File

@ -188,5 +188,8 @@ sphinx_*/
# Rust analyzer configuration
/rust-project.json
# rustc error message long types
*.long-type-*.txt
# bc language scripts (not LLVM bitcode)
!kernel/time/timeconst.bc

5
Documentation/rust/testing.rst
View File

@ -140,11 +140,15 @@ are also mapped to KUnit.
These tests are introduced by the ``kunit_tests`` procedural macro, which takes
the name of the test suite as an argument.
Each test suite should be guarded by a Kconfig option in
``rust/kernel/Kconfig.test``.
For instance, assume we want to test the function ``f`` from the documentation
tests section. We could write, in the same file where we have our function:
.. code-block:: rust
#[cfg(CONFIG_RUST_MYMOD_KUNIT_TEST)]
#[kunit_tests(rust_kernel_mymod)]
mod tests {
use super::*;
@ -173,6 +177,7 @@ the unit type ``()``) or ``Result`` (i.e. any ``Result<T, E>``). For instance:
.. code-block:: rust
#[cfg(CONFIG_RUST_MYMOD_KUNIT_TEST)]
#[kunit_tests(rust_kernel_mymod)]
mod tests {
use super::*;

11
MAINTAINERS
View File

@ -23400,6 +23400,10 @@ R: Andreas Hindborg <a.hindborg@kernel.org>
R: Alice Ryhl <aliceryhl@google.com>
R: Trevor Gross <tmgross@umich.edu>
R: Danilo Krummrich <dakr@kernel.org>
R: Daniel Almeida <daniel.almeida@collabora.com>
R: Tamir Duberstein <tamird@kernel.org>
R: Alexandre Courbot <acourbot@nvidia.com>
R: Onur Özkan <work@onurozkan.dev>
L: rust-for-linux@vger.kernel.org
S: Supported
W: https://rust-for-linux.com
@ -23429,6 +23433,13 @@ T: git https://github.com/Rust-for-Linux/linux.git alloc-next
F: rust/kernel/alloc.rs
F: rust/kernel/alloc/
RUST [BITFIELD]
M: Alexandre Courbot <acourbot@nvidia.com>
R: Yury Norov <yury.norov@gmail.com>
L: rust-for-linux@vger.kernel.org
S: Maintained
F: rust/kernel/bitfield.rs
RUST [INTEROP]
M: Joel Fernandes <joelagnelf@nvidia.com>
M: Alexandre Courbot <acourbot@nvidia.com>

6
Makefile
View File

@ -1709,7 +1709,8 @@ MRPROPER_FILES += include/config include/generated \
vmlinux-gdb.py \
rpmbuild \
rust/libmacros.so rust/libmacros.dylib \
rust/libpin_init_internal.so rust/libpin_init_internal.dylib
rust/libpin_init_internal.so rust/libpin_init_internal.dylib \
rust/libzerocopy_derive.so rust/libzerocopy_derive.dylib
# clean - Delete most, but leave enough to build external modules
#
@ -1961,6 +1962,8 @@ rustfmt:
-path $(srctree)/rust/proc-macro2 \
-o -path $(srctree)/rust/quote \
-o -path $(srctree)/rust/syn \
-o -path $(srctree)/rust/zerocopy \
-o -path $(srctree)/rust/zerocopy-derive \
\) -prune -o \
-type f -a -name '*.rs' -a ! -name '*generated*' -print \
| xargs $(RUSTFMT) $(rustfmt_flags)
@ -2169,6 +2172,7 @@ clean: $(clean-dirs)
-o -name '*.c.[012]*.*' \
-o -name '*.ll' \
-o -name '*.gcno' \
-o -name '*.long-type-*.txt' \
\) -type f -print \
-o -name '.tmp_*' -print \
| xargs rm -rf

4
drivers/gpu/nova-core/bitfield.rs
View File

@ -170,7 +170,7 @@ impl $name {
(@check_field_bounds $hi:tt:$lo:tt $field:ident as bool) => {
#[allow(clippy::eq_op)]
const _: () = {
::kernel::build_assert!(
::kernel::build_assert::build_assert!(
$hi == $lo,
concat!("boolean field `", stringify!($field), "` covers more than one bit")
);
@ -181,7 +181,7 @@ impl $name {
(@check_field_bounds $hi:tt:$lo:tt $field:ident as $type:tt) => {
#[allow(clippy::eq_op)]
const _: () = {
::kernel::build_assert!(
::kernel::build_assert::build_assert!(
$hi >= $lo,
concat!("field `", stringify!($field), "`'s MSB is smaller than its LSB")
);

5
drivers/gpu/nova-core/firmware.rs
View File

@ -48,7 +48,7 @@ fn request_firmware(
/// Structure used to describe some firmwares, notably FWSEC-FRTS.
#[repr(C)]
#[derive(Debug, Clone)]
#[derive(Debug, Clone, FromBytes)]
pub(crate) struct FalconUCodeDescV2 {
/// Header defined by 'NV_BIT_FALCON_UCODE_DESC_HEADER_VDESC*' in OpenRM.
hdr: u32,
@ -84,9 +84,6 @@ pub(crate) struct FalconUCodeDescV2 {
pub(crate) alt_dmem_load_size: u32,
}
// SAFETY: all bit patterns are valid for this type, and it doesn't use interior mutability.
unsafe impl FromBytes for FalconUCodeDescV2 {}
/// Structure used to describe some firmwares, notably FWSEC-FRTS.
#[repr(C)]
#[derive(Debug, Clone)]

6
drivers/gpu/nova-core/gsp/cmdq.rs
View File

@ -237,7 +237,7 @@ fn new(dev: &device::Device<device::Bound>) -> Result<Self> {
let start = gsp_mem.dma_handle();
// Write values one by one to avoid an on-stack instance of `PteArray`.
for i in 0..GspMem::PTE_ARRAY_SIZE {
dma_write!(gsp_mem, .ptes.0[i], PteArray::<0>::entry(start, i)?);
dma_write!(gsp_mem, .ptes.0[build: i], PteArray::<0>::entry(start, i)?);
}
dma_write!(
@ -260,7 +260,7 @@ fn new(dev: &device::Device<device::Bound>) -> Result<Self> {
let rx = self.gsp_read_ptr();
// Pointer to the first entry of the CPU message queue.
let data = ptr::project!(mut self.0.as_mut_ptr(), .cpuq.msgq.data[0]);
let data = ptr::project!(mut self.0.as_mut_ptr(), .cpuq.msgq.data[build: 0]);
let (tail_end, wrap_end) = if rx == 0 {
// The write area is non-wrapping, and stops at the second-to-last entry of the command
@ -322,7 +322,7 @@ fn driver_write_area_size(&self) -> usize {
let rx = self.cpu_read_ptr();
// Pointer to the first entry of the GSP message queue.
let data = ptr::project!(self.0.as_ptr(), .gspq.msgq.data[0]);
let data = ptr::project!(self.0.as_ptr(), .gspq.msgq.data[build: 0]);
let (tail_end, wrap_end) = if rx <= tx {
// Read area is non-wrapping and stops right before `tx`.

2
drivers/gpu/nova-core/num.rs
View File

@ -49,7 +49,7 @@ macro_rules! impl_safe_as {
#[allow(unused)]
#[inline(always)]
pub(crate) const fn [<$from _as_ $into>](value: $from) -> $into {
kernel::static_assert!(size_of::<$into>() >= size_of::<$from>());
::kernel::build_assert::static_assert!(size_of::<$into>() >= size_of::<$from>());
value as $into
}

6
drivers/gpu/nova-core/vbios.rs
View File

@ -16,6 +16,8 @@
transmute::FromBytes,
};
use zerocopy::FromBytes as _;
use crate::{
driver::Bar0,
firmware::{
@ -1011,8 +1013,8 @@ pub(crate) fn header(&self) -> Result<FalconUCodeDesc> {
let data = self.base.data.get(falcon_ucode_offset..).ok_or(EINVAL)?;
match ver {
2 => {
let v2 = FalconUCodeDescV2::from_bytes_copy_prefix(data)
.ok_or(EINVAL)?
let v2 = FalconUCodeDescV2::read_from_prefix(data)
.map_err(|_| EINVAL)?
.0;
Ok(FalconUCodeDesc::V2(v2))
}

5
init/Kconfig
View File

@ -2195,7 +2195,8 @@ config RUST
depends on !DEBUG_INFO_BTF || (PAHOLE_HAS_LANG_EXCLUDE && !LTO)
depends on !CFI || HAVE_CFI_ICALL_NORMALIZE_INTEGERS_RUSTC
select CFI_ICALL_NORMALIZE_INTEGERS if CFI
depends on !KASAN_SW_TAGS
depends on !KASAN || CC_IS_CLANG
depends on !KASAN_SW_TAGS || RUSTC_VERSION >= 109600
help
Enables Rust support in the kernel.
@ -2209,6 +2210,8 @@ config RUST
If unsure, say N.
source "rust/kernel/Kconfig.test"
config RUSTC_VERSION_TEXT
string
depends on RUST

96
rust/Makefile
View File

@ -6,6 +6,8 @@ rustdoc_output := $(objtree)/Documentation/output/rust/rustdoc
obj-$(CONFIG_RUST) += core.o compiler_builtins.o ffi.o
always-$(CONFIG_RUST) += exports_core_generated.h
obj-$(CONFIG_RUST) += zerocopy.o
ifdef CONFIG_RUST_INLINE_HELPERS
always-$(CONFIG_RUST) += helpers/helpers.bc helpers/helpers_module.bc
else
@ -47,13 +49,14 @@ endif
# Avoids running `$(RUSTC)` when it may not be available.
ifdef CONFIG_RUST
libmacros_name := $(shell MAKEFLAGS= $(RUSTC) --print file-names --crate-name macros --crate-type proc-macro - </dev/null)
libmacros_extension := $(patsubst libmacros.%,%,$(libmacros_name))
procmacro-name = $(shell MAKEFLAGS= $(RUSTC) --print file-names --crate-name $(1) --crate-type proc-macro - </dev/null)
procmacro-extension := $(patsubst libname.%,%,$(call procmacro-name,name))
libpin_init_internal_name := $(shell MAKEFLAGS= $(RUSTC) --print file-names --crate-name pin_init_internal --crate-type proc-macro - </dev/null)
libpin_init_internal_extension := $(patsubst libpin_init_internal.%,%,$(libpin_init_internal_name))
libzerocopy_derive_name := $(call procmacro-name,zerocopy_derive)
libmacros_name := $(call procmacro-name,macros)
libpin_init_internal_name := $(call procmacro-name,pin_init_internal)
always-$(CONFIG_RUST) += $(libmacros_name) $(libpin_init_internal_name)
always-$(CONFIG_RUST) += $(libzerocopy_derive_name) $(libmacros_name) $(libpin_init_internal_name)
# `$(rust_flags)` is passed in case the user added `--sysroot`.
rustc_sysroot := $(shell MAKEFLAGS= $(RUSTC) $(rust_flags) --print sysroot)
@ -81,6 +84,12 @@ core-flags := \
--edition=$(core-edition) \
$(call cfgs-to-flags,$(core-cfgs))
zerocopy-flags := \
--cap-lints=allow
zerocopy-envs := \
CARGO_PKG_VERSION=0.8.50
proc_macro2-cfgs := \
feature="proc-macro" \
wrap_proc_macro \
@ -118,6 +127,12 @@ syn-flags := \
--extern quote \
$(call cfgs-to-flags,$(syn-cfgs))
zerocopy_derive-flags := \
--cap-lints=allow \
--extern proc_macro2 \
--extern quote \
--extern syn
pin_init_internal-cfgs := \
kernel USE_RUSTC_FEATURES
@ -144,6 +159,7 @@ doctests_modifiers_workaround := $(rustdoc_modifiers_workaround)$(if $(call rust
quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $<
cmd_rustdoc = \
$(rustc_target_envs) \
OBJTREE=$(abspath $(objtree)) \
$(RUSTDOC) $(filter-out $(skip_flags) --remap-path-scope=%,$(if $(rustdoc_host),$(rust_common_flags),$(rust_flags))) \
$(rustc_target_flags) -L$(objtree)/$(obj) \
@ -166,7 +182,7 @@ quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $<
# command-like flags to solve the issue. Meanwhile, we use the non-custom case
# and then retouch the generated files.
rustdoc: rustdoc-core rustdoc-macros rustdoc-compiler_builtins \
rustdoc-kernel rustdoc-pin_init
rustdoc-kernel rustdoc-pin_init rustdoc-zerocopy rustdoc-zerocopy_derive
$(Q)grep -Ehro '<a href="srctree/([^"]+)"' $(rustdoc_output) | \
cut -d'"' -f2 | cut -d/ -f2- | while read f; do \
if [ ! -e "$(srctree)/$$f" ]; then \
@ -199,6 +215,12 @@ rustdoc-syn: private rustc_target_flags = $(syn-flags)
rustdoc-syn: $(src)/syn/lib.rs rustdoc-clean rustdoc-quote FORCE
+$(call if_changed,rustdoc)
rustdoc-zerocopy_derive: private rustdoc_host = yes
rustdoc-zerocopy_derive: private rustc_target_flags = $(zerocopy_derive-flags) \
--extern proc_macro --crate-type proc-macro
rustdoc-zerocopy_derive: $(src)/zerocopy-derive/lib.rs rustdoc-clean rustdoc-syn FORCE
+$(call if_changed,rustdoc)
rustdoc-macros: private rustdoc_host = yes
rustdoc-macros: private rustc_target_flags = --crate-type proc-macro \
--extern proc_macro --extern proc_macro2 --extern quote --extern syn
@ -218,6 +240,13 @@ rustdoc-compiler_builtins: private is-kernel-object := y
rustdoc-compiler_builtins: $(src)/compiler_builtins.rs rustdoc-core FORCE
+$(call if_changed,rustdoc)
rustdoc-zerocopy: private rustc_target_envs := $(zerocopy-envs)
rustdoc-zerocopy: private is-kernel-object := y
rustdoc-zerocopy: private rustc_target_flags = $(zerocopy-flags) \
--extend-css $(src)/zerocopy/rustdoc/style.css
rustdoc-zerocopy: $(src)/zerocopy/src/lib.rs rustdoc-clean rustdoc-core FORCE
+$(call if_changed,rustdoc)
rustdoc-ffi: private is-kernel-object := y
rustdoc-ffi: $(src)/ffi.rs rustdoc-core FORCE
+$(call if_changed,rustdoc)
@ -239,7 +268,8 @@ rustdoc-pin_init: $(src)/pin-init/src/lib.rs rustdoc-pin_init_internal \
rustdoc-kernel: private is-kernel-object := y
rustdoc-kernel: private rustc_target_flags = --extern ffi --extern pin_init \
--extern build_error --extern macros \
--extern bindings --extern uapi
--extern bindings --extern uapi \
--extern zerocopy --extern zerocopy_derive
rustdoc-kernel: $(src)/kernel/lib.rs rustdoc-core rustdoc-ffi rustdoc-macros \
rustdoc-pin_init rustdoc-compiler_builtins $(obj)/$(libmacros_name) \
$(obj)/bindings.o FORCE
@ -250,6 +280,7 @@ rustdoc-clean: FORCE
quiet_cmd_rustc_test_library = $(RUSTC_OR_CLIPPY_QUIET) TL $<
cmd_rustc_test_library = \
$(rustc_target_envs) \
OBJTREE=$(abspath $(objtree)) \
$(RUSTC_OR_CLIPPY) $(filter-out $(skip_flags),$(rust_common_flags) $(rustc_target_flags)) \
@$(objtree)/include/generated/rustc_cfg \
@ -258,6 +289,11 @@ quiet_cmd_rustc_test_library = $(RUSTC_OR_CLIPPY_QUIET) TL $<
-L$(objtree)/$(obj)/test \
--crate-name $(subst rusttest-,,$(subst rusttestlib-,,$@)) $<
rusttestlib-zerocopy: private rustc_target_envs := $(zerocopy-envs)
rusttestlib-zerocopy: private rustc_target_flags = $(zerocopy-flags)
rusttestlib-zerocopy: $(src)/zerocopy/src/lib.rs FORCE
+$(call if_changed,rustc_test_library)
rusttestlib-build_error: $(src)/build_error.rs FORCE
+$(call if_changed,rustc_test_library)
@ -277,6 +313,12 @@ rusttestlib-syn: private rustc_target_flags = $(syn-flags)
rusttestlib-syn: $(src)/syn/lib.rs rusttestlib-quote FORCE
+$(call if_changed,rustc_test_library)
rusttestlib-zerocopy_derive: private rustc_target_flags = $(zerocopy_derive-flags) \
--extern proc_macro
rusttestlib-zerocopy_derive: private rustc_test_library_proc = yes
rusttestlib-zerocopy_derive: $(src)/zerocopy-derive/lib.rs rusttestlib-syn FORCE
+$(call if_changed,rustc_test_library)
rusttestlib-macros: private rustc_target_flags = --extern proc_macro \
--extern proc_macro2 --extern quote --extern syn
rusttestlib-macros: private rustc_test_library_proc = yes
@ -298,10 +340,11 @@ rusttestlib-pin_init: $(src)/pin-init/src/lib.rs rusttestlib-macros \
rusttestlib-kernel: private rustc_target_flags = --extern ffi \
--extern build_error --extern macros --extern pin_init \
--extern bindings --extern uapi
--extern bindings --extern uapi \
--extern zerocopy --extern zerocopy_derive
rusttestlib-kernel: $(src)/kernel/lib.rs rusttestlib-bindings rusttestlib-uapi \
rusttestlib-build_error rusttestlib-pin_init $(obj)/$(libmacros_name) \
$(obj)/bindings.o FORCE
$(obj)/bindings.o rusttestlib-zerocopy rusttestlib-zerocopy_derive FORCE
+$(call if_changed,rustc_test_library)
rusttestlib-bindings: private rustc_target_flags = --extern ffi --extern pin_init
@ -314,6 +357,7 @@ rusttestlib-uapi: $(src)/uapi/lib.rs rusttestlib-ffi rusttestlib-pin_init FORCE
quiet_cmd_rustdoc_test = RUSTDOC T $<
cmd_rustdoc_test = \
$(rustc_target_envs) \
RUST_MODFILE=test.rs \
OBJTREE=$(abspath $(objtree)) \
$(RUSTDOC) --test $(rust_common_flags) \
@ -328,11 +372,13 @@ quiet_cmd_rustdoc_test_kernel = RUSTDOC TK $<
cmd_rustdoc_test_kernel = \
rm -rf $(objtree)/$(obj)/test/doctests/kernel; \
mkdir -p $(objtree)/$(obj)/test/doctests/kernel; \
$(rustc_target_envs) \
OBJTREE=$(abspath $(objtree)) \
$(RUSTDOC) --test $(filter-out --remap-path-scope=%,$(rust_flags)) \
-L$(objtree)/$(obj) --extern ffi --extern pin_init \
--extern kernel --extern build_error --extern macros \
--extern bindings --extern uapi \
--extern zerocopy --extern zerocopy_derive \
--no-run --crate-name kernel -Zunstable-options \
--sysroot=/dev/null \
$(doctests_modifiers_workaround) \
@ -350,6 +396,7 @@ quiet_cmd_rustdoc_test_kernel = RUSTDOC TK $<
# so for the moment we skip `-Cpanic=abort`.
quiet_cmd_rustc_test = $(RUSTC_OR_CLIPPY_QUIET) T $<
cmd_rustc_test = \
$(rustc_target_envs) \
OBJTREE=$(abspath $(objtree)) \
$(RUSTC_OR_CLIPPY) --test $(rust_common_flags) \
@$(objtree)/include/generated/rustc_cfg \
@ -517,8 +564,9 @@ $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE
$(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE
$(call if_changed,exports)
quiet_cmd_rustc_procmacrolibrary = $(RUSTC_OR_CLIPPY_QUIET) PL $@
quiet_cmd_rustc_procmacrolibrary = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) PL $@
cmd_rustc_procmacrolibrary = \
$(rustc_target_envs) \
$(if $(skip_clippy),$(RUSTC),$(RUSTC_OR_CLIPPY)) \
$(filter-out $(skip_flags),$(rust_common_flags) $(rustc_target_flags)) \
--emit=dep-info=$(depfile) --emit=link=$@ --crate-type rlib -O \
@ -541,17 +589,24 @@ $(obj)/libsyn.rlib: private rustc_target_flags = $(syn-flags)
$(obj)/libsyn.rlib: $(src)/syn/lib.rs $(obj)/libquote.rlib FORCE
+$(call if_changed_dep,rustc_procmacrolibrary)
quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@
quiet_cmd_rustc_procmacro = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) P $@
cmd_rustc_procmacro = \
$(RUSTC_OR_CLIPPY) $(rust_common_flags) $(rustc_target_flags) \
$(rustc_target_envs) \
$(if $(skip_clippy),$(RUSTC),$(RUSTC_OR_CLIPPY)) $(rust_common_flags) $(rustc_target_flags) \
-Clinker-flavor=gcc -Clinker=$(HOSTCC) \
-Clink-args='$(call escsq,$(KBUILD_PROCMACROLDFLAGS))' \
--emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \
--crate-type proc-macro -L$(objtree)/$(obj) \
--crate-name $(patsubst lib%.$(libmacros_extension),%,$(notdir $@)) \
--crate-name $(patsubst lib%.$(procmacro-extension),%,$(notdir $@)) \
@$(objtree)/include/generated/rustc_cfg $<
# Procedural macros can only be used with the `rustc` that compiled it.
$(obj)/$(libzerocopy_derive_name): private skip_clippy = 1
$(obj)/$(libzerocopy_derive_name): private rustc_target_flags = $(zerocopy_derive-flags)
$(obj)/$(libzerocopy_derive_name): $(src)/zerocopy-derive/lib.rs $(obj)/libproc_macro2.rlib \
$(obj)/libquote.rlib $(obj)/libsyn.rlib FORCE
+$(call if_changed_dep,rustc_procmacro)
$(obj)/$(libmacros_name): private rustc_target_flags = \
--extern proc_macro2 --extern quote --extern syn
$(obj)/$(libmacros_name): $(src)/macros/lib.rs $(obj)/libproc_macro2.rlib \
@ -567,6 +622,7 @@ $(obj)/$(libpin_init_internal_name): $(src)/pin-init/internal/src/lib.rs \
# since Rust 1.95.0 (https://github.com/rust-lang/rust/pull/151534).
quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L $@
cmd_rustc_library = \
$(rustc_target_envs) \
OBJTREE=$(abspath $(objtree)) \
$(if $(skip_clippy),$(RUSTC),$(RUSTC_OR_CLIPPY)) \
$(filter-out $(skip_flags),$(rust_flags)) $(rustc_target_flags) \
@ -591,6 +647,7 @@ rust-analyzer:
--cfgs='syn=$(syn-cfgs)' \
--cfgs='pin_init_internal=$(pin_init_internal-cfgs)' \
--cfgs='pin_init=$(pin_init-cfgs)' \
--envs='zerocopy=$(zerocopy-envs)' \
$(realpath $(srctree)) $(realpath $(objtree)) \
$(rustc_sysroot) $(RUST_LIB_SRC) $(if $(KBUILD_EXTMOD),$(srcroot)) \
> rust-project.json
@ -662,6 +719,13 @@ $(obj)/compiler_builtins.o: private rustc_objcopy = -w -W '__*'
$(obj)/compiler_builtins.o: $(src)/compiler_builtins.rs $(obj)/core.o FORCE
+$(call if_changed_rule,rustc_library)
$(obj)/zerocopy.o: private skip_clippy = 1
$(obj)/zerocopy.o: private skip_gendwarfksyms = 1
$(obj)/zerocopy.o: private rustc_target_envs := $(zerocopy-envs)
$(obj)/zerocopy.o: private rustc_target_flags = $(zerocopy-flags)
$(obj)/zerocopy.o: $(src)/zerocopy/src/lib.rs $(obj)/compiler_builtins.o FORCE
+$(call if_changed_rule,rustc_library)
$(obj)/pin_init.o: private skip_gendwarfksyms = 1
$(obj)/pin_init.o: private rustc_target_flags = $(pin_init-flags)
$(obj)/pin_init.o: $(src)/pin-init/src/lib.rs $(obj)/compiler_builtins.o \
@ -697,9 +761,11 @@ $(obj)/uapi.o: $(src)/uapi/lib.rs \
+$(call if_changed_rule,rustc_library)
$(obj)/kernel.o: private rustc_target_flags = --extern ffi --extern pin_init \
--extern build_error --extern macros --extern bindings --extern uapi
--extern build_error --extern macros --extern bindings --extern uapi \
--extern zerocopy --extern zerocopy_derive
$(obj)/kernel.o: $(src)/kernel/lib.rs $(obj)/build_error.o $(obj)/pin_init.o \
$(obj)/$(libmacros_name) $(obj)/bindings.o $(obj)/uapi.o FORCE
$(obj)/$(libmacros_name) $(obj)/bindings.o $(obj)/uapi.o \
$(obj)/zerocopy.o $(obj)/$(libzerocopy_derive_name) FORCE
+$(call if_changed_rule,rustc_library)
ifdef CONFIG_JUMP_LABEL

86
rust/kernel/Kconfig.test Normal file
View File

@ -0,0 +1,86 @@
# SPDX-License-Identifier: GPL-2.0-only
menuconfig RUST_KUNIT_TESTS
bool "Rust KUnit tests"
depends on KUNIT && RUST
default KUNIT_ALL_TESTS
help
This menu collects all options for Rust KUnit tests.
See Documentation/rust/testing.rst for how to protect
unit tests with these options.
Say Y here to enable Rust KUnit tests.
If unsure, say N.
if RUST_KUNIT_TESTS
config RUST_ALLOCATOR_KUNIT_TEST
bool "KUnit tests for Rust allocator API" if !KUNIT_ALL_TESTS
default KUNIT_ALL_TESTS
help
This option enables KUnit tests for the Rust allocator API.
These are only for development and testing, not for regular
kernel use cases.
If unsure, say N.
config RUST_KVEC_KUNIT_TEST
bool "KUnit tests for Rust KVec API" if !KUNIT_ALL_TESTS
default KUNIT_ALL_TESTS
help
This option enables KUnit tests for the Rust KVec API.
These are only for development and testing, not for
regular kernel use cases.
If unsure, say N.
config RUST_BITMAP_KUNIT_TEST
bool "KUnit tests for Rust bitmap API" if !KUNIT_ALL_TESTS
default KUNIT_ALL_TESTS
help
This option enables KUnit tests for the Rust bitmap API.
These are only for development and testing, not for regular
kernel use cases.
If unsure, say N.
config RUST_KUNIT_SELFTEST
bool "KUnit selftests for Rust" if !KUNIT_ALL_TESTS
default KUNIT_ALL_TESTS
help
This option enables KUnit selftests. These are only
for development and testing, not for regular kernel
use cases.
If unsure, say N.
config RUST_STR_KUNIT_TEST
bool "KUnit tests for Rust strings API" if !KUNIT_ALL_TESTS
default KUNIT_ALL_TESTS
help
This option enables KUnit tests for the Rust strings API.
These are only for development and testing, not for regular
kernel use cases.
If unsure, say N.
config RUST_ATOMICS_KUNIT_TEST
bool "KUnit tests for Rust atomics API" if !KUNIT_ALL_TESTS
default KUNIT_ALL_TESTS
help
This option enables KUnit tests for the Rust atomics API.
These are only for development and testing, not for regular
kernel use cases.
If unsure, say N.
config RUST_BITFIELD_KUNIT_TEST
bool "KUnit tests for the Rust `bitfield!` macro" if !KUNIT_ALL_TESTS
default KUNIT_ALL_TESTS
help
This option enables KUnit tests for the Rust `bitfield!` macro.
These are only for development and testing, not for regular
kernel use cases.
If unsure, say N.
endif

8
rust/kernel/alloc.rs
View File

@ -22,8 +22,12 @@
#[derive(Copy, Clone, PartialEq, Eq, Debug)]
pub struct AllocError;
use crate::error::{code::EINVAL, Result};
use core::{alloc::Layout, ptr::NonNull};
use crate::prelude::*;
use core::{
alloc::Layout,
ptr::NonNull, //
};
/// Flags to be used when allocating memory.
///

33
rust/kernel/alloc/allocator.rs
View File

@ -8,14 +8,25 @@
//!
//! Reference: <https://docs.kernel.org/core-api/memory-allocation.html>
use super::Flags;
use core::alloc::Layout;
use core::ptr;
use core::ptr::NonNull;
use super::{
AllocError,
Allocator,
Flags,
NumaNode, //
};
use crate::alloc::{AllocError, Allocator, NumaNode};
use crate::bindings;
use crate::page;
use crate::{
bindings,
page, //
};
use core::{
alloc::Layout,
ptr::{
self,
NonNull, //
}, //
};
const ARCH_KMALLOC_MINALIGN: usize = bindings::ARCH_KMALLOC_MINALIGN;
@ -163,8 +174,11 @@ impl Vmalloc {
/// # Examples
///
/// ```
/// # use core::ptr::{NonNull, from_mut};
/// # use kernel::{page, prelude::*};
/// # use core::ptr::{
/// # from_mut,
/// # NonNull, //
/// # };
/// # use kernel::page;
/// use kernel::alloc::allocator::Vmalloc;
///
/// let mut vbox = VBox::<[u8; page::PAGE_SIZE]>::new_uninit(GFP_KERNEL)?;
@ -251,6 +265,7 @@ unsafe fn realloc(
}
}
#[cfg(CONFIG_RUST_ALLOCATOR_KUNIT_TEST)]
#[macros::kunit_tests(rust_allocator)]
mod tests {
use super::*;

8
rust/kernel/alloc/allocator/iter.rs
View File

@ -1,9 +1,13 @@
// SPDX-License-Identifier: GPL-2.0
use super::Vmalloc;
use crate::page;
use core::marker::PhantomData;
use core::ptr::NonNull;
use core::{
marker::PhantomData,
ptr::NonNull, //
};
/// An [`Iterator`] of [`page::BorrowedPage`] items owned by a [`Vmalloc`] allocation.
///

76
rust/kernel/alloc/kbox.rs
View File

@ -3,24 +3,47 @@
//! Implementation of [`Box`].
#[allow(unused_imports)] // Used in doc comments.
use super::allocator::{KVmalloc, Kmalloc, Vmalloc, VmallocPageIter};
use super::{AllocError, Allocator, Flags, NumaNode};
use core::alloc::Layout;
use core::borrow::{Borrow, BorrowMut};
use core::marker::PhantomData;
use core::mem::ManuallyDrop;
use core::mem::MaybeUninit;
use core::ops::{Deref, DerefMut};
use core::pin::Pin;
use core::ptr::NonNull;
use core::result::Result;
use super::allocator::{
KVmalloc,
Kmalloc,
Vmalloc,
VmallocPageIter, //
};
use crate::ffi::c_void;
use crate::fmt;
use crate::init::InPlaceInit;
use crate::page::AsPageIter;
use crate::types::ForeignOwnable;
use pin_init::{InPlaceWrite, Init, PinInit, ZeroableOption};
use super::{
AllocError,
Allocator,
Flags,
NumaNode, //
};
use crate::{
fmt,
page::AsPageIter,
prelude::*,
types::ForeignOwnable, //
};
use core::{
alloc::Layout,
borrow::{
Borrow,
BorrowMut, //
},
marker::PhantomData,
mem::{
ManuallyDrop,
MaybeUninit, //
},
ops::{
Deref,
DerefMut, //
},
ptr::NonNull,
result::Result, //
};
use pin_init::ZeroableOption;
/// The kernel's [`Box`] type -- a heap allocation for a single value of type `T`.
///
@ -274,7 +297,10 @@ pub fn pin(x: T, flags: Flags) -> Result<Pin<Box<T, A>>, AllocError>
/// # Examples
///
/// ```
/// use kernel::sync::{new_spinlock, SpinLock};
/// use kernel::sync::{
/// new_spinlock,
/// SpinLock, //
/// };
///
/// struct Inner {
/// a: u32,
@ -411,6 +437,7 @@ impl<T, A> InPlaceWrite<T> for Box<MaybeUninit<T>, A>
{
type Initialized = Box<T, A>;
#[inline]
fn write_init<E>(mut self, init: impl Init<T, E>) -> Result<Self::Initialized, E> {
let slot = self.as_mut_ptr();
// SAFETY: When init errors/panics, slot will get deallocated but not dropped,
@ -420,6 +447,7 @@ fn write_init<E>(mut self, init: impl Init<T, E>) -> Result<Self::Initialized, E
Ok(unsafe { Box::assume_init(self) })
}
#[inline]
fn write_pin_init<E>(mut self, init: impl PinInit<T, E>) -> Result<Pin<Self::Initialized>, E> {
let slot = self.as_mut_ptr();
// SAFETY: When init errors/panics, slot will get deallocated but not dropped,
@ -567,7 +595,6 @@ fn deref_mut(&mut self) -> &mut T {
///
/// ```
/// # use core::borrow::Borrow;
/// # use kernel::alloc::KBox;
/// struct Foo<B: Borrow<u32>>(B);
///
/// // Owned instance.
@ -595,7 +622,6 @@ fn borrow(&self) -> &T {
///
/// ```
/// # use core::borrow::BorrowMut;
/// # use kernel::alloc::KBox;
/// struct Foo<B: BorrowMut<u32>>(B);
///
/// // Owned instance.
@ -660,9 +686,13 @@ fn drop(&mut self) {
/// # Examples
///
/// ```
/// # use kernel::prelude::*;
/// use kernel::alloc::allocator::VmallocPageIter;
/// use kernel::page::{AsPageIter, PAGE_SIZE};
/// use kernel::{
/// alloc::allocator::VmallocPageIter,
/// page::{
/// AsPageIter,
/// PAGE_SIZE, //
/// }, //
/// };
///
/// let mut vbox = VBox::new((), GFP_KERNEL)?;
///

82
rust/kernel/alloc/kvec.rs
View File

@ -3,29 +3,52 @@
//! Implementation of [`Vec`].
use super::{
allocator::{KVmalloc, Kmalloc, Vmalloc, VmallocPageIter},
allocator::{
KVmalloc,
Kmalloc,
Vmalloc,
VmallocPageIter, //
},
layout::ArrayLayout,
AllocError, Allocator, Box, Flags, NumaNode,
AllocError,
Allocator,
Box,
Flags,
NumaNode, //
};
use crate::{
fmt,
page::{
AsPageIter,
PAGE_SIZE, //
},
}, //
};
use core::{
borrow::{Borrow, BorrowMut},
borrow::{
Borrow,
BorrowMut, //
},
marker::PhantomData,
mem::{ManuallyDrop, MaybeUninit},
ops::Deref,
ops::DerefMut,
ops::Index,
ops::IndexMut,
ptr,
ptr::NonNull,
slice,
slice::SliceIndex,
mem::{
ManuallyDrop,
MaybeUninit, //
},
ops::{
Deref,
DerefMut,
Index,
IndexMut, //
},
ptr::{
self,
NonNull, //
},
slice::{
self,
SliceIndex, //
}, //
};
mod errors;
@ -614,7 +637,7 @@ pub fn clear(&mut self) {
///
/// v.reserve(10, GFP_KERNEL)?;
/// let cap = v.capacity();
/// assert!(cap >= 10);
/// assert!(cap >= v.len() + 10);
///
/// v.reserve(10, GFP_KERNEL)?;
/// let new_cap = v.capacity();
@ -849,6 +872,24 @@ pub fn shrink_to(&mut self, min_capacity: usize, flags: Flags) -> Result<(), All
impl<T: Clone, A: Allocator> Vec<T, A> {
/// Extend the vector by `n` clones of `value`.
///
/// # Examples
///
/// ```
/// let mut v = KVec::new();
/// v.push(1, GFP_KERNEL)?;
///
/// v.extend_with(3, 5, GFP_KERNEL)?;
/// assert_eq!(&v, &[1, 5, 5, 5]);
///
/// v.extend_with(2, 8, GFP_KERNEL)?;
/// assert_eq!(&v, &[1, 5, 5, 5, 8, 8]);
///
/// v.extend_with(0, 3, GFP_KERNEL)?;
/// assert_eq!(&v, &[1, 5, 5, 5, 8, 8]);
///
/// # Ok::<(), Error>(())
/// ```
pub fn extend_with(&mut self, n: usize, value: T, flags: Flags) -> Result<(), AllocError> {
if n == 0 {
return Ok(());
@ -866,7 +907,7 @@ pub fn extend_with(&mut self, n: usize, value: T, flags: Flags) -> Result<(), Al
spare[n - 1].write(value);
// SAFETY:
// - `self.len() + n < self.capacity()` due to the call to reserve above,
// - `self.len() + n <= self.capacity()` due to the call to reserve above,
// - the loop and the line above initialized the next `n` elements.
unsafe { self.inc_len(n) };
@ -1146,9 +1187,13 @@ fn into_iter(self) -> Self::IntoIter {
/// # Examples
///
/// ```
/// # use kernel::prelude::*;
/// use kernel::alloc::allocator::VmallocPageIter;
/// use kernel::page::{AsPageIter, PAGE_SIZE};
/// use kernel::{
/// alloc::allocator::VmallocPageIter,
/// page::{
/// AsPageIter,
/// PAGE_SIZE, //
/// }, //
/// };
///
/// let mut vec = VVec::<u8>::new();
///
@ -1463,6 +1508,7 @@ fn drop(&mut self) {
}
}
#[cfg(CONFIG_RUST_KVEC_KUNIT_TEST)]
#[macros::kunit_tests(rust_kvec)]
mod tests {
use super::*;

6
rust/kernel/alloc/kvec/errors.rs
View File

@ -2,8 +2,10 @@
//! Errors for the [`Vec`] type.
use kernel::fmt;
use kernel::prelude::*;
use crate::{
fmt,
prelude::*, //
};
/// Error type for [`Vec::push_within_capacity`].
pub struct PushError<T>(pub T);

10
rust/kernel/alloc/layout.rs
View File

@ -4,7 +4,10 @@
//!
//! Custom layout types extending or improving [`Layout`].
use core::{alloc::Layout, marker::PhantomData};
use core::{
alloc::Layout,
marker::PhantomData, //
};
/// Error when constructing an [`ArrayLayout`].
pub struct LayoutError;
@ -47,7 +50,10 @@ pub const fn empty() -> Self {
/// # Examples
///
/// ```
/// # use kernel::alloc::layout::{ArrayLayout, LayoutError};
/// # use kernel::alloc::layout::{
/// # ArrayLayout,
/// # LayoutError, //
/// # };
/// let layout = ArrayLayout::<i32>::new(15)?;
/// assert_eq!(layout.len(), 15);
///

862
rust/kernel/bitfield.rs Normal file
View File

@ -0,0 +1,862 @@
// SPDX-License-Identifier: GPL-2.0
//! Support for defining bitfields as Rust structures.
//!
//! The [`bitfield!`](kernel::bitfield!) macro declares integer types that are split into distinct
//! bit fields of arbitrary length. Each field is typed using [`Bounded`](kernel::num::Bounded) to
//! ensure values are properly validated and to avoid implicit data loss.
//!
//! # Example
//!
//! ```rust
//! use kernel::bitfield;
//! use kernel::num::Bounded;
//!
//! bitfield! {
//! pub struct Rgb(u16) {
//! 15:11 blue;
//! 10:5 green;
//! 4:0 red;
//! }
//! }
//!
//! // Valid value for the `blue` field.
//! let blue = Bounded::<u16, 5>::new::<0x18>();
//!
//! // Setters can be chained. Values ranges are checked at compile-time.
//! let color = Rgb::zeroed()
//! // Compile-time bounds check of constant value.
//! .with_const_red::<0x10>()
//! .with_const_green::<0x1f>()
//! // A `Bounded` can also be passed.
//! .with_blue(blue);
//!
//! assert_eq!(color.red(), 0x10);
//! assert_eq!(color.green(), 0x1f);
//! assert_eq!(color.blue(), 0x18);
//! assert_eq!(
//! color.into_raw(),
//! (0x18 << Rgb::BLUE_SHIFT) + (0x1f << Rgb::GREEN_SHIFT) + 0x10,
//! );
//!
//! // Convert to/from the backing storage type.
//! let raw: u16 = color.into();
//! assert_eq!(Rgb::from(raw), color);
//! ```
//!
//! # Syntax
//!
//! ```text
//! bitfield! {
//! #[attributes]
//! // Documentation for `Name`.
//! pub struct Name(storage_type) {
//! // `field_1` documentation.
//! hi:lo field_1;
//! // `field_2` documentation.
//! hi:lo field_2 => ConvertedType;
//! // `field_3` documentation.
//! hi:lo field_3 ?=> ConvertedType;
//! ...
//! }
//! }
//! ```
//!
//! - `storage_type`: The underlying unsigned integer type ([`u8`], [`u16`], [`u32`], [`u64`]).
//! Signed integer storage types are not supported.
//! - `hi:lo`: Bit range (inclusive), where `hi >= lo`.
//! - `=> Type`: Optional infallible conversion (see [below](#infallible-conversion-)).
//! - `?=> Type`: Optional fallible conversion (see [below](#fallible-conversion-)).
//! - Documentation strings and attributes are optional.
//!
//! # Generated code
//!
//! Each field is internally represented as a [`Bounded`] parameterized by its bit width. Field
//! values can either be set/retrieved directly, or converted from/to another type.
//!
//! The use of [`Bounded`] for each field enforces bounds-checking (at build time or runtime) of
//! every value assigned to a field. This ensures that data is never accidentally truncated.
//!
//! The macro generates the bitfield type, [`From`] and [`Into`] implementations for its storage
//! type, as well as [`Debug`] and [`Zeroable`](pin_init::Zeroable) implementations.
//!
//! For each field, it also generates:
//!
//! - `field()`: Getter method for the field value.
//! - `with_field(value)`: Infallible setter; the argument type must fit within the field's width.
//! - `with_const_field::<VALUE>()`: `const` setter; the value is validated at compile time.
//! Usually shorter to use than `with_field` for constant values as it doesn't require
//! constructing a [`Bounded`].
//! - `try_with_field(value)`: Fallible setter. Returns an error if the value is out of range.
//! - `FIELD_MASK`, `FIELD_SHIFT`, `FIELD_RANGE`: Constants for manual bit manipulation.
//!
//! # Reserved names for field identifiers
//!
//! Field identifiers are used to generate methods and associated constants on the bitfield type.
//! For a field named `field`, the macro may generate methods named `field`, `with_field`,
//! `with_const_field`, `try_with_field`, `__field` and `__with_field`, as well as constants named
//! `FIELD_MASK`, `FIELD_SHIFT` and `FIELD_RANGE`.
//!
//! Therefore, field identifiers must not use names that would collide with generated items for
//! any field in the same bitfield. The following prefixes are thus reserved for field identifiers:
//!
//! - `with_`
//! - `const_`
//! - `try_with_`
//! - `__`
//!
//! The field identifiers `from_raw`, `into_raw`, and `into` are also reserved.
//!
//! In addition, field identifiers should follow Rust `snake_case` conventions, since the associated
//! constants are generated by uppercasing the field name.
//!
//! # Implicit conversions
//!
//! Types that fit entirely within a field's bit width can be used directly with setters. For
//! example, [`bool`] works with single-bit fields, and [`u8`] works with 8-bit fields:
//!
//! ```rust
//! use kernel::bitfield;
//!
//! bitfield! {
//! pub struct Flags(u32) {
//! 15:8 byte_field;
//! 0:0 flag;
//! }
//! }
//!
//! let flags = Flags::zeroed()
//! .with_byte_field(0x42_u8)
//! .with_flag(true);
//!
//! assert_eq!(flags.into_raw(), (0x42 << Flags::BYTE_FIELD_SHIFT) | 1);
//! ```
//!
//! # Runtime bounds checking
//!
//! When a value is not known at compile time, use `try_with_field()` to check bounds at runtime:
//!
//! ```rust
//! use kernel::bitfield;
//!
//! bitfield! {
//! pub struct Config(u8) {
//! 3:0 nibble;
//! }
//! }
//!
//! fn set_nibble(config: Config, value: u8) -> Result<Config, Error> {
//! // Returns `EOVERFLOW` if `value > 0xf`.
//! config.try_with_nibble(value)
//! }
//! # Ok::<(), Error>(())
//! ```
//!
//! # Type conversion
//!
//! Fields can be automatically converted to/from a custom type using `=>` (infallible) or `?=>`
//! (fallible). The custom type must implement the appropriate [`From`] or [`TryFrom`] traits with
//! [`Bounded`].
//!
//! ## Infallible conversion (`=>`)
//!
//! Use this when all possible bit patterns of a field map to valid values:
//!
//! ```rust
//! use kernel::bitfield;
//! use kernel::num::Bounded;
//!
//! #[derive(Debug, Clone, Copy, PartialEq)]
//! enum Power {
//! Off,
//! On,
//! }
//!
//! impl From<Bounded<u32, 1>> for Power {
//! fn from(v: Bounded<u32, 1>) -> Self {
//! match *v {
//! 0 => Power::Off,
//! _ => Power::On,
//! }
//! }
//! }
//!
//! impl From<Power> for Bounded<u32, 1> {
//! fn from(p: Power) -> Self {
//! (p as u32 != 0).into()
//! }
//! }
//!
//! bitfield! {
//! pub struct Control(u32) {
//! 0:0 power => Power;
//! }
//! }
//!
//! let ctrl = Control::zeroed().with_power(Power::On);
//! assert_eq!(ctrl.power(), Power::On);
//! ```
//!
//! ## Fallible conversion (`?=>`)
//!
//! Use this when some bit patterns of a field are invalid. The getter returns a [`Result`]:
//!
//! ```rust
//! use kernel::bitfield;
//! use kernel::num::Bounded;
//!
//! #[derive(Debug, Clone, Copy, PartialEq)]
//! enum Mode {
//! Low = 0,
//! High = 1,
//! Auto = 2,
//! // 3 is invalid
//! }
//!
//! impl TryFrom<Bounded<u32, 2>> for Mode {
//! type Error = u32;
//!
//! fn try_from(v: Bounded<u32, 2>) -> Result<Self, u32> {
//! match *v {
//! 0 => Ok(Mode::Low),
//! 1 => Ok(Mode::High),
//! 2 => Ok(Mode::Auto),
//! n => Err(n),
//! }
//! }
//! }
//!
//! impl From<Mode> for Bounded<u32, 2> {
//! fn from(m: Mode) -> Self {
//! match m {
//! Mode::Low => Bounded::<u32, _>::new::<0>(),
//! Mode::High => Bounded::<u32, _>::new::<1>(),
//! Mode::Auto => Bounded::<u32, _>::new::<2>(),
//! }
//! }
//! }
//!
//! bitfield! {
//! pub struct Config(u32) {
//! 1:0 mode ?=> Mode;
//! }
//! }
//!
//! let cfg = Config::zeroed().with_mode(Mode::Auto);
//! assert_eq!(cfg.mode(), Ok(Mode::Auto));
//!
//! // Invalid bit pattern returns an error.
//! assert_eq!(Config::from(0b11).mode(), Err(3));
//! ```
//!
//! # Bits outside of declared fields
//!
//! Bits of the storage type that are not part of any declared field are preserved by the setter
//! methods, and can only be modified through `from_raw` or the [`From`] implementation from the
//! storage type.
//!
//! ```rust
//! use kernel::bitfield;
//!
//! bitfield! {
//! pub struct Sparse(u8) {
//! 7:6 high;
//! // Bits 5:1 are not covered by any field.
//! 0:0 low;
//! }
//! }
//!
//! // Set the gap bits via `from_raw`, then mutate the declared fields.
//! let val = Sparse::from_raw(0b0010_1010)
//! .with_const_high::<0b11>()
//! .with_low(true);
//!
//! // Bits 5:1 are unchanged.
//! assert_eq!(val.into_raw(), 0b1110_1011);
//! ```
//!
//! # Signed field values
//!
//! Bitfield storage types are unsigned. Since field getter methods return a [`Bounded`] of the
//! storage type, fields are also unsigned by default.
//!
//! If a field needs to encode a signed value, use a custom conversion type with `=>` or `?=>` to
//! perform the sign interpretation explicitly.
//!
//! [`Bounded`]: kernel::num::Bounded
/// Defines a bitfield struct with bounds-checked accessors for individual bit ranges.
///
/// See the [`mod@kernel::bitfield`] module for full documentation and examples.
#[macro_export]
macro_rules! bitfield {
// Entry point defining the bitfield struct, its implementations and its field accessors.
(
$(#[$attr:meta])* $vis:vis struct $name:ident($storage:ty) { $($fields:tt)* }
) => {
$crate::bitfield!(@core
#[allow(non_camel_case_types)]
$(#[$attr])* $vis $name $storage
);
$crate::bitfield!(@fields $vis $name $storage { $($fields)* });
};
// All rules below are helpers.
// Defines the wrapper `$name` type and its conversions from/to the storage type.
(@core $(#[$attr:meta])* $vis:vis $name:ident $storage:ty) => {
$(#[$attr])*
#[repr(transparent)]
#[derive(Clone, Copy, PartialEq, Eq)]
$vis struct $name {
inner: $storage,
}
#[allow(dead_code)]
impl $name {
/// Creates a bitfield from a raw value.
#[inline(always)]
$vis const fn from_raw(value: $storage) -> Self {
Self{ inner: value }
}
/// Turns this bitfield into its raw value.
///
/// This is similar to the [`From`] implementation, but is shorter to invoke in
/// most cases.
#[inline(always)]
$vis const fn into_raw(self) -> $storage {
self.inner
}
}
// SAFETY: `$storage` is `Zeroable` and `$name` is transparent.
unsafe impl ::pin_init::Zeroable for $name {}
impl ::core::convert::From<$name> for $storage {
#[inline(always)]
fn from(val: $name) -> $storage {
val.into_raw()
}
}
impl ::core::convert::From<$storage> for $name {
#[inline(always)]
fn from(val: $storage) -> $name {
Self::from_raw(val)
}
}
};
// Definitions requiring knowledge of individual fields: private and public field accessors,
// and `Debug` implementation.
(@fields $vis:vis $name:ident $storage:ty {
$($(#[doc = $doc:expr])* $hi:literal:$lo:literal $field:ident
$(?=> $try_into_type:ty)?
$(=> $into_type:ty)?
;
)*
}
) => {
#[allow(dead_code)]
impl $name {
$(
$crate::bitfield!(@private_field_accessors $vis $name $storage : $hi:$lo $field);
$crate::bitfield!(
@public_field_accessors $(#[doc = $doc])* $vis $name $storage : $hi:$lo $field
$(?=> $try_into_type)?
$(=> $into_type)?
);
)*
}
$crate::bitfield!(@debug $name { $($field;)* });
};
// Private field accessors working with the exact `Bounded` type for the field.
(
@private_field_accessors $vis:vis $name:ident $storage:ty : $hi:tt:$lo:tt $field:ident
) => {
::kernel::macros::paste!(
$vis const [<$field:upper _RANGE>]: ::core::ops::RangeInclusive<u8> = $lo..=$hi;
$vis const [<$field:upper _MASK>]: $storage =
((((1 << $hi) - 1) << 1) + 1) - ((1 << $lo) - 1);
$vis const [<$field:upper _SHIFT>]: u32 = $lo;
);
::kernel::macros::paste!(
#[inline(always)]
fn [<__ $field>](self) ->
::kernel::num::Bounded<$storage, { $hi + 1 - $lo }> {
// Left shift to align the field's MSB with the storage MSB.
const ALIGN_TOP: u32 = $storage::BITS - ($hi + 1);
// Right shift to move the top-aligned field to bit 0 of the storage.
const ALIGN_BOTTOM: u32 = ALIGN_TOP + $lo;
// Extract the field using two shifts. `Bounded::shr` produces the correctly-sized
// output type.
let val = ::kernel::num::Bounded::<$storage, { $storage::BITS }>::from(
self.inner << ALIGN_TOP
);
val.shr::<ALIGN_BOTTOM, { $hi + 1 - $lo } >()
}
#[inline(always)]
const fn [<__with_ $field>](
mut self,
value: ::kernel::num::Bounded<$storage, { $hi + 1 - $lo }>,
) -> Self
{
const MASK: $storage = <$name>::[<$field:upper _MASK>];
const SHIFT: u32 = <$name>::[<$field:upper _SHIFT>];
let value = value.get() << SHIFT;
self.inner = (self.inner & !MASK) | value;
self
}
);
};
// Public accessors for fields infallibly (`=>`) converted to a type.
(
@public_field_accessors $(#[doc = $doc:expr])* $vis:vis $name:ident $storage:ty :
$hi:literal:$lo:literal $field:ident => $into_type:ty
) => {
::kernel::macros::paste!(
$(#[doc = $doc])*
#[doc = "Returns the value of this field."]
#[inline(always)]
$vis fn $field(self) -> $into_type
{
self.[<__ $field>]().into()
}
$(#[doc = $doc])*
#[doc = "Sets this field to the given `value`."]
#[inline(always)]
$vis fn [<with_ $field>](self, value: $into_type) -> Self
{
self.[<__with_ $field>](value.into())
}
);
};
// Public accessors for fields fallibly (`?=>`) converted to a type.
(
@public_field_accessors $(#[doc = $doc:expr])* $vis:vis $name:ident $storage:ty :
$hi:tt:$lo:tt $field:ident ?=> $try_into_type:ty
) => {
::kernel::macros::paste!(
$(#[doc = $doc])*
#[doc = "Returns the value of this field."]
#[inline(always)]
$vis fn $field(self) ->
::core::result::Result<
$try_into_type,
<$try_into_type as ::core::convert::TryFrom<
::kernel::num::Bounded<$storage, { $hi + 1 - $lo }>
>>::Error
>
{
self.[<__ $field>]().try_into()
}
$(#[doc = $doc])*
#[doc = "Sets this field to the given `value`."]
#[inline(always)]
$vis fn [<with_ $field>](self, value: $try_into_type) -> Self
{
self.[<__with_ $field>](value.into())
}
);
};
// Public accessors for fields not converted to a type.
(
@public_field_accessors $(#[doc = $doc:expr])* $vis:vis $name:ident $storage:ty :
$hi:tt:$lo:tt $field:ident
) => {
::kernel::macros::paste!(
$(#[doc = $doc])*
#[doc = "Returns the value of this field."]
#[inline(always)]
$vis fn $field(self) ->
::kernel::num::Bounded<$storage, { $hi + 1 - $lo }>
{
self.[<__ $field>]()
}
$(#[doc = $doc])*
#[doc = "Sets this field to the compile-time constant `VALUE`."]
#[inline(always)]
$vis const fn [<with_const_ $field>]<const VALUE: $storage>(self) -> Self {
self.[<__with_ $field>](
::kernel::num::Bounded::<$storage, { $hi + 1 - $lo }>::new::<VALUE>()
)
}
$(#[doc = $doc])*
#[doc = "Sets this field to the given `value`."]
#[inline(always)]
$vis fn [<with_ $field>]<T>(
self,
value: T,
) -> Self
where T: ::core::convert::Into<::kernel::num::Bounded<$storage, { $hi + 1 - $lo }>>,
{
self.[<__with_ $field>](value.into())
}
$(#[doc = $doc])*
#[doc = "Tries to set this field to `value`, returning an error if it is out of range."]
#[inline(always)]
$vis fn [<try_with_ $field>]<T>(
self,
value: T,
) -> ::kernel::error::Result<Self>
where T: ::kernel::num::TryIntoBounded<$storage, { $hi + 1 - $lo }>,
{
Ok(
self.[<__with_ $field>](
value.try_into_bounded().ok_or(::kernel::error::code::EOVERFLOW)?
)
)
}
);
};
// `Debug` implementation.
(@debug $name:ident { $($field:ident;)* }) => {
impl ::kernel::fmt::Debug for $name {
fn fmt(&self, f: &mut ::kernel::fmt::Formatter<'_>) -> ::kernel::fmt::Result {
f.debug_struct(stringify!($name))
.field("<raw>", &::kernel::prelude::fmt!("{:#x}", self.inner))
$(
.field(stringify!($field), &self.$field())
)*
.finish()
}
}
};
}
#[cfg(CONFIG_RUST_BITFIELD_KUNIT_TEST)]
#[::kernel::macros::kunit_tests(rust_kernel_bitfield)]
mod tests {
use core::convert::TryFrom;
use pin_init::Zeroable;
use kernel::num::Bounded;
// Enum types for testing `=>` and `?=>` conversions.
#[derive(Debug, Clone, Copy, PartialEq)]
enum MemoryType {
Unmapped = 0,
Normal = 1,
Device = 2,
Reserved = 3,
}
impl TryFrom<Bounded<u64, 4>> for MemoryType {
type Error = u64;
fn try_from(value: Bounded<u64, 4>) -> Result<Self, Self::Error> {
match value.get() {
0 => Ok(MemoryType::Unmapped),
1 => Ok(MemoryType::Normal),
2 => Ok(MemoryType::Device),
3 => Ok(MemoryType::Reserved),
_ => Err(value.get()),
}
}
}
impl From<MemoryType> for Bounded<u64, 4> {
fn from(mt: MemoryType) -> Bounded<u64, 4> {
Bounded::from_expr(mt as u64)
}
}
#[derive(Debug, Clone, Copy, PartialEq)]
enum Priority {
Low = 0,
Medium = 1,
High = 2,
Critical = 3,
}
impl From<Bounded<u16, 2>> for Priority {
fn from(value: Bounded<u16, 2>) -> Self {
match value & 0x3 {
0 => Priority::Low,
1 => Priority::Medium,
2 => Priority::High,
_ => Priority::Critical,
}
}
}
impl From<Priority> for Bounded<u16, 2> {
fn from(p: Priority) -> Bounded<u16, 2> {
Bounded::from_expr(p as u16)
}
}
bitfield! {
struct TestU64(u64) {
63:63 field_63;
61:52 field_61_52;
51:16 field_51_16;
15:12 field_15_12 ?=> MemoryType;
11:9 field_11_9;
1:1 field_1;
0:0 field_0;
}
}
bitfield! {
struct TestU16(u16) {
15:8 field_15_8;
7:4 field_7_4; // Partial overlap with `field_5_4`.
5:4 field_5_4 => Priority;
3:1 field_3_1;
0:0 field_0;
}
}
bitfield! {
struct TestU8(u8) {
7:0 field_7_0; // Full byte overlap.
7:4 field_7_4;
3:2 field_3_2;
1:1 field_1;
0:0 field_0;
}
}
// Single and multi-bit fields basic access.
#[test]
fn test_basic_access() {
// `TestU64`.
let mut val = TestU64::zeroed();
assert_eq!(val.into_raw(), 0x0);
val = val.with_field_0(true);
assert!(val.field_0().into_bool());
assert_eq!(val.into_raw(), 0x1);
val = val.with_field_1(true);
assert!(val.field_1().into_bool());
val = val.with_field_1(false);
assert!(!val.field_1().into_bool());
assert_eq!(val.into_raw(), 0x1);
val = val.with_const_field_11_9::<0x5>();
assert_eq!(val.field_11_9(), 0x5);
assert_eq!(val.into_raw(), 0xA01);
val = val.with_const_field_51_16::<0x123456>();
assert_eq!(val.field_51_16(), 0x123456);
assert_eq!(val.into_raw(), 0x0012_3456_0A01);
const MAX_FIELD_51_16: u64 = ::kernel::bits::genmask_u64(0..=35);
val = val.with_const_field_51_16::<{ MAX_FIELD_51_16 }>();
assert_eq!(val.field_51_16(), MAX_FIELD_51_16);
val = val.with_const_field_61_52::<0x3FF>();
assert_eq!(val.field_61_52(), 0x3FF);
val = val.with_field_63(true);
assert!(val.field_63().into_bool());
// `TestU16`.
let mut val = TestU16::zeroed();
assert_eq!(val.into_raw(), 0x0);
val = val.with_field_0(true);
assert!(val.field_0().into_bool());
assert_eq!(val.into_raw(), 0x1);
val = val.with_const_field_3_1::<0x5>();
assert_eq!(val.field_3_1(), 0x5);
assert_eq!(val.into_raw(), 0xB);
val = val.with_const_field_7_4::<0xA>();
assert_eq!(val.field_7_4(), 0xA);
assert_eq!(val.into_raw(), 0xAB);
val = val.with_const_field_15_8::<0x42>();
assert_eq!(val.field_15_8(), 0x42);
assert_eq!(val.into_raw(), 0x42AB);
// `TestU8`.
let mut val = TestU8::zeroed();
assert_eq!(val.into_raw(), 0x0);
val = val.with_field_0(true);
assert!(val.field_0().into_bool());
assert_eq!(val.into_raw(), 0x1);
val = val.with_field_1(true);
assert!(val.field_1().into_bool());
assert_eq!(val.into_raw(), 0x3);
val = val.with_const_field_3_2::<0x3>();
assert_eq!(val.field_3_2(), 0x3);
assert_eq!(val.into_raw(), 0xF);
val = val.with_const_field_7_4::<0xA>();
assert_eq!(val.field_7_4(), 0xA);
assert_eq!(val.into_raw(), 0xAF);
}
// `=>` infallible conversion.
#[test]
fn test_infallible_conversion() {
let mut val = TestU16::zeroed();
val = val.with_field_5_4(Priority::Low);
assert_eq!(val.field_5_4(), Priority::Low);
assert_eq!(val.into_raw() & 0x30, 0x00);
val = val.with_field_5_4(Priority::Medium);
assert_eq!(val.field_5_4(), Priority::Medium);
assert_eq!(val.into_raw() & 0x30, 0x10);
val = val.with_field_5_4(Priority::High);
assert_eq!(val.field_5_4(), Priority::High);
assert_eq!(val.into_raw() & 0x30, 0x20);
val = val.with_field_5_4(Priority::Critical);
assert_eq!(val.field_5_4(), Priority::Critical);
assert_eq!(val.into_raw() & 0x30, 0x30);
}
// `?=>` fallible conversion.
#[test]
fn test_fallible_conversion() {
let mut val = TestU64::zeroed();
val = val.with_field_15_12(MemoryType::Unmapped);
assert_eq!(val.field_15_12(), Ok(MemoryType::Unmapped));
val = val.with_field_15_12(MemoryType::Normal);
assert_eq!(val.field_15_12(), Ok(MemoryType::Normal));
val = val.with_field_15_12(MemoryType::Device);
assert_eq!(val.field_15_12(), Ok(MemoryType::Device));
val = val.with_field_15_12(MemoryType::Reserved);
assert_eq!(val.field_15_12(), Ok(MemoryType::Reserved));
// `field_15_12` is 4 bits wide (0-15); `MemoryType` only covers 0-3, so 4-15 return `Err`.
let raw = (val.into_raw() & !::kernel::bits::genmask_u64(12..=15)) | (0x7 << 12);
assert_eq!(TestU64::from_raw(raw).field_15_12(), Err(0x7));
}
// Test that setting an overlapping field affects the overlapped one as expected.
#[test]
fn test_overlapping_fields() {
let mut val = TestU16::zeroed();
val = val.with_field_5_4(Priority::High); // High == 2 == 0b10.
assert_eq!(val.field_5_4(), Priority::High);
assert_eq!(val.field_7_4(), 0x2); // Bits 7:6 == 0, bits 5:4 == 0b10.
val = val.with_const_field_7_4::<0xF>();
assert_eq!(val.field_7_4(), 0xF);
assert_eq!(val.field_5_4(), Priority::Critical); // Bits 5:4 == 0b11.
// `field_7_0` should encompass all other fields.
let mut val = TestU8::zeroed()
.with_field_0(true)
.with_field_1(true)
.with_const_field_3_2::<0x3>()
.with_const_field_7_4::<0xA>();
assert_eq!(val.into_raw(), 0xAF);
val = val.with_field_7_0(0x55);
assert_eq!(val.field_7_0(), 0x55);
assert!(val.field_0().into_bool());
assert!(!val.field_1().into_bool());
assert_eq!(val.field_3_2(), 0x1);
assert_eq!(val.field_7_4(), 0x5);
}
// Checks that bits not mapped to any field are left untouched.
#[test]
fn test_unallocated_bits() {
let gap_bits = (1u64 << 62) | 0x1FC;
let set_all_fields = |val: TestU64| {
val.with_field_63(true)
.with_const_field_61_52::<0x155>()
.with_const_field_51_16::<0x123456>()
.with_field_15_12(MemoryType::Device)
.with_const_field_11_9::<0x5>()
.with_field_1(true)
.with_field_0(true)
};
// Gap bits to 0.
let val = set_all_fields(TestU64::from_raw(0));
assert_eq!(val.into_raw() & gap_bits, 0);
// Gap bits to 1.
let val = set_all_fields(TestU64::from_raw(gap_bits));
assert_eq!(val.into_raw() & gap_bits, gap_bits);
}
#[test]
fn test_try_with() {
let val = TestU64::zeroed().try_with_field_51_16(0x123456).unwrap();
assert_eq!(val.field_51_16(), 0x123456);
let err = TestU64::zeroed().try_with_field_51_16(u64::MAX);
assert_eq!(err, Err(::kernel::error::code::EOVERFLOW));
let val = TestU64::zeroed()
.try_with_field_51_16(0xABCDEF)
.and_then(|p| p.try_with_field_0(1))
.unwrap();
assert_eq!(val.field_51_16(), 0xABCDEF);
assert!(val.field_0().into_bool());
}
// `from_raw`/`into_raw` and `From`/`Into` round-trips.
#[test]
fn test_raw() {
let raw: u64 = 0xBFF0_0000_3123_3E03;
let val = TestU64::from_raw(raw);
assert_eq!(u64::from(val), raw);
assert!(val.field_0().into_bool());
assert!(val.field_1().into_bool());
assert_eq!(val.field_11_9(), 0x7);
assert_eq!(val.field_51_16(), 0x3123);
assert_eq!(val.field_15_12(), Ok(MemoryType::Reserved));
assert_eq!(val.field_61_52(), 0x3FF);
assert!(val.field_63().into_bool());
let raw: u16 = 0x42AB;
let val = TestU16::from_raw(raw);
assert_eq!(u16::from(val), raw);
assert!(val.field_0().into_bool());
assert_eq!(val.field_3_1(), 0x5);
assert_eq!(val.field_7_4(), 0xA);
assert_eq!(val.field_15_8(), 0x42);
let raw: u8 = 0xAF;
let val = TestU8::from_raw(raw);
assert_eq!(u8::from(val), raw);
assert!(val.field_0().into_bool());
assert!(val.field_1().into_bool());
assert_eq!(val.field_3_2(), 0x3);
assert_eq!(val.field_7_4(), 0xA);
assert_eq!(val.field_7_0(), 0xAF);
}
}

5
rust/kernel/bitmap.rs
View File

@ -499,9 +499,8 @@ pub fn next_zero_bit(&self, start: usize) -> Option<usize> {
}
}
use macros::kunit_tests;
#[kunit_tests(rust_kernel_bitmap)]
#[cfg(CONFIG_RUST_BITMAP_KUNIT_TEST)]
#[macros::kunit_tests(rust_kernel_bitmap)]
mod tests {
use super::*;
use kernel::alloc::flags::GFP_KERNEL;

19
rust/kernel/build_assert.rs
View File

@ -61,15 +61,16 @@
//! undefined symbols and linker errors, it is not developer friendly to debug, so it is recommended
//! to avoid it and prefer other two assertions where possible.
#[doc(inline)]
pub use crate::{
build_assert,
build_assert_macro as build_assert,
build_error,
const_assert,
static_assert, //
};
#[doc(hidden)]
pub use build_error::build_error;
pub use build_error::build_error as build_error_fn;
/// Static assert (i.e. compile-time assert).
///
@ -105,6 +106,7 @@
/// static_assert!(f(40) == 42, "f(x) must add 2 to the given input.");
/// ```
#[macro_export]
#[doc(hidden)]
macro_rules! static_assert {
($condition:expr $(,$arg:literal)?) => {
const _: () = ::core::assert!($condition $(,$arg)?);
@ -133,6 +135,7 @@ macro_rules! static_assert {
/// }
/// ```
#[macro_export]
#[doc(hidden)]
macro_rules! const_assert {
($condition:expr $(,$arg:literal)?) => {
const { ::core::assert!($condition $(,$arg)?) };
@ -157,12 +160,13 @@ macro_rules! const_assert {
/// // foo(usize::MAX); // Fails to compile.
/// ```
#[macro_export]
#[doc(hidden)]
macro_rules! build_error {
() => {{
$crate::build_assert::build_error("")
$crate::build_assert::build_error_fn("")
}};
($msg:expr) => {{
$crate::build_assert::build_error($msg)
$crate::build_assert::build_error_fn($msg)
}};
}
@ -200,15 +204,16 @@ macro_rules! build_error {
/// const _: () = const_bar(2);
/// ```
#[macro_export]
macro_rules! build_assert {
#[doc(hidden)]
macro_rules! build_assert_macro {
($cond:expr $(,)?) => {{
if !$cond {
$crate::build_assert::build_error(concat!("assertion failed: ", stringify!($cond)));
$crate::build_assert::build_error_fn(concat!("assertion failed: ", stringify!($cond)));
}
}};
($cond:expr, $msg:expr) => {{
if !$cond {
$crate::build_assert::build_error($msg);
$crate::build_assert::build_error_fn($msg);
}
}};
}

2
rust/kernel/cpufreq.rs
View File

@ -1323,7 +1323,7 @@ impl<T: Driver> Registration<T> {
// SAFETY: The C API guarantees that `cpu` refers to a valid CPU number.
let cpu_id = unsafe { CpuId::from_u32_unchecked(cpu) };
PolicyCpu::from_cpu(cpu_id).map_or(0, |mut policy| T::get(&mut policy).map_or(0, |f| f))
PolicyCpu::from_cpu(cpu_id).map_or(0, |mut policy| T::get(&mut policy).unwrap_or(0))
}
/// Driver's `update_limit` callback.

15
rust/kernel/dma.rs
View File

@ -1152,8 +1152,8 @@ unsafe impl Sync for CoherentHandle {}
/// unsafe impl kernel::transmute::AsBytes for MyStruct{};
///
/// # fn test(alloc: &kernel::dma::Coherent<[MyStruct]>) -> Result {
/// let whole = kernel::dma_read!(alloc, [2]?);
/// let field = kernel::dma_read!(alloc, [1]?.field);
/// let whole = kernel::dma_read!(alloc, [try: 2]);
/// let field = kernel::dma_read!(alloc, [panic: 1].field);
/// # Ok::<(), Error>(()) }
/// ```
#[macro_export]
@ -1189,8 +1189,8 @@ macro_rules! dma_read {
/// unsafe impl kernel::transmute::AsBytes for MyStruct{};
///
/// # fn test(alloc: &kernel::dma::Coherent<[MyStruct]>) -> Result {
/// kernel::dma_write!(alloc, [2]?.member, 0xf);
/// kernel::dma_write!(alloc, [1]?, MyStruct { member: 0xf });
/// kernel::dma_write!(alloc, [try: 2].member, 0xf);
/// kernel::dma_write!(alloc, [panic: 1], MyStruct { member: 0xf });
/// # Ok::<(), Error>(()) }
/// ```
#[macro_export]
@ -1207,11 +1207,8 @@ macro_rules! dma_write {
(@parse [$dma:expr] [$($proj:tt)*] [.$field:tt $($rest:tt)*]) => {
$crate::dma_write!(@parse [$dma] [$($proj)* .$field] [$($rest)*])
};
(@parse [$dma:expr] [$($proj:tt)*] [[$index:expr]? $($rest:tt)*]) => {
$crate::dma_write!(@parse [$dma] [$($proj)* [$index]?] [$($rest)*])
};
(@parse [$dma:expr] [$($proj:tt)*] [[$index:expr] $($rest:tt)*]) => {
$crate::dma_write!(@parse [$dma] [$($proj)* [$index]] [$($rest)*])
(@parse [$dma:expr] [$($proj:tt)*] [[$flavor:ident: $index:expr] $($rest:tt)*]) => {
$crate::dma_write!(@parse [$dma] [$($proj)* [$flavor: $index]] [$($rest)*])
};
($dma:expr, $($rest:tt)*) => {
$crate::dma_write!(@parse [$dma] [] [$($rest)*])

6
rust/kernel/error.rs
View File

@ -25,10 +25,8 @@ macro_rules! declare_err {
#[doc = $doc]
)*
pub const $err: super::Error =
match super::Error::try_from_errno(-(crate::bindings::$err as i32)) {
Some(err) => err,
None => panic!("Invalid errno in `declare_err!`"),
};
super::Error::try_from_errno(-(crate::bindings::$err as i32))
.expect("Invalid errno in `declare_err!`");
};
}

19
rust/kernel/fmt.rs
View File

@ -4,7 +4,14 @@
//!
//! This module is intended to be used in place of `core::fmt` in kernel code.
pub use core::fmt::{Arguments, Debug, Error, Formatter, Result, Write};
pub use core::fmt::{
Arguments,
Debug,
Error,
Formatter,
Result,
Write, //
};
/// Internal adapter used to route and allow implementations of formatting traits for foreign types.
///
@ -27,7 +34,15 @@ fn fmt(&self, f: &mut Formatter<'_>) -> Result {
};
}
use core::fmt::{Binary, LowerExp, LowerHex, Octal, Pointer, UpperExp, UpperHex};
use core::fmt::{
Binary,
LowerExp,
LowerHex,
Octal,
Pointer,
UpperExp,
UpperHex, //
};
impl_fmt_adapter_forward!(Debug, LowerHex, UpperHex, Octal, Binary, Pointer, LowerExp, UpperExp);
/// A copy of [`core::fmt::Display`] that allows us to implement it for foreign types.

2
rust/kernel/init.rs
View File

@ -151,6 +151,7 @@ fn try_pin_init<E>(init: impl PinInit<T, E>, flags: Flags) -> Result<Self::Pinne
/// type.
///
/// If `T: !Unpin` it will not be able to move afterwards.
#[inline]
fn pin_init<E>(init: impl PinInit<T, E>, flags: Flags) -> error::Result<Self::PinnedSelf>
where
Error: From<E>,
@ -168,6 +169,7 @@ fn try_init<E>(init: impl Init<T, E>, flags: Flags) -> Result<Self, E>
E: From<AllocError>;
/// Use the given initializer to in-place initialize a `T`.
#[inline]
fn init<E>(init: impl Init<T, E>, flags: Flags) -> error::Result<Self>
where
Error: From<E>,

265
rust/kernel/io/register.rs
View File

@ -108,9 +108,10 @@
use core::marker::PhantomData;
use crate::io::IoLoc;
use kernel::build_assert;
use crate::{
build_assert::build_assert,
io::IoLoc, //
};
/// Trait implemented by all registers.
pub trait Register: Sized {
@ -872,7 +873,7 @@ macro_rules! register {
@reg $(#[$attr:meta])* $vis:vis $name:ident ($storage:ty)
[ $size:expr, stride = $stride:expr ] @ $offset:literal { $($fields:tt)* }
) => {
::kernel::static_assert!(::core::mem::size_of::<$storage>() <= $stride);
$crate::build_assert::static_assert!(::core::mem::size_of::<$storage>() <= $stride);
$crate::register!(@bitfield $(#[$attr])* $vis struct $name($storage) { $($fields)* });
$crate::register!(@io_base $name($storage) @ $offset);
@ -895,7 +896,9 @@ macro_rules! register {
@reg $(#[$attr:meta])* $vis:vis $name:ident ($storage:ty) => $alias:ident [ $idx:expr ]
{ $($fields:tt)* }
) => {
::kernel::static_assert!($idx < <$alias as $crate::io::register::RegisterArray>::SIZE);
$crate::build_assert::static_assert!(
$idx < <$alias as $crate::io::register::RegisterArray>::SIZE
);
$crate::register!(@bitfield $(#[$attr])* $vis struct $name($storage) { $($fields)* });
$crate::register!(
@ -912,7 +915,7 @@ macro_rules! register {
[ $size:expr, stride = $stride:expr ]
@ $base:ident + $offset:literal { $($fields:tt)* }
) => {
::kernel::static_assert!(::core::mem::size_of::<$storage>() <= $stride);
$crate::build_assert::static_assert!(::core::mem::size_of::<$storage>() <= $stride);
$crate::register!(@bitfield $(#[$attr])* $vis struct $name($storage) { $($fields)* });
$crate::register!(@io_base $name($storage) @ $offset);
@ -938,7 +941,9 @@ macro_rules! register {
@reg $(#[$attr:meta])* $vis:vis $name:ident ($storage:ty)
=> $base:ident + $alias:ident [ $idx:expr ] { $($fields:tt)* }
) => {
::kernel::static_assert!($idx < <$alias as $crate::io::register::RegisterArray>::SIZE);
$crate::build_assert::static_assert!(
$idx < <$alias as $crate::io::register::RegisterArray>::SIZE
);
$crate::register!(@bitfield $(#[$attr])* $vis struct $name($storage) { $($fields)* });
$crate::register!(
@ -956,11 +961,10 @@ macro_rules! register {
(
@bitfield $(#[$attr:meta])* $vis:vis struct $name:ident($storage:ty) { $($fields:tt)* }
) => {
$crate::register!(@bitfield_core
$crate::bitfield!(
#[allow(non_camel_case_types)]
$(#[$attr])* $vis $name $storage
$(#[$attr])* $vis struct $name($storage) { $($fields)* }
);
$crate::register!(@bitfield_fields $vis $name $storage { $($fields)* });
};
// Implementations shared by all registers types.
@ -1016,245 +1020,4 @@ impl $crate::io::register::RegisterArray for $name {
impl $crate::io::register::RelativeRegisterArray for $name {}
};
// Defines the wrapper `$name` type and its conversions from/to the storage type.
(@bitfield_core $(#[$attr:meta])* $vis:vis $name:ident $storage:ty) => {
$(#[$attr])*
#[repr(transparent)]
#[derive(Clone, Copy, PartialEq, Eq)]
$vis struct $name {
inner: $storage,
}
#[allow(dead_code)]
impl $name {
/// Creates a bitfield from a raw value.
#[inline(always)]
$vis const fn from_raw(value: $storage) -> Self {
Self{ inner: value }
}
/// Turns this bitfield into its raw value.
///
/// This is similar to the [`From`] implementation, but is shorter to invoke in
/// most cases.
#[inline(always)]
$vis const fn into_raw(self) -> $storage {
self.inner
}
}
// SAFETY: `$storage` is `Zeroable` and `$name` is transparent.
unsafe impl ::pin_init::Zeroable for $name {}
impl ::core::convert::From<$name> for $storage {
#[inline(always)]
fn from(val: $name) -> $storage {
val.into_raw()
}
}
impl ::core::convert::From<$storage> for $name {
#[inline(always)]
fn from(val: $storage) -> $name {
Self::from_raw(val)
}
}
};
// Definitions requiring knowledge of individual fields: private and public field accessors,
// and `Debug` implementation.
(@bitfield_fields $vis:vis $name:ident $storage:ty {
$($(#[doc = $doc:expr])* $hi:literal:$lo:literal $field:ident
$(?=> $try_into_type:ty)?
$(=> $into_type:ty)?
;
)*
}
) => {
#[allow(dead_code)]
impl $name {
$(
$crate::register!(@private_field_accessors $vis $name $storage : $hi:$lo $field);
$crate::register!(
@public_field_accessors $(#[doc = $doc])* $vis $name $storage : $hi:$lo $field
$(?=> $try_into_type)?
$(=> $into_type)?
);
)*
}
$crate::register!(@debug $name { $($field;)* });
};
// Private field accessors working with the exact `Bounded` type for the field.
(
@private_field_accessors $vis:vis $name:ident $storage:ty : $hi:tt:$lo:tt $field:ident
) => {
::kernel::macros::paste!(
$vis const [<$field:upper _RANGE>]: ::core::ops::RangeInclusive<u8> = $lo..=$hi;
$vis const [<$field:upper _MASK>]: $storage =
((((1 << $hi) - 1) << 1) + 1) - ((1 << $lo) - 1);
$vis const [<$field:upper _SHIFT>]: u32 = $lo;
);
::kernel::macros::paste!(
fn [<__ $field>](self) ->
::kernel::num::Bounded<$storage, { $hi + 1 - $lo }> {
// Left shift to align the field's MSB with the storage MSB.
const ALIGN_TOP: u32 = $storage::BITS - ($hi + 1);
// Right shift to move the top-aligned field to bit 0 of the storage.
const ALIGN_BOTTOM: u32 = ALIGN_TOP + $lo;
// Extract the field using two shifts. `Bounded::shr` produces the correctly-sized
// output type.
let val = ::kernel::num::Bounded::<$storage, { $storage::BITS }>::from(
self.inner << ALIGN_TOP
);
val.shr::<ALIGN_BOTTOM, { $hi + 1 - $lo } >()
}
const fn [<__with_ $field>](
mut self,
value: ::kernel::num::Bounded<$storage, { $hi + 1 - $lo }>,
) -> Self
{
const MASK: $storage = <$name>::[<$field:upper _MASK>];
const SHIFT: u32 = <$name>::[<$field:upper _SHIFT>];
let value = value.get() << SHIFT;
self.inner = (self.inner & !MASK) | value;
self
}
);
};
// Public accessors for fields infallibly (`=>`) converted to a type.
(
@public_field_accessors $(#[doc = $doc:expr])* $vis:vis $name:ident $storage:ty :
$hi:literal:$lo:literal $field:ident => $into_type:ty
) => {
::kernel::macros::paste!(
$(#[doc = $doc])*
#[doc = "Returns the value of this field."]
#[inline(always)]
$vis fn $field(self) -> $into_type
{
self.[<__ $field>]().into()
}
$(#[doc = $doc])*
#[doc = "Sets this field to the given `value`."]
#[inline(always)]
$vis fn [<with_ $field>](self, value: $into_type) -> Self
{
self.[<__with_ $field>](value.into())
}
);
};
// Public accessors for fields fallibly (`?=>`) converted to a type.
(
@public_field_accessors $(#[doc = $doc:expr])* $vis:vis $name:ident $storage:ty :
$hi:tt:$lo:tt $field:ident ?=> $try_into_type:ty
) => {
::kernel::macros::paste!(
$(#[doc = $doc])*
#[doc = "Returns the value of this field."]
#[inline(always)]
$vis fn $field(self) ->
Result<
$try_into_type,
<$try_into_type as ::core::convert::TryFrom<
::kernel::num::Bounded<$storage, { $hi + 1 - $lo }>
>>::Error
>
{
self.[<__ $field>]().try_into()
}
$(#[doc = $doc])*
#[doc = "Sets this field to the given `value`."]
#[inline(always)]
$vis fn [<with_ $field>](self, value: $try_into_type) -> Self
{
self.[<__with_ $field>](value.into())
}
);
};
// Public accessors for fields not converted to a type.
(
@public_field_accessors $(#[doc = $doc:expr])* $vis:vis $name:ident $storage:ty :
$hi:tt:$lo:tt $field:ident
) => {
::kernel::macros::paste!(
$(#[doc = $doc])*
#[doc = "Returns the value of this field."]
#[inline(always)]
$vis fn $field(self) ->
::kernel::num::Bounded<$storage, { $hi + 1 - $lo }>
{
self.[<__ $field>]()
}
$(#[doc = $doc])*
#[doc = "Sets this field to the compile-time constant `VALUE`."]
#[inline(always)]
$vis const fn [<with_const_ $field>]<const VALUE: $storage>(self) -> Self {
self.[<__with_ $field>](
::kernel::num::Bounded::<$storage, { $hi + 1 - $lo }>::new::<VALUE>()
)
}
$(#[doc = $doc])*
#[doc = "Sets this field to the given `value`."]
#[inline(always)]
$vis fn [<with_ $field>]<T>(
self,
value: T,
) -> Self
where T: Into<::kernel::num::Bounded<$storage, { $hi + 1 - $lo }>>,
{
self.[<__with_ $field>](value.into())
}
$(#[doc = $doc])*
#[doc = "Tries to set this field to `value`, returning an error if it is out of range."]
#[inline(always)]
$vis fn [<try_with_ $field>]<T>(
self,
value: T,
) -> ::kernel::error::Result<Self>
where T: ::kernel::num::TryIntoBounded<$storage, { $hi + 1 - $lo }>,
{
Ok(
self.[<__with_ $field>](
value.try_into_bounded().ok_or(::kernel::error::code::EOVERFLOW)?
)
)
}
);
};
// `Debug` implementation.
(@debug $name:ident { $($field:ident;)* }) => {
impl ::kernel::fmt::Debug for $name {
fn fmt(&self, f: &mut ::kernel::fmt::Formatter<'_>) -> ::kernel::fmt::Result {
f.debug_struct(stringify!($name))
.field("<raw>", &::kernel::prelude::fmt!("{:#x}", self.inner))
$(
.field(stringify!($field), &self.$field())
)*
.finish()
}
}
};
}

2
rust/kernel/io/resource.rs
View File

@ -229,7 +229,7 @@ impl Flags {
// Always inline to optimize out error path of `build_assert`.
#[inline(always)]
const fn new(value: u32) -> Self {
crate::build_assert!(value as u64 <= c_ulong::MAX as u64);
build_assert!(value as u64 <= c_ulong::MAX as u64);
Flags(value as c_ulong)
}
}

2
rust/kernel/ioctl.rs
View File

@ -6,7 +6,7 @@
#![expect(non_snake_case)]
use crate::build_assert;
use crate::build_assert::build_assert;
/// Build an ioctl number, analogous to the C macro of the same name.
#[inline(always)]

1
rust/kernel/kunit.rs
View File

@ -329,6 +329,7 @@ pub fn in_kunit_test() -> bool {
!unsafe { bindings::kunit_get_current_test() }.is_null()
}
#[cfg(CONFIG_RUST_KUNIT_SELFTEST)]
#[kunit_tests(rust_kernel_kunit)]
mod tests {
use super::*;

1
rust/kernel/lib.rs
View File

@ -44,6 +44,7 @@
pub mod alloc;
#[cfg(CONFIG_AUXILIARY_BUS)]
pub mod auxiliary;
pub mod bitfield;
pub mod bitmap;
pub mod bits;
#[cfg(CONFIG_BLOCK)]

8
rust/kernel/net/phy/reg.rs
View File

@ -9,9 +9,11 @@
//! defined in IEEE 802.3.
use super::Device;
use crate::build_assert;
use crate::error::*;
use crate::uapi;
use crate::{
build_assert::build_assert,
error::*,
uapi, //
};
mod private {
/// Marker that a trait cannot be implemented outside of this crate

2
rust/kernel/num/bounded.rs
View File

@ -364,7 +364,7 @@ pub fn try_new(value: T) -> Option<Self> {
// Always inline to optimize out error path of `build_assert`.
#[inline(always)]
pub fn from_expr(expr: T) -> Self {
crate::build_assert!(
crate::build_assert::build_assert!(
fits_within(expr, N),
"Requested value larger than maximal representable value."
);

18
rust/kernel/page.rs
View File

@ -3,17 +3,25 @@
//! Kernel page allocation and management.
use crate::{
alloc::{AllocError, Flags},
alloc::{
AllocError,
Flags, //
},
bindings,
error::code::*,
error::Result,
uaccess::UserSliceReader,
error::{
code::*,
Result, //
},
uaccess::UserSliceReader, //
};
use core::{
marker::PhantomData,
mem::ManuallyDrop,
ops::Deref,
ptr::{self, NonNull},
ptr::{
self,
NonNull, //
}, //
};
/// A bitwise shift for the page size.

19
rust/kernel/prelude.rs
View File

@ -22,6 +22,7 @@
pin::Pin, //
};
#[doc(no_inline)]
pub use ::ffi::{
c_char,
c_int,
@ -47,6 +48,7 @@
vtable, //
};
#[doc(no_inline)]
pub use pin_init::{
init,
pin_data,
@ -58,6 +60,13 @@
Zeroable, //
};
#[doc(no_inline)]
pub use zerocopy::FromBytes;
#[doc(no_inline)]
pub use zerocopy_derive::FromBytes;
#[doc(no_inline)]
pub use super::{
alloc::{
flags::*,
@ -70,9 +79,12 @@
VVec,
Vec, //
},
build_assert,
build_error,
const_assert,
build_assert::{
build_assert,
build_error,
const_assert,
static_assert, //
},
current,
dev_alert,
dev_crit,
@ -96,7 +108,6 @@
pr_info,
pr_notice,
pr_warn,
static_assert,
str::CStrExt as _,
try_init,
try_pin_init,

99
rust/kernel/ptr/projection.rs
View File

@ -26,14 +26,14 @@ fn from(_: OutOfBound) -> Self {
///
/// # Safety
///
/// The implementation of `index` and `get` (if [`Some`] is returned) must ensure that, if provided
/// input pointer `slice` and returned pointer `output`, then:
/// For a given input pointer `slice` and return value `output`, the implementation of `index`,
/// `build_index` and `get` (if [`Some`] is returned) must ensure that:
/// - `output` has the same provenance as `slice`;
/// - `output.byte_offset_from(slice)` is between 0 to
/// `KnownSize::size(slice) - KnownSize::size(output)`.
///
/// This means that if the input pointer is valid, then pointer returned by `get` or `index` is
/// also valid.
/// This means that if the input pointer is valid, then the pointer returned by `get`, `index`
/// or `build_index` is also valid.
#[diagnostic::on_unimplemented(message = "`{Self}` cannot be used to index `{T}`")]
#[doc(hidden)]
pub unsafe trait ProjectIndex<T: ?Sized>: Sized {
@ -42,10 +42,16 @@ pub unsafe trait ProjectIndex<T: ?Sized>: Sized {
/// Returns an index-projected pointer, if in bounds.
fn get(self, slice: *mut T) -> Option<*mut Self::Output>;
/// Returns an index-projected pointer; panic if out of bounds.
fn index(self, slice: *mut T) -> *mut Self::Output;
/// Returns an index-projected pointer; fail the build if it cannot be proved to be in bounds.
#[inline(always)]
fn index(self, slice: *mut T) -> *mut Self::Output {
Self::get(self, slice).unwrap_or_else(|| build_error!())
fn build_index(self, slice: *mut T) -> *mut Self::Output {
match Self::get(self, slice) {
Some(v) => v,
None => build_error!(),
}
}
}
@ -67,6 +73,11 @@ fn index(self, slice: *mut T) -> *mut Self::Output {
fn index(self, slice: *mut [T; N]) -> *mut Self::Output {
<I as ProjectIndex<[T]>>::index(self, slice)
}
#[inline(always)]
fn build_index(self, slice: *mut [T; N]) -> *mut Self::Output {
<I as ProjectIndex<[T]>>::build_index(self, slice)
}
}
// SAFETY: `get`-returned pointer has the same provenance as `slice` and the offset is checked to
@ -82,6 +93,16 @@ fn get(self, slice: *mut [T]) -> Option<*mut T> {
Some(slice.cast::<T>().wrapping_add(self))
}
}
#[inline(always)]
fn index(self, slice: *mut [T]) -> *mut T {
// Leverage Rust built-in operators for bounds checking.
// SAFETY: All non-null and aligned pointers are valid for ZST read.
let zst_slice =
unsafe { core::slice::from_raw_parts::<()>(core::ptr::dangling(), slice.len()) };
let () = zst_slice[self];
slice.cast::<T>().wrapping_add(self)
}
}
// SAFETY: `get`-returned pointer has the same provenance as `slice` and the offset is checked to
@ -100,6 +121,18 @@ fn get(self, slice: *mut [T]) -> Option<*mut [T]> {
new_len,
))
}
#[inline(always)]
fn index(self, slice: *mut [T]) -> *mut [T] {
// Leverage Rust built-in operators for bounds checking.
// SAFETY: All non-null and aligned pointers are valid for ZST read.
let zst_slice =
unsafe { core::slice::from_raw_parts::<()>(core::ptr::dangling(), slice.len()) };
_ = zst_slice[self.clone()];
// SAFETY: Bounds checked.
unsafe { self.get(slice).unwrap_unchecked() }
}
}
// SAFETY: Safety requirement guaranteed by the forwarded impl.
@ -110,6 +143,11 @@ unsafe impl<T> ProjectIndex<[T]> for core::ops::RangeTo<usize> {
fn get(self, slice: *mut [T]) -> Option<*mut [T]> {
(0..self.end).get(slice)
}
#[inline(always)]
fn index(self, slice: *mut [T]) -> *mut [T] {
(0..self.end).index(slice)
}
}
// SAFETY: Safety requirement guaranteed by the forwarded impl.
@ -120,6 +158,11 @@ unsafe impl<T> ProjectIndex<[T]> for core::ops::RangeFrom<usize> {
fn get(self, slice: *mut [T]) -> Option<*mut [T]> {
(self.start..slice.len()).get(slice)
}
#[inline(always)]
fn index(self, slice: *mut [T]) -> *mut [T] {
(self.start..slice.len()).index(slice)
}
}
// SAFETY: `get` returned the pointer as is, so it always has the same provenance and offset of 0.
@ -130,6 +173,11 @@ unsafe impl<T> ProjectIndex<[T]> for core::ops::RangeFull {
fn get(self, slice: *mut [T]) -> Option<*mut [T]> {
Some(slice)
}
#[inline(always)]
fn index(self, slice: *mut [T]) -> *mut [T] {
slice
}
}
/// A helper trait to perform field projection.
@ -207,10 +255,13 @@ unsafe fn proj<F>(_: *mut Self, _: impl FnOnce(*mut Self) -> *mut F) -> *mut F {
/// If a mutable pointer is needed, the macro input can be prefixed with the `mut` keyword, i.e.
/// `kernel::ptr::project!(mut ptr, projection)`. By default, a const pointer is created.
///
/// `ptr::project!` macro can perform both fallible indexing and build-time checked indexing.
/// `[index]` form performs build-time bounds checking; if compiler fails to prove `[index]` is in
/// bounds, compilation will fail. `[index]?` can be used to perform runtime bounds checking;
/// `OutOfBound` error is raised via `?` if the index is out of bounds.
/// The `ptr::project!` macro can perform both fallible indexing and build-time checked indexing.
/// The syntax is of the form `[<flavor>: index]` where `flavor` indicates the way of handling
/// index out-of-bounds errors.
/// - `try` will raise an [`OutOfBound`] error (which is convertible to [`ERANGE`]).
/// - `build` will use the [`build_assert!`] mechanism to have the compiler validate the index is
/// in bounds.
/// - `panic` will cause a Rust [`panic!`] if the index goes out of bounds.
///
/// # Examples
///
@ -228,17 +279,21 @@ unsafe fn proj<F>(_: *mut Self, _: impl FnOnce(*mut Self) -> *mut F) -> *mut F {
/// }
/// ```
///
/// Index projections are performed with `[index]`:
/// Index projections are performed with `[<flavor>: index]`, where `flavor` is `try`, `build` or
/// `panic`:
///
/// ```
/// fn proj(ptr: *const [u8; 32]) -> Result {
/// let field_ptr: *const u8 = kernel::ptr::project!(ptr, [1]);
/// let field_ptr: *const u8 = kernel::ptr::project!(ptr, [build: 1]);
/// // The following invocation, if uncommented, would fail the build.
/// //
/// // kernel::ptr::project!(ptr, [128]);
/// // kernel::ptr::project!(ptr, [build: 128]);
///
/// // This will raise an `OutOfBound` error (which is convertible to `ERANGE`).
/// kernel::ptr::project!(ptr, [128]?);
/// kernel::ptr::project!(ptr, [try: 128]);
///
/// // This will panic at runtime if executed.
/// kernel::ptr::project!(ptr, [panic: 128]);
/// Ok(())
/// }
/// ```
@ -248,7 +303,7 @@ unsafe fn proj<F>(_: *mut Self, _: impl FnOnce(*mut Self) -> *mut F) -> *mut F {
/// ```
/// let ptr: *const [u8; 32] = core::ptr::dangling();
/// let field_ptr: Result<*const u8> = (|| -> Result<_> {
/// Ok(kernel::ptr::project!(ptr, [128]?))
/// Ok(kernel::ptr::project!(ptr, [try: 128]))
/// })();
/// assert!(field_ptr.is_err());
/// ```
@ -257,7 +312,7 @@ unsafe fn proj<F>(_: *mut Self, _: impl FnOnce(*mut Self) -> *mut F) -> *mut F {
///
/// ```
/// let ptr: *mut [(u8, u16); 32] = core::ptr::dangling_mut();
/// let field_ptr: *mut u16 = kernel::ptr::project!(mut ptr, [1].1);
/// let field_ptr: *mut u16 = kernel::ptr::project!(mut ptr, [build: 1].1);
/// ```
#[macro_export]
macro_rules! project_pointer {
@ -280,16 +335,22 @@ macro_rules! project_pointer {
$crate::ptr::project!(@gen $ptr, $($rest)*)
};
// Fallible index projection.
(@gen $ptr:ident, [$index:expr]? $($rest:tt)*) => {
(@gen $ptr:ident, [try: $index:expr] $($rest:tt)*) => {
let $ptr = $crate::ptr::projection::ProjectIndex::get($index, $ptr)
.ok_or($crate::ptr::projection::OutOfBound)?;
$crate::ptr::project!(@gen $ptr, $($rest)*)
};
// Build-time checked index projection.
(@gen $ptr:ident, [$index:expr] $($rest:tt)*) => {
// Panicking index projection.
(@gen $ptr:ident, [panic: $index:expr] $($rest:tt)*) => {
let $ptr = $crate::ptr::projection::ProjectIndex::index($index, $ptr);
$crate::ptr::project!(@gen $ptr, $($rest)*)
};
// Build-time checked index projection.
(@gen $ptr:ident, [build: $index:expr] $($rest:tt)*) => {
let $ptr = $crate::ptr::projection::ProjectIndex::build_index($index, $ptr);
$crate::ptr::project!(@gen $ptr, $($rest)*)
};
(mut $ptr:expr, $($proj:tt)*) => {{
let ptr: *mut _ = $ptr;
$crate::ptr::project!(@gen ptr, $($proj)*);

24
rust/kernel/str.rs
View File

@ -3,14 +3,27 @@
//! String representations.
use crate::{
alloc::{flags::*, AllocError, KVec},
error::{to_result, Result},
fmt::{self, Write},
prelude::*,
alloc::{
AllocError,
KVec, //
},
error::{
to_result,
Result, //
},
fmt::{
self,
Write, //
},
prelude::*, //
};
use core::{
marker::PhantomData,
ops::{Deref, DerefMut, Index},
ops::{
Deref,
DerefMut,
Index, //
}, //
};
pub use crate::prelude::CStr;
@ -415,6 +428,7 @@ macro_rules! c_str {
}};
}
#[cfg(CONFIG_RUST_STR_KUNIT_TEST)]
#[kunit_tests(rust_kernel_str)]
mod tests {
use super::*;

12
rust/kernel/sync/arc.rs
View File

@ -712,6 +712,7 @@ fn try_init<E>(init: impl Init<T, E>, flags: Flags) -> Result<Self, E>
impl<T> InPlaceWrite<T> for UniqueArc<MaybeUninit<T>> {
type Initialized = UniqueArc<T>;
#[inline]
fn write_init<E>(mut self, init: impl Init<T, E>) -> Result<Self::Initialized, E> {
let slot = self.as_mut_ptr();
// SAFETY: When init errors/panics, slot will get deallocated but not dropped,
@ -721,6 +722,7 @@ fn write_init<E>(mut self, init: impl Init<T, E>) -> Result<Self::Initialized, E
Ok(unsafe { self.assume_init() })
}
#[inline]
fn write_pin_init<E>(mut self, init: impl PinInit<T, E>) -> Result<Pin<Self::Initialized>, E> {
let slot = self.as_mut_ptr();
// SAFETY: When init errors/panics, slot will get deallocated but not dropped,
@ -758,6 +760,14 @@ pub fn new_uninit(flags: Flags) -> Result<UniqueArc<MaybeUninit<T>>, AllocError>
}
}
impl<T: ?Sized> UniqueArc<T> {
/// Return a raw pointer to the data in this [`UniqueArc`].
#[inline]
pub fn as_ptr(this: &Self) -> *const T {
Arc::as_ptr(&this.inner)
}
}
impl<T> UniqueArc<MaybeUninit<T>> {
/// Converts a `UniqueArc<MaybeUninit<T>>` into a `UniqueArc<T>` by writing a value into it.
pub fn write(mut self, value: T) -> UniqueArc<T> {
@ -782,6 +792,7 @@ pub unsafe fn assume_init(self) -> UniqueArc<T> {
}
/// Initialize `self` using the given initializer.
#[inline]
pub fn init_with<E>(mut self, init: impl Init<T, E>) -> core::result::Result<UniqueArc<T>, E> {
// SAFETY: The supplied pointer is valid for initialization.
match unsafe { init.__init(self.as_mut_ptr()) } {
@ -792,6 +803,7 @@ pub fn init_with<E>(mut self, init: impl Init<T, E>) -> core::result::Result<Uni
}
/// Pin-initialize `self` using the given pin-initializer.
#[inline]
pub fn pin_init_with<E>(
mut self,
init: impl PinInit<T, E>,

7
rust/kernel/sync/aref.rs
View File

@ -17,7 +17,12 @@
//! [`Arc`]: crate::sync::Arc
//! [`Arc<T>`]: crate::sync::Arc
use core::{marker::PhantomData, mem::ManuallyDrop, ops::Deref, ptr::NonNull};
use core::{
marker::PhantomData,
mem::ManuallyDrop,
ops::Deref,
ptr::NonNull, //
};
/// Types that are _always_ reference counted.
///

9
rust/kernel/sync/atomic/internal.rs
View File

@ -4,8 +4,11 @@
//!
//! Provides 1:1 mapping to the C atomic operations.
use crate::bindings;
use crate::macros::paste;
use crate::{
bindings,
build_assert::static_assert,
macros::paste, //
};
use core::cell::UnsafeCell;
use ffi::c_void;
@ -46,7 +49,7 @@ pub trait AtomicImpl: Sized + Copy + private::Sealed {
// In the future when a CONFIG_ARCH_SUPPORTS_ATOMIC_RMW=n architecture plans to support Rust, the
// load/store helpers that guarantee atomicity against RmW operations (usually via a lock) need to
// be added.
crate::static_assert!(
static_assert!(
cfg!(CONFIG_ARCH_SUPPORTS_ATOMIC_RMW),
"The current implementation of atomic i8/i16/ptr relies on the architecure being \
ARCH_SUPPORTS_ATOMIC_RMW"

9
rust/kernel/sync/atomic/predefine.rs
View File

@ -2,9 +2,7 @@
//! Pre-defined atomic types
use crate::static_assert;
use core::mem::{align_of, size_of};
use ffi::c_void;
use crate::prelude::*;
// Ensure size and alignment requirements are checked.
static_assert!(size_of::<bool>() == size_of::<i8>());
@ -154,9 +152,8 @@ fn rhs_into_delta(rhs: usize) -> isize_atomic_repr {
}
}
use crate::macros::kunit_tests;
#[kunit_tests(rust_atomics)]
#[cfg(CONFIG_RUST_ATOMICS_KUNIT_TEST)]
#[macros::kunit_tests(rust_atomics)]
mod tests {
use super::super::*;

2
rust/kernel/sync/lock/global.rs
View File

@ -85,6 +85,7 @@ pub fn lock(&'static self) -> GlobalGuard<B> {
}
/// Try to lock this global lock.
#[must_use = "if unused, the lock will be immediately unlocked"]
#[inline]
pub fn try_lock(&'static self) -> Option<GlobalGuard<B>> {
Some(GlobalGuard {
@ -96,6 +97,7 @@ pub fn try_lock(&'static self) -> Option<GlobalGuard<B>> {
/// A guard for a [`GlobalLock`].
///
/// See [`global_lock!`] for examples.
#[must_use = "the lock unlocks immediately when the guard is unused"]
pub struct GlobalGuard<B: GlobalLockBackend> {
inner: Guard<'static, B::Item, B::Backend>,
}

2
rust/kernel/sync/locked_by.rs
View File

@ -3,7 +3,7 @@
//! A wrapper for data protected by a lock that does not wrap it.
use super::{lock::Backend, lock::Lock};
use crate::build_assert;
use crate::build_assert::build_assert;
use core::{cell::UnsafeCell, mem::size_of, ptr};
/// Allows access to some data to be serialised by a lock that does not wrap it.

8
rust/kernel/sync/refcount.rs
View File

@ -4,9 +4,11 @@
//!
//! C header: [`include/linux/refcount.h`](srctree/include/linux/refcount.h)
use crate::build_assert;
use crate::sync::atomic::Atomic;
use crate::types::Opaque;
use crate::{
build_assert::build_assert,
sync::atomic::Atomic,
types::Opaque, //
};
/// Atomic reference counter.
///

10
rust/kernel/xarray.rs
View File

@ -5,10 +5,16 @@
//! C header: [`include/linux/xarray.h`](srctree/include/linux/xarray.h)
use crate::{
alloc, bindings, build_assert,
alloc,
bindings,
build_assert::build_assert,
error::{Error, Result},
ffi::c_void,
types::{ForeignOwnable, NotThreadSafe, Opaque},
types::{
ForeignOwnable,
NotThreadSafe,
Opaque, //
}, //
};
use core::{iter, marker::PhantomData, pin::Pin, ptr::NonNull};
use pin_init::{pin_data, pin_init, pinned_drop, PinInit};

2
rust/pin-init/README.md
View File

@ -3,7 +3,7 @@
[![Dependency status](https://deps.rs/repo/github/Rust-for-Linux/pin-init/status.svg)](https://deps.rs/repo/github/Rust-for-Linux/pin-init)
![License](https://img.shields.io/crates/l/pin-init)
[![Toolchain](https://img.shields.io/badge/toolchain-nightly-red)](#nightly-only)
![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/Rust-for-Linux/pin-init/test.yml)
![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/Rust-for-Linux/pin-init/ci.yml)
# `pin-init`
> [!NOTE]

3
rust/pin-init/examples/big_struct_in_place.rs
View File

@ -1,8 +1,5 @@
// SPDX-License-Identifier: Apache-2.0 OR MIT
#![cfg_attr(USE_RUSTC_FEATURES, feature(lint_reasons))]
#![cfg_attr(USE_RUSTC_FEATURES, feature(raw_ref_op))]
use pin_init::*;
// Struct with size over 1GiB

2
rust/pin-init/examples/error.rs
View File

@ -11,6 +11,7 @@
pub struct Error;
impl From<Infallible> for Error {
#[inline]
fn from(e: Infallible) -> Self {
match e {}
}
@ -18,6 +19,7 @@ fn from(e: Infallible) -> Self {
#[cfg(feature = "alloc")]
impl From<AllocError> for Error {
#[inline]
fn from(_: AllocError) -> Self {
Self
}

2
rust/pin-init/examples/linked_list.rs
View File

@ -2,8 +2,6 @@
#![allow(clippy::undocumented_unsafe_blocks)]
#![cfg_attr(feature = "alloc", feature(allocator_api))]
#![cfg_attr(USE_RUSTC_FEATURES, feature(lint_reasons))]
#![cfg_attr(USE_RUSTC_FEATURES, feature(raw_ref_op))]
use core::{
cell::Cell,

4
rust/pin-init/examples/mutex.rs
View File

@ -2,8 +2,6 @@
#![allow(clippy::undocumented_unsafe_blocks)]
#![cfg_attr(feature = "alloc", feature(allocator_api))]
#![cfg_attr(USE_RUSTC_FEATURES, feature(lint_reasons))]
#![cfg_attr(USE_RUSTC_FEATURES, feature(raw_ref_op))]
#![allow(clippy::missing_safety_doc)]
use core::{
@ -220,7 +218,7 @@ fn main() {
for h in handles {
h.join().expect("thread panicked");
}
println!("{:?}", &*mtx.lock());
println!("{:?}", *mtx.lock());
assert_eq!(*mtx.lock(), workload * thread_count * 2);
}
}

4
rust/pin-init/examples/pthread_mutex.rs
View File

@ -3,8 +3,6 @@
// inspired by <https://github.com/nbdd0121/pin-init/blob/trunk/examples/pthread_mutex.rs>
#![allow(clippy::undocumented_unsafe_blocks)]
#![cfg_attr(feature = "alloc", feature(allocator_api))]
#![cfg_attr(USE_RUSTC_FEATURES, feature(lint_reasons))]
#![cfg_attr(USE_RUSTC_FEATURES, feature(raw_ref_op))]
#[cfg(not(windows))]
mod pthread_mtx {
@ -179,7 +177,7 @@ fn main() {
for h in handles {
h.join().expect("thread panicked");
}
println!("{:?}", &*mtx.lock());
println!("{:?}", *mtx.lock());
assert_eq!(*mtx.lock(), workload * thread_count * 2);
}
}

4
rust/pin-init/examples/static_init.rs
View File

@ -2,8 +2,6 @@
#![allow(clippy::undocumented_unsafe_blocks)]
#![cfg_attr(feature = "alloc", feature(allocator_api))]
#![cfg_attr(USE_RUSTC_FEATURES, feature(lint_reasons))]
#![cfg_attr(USE_RUSTC_FEATURES, feature(raw_ref_op))]
#![allow(unused_imports)]
use core::{
@ -119,7 +117,7 @@ fn main() {
for h in handles {
h.join().expect("thread panicked");
}
println!("{:?}, {:?}", &*mtx.lock(), &*COUNT.lock());
println!("{:?}, {:?}", *mtx.lock(), *COUNT.lock());
assert_eq!(*mtx.lock(), workload * thread_count * 2);
}
}

14
rust/pin-init/internal/src/diagnostics.rs
View File

@ -3,6 +3,7 @@
use std::fmt::Display;
use proc_macro2::TokenStream;
use quote::quote_spanned;
use syn::{spanned::Spanned, Error};
pub(crate) struct DiagCtxt(TokenStream);
@ -15,6 +16,19 @@ pub(crate) fn error(&mut self, span: impl Spanned, msg: impl Display) -> ErrorGu
ErrorGuaranteed(())
}
pub(crate) fn warn(&mut self, span: impl Spanned, msg: impl Display) {
// Have the message start on a new line for visual clarity.
let msg = format!("\n{}", msg);
self.0.extend(quote_spanned!(span.span() =>
// Approximate using deprecated warning while `proc_macro_diagnostic` is unstable.
const _: () = {
#[deprecated = #msg]
const fn warn() {}
warn();
};
));
}
pub(crate) fn with(
fun: impl FnOnce(&mut DiagCtxt) -> Result<TokenStream, ErrorGuaranteed>,
) -> TokenStream {

179
rust/pin-init/internal/src/init.rs
View File

@ -1,7 +1,7 @@
// SPDX-License-Identifier: Apache-2.0 OR MIT
use proc_macro2::{Span, TokenStream};
use quote::{format_ident, quote, quote_spanned};
use quote::{format_ident, quote};
use syn::{
braced,
parse::{End, Parse},
@ -103,17 +103,15 @@ pub(crate) fn expand(
|(_, err)| Box::new(err),
);
let slot = format_ident!("slot");
let (has_data_trait, data_trait, get_data, init_from_closure) = if pinned {
let (has_data_trait, get_data, init_from_closure) = if pinned {
(
format_ident!("HasPinData"),
format_ident!("PinData"),
format_ident!("__pin_data"),
format_ident!("pin_init_from_closure"),
)
} else {
(
format_ident!("HasInitData"),
format_ident!("InitData"),
format_ident!("__init_data"),
format_ident!("init_from_closure"),
)
@ -157,8 +155,7 @@ fn assert_zeroable<T: ?::core::marker::Sized>(_: *mut T)
#path::#get_data()
};
// Ensure that `#data` really is of type `#data` and help with type inference:
let init = ::pin_init::__internal::#data_trait::make_closure::<_, #error>(
#data,
let init = #data.__make_closure::<_, #error>(
move |slot| {
#zeroable_check
#this
@ -172,14 +169,7 @@ fn assert_zeroable<T: ?::core::marker::Sized>(_: *mut T)
init(slot).map(|__InitOk| ())
};
// SAFETY: TODO
let init = unsafe { ::pin_init::#init_from_closure::<_, #error>(init) };
// FIXME: this let binding is required to avoid a compiler error (cycle when computing the
// opaque type returned by this function) before Rust 1.81. Remove after MSRV bump.
#[allow(
clippy::let_and_return,
reason = "some clippy versions warn about the let binding"
)]
init
unsafe { ::pin_init::#init_from_closure::<_, #error>(init) }
}})
}
@ -238,107 +228,82 @@ fn init_fields(
cfgs.retain(|attr| attr.path().is_ident("cfg"));
cfgs
};
let init = match kind {
InitializerKind::Value { ident, value } => {
let mut value_ident = ident.clone();
let value_prep = value.as_ref().map(|value| &value.1).map(|value| {
// Setting the span of `value_ident` to `value`'s span improves error messages
// when the type of `value` is wrong.
value_ident.set_span(value.span());
quote!(let #value_ident = #value;)
let ident = match kind {
InitializerKind::Value { ident, .. } => ident,
InitializerKind::Init { ident, .. } => ident,
InitializerKind::Code { block, .. } => {
res.extend(quote! {
#(#attrs)*
#[allow(unused_braces)]
#block
});
// Again span for better diagnostics
let write = quote_spanned!(ident.span()=> ::core::ptr::write);
quote! {
#(#attrs)*
{
#value_prep
// SAFETY: TODO
unsafe { #write(&raw mut (*#slot).#ident, #value_ident) };
}
}
continue;
}
InitializerKind::Init { ident, value, .. } => {
// Again span for better diagnostics
let init = format_ident!("init", span = value.span());
let value_init = if pinned {
quote! {
// SAFETY:
// - `slot` is valid, because we are inside of an initializer closure, we
// return when an error/panic occurs.
// - We also use `#data` to require the correct trait (`Init` or `PinInit`)
// for `#ident`.
unsafe { #data.#ident(&raw mut (*#slot).#ident, #init)? };
}
} else {
quote! {
// SAFETY: `slot` is valid, because we are inside of an initializer
// closure, we return when an error/panic occurs.
unsafe {
::pin_init::Init::__init(
#init,
&raw mut (*#slot).#ident,
)?
};
}
};
quote! {
#(#attrs)*
{
let #init = #value;
#value_init
}
}
}
InitializerKind::Code { block: value, .. } => quote! {
#(#attrs)*
#[allow(unused_braces)]
#value
},
};
res.extend(init);
if let Some(ident) = kind.ident() {
// `mixed_site` ensures that the guard is not accessible to the user-controlled code.
let guard = format_ident!("__{ident}_guard", span = Span::mixed_site());
// NOTE: The reference is derived from the guard so that it only lives as long as the
// guard does and cannot escape the scope. If it's created via `&mut (*#slot).#ident`
// like the unaligned field guard, it will become effectively `'static`.
let accessor = if pinned {
let project_ident = format_ident!("__project_{ident}");
quote! {
// SAFETY: the initialization is pinned.
unsafe { #data.#project_ident(#guard.let_binding()) }
}
} else {
quote! {
#guard.let_binding()
}
};
res.extend(quote! {
#(#cfgs)*
// Create the drop guard.
//
let slot = if pinned {
quote! {
// SAFETY:
// - `slot` is valid and properly aligned.
// - `make_field_check` checks that `&raw mut (*slot).#ident` is properly aligned.
// - `make_field_check` prevents `#ident` from being used twice, therefore
// `(*slot).#ident` is exclusively accessed and has not been initialized.
(unsafe { #data.#ident(#slot) })
}
} else {
quote! {
// For `init!()` macro, everything is unpinned.
// SAFETY:
// - `&raw mut (*slot).#ident` is valid.
// - `make_field_check` checks that `&raw mut (*slot).#ident` is properly aligned.
// - `(*slot).#ident` has been initialized above.
// - We only need the ownership to the pointee back when initialization has
// succeeded, where we `forget` the guard.
let mut #guard = unsafe {
::pin_init::__internal::DropGuard::new(
&raw mut (*slot).#ident
// - `make_field_check` prevents `#ident` from being used twice, therefore
// `(*slot).#ident` is exclusively accessed and has not been initialized.
(unsafe {
::pin_init::__internal::Slot::<::pin_init::__internal::Unpinned, _>::new(
&raw mut (*#slot).#ident
)
};
})
}
};
#(#cfgs)*
#[allow(unused_variables)]
let #ident = #accessor;
});
guards.push(guard);
guard_attrs.push(cfgs);
}
// `mixed_site` ensures that the guard is not accessible to the user-controlled code.
let guard = format_ident!("__{ident}_guard", span = Span::mixed_site());
let init = match kind {
InitializerKind::Value { ident, value } => {
let value = value
.as_ref()
.map(|(_, value)| quote!(#value))
.unwrap_or_else(|| quote!(#ident));
quote! {
#(#attrs)*
let mut #guard = #slot.write(#value);
}
}
InitializerKind::Init { value, .. } => {
quote! {
#(#attrs)*
let mut #guard = #slot.init(#value)?;
}
}
InitializerKind::Code { .. } => unreachable!(),
};
res.extend(quote! {
#init
#(#cfgs)*
// Allow `non_snake_case` since the same warning is going to be reported for the struct
// field.
#[allow(unused_variables, non_snake_case)]
let #ident = #guard.let_binding();
});
guards.push(guard);
guard_attrs.push(cfgs);
}
quote! {
#res
@ -389,7 +354,7 @@ fn make_field_check(
::core::ptr::write(slot, #path {
#(
#(#field_attrs)*
#field_name: ::core::panic!(),
#field_name: loop {},
)*
#zeroing_trailer
})

1
rust/pin-init/internal/src/lib.rs
View File

@ -6,7 +6,6 @@
//! `pin-init` proc macros.
#![cfg_attr(USE_RUSTC_FEATURES, feature(lint_reasons))]
// Documentation is done in the pin-init crate instead.
#![allow(missing_docs)]

268
rust/pin-init/internal/src/pin_data.rs
View File

@ -7,7 +7,7 @@
parse_quote, parse_quote_spanned,
spanned::Spanned,
visit_mut::VisitMut,
Field, Generics, Ident, Item, PathSegment, Type, TypePath, Visibility, WhereClause,
Attribute, Field, Generics, Ident, Item, PathSegment, Type, TypePath, Visibility, WhereClause,
};
use crate::diagnostics::{DiagCtxt, ErrorGuaranteed};
@ -35,6 +35,12 @@ fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
}
}
struct FieldInfo<'a> {
field: &'a Field,
pinned: bool,
cfg_attrs: Vec<&'a Attribute>,
}
pub(crate) fn pin_data(
args: Args,
input: Item,
@ -73,24 +79,37 @@ pub(crate) fn pin_data(
replacer.visit_generics_mut(&mut struct_.generics);
replacer.visit_fields_mut(&mut struct_.fields);
let fields: Vec<(bool, &Field)> = struct_
let fields: Vec<FieldInfo<'_>> = struct_
.fields
.iter_mut()
.map(|field| {
let len = field.attrs.len();
field.attrs.retain(|a| !a.path().is_ident("pin"));
(len != field.attrs.len(), &*field)
let pinned = len != field.attrs.len();
let cfg_attrs = field
.attrs
.iter()
.filter(|a| a.path().is_ident("cfg"))
.collect();
FieldInfo {
field: &*field,
pinned,
cfg_attrs,
}
})
.collect();
for (pinned, field) in &fields {
if !pinned && is_phantom_pinned(&field.ty) {
dcx.error(
field,
for field in &fields {
let ident = field.field.ident.as_ref().unwrap();
if !field.pinned && is_phantom_pinned(&field.field.ty) {
dcx.warn(
field.field,
format!(
"The field `{}` of type `PhantomPinned` only has an effect \
"The field `{ident}` of type `PhantomPinned` only has an effect \
if it has the `#[pin]` attribute",
field.ident.as_ref().unwrap(),
),
);
}
@ -143,7 +162,7 @@ fn is_phantom_pinned(ty: &Type) -> bool {
fn generate_unpin_impl(
ident: &Ident,
generics: &Generics,
fields: &[(bool, &Field)],
fields: &[FieldInfo<'_>],
) -> TokenStream {
let (_, ty_generics, _) = generics.split_for_impl();
let mut generics_with_pin_lt = generics.clone();
@ -160,19 +179,28 @@ fn generate_unpin_impl(
else {
unreachable!()
};
let pinned_fields = fields.iter().filter_map(|(b, f)| b.then_some(f));
let pinned_fields = fields.iter().filter(|f| f.pinned).map(|f| {
let ident = f.field.ident.as_ref().unwrap();
let ty = &f.field.ty;
let cfg_attrs = &f.cfg_attrs;
quote!(
#(#cfg_attrs)*
#ident: #ty
)
});
quote! {
// This struct will be used for the unpin analysis. It is needed, because only structurally
// pinned fields are relevant whether the struct should implement `Unpin`.
#[allow(dead_code)] // The fields below are never used.
#[allow(
dead_code, // The fields below are never used.
non_snake_case // The warning will be emitted on the struct definition.
)]
struct __Unpin #generics_with_pin_lt
#where_token
#predicates
{
__phantom_pin: ::core::marker::PhantomData<fn(&'__pin ()) -> &'__pin ()>,
__phantom: ::core::marker::PhantomData<
fn(#ident #ty_generics) -> #ident #ty_generics
>,
__phantom_pin: ::pin_init::__internal::PhantomInvariantLifetime<'__pin>,
__phantom: ::pin_init::__internal::PhantomInvariant<#ident #ty_generics>,
#(#pinned_fields),*
}
@ -238,7 +266,7 @@ fn generate_projections(
vis: &Visibility,
ident: &Ident,
generics: &Generics,
fields: &[(bool, &Field)],
fields: &[FieldInfo<'_>],
) -> TokenStream {
let (impl_generics, ty_generics, _) = generics.split_for_impl();
let mut generics_with_pin_lt = generics.clone();
@ -247,32 +275,23 @@ fn generate_projections(
let projection = format_ident!("{ident}Projection");
let this = format_ident!("this");
let (fields_decl, fields_proj) = collect_tuple(fields.iter().map(
|(
pinned,
Field {
vis,
ident,
ty,
attrs,
..
},
)| {
let mut attrs = attrs.clone();
attrs.retain(|a| !a.path().is_ident("pin"));
let mut no_doc_attrs = attrs.clone();
no_doc_attrs.retain(|a| !a.path().is_ident("doc"));
let (fields_decl, fields_proj): (Vec<_>, Vec<_>) = fields
.iter()
.map(|field| {
let Field { vis, ident, ty, .. } = &field.field;
let cfg_attrs = &field.cfg_attrs;
let ident = ident
.as_ref()
.expect("only structs with named fields are supported");
if *pinned {
if field.pinned {
(
quote!(
#(#attrs)*
#(#cfg_attrs)*
#vis #ident: ::core::pin::Pin<&'__pin mut #ty>,
),
quote!(
#(#no_doc_attrs)*
#(#cfg_attrs)*
// SAFETY: this field is structurally pinned.
#ident: unsafe { ::core::pin::Pin::new_unchecked(&mut #this.#ident) },
),
@ -280,31 +299,35 @@ fn generate_projections(
} else {
(
quote!(
#(#attrs)*
#(#cfg_attrs)*
#vis #ident: &'__pin mut #ty,
),
quote!(
#(#no_doc_attrs)*
#(#cfg_attrs)*
#ident: &mut #this.#ident,
),
)
}
},
));
})
.collect();
let structurally_pinned_fields_docs = fields
.iter()
.filter_map(|(pinned, field)| pinned.then_some(field))
.map(|Field { ident, .. }| format!(" - `{}`", ident.as_ref().unwrap()));
.filter(|f| f.pinned)
.map(|f| format!(" - `{}`", f.field.ident.as_ref().unwrap()));
let not_structurally_pinned_fields_docs = fields
.iter()
.filter_map(|(pinned, field)| (!pinned).then_some(field))
.map(|Field { ident, .. }| format!(" - `{}`", ident.as_ref().unwrap()));
.filter(|f| !f.pinned)
.map(|f| format!(" - `{}`", f.field.ident.as_ref().unwrap()));
let docs = format!(" Pin-projections of [`{ident}`]");
quote! {
#[doc = #docs]
#[allow(dead_code)]
// Allow `non_snake_case` since the same warning will be emitted on
// the struct definition.
#[allow(dead_code, non_snake_case)]
#[doc(hidden)]
#vis struct #projection #generics_with_pin_lt {
#vis struct #projection #generics_with_pin_lt
#whr
{
#(#fields_decl)*
___pin_phantom_data: ::core::marker::PhantomData<&'__pin mut ()>,
}
@ -336,91 +359,54 @@ impl #impl_generics #ident #ty_generics
fn generate_the_pin_data(
vis: &Visibility,
ident: &Ident,
struct_name: &Ident,
generics: &Generics,
fields: &[(bool, &Field)],
fields: &[FieldInfo<'_>],
) -> TokenStream {
let (impl_generics, ty_generics, whr) = generics.split_for_impl();
// For every field, we create an initializing projection function according to its projection
// type. If a field is structurally pinned, then it must be initialized via `PinInit`, if it is
// not structurally pinned, then it can be initialized via `Init`.
//
// The functions are `unsafe` to prevent accidentally calling them.
fn handle_field(
Field {
vis,
ident,
ty,
attrs,
..
}: &Field,
struct_ident: &Ident,
pinned: bool,
) -> TokenStream {
let mut attrs = attrs.clone();
attrs.retain(|a| !a.path().is_ident("pin"));
let ident = ident
.as_ref()
.expect("only structs with named fields are supported");
let project_ident = format_ident!("__project_{ident}");
let (init_ty, init_fn, project_ty, project_body, pin_safety) = if pinned {
(
quote!(PinInit),
quote!(__pinned_init),
quote!(::core::pin::Pin<&'__slot mut #ty>),
// SAFETY: this field is structurally pinned.
quote!(unsafe { ::core::pin::Pin::new_unchecked(slot) }),
quote!(
/// - `slot` will not move until it is dropped, i.e. it will be pinned.
),
)
} else {
(
quote!(Init),
quote!(__init),
quote!(&'__slot mut #ty),
quote!(slot),
quote!(),
)
};
let slot_safety = format!(
" `slot` points at the field `{ident}` inside of `{struct_ident}`, which is pinned.",
);
quote! {
/// # Safety
///
/// - `slot` is a valid pointer to uninitialized memory.
/// - the caller does not touch `slot` when `Err` is returned, they are only permitted
/// to deallocate.
#pin_safety
#(#attrs)*
#vis unsafe fn #ident<E>(
self,
slot: *mut #ty,
init: impl ::pin_init::#init_ty<#ty, E>,
) -> ::core::result::Result<(), E> {
// SAFETY: this function has the same safety requirements as the __init function
// called below.
unsafe { ::pin_init::#init_ty::#init_fn(init, slot) }
}
/// # Safety
///
#[doc = #slot_safety]
#(#attrs)*
#vis unsafe fn #project_ident<'__slot>(
self,
slot: &'__slot mut #ty,
) -> #project_ty {
#project_body
}
}
}
// type. If a field is structurally pinned, we create a `Slot` with `Pinned` which must be
// initialized via `PinInit`; if it is not structurally pinned, then we create a `Slot` with
// `Unpinned` which allows initialization via `Init`.
let field_accessors = fields
.iter()
.map(|(pinned, field)| handle_field(field, ident, *pinned))
.map(|f| {
let Field { vis, ident, ty, .. } = f.field;
let cfg_attrs = &f.cfg_attrs;
let field_name = ident
.as_ref()
.expect("only structs with named fields are supported");
let pin_marker = if f.pinned {
quote!(Pinned)
} else {
quote!(Unpinned)
};
quote! {
/// # Safety
///
/// - `slot` is valid and properly aligned.
/// - `(*slot).#field_name` is properly aligned.
/// - `(*slot).#field_name` points to uninitialized and exclusively accessed
/// memory.
#(#cfg_attrs)*
// Allow `non_snake_case` since the same warning will be emitted on
// the struct definition.
#[allow(non_snake_case)]
#[inline(always)]
#vis unsafe fn #field_name(
self,
slot: *mut #struct_name #ty_generics,
) -> ::pin_init::__internal::Slot<::pin_init::__internal::#pin_marker, #ty> {
// SAFETY:
// - If `#pin_marker` is `Pinned`, the corresponding field is structurally
// pinned.
// - Other safety requirements follows the safety requirement.
unsafe { ::pin_init::__internal::Slot::new(&raw mut (*slot).#field_name) }
}
}
})
.collect::<TokenStream>();
quote! {
// We declare this struct which will host all of the projection function for our type. It
@ -429,9 +415,7 @@ fn handle_field(
#vis struct __ThePinData #generics
#whr
{
__phantom: ::core::marker::PhantomData<
fn(#ident #ty_generics) -> #ident #ty_generics
>,
__phantom: ::pin_init::__internal::PhantomInvariant<#struct_name #ty_generics>,
}
impl #impl_generics ::core::clone::Clone for __ThePinData #ty_generics
@ -449,27 +433,30 @@ impl #impl_generics ::core::marker::Copy for __ThePinData #ty_generics
impl #impl_generics __ThePinData #ty_generics
#whr
{
/// Type inference helper function.
#[inline(always)]
#vis fn __make_closure<__F, __E>(self, f: __F) -> __F
where
__F: FnOnce(*mut #struct_name #ty_generics) ->
::core::result::Result<::pin_init::__internal::InitOk, __E>,
{
f
}
#field_accessors
}
// SAFETY: We have added the correct projection functions above to `__ThePinData` and
// we also use the least restrictive generics possible.
unsafe impl #impl_generics ::pin_init::__internal::HasPinData for #ident #ty_generics
unsafe impl #impl_generics ::pin_init::__internal::HasPinData for #struct_name #ty_generics
#whr
{
type PinData = __ThePinData #ty_generics;
unsafe fn __pin_data() -> Self::PinData {
__ThePinData { __phantom: ::core::marker::PhantomData }
__ThePinData { __phantom: ::pin_init::__internal::PhantomInvariant::new() }
}
}
// SAFETY: TODO
unsafe impl #impl_generics ::pin_init::__internal::PinData for __ThePinData #ty_generics
#whr
{
type Datee = #ident #ty_generics;
}
}
}
@ -500,14 +487,3 @@ fn visit_item_mut(&mut self, _: &mut Item) {
// Do not descend into items, since items reset/change what `Self` refers to.
}
}
// replace with `.collect()` once MSRV is above 1.79
fn collect_tuple<A, B>(iter: impl Iterator<Item = (A, B)>) -> (Vec<A>, Vec<B>) {
let mut res_a = vec![];
let mut res_b = vec![];
for (a, b) in iter {
res_a.push(a);
res_b.push(b);
}
(res_a, res_b)
}

2
rust/pin-init/internal/src/zeroable.rs
View File

@ -1,4 +1,4 @@
// SPDX-License-Identifier: GPL-2.0
// SPDX-License-Identifier: Apache-2.0 OR MIT
use proc_macro2::TokenStream;
use quote::quote;

231
rust/pin-init/src/__internal.rs
View File

@ -7,42 +7,54 @@
use super::*;
/// Zero-sized type used to mark a type as invariant.
///
/// This is a polyfill for the [unstable type] in the standard library of the same name.
///
/// See the [nomicon] for what subtyping is. See also [this table].
///
/// The reason for not using `PhantomData<*mut T>` is that that type never implements [`Send`] and
/// [`Sync`]. Hence `fn(*mut T) -> *mut T` is used, as that type always implements them.
///
/// [unstable type]: https://doc.rust-lang.org/nightly/std/marker/struct.PhantomInvariant.html
/// [nomicon]: https://doc.rust-lang.org/nomicon/subtyping.html
/// [this table]: https://doc.rust-lang.org/nomicon/phantom-data.html#table-of-phantomdata-patterns
pub(crate) type Invariant<T> = PhantomData<fn(*mut T) -> *mut T>;
#[repr(transparent)]
pub struct PhantomInvariant<T: ?Sized>(PhantomData<fn(T) -> T>);
/// Module-internal type implementing `PinInit` and `Init`.
///
/// It is unsafe to create this type, since the closure needs to fulfill the same safety
/// requirement as the `__pinned_init`/`__init` functions.
pub(crate) struct InitClosure<F, T: ?Sized, E>(pub(crate) F, pub(crate) Invariant<(E, T)>);
// SAFETY: While constructing the `InitClosure`, the user promised that it upholds the
// `__init` invariants.
unsafe impl<T: ?Sized, F, E> Init<T, E> for InitClosure<F, T, E>
where
F: FnOnce(*mut T) -> Result<(), E>,
{
#[inline]
unsafe fn __init(self, slot: *mut T) -> Result<(), E> {
(self.0)(slot)
impl<T: ?Sized> Clone for PhantomInvariant<T> {
#[inline(always)]
fn clone(&self) -> Self {
*self
}
}
// SAFETY: While constructing the `InitClosure`, the user promised that it upholds the
// `__pinned_init` invariants.
unsafe impl<T: ?Sized, F, E> PinInit<T, E> for InitClosure<F, T, E>
where
F: FnOnce(*mut T) -> Result<(), E>,
{
#[inline]
unsafe fn __pinned_init(self, slot: *mut T) -> Result<(), E> {
(self.0)(slot)
impl<T: ?Sized> Copy for PhantomInvariant<T> {}
impl<T: ?Sized> Default for PhantomInvariant<T> {
#[inline(always)]
fn default() -> Self {
Self::new()
}
}
impl<T: ?Sized> PhantomInvariant<T> {
#[inline(always)]
pub const fn new() -> Self {
Self(PhantomData)
}
}
/// Zero-sized type used to mark a lifetime as invariant.
///
/// This is a polyfill for the [unstable type] in the standard library of the same name.
///
/// [unstable type]: https://doc.rust-lang.org/nightly/std/marker/struct.PhantomInvariantLifetime.html
#[repr(transparent)]
#[derive(Clone, Copy, Default)]
pub struct PhantomInvariantLifetime<'a>(PhantomInvariant<&'a ()>);
impl PhantomInvariantLifetime<'_> {
#[inline(always)]
pub const fn new() -> Self {
Self(PhantomInvariant::new())
}
}
@ -71,30 +83,12 @@ pub unsafe fn new() -> Self {
///
/// Only the `init` module is allowed to use this trait.
pub unsafe trait HasPinData {
type PinData: PinData;
type PinData;
#[expect(clippy::missing_safety_doc)]
unsafe fn __pin_data() -> Self::PinData;
}
/// Marker trait for pinning data of structs.
///
/// # Safety
///
/// Only the `init` module is allowed to use this trait.
pub unsafe trait PinData: Copy {
type Datee: ?Sized + HasPinData;
/// Type inference helper function.
#[inline(always)]
fn make_closure<F, E>(self, f: F) -> F
where
F: FnOnce(*mut Self::Datee) -> Result<InitOk, E>,
{
f
}
}
/// This trait is automatically implemented for every type. It aims to provide the same type
/// inference help as `HasPinData`.
///
@ -102,31 +96,13 @@ fn make_closure<F, E>(self, f: F) -> F
///
/// Only the `init` module is allowed to use this trait.
pub unsafe trait HasInitData {
type InitData: InitData;
type InitData;
#[expect(clippy::missing_safety_doc)]
unsafe fn __init_data() -> Self::InitData;
}
/// Same function as `PinData`, but for arbitrary data.
///
/// # Safety
///
/// Only the `init` module is allowed to use this trait.
pub unsafe trait InitData: Copy {
type Datee: ?Sized + HasInitData;
/// Type inference helper function.
#[inline(always)]
fn make_closure<F, E>(self, f: F) -> F
where
F: FnOnce(*mut Self::Datee) -> Result<InitOk, E>,
{
f
}
}
pub struct AllData<T: ?Sized>(Invariant<T>);
pub struct AllData<T: ?Sized>(PhantomInvariant<T>);
impl<T: ?Sized> Clone for AllData<T> {
fn clone(&self) -> Self {
@ -136,9 +112,15 @@ fn clone(&self) -> Self {
impl<T: ?Sized> Copy for AllData<T> {}
// SAFETY: TODO.
unsafe impl<T: ?Sized> InitData for AllData<T> {
type Datee = T;
impl<T: ?Sized> AllData<T> {
/// Type inference helper function.
#[inline(always)]
pub fn __make_closure<F, E>(self, f: F) -> F
where
F: FnOnce(*mut T) -> Result<InitOk, E>,
{
f
}
}
// SAFETY: TODO.
@ -146,7 +128,7 @@ unsafe impl<T: ?Sized> HasInitData for T {
type InitData = AllData<T>;
unsafe fn __init_data() -> Self::InitData {
AllData(PhantomData)
AllData(PhantomInvariant::new())
}
}
@ -235,6 +217,87 @@ struct Foo {
println!("{value:?}");
}
// Marker types that determines type of `DropGuard`'s let bindings.
pub struct Pinned;
pub struct Unpinned;
/// Represent an uninitialized field.
///
/// # Invariants
///
/// - `ptr` is valid, properly aligned and points to uninitialized and exclusively accessed memory.
/// - If `P` is `Pinned`, then `ptr` is structurally pinned.
pub struct Slot<P, T: ?Sized> {
ptr: *mut T,
_phantom: PhantomData<P>,
}
impl<P, T: ?Sized> Slot<P, T> {
/// # Safety
///
/// - `ptr` is valid, properly aligned and points to uninitialized and exclusively accessed
/// memory.
/// - If `P` is `Pinned`, then `ptr` is structurally pinned.
#[inline(always)]
pub unsafe fn new(ptr: *mut T) -> Self {
// INVARIANT: Per safety requirement.
Self {
ptr,
_phantom: PhantomData,
}
}
/// Initialize the field by value.
#[inline(always)]
pub fn write(self, value: T) -> DropGuard<P, T>
where
T: Sized,
{
// SAFETY: `self.ptr` is a valid and aligned pointer for write.
unsafe { self.ptr.write(value) }
// SAFETY:
// - `self.ptr` is valid and properly aligned per type invariant.
// - `*self.ptr` is initialized above and the ownership is transferred to the guard.
// - If `P` is `Pinned`, `self.ptr` is pinned.
unsafe { DropGuard::new(self.ptr) }
}
}
impl<T: ?Sized> Slot<Unpinned, T> {
/// Initialize the field.
#[inline(always)]
pub fn init<E>(self, init: impl Init<T, E>) -> Result<DropGuard<Unpinned, T>, E> {
// SAFETY:
// - `self.ptr` is valid and properly aligned.
// - when `Err` is returned, we also propagate the error without touching `slot`;
// also `self` is consumed so it cannot be touched further.
unsafe { init.__init(self.ptr)? };
// SAFETY:
// - `self.ptr` is valid and properly aligned per type invariant.
// - `*self.ptr` is initialized above and the ownership is transferred to the guard.
Ok(unsafe { DropGuard::new(self.ptr) })
}
}
impl<T: ?Sized> Slot<Pinned, T> {
/// Initialize the field.
#[inline(always)]
pub fn init<E>(self, init: impl PinInit<T, E>) -> Result<DropGuard<Pinned, T>, E> {
// SAFETY:
// - `self.ptr` is valid and properly aligned.
// - when `Err` is returned, we also propagate the error without touching `ptr`;
// also `self` is consumed so it cannot be touched further.
// - the drop guard will not hand out `&mut` (only `Pin<&mut T>`).
unsafe { init.__pinned_init(self.ptr)? };
// SAFETY:
// - `self.ptr` is valid, properly aligned and pinned per type invariant.
// - `*self.ptr` is initialized above and the ownership is transferred to the guard.
Ok(unsafe { DropGuard::new(self.ptr) })
}
}
/// When a value of this type is dropped, it drops a `T`.
///
/// Can be forgotten to prevent the drop.
@ -243,11 +306,13 @@ struct Foo {
///
/// - `ptr` is valid and properly aligned.
/// - `*ptr` is initialized and owned by this guard.
pub struct DropGuard<T: ?Sized> {
/// - if `P` is `Pinned`, `ptr` is pinned.
pub struct DropGuard<P, T: ?Sized> {
ptr: *mut T,
phantom: PhantomData<P>,
}
impl<T: ?Sized> DropGuard<T> {
impl<P, T: ?Sized> DropGuard<P, T> {
/// Creates a drop guard and transfer the ownership of the pointer content.
///
/// The ownership is only relinguished if the guard is forgotten via [`core::mem::forget`].
@ -256,12 +321,18 @@ impl<T: ?Sized> DropGuard<T> {
///
/// - `ptr` is valid and properly aligned.
/// - `*ptr` is initialized, and the ownership is transferred to this guard.
/// - if `P` is `Pinned`, `ptr` is pinned.
#[inline]
pub unsafe fn new(ptr: *mut T) -> Self {
// INVARIANT: By safety requirement.
Self { ptr }
Self {
ptr,
phantom: PhantomData,
}
}
}
impl<T: ?Sized> DropGuard<Unpinned, T> {
/// Create a let binding for accessor use.
#[inline]
pub fn let_binding(&mut self) -> &mut T {
@ -270,7 +341,17 @@ pub fn let_binding(&mut self) -> &mut T {
}
}
impl<T: ?Sized> Drop for DropGuard<T> {
impl<T: ?Sized> DropGuard<Pinned, T> {
/// Create a let binding for accessor use.
#[inline]
pub fn let_binding(&mut self) -> Pin<&mut T> {
// SAFETY: `self.ptr` is valid, properly aligned, initialized, exclusively accessible and
// pinned per type invariant.
unsafe { Pin::new_unchecked(&mut *self.ptr) }
}
}
impl<P, T: ?Sized> Drop for DropGuard<P, T> {
#[inline]
fn drop(&mut self) {
// SAFETY: `self.ptr` is valid, properly aligned and `*self.ptr` is owned by this guard.

137
rust/pin-init/src/lib.rs
View File

@ -263,12 +263,6 @@
//! [`impl Init<T, E>`]: crate::Init
//! [Rust-for-Linux]: https://rust-for-linux.com/
#![cfg_attr(USE_RUSTC_FEATURES, feature(lint_reasons))]
#![cfg_attr(USE_RUSTC_FEATURES, feature(raw_ref_op))]
#![cfg_attr(
all(any(feature = "alloc", feature = "std"), USE_RUSTC_FEATURES),
feature(new_uninit)
)]
#![forbid(missing_docs, unsafe_op_in_unsafe_fn)]
#![cfg_attr(not(feature = "std"), no_std)]
#![cfg_attr(feature = "alloc", feature(allocator_api))]
@ -437,7 +431,7 @@
/// ```
/// use pin_init::MaybeZeroable;
///
/// // implmements `Zeroable`
/// // implements `Zeroable`
/// #[derive(MaybeZeroable)]
/// pub struct DriverData {
/// pub(crate) id: i64,
@ -445,7 +439,7 @@
/// len: usize,
/// }
///
/// // does not implmement `Zeroable`
/// // does not implement `Zeroable`
/// #[derive(MaybeZeroable)]
/// pub struct DriverData2 {
/// pub(crate) id: i64,
@ -873,12 +867,12 @@ macro_rules! stack_try_pin_init {
#[macro_export]
macro_rules! assert_pinned {
($ty:ty, $field:ident, $field_ty:ty, inline) => {
let _ = move |ptr: *mut $field_ty| {
// SAFETY: This code is unreachable.
let data = unsafe { <$ty as $crate::__internal::HasPinData>::__pin_data() };
let init = $crate::__internal::AlwaysFail::<$field_ty>::new();
// SAFETY: This code is unreachable.
unsafe { data.$field(ptr, init) }.ok();
// SAFETY: This code is unreachable.
let _ = move |ptr: *mut $ty| unsafe {
let data = <$ty as $crate::__internal::HasPinData>::__pin_data();
_ = data
.$field(ptr)
.init($crate::__internal::AlwaysFail::<$field_ty>::new());
};
};
@ -953,12 +947,12 @@ fn pin_chain<F>(self, f: F) -> ChainPinInit<Self, F, T, E>
where
F: FnOnce(Pin<&mut T>) -> Result<(), E>,
{
ChainPinInit(self, f, PhantomData)
ChainPinInit(self, f, __internal::PhantomInvariant::new())
}
}
/// An initializer returned by [`PinInit::pin_chain`].
pub struct ChainPinInit<I, F, T: ?Sized, E>(I, F, __internal::Invariant<(E, T)>);
pub struct ChainPinInit<I, F, T: ?Sized, E>(I, F, __internal::PhantomInvariant<(E, T)>);
// SAFETY: The `__pinned_init` function is implemented such that it
// - returns `Ok(())` on successful initialization,
@ -1061,12 +1055,12 @@ fn chain<F>(self, f: F) -> ChainInit<Self, F, T, E>
where
F: FnOnce(&mut T) -> Result<(), E>,
{
ChainInit(self, f, PhantomData)
ChainInit(self, f, __internal::PhantomInvariant::new())
}
}
/// An initializer returned by [`Init::chain`].
pub struct ChainInit<I, F, T: ?Sized, E>(I, F, __internal::Invariant<(E, T)>);
pub struct ChainInit<I, F, T: ?Sized, E>(I, F, __internal::PhantomInvariant<(E, T)>);
// SAFETY: The `__init` function is implemented such that it
// - returns `Ok(())` on successful initialization,
@ -1098,6 +1092,36 @@ unsafe fn __pinned_init(self, slot: *mut T) -> Result<(), E> {
}
}
/// Implement `PinInit` and `Init` for closures.
///
/// It is unsafe to create this type, since the closure needs to fulfill the same safety
/// requirement as the `__pinned_init`/`__init` functions.
struct InitClosure<F, T: ?Sized>(F, __internal::PhantomInvariant<T>);
// SAFETY: While constructing the `InitClosure`, the user promised that it upholds the
// `__init` invariants.
unsafe impl<T: ?Sized, F, E> Init<T, E> for InitClosure<F, T>
where
F: FnOnce(*mut T) -> Result<(), E>,
{
#[inline]
unsafe fn __init(self, slot: *mut T) -> Result<(), E> {
(self.0)(slot)
}
}
// SAFETY: While constructing the `InitClosure`, the user promised that it upholds the
// `__pinned_init` invariants.
unsafe impl<T: ?Sized, F, E> PinInit<T, E> for InitClosure<F, T>
where
F: FnOnce(*mut T) -> Result<(), E>,
{
#[inline]
unsafe fn __pinned_init(self, slot: *mut T) -> Result<(), E> {
(self.0)(slot)
}
}
/// Creates a new [`PinInit<T, E>`] from the given closure.
///
/// # Safety
@ -1114,7 +1138,7 @@ unsafe fn __pinned_init(self, slot: *mut T) -> Result<(), E> {
pub const unsafe fn pin_init_from_closure<T: ?Sized, E>(
f: impl FnOnce(*mut T) -> Result<(), E>,
) -> impl PinInit<T, E> {
__internal::InitClosure(f, PhantomData)
InitClosure(f, __internal::PhantomInvariant::new())
}
/// Creates a new [`Init<T, E>`] from the given closure.
@ -1133,7 +1157,7 @@ unsafe fn __pinned_init(self, slot: *mut T) -> Result<(), E> {
pub const unsafe fn init_from_closure<T: ?Sized, E>(
f: impl FnOnce(*mut T) -> Result<(), E>,
) -> impl Init<T, E> {
__internal::InitClosure(f, PhantomData)
InitClosure(f, __internal::PhantomInvariant::new())
}
/// Changes the to be initialized type.
@ -1145,14 +1169,7 @@ unsafe fn __pinned_init(self, slot: *mut T) -> Result<(), E> {
pub const unsafe fn cast_pin_init<T, U, E>(init: impl PinInit<T, E>) -> impl PinInit<U, E> {
// SAFETY: initialization delegated to a valid initializer. Cast is valid by function safety
// requirements.
let res = unsafe { pin_init_from_closure(|ptr: *mut U| init.__pinned_init(ptr.cast::<T>())) };
// FIXME: this let binding is required to avoid a compiler error (cycle when computing the opaque
// type returned by this function) before Rust 1.81. Remove after MSRV bump.
#[allow(
clippy::let_and_return,
reason = "some clippy versions warn about the let binding"
)]
res
unsafe { pin_init_from_closure(|ptr: *mut U| init.__pinned_init(ptr.cast::<T>())) }
}
/// Changes the to be initialized type.
@ -1164,14 +1181,7 @@ unsafe fn __pinned_init(self, slot: *mut T) -> Result<(), E> {
pub const unsafe fn cast_init<T, U, E>(init: impl Init<T, E>) -> impl Init<U, E> {
// SAFETY: initialization delegated to a valid initializer. Cast is valid by function safety
// requirements.
let res = unsafe { init_from_closure(|ptr: *mut U| init.__init(ptr.cast::<T>())) };
// FIXME: this let binding is required to avoid a compiler error (cycle when computing the opaque
// type returned by this function) before Rust 1.81. Remove after MSRV bump.
#[allow(
clippy::let_and_return,
reason = "some clippy versions warn about the let binding"
)]
res
unsafe { init_from_closure(|ptr: *mut U| init.__init(ptr.cast::<T>())) }
}
/// An initializer that leaves the memory uninitialized.
@ -1523,27 +1533,6 @@ fn zeroed() -> Self
}
}
/// Marker trait for types that allow `Option<Self>` to be set to all zeroes in order to write
/// `None` to that location.
///
/// # Safety
///
/// The implementer needs to ensure that `unsafe impl Zeroable for Option<Self> {}` is sound.
pub unsafe trait ZeroableOption {}
// SAFETY: by the safety requirement of `ZeroableOption`, this is valid.
unsafe impl<T: ZeroableOption> Zeroable for Option<T> {}
// SAFETY: `Option<&T>` is part of the option layout optimization guarantee:
// <https://doc.rust-lang.org/stable/std/option/index.html#representation>.
unsafe impl<T> ZeroableOption for &T {}
// SAFETY: `Option<&mut T>` is part of the option layout optimization guarantee:
// <https://doc.rust-lang.org/stable/std/option/index.html#representation>.
unsafe impl<T> ZeroableOption for &mut T {}
// SAFETY: `Option<NonNull<T>>` is part of the option layout optimization guarantee:
// <https://doc.rust-lang.org/stable/std/option/index.html#representation>.
unsafe impl<T> ZeroableOption for NonNull<T> {}
/// Create an initializer for a zeroed `T`.
///
/// The returned initializer will write `0x00` to every byte of the given `slot`.
@ -1649,6 +1638,17 @@ unsafe impl<$first: Zeroable, $($t: Zeroable),*> Zeroable for ($first, $($t),*)
impl_tuple_zeroable!(A, B, C, D, E, F, G, H, I, J);
/// Marker trait for types that allow `Option<Self>` to be set to all zeroes in order to write
/// `None` to that location.
///
/// # Safety
///
/// The implementer needs to ensure that `unsafe impl Zeroable for Option<Self> {}` is sound.
pub unsafe trait ZeroableOption {}
// SAFETY: by the safety requirement of `ZeroableOption`, this is valid.
unsafe impl<T: ZeroableOption> Zeroable for Option<T> {}
macro_rules! impl_fn_zeroable_option {
([$($abi:literal),* $(,)?] $args:tt) => {
$(impl_fn_zeroable_option!({extern $abi} $args);)*
@ -1674,18 +1674,27 @@ unsafe impl<$ret, $($rest),*> ZeroableOption for $($prefix)* fn($($rest),*) -> $
impl_fn_zeroable_option!(["Rust", "C"] { A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U });
macro_rules! impl_non_zero_int_zeroable_option {
($($int:ty),* $(,)?) => {
// SAFETY: Safety comment written in the macro invocation.
$(unsafe impl ZeroableOption for $int {})*
macro_rules! impl_zeroable_option {
($($({$($generics:tt)*})? $t:ty, )*) => {
// SAFETY: Safety comments written in the macro invocation.
$(unsafe impl$($($generics)*)? ZeroableOption for $t {})*
};
}
impl_non_zero_int_zeroable_option! {
impl_zeroable_option! {
// SAFETY: `Option<&T>` is part of the option layout optimization guarantee:
// <https://doc.rust-lang.org/stable/std/option/index.html#representation>.
{<T: ?Sized>} &T,
// SAFETY: `Option<&mut T>` is part of the option layout optimization guarantee:
// <https://doc.rust-lang.org/stable/std/option/index.html#representation>.
{<T: ?Sized>} &mut T,
// SAFETY: `Option<NonNull<T>>` is part of the option layout optimization guarantee:
// <https://doc.rust-lang.org/stable/std/option/index.html#representation>.
{<T: ?Sized>} NonNull<T>,
// SAFETY: All zeros is equivalent to `None` (option layout optimization guarantee:
// <https://doc.rust-lang.org/stable/std/option/index.html#representation>).
NonZeroU8, NonZeroU16, NonZeroU32, NonZeroU64, NonZeroU128, NonZeroUsize,
NonZeroI8, NonZeroI16, NonZeroI32, NonZeroI64, NonZeroI128, NonZeroIsize,
NonZero<u8>, NonZero<u16>, NonZero<u32>, NonZero<u64>, NonZero<u128>, NonZero<usize>,
NonZero<i8>, NonZero<i16>, NonZero<i32>, NonZero<i64>, NonZero<i128>, NonZero<isize>,
}
/// This trait allows creating an instance of `Self` which contains exactly one

14
rust/zerocopy-derive/README.md Normal file
View File

@ -0,0 +1,14 @@
# `zerocopy-derive`
These source files come from the Rust `zerocopy-derive` crate, version v0.8.50
(released 2026-05-31), hosted in the <https://github.com/google/zerocopy>
repository, licensed under "BSD-2-Clause OR Apache-2.0 OR MIT" and only
modified to add the SPDX license identifiers and to remove the generation of
non-ASCII identifiers.
For copyright details, please see:
https://github.com/google/zerocopy/blob/v0.8.50/README.md?plain=1
https://github.com/google/zerocopy/blob/v0.8.50/LICENSE-BSD
https://github.com/google/zerocopy/blob/v0.8.50/LICENSE-APACHE
https://github.com/google/zerocopy/blob/v0.8.50/LICENSE-MIT

192
rust/zerocopy-derive/derive/from_bytes.rs Normal file
View File

@ -0,0 +1,192 @@
// SPDX-License-Identifier: (BSD-2-Clause OR Apache-2.0) OR MIT
use proc_macro2::{Span, TokenStream};
use syn::{
parse_quote, Data, DataEnum, DataStruct, DataUnion, Error, Expr, ExprLit, ExprUnary, Lit, UnOp,
WherePredicate,
};
use crate::{
derive::try_from_bytes::derive_try_from_bytes,
repr::{CompoundRepr, EnumRepr, Repr, Spanned},
util::{enum_size_from_repr, Ctx, FieldBounds, ImplBlockBuilder, Trait, TraitBound},
};
/// Returns `Ok(index)` if variant `index` of the enum has a discriminant of
/// zero. If `Err(bool)` is returned, the boolean is true if the enum has
/// unknown discriminants (e.g. discriminants set to const expressions which we
/// can't evaluate in a proc macro). If the enum has unknown discriminants, then
/// it might have a zero variant that we just can't detect.
pub(crate) fn find_zero_variant(enm: &DataEnum) -> Result<usize, bool> {
// Discriminants can be anywhere in the range [i128::MIN, u128::MAX] because
// the discriminant type may be signed or unsigned. Since we only care about
// tracking the discriminant when it's less than or equal to zero, we can
// avoid u128 -> i128 conversions and bounds checking by making the "next
// discriminant" value implicitly negative.
// Technically 64 bits is enough, but 128 is better for future compatibility
// with https://github.com/rust-lang/rust/issues/56071
let mut next_negative_discriminant = Some(0);
// Sometimes we encounter explicit discriminants that we can't know the
// value of (e.g. a constant expression that requires evaluation). These
// could evaluate to zero or a negative number, but we can't assume that
// they do (no false positives allowed!). So we treat them like strictly-
// positive values that can't result in any zero variants, and track whether
// we've encountered any unknown discriminants.
let mut has_unknown_discriminants = false;
for (i, v) in enm.variants.iter().enumerate() {
match v.discriminant.as_ref() {
// Implicit discriminant
None => {
match next_negative_discriminant.as_mut() {
Some(0) => return Ok(i),
// n is nonzero so subtraction is always safe
Some(n) => *n -= 1,
None => (),
}
}
// Explicit positive discriminant
Some((_, Expr::Lit(ExprLit { lit: Lit::Int(int), .. }))) => {
match int.base10_parse::<u128>().ok() {
Some(0) => return Ok(i),
Some(_) => next_negative_discriminant = None,
None => {
// Numbers should never fail to parse, but just in case:
has_unknown_discriminants = true;
next_negative_discriminant = None;
}
}
}
// Explicit negative discriminant
Some((_, Expr::Unary(ExprUnary { op: UnOp::Neg(_), expr, .. }))) => match &**expr {
Expr::Lit(ExprLit { lit: Lit::Int(int), .. }) => {
match int.base10_parse::<u128>().ok() {
Some(0) => return Ok(i),
// x is nonzero so subtraction is always safe
Some(x) => next_negative_discriminant = Some(x - 1),
None => {
// Numbers should never fail to parse, but just in
// case:
has_unknown_discriminants = true;
next_negative_discriminant = None;
}
}
}
// Unknown negative discriminant (e.g. const repr)
_ => {
has_unknown_discriminants = true;
next_negative_discriminant = None;
}
},
// Unknown discriminant (e.g. const expr)
_ => {
has_unknown_discriminants = true;
next_negative_discriminant = None;
}
}
}
Err(has_unknown_discriminants)
}
pub(crate) fn derive_from_zeros(ctx: &Ctx, top_level: Trait) -> Result<TokenStream, Error> {
let try_from_bytes = derive_try_from_bytes(ctx, top_level)?;
let from_zeros = match &ctx.ast.data {
Data::Struct(strct) => derive_from_zeros_struct(ctx, strct),
Data::Enum(enm) => derive_from_zeros_enum(ctx, enm)?,
Data::Union(unn) => derive_from_zeros_union(ctx, unn),
};
Ok(IntoIterator::into_iter([try_from_bytes, from_zeros]).collect())
}
pub(crate) fn derive_from_bytes(ctx: &Ctx, top_level: Trait) -> Result<TokenStream, Error> {
let from_zeros = derive_from_zeros(ctx, top_level)?;
let from_bytes = match &ctx.ast.data {
Data::Struct(strct) => derive_from_bytes_struct(ctx, strct),
Data::Enum(enm) => derive_from_bytes_enum(ctx, enm)?,
Data::Union(unn) => derive_from_bytes_union(ctx, unn),
};
Ok(IntoIterator::into_iter([from_zeros, from_bytes]).collect())
}
fn derive_from_zeros_struct(ctx: &Ctx, strct: &DataStruct) -> TokenStream {
ImplBlockBuilder::new(ctx, strct, Trait::FromZeros, FieldBounds::ALL_SELF).build()
}
fn derive_from_zeros_enum(ctx: &Ctx, enm: &DataEnum) -> Result<TokenStream, Error> {
let repr = EnumRepr::from_attrs(&ctx.ast.attrs)?;
// We don't actually care what the repr is; we just care that it's one of
// the allowed ones.
match repr {
Repr::Compound(Spanned { t: CompoundRepr::C | CompoundRepr::Primitive(_), span: _ }, _) => {
}
Repr::Transparent(_) | Repr::Compound(Spanned { t: CompoundRepr::Rust, span: _ }, _) => {
return ctx.error_or_skip(
Error::new(
Span::call_site(),
"must have #[repr(C)] or #[repr(Int)] attribute in order to guarantee this type's memory layout",
),
);
}
}
let zero_variant = match find_zero_variant(enm) {
Ok(index) => enm.variants.iter().nth(index).unwrap(),
// Has unknown variants
Err(true) => {
return ctx.error_or_skip(Error::new_spanned(
&ctx.ast,
"FromZeros only supported on enums with a variant that has a discriminant of `0`\n\
help: This enum has discriminants which are not literal integers. One of those may \
define or imply which variant has a discriminant of zero. Use a literal integer to \
define or imply the variant with a discriminant of zero.",
));
}
// Does not have unknown variants
Err(false) => {
return ctx.error_or_skip(Error::new_spanned(
&ctx.ast,
"FromZeros only supported on enums with a variant that has a discriminant of `0`",
));
}
};
let zerocopy_crate = &ctx.zerocopy_crate;
let explicit_bounds = zero_variant
.fields
.iter()
.map(|field| {
let ty = &field.ty;
parse_quote! { #ty: #zerocopy_crate::FromZeros }
})
.collect::<Vec<WherePredicate>>();
Ok(ImplBlockBuilder::new(ctx, enm, Trait::FromZeros, FieldBounds::Explicit(explicit_bounds))
.build())
}
fn derive_from_zeros_union(ctx: &Ctx, unn: &DataUnion) -> TokenStream {
let field_type_trait_bounds = FieldBounds::All(&[TraitBound::Slf]);
ImplBlockBuilder::new(ctx, unn, Trait::FromZeros, field_type_trait_bounds).build()
}
fn derive_from_bytes_struct(ctx: &Ctx, strct: &DataStruct) -> TokenStream {
ImplBlockBuilder::new(ctx, strct, Trait::FromBytes, FieldBounds::ALL_SELF).build()
}
fn derive_from_bytes_enum(ctx: &Ctx, enm: &DataEnum) -> Result<TokenStream, Error> {
let repr = EnumRepr::from_attrs(&ctx.ast.attrs)?;
let variants_required = 1usize << enum_size_from_repr(&repr)?;
if enm.variants.len() != variants_required {
return ctx.error_or_skip(Error::new_spanned(
&ctx.ast,
format!(
"FromBytes only supported on {} enum with {} variants",
repr.repr_type_name(),
variants_required
),
));
}
Ok(ImplBlockBuilder::new(ctx, enm, Trait::FromBytes, FieldBounds::ALL_SELF).build())
}
fn derive_from_bytes_union(ctx: &Ctx, unn: &DataUnion) -> TokenStream {
let field_type_trait_bounds = FieldBounds::All(&[TraitBound::Slf]);
ImplBlockBuilder::new(ctx, unn, Trait::FromBytes, field_type_trait_bounds).build()
}

164
rust/zerocopy-derive/derive/into_bytes.rs Normal file
View File

@ -0,0 +1,164 @@
// SPDX-License-Identifier: (BSD-2-Clause OR Apache-2.0) OR MIT
use proc_macro2::{Span, TokenStream};
use quote::quote;
use syn::{Data, DataEnum, DataStruct, DataUnion, Error, Type};
use crate::{
repr::{EnumRepr, StructUnionRepr},
util::{
generate_tag_enum, Ctx, DataExt, FieldBounds, ImplBlockBuilder, PaddingCheck, Trait,
TraitBound,
},
};
pub(crate) fn derive_into_bytes(ctx: &Ctx, _top_level: Trait) -> Result<TokenStream, Error> {
match &ctx.ast.data {
Data::Struct(strct) => derive_into_bytes_struct(ctx, strct),
Data::Enum(enm) => derive_into_bytes_enum(ctx, enm),
Data::Union(unn) => derive_into_bytes_union(ctx, unn),
}
}
fn derive_into_bytes_struct(ctx: &Ctx, strct: &DataStruct) -> Result<TokenStream, Error> {
let repr = StructUnionRepr::from_attrs(&ctx.ast.attrs)?;
let is_transparent = repr.is_transparent();
let is_c = repr.is_c();
let is_packed_1 = repr.is_packed_1();
let num_fields = strct.fields().len();
let (padding_check, require_unaligned_fields) = if is_transparent || is_packed_1 {
// No padding check needed.
// - repr(transparent): The layout and ABI of the whole struct is the
// same as its only non-ZST field (meaning there's no padding outside
// of that field) and we require that field to be `IntoBytes` (meaning
// there's no padding in that field).
// - repr(packed): Any inter-field padding bytes are removed, meaning
// that any padding bytes would need to come from the fields, all of
// which we require to be `IntoBytes` (meaning they don't have any
// padding). Note that this holds regardless of other `repr`
// attributes, including `repr(Rust)`. [1]
//
// [1] Per https://doc.rust-lang.org/1.81.0/reference/type-layout.html#the-alignment-modifiers:
//
// An important consequence of these rules is that a type with
// `#[repr(packed(1))]`` (or `#[repr(packed)]``) will have no
// inter-field padding.
(None, false)
} else if is_c && !repr.is_align_gt_1() && num_fields <= 1 {
// No padding check needed. A repr(C) struct with zero or one field has
// no padding unless #[repr(align)] explicitly adds padding, which we
// check for in this branch's condition.
(None, false)
} else if ctx.ast.generics.params.is_empty() {
// Is the last field a syntactic slice, i.e., `[SomeType]`.
let is_syntactic_dst =
strct.fields().last().map(|(_, _, ty)| matches!(ty, Type::Slice(_))).unwrap_or(false);
// Since there are no generics, we can emit a padding check. All reprs
// guarantee that fields won't overlap [1], so the padding check is
// sound. This is more permissive than the next case, which requires
// that all field types implement `Unaligned`.
//
// [1] Per https://doc.rust-lang.org/1.81.0/reference/type-layout.html#the-rust-representation:
//
// The only data layout guarantees made by [`repr(Rust)`] are those
// required for soundness. They are:
// ...
// 2. The fields do not overlap.
// ...
if is_c && is_syntactic_dst {
(Some(PaddingCheck::ReprCStruct), false)
} else {
(Some(PaddingCheck::Struct), false)
}
} else if is_c && !repr.is_align_gt_1() {
// We can't use a padding check since there are generic type arguments.
// Instead, we require all field types to implement `Unaligned`. This
// ensures that the `repr(C)` layout algorithm will not insert any
// padding unless #[repr(align)] explicitly adds padding, which we check
// for in this branch's condition.
//
// FIXME(#10): Support type parameters for non-transparent, non-packed
// structs without requiring `Unaligned`.
(None, true)
} else {
return ctx.error_or_skip(Error::new(
Span::call_site(),
"must have a non-align #[repr(...)] attribute in order to guarantee this type's memory layout",
));
};
let field_bounds = if require_unaligned_fields {
FieldBounds::All(&[TraitBound::Slf, TraitBound::Other(Trait::Unaligned)])
} else {
FieldBounds::ALL_SELF
};
Ok(ImplBlockBuilder::new(ctx, strct, Trait::IntoBytes, field_bounds)
.padding_check(padding_check)
.build())
}
fn derive_into_bytes_enum(ctx: &Ctx, enm: &DataEnum) -> Result<TokenStream, Error> {
let repr = EnumRepr::from_attrs(&ctx.ast.attrs)?;
if !repr.is_c() && !repr.is_primitive() {
return ctx.error_or_skip(Error::new(
Span::call_site(),
"must have #[repr(C)] or #[repr(Int)] attribute in order to guarantee this type's memory layout",
));
}
let tag_type_definition = generate_tag_enum(ctx, &repr, enm);
Ok(ImplBlockBuilder::new(ctx, enm, Trait::IntoBytes, FieldBounds::ALL_SELF)
.padding_check(PaddingCheck::Enum { tag_type_definition })
.build())
}
fn derive_into_bytes_union(ctx: &Ctx, unn: &DataUnion) -> Result<TokenStream, Error> {
// See #1792 for more context.
//
// By checking for `zerocopy_derive_union_into_bytes` both here and in the
// generated code, we ensure that `--cfg zerocopy_derive_union_into_bytes`
// need only be passed *either* when compiling this crate *or* when
// compiling the user's crate. The former is preferable, but in some
// situations (such as when cross-compiling using `cargo build --target`),
// it doesn't get propagated to this crate's build by default.
let cfg_compile_error = if cfg!(zerocopy_derive_union_into_bytes) {
quote!()
} else {
let core = ctx.core_path();
let error_message = "requires --cfg zerocopy_derive_union_into_bytes;
please let us know you use this feature: https://github.com/google/zerocopy/discussions/1802";
quote!(
#[allow(unused_attributes, unexpected_cfgs)]
const _: () = {
#[cfg(not(zerocopy_derive_union_into_bytes))]
#core::compile_error!(#error_message);
};
)
};
// FIXME(#10): Support type parameters.
if !ctx.ast.generics.params.is_empty() {
return ctx.error_or_skip(Error::new(
Span::call_site(),
"unsupported on types with type parameters",
));
}
// Because we don't support generics, we don't need to worry about
// special-casing different reprs. So long as there is *some* repr which
// guarantees the layout, our `PaddingCheck::Union` guarantees that there is
// no padding.
let repr = StructUnionRepr::from_attrs(&ctx.ast.attrs)?;
if !repr.is_c() && !repr.is_transparent() && !repr.is_packed_1() {
return ctx.error_or_skip(Error::new(
Span::call_site(),
"must be #[repr(C)], #[repr(packed)], or #[repr(transparent)]",
));
}
let impl_block = ImplBlockBuilder::new(ctx, unn, Trait::IntoBytes, FieldBounds::ALL_SELF)
.padding_check(PaddingCheck::Union)
.build();
Ok(quote!(#cfg_compile_error #impl_block))
}

350
rust/zerocopy-derive/derive/known_layout.rs Normal file
View File

@ -0,0 +1,350 @@
// SPDX-License-Identifier: (BSD-2-Clause OR Apache-2.0) OR MIT
use proc_macro2::TokenStream;
use quote::quote;
use syn::{parse_quote, Data, Error, Type};
use crate::{
repr::StructUnionRepr,
util::{Ctx, DataExt, FieldBounds, ImplBlockBuilder, SelfBounds, Trait},
};
fn derive_known_layout_for_repr_c_struct<'a>(
ctx: &'a Ctx,
repr: &StructUnionRepr,
fields: &[(&'a syn::Visibility, TokenStream, &'a Type)],
) -> Option<(SelfBounds<'a>, TokenStream, Option<TokenStream>)> {
let (trailing_field, leading_fields) = fields.split_last()?;
let (_vis, trailing_field_name, trailing_field_ty) = trailing_field;
let leading_fields_tys = leading_fields.iter().map(|(_vis, _name, ty)| ty);
let core = ctx.core_path();
let repr_align = repr
.get_align()
.map(|align| {
let align = align.t.get();
quote!(#core::num::NonZeroUsize::new(#align as usize))
})
.unwrap_or_else(|| quote!(#core::option::Option::None));
let repr_packed = repr
.get_packed()
.map(|packed| {
let packed = packed.get();
quote!(#core::num::NonZeroUsize::new(#packed as usize))
})
.unwrap_or_else(|| quote!(#core::option::Option::None));
let zerocopy_crate = &ctx.zerocopy_crate;
let make_methods = |trailing_field_ty| {
quote! {
// SAFETY:
// - The returned pointer has the same address and provenance as
// `bytes`:
// - The recursive call to `raw_from_ptr_len` preserves both
// address and provenance.
// - The `as` cast preserves both address and provenance.
// - `NonNull::new_unchecked` preserves both address and
// provenance.
// - If `Self` is a slice DST, the returned pointer encodes
// `elems` elements in the trailing slice:
// - This is true of the recursive call to `raw_from_ptr_len`.
// - `trailing.as_ptr() as *mut Self` preserves trailing slice
// element count [1].
// - `NonNull::new_unchecked` preserves trailing slice element
// count.
//
// [1] Per https://doc.rust-lang.org/reference/expressions/operator-expr.html#pointer-to-pointer-cast:
//
// `*const T`` / `*mut T` can be cast to `*const U` / `*mut U`
// with the following behavior:
// ...
// - If `T` and `U` are both unsized, the pointer is also
// returned unchanged. In particular, the metadata is
// preserved exactly.
//
// For instance, a cast from `*const [T]` to `*const [U]`
// preserves the number of elements. ... The same holds
// for str and any compound type whose unsized tail is a
// slice type, such as struct `Foo(i32, [u8])` or
// `(u64, Foo)`.
#[inline(always)]
fn raw_from_ptr_len(
bytes: #core::ptr::NonNull<u8>,
meta: <Self as #zerocopy_crate::KnownLayout>::PointerMetadata,
) -> #core::ptr::NonNull<Self> {
let trailing = <#trailing_field_ty as #zerocopy_crate::KnownLayout>::raw_from_ptr_len(bytes, meta);
let slf = trailing.as_ptr() as *mut Self;
// SAFETY: Constructed from `trailing`, which is non-null.
unsafe { #core::ptr::NonNull::new_unchecked(slf) }
}
#[inline(always)]
fn pointer_to_metadata(ptr: *mut Self) -> <Self as #zerocopy_crate::KnownLayout>::PointerMetadata {
<#trailing_field_ty>::pointer_to_metadata(ptr as *mut _)
}
}
};
let inner_extras = {
let leading_fields_tys = leading_fields_tys.clone();
let methods = make_methods(*trailing_field_ty);
let (_, ty_generics, _) = ctx.ast.generics.split_for_impl();
quote!(
type PointerMetadata = <#trailing_field_ty as #zerocopy_crate::KnownLayout>::PointerMetadata;
type MaybeUninit = __ZerocopyKnownLayoutMaybeUninit #ty_generics;
// SAFETY: `LAYOUT` accurately describes the layout of `Self`.
// The documentation of `DstLayout::for_repr_c_struct` vows that
// invocations in this manner will accurately describe a type,
// so long as:
//
// - that type is `repr(C)`,
// - its fields are enumerated in the order they appear,
// - the presence of `repr_align` and `repr_packed` are
// correctly accounted for.
//
// We respect all three of these preconditions here. This
// expansion is only used if `is_repr_c_struct`, we enumerate
// the fields in order, and we extract the values of `align(N)`
// and `packed(N)`.
const LAYOUT: #zerocopy_crate::DstLayout = #zerocopy_crate::DstLayout::for_repr_c_struct(
#repr_align,
#repr_packed,
&[
#(#zerocopy_crate::DstLayout::for_type::<#leading_fields_tys>(),)*
<#trailing_field_ty as #zerocopy_crate::KnownLayout>::LAYOUT
],
);
#methods
)
};
let outer_extras = {
let ident = &ctx.ast.ident;
let vis = &ctx.ast.vis;
let params = &ctx.ast.generics.params;
let (impl_generics, ty_generics, where_clause) = ctx.ast.generics.split_for_impl();
let predicates = if let Some(where_clause) = where_clause {
where_clause.predicates.clone()
} else {
Default::default()
};
// Generate a valid ident for a type-level handle to a field of a
// given `name`.
let field_index = |name: &TokenStream| ident!(("__Zerocopy_Field_{}", name), ident.span());
let field_indices: Vec<_> =
fields.iter().map(|(_vis, name, _ty)| field_index(name)).collect();
// Define the collection of type-level field handles.
let field_defs = field_indices.iter().zip(fields).map(|(idx, (vis, _, _))| {
quote! {
#vis struct #idx;
}
});
let field_impls = field_indices.iter().zip(fields).map(|(idx, (_, _, ty))| quote! {
// SAFETY: `#ty` is the type of `#ident`'s field at `#idx`.
//
// We implement `Field` for each field of the struct to create a
// projection from the field index to its type. This allows us
// to refer to the field's type in a way that respects `Self`
// hygiene. If we just copy-pasted the tokens of `#ty`, we
// would not respect `Self` hygiene, as `Self` would refer to
// the helper struct we are generating, not the derive target
// type.
unsafe impl #impl_generics #zerocopy_crate::util::macro_util::Field<#idx> for #ident #ty_generics
where
#predicates
{
type Type = #ty;
}
});
let trailing_field_index = field_index(trailing_field_name);
let leading_field_indices =
leading_fields.iter().map(|(_vis, name, _ty)| field_index(name));
// We use `Field` to project the type of the trailing field. This is
// required to ensure that if the field type uses `Self`, it
// resolves to the derive target type, not the helper struct we are
// generating.
let trailing_field_ty = quote! {
<#ident #ty_generics as
#zerocopy_crate::util::macro_util::Field<#trailing_field_index>
>::Type
};
let methods = make_methods(&parse_quote! {
<#trailing_field_ty as #zerocopy_crate::KnownLayout>::MaybeUninit
});
let core = ctx.core_path();
quote! {
#(#field_defs)*
#(#field_impls)*
// SAFETY: This has the same layout as the derive target type,
// except that it admits uninit bytes. This is ensured by using
// the same repr as the target type, and by using field types
// which have the same layout as the target type's fields,
// except that they admit uninit bytes. We indirect through
// `Field` to ensure that occurrences of `Self` resolve to
// `#ty`, not `__ZerocopyKnownLayoutMaybeUninit` (see #2116).
#repr
#[doc(hidden)]
#vis struct __ZerocopyKnownLayoutMaybeUninit<#params> (
#(#core::mem::MaybeUninit<
<#ident #ty_generics as
#zerocopy_crate::util::macro_util::Field<#leading_field_indices>
>::Type
>,)*
// NOTE(#2302): We wrap in `ManuallyDrop` here in case the
// type we're operating on is both generic and
// `repr(packed)`. In that case, Rust needs to know that the
// type is *either* `Sized` or has a trivial `Drop`.
// `ManuallyDrop` has a trivial `Drop`, and so satisfies
// this requirement.
#core::mem::ManuallyDrop<
<#trailing_field_ty as #zerocopy_crate::KnownLayout>::MaybeUninit
>
)
where
#trailing_field_ty: #zerocopy_crate::KnownLayout,
#predicates;
// SAFETY: We largely defer to the `KnownLayout` implementation
// on the derive target type (both by using the same tokens, and
// by deferring to impl via type-level indirection). This is
// sound, since `__ZerocopyKnownLayoutMaybeUninit` is guaranteed
// to have the same layout as the derive target type, except
// that `__ZerocopyKnownLayoutMaybeUninit` admits uninit bytes.
unsafe impl #impl_generics #zerocopy_crate::KnownLayout for __ZerocopyKnownLayoutMaybeUninit #ty_generics
where
#trailing_field_ty: #zerocopy_crate::KnownLayout,
#predicates
{
fn only_derive_is_allowed_to_implement_this_trait() {}
type PointerMetadata = <#ident #ty_generics as #zerocopy_crate::KnownLayout>::PointerMetadata;
type MaybeUninit = Self;
const LAYOUT: #zerocopy_crate::DstLayout = <#ident #ty_generics as #zerocopy_crate::KnownLayout>::LAYOUT;
#methods
}
}
};
Some((SelfBounds::None, inner_extras, Some(outer_extras)))
}
pub(crate) fn derive(ctx: &Ctx, _top_level: Trait) -> Result<TokenStream, Error> {
// If this is a `repr(C)` struct, then `c_struct_repr` contains the entire
// `repr` attribute.
let c_struct_repr = match &ctx.ast.data {
Data::Struct(..) => {
let repr = StructUnionRepr::from_attrs(&ctx.ast.attrs)?;
if repr.is_c() {
Some(repr)
} else {
None
}
}
Data::Enum(..) | Data::Union(..) => None,
};
let fields = ctx.ast.data.fields();
let (self_bounds, inner_extras, outer_extras) = c_struct_repr
.as_ref()
.and_then(|repr| {
derive_known_layout_for_repr_c_struct(ctx, repr, &fields)
})
.unwrap_or_else(|| {
let zerocopy_crate = &ctx.zerocopy_crate;
let core = ctx.core_path();
// For enums, unions, and non-`repr(C)` structs, we require that
// `Self` is sized, and as a result don't need to reason about the
// internals of the type.
(
SelfBounds::SIZED,
quote!(
type PointerMetadata = ();
type MaybeUninit =
#core::mem::MaybeUninit<Self>;
// SAFETY: `LAYOUT` is guaranteed to accurately describe the
// layout of `Self`, because that is the documented safety
// contract of `DstLayout::for_type`.
const LAYOUT: #zerocopy_crate::DstLayout = #zerocopy_crate::DstLayout::for_type::<Self>();
// SAFETY: `.cast` preserves address and provenance.
//
// FIXME(#429): Add documentation to `.cast` that promises that
// it preserves provenance.
#[inline(always)]
fn raw_from_ptr_len(bytes: #core::ptr::NonNull<u8>, _meta: ()) -> #core::ptr::NonNull<Self> {
bytes.cast::<Self>()
}
#[inline(always)]
fn pointer_to_metadata(_ptr: *mut Self) -> () {}
),
None,
)
});
Ok(match &ctx.ast.data {
Data::Struct(strct) => {
let require_trait_bound_on_field_types =
if matches!(self_bounds, SelfBounds::All(&[Trait::Sized])) {
FieldBounds::None
} else {
FieldBounds::TRAILING_SELF
};
// A bound on the trailing field is required, since structs are
// unsized if their trailing field is unsized. Reflecting the layout
// of an usized trailing field requires that the field is
// `KnownLayout`.
ImplBlockBuilder::new(
ctx,
strct,
Trait::KnownLayout,
require_trait_bound_on_field_types,
)
.self_type_trait_bounds(self_bounds)
.inner_extras(inner_extras)
.outer_extras(outer_extras)
.build()
}
Data::Enum(enm) => {
// A bound on the trailing field is not required, since enums cannot
// currently be unsized.
ImplBlockBuilder::new(ctx, enm, Trait::KnownLayout, FieldBounds::None)
.self_type_trait_bounds(SelfBounds::SIZED)
.inner_extras(inner_extras)
.outer_extras(outer_extras)
.build()
}
Data::Union(unn) => {
// A bound on the trailing field is not required, since unions
// cannot currently be unsized.
ImplBlockBuilder::new(ctx, unn, Trait::KnownLayout, FieldBounds::None)
.self_type_trait_bounds(SelfBounds::SIZED)
.inner_extras(inner_extras)
.outer_extras(outer_extras)
.build()
}
})
}

132
rust/zerocopy-derive/derive/mod.rs Normal file
View File

@ -0,0 +1,132 @@
// SPDX-License-Identifier: (BSD-2-Clause OR Apache-2.0) OR MIT
pub mod from_bytes;
pub mod into_bytes;
pub mod known_layout;
pub mod try_from_bytes;
pub mod unaligned;
use proc_macro2::{Span, TokenStream};
use quote::quote;
use syn::{Data, Error};
use crate::{
repr::StructUnionRepr,
util::{Ctx, DataExt, FieldBounds, ImplBlockBuilder, Trait},
};
pub(crate) fn derive_immutable(ctx: &Ctx, _top_level: Trait) -> TokenStream {
match &ctx.ast.data {
Data::Struct(strct) => {
ImplBlockBuilder::new(ctx, strct, Trait::Immutable, FieldBounds::ALL_SELF).build()
}
Data::Enum(enm) => {
ImplBlockBuilder::new(ctx, enm, Trait::Immutable, FieldBounds::ALL_SELF).build()
}
Data::Union(unn) => {
ImplBlockBuilder::new(ctx, unn, Trait::Immutable, FieldBounds::ALL_SELF).build()
}
}
}
pub(crate) fn derive_hash(ctx: &Ctx, _top_level: Trait) -> Result<TokenStream, Error> {
// This doesn't delegate to `impl_block` because `impl_block` assumes it is
// deriving a `zerocopy`-defined trait, and these trait impls share a common
// shape that `Hash` does not. In particular, `zerocopy` traits contain a
// method that only `zerocopy_derive` macros are supposed to implement, and
// `impl_block` generating this trait method is incompatible with `Hash`.
let type_ident = &ctx.ast.ident;
let (impl_generics, ty_generics, where_clause) = ctx.ast.generics.split_for_impl();
let where_predicates = where_clause.map(|clause| &clause.predicates);
let zerocopy_crate = &ctx.zerocopy_crate;
let core = ctx.core_path();
Ok(quote! {
impl #impl_generics #core::hash::Hash for #type_ident #ty_generics
where
Self: #zerocopy_crate::IntoBytes + #zerocopy_crate::Immutable,
#where_predicates
{
fn hash<H: #core::hash::Hasher>(&self, state: &mut H) {
#core::hash::Hasher::write(state, #zerocopy_crate::IntoBytes::as_bytes(self))
}
fn hash_slice<H: #core::hash::Hasher>(data: &[Self], state: &mut H) {
#core::hash::Hasher::write(state, #zerocopy_crate::IntoBytes::as_bytes(data))
}
}
})
}
pub(crate) fn derive_eq(ctx: &Ctx, _top_level: Trait) -> Result<TokenStream, Error> {
// This doesn't delegate to `impl_block` because `impl_block` assumes it is
// deriving a `zerocopy`-defined trait, and these trait impls share a common
// shape that `Eq` does not. In particular, `zerocopy` traits contain a
// method that only `zerocopy_derive` macros are supposed to implement, and
// `impl_block` generating this trait method is incompatible with `Eq`.
let type_ident = &ctx.ast.ident;
let (impl_generics, ty_generics, where_clause) = ctx.ast.generics.split_for_impl();
let where_predicates = where_clause.map(|clause| &clause.predicates);
let zerocopy_crate = &ctx.zerocopy_crate;
let core = ctx.core_path();
Ok(quote! {
impl #impl_generics #core::cmp::PartialEq for #type_ident #ty_generics
where
Self: #zerocopy_crate::IntoBytes + #zerocopy_crate::Immutable,
#where_predicates
{
fn eq(&self, other: &Self) -> bool {
#core::cmp::PartialEq::eq(
#zerocopy_crate::IntoBytes::as_bytes(self),
#zerocopy_crate::IntoBytes::as_bytes(other),
)
}
}
impl #impl_generics #core::cmp::Eq for #type_ident #ty_generics
where
Self: #zerocopy_crate::IntoBytes + #zerocopy_crate::Immutable,
#where_predicates
{
}
})
}
pub(crate) fn derive_split_at(ctx: &Ctx, _top_level: Trait) -> Result<TokenStream, Error> {
let repr = StructUnionRepr::from_attrs(&ctx.ast.attrs)?;
match &ctx.ast.data {
Data::Struct(_) => {}
Data::Enum(_) | Data::Union(_) => {
return Err(Error::new(Span::call_site(), "can only be applied to structs"));
}
};
if repr.get_packed().is_some() {
return Err(Error::new(Span::call_site(), "must not have #[repr(packed)] attribute"));
}
if !(repr.is_c() || repr.is_transparent()) {
return Err(Error::new(
Span::call_site(),
"must have #[repr(C)] or #[repr(transparent)] in order to guarantee this type's layout is splitable",
));
}
let fields = ctx.ast.data.fields();
let trailing_field = if let Some(((_, _, trailing_field), _)) = fields.split_last() {
trailing_field
} else {
return Err(Error::new(Span::call_site(), "must at least one field"));
};
let zerocopy_crate = &ctx.zerocopy_crate;
// SAFETY: `#ty`, per the above checks, is `repr(C)` or `repr(transparent)`
// and is not packed; its trailing field is guaranteed to be well-aligned
// for its type. By invariant on `FieldBounds::TRAILING_SELF`, the trailing
// slice of the trailing field is also well-aligned for its type.
Ok(ImplBlockBuilder::new(ctx, &ctx.ast.data, Trait::SplitAt, FieldBounds::TRAILING_SELF)
.inner_extras(quote! {
type Elem = <#trailing_field as #zerocopy_crate::SplitAt>::Elem;
})
.build())
}

765
rust/zerocopy-derive/derive/try_from_bytes.rs Normal file
View File

@ -0,0 +1,765 @@
// SPDX-License-Identifier: (BSD-2-Clause OR Apache-2.0) OR MIT
use proc_macro2::TokenStream;
use quote::quote;
use syn::{
parse_quote, spanned::Spanned as _, Data, DataEnum, DataStruct, DataUnion, DeriveInput, Error,
Expr, Fields, Ident, Index, Type,
};
use crate::{
repr::{EnumRepr, StructUnionRepr},
util::{
const_block, enum_size_from_repr, generate_tag_enum, Ctx, DataExt, FieldBounds,
ImplBlockBuilder, Trait, TraitBound,
},
};
fn tag_ident(variant_ident: &Ident) -> Ident {
ident!(("___ZEROCOPY_TAG_{}", variant_ident), variant_ident.span())
}
/// Generates a constant for the tag associated with each variant of the enum.
/// When we match on the enum's tag, each arm matches one of these constants. We
/// have to use constants here because:
///
/// - The type that we're matching on is not the type of the tag, it's an
/// integer of the same size as the tag type and with the same bit patterns.
/// - We can't read the enum tag as an enum because the bytes may not represent
/// a valid variant.
/// - Patterns do not currently support const expressions, so we have to assign
/// these constants to names rather than use them inline in the `match`
/// statement.
fn generate_tag_consts(data: &DataEnum) -> TokenStream {
let tags = data.variants.iter().map(|v| {
let variant_ident = &v.ident;
let tag_ident = tag_ident(variant_ident);
quote! {
// This casts the enum variant to its discriminant, and then
// converts the discriminant to the target integral type via a
// numeric cast [1].
//
// Because these are the same size, this is defined to be a no-op
// and therefore is a lossless conversion [2].
//
// [1] Per https://doc.rust-lang.org/1.81.0/reference/expressions/operator-expr.html#enum-cast:
//
// Casts an enum to its discriminant.
//
// [2] Per https://doc.rust-lang.org/1.81.0/reference/expressions/operator-expr.html#numeric-cast:
//
// Casting between two integers of the same size (e.g. i32 -> u32)
// is a no-op.
const #tag_ident: ___ZerocopyTagPrimitive =
___ZerocopyTag::#variant_ident as ___ZerocopyTagPrimitive;
}
});
quote! {
#(#tags)*
}
}
fn variant_struct_ident(variant_ident: &Ident) -> Ident {
ident!(("___ZerocopyVariantStruct_{}", variant_ident), variant_ident.span())
}
/// Generates variant structs for the given enum variant.
///
/// These are structs associated with each variant of an enum. They are
/// `repr(C)` tuple structs with the same fields as the variant after a
/// `MaybeUninit<___ZerocopyInnerTag>`.
///
/// In order to unify the generated types for `repr(C)` and `repr(int)` enums,
/// we use a "fused" representation with fields for both an inner tag and an
/// outer tag. Depending on the repr, we will set one of these tags to the tag
/// type and the other to `()`. This lets us generate the same code but put the
/// tags in different locations.
fn generate_variant_structs(ctx: &Ctx, data: &DataEnum) -> TokenStream {
let (impl_generics, ty_generics, where_clause) = ctx.ast.generics.split_for_impl();
let enum_name = &ctx.ast.ident;
// All variant structs have a `PhantomData<MyEnum<...>>` field because we
// don't know which generic parameters each variant will use, and unused
// generic parameters are a compile error.
let core = ctx.core_path();
let phantom_ty = quote! {
#core::marker::PhantomData<#enum_name #ty_generics>
};
let variant_structs = data.variants.iter().filter_map(|variant| {
// We don't generate variant structs for unit variants because we only
// need to check the tag. This helps cut down our generated code a bit.
if matches!(variant.fields, Fields::Unit) {
return None;
}
let variant_struct_ident = variant_struct_ident(&variant.ident);
let field_types = variant.fields.iter().map(|f| &f.ty);
let variant_struct = parse_quote! {
#[repr(C)]
struct #variant_struct_ident #impl_generics (
#core::mem::MaybeUninit<___ZerocopyInnerTag>,
#(#field_types,)*
#phantom_ty,
) #where_clause;
};
// We do this rather than emitting `#[derive(::zerocopy::TryFromBytes)]`
// because that is not hygienic, and this is also more performant.
let try_from_bytes_impl =
derive_try_from_bytes(&ctx.with_input(&variant_struct), Trait::TryFromBytes)
.expect("derive_try_from_bytes should not fail on synthesized type");
Some(quote! {
#variant_struct
#try_from_bytes_impl
})
});
quote! {
#(#variant_structs)*
}
}
fn variants_union_field_ident(ident: &Ident) -> Ident {
// Field names are prefixed with `__field_` to prevent name collision
// with the `__nonempty` field.
ident!(("__field_{}", ident), ident.span())
}
fn generate_variants_union(ctx: &Ctx, data: &DataEnum) -> TokenStream {
let generics = &ctx.ast.generics;
let (_, ty_generics, _) = generics.split_for_impl();
let fields = data.variants.iter().filter_map(|variant| {
// We don't generate variant structs for unit variants because we only
// need to check the tag. This helps cut down our generated code a bit.
if matches!(variant.fields, Fields::Unit) {
return None;
}
let field_name = variants_union_field_ident(&variant.ident);
let variant_struct_ident = variant_struct_ident(&variant.ident);
let core = ctx.core_path();
Some(quote! {
#field_name: #core::mem::ManuallyDrop<#variant_struct_ident #ty_generics>,
})
});
let variants_union = parse_quote! {
#[repr(C)]
union ___ZerocopyVariants #generics {
#(#fields)*
// Enums can have variants with no fields, but unions must
// have at least one field. So we just add a trailing unit
// to ensure that this union always has at least one field.
// Because this union is `repr(C)`, this unit type does not
// affect the layout.
__nonempty: (),
}
};
let has_field =
derive_has_field_struct_union(&ctx.with_input(&variants_union), &variants_union.data);
quote! {
#variants_union
#has_field
}
}
/// Generates an implementation of `is_bit_valid` for an arbitrary enum.
///
/// The general process is:
///
/// 1. Generate a tag enum. This is an enum with the same repr, variants, and
/// corresponding discriminants as the original enum, but without any fields
/// on the variants. This gives us access to an enum where the variants have
/// the same discriminants as the one we're writing `is_bit_valid` for.
/// 2. Make constants from the variants of the tag enum. We need these because
/// we can't put const exprs in match arms.
/// 3. Generate variant structs. These are structs which have the same fields as
/// each variant of the enum, and are `#[repr(C)]` with an optional "inner
/// tag".
/// 4. Generate a variants union, with one field for each variant struct type.
/// 5. And finally, our raw enum is a `#[repr(C)]` struct of an "outer tag" and
/// the variants union.
///
/// See these reference links for fully-worked example decompositions.
///
/// - `repr(C)`: <https://doc.rust-lang.org/reference/type-layout.html#reprc-enums-with-fields>
/// - `repr(int)`: <https://doc.rust-lang.org/reference/type-layout.html#primitive-representation-of-enums-with-fields>
/// - `repr(C, int)`: <https://doc.rust-lang.org/reference/type-layout.html#combining-primitive-representations-of-enums-with-fields-and-reprc>
pub(crate) fn derive_is_bit_valid(
ctx: &Ctx,
data: &DataEnum,
repr: &EnumRepr,
) -> Result<TokenStream, Error> {
let trait_path = Trait::TryFromBytes.crate_path(ctx);
let tag_enum = generate_tag_enum(ctx, repr, data);
let tag_consts = generate_tag_consts(data);
let (outer_tag_type, inner_tag_type) = if repr.is_c() {
(quote! { ___ZerocopyTag }, quote! { () })
} else if repr.is_primitive() {
(quote! { () }, quote! { ___ZerocopyTag })
} else {
return Err(Error::new(
ctx.ast.span(),
"must have #[repr(C)] or #[repr(Int)] attribute in order to guarantee this type's memory layout",
));
};
let variant_structs = generate_variant_structs(ctx, data);
let variants_union = generate_variants_union(ctx, data);
let (impl_generics, ty_generics, where_clause) = ctx.ast.generics.split_for_impl();
let zerocopy_crate = &ctx.zerocopy_crate;
let has_tag = ImplBlockBuilder::new(ctx, data, Trait::HasTag, FieldBounds::None)
.inner_extras(quote! {
type Tag = ___ZerocopyTag;
type ProjectToTag = #zerocopy_crate::pointer::cast::CastSized;
})
.build();
let has_fields = data.variants().into_iter().flat_map(|(variant, fields)| {
let variant_ident = &variant.unwrap().ident;
let variants_union_field_ident = variants_union_field_ident(variant_ident);
let field: Box<syn::Type> = parse_quote!(());
fields.into_iter().enumerate().map(move |(idx, (vis, ident, ty))| {
// Rust does not presently support explicit visibility modifiers on
// enum fields, but we guard against the possibility to ensure this
// derive remains sound.
assert!(matches!(vis, syn::Visibility::Inherited));
let variant_struct_field_index = Index::from(idx + 1);
let (_, ty_generics, _) = ctx.ast.generics.split_for_impl();
let has_field_trait = Trait::HasField {
variant_id: parse_quote!({ #zerocopy_crate::ident_id!(#variant_ident) }),
// Since Rust does not presently support explicit visibility
// modifiers on enum fields, any public type is suitable here;
// we use `()`.
field: field.clone(),
field_id: parse_quote!({ #zerocopy_crate::ident_id!(#ident) }),
};
let has_field_path = has_field_trait.crate_path(ctx);
let has_field = ImplBlockBuilder::new(
ctx,
data,
has_field_trait,
FieldBounds::None,
)
.inner_extras(quote! {
type Type = #ty;
#[inline(always)]
fn project(slf: #zerocopy_crate::pointer::PtrInner<'_, Self>) -> *mut <Self as #has_field_path>::Type {
use #zerocopy_crate::pointer::cast::{CastSized, Projection};
slf.project::<___ZerocopyRawEnum #ty_generics, CastSized>()
.project::<_, Projection<_, { #zerocopy_crate::STRUCT_VARIANT_ID }, { #zerocopy_crate::ident_id!(variants) }>>()
.project::<_, Projection<_, { #zerocopy_crate::REPR_C_UNION_VARIANT_ID }, { #zerocopy_crate::ident_id!(#variants_union_field_ident) }>>()
.project::<_, Projection<_, { #zerocopy_crate::STRUCT_VARIANT_ID }, { #zerocopy_crate::ident_id!(value) }>>()
.project::<_, Projection<_, { #zerocopy_crate::STRUCT_VARIANT_ID }, { #zerocopy_crate::ident_id!(#variant_struct_field_index) }>>()
.as_ptr()
}
})
.build();
let project = ImplBlockBuilder::new(
ctx,
data,
Trait::ProjectField {
variant_id: parse_quote!({ #zerocopy_crate::ident_id!(#variant_ident) }),
// Since Rust does not presently support explicit visibility
// modifiers on enum fields, any public type is suitable
// here; we use `()`.
field: field.clone(),
field_id: parse_quote!({ #zerocopy_crate::ident_id!(#ident) }),
invariants: parse_quote!((Aliasing, Alignment, #zerocopy_crate::invariant::Initialized)),
},
FieldBounds::None,
)
.param_extras(vec![
parse_quote!(Aliasing: #zerocopy_crate::invariant::Aliasing),
parse_quote!(Alignment: #zerocopy_crate::invariant::Alignment),
])
.inner_extras(quote! {
type Error = #zerocopy_crate::util::macro_util::core_reexport::convert::Infallible;
type Invariants = (Aliasing, Alignment, #zerocopy_crate::invariant::Initialized);
})
.build();
quote! {
#has_field
#project
}
})
});
let core = ctx.core_path();
let match_arms = data.variants.iter().map(|variant| {
let tag_ident = tag_ident(&variant.ident);
let variant_struct_ident = variant_struct_ident(&variant.ident);
let variants_union_field_ident = variants_union_field_ident(&variant.ident);
if matches!(variant.fields, Fields::Unit) {
// Unit variants don't need any further validation beyond checking
// the tag.
quote! {
#tag_ident => true
}
} else {
quote! {
#tag_ident => {
// SAFETY: Since we know that the tag is `#tag_ident`, we
// know that no other `&`s exist which refer to this enum
// as any other variant.
let variant_md = variants.cast::<
_,
#zerocopy_crate::pointer::cast::Projection<
// #zerocopy_crate::ReadOnly<_>,
_,
{ #zerocopy_crate::REPR_C_UNION_VARIANT_ID },
{ #zerocopy_crate::ident_id!(#variants_union_field_ident) }
>,
_
>();
let variant = variant_md.cast::<
#zerocopy_crate::ReadOnly<#variant_struct_ident #ty_generics>,
#zerocopy_crate::pointer::cast::CastSized,
(#zerocopy_crate::pointer::BecauseRead, _)
>();
<
#variant_struct_ident #ty_generics as #trait_path
>::is_bit_valid(variant)
}
}
}
});
let generics = &ctx.ast.generics;
let raw_enum: DeriveInput = parse_quote! {
#[repr(C)]
struct ___ZerocopyRawEnum #generics {
tag: ___ZerocopyOuterTag,
variants: ___ZerocopyVariants #ty_generics,
}
};
let self_ident = &ctx.ast.ident;
let invariants_eq_impl = quote! {
// SAFETY: `___ZerocopyRawEnum` is designed to have the same layout,
// validity, and invariants as `Self`.
unsafe impl #impl_generics #zerocopy_crate::pointer::InvariantsEq<___ZerocopyRawEnum #ty_generics> for #self_ident #ty_generics #where_clause {}
};
let raw_enum_projections =
derive_has_field_struct_union(&ctx.with_input(&raw_enum), &raw_enum.data);
let raw_enum = quote! {
#raw_enum
#invariants_eq_impl
#raw_enum_projections
};
Ok(quote! {
// SAFETY: We use `is_bit_valid` to validate that the bit pattern of the
// enum's tag corresponds to one of the enum's discriminants. Then, we
// check the bit validity of each field of the corresponding variant.
// Thus, this is a sound implementation of `is_bit_valid`.
#[inline]
fn is_bit_valid<___ZcAlignment>(
mut candidate: #zerocopy_crate::Maybe<'_, Self, ___ZcAlignment>,
) -> #core::primitive::bool
where
___ZcAlignment: #zerocopy_crate::invariant::Alignment,
{
#tag_enum
type ___ZerocopyTagPrimitive = #zerocopy_crate::util::macro_util::SizeToTag<
{ #core::mem::size_of::<___ZerocopyTag>() },
>;
#tag_consts
type ___ZerocopyOuterTag = #outer_tag_type;
type ___ZerocopyInnerTag = #inner_tag_type;
#variant_structs
#variants_union
#raw_enum
#has_tag
#(#has_fields)*
let tag = {
// SAFETY:
// - The provided cast addresses a subset of the bytes addressed
// by `candidate` because it addresses the starting tag of the
// enum.
// - Because the pointer is cast from `candidate`, it has the
// same provenance as it.
// - There are no `UnsafeCell`s in the tag because it is a
// primitive integer.
// - `tag_ptr` is casted from `candidate`, whose referent is
// `Initialized`. Since we have not written uninitialized
// bytes into the referent, `tag_ptr` is also `Initialized`.
//
// FIXME(#2874): Revise this to a `cast` once `candidate`
// references a `ReadOnly<Self>`.
let tag_ptr = unsafe {
candidate.reborrow().project_transmute_unchecked::<
_,
#zerocopy_crate::invariant::Initialized,
#zerocopy_crate::pointer::cast::CastSized
>()
};
tag_ptr.recall_validity::<_, (_, (_, _))>().read::<#zerocopy_crate::BecauseImmutable>()
};
let mut raw_enum = candidate.cast::<
#zerocopy_crate::ReadOnly<___ZerocopyRawEnum #ty_generics>,
#zerocopy_crate::pointer::cast::CastSized,
(#zerocopy_crate::pointer::BecauseRead, _)
>();
let variants = #zerocopy_crate::into_inner!(raw_enum.project::<
_,
{ #zerocopy_crate::STRUCT_VARIANT_ID },
{ #zerocopy_crate::ident_id!(variants) }
>());
match tag {
#(#match_arms,)*
_ => false,
}
}
})
}
pub(crate) fn derive_try_from_bytes(ctx: &Ctx, top_level: Trait) -> Result<TokenStream, Error> {
match &ctx.ast.data {
Data::Struct(strct) => derive_try_from_bytes_struct(ctx, strct, top_level),
Data::Enum(enm) => derive_try_from_bytes_enum(ctx, enm, top_level),
Data::Union(unn) => Ok(derive_try_from_bytes_union(ctx, unn, top_level)),
}
}
fn derive_has_field_struct_union(ctx: &Ctx, data: &dyn DataExt) -> TokenStream {
let fields = ctx.ast.data.fields();
if fields.is_empty() {
return quote! {};
}
let field_tokens = fields.iter().map(|(vis, ident, _)| {
let ident = ident!(("__z{}", ident), ident.span());
quote!(
#vis enum #ident {}
)
});
let zerocopy_crate = &ctx.zerocopy_crate;
let variant_id: Box<Expr> = match &ctx.ast.data {
Data::Struct(_) => parse_quote!({ #zerocopy_crate::STRUCT_VARIANT_ID }),
Data::Union(_) => {
let is_repr_c = StructUnionRepr::from_attrs(&ctx.ast.attrs)
.map(|repr| repr.is_c())
.unwrap_or(false);
if is_repr_c {
parse_quote!({ #zerocopy_crate::REPR_C_UNION_VARIANT_ID })
} else {
parse_quote!({ #zerocopy_crate::UNION_VARIANT_ID })
}
}
_ => unreachable!(),
};
let core = ctx.core_path();
let has_tag = ImplBlockBuilder::new(ctx, data, Trait::HasTag, FieldBounds::None)
.inner_extras(quote! {
type Tag = ();
type ProjectToTag = #zerocopy_crate::pointer::cast::CastToUnit;
})
.build();
let has_fields = fields.iter().map(move |(_, ident, ty)| {
let field_token = ident!(("__z{}", ident), ident.span());
let field: Box<Type> = parse_quote!(#field_token);
let field_id: Box<Expr> = parse_quote!({ #zerocopy_crate::ident_id!(#ident) });
let has_field_trait = Trait::HasField {
variant_id: variant_id.clone(),
field: field.clone(),
field_id: field_id.clone(),
};
let has_field_path = has_field_trait.crate_path(ctx);
ImplBlockBuilder::new(
ctx,
data,
has_field_trait,
FieldBounds::None,
)
.inner_extras(quote! {
type Type = #ty;
#[inline(always)]
fn project(slf: #zerocopy_crate::pointer::PtrInner<'_, Self>) -> *mut <Self as #has_field_path>::Type {
let slf = slf.as_ptr();
// SAFETY: By invariant on `PtrInner`, `slf` is a non-null
// pointer whose referent is zero-sized or lives in a valid
// allocation. Since `#ident` is a struct or union field of
// `Self`, this projection preserves or shrinks the referent
// size, and so the resulting referent also fits in the same
// allocation.
unsafe { #core::ptr::addr_of_mut!((*slf).#ident) }
}
}).outer_extras(if matches!(&ctx.ast.data, Data::Struct(..)) {
let fields_preserve_alignment = StructUnionRepr::from_attrs(&ctx.ast.attrs)
.map(|repr| repr.get_packed().is_none())
.unwrap();
let alignment = if fields_preserve_alignment {
quote! { Alignment }
} else {
quote! { #zerocopy_crate::invariant::Unaligned }
};
// SAFETY: See comments on items.
ImplBlockBuilder::new(
ctx,
data,
Trait::ProjectField {
variant_id: variant_id.clone(),
field: field.clone(),
field_id: field_id.clone(),
invariants: parse_quote!((Aliasing, Alignment, #zerocopy_crate::invariant::Initialized)),
},
FieldBounds::None,
)
.param_extras(vec![
parse_quote!(Aliasing: #zerocopy_crate::invariant::Aliasing),
parse_quote!(Alignment: #zerocopy_crate::invariant::Alignment),
])
.inner_extras(quote! {
// SAFETY: Projection into structs is always infallible.
type Error = #zerocopy_crate::util::macro_util::core_reexport::convert::Infallible;
// SAFETY: The alignment of the projected `Ptr` is `Unaligned`
// if the structure is packed; otherwise inherited from the
// outer `Ptr`. If the validity of the outer pointer is
// `Initialized`, so too is the validity of its fields.
type Invariants = (Aliasing, #alignment, #zerocopy_crate::invariant::Initialized);
})
.build()
} else {
quote! {}
})
.build()
});
const_block(field_tokens.into_iter().chain(Some(has_tag)).chain(has_fields).map(Some))
}
fn derive_try_from_bytes_struct(
ctx: &Ctx,
strct: &DataStruct,
top_level: Trait,
) -> Result<TokenStream, Error> {
let extras = try_gen_trivial_is_bit_valid(ctx, top_level).unwrap_or_else(|| {
let zerocopy_crate = &ctx.zerocopy_crate;
let fields = strct.fields();
let field_names = fields.iter().map(|(_vis, name, _ty)| name);
let field_tys = fields.iter().map(|(_vis, _name, ty)| ty);
let core = ctx.core_path();
quote!(
// SAFETY: We use `is_bit_valid` to validate that each field is
// bit-valid, and only return `true` if all of them are. The bit
// validity of a struct is just the composition of the bit
// validities of its fields, so this is a sound implementation
// of `is_bit_valid`.
#[inline]
fn is_bit_valid<___ZcAlignment>(
mut candidate: #zerocopy_crate::Maybe<'_, Self, ___ZcAlignment>,
) -> #core::primitive::bool
where
___ZcAlignment: #zerocopy_crate::invariant::Alignment,
{
true #(&& {
let field_candidate = #zerocopy_crate::into_inner!(candidate.reborrow().project::<
_,
{ #zerocopy_crate::STRUCT_VARIANT_ID },
{ #zerocopy_crate::ident_id!(#field_names) }
>());
<#field_tys as #zerocopy_crate::TryFromBytes>::is_bit_valid(field_candidate)
})*
}
)
});
Ok(ImplBlockBuilder::new(ctx, strct, Trait::TryFromBytes, FieldBounds::ALL_SELF)
.inner_extras(extras)
.outer_extras(derive_has_field_struct_union(ctx, strct))
.build())
}
fn derive_try_from_bytes_union(ctx: &Ctx, unn: &DataUnion, top_level: Trait) -> TokenStream {
let field_type_trait_bounds = FieldBounds::All(&[TraitBound::Slf]);
let zerocopy_crate = &ctx.zerocopy_crate;
let variant_id: Box<Expr> = {
let is_repr_c =
StructUnionRepr::from_attrs(&ctx.ast.attrs).map(|repr| repr.is_c()).unwrap_or(false);
if is_repr_c {
parse_quote!({ #zerocopy_crate::REPR_C_UNION_VARIANT_ID })
} else {
parse_quote!({ #zerocopy_crate::UNION_VARIANT_ID })
}
};
let extras = try_gen_trivial_is_bit_valid(ctx, top_level).unwrap_or_else(|| {
let fields = unn.fields();
let field_names = fields.iter().map(|(_vis, name, _ty)| name);
let field_tys = fields.iter().map(|(_vis, _name, ty)| ty);
let core = ctx.core_path();
quote!(
// SAFETY: We use `is_bit_valid` to validate that any field is
// bit-valid; we only return `true` if at least one of them is.
// The bit validity of a union is not yet well defined in Rust,
// but it is guaranteed to be no more strict than this
// definition. See #696 for a more in-depth discussion.
#[inline]
fn is_bit_valid<___ZcAlignment>(
mut candidate: #zerocopy_crate::Maybe<'_, Self, ___ZcAlignment>,
) -> #core::primitive::bool
where
___ZcAlignment: #zerocopy_crate::invariant::Alignment,
{
false #(|| {
// SAFETY:
// - Since `ReadOnly<Self>: Immutable` unconditionally,
// neither `*slf` nor the returned pointer's referent
// permit interior mutation.
// - Both source and destination validity are
// `Initialized`, which is always a sound
// transmutation.
let field_candidate = unsafe {
candidate.reborrow().project_transmute_unchecked::<
_,
_,
#zerocopy_crate::pointer::cast::Projection<
_,
#variant_id,
{ #zerocopy_crate::ident_id!(#field_names) }
>
>()
};
<#field_tys as #zerocopy_crate::TryFromBytes>::is_bit_valid(field_candidate)
})*
}
)
});
ImplBlockBuilder::new(ctx, unn, Trait::TryFromBytes, field_type_trait_bounds)
.inner_extras(extras)
.outer_extras(derive_has_field_struct_union(ctx, unn))
.build()
}
fn derive_try_from_bytes_enum(
ctx: &Ctx,
enm: &DataEnum,
top_level: Trait,
) -> Result<TokenStream, Error> {
let repr = EnumRepr::from_attrs(&ctx.ast.attrs)?;
// If an enum has no fields, it has a well-defined integer representation,
// and every possible bit pattern corresponds to a valid discriminant tag,
// then it *could* be `FromBytes` (even if the user hasn't derived
// `FromBytes`). This holds if, for `repr(uN)` or `repr(iN)`, there are 2^N
// variants.
let could_be_from_bytes = enum_size_from_repr(&repr)
.map(|size| enm.fields().is_empty() && enm.variants.len() == 1usize << size)
.unwrap_or(false);
let trivial_is_bit_valid = try_gen_trivial_is_bit_valid(ctx, top_level);
let extra = match (trivial_is_bit_valid, could_be_from_bytes) {
(Some(is_bit_valid), _) => is_bit_valid,
// SAFETY: It would be sound for the enum to implement `FromBytes`, as
// required by `gen_trivial_is_bit_valid_unchecked`.
(None, true) => unsafe { gen_trivial_is_bit_valid_unchecked(ctx) },
(None, false) => match derive_is_bit_valid(ctx, enm, &repr) {
Ok(extra) => extra,
Err(_) if ctx.skip_on_error => return Ok(TokenStream::new()),
Err(e) => return Err(e),
},
};
Ok(ImplBlockBuilder::new(ctx, enm, Trait::TryFromBytes, FieldBounds::ALL_SELF)
.inner_extras(extra)
.build())
}
fn try_gen_trivial_is_bit_valid(ctx: &Ctx, top_level: Trait) -> Option<proc_macro2::TokenStream> {
// If the top-level trait is `FromBytes` and `Self` has no type parameters,
// then the `FromBytes` derive will fail compilation if `Self` is not
// actually soundly `FromBytes`, and so we can rely on that for our
// `is_bit_valid` impl. It's plausible that we could make changes - or Rust
// could make changes (such as the "trivial bounds" language feature) - that
// make this no longer true. To hedge against these, we include an explicit
// `Self: FromBytes` check in the generated `is_bit_valid`, which is
// bulletproof.
//
// If `ctx.skip_on_error` is true, we can't rely on the `FromBytes` derive
// to fail compilation if `Self` is not actually soundly `FromBytes`.
if matches!(top_level, Trait::FromBytes)
&& ctx.ast.generics.params.is_empty()
&& !ctx.skip_on_error
{
let zerocopy_crate = &ctx.zerocopy_crate;
let core = ctx.core_path();
Some(quote!(
// SAFETY: See inline.
#[inline(always)]
fn is_bit_valid<___ZcAlignment>(
_candidate: #zerocopy_crate::Maybe<'_, Self, ___ZcAlignment>,
) -> #core::primitive::bool
where
___ZcAlignment: #zerocopy_crate::invariant::Alignment,
{
if false {
fn assert_is_from_bytes<T>()
where
T: #zerocopy_crate::FromBytes,
T: ?#core::marker::Sized,
{
}
assert_is_from_bytes::<Self>();
}
// SAFETY: The preceding code only compiles if `Self:
// FromBytes`. Thus, this code only compiles if all initialized
// byte sequences represent valid instances of `Self`.
true
}
))
} else {
None
}
}
/// # Safety
///
/// All initialized bit patterns must be valid for `Self`.
unsafe fn gen_trivial_is_bit_valid_unchecked(ctx: &Ctx) -> proc_macro2::TokenStream {
let zerocopy_crate = &ctx.zerocopy_crate;
let core = ctx.core_path();
quote!(
// SAFETY: The caller of `gen_trivial_is_bit_valid_unchecked` has
// promised that all initialized bit patterns are valid for `Self`.
#[inline(always)]
fn is_bit_valid<___ZcAlignment>(
_candidate: #zerocopy_crate::Maybe<'_, Self, ___ZcAlignment>,
) -> #core::primitive::bool
where
___ZcAlignment: #zerocopy_crate::invariant::Alignment,
{
true
}
)
}

80
rust/zerocopy-derive/derive/unaligned.rs Normal file
View File

@ -0,0 +1,80 @@
// SPDX-License-Identifier: (BSD-2-Clause OR Apache-2.0) OR MIT
use proc_macro2::{Span, TokenStream};
use syn::{Data, DataEnum, DataStruct, DataUnion, Error};
use crate::{
repr::{EnumRepr, StructUnionRepr},
util::{Ctx, FieldBounds, ImplBlockBuilder, Trait},
};
pub(crate) fn derive_unaligned(ctx: &Ctx, _top_level: Trait) -> Result<TokenStream, Error> {
match &ctx.ast.data {
Data::Struct(strct) => derive_unaligned_struct(ctx, strct),
Data::Enum(enm) => derive_unaligned_enum(ctx, enm),
Data::Union(unn) => derive_unaligned_union(ctx, unn),
}
}
/// A struct is `Unaligned` if:
/// - `repr(align)` is no more than 1 and either
/// - `repr(C)` or `repr(transparent)` and
/// - all fields `Unaligned`
/// - `repr(packed)`
fn derive_unaligned_struct(ctx: &Ctx, strct: &DataStruct) -> Result<TokenStream, Error> {
let repr = StructUnionRepr::from_attrs(&ctx.ast.attrs)?;
repr.unaligned_validate_no_align_gt_1()?;
let field_bounds = if repr.is_packed_1() {
FieldBounds::None
} else if repr.is_c() || repr.is_transparent() {
FieldBounds::ALL_SELF
} else {
return ctx.error_or_skip(Error::new(
Span::call_site(),
"must have #[repr(C)], #[repr(transparent)], or #[repr(packed)] attribute in order to guarantee this type's alignment",
));
};
Ok(ImplBlockBuilder::new(ctx, strct, Trait::Unaligned, field_bounds).build())
}
/// An enum is `Unaligned` if:
/// - No `repr(align(N > 1))`
/// - `repr(u8)` or `repr(i8)`
fn derive_unaligned_enum(ctx: &Ctx, enm: &DataEnum) -> Result<TokenStream, Error> {
let repr = EnumRepr::from_attrs(&ctx.ast.attrs)?;
repr.unaligned_validate_no_align_gt_1()?;
if !repr.is_u8() && !repr.is_i8() {
return ctx.error_or_skip(Error::new(
Span::call_site(),
"must have #[repr(u8)] or #[repr(i8)] attribute in order to guarantee this type's alignment",
));
}
Ok(ImplBlockBuilder::new(ctx, enm, Trait::Unaligned, FieldBounds::ALL_SELF).build())
}
/// Like structs, a union is `Unaligned` if:
/// - `repr(align)` is no more than 1 and either
/// - `repr(C)` or `repr(transparent)` and
/// - all fields `Unaligned`
/// - `repr(packed)`
fn derive_unaligned_union(ctx: &Ctx, unn: &DataUnion) -> Result<TokenStream, Error> {
let repr = StructUnionRepr::from_attrs(&ctx.ast.attrs)?;
repr.unaligned_validate_no_align_gt_1()?;
let field_type_trait_bounds = if repr.is_packed_1() {
FieldBounds::None
} else if repr.is_c() || repr.is_transparent() {
FieldBounds::ALL_SELF
} else {
return ctx.error_or_skip(Error::new(
Span::call_site(),
"must have #[repr(C)], #[repr(transparent)], or #[repr(packed)] attribute in order to guarantee this type's alignment",
));
};
Ok(ImplBlockBuilder::new(ctx, unn, Trait::Unaligned, field_type_trait_bounds).build())
}

146
rust/zerocopy-derive/lib.rs Normal file
View File

@ -0,0 +1,146 @@
// SPDX-License-Identifier: (BSD-2-Clause OR Apache-2.0) OR MIT
// Copyright 2019 The Fuchsia Authors
//
// Licensed under a BSD-style license <LICENSE-BSD>, Apache License, Version 2.0
// <LICENSE-APACHE or https://www.apache.org/licenses/LICENSE-2.0>, or the MIT
// license <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your option.
// This file may not be copied, modified, or distributed except according to
// those terms.
//! Derive macros for [zerocopy]'s traits.
//!
//! [zerocopy]: https://docs.rs/zerocopy
// Sometimes we want to use lints which were added after our MSRV.
// `unknown_lints` is `warn` by default and we deny warnings in CI, so without
// this attribute, any unknown lint would cause a CI failure when testing with
// our MSRV.
#![allow(unknown_lints)]
#![deny(renamed_and_removed_lints)]
#![deny(
clippy::all,
clippy::missing_safety_doc,
clippy::multiple_unsafe_ops_per_block,
clippy::undocumented_unsafe_blocks
)]
// We defer to own discretion on type complexity.
#![allow(clippy::type_complexity)]
// Inlining format args isn't supported on our MSRV.
#![allow(clippy::uninlined_format_args)]
#![deny(
rustdoc::bare_urls,
rustdoc::broken_intra_doc_links,
rustdoc::invalid_codeblock_attributes,
rustdoc::invalid_html_tags,
rustdoc::invalid_rust_codeblocks,
rustdoc::missing_crate_level_docs,
rustdoc::private_intra_doc_links
)]
#![recursion_limit = "128"]
macro_rules! ident {
(($fmt:literal $(, $arg:expr)*), $span:expr) => {
syn::Ident::new(&format!($fmt $(, crate::util::to_ident_str($arg))*), $span)
};
}
mod derive;
#[cfg(test)]
mod output_tests;
mod repr;
mod util;
use syn::{DeriveInput, Error};
use crate::util::*;
// FIXME(https://github.com/rust-lang/rust/issues/54140): Some errors could be
// made better if we could add multiple lines of error output like this:
//
// error: unsupported representation
// --> enum.rs:28:8
// |
// 28 | #[repr(transparent)]
// |
// help: required by the derive of FromBytes
//
// Instead, we have more verbose error messages like "unsupported representation
// for deriving FromZeros, FromBytes, IntoBytes, or Unaligned on an enum"
//
// This will probably require Span::error
// (https://doc.rust-lang.org/nightly/proc_macro/struct.Span.html#method.error),
// which is currently unstable. Revisit this once it's stable.
/// Defines a derive function named `$outer` which parses its input
/// `TokenStream` as a `DeriveInput` and then invokes the `$inner` function.
///
/// Note that the separate `$outer` parameter is required - proc macro functions
/// are currently required to live at the crate root, and so the caller must
/// specify the name in order to avoid name collisions.
macro_rules! derive {
($trait:ident => $outer:ident => $inner:path) => {
#[proc_macro_derive($trait, attributes(zerocopy))]
pub fn $outer(ts: proc_macro::TokenStream) -> proc_macro::TokenStream {
let ast = syn::parse_macro_input!(ts as DeriveInput);
let ctx = match Ctx::try_from_derive_input(ast) {
Ok(ctx) => ctx,
Err(e) => return e.into_compile_error().into(),
};
let ts = $inner(&ctx, Trait::$trait).into_ts();
// We wrap in `const_block` as a backstop in case any derive fails
// to wrap its output in `const_block` (and thus fails to annotate)
// with the full set of `#[allow(...)]` attributes).
let ts = const_block([Some(ts)]);
#[cfg(test)]
crate::util::testutil::check_hygiene(ts.clone());
ts.into()
}
};
}
trait IntoTokenStream {
fn into_ts(self) -> proc_macro2::TokenStream;
}
impl IntoTokenStream for proc_macro2::TokenStream {
fn into_ts(self) -> proc_macro2::TokenStream {
self
}
}
impl IntoTokenStream for Result<proc_macro2::TokenStream, Error> {
fn into_ts(self) -> proc_macro2::TokenStream {
match self {
Ok(ts) => ts,
Err(err) => err.to_compile_error(),
}
}
}
derive!(KnownLayout => derive_known_layout => crate::derive::known_layout::derive);
derive!(Immutable => derive_immutable => crate::derive::derive_immutable);
derive!(TryFromBytes => derive_try_from_bytes => crate::derive::try_from_bytes::derive_try_from_bytes);
derive!(FromZeros => derive_from_zeros => crate::derive::from_bytes::derive_from_zeros);
derive!(FromBytes => derive_from_bytes => crate::derive::from_bytes::derive_from_bytes);
derive!(IntoBytes => derive_into_bytes => crate::derive::into_bytes::derive_into_bytes);
derive!(Unaligned => derive_unaligned => crate::derive::unaligned::derive_unaligned);
derive!(ByteHash => derive_hash => crate::derive::derive_hash);
derive!(ByteEq => derive_eq => crate::derive::derive_eq);
derive!(SplitAt => derive_split_at => crate::derive::derive_split_at);
/// Deprecated: prefer [`FromZeros`] instead.
#[deprecated(since = "0.8.0", note = "`FromZeroes` was renamed to `FromZeros`")]
#[doc(hidden)]
#[proc_macro_derive(FromZeroes)]
pub fn derive_from_zeroes(ts: proc_macro::TokenStream) -> proc_macro::TokenStream {
derive_from_zeros(ts)
}
/// Deprecated: prefer [`IntoBytes`] instead.
#[deprecated(since = "0.8.0", note = "`AsBytes` was renamed to `IntoBytes`")]
#[doc(hidden)]
#[proc_macro_derive(AsBytes)]
pub fn derive_as_bytes(ts: proc_macro::TokenStream) -> proc_macro::TokenStream {
derive_into_bytes(ts)
}

851
rust/zerocopy-derive/repr.rs Normal file
View File

@ -0,0 +1,851 @@
// SPDX-License-Identifier: (BSD-2-Clause OR Apache-2.0) OR MIT
// Copyright 2019 The Fuchsia Authors
//
// Licensed under a BSD-style license <LICENSE-BSD>, Apache License, Version 2.0
// <LICENSE-APACHE or https://www.apache.org/licenses/LICENSE-2.0>, or the MIT
// license <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your option.
// This file may not be copied, modified, or distributed except according to
// those terms.
use core::{
convert::{Infallible, TryFrom},
num::NonZeroU32,
};
use proc_macro2::{Span, TokenStream};
use quote::{quote_spanned, ToTokens, TokenStreamExt as _};
use syn::{
punctuated::Punctuated, spanned::Spanned as _, token::Comma, Attribute, Error, LitInt, Meta,
MetaList,
};
/// The computed representation of a type.
///
/// This is the result of processing all `#[repr(...)]` attributes on a type, if
/// any. A `Repr` is only capable of representing legal combinations of
/// `#[repr(...)]` attributes.
#[cfg_attr(test, derive(Copy, Clone, Debug))]
pub(crate) enum Repr<Prim, Packed> {
/// `#[repr(transparent)]`
Transparent(Span),
/// A compound representation: `repr(C)`, `repr(Rust)`, or `repr(Int)`
/// optionally combined with `repr(packed(...))` or `repr(align(...))`
Compound(Spanned<CompoundRepr<Prim>>, Option<Spanned<AlignRepr<Packed>>>),
}
/// A compound representation: `repr(C)`, `repr(Rust)`, or `repr(Int)`.
#[cfg_attr(test, derive(Copy, Clone, Debug, Eq, PartialEq))]
pub(crate) enum CompoundRepr<Prim> {
C,
Rust,
Primitive(Prim),
}
/// `repr(Int)`
#[derive(Copy, Clone)]
#[cfg_attr(test, derive(Debug, Eq, PartialEq))]
pub(crate) enum PrimitiveRepr {
U8,
U16,
U32,
U64,
U128,
Usize,
I8,
I16,
I32,
I64,
I128,
Isize,
}
/// `repr(packed(...))` or `repr(align(...))`
#[cfg_attr(test, derive(Copy, Clone, Debug, Eq, PartialEq))]
pub(crate) enum AlignRepr<Packed> {
Packed(Packed),
Align(NonZeroU32),
}
/// The representations which can legally appear on a struct or union type.
pub(crate) type StructUnionRepr = Repr<Infallible, NonZeroU32>;
/// The representations which can legally appear on an enum type.
pub(crate) type EnumRepr = Repr<PrimitiveRepr, Infallible>;
impl<Prim, Packed> Repr<Prim, Packed> {
/// Gets the name of this "repr type" - the non-align `repr(X)` that is used
/// in prose to refer to this type.
///
/// For example, we would refer to `#[repr(C, align(4))] struct Foo { ... }`
/// as a "`repr(C)` struct".
pub(crate) fn repr_type_name(&self) -> &str
where
Prim: Copy + With<PrimitiveRepr>,
{
use CompoundRepr::*;
use PrimitiveRepr::*;
use Repr::*;
match self {
Transparent(_span) => "repr(transparent)",
Compound(Spanned { t: repr, span: _ }, _align) => match repr {
C => "repr(C)",
Rust => "repr(Rust)",
Primitive(prim) => prim.with(|prim| match prim {
U8 => "repr(u8)",
U16 => "repr(u16)",
U32 => "repr(u32)",
U64 => "repr(u64)",
U128 => "repr(u128)",
Usize => "repr(usize)",
I8 => "repr(i8)",
I16 => "repr(i16)",
I32 => "repr(i32)",
I64 => "repr(i64)",
I128 => "repr(i128)",
Isize => "repr(isize)",
}),
},
}
}
pub(crate) fn is_transparent(&self) -> bool {
matches!(self, Repr::Transparent(_))
}
pub(crate) fn is_c(&self) -> bool {
use CompoundRepr::*;
matches!(self, Repr::Compound(Spanned { t: C, span: _ }, _align))
}
pub(crate) fn is_primitive(&self) -> bool {
use CompoundRepr::*;
matches!(self, Repr::Compound(Spanned { t: Primitive(_), span: _ }, _align))
}
pub(crate) fn get_packed(&self) -> Option<&Packed> {
use AlignRepr::*;
use Repr::*;
if let Compound(_, Some(Spanned { t: Packed(p), span: _ })) = self {
Some(p)
} else {
None
}
}
pub(crate) fn get_align(&self) -> Option<Spanned<NonZeroU32>> {
use AlignRepr::*;
use Repr::*;
if let Compound(_, Some(Spanned { t: Align(n), span })) = self {
Some(Spanned::new(*n, *span))
} else {
None
}
}
pub(crate) fn is_align_gt_1(&self) -> bool {
self.get_align().map(|n| n.t.get() > 1).unwrap_or(false)
}
/// When deriving `Unaligned`, validate that the decorated type has no
/// `#[repr(align(N))]` attribute where `N > 1`. If no such attribute exists
/// (including if `N == 1`), this returns `Ok(())`, and otherwise it returns
/// a descriptive error.
pub(crate) fn unaligned_validate_no_align_gt_1(&self) -> Result<(), Error> {
if let Some(n) = self.get_align().filter(|n| n.t.get() > 1) {
Err(Error::new(
n.span,
"cannot derive `Unaligned` on type with alignment greater than 1",
))
} else {
Ok(())
}
}
}
impl<Prim> Repr<Prim, NonZeroU32> {
/// Does `self` describe a `#[repr(packed)]` or `#[repr(packed(1))]` type?
pub(crate) fn is_packed_1(&self) -> bool {
self.get_packed().map(|n| n.get() == 1).unwrap_or(false)
}
}
impl<Packed> Repr<PrimitiveRepr, Packed> {
fn get_primitive(&self) -> Option<&PrimitiveRepr> {
use CompoundRepr::*;
use Repr::*;
if let Compound(Spanned { t: Primitive(p), span: _ }, _align) = self {
Some(p)
} else {
None
}
}
/// Does `self` describe a `#[repr(u8)]` type?
pub(crate) fn is_u8(&self) -> bool {
matches!(self.get_primitive(), Some(PrimitiveRepr::U8))
}
/// Does `self` describe a `#[repr(i8)]` type?
pub(crate) fn is_i8(&self) -> bool {
matches!(self.get_primitive(), Some(PrimitiveRepr::I8))
}
}
impl<Prim, Packed> ToTokens for Repr<Prim, Packed>
where
Prim: With<PrimitiveRepr> + Copy,
Packed: With<NonZeroU32> + Copy,
{
fn to_tokens(&self, ts: &mut TokenStream) {
use Repr::*;
match self {
Transparent(span) => ts.append_all(quote_spanned! { *span=> #[repr(transparent)] }),
Compound(repr, align) => {
repr.to_tokens(ts);
if let Some(align) = align {
align.to_tokens(ts);
}
}
}
}
}
impl<Prim: With<PrimitiveRepr> + Copy> ToTokens for Spanned<CompoundRepr<Prim>> {
fn to_tokens(&self, ts: &mut TokenStream) {
use CompoundRepr::*;
match &self.t {
C => ts.append_all(quote_spanned! { self.span=> #[repr(C)] }),
Rust => ts.append_all(quote_spanned! { self.span=> #[repr(Rust)] }),
Primitive(prim) => prim.with(|prim| Spanned::new(prim, self.span).to_tokens(ts)),
}
}
}
impl ToTokens for Spanned<PrimitiveRepr> {
fn to_tokens(&self, ts: &mut TokenStream) {
use PrimitiveRepr::*;
match self.t {
U8 => ts.append_all(quote_spanned! { self.span => #[repr(u8)] }),
U16 => ts.append_all(quote_spanned! { self.span => #[repr(u16)] }),
U32 => ts.append_all(quote_spanned! { self.span => #[repr(u32)] }),
U64 => ts.append_all(quote_spanned! { self.span => #[repr(u64)] }),
U128 => ts.append_all(quote_spanned! { self.span => #[repr(u128)] }),
Usize => ts.append_all(quote_spanned! { self.span => #[repr(usize)] }),
I8 => ts.append_all(quote_spanned! { self.span => #[repr(i8)] }),
I16 => ts.append_all(quote_spanned! { self.span => #[repr(i16)] }),
I32 => ts.append_all(quote_spanned! { self.span => #[repr(i32)] }),
I64 => ts.append_all(quote_spanned! { self.span => #[repr(i64)] }),
I128 => ts.append_all(quote_spanned! { self.span => #[repr(i128)] }),
Isize => ts.append_all(quote_spanned! { self.span => #[repr(isize)] }),
}
}
}
impl<Packed: With<NonZeroU32> + Copy> ToTokens for Spanned<AlignRepr<Packed>> {
fn to_tokens(&self, ts: &mut TokenStream) {
use AlignRepr::*;
// We use `syn::Index` instead of `u32` because `quote_spanned!`
// serializes `u32` literals as `123u32`, not just `123`. Rust doesn't
// recognize that as a valid argument to `#[repr(align(...))]` or
// `#[repr(packed(...))]`.
let to_index = |n: NonZeroU32| syn::Index { index: n.get(), span: self.span };
match self.t {
Packed(n) => n.with(|n| {
let n = to_index(n);
ts.append_all(quote_spanned! { self.span => #[repr(packed(#n))] })
}),
Align(n) => {
let n = to_index(n);
ts.append_all(quote_spanned! { self.span => #[repr(align(#n))] })
}
}
}
}
/// The result of parsing a single `#[repr(...)]` attribute or a single
/// directive inside a compound `#[repr(..., ...)]` attribute.
#[derive(Copy, Clone, PartialEq, Eq)]
#[cfg_attr(test, derive(Debug))]
pub(crate) enum RawRepr {
Transparent,
C,
Rust,
U8,
U16,
U32,
U64,
U128,
Usize,
I8,
I16,
I32,
I64,
I128,
Isize,
Align(NonZeroU32),
PackedN(NonZeroU32),
Packed,
}
/// The error from converting from a `RawRepr`.
#[cfg_attr(test, derive(Debug, Eq, PartialEq))]
pub(crate) enum FromRawReprError<E> {
/// The `RawRepr` doesn't affect the high-level repr we're parsing (e.g.
/// it's `align(...)` and we're parsing a `CompoundRepr`).
None,
/// The `RawRepr` is invalid for the high-level repr we're parsing (e.g.
/// it's `packed` repr and we're parsing an `AlignRepr` for an enum type).
Err(E),
}
/// The representation hint is not supported for the decorated type.
#[cfg_attr(test, derive(Copy, Clone, Debug, Eq, PartialEq))]
pub(crate) struct UnsupportedReprError;
impl<Prim: With<PrimitiveRepr>> TryFrom<RawRepr> for CompoundRepr<Prim> {
type Error = FromRawReprError<UnsupportedReprError>;
fn try_from(
raw: RawRepr,
) -> Result<CompoundRepr<Prim>, FromRawReprError<UnsupportedReprError>> {
use RawRepr::*;
match raw {
C => Ok(CompoundRepr::C),
Rust => Ok(CompoundRepr::Rust),
raw @ (U8 | U16 | U32 | U64 | U128 | Usize | I8 | I16 | I32 | I64 | I128 | Isize) => {
Prim::try_with_or(
|| match raw {
U8 => Ok(PrimitiveRepr::U8),
U16 => Ok(PrimitiveRepr::U16),
U32 => Ok(PrimitiveRepr::U32),
U64 => Ok(PrimitiveRepr::U64),
U128 => Ok(PrimitiveRepr::U128),
Usize => Ok(PrimitiveRepr::Usize),
I8 => Ok(PrimitiveRepr::I8),
I16 => Ok(PrimitiveRepr::I16),
I32 => Ok(PrimitiveRepr::I32),
I64 => Ok(PrimitiveRepr::I64),
I128 => Ok(PrimitiveRepr::I128),
Isize => Ok(PrimitiveRepr::Isize),
Transparent | C | Rust | Align(_) | PackedN(_) | Packed => {
Err(UnsupportedReprError)
}
},
UnsupportedReprError,
)
.map(CompoundRepr::Primitive)
.map_err(FromRawReprError::Err)
}
Transparent | Align(_) | PackedN(_) | Packed => Err(FromRawReprError::None),
}
}
}
impl<Pcked: With<NonZeroU32>> TryFrom<RawRepr> for AlignRepr<Pcked> {
type Error = FromRawReprError<UnsupportedReprError>;
fn try_from(raw: RawRepr) -> Result<AlignRepr<Pcked>, FromRawReprError<UnsupportedReprError>> {
use RawRepr::*;
match raw {
Packed | PackedN(_) => Pcked::try_with_or(
|| match raw {
Packed => Ok(NonZeroU32::new(1).unwrap()),
PackedN(n) => Ok(n),
U8 | U16 | U32 | U64 | U128 | Usize | I8 | I16 | I32 | I64 | I128 | Isize
| Transparent | C | Rust | Align(_) => Err(UnsupportedReprError),
},
UnsupportedReprError,
)
.map(AlignRepr::Packed)
.map_err(FromRawReprError::Err),
Align(n) => Ok(AlignRepr::Align(n)),
U8 | U16 | U32 | U64 | U128 | Usize | I8 | I16 | I32 | I64 | I128 | Isize
| Transparent | C | Rust => Err(FromRawReprError::None),
}
}
}
/// The error from extracting a high-level repr type from a list of `RawRepr`s.
#[cfg_attr(test, derive(Copy, Clone, Debug, Eq, PartialEq))]
enum FromRawReprsError<E> {
/// One of the `RawRepr`s is invalid for the high-level repr we're parsing
/// (e.g. there's a `packed` repr and we're parsing an `AlignRepr` for an
/// enum type).
Single(E),
/// Two `RawRepr`s appear which both affect the high-level repr we're
/// parsing (e.g., the list is `#[repr(align(2), packed)]`). Note that we
/// conservatively treat redundant reprs as conflicting (e.g.
/// `#[repr(packed, packed)]`).
Conflict,
}
/// Tries to extract a high-level repr from a list of `RawRepr`s.
fn try_from_raw_reprs<'a, E, R: TryFrom<RawRepr, Error = FromRawReprError<E>>>(
r: impl IntoIterator<Item = &'a Spanned<RawRepr>>,
) -> Result<Option<Spanned<R>>, Spanned<FromRawReprsError<E>>> {
// Walk the list of `RawRepr`s and attempt to convert each to an `R`. Bail
// if we find any errors. If we find more than one which converts to an `R`,
// bail with a `Conflict` error.
r.into_iter().try_fold(None, |found: Option<Spanned<R>>, raw| {
let new = match Spanned::<R>::try_from(*raw) {
Ok(r) => r,
// This `RawRepr` doesn't convert to an `R`, so keep the current
// found `R`, if any.
Err(FromRawReprError::None) => return Ok(found),
// This repr is unsupported for the decorated type (e.g.
// `repr(packed)` on an enum).
Err(FromRawReprError::Err(Spanned { t: err, span })) => {
return Err(Spanned::new(FromRawReprsError::Single(err), span))
}
};
if let Some(found) = found {
// We already found an `R`, but this `RawRepr` also converts to an
// `R`, so that's a conflict.
//
// `Span::join` returns `None` if the two spans are from different
// files or if we're not on the nightly compiler. In that case, just
// use `new`'s span.
let span = found.span.join(new.span).unwrap_or(new.span);
Err(Spanned::new(FromRawReprsError::Conflict, span))
} else {
Ok(Some(new))
}
})
}
/// The error returned from [`Repr::from_attrs`].
#[cfg_attr(test, derive(Copy, Clone, Debug, Eq, PartialEq))]
enum FromAttrsError {
FromRawReprs(FromRawReprsError<UnsupportedReprError>),
Unrecognized,
}
impl From<FromRawReprsError<UnsupportedReprError>> for FromAttrsError {
fn from(err: FromRawReprsError<UnsupportedReprError>) -> FromAttrsError {
FromAttrsError::FromRawReprs(err)
}
}
impl From<UnrecognizedReprError> for FromAttrsError {
fn from(_err: UnrecognizedReprError) -> FromAttrsError {
FromAttrsError::Unrecognized
}
}
impl From<Spanned<FromAttrsError>> for Error {
fn from(err: Spanned<FromAttrsError>) -> Error {
let Spanned { t: err, span } = err;
match err {
FromAttrsError::FromRawReprs(FromRawReprsError::Single(
_err @ UnsupportedReprError,
)) => Error::new(span, "unsupported representation hint for the decorated type"),
FromAttrsError::FromRawReprs(FromRawReprsError::Conflict) => {
// NOTE: This says "another" rather than "a preceding" because
// when one of the reprs involved is `transparent`, we detect
// that condition in `Repr::from_attrs`, and at that point we
// can't tell which repr came first, so we might report this on
// the first involved repr rather than the second, third, etc.
Error::new(span, "this conflicts with another representation hint")
}
FromAttrsError::Unrecognized => Error::new(span, "unrecognized representation hint"),
}
}
}
impl<Prim, Packed> Repr<Prim, Packed> {
fn from_attrs_inner(attrs: &[Attribute]) -> Result<Repr<Prim, Packed>, Spanned<FromAttrsError>>
where
Prim: With<PrimitiveRepr>,
Packed: With<NonZeroU32>,
{
let raw_reprs = RawRepr::from_attrs(attrs).map_err(Spanned::from)?;
let transparent = {
let mut transparents = raw_reprs.iter().filter_map(|Spanned { t, span }| match t {
RawRepr::Transparent => Some(span),
_ => None,
});
let first = transparents.next();
let second = transparents.next();
match (first, second) {
(None, None) => None,
(Some(span), None) => Some(*span),
(Some(_), Some(second)) => {
return Err(Spanned::new(
FromAttrsError::FromRawReprs(FromRawReprsError::Conflict),
*second,
))
}
// An iterator can't produce a value only on the second call to
// `.next()`.
(None, Some(_)) => unreachable!(),
}
};
let compound: Option<Spanned<CompoundRepr<Prim>>> =
try_from_raw_reprs(raw_reprs.iter()).map_err(Spanned::from)?;
let align: Option<Spanned<AlignRepr<Packed>>> =
try_from_raw_reprs(raw_reprs.iter()).map_err(Spanned::from)?;
if let Some(span) = transparent {
if compound.is_some() || align.is_some() {
// Arbitrarily report the problem on the `transparent` span. Any
// span will do.
return Err(Spanned::new(FromRawReprsError::Conflict.into(), span));
}
Ok(Repr::Transparent(span))
} else {
Ok(Repr::Compound(
compound.unwrap_or(Spanned::new(CompoundRepr::Rust, Span::call_site())),
align,
))
}
}
}
impl<Prim, Packed> Repr<Prim, Packed> {
pub(crate) fn from_attrs(attrs: &[Attribute]) -> Result<Repr<Prim, Packed>, Error>
where
Prim: With<PrimitiveRepr>,
Packed: With<NonZeroU32>,
{
Repr::from_attrs_inner(attrs).map_err(Into::into)
}
}
/// The representation hint could not be parsed or was unrecognized.
struct UnrecognizedReprError;
impl RawRepr {
fn from_attrs(
attrs: &[Attribute],
) -> Result<Vec<Spanned<RawRepr>>, Spanned<UnrecognizedReprError>> {
let mut reprs = Vec::new();
for attr in attrs {
// Ignore documentation attributes.
if attr.path().is_ident("doc") {
continue;
}
if let Meta::List(ref meta_list) = attr.meta {
if meta_list.path.is_ident("repr") {
let parsed: Punctuated<Meta, Comma> =
match meta_list.parse_args_with(Punctuated::parse_terminated) {
Ok(parsed) => parsed,
Err(_) => {
return Err(Spanned::new(
UnrecognizedReprError,
meta_list.tokens.span(),
))
}
};
for meta in parsed {
let s = meta.span();
reprs.push(
RawRepr::from_meta(&meta)
.map(|r| Spanned::new(r, s))
.map_err(|e| Spanned::new(e, s))?,
);
}
}
}
}
Ok(reprs)
}
fn from_meta(meta: &Meta) -> Result<RawRepr, UnrecognizedReprError> {
let (path, list) = match meta {
Meta::Path(path) => (path, None),
Meta::List(list) => (&list.path, Some(list)),
_ => return Err(UnrecognizedReprError),
};
let ident = path.get_ident().ok_or(UnrecognizedReprError)?;
// Only returns `Ok` for non-zero power-of-two values.
let parse_nzu64 = |list: &MetaList| {
list.parse_args::<LitInt>()
.and_then(|int| int.base10_parse::<NonZeroU32>())
.map_err(|_| UnrecognizedReprError)
.and_then(|nz| {
if nz.get().is_power_of_two() {
Ok(nz)
} else {
Err(UnrecognizedReprError)
}
})
};
use RawRepr::*;
Ok(match (ident.to_string().as_str(), list) {
("u8", None) => U8,
("u16", None) => U16,
("u32", None) => U32,
("u64", None) => U64,
("u128", None) => U128,
("usize", None) => Usize,
("i8", None) => I8,
("i16", None) => I16,
("i32", None) => I32,
("i64", None) => I64,
("i128", None) => I128,
("isize", None) => Isize,
("C", None) => C,
("transparent", None) => Transparent,
("Rust", None) => Rust,
("packed", None) => Packed,
("packed", Some(list)) => PackedN(parse_nzu64(list)?),
("align", Some(list)) => Align(parse_nzu64(list)?),
_ => return Err(UnrecognizedReprError),
})
}
}
pub(crate) use util::*;
mod util {
use super::*;
/// A value with an associated span.
#[derive(Copy, Clone)]
#[cfg_attr(test, derive(Debug))]
pub(crate) struct Spanned<T> {
pub(crate) t: T,
pub(crate) span: Span,
}
impl<T> Spanned<T> {
pub(super) fn new(t: T, span: Span) -> Spanned<T> {
Spanned { t, span }
}
pub(super) fn from<U>(s: Spanned<U>) -> Spanned<T>
where
T: From<U>,
{
let Spanned { t: u, span } = s;
Spanned::new(u.into(), span)
}
/// Delegates to `T: TryFrom`, preserving span information in both the
/// success and error cases.
pub(super) fn try_from<E, U>(
u: Spanned<U>,
) -> Result<Spanned<T>, FromRawReprError<Spanned<E>>>
where
T: TryFrom<U, Error = FromRawReprError<E>>,
{
let Spanned { t: u, span } = u;
T::try_from(u).map(|t| Spanned { t, span }).map_err(|err| match err {
FromRawReprError::None => FromRawReprError::None,
FromRawReprError::Err(e) => FromRawReprError::Err(Spanned::new(e, span)),
})
}
}
// Used to permit implementing `With<T> for T: Inhabited` and for
// `Infallible` without a blanket impl conflict.
pub(crate) trait Inhabited {}
impl Inhabited for PrimitiveRepr {}
impl Inhabited for NonZeroU32 {}
pub(crate) trait With<T> {
fn with<O, F: FnOnce(T) -> O>(self, f: F) -> O;
fn try_with_or<E, F: FnOnce() -> Result<T, E>>(f: F, err: E) -> Result<Self, E>
where
Self: Sized;
}
impl<T: Inhabited> With<T> for T {
fn with<O, F: FnOnce(T) -> O>(self, f: F) -> O {
f(self)
}
fn try_with_or<E, F: FnOnce() -> Result<T, E>>(f: F, _err: E) -> Result<Self, E> {
f()
}
}
impl<T> With<T> for Infallible {
fn with<O, F: FnOnce(T) -> O>(self, _f: F) -> O {
match self {}
}
fn try_with_or<E, F: FnOnce() -> Result<T, E>>(_f: F, err: E) -> Result<Self, E> {
Err(err)
}
}
}
#[cfg(test)]
mod tests {
use syn::parse_quote;
use super::*;
impl<T> From<T> for Spanned<T> {
fn from(t: T) -> Spanned<T> {
Spanned::new(t, Span::call_site())
}
}
// We ignore spans for equality in testing since real spans are hard to
// synthesize and don't implement `PartialEq`.
impl<T: PartialEq> PartialEq for Spanned<T> {
fn eq(&self, other: &Spanned<T>) -> bool {
self.t.eq(&other.t)
}
}
impl<T: Eq> Eq for Spanned<T> {}
impl<Prim: PartialEq, Packed: PartialEq> PartialEq for Repr<Prim, Packed> {
fn eq(&self, other: &Repr<Prim, Packed>) -> bool {
match (self, other) {
(Repr::Transparent(_), Repr::Transparent(_)) => true,
(Repr::Compound(sc, sa), Repr::Compound(oc, oa)) => (sc, sa) == (oc, oa),
_ => false,
}
}
}
fn s() -> Span {
Span::call_site()
}
#[test]
fn test() {
// Test that a given `#[repr(...)]` attribute parses and returns the
// given `Repr` or error.
macro_rules! test {
($(#[$attr:meta])* => $repr:expr) => {
test!(@inner $(#[$attr])* => Repr => Ok($repr));
};
// In the error case, the caller must explicitly provide the name of
// the `Repr` type to assist in type inference.
(@error $(#[$attr:meta])* => $typ:ident => $repr:expr) => {
test!(@inner $(#[$attr])* => $typ => Err($repr));
};
(@inner $(#[$attr:meta])* => $typ:ident => $repr:expr) => {
let attr: Attribute = parse_quote!($(#[$attr])*);
let mut got = $typ::from_attrs_inner(&[attr]);
let expect: Result<Repr<_, _>, _> = $repr;
if false {
// Force Rust to infer `got` as having the same type as
// `expect`.
got = expect;
}
assert_eq!(got, expect, stringify!($(#[$attr])*));
};
}
use AlignRepr::*;
use CompoundRepr::*;
use PrimitiveRepr::*;
let nz = |n: u32| NonZeroU32::new(n).unwrap();
test!(#[repr(transparent)] => StructUnionRepr::Transparent(s()));
test!(#[repr()] => StructUnionRepr::Compound(Rust.into(), None));
test!(#[repr(packed)] => StructUnionRepr::Compound(Rust.into(), Some(Packed(nz(1)).into())));
test!(#[repr(packed(2))] => StructUnionRepr::Compound(Rust.into(), Some(Packed(nz(2)).into())));
test!(#[repr(align(1))] => StructUnionRepr::Compound(Rust.into(), Some(Align(nz(1)).into())));
test!(#[repr(align(2))] => StructUnionRepr::Compound(Rust.into(), Some(Align(nz(2)).into())));
test!(#[repr(C)] => StructUnionRepr::Compound(C.into(), None));
test!(#[repr(C, packed)] => StructUnionRepr::Compound(C.into(), Some(Packed(nz(1)).into())));
test!(#[repr(C, packed(2))] => StructUnionRepr::Compound(C.into(), Some(Packed(nz(2)).into())));
test!(#[repr(C, align(1))] => StructUnionRepr::Compound(C.into(), Some(Align(nz(1)).into())));
test!(#[repr(C, align(2))] => StructUnionRepr::Compound(C.into(), Some(Align(nz(2)).into())));
test!(#[repr(transparent)] => EnumRepr::Transparent(s()));
test!(#[repr()] => EnumRepr::Compound(Rust.into(), None));
test!(#[repr(align(1))] => EnumRepr::Compound(Rust.into(), Some(Align(nz(1)).into())));
test!(#[repr(align(2))] => EnumRepr::Compound(Rust.into(), Some(Align(nz(2)).into())));
macro_rules! for_each_compound_repr {
($($r:tt => $var:expr),*) => {
$(
test!(#[repr($r)] => EnumRepr::Compound($var.into(), None));
test!(#[repr($r, align(1))] => EnumRepr::Compound($var.into(), Some(Align(nz(1)).into())));
test!(#[repr($r, align(2))] => EnumRepr::Compound($var.into(), Some(Align(nz(2)).into())));
)*
}
}
for_each_compound_repr!(
C => C,
u8 => Primitive(U8),
u16 => Primitive(U16),
u32 => Primitive(U32),
u64 => Primitive(U64),
usize => Primitive(Usize),
i8 => Primitive(I8),
i16 => Primitive(I16),
i32 => Primitive(I32),
i64 => Primitive(I64),
isize => Primitive(Isize)
);
use FromAttrsError::*;
use FromRawReprsError::*;
// Run failure tests which are valid for both `StructUnionRepr` and
// `EnumRepr`.
macro_rules! for_each_repr_type {
($($repr:ident),*) => {
$(
// Invalid packed or align attributes
test!(@error #[repr(packed(0))] => $repr => Unrecognized.into());
test!(@error #[repr(packed(3))] => $repr => Unrecognized.into());
test!(@error #[repr(align(0))] => $repr => Unrecognized.into());
test!(@error #[repr(align(3))] => $repr => Unrecognized.into());
// Conflicts
test!(@error #[repr(transparent, transparent)] => $repr => FromRawReprs(Conflict).into());
test!(@error #[repr(transparent, C)] => $repr => FromRawReprs(Conflict).into());
test!(@error #[repr(transparent, Rust)] => $repr => FromRawReprs(Conflict).into());
test!(@error #[repr(C, transparent)] => $repr => FromRawReprs(Conflict).into());
test!(@error #[repr(C, C)] => $repr => FromRawReprs(Conflict).into());
test!(@error #[repr(C, Rust)] => $repr => FromRawReprs(Conflict).into());
test!(@error #[repr(Rust, transparent)] => $repr => FromRawReprs(Conflict).into());
test!(@error #[repr(Rust, C)] => $repr => FromRawReprs(Conflict).into());
test!(@error #[repr(Rust, Rust)] => $repr => FromRawReprs(Conflict).into());
)*
}
}
for_each_repr_type!(StructUnionRepr, EnumRepr);
// Enum-specific conflicts.
//
// We don't bother to test every combination since that would be a huge
// number (enums can have primitive reprs u8, u16, u32, u64, usize, i8,
// i16, i32, i64, and isize). Instead, since the conflict logic doesn't
// care what specific value of `PrimitiveRepr` is present, we assume
// that testing against u8 alone is fine.
test!(@error #[repr(transparent, u8)] => EnumRepr => FromRawReprs(Conflict).into());
test!(@error #[repr(u8, transparent)] => EnumRepr => FromRawReprs(Conflict).into());
test!(@error #[repr(C, u8)] => EnumRepr => FromRawReprs(Conflict).into());
test!(@error #[repr(u8, C)] => EnumRepr => FromRawReprs(Conflict).into());
test!(@error #[repr(Rust, u8)] => EnumRepr => FromRawReprs(Conflict).into());
test!(@error #[repr(u8, Rust)] => EnumRepr => FromRawReprs(Conflict).into());
test!(@error #[repr(u8, u8)] => EnumRepr => FromRawReprs(Conflict).into());
// Illegal struct/union reprs
test!(@error #[repr(u8)] => StructUnionRepr => FromRawReprs(Single(UnsupportedReprError)).into());
test!(@error #[repr(u16)] => StructUnionRepr => FromRawReprs(Single(UnsupportedReprError)).into());
test!(@error #[repr(u32)] => StructUnionRepr => FromRawReprs(Single(UnsupportedReprError)).into());
test!(@error #[repr(u64)] => StructUnionRepr => FromRawReprs(Single(UnsupportedReprError)).into());
test!(@error #[repr(usize)] => StructUnionRepr => FromRawReprs(Single(UnsupportedReprError)).into());
test!(@error #[repr(i8)] => StructUnionRepr => FromRawReprs(Single(UnsupportedReprError)).into());
test!(@error #[repr(i16)] => StructUnionRepr => FromRawReprs(Single(UnsupportedReprError)).into());
test!(@error #[repr(i32)] => StructUnionRepr => FromRawReprs(Single(UnsupportedReprError)).into());
test!(@error #[repr(i64)] => StructUnionRepr => FromRawReprs(Single(UnsupportedReprError)).into());
test!(@error #[repr(isize)] => StructUnionRepr => FromRawReprs(Single(UnsupportedReprError)).into());
// Illegal enum reprs
test!(@error #[repr(packed)] => EnumRepr => FromRawReprs(Single(UnsupportedReprError)).into());
test!(@error #[repr(packed(1))] => EnumRepr => FromRawReprs(Single(UnsupportedReprError)).into());
test!(@error #[repr(packed(2))] => EnumRepr => FromRawReprs(Single(UnsupportedReprError)).into());
}
}

845
rust/zerocopy-derive/util.rs Normal file
View File

@ -0,0 +1,845 @@
// SPDX-License-Identifier: (BSD-2-Clause OR Apache-2.0) OR MIT
// Copyright 2019 The Fuchsia Authors
//
// Licensed under a BSD-style license <LICENSE-BSD>, Apache License, Version 2.0
// <LICENSE-APACHE or https://www.apache.org/licenses/LICENSE-2.0>, or the MIT
// license <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your option.
// This file may not be copied, modified, or distributed except according to
// those terms.
use std::num::NonZeroU32;
use proc_macro2::{Span, TokenStream};
use quote::{quote, quote_spanned, ToTokens};
use syn::{
parse_quote, spanned::Spanned as _, Data, DataEnum, DataStruct, DataUnion, DeriveInput, Error,
Expr, ExprLit, Field, GenericParam, Ident, Index, Lit, LitStr, Meta, Path, Type, Variant,
Visibility, WherePredicate,
};
use crate::repr::{CompoundRepr, EnumRepr, PrimitiveRepr, Repr, Spanned};
pub(crate) struct Ctx {
pub(crate) ast: DeriveInput,
pub(crate) zerocopy_crate: Path,
// The value of the last `#[zerocopy(on_error = ...)]` attribute, or `false`
// if none is provided.
pub(crate) skip_on_error: bool,
// The span of the last `#[zerocopy(on_error = ...)]` attribute, if any.
pub(crate) on_error_span: Option<proc_macro2::Span>,
}
impl Ctx {
/// Attempt to extract a crate path from the provided attributes. Defaults to
/// `::zerocopy` if not found.
pub(crate) fn try_from_derive_input(ast: DeriveInput) -> Result<Self, Error> {
let mut path = parse_quote!(::zerocopy);
let mut skip_on_error = false;
let mut on_error_span = None;
for attr in &ast.attrs {
if let Meta::List(ref meta_list) = attr.meta {
if meta_list.path.is_ident("zerocopy") {
attr.parse_nested_meta(|meta| {
if meta.path.is_ident("crate") {
let expr = meta.value().and_then(|value| value.parse());
if let Ok(Expr::Lit(ExprLit { lit: Lit::Str(lit), .. })) = expr {
if let Ok(path_lit) = lit.parse::<Ident>() {
path = parse_quote!(::#path_lit);
return Ok(());
}
}
return Err(Error::new(
Span::call_site(),
"`crate` attribute requires a path as the value",
));
}
if meta.path.is_ident("on_error") {
on_error_span = Some(meta.path.span());
let value = meta.value()?;
let s: LitStr = value.parse()?;
match s.value().as_str() {
"skip" => skip_on_error = true,
"fail" => skip_on_error = false,
_ => return Err(Error::new(
s.span(),
"unrecognized value for `on_error` attribute from `zerocopy`; expected `skip` or `fail`",
)),
}
return Ok(());
}
Err(Error::new(
Span::call_site(),
format!(
"unknown attribute encountered: {}",
meta.path.into_token_stream()
),
))
})?;
}
}
}
Ok(Self { ast, zerocopy_crate: path, skip_on_error, on_error_span })
}
pub(crate) fn with_input(&self, input: &DeriveInput) -> Self {
Self {
ast: input.clone(),
zerocopy_crate: self.zerocopy_crate.clone(),
skip_on_error: self.skip_on_error,
on_error_span: self.on_error_span,
}
}
pub(crate) fn core_path(&self) -> TokenStream {
let zerocopy_crate = &self.zerocopy_crate;
quote!(#zerocopy_crate::util::macro_util::core_reexport)
}
pub(crate) fn cfg_compile_error(&self) -> TokenStream {
// By checking both during the compilation of the proc macro *and* in
// the generated code, we ensure that `--cfg
// zerocopy_unstable_derive_on_error` need only be passed *either* when
// compiling this crate *or* when compiling the user's crate. The former
// is preferable, but in some situations (such as when cross-compiling
// using `cargo build --target`), it doesn't get propagated to this
// crate's build by default.
if cfg!(zerocopy_unstable_derive_on_error) {
quote!()
} else if let Some(span) = self.on_error_span {
let core = self.core_path();
let error_message = "`on_error` is experimental; pass '--cfg zerocopy_unstable_derive_on_error' to enable";
quote::quote_spanned! {span=>
#[allow(unused_attributes, unexpected_cfgs)]
const _: () = {
#[cfg(not(zerocopy_unstable_derive_on_error))]
#core::compile_error!(#error_message);
};
}
} else {
quote!()
}
}
pub(crate) fn error_or_skip<E>(&self, error: E) -> Result<TokenStream, E> {
if self.skip_on_error {
Ok(self.cfg_compile_error())
} else {
Err(error)
}
}
}
pub(crate) trait DataExt {
/// Extracts the names and types of all fields. For enums, extracts the
/// names and types of fields from each variant. For tuple structs, the
/// names are the indices used to index into the struct (ie, `0`, `1`, etc).
///
/// FIXME: Extracting field names for enums doesn't really make sense. Types
/// makes sense because we don't care about where they live - we just care
/// about transitive ownership. But for field names, we'd only use them when
/// generating is_bit_valid, which cares about where they live.
fn fields(&self) -> Vec<(&Visibility, TokenStream, &Type)>;
fn variants(&self) -> Vec<(Option<&Variant>, Vec<(&Visibility, TokenStream, &Type)>)>;
fn tag(&self) -> Option<Ident>;
}
impl DataExt for Data {
fn fields(&self) -> Vec<(&Visibility, TokenStream, &Type)> {
match self {
Data::Struct(strc) => strc.fields(),
Data::Enum(enm) => enm.fields(),
Data::Union(un) => un.fields(),
}
}
fn variants(&self) -> Vec<(Option<&Variant>, Vec<(&Visibility, TokenStream, &Type)>)> {
match self {
Data::Struct(strc) => strc.variants(),
Data::Enum(enm) => enm.variants(),
Data::Union(un) => un.variants(),
}
}
fn tag(&self) -> Option<Ident> {
match self {
Data::Struct(strc) => strc.tag(),
Data::Enum(enm) => enm.tag(),
Data::Union(un) => un.tag(),
}
}
}
impl DataExt for DataStruct {
fn fields(&self) -> Vec<(&Visibility, TokenStream, &Type)> {
map_fields(&self.fields)
}
fn variants(&self) -> Vec<(Option<&Variant>, Vec<(&Visibility, TokenStream, &Type)>)> {
vec![(None, self.fields())]
}
fn tag(&self) -> Option<Ident> {
None
}
}
impl DataExt for DataEnum {
fn fields(&self) -> Vec<(&Visibility, TokenStream, &Type)> {
map_fields(self.variants.iter().flat_map(|var| &var.fields))
}
fn variants(&self) -> Vec<(Option<&Variant>, Vec<(&Visibility, TokenStream, &Type)>)> {
self.variants.iter().map(|var| (Some(var), map_fields(&var.fields))).collect()
}
fn tag(&self) -> Option<Ident> {
Some(Ident::new("___ZerocopyTag", Span::call_site()))
}
}
impl DataExt for DataUnion {
fn fields(&self) -> Vec<(&Visibility, TokenStream, &Type)> {
map_fields(&self.fields.named)
}
fn variants(&self) -> Vec<(Option<&Variant>, Vec<(&Visibility, TokenStream, &Type)>)> {
vec![(None, self.fields())]
}
fn tag(&self) -> Option<Ident> {
None
}
}
fn map_fields<'a>(
fields: impl 'a + IntoIterator<Item = &'a Field>,
) -> Vec<(&'a Visibility, TokenStream, &'a Type)> {
fields
.into_iter()
.enumerate()
.map(|(idx, f)| {
(
&f.vis,
f.ident
.as_ref()
.map(ToTokens::to_token_stream)
.unwrap_or_else(|| Index::from(idx).to_token_stream()),
&f.ty,
)
})
.collect()
}
pub(crate) fn to_ident_str(t: &impl ToString) -> String {
let s = t.to_string();
if let Some(stripped) = s.strip_prefix("r#") {
stripped.to_string()
} else {
s
}
}
/// This enum describes what kind of padding check needs to be generated for the
/// associated impl.
pub(crate) enum PaddingCheck {
/// Check that the sum of the fields' sizes exactly equals the struct's
/// size.
Struct,
/// Check that a `repr(C)` struct has no padding.
ReprCStruct,
/// Check that the size of each field exactly equals the union's size.
Union,
/// Check that every variant of the enum contains no padding.
///
/// Because doing so requires a tag enum, this padding check requires an
/// additional `TokenStream` which defines the tag enum as `___ZerocopyTag`.
Enum { tag_type_definition: TokenStream },
}
impl PaddingCheck {
/// Returns the idents of the trait to use and the macro to call in order to
/// validate that a type passes the relevant padding check.
pub(crate) fn validator_trait_and_macro_idents(&self) -> (Ident, Ident) {
let (trt, mcro) = match self {
PaddingCheck::Struct => ("PaddingFree", "struct_padding"),
PaddingCheck::ReprCStruct => ("DynamicPaddingFree", "repr_c_struct_has_padding"),
PaddingCheck::Union => ("PaddingFree", "union_padding"),
PaddingCheck::Enum { .. } => ("PaddingFree", "enum_padding"),
};
let trt = Ident::new(trt, Span::call_site());
let mcro = Ident::new(mcro, Span::call_site());
(trt, mcro)
}
/// Sometimes performing the padding check requires some additional
/// "context" code. For enums, this is the definition of the tag enum.
pub(crate) fn validator_macro_context(&self) -> Option<&TokenStream> {
match self {
PaddingCheck::Struct | PaddingCheck::ReprCStruct | PaddingCheck::Union => None,
PaddingCheck::Enum { tag_type_definition } => Some(tag_type_definition),
}
}
}
#[derive(Clone)]
pub(crate) enum Trait {
KnownLayout,
HasTag,
HasField {
variant_id: Box<Expr>,
field: Box<Type>,
field_id: Box<Expr>,
},
ProjectField {
variant_id: Box<Expr>,
field: Box<Type>,
field_id: Box<Expr>,
invariants: Box<Type>,
},
Immutable,
TryFromBytes,
FromZeros,
FromBytes,
IntoBytes,
Unaligned,
Sized,
ByteHash,
ByteEq,
SplitAt,
}
impl ToTokens for Trait {
fn to_tokens(&self, tokens: &mut TokenStream) {
// According to [1], the format of the derived `Debug`` output is not
// stable and therefore not guaranteed to represent the variant names.
// Indeed with the (unstable) `fmt-debug` compiler flag [2], it can
// return only a minimalized output or empty string. To make sure this
// code will work in the future and independent of the compiler flag, we
// translate the variants to their names manually here.
//
// [1] https://doc.rust-lang.org/1.81.0/std/fmt/trait.Debug.html#stability
// [2] https://doc.rust-lang.org/beta/unstable-book/compiler-flags/fmt-debug.html
let s = match self {
Trait::HasField { .. } => "HasField",
Trait::ProjectField { .. } => "ProjectField",
Trait::KnownLayout => "KnownLayout",
Trait::HasTag => "HasTag",
Trait::Immutable => "Immutable",
Trait::TryFromBytes => "TryFromBytes",
Trait::FromZeros => "FromZeros",
Trait::FromBytes => "FromBytes",
Trait::IntoBytes => "IntoBytes",
Trait::Unaligned => "Unaligned",
Trait::Sized => "Sized",
Trait::ByteHash => "ByteHash",
Trait::ByteEq => "ByteEq",
Trait::SplitAt => "SplitAt",
};
let ident = Ident::new(s, Span::call_site());
let arguments: Option<syn::AngleBracketedGenericArguments> = match self {
Trait::HasField { variant_id, field, field_id } => {
Some(parse_quote!(<#field, #variant_id, #field_id>))
}
Trait::ProjectField { variant_id, field, field_id, invariants } => {
Some(parse_quote!(<#field, #invariants, #variant_id, #field_id>))
}
Trait::KnownLayout
| Trait::HasTag
| Trait::Immutable
| Trait::TryFromBytes
| Trait::FromZeros
| Trait::FromBytes
| Trait::IntoBytes
| Trait::Unaligned
| Trait::Sized
| Trait::ByteHash
| Trait::ByteEq
| Trait::SplitAt => None,
};
tokens.extend(quote!(#ident #arguments));
}
}
impl Trait {
pub(crate) fn crate_path(&self, ctx: &Ctx) -> Path {
let zerocopy_crate = &ctx.zerocopy_crate;
let core = ctx.core_path();
match self {
Self::Sized => parse_quote!(#core::marker::#self),
_ => parse_quote!(#zerocopy_crate::#self),
}
}
}
pub(crate) enum TraitBound {
Slf,
Other(Trait),
}
pub(crate) enum FieldBounds<'a> {
None,
All(&'a [TraitBound]),
Trailing(&'a [TraitBound]),
Explicit(Vec<WherePredicate>),
}
impl<'a> FieldBounds<'a> {
pub(crate) const ALL_SELF: FieldBounds<'a> = FieldBounds::All(&[TraitBound::Slf]);
pub(crate) const TRAILING_SELF: FieldBounds<'a> = FieldBounds::Trailing(&[TraitBound::Slf]);
}
pub(crate) enum SelfBounds<'a> {
None,
All(&'a [Trait]),
}
// FIXME(https://github.com/rust-lang/rust-clippy/issues/12908): This is a false
// positive. Explicit lifetimes are actually necessary here.
#[allow(clippy::needless_lifetimes)]
impl<'a> SelfBounds<'a> {
pub(crate) const SIZED: Self = Self::All(&[Trait::Sized]);
}
/// Normalizes a slice of bounds by replacing [`TraitBound::Slf`] with `slf`.
pub(crate) fn normalize_bounds<'a>(
slf: &'a Trait,
bounds: &'a [TraitBound],
) -> impl 'a + Iterator<Item = Trait> {
bounds.iter().map(move |bound| match bound {
TraitBound::Slf => slf.clone(),
TraitBound::Other(trt) => trt.clone(),
})
}
pub(crate) struct ImplBlockBuilder<'a> {
ctx: &'a Ctx,
data: &'a dyn DataExt,
trt: Trait,
field_type_trait_bounds: FieldBounds<'a>,
self_type_trait_bounds: SelfBounds<'a>,
padding_check: Option<PaddingCheck>,
param_extras: Vec<GenericParam>,
inner_extras: Option<TokenStream>,
outer_extras: Option<TokenStream>,
}
impl<'a> ImplBlockBuilder<'a> {
pub(crate) fn new(
ctx: &'a Ctx,
data: &'a dyn DataExt,
trt: Trait,
field_type_trait_bounds: FieldBounds<'a>,
) -> Self {
Self {
ctx,
data,
trt,
field_type_trait_bounds,
self_type_trait_bounds: SelfBounds::None,
padding_check: None,
param_extras: Vec::new(),
inner_extras: None,
outer_extras: None,
}
}
pub(crate) fn self_type_trait_bounds(mut self, self_type_trait_bounds: SelfBounds<'a>) -> Self {
self.self_type_trait_bounds = self_type_trait_bounds;
self
}
pub(crate) fn padding_check<P: Into<Option<PaddingCheck>>>(mut self, padding_check: P) -> Self {
self.padding_check = padding_check.into();
self
}
pub(crate) fn param_extras(mut self, param_extras: Vec<GenericParam>) -> Self {
self.param_extras.extend(param_extras);
self
}
pub(crate) fn inner_extras(mut self, inner_extras: TokenStream) -> Self {
self.inner_extras = Some(inner_extras);
self
}
pub(crate) fn outer_extras<T: Into<Option<TokenStream>>>(mut self, outer_extras: T) -> Self {
self.outer_extras = outer_extras.into();
self
}
pub(crate) fn build(self) -> TokenStream {
// In this documentation, we will refer to this hypothetical struct:
//
// #[derive(FromBytes)]
// struct Foo<T, I: Iterator>
// where
// T: Copy,
// I: Clone,
// I::Item: Clone,
// {
// a: u8,
// b: T,
// c: I::Item,
// }
//
// We extract the field types, which in this case are `u8`, `T`, and
// `I::Item`. We re-use the existing parameters and where clauses. If
// `require_trait_bound == true` (as it is for `FromBytes), we add where
// bounds for each field's type:
//
// impl<T, I: Iterator> FromBytes for Foo<T, I>
// where
// T: Copy,
// I: Clone,
// I::Item: Clone,
// T: FromBytes,
// I::Item: FromBytes,
// {
// }
//
// NOTE: It is standard practice to only emit bounds for the type
// parameters themselves, not for field types based on those parameters
// (e.g., `T` vs `T::Foo`). For a discussion of why this is standard
// practice, see https://github.com/rust-lang/rust/issues/26925.
//
// The reason we diverge from this standard is that doing it that way
// for us would be unsound. E.g., consider a type, `T` where `T:
// FromBytes` but `T::Foo: !FromBytes`. It would not be sound for us to
// accept a type with a `T::Foo` field as `FromBytes` simply because `T:
// FromBytes`.
//
// While there's no getting around this requirement for us, it does have
// the pretty serious downside that, when lifetimes are involved, the
// trait solver ties itself in knots:
//
// #[derive(Unaligned)]
// #[repr(C)]
// struct Dup<'a, 'b> {
// a: PhantomData<&'a u8>,
// b: PhantomData<&'b u8>,
// }
//
// error[E0283]: type annotations required: cannot resolve `core::marker::PhantomData<&'a u8>: zerocopy::Unaligned`
// --> src/main.rs:6:10
// |
// 6 | #[derive(Unaligned)]
// | ^^^^^^^^^
// |
// = note: required by `zerocopy::Unaligned`
let type_ident = &self.ctx.ast.ident;
let trait_path = self.trt.crate_path(self.ctx);
let fields = self.data.fields();
let variants = self.data.variants();
let tag = self.data.tag();
let zerocopy_crate = &self.ctx.zerocopy_crate;
fn bound_tt(ty: &Type, traits: impl Iterator<Item = Trait>, ctx: &Ctx) -> WherePredicate {
let traits = traits.map(|t| t.crate_path(ctx));
parse_quote!(#ty: #(#traits)+*)
}
let field_type_bounds: Vec<_> = match (self.field_type_trait_bounds, &fields[..]) {
(FieldBounds::All(traits), _) => fields
.iter()
.map(|(_vis, _name, ty)| {
bound_tt(ty, normalize_bounds(&self.trt, traits), self.ctx)
})
.collect(),
(FieldBounds::None, _) | (FieldBounds::Trailing(..), []) => vec![],
(FieldBounds::Trailing(traits), [.., last]) => {
vec![bound_tt(last.2, normalize_bounds(&self.trt, traits), self.ctx)]
}
(FieldBounds::Explicit(bounds), _) => bounds,
};
let padding_check_bound = self
.padding_check
.map(|check| {
// Parse the repr for `align` and `packed` modifiers. Note that
// `Repr::<PrimitiveRepr, NonZeroU32>` is more permissive than
// what Rust supports for structs, enums, or unions, and thus
// reliably extracts these modifiers for any kind of type.
let repr =
Repr::<PrimitiveRepr, NonZeroU32>::from_attrs(&self.ctx.ast.attrs).unwrap();
let core = self.ctx.core_path();
let option = quote! { #core::option::Option };
let nonzero = quote! { #core::num::NonZeroUsize };
let none = quote! { #option::None::<#nonzero> };
let repr_align =
repr.get_align().map(|spanned| {
let n = spanned.t.get();
quote_spanned! { spanned.span => (#nonzero::new(#n as usize)) }
}).unwrap_or(quote! { (#none) });
let repr_packed =
repr.get_packed().map(|packed| {
let n = packed.get();
quote! { (#nonzero::new(#n as usize)) }
}).unwrap_or(quote! { (#none) });
let variant_types = variants.iter().map(|(_, fields)| {
let types = fields.iter().map(|(_vis, _name, ty)| ty);
quote!([#((#types)),*])
});
let validator_context = check.validator_macro_context();
let (trt, validator_macro) = check.validator_trait_and_macro_idents();
let t = tag.iter();
parse_quote! {
(): #zerocopy_crate::util::macro_util::#trt<
Self,
{
#validator_context
#zerocopy_crate::#validator_macro!(Self, #repr_align, #repr_packed, #(#t,)* #(#variant_types),*)
}
>
}
});
let self_bounds: Option<WherePredicate> = match self.self_type_trait_bounds {
SelfBounds::None => None,
SelfBounds::All(traits) => {
Some(bound_tt(&parse_quote!(Self), traits.iter().cloned(), self.ctx))
}
};
let bounds = self
.ctx
.ast
.generics
.where_clause
.as_ref()
.map(|where_clause| where_clause.predicates.iter())
.into_iter()
.flatten()
.chain(field_type_bounds.iter())
.chain(padding_check_bound.iter())
.chain(self_bounds.iter());
// The parameters with trait bounds, but without type defaults.
let mut params: Vec<_> = self
.ctx
.ast
.generics
.params
.clone()
.into_iter()
.map(|mut param| {
match &mut param {
GenericParam::Type(ty) => ty.default = None,
GenericParam::Const(cnst) => cnst.default = None,
GenericParam::Lifetime(_) => {}
}
parse_quote!(#param)
})
.chain(self.param_extras)
.collect();
// For MSRV purposes, ensure that lifetimes precede types precede const
// generics.
params.sort_by_cached_key(|param| match param {
GenericParam::Lifetime(_) => 0,
GenericParam::Type(_) => 1,
GenericParam::Const(_) => 2,
});
// The identifiers of the parameters without trait bounds or type
// defaults.
let param_idents = self.ctx.ast.generics.params.iter().map(|param| match param {
GenericParam::Type(ty) => {
let ident = &ty.ident;
quote!(#ident)
}
GenericParam::Lifetime(l) => {
let ident = &l.lifetime;
quote!(#ident)
}
GenericParam::Const(cnst) => {
let ident = &cnst.ident;
quote!({#ident})
}
});
let inner_extras = self.inner_extras;
let allow_trivial_bounds =
if self.ctx.skip_on_error { quote!(#[allow(trivial_bounds)]) } else { quote!() };
let impl_tokens = quote! {
#allow_trivial_bounds
unsafe impl < #(#params),* > #trait_path for #type_ident < #(#param_idents),* >
where
#(#bounds,)*
{
fn only_derive_is_allowed_to_implement_this_trait() {}
#inner_extras
}
};
let outer_extras = self.outer_extras.filter(|e| !e.is_empty());
let cfg_compile_error = self.ctx.cfg_compile_error();
const_block([Some(cfg_compile_error), Some(impl_tokens), outer_extras])
}
}
// A polyfill for `Option::then_some`, which was added after our MSRV.
//
// The `#[allow(unused)]` is necessary because, on sufficiently recent toolchain
// versions, `b.then_some(...)` resolves to the inherent method rather than to
// this trait, and so this trait is considered unused.
//
// FIXME(#67): Remove this once our MSRV is >= 1.62.
#[allow(unused)]
trait BoolExt {
fn then_some<T>(self, t: T) -> Option<T>;
}
impl BoolExt for bool {
fn then_some<T>(self, t: T) -> Option<T> {
if self {
Some(t)
} else {
None
}
}
}
pub(crate) fn const_block(items: impl IntoIterator<Item = Option<TokenStream>>) -> TokenStream {
let items = items.into_iter().flatten();
quote! {
#[allow(
// FIXME(#553): Add a test that generates a warning when
// `#[allow(deprecated)]` isn't present.
deprecated,
// Required on some rustc versions due to a lint that is only
// triggered when `derive(KnownLayout)` is applied to `repr(C)`
// structs that are generated by macros. See #2177 for details.
private_bounds,
non_local_definitions,
non_camel_case_types,
non_upper_case_globals,
non_snake_case,
non_ascii_idents,
clippy::missing_inline_in_public_items,
)]
#[deny(ambiguous_associated_items)]
// While there are not currently any warnings that this suppresses
// (that we're aware of), it's good future-proofing hygiene.
#[automatically_derived]
const _: () = {
#(#items)*
};
}
}
pub(crate) fn generate_tag_enum(ctx: &Ctx, repr: &EnumRepr, data: &DataEnum) -> TokenStream {
let zerocopy_crate = &ctx.zerocopy_crate;
let variants = data.variants.iter().map(|v| {
let ident = &v.ident;
if let Some((eq, discriminant)) = &v.discriminant {
quote! { #ident #eq #discriminant }
} else {
quote! { #ident }
}
});
// Don't include any `repr(align)` when generating the tag enum, as that
// could add padding after the tag but before any variants, which is not the
// correct behavior.
let repr = match repr {
EnumRepr::Transparent(span) => quote::quote_spanned! { *span => #[repr(transparent)] },
EnumRepr::Compound(c, _) => quote! { #c },
};
quote! {
#repr
#[allow(dead_code)]
pub enum ___ZerocopyTag {
#(#variants,)*
}
// SAFETY: `___ZerocopyTag` has no fields, and so it does not permit
// interior mutation.
unsafe impl #zerocopy_crate::Immutable for ___ZerocopyTag {
fn only_derive_is_allowed_to_implement_this_trait() {}
}
}
}
pub(crate) fn enum_size_from_repr(repr: &EnumRepr) -> Result<usize, Error> {
use CompoundRepr::*;
use PrimitiveRepr::*;
use Repr::*;
match repr {
Transparent(span)
| Compound(
Spanned {
t: C | Rust | Primitive(U32 | I32 | U64 | I64 | U128 | I128 | Usize | Isize),
span,
},
_,
) => Err(Error::new(
*span,
"`FromBytes` only supported on enums with `#[repr(...)]` attributes `u8`, `i8`, `u16`, or `i16`",
)),
Compound(Spanned { t: Primitive(U8 | I8), span: _ }, _align) => Ok(8),
Compound(Spanned { t: Primitive(U16 | I16), span: _ }, _align) => Ok(16),
}
}
#[cfg(test)]
pub(crate) mod testutil {
use proc_macro2::TokenStream;
use syn::visit::{self, Visit};
/// Checks for hygiene violations in the generated code.
///
/// # Panics
///
/// Panics if a hygiene violation is found.
pub(crate) fn check_hygiene(ts: TokenStream) {
struct AmbiguousItemVisitor;
impl<'ast> Visit<'ast> for AmbiguousItemVisitor {
fn visit_path(&mut self, i: &'ast syn::Path) {
if i.segments.len() > 1 && i.segments.first().unwrap().ident == "Self" {
panic!(
"Found ambiguous path `{}` in generated output. \
All associated item access must be fully qualified (e.g., `<Self as Trait>::Item`) \
to prevent hygiene issues.",
quote::quote!(#i)
);
}
visit::visit_path(self, i);
}
}
let file = syn::parse2::<syn::File>(ts).expect("failed to parse generated output as File");
AmbiguousItemVisitor.visit_file(&file);
}
#[test]
fn test_check_hygiene_success() {
check_hygiene(quote::quote! {
fn foo() {
let _ = <Self as Trait>::Item;
}
});
}
#[test]
#[should_panic(expected = "Found ambiguous path `Self :: Ambiguous`")]
fn test_check_hygiene_failure() {
check_hygiene(quote::quote! {
fn foo() {
let _ = Self::Ambiguous;
}
});
}
}

14
rust/zerocopy/README.md Normal file
View File

@ -0,0 +1,14 @@
# `zerocopy`
These source files come from the Rust `zerocopy` crate, version v0.8.50
(released 2026-05-31), hosted in the <https://github.com/google/zerocopy>
repository, licensed under "BSD-2-Clause OR Apache-2.0 OR MIT" and only
modified to add the SPDX license identifiers and to remove `Display`
for `f32` and `f64`.
For copyright details, please see:
https://github.com/google/zerocopy/blob/v0.8.50/README.md?plain=1
https://github.com/google/zerocopy/blob/v0.8.50/LICENSE-BSD
https://github.com/google/zerocopy/blob/v0.8.50/LICENSE-APACHE
https://github.com/google/zerocopy/blob/v0.8.50/LICENSE-MIT

9
rust/zerocopy/benches/as_bytes_dynamic_size.rs Normal file
View File

@ -0,0 +1,9 @@
use zerocopy::*;
#[path = "formats/coco_dynamic_size.rs"]
mod format;
#[unsafe(no_mangle)]
fn bench_as_bytes_dynamic_size(source: &format::CocoPacket) -> &[u8] {
source.as_bytes()
}

5
rust/zerocopy/benches/as_bytes_dynamic_size.x86-64 Normal file
View File

@ -0,0 +1,5 @@
bench_as_bytes_dynamic_size:
mov rax, rdi
lea rdx, [2*rsi + 5]
and rdx, -2
ret

47
rust/zerocopy/benches/as_bytes_dynamic_size.x86-64.mca Normal file
View File

@ -0,0 +1,47 @@
Iterations: 100
Instructions: 400
Total Cycles: 137
Total uOps: 400
Dispatch Width: 4
uOps Per Cycle: 2.92
IPC: 2.92
Block RThroughput: 1.0
Instruction Info:
[1]: #uOps
[2]: Latency
[3]: RThroughput
[4]: MayLoad
[5]: MayStore
[6]: HasSideEffects (U)
[1] [2] [3] [4] [5] [6] Instructions:
1 1 0.33 mov rax, rdi
1 1 0.50 lea rdx, [2*rsi + 5]
1 1 0.33 and rdx, -2
1 1 1.00 U ret
Resources:
[0] - SBDivider
[1] - SBFPDivider
[2] - SBPort0
[3] - SBPort1
[4] - SBPort4
[5] - SBPort5
[6.0] - SBPort23
[6.1] - SBPort23
Resource pressure per iteration:
[0] [1] [2] [3] [4] [5] [6.0] [6.1]
- - 1.33 1.33 - 1.34 - -
Resource pressure by instruction:
[0] [1] [2] [3] [4] [5] [6.0] [6.1] Instructions:
- - - 0.66 - 0.34 - - mov rax, rdi
- - 0.33 0.67 - - - - lea rdx, [2*rsi + 5]
- - 1.00 - - - - - and rdx, -2
- - - - - 1.00 - - ret

9
rust/zerocopy/benches/as_bytes_static_size.rs Normal file
View File

@ -0,0 +1,9 @@
use zerocopy::*;
#[path = "formats/coco_static_size.rs"]
mod format;
#[unsafe(no_mangle)]
fn bench_as_bytes_static_size(source: &format::CocoPacket) -> &[u8] {
source.as_bytes()
}

4
rust/zerocopy/benches/as_bytes_static_size.x86-64 Normal file
View File

@ -0,0 +1,4 @@
bench_as_bytes_static_size:
mov rax, rdi
mov edx, 6
ret

45
rust/zerocopy/benches/as_bytes_static_size.x86-64.mca Normal file
View File

@ -0,0 +1,45 @@
Iterations: 100
Instructions: 300
Total Cycles: 104
Total uOps: 300
Dispatch Width: 4
uOps Per Cycle: 2.88
IPC: 2.88
Block RThroughput: 1.0
Instruction Info:
[1]: #uOps
[2]: Latency
[3]: RThroughput
[4]: MayLoad
[5]: MayStore
[6]: HasSideEffects (U)
[1] [2] [3] [4] [5] [6] Instructions:
1 1 0.33 mov rax, rdi
1 1 0.33 mov edx, 6
1 1 1.00 U ret
Resources:
[0] - SBDivider
[1] - SBFPDivider
[2] - SBPort0
[3] - SBPort1
[4] - SBPort4
[5] - SBPort5
[6.0] - SBPort23
[6.1] - SBPort23
Resource pressure per iteration:
[0] [1] [2] [3] [4] [5] [6.0] [6.1]
- - 0.99 1.00 - 1.01 - -
Resource pressure by instruction:
[0] [1] [2] [3] [4] [5] [6.0] [6.1] Instructions:
- - 0.99 - - 0.01 - - mov rax, rdi
- - - 1.00 - - - - mov edx, 6
- - - - - 1.00 - - ret

9
rust/zerocopy/benches/extend_vec_zeroed.rs Normal file
View File

@ -0,0 +1,9 @@
use zerocopy::*;
#[path = "formats/coco_static_size.rs"]
mod format;
#[unsafe(no_mangle)]
fn bench_extend_vec_zeroed(v: &mut Vec<format::LocoPacket>, additional: usize) -> Option<()> {
FromZeros::extend_vec_zeroed(v, additional).ok()
}

60
rust/zerocopy/benches/extend_vec_zeroed.x86-64 Normal file
View File

@ -0,0 +1,60 @@
bench_extend_vec_zeroed:
push r15
push r14
push r13
push r12
push rbx
sub rsp, 32
mov rbx, rdi
mov rax, qword ptr [rdi]
mov r12, qword ptr [rdi + 16]
mov rcx, rax
sub rcx, r12
cmp rsi, rcx
jbe .LBB6_3
mov r15, r12
add r15, rsi
jae .LBB6_6
.LBB6_2:
xor eax, eax
jmp .LBB6_5
.LBB6_3:
mov rax, qword ptr [rbx + 8]
lea r15, [r12 + rsi]
.LBB6_4:
lea rcx, [r12 + 2*r12]
lea rdi, [rax + 2*rcx]
add rsi, rsi
lea rdx, [rsi + 2*rsi]
xor esi, esi
call qword ptr [rip + memset@GOTPCREL]
mov qword ptr [rbx + 16], r15
mov al, 1
.LBB6_5:
add rsp, 32
pop rbx
pop r12
pop r13
pop r14
pop r15
ret
.LBB6_6:
mov r13, rsi
lea rcx, [rax + rax]
cmp r15, rcx
cmova rcx, r15
cmp rcx, 5
mov r14d, 4
cmovae r14, rcx
mov rdx, qword ptr [rbx + 8]
lea rdi, [rsp + 8]
mov rsi, rax
mov rcx, r14
call <alloc::raw_vec::RawVecInner>::finish_grow
cmp dword ptr [rsp + 8], 1
je .LBB6_2
mov rax, qword ptr [rsp + 16]
mov qword ptr [rbx + 8], rax
mov qword ptr [rbx], r14
mov rsi, r13
jmp .LBB6_4

147
rust/zerocopy/benches/extend_vec_zeroed.x86-64.mca Normal file
View File

@ -0,0 +1,147 @@
Iterations: 100
Instructions: 5400
Total Cycles: 6595
Total uOps: 6800
Dispatch Width: 4
uOps Per Cycle: 1.03
IPC: 0.82
Block RThroughput: 17.0
Instruction Info:
[1]: #uOps
[2]: Latency
[3]: RThroughput
[4]: MayLoad
[5]: MayStore
[6]: HasSideEffects (U)
[1] [2] [3] [4] [5] [6] Instructions:
2 5 1.00 * push r15
2 5 1.00 * push r14
2 5 1.00 * push r13
2 5 1.00 * push r12
2 5 1.00 * push rbx
1 1 0.33 sub rsp, 32
1 1 0.33 mov rbx, rdi
1 5 0.50 * mov rax, qword ptr [rdi]
1 5 0.50 * mov r12, qword ptr [rdi + 16]
1 1 0.33 mov rcx, rax
1 1 0.33 sub rcx, r12
1 1 0.33 cmp rsi, rcx
1 1 1.00 jbe .LBB6_3
1 1 0.33 mov r15, r12
1 1 0.33 add r15, rsi
1 1 1.00 jae .LBB6_6
1 0 0.25 xor eax, eax
1 1 1.00 jmp .LBB6_5
1 5 0.50 * mov rax, qword ptr [rbx + 8]
1 1 0.50 lea r15, [r12 + rsi]
1 1 0.50 lea rcx, [r12 + 2*r12]
1 1 0.50 lea rdi, [rax + 2*rcx]
1 1 0.33 add rsi, rsi
1 1 0.50 lea rdx, [rsi + 2*rsi]
1 0 0.25 xor esi, esi
4 7 1.00 * call qword ptr [rip + memset@GOTPCREL]
1 1 1.00 * mov qword ptr [rbx + 16], r15
1 1 0.33 mov al, 1
1 1 0.33 add rsp, 32
1 6 0.50 * pop rbx
1 6 0.50 * pop r12
1 6 0.50 * pop r13
1 6 0.50 * pop r14
1 6 0.50 * pop r15
1 1 1.00 U ret
1 1 0.33 mov r13, rsi
1 1 0.50 lea rcx, [rax + rax]
1 1 0.33 cmp r15, rcx
3 3 1.00 cmova rcx, r15
1 1 0.33 cmp rcx, 5
1 1 0.33 mov r14d, 4
2 2 0.67 cmovae r14, rcx
1 5 0.50 * mov rdx, qword ptr [rbx + 8]
1 1 0.50 lea rdi, [rsp + 8]
1 1 0.33 mov rsi, rax
1 1 0.33 mov rcx, r14
3 5 1.00 call <alloc::raw_vec::RawVecInner>::finish_grow
2 6 0.50 * cmp dword ptr [rsp + 8], 1
1 1 1.00 je .LBB6_2
1 5 0.50 * mov rax, qword ptr [rsp + 16]
1 1 1.00 * mov qword ptr [rbx + 8], rax
1 1 1.00 * mov qword ptr [rbx], r14
1 1 0.33 mov rsi, r13
1 1 1.00 jmp .LBB6_4
Resources:
[0] - SBDivider
[1] - SBFPDivider
[2] - SBPort0
[3] - SBPort1
[4] - SBPort4
[5] - SBPort5
[6.0] - SBPort23
[6.1] - SBPort23
Resource pressure per iteration:
[0] [1] [2] [3] [4] [5] [6.0] [6.1]
- - 12.00 12.00 10.00 13.00 11.00 11.00
Resource pressure by instruction:
[0] [1] [2] [3] [4] [5] [6.0] [6.1] Instructions:
- - - - 1.00 - 0.49 0.51 push r15
- - - - 1.00 - 0.51 0.49 push r14
- - - - 1.00 - 0.50 0.50 push r13
- - - - 1.00 - 0.50 0.50 push r12
- - - - 1.00 - 0.50 0.50 push rbx
- - 0.01 0.99 - - - - sub rsp, 32
- - - - - 1.00 - - mov rbx, rdi
- - - - - - 0.50 0.50 mov rax, qword ptr [rdi]
- - - - - - 0.50 0.50 mov r12, qword ptr [rdi + 16]
- - - 1.00 - - - - mov rcx, rax
- - - 0.99 - 0.01 - - sub rcx, r12
- - - - - 1.00 - - cmp rsi, rcx
- - - - - 1.00 - - jbe .LBB6_3
- - 0.01 0.98 - 0.01 - - mov r15, r12
- - 0.99 0.01 - - - - add r15, rsi
- - - - - 1.00 - - jae .LBB6_6
- - - - - - - - xor eax, eax
- - - - - 1.00 - - jmp .LBB6_5
- - - - - - 0.50 0.50 mov rax, qword ptr [rbx + 8]
- - 1.00 - - - - - lea r15, [r12 + rsi]
- - 0.98 0.02 - - - - lea rcx, [r12 + 2*r12]
- - 0.99 0.01 - - - - lea rdi, [rax + 2*rcx]
- - - 1.00 - - - - add rsi, rsi
- - 0.99 0.01 - - - - lea rdx, [rsi + 2*rsi]
- - - - - - - - xor esi, esi
- - - - 1.00 1.00 1.00 1.00 call qword ptr [rip + memset@GOTPCREL]
- - - - 1.00 - 0.50 0.50 mov qword ptr [rbx + 16], r15
- - 0.01 0.99 - - - - mov al, 1
- - 1.00 - - - - - add rsp, 32
- - - - - - 0.50 0.50 pop rbx
- - - - - - 0.50 0.50 pop r12
- - - - - - 0.50 0.50 pop r13
- - - - - - 0.50 0.50 pop r14
- - - - - - 0.50 0.50 pop r15
- - - - - 1.00 - - ret
- - 1.00 - - - - - mov r13, rsi
- - 0.01 0.99 - - - - lea rcx, [rax + rax]
- - 0.99 0.01 - - - - cmp r15, rcx
- - 2.00 0.01 - 0.99 - - cmova rcx, r15
- - 0.01 0.99 - - - - cmp rcx, 5
- - 0.01 0.99 - - - - mov r14d, 4
- - 1.00 0.01 - 0.99 - - cmovae r14, rcx
- - - - - - 0.50 0.50 mov rdx, qword ptr [rbx + 8]
- - 0.01 0.99 - - - - lea rdi, [rsp + 8]
- - - 1.00 - - - - mov rsi, rax
- - - 0.01 - 0.99 - - mov rcx, r14
- - - - 1.00 1.00 0.50 0.50 call <alloc::raw_vec::RawVecInner>::finish_grow
- - - 0.99 - 0.01 0.50 0.50 cmp dword ptr [rsp + 8], 1
- - - - - 1.00 - - je .LBB6_2
- - - - - - 0.50 0.50 mov rax, qword ptr [rsp + 16]
- - - - 1.00 - 0.49 0.51 mov qword ptr [rbx + 8], rax
- - - - 1.00 - 0.51 0.49 mov qword ptr [rbx], r14
- - 0.99 0.01 - - - - mov rsi, r13
- - - - - 1.00 - - jmp .LBB6_4

24
rust/zerocopy/benches/formats/coco_dynamic_padding.rs Normal file
View File

@ -0,0 +1,24 @@
use zerocopy_derive::*;
// The only valid value of this type are the bytes `0xC0C0`.
#[derive(TryFromBytes, KnownLayout, Immutable)]
#[repr(u16)]
pub enum C0C0 {
_XC0C0 = 0xC0C0,
}
#[derive(FromBytes, KnownLayout, Immutable, SplitAt)]
#[repr(C, align(4))]
pub struct Packet<Magic> {
magic_number: Magic,
milk: u8,
mug_size: u8,
temperature: [u8; 5],
marshmallows: [[u8; 3]],
}
/// A packet begining with the magic number `0xC0C0`.
pub type CocoPacket = Packet<C0C0>;
/// A packet beginning with any two initialized bytes.
pub type LocoPacket = Packet<[u8; 2]>;

27
rust/zerocopy/benches/formats/coco_dynamic_size.rs Normal file
View File

@ -0,0 +1,27 @@
use zerocopy_derive::*;
// The only valid value of this type are the bytes `0xC0C0`.
#[derive(TryFromBytes, KnownLayout, Immutable, IntoBytes)]
#[repr(u16)]
pub enum C0C0 {
_XC0C0 = 0xC0C0,
}
macro_rules! define_packet {
($name: ident, $trait: ident, $leading_field: ty) => {
#[derive($trait, KnownLayout, Immutable, IntoBytes, SplitAt)]
#[repr(C, align(2))]
pub struct $name {
magic_number: $leading_field,
mug_size: u8,
temperature: u8,
marshmallows: [[u8; 2]],
}
};
}
/// Packet begins with bytes 0xC0C0.
define_packet!(CocoPacket, TryFromBytes, C0C0);
/// Packet begins with any two bytes.
define_packet!(LocoPacket, FromBytes, [u8; 2]);

27
rust/zerocopy/benches/formats/coco_static_size.rs Normal file
View File

@ -0,0 +1,27 @@
use zerocopy_derive::*;
// The only valid value of this type are the bytes `0xC0C0`.
#[derive(TryFromBytes, KnownLayout, Immutable, IntoBytes)]
#[repr(u16)]
pub enum C0C0 {
_XC0C0 = 0xC0C0,
}
macro_rules! define_packet {
($name: ident, $trait: ident, $leading_field: ty) => {
#[derive($trait, KnownLayout, Immutable, IntoBytes)]
#[repr(C, align(2))]
pub struct $name {
magic_number: $leading_field,
mug_size: u8,
temperature: u8,
marshmallows: [u8; 2],
}
};
}
/// Packet begins with bytes 0xC0C0.
define_packet!(CocoPacket, TryFromBytes, C0C0);
/// Packet begins with any two bytes.
define_packet!(LocoPacket, FromBytes, [u8; 2]);

13
rust/zerocopy/benches/insert_vec_zeroed.rs Normal file
View File

@ -0,0 +1,13 @@
use zerocopy::*;
#[path = "formats/coco_static_size.rs"]
mod format;
#[unsafe(no_mangle)]
fn bench_insert_vec_zeroed(
v: &mut Vec<format::LocoPacket>,
position: usize,
additional: usize,
) -> Option<()> {
FromZeros::insert_vec_zeroed(v, position, additional).ok()
}

79
rust/zerocopy/benches/insert_vec_zeroed.x86-64 Normal file
View File

@ -0,0 +1,79 @@
bench_insert_vec_zeroed:
push rbp
push r15
push r14
push r13
push r12
push rbx
sub rsp, 24
mov r12, qword ptr [rdi + 16]
mov r13, r12
sub r13, rsi
jb .LBB6_10
mov rbx, rdi
mov rax, qword ptr [rdi]
mov rcx, rax
sub rcx, r12
cmp rdx, rcx
jbe .LBB6_4
add r12, rdx
jae .LBB6_7
.LBB6_3:
xor eax, eax
jmp .LBB6_6
.LBB6_4:
mov rax, qword ptr [rbx + 8]
add r12, rdx
.LBB6_5:
lea rcx, [rsi + 2*rsi]
lea r14, [rax + 2*rcx]
add rdx, rdx
lea r15, [rdx + 2*rdx]
lea rdi, [r14 + r15]
add r13, r13
lea rdx, [2*r13]
add rdx, r13
mov rsi, r14
call qword ptr [rip + memmove@GOTPCREL]
mov rdi, r14
xor esi, esi
mov rdx, r15
call qword ptr [rip + memset@GOTPCREL]
mov qword ptr [rbx + 16], r12
mov al, 1
.LBB6_6:
add rsp, 24
pop rbx
pop r12
pop r13
pop r14
pop r15
pop rbp
ret
.LBB6_7:
mov r15, rsi
mov rbp, rdx
lea rcx, [rax + rax]
cmp r12, rcx
cmova rcx, r12
cmp rcx, 5
mov r14d, 4
cmovae r14, rcx
mov rdx, qword ptr [rbx + 8]
mov rdi, rsp
mov rsi, rax
mov rcx, r14
call <alloc::raw_vec::RawVecInner>::finish_grow
cmp dword ptr [rsp], 1
je .LBB6_3
mov rax, qword ptr [rsp + 8]
mov qword ptr [rbx + 8], rax
mov qword ptr [rbx], r14
mov rdx, rbp
mov rsi, r15
jmp .LBB6_5
.LBB6_10:
lea rdi, [rip + .Lanon.HASH.1]
lea rdx, [rip + .Lanon.HASH.3]
mov esi, 37
call qword ptr [rip + core::panicking::panic@GOTPCREL]

183
rust/zerocopy/benches/insert_vec_zeroed.x86-64.mca Normal file
View File

@ -0,0 +1,183 @@
Iterations: 100
Instructions: 7200
Total Cycles: 7648
Total uOps: 9300
Dispatch Width: 4
uOps Per Cycle: 1.22
IPC: 0.94
Block RThroughput: 23.3
Instruction Info:
[1]: #uOps
[2]: Latency
[3]: RThroughput
[4]: MayLoad
[5]: MayStore
[6]: HasSideEffects (U)
[1] [2] [3] [4] [5] [6] Instructions:
2 5 1.00 * push rbp
2 5 1.00 * push r15
2 5 1.00 * push r14
2 5 1.00 * push r13
2 5 1.00 * push r12
2 5 1.00 * push rbx
1 1 0.33 sub rsp, 24
1 5 0.50 * mov r12, qword ptr [rdi + 16]
1 1 0.33 mov r13, r12
1 1 0.33 sub r13, rsi
1 1 1.00 jb .LBB6_10
1 1 0.33 mov rbx, rdi
1 5 0.50 * mov rax, qword ptr [rdi]
1 1 0.33 mov rcx, rax
1 1 0.33 sub rcx, r12
1 1 0.33 cmp rdx, rcx
1 1 1.00 jbe .LBB6_4
1 1 0.33 add r12, rdx
1 1 1.00 jae .LBB6_7
1 0 0.25 xor eax, eax
1 1 1.00 jmp .LBB6_6
1 5 0.50 * mov rax, qword ptr [rbx + 8]
1 1 0.33 add r12, rdx
1 1 0.50 lea rcx, [rsi + 2*rsi]
1 1 0.50 lea r14, [rax + 2*rcx]
1 1 0.33 add rdx, rdx
1 1 0.50 lea r15, [rdx + 2*rdx]
1 1 0.50 lea rdi, [r14 + r15]
1 1 0.33 add r13, r13
1 1 0.50 lea rdx, [2*r13]
1 1 0.33 add rdx, r13
1 1 0.33 mov rsi, r14
4 7 1.00 * call qword ptr [rip + memmove@GOTPCREL]
1 1 0.33 mov rdi, r14
1 0 0.25 xor esi, esi
1 1 0.33 mov rdx, r15
4 7 1.00 * call qword ptr [rip + memset@GOTPCREL]
1 1 1.00 * mov qword ptr [rbx + 16], r12
1 1 0.33 mov al, 1
1 1 0.33 add rsp, 24
1 6 0.50 * pop rbx
1 6 0.50 * pop r12
1 6 0.50 * pop r13
1 6 0.50 * pop r14
1 6 0.50 * pop r15
1 6 0.50 * pop rbp
1 1 1.00 U ret
1 1 0.33 mov r15, rsi
1 1 0.33 mov rbp, rdx
1 1 0.50 lea rcx, [rax + rax]
1 1 0.33 cmp r12, rcx
3 3 1.00 cmova rcx, r12
1 1 0.33 cmp rcx, 5
1 1 0.33 mov r14d, 4
2 2 0.67 cmovae r14, rcx
1 5 0.50 * mov rdx, qword ptr [rbx + 8]
1 1 0.33 mov rdi, rsp
1 1 0.33 mov rsi, rax
1 1 0.33 mov rcx, r14
3 5 1.00 call <alloc::raw_vec::RawVecInner>::finish_grow
2 6 0.50 * cmp dword ptr [rsp], 1
1 1 1.00 je .LBB6_3
1 5 0.50 * mov rax, qword ptr [rsp + 8]
1 1 1.00 * mov qword ptr [rbx + 8], rax
1 1 1.00 * mov qword ptr [rbx], r14
1 1 0.33 mov rdx, rbp
1 1 0.33 mov rsi, r15
1 1 1.00 jmp .LBB6_5
1 1 0.50 lea rdi, [rip + .Lanon.HASH.1]
1 1 0.50 lea rdx, [rip + .Lanon.HASH.3]
1 1 0.33 mov esi, 37
4 7 1.00 * call qword ptr [rip + core::panicking::panic@GOTPCREL]
Resources:
[0] - SBDivider
[1] - SBFPDivider
[2] - SBPort0
[3] - SBPort1
[4] - SBPort4
[5] - SBPort5
[6.0] - SBPort23
[6.1] - SBPort23
Resource pressure per iteration:
[0] [1] [2] [3] [4] [5] [6.0] [6.1]
- - 17.02 16.50 13.00 19.48 14.00 14.00
Resource pressure by instruction:
[0] [1] [2] [3] [4] [5] [6.0] [6.1] Instructions:
- - - - 1.00 - 0.98 0.02 push rbp
- - - - 1.00 - 0.02 0.98 push r15
- - - - 1.00 - 0.99 0.01 push r14
- - - - 1.00 - 0.01 0.99 push r13
- - - - 1.00 - 0.99 0.01 push r12
- - - - 1.00 - 0.01 0.99 push rbx
- - 0.49 0.51 - - - - sub rsp, 24
- - - - - - 0.04 0.96 mov r12, qword ptr [rdi + 16]
- - 0.49 0.50 - 0.01 - - mov r13, r12
- - 0.48 0.51 - 0.01 - - sub r13, rsi
- - - - - 1.00 - - jb .LBB6_10
- - 0.49 0.49 - 0.02 - - mov rbx, rdi
- - - - - - 0.97 0.03 mov rax, qword ptr [rdi]
- - 0.51 0.49 - - - - mov rcx, rax
- - 0.49 0.02 - 0.49 - - sub rcx, r12
- - 0.49 0.50 - 0.01 - - cmp rdx, rcx
- - - - - 1.00 - - jbe .LBB6_4
- - 0.02 0.49 - 0.49 - - add r12, rdx
- - - - - 1.00 - - jae .LBB6_7
- - - - - - - - xor eax, eax
- - - - - 1.00 - - jmp .LBB6_6
- - - - - - 0.97 0.03 mov rax, qword ptr [rbx + 8]
- - 0.51 0.49 - - - - add r12, rdx
- - 0.49 0.51 - - - - lea rcx, [rsi + 2*rsi]
- - 0.50 0.50 - - - - lea r14, [rax + 2*rcx]
- - 0.51 0.49 - - - - add rdx, rdx
- - 0.50 0.50 - - - - lea r15, [rdx + 2*rdx]
- - 0.49 0.51 - - - - lea rdi, [r14 + r15]
- - 0.50 0.49 - 0.01 - - add r13, r13
- - 0.51 0.49 - - - - lea rdx, [2*r13]
- - 0.01 0.01 - 0.98 - - add rdx, r13
- - 0.01 - - 0.99 - - mov rsi, r14
- - - - 1.00 1.00 1.98 0.02 call qword ptr [rip + memmove@GOTPCREL]
- - 0.49 0.50 - 0.01 - - mov rdi, r14
- - - - - - - - xor esi, esi
- - 0.50 0.50 - - - - mov rdx, r15
- - - - 1.00 1.00 1.96 0.04 call qword ptr [rip + memset@GOTPCREL]
- - - - 1.00 - 0.01 0.99 mov qword ptr [rbx + 16], r12
- - 0.50 - - 0.50 - - mov al, 1
- - 0.51 0.49 - - - - add rsp, 24
- - - - - - 0.02 0.98 pop rbx
- - - - - - 0.03 0.97 pop r12
- - - - - - 0.03 0.97 pop r13
- - - - - - 0.97 0.03 pop r14
- - - - - - 0.03 0.97 pop r15
- - - - - - 0.01 0.99 pop rbp
- - - - - 1.00 - - ret
- - 0.49 0.51 - - - - mov r15, rsi
- - 0.51 0.49 - - - - mov rbp, rdx
- - 0.49 0.51 - - - - lea rcx, [rax + rax]
- - 0.49 0.50 - 0.01 - - cmp r12, rcx
- - 1.04 0.50 - 1.46 - - cmova rcx, r12
- - 0.49 0.49 - 0.02 - - cmp rcx, 5
- - 0.50 - - 0.50 - - mov r14d, 4
- - 0.50 0.51 - 0.99 - - cmovae r14, rcx
- - - - - - 0.97 0.03 mov rdx, qword ptr [rbx + 8]
- - - 0.51 - 0.49 - - mov rdi, rsp
- - 0.01 0.50 - 0.49 - - mov rsi, rax
- - 0.49 0.50 - 0.01 - - mov rcx, r14
- - - - 1.00 1.00 0.99 0.01 call <alloc::raw_vec::RawVecInner>::finish_grow
- - 0.51 0.49 - - 0.50 0.50 cmp dword ptr [rsp], 1
- - - - - 1.00 - - je .LBB6_3
- - - - - - 0.50 0.50 mov rax, qword ptr [rsp + 8]
- - - - 1.00 - 0.99 0.01 mov qword ptr [rbx + 8], rax
- - - - 1.00 - 0.01 0.99 mov qword ptr [rbx], r14
- - 0.49 0.50 - 0.01 - - mov rdx, rbp
- - 0.50 0.01 - 0.49 - - mov rsi, r15
- - - - - 1.00 - - jmp .LBB6_5
- - 0.01 0.99 - - - - lea rdi, [rip + .Lanon.HASH.1]
- - 0.99 0.01 - - - - lea rdx, [rip + .Lanon.HASH.3]
- - 0.02 0.49 - 0.49 - - mov esi, 37
- - - - 1.00 1.00 0.02 1.98 call qword ptr [rip + core::panicking::panic@GOTPCREL]

9
rust/zerocopy/benches/new_box_zeroed.rs Normal file
View File

@ -0,0 +1,9 @@
use zerocopy::*;
#[path = "formats/coco_static_size.rs"]
mod format;
#[unsafe(no_mangle)]
fn bench_new_box_zeroed() -> Option<Box<format::LocoPacket>> {
FromZeros::new_box_zeroed().ok()
}

7
rust/zerocopy/benches/new_box_zeroed.x86-64 Normal file
View File

@ -0,0 +1,7 @@
bench_new_box_zeroed:
push rax
call qword ptr [rip + __rustc::__rust_no_alloc_shim_is_unstable_v2@GOTPCREL]
mov edi, 6
mov esi, 2
pop rax
jmp qword ptr [rip + __rustc::__rust_alloc_zeroed@GOTPCREL]

51
rust/zerocopy/benches/new_box_zeroed.x86-64.mca Normal file
View File

@ -0,0 +1,51 @@
Iterations: 100
Instructions: 600
Total Cycles: 1197
Total uOps: 1100
Dispatch Width: 4
uOps Per Cycle: 0.92
IPC: 0.50
Block RThroughput: 2.8
Instruction Info:
[1]: #uOps
[2]: Latency
[3]: RThroughput
[4]: MayLoad
[5]: MayStore
[6]: HasSideEffects (U)
[1] [2] [3] [4] [5] [6] Instructions:
2 5 1.00 * push rax
4 7 1.00 * call qword ptr [rip + __rustc::__rust_no_alloc_shim_is_unstable_v2@GOTPCREL]
1 1 0.33 mov edi, 6
1 1 0.33 mov esi, 2
1 6 0.50 * pop rax
2 6 1.00 * jmp qword ptr [rip + __rustc::__rust_alloc_zeroed@GOTPCREL]
Resources:
[0] - SBDivider
[1] - SBFPDivider
[2] - SBPort0
[3] - SBPort1
[4] - SBPort4
[5] - SBPort5
[6.0] - SBPort23
[6.1] - SBPort23
Resource pressure per iteration:
[0] [1] [2] [3] [4] [5] [6.0] [6.1]
- - 0.99 1.00 2.00 2.01 2.07 2.93
Resource pressure by instruction:
[0] [1] [2] [3] [4] [5] [6.0] [6.1] Instructions:
- - - - 1.00 - 0.93 0.07 push rax
- - - - 1.00 1.00 0.12 1.88 call qword ptr [rip + __rustc::__rust_no_alloc_shim_is_unstable_v2@GOTPCREL]
- - 0.99 - - 0.01 - - mov edi, 6
- - - 1.00 - - - - mov esi, 2
- - - - - - 0.94 0.06 pop rax
- - - - - 1.00 0.08 0.92 jmp qword ptr [rip + __rustc::__rust_alloc_zeroed@GOTPCREL]

11
rust/zerocopy/benches/new_box_zeroed_with_elems_dynamic_padding.rs Normal file
View File

@ -0,0 +1,11 @@
use zerocopy::*;
#[path = "formats/coco_dynamic_padding.rs"]
mod format;
#[unsafe(no_mangle)]
fn bench_new_box_zeroed_with_elems_dynamic_padding(
count: usize,
) -> Option<Box<format::LocoPacket>> {
FromZeros::new_box_zeroed_with_elems(count).ok()
}

24
rust/zerocopy/benches/new_box_zeroed_with_elems_dynamic_padding.x86-64 Normal file
View File

@ -0,0 +1,24 @@
bench_new_box_zeroed_with_elems_dynamic_padding:
push r14
push rbx
push rax
mov rbx, rdi
movabs rax, 3074457345618258598
cmp rdi, rax
ja .LBB5_1
lea r14, [rbx + 2*rbx]
or r14, 3
add r14, 9
call qword ptr [rip + __rustc::__rust_no_alloc_shim_is_unstable_v2@GOTPCREL]
mov esi, 4
mov rdi, r14
call qword ptr [rip + __rustc::__rust_alloc_zeroed@GOTPCREL]
jmp .LBB5_3
.LBB5_1:
xor eax, eax
.LBB5_3:
mov rdx, rbx
add rsp, 8
pop rbx
pop r14
ret

81
rust/zerocopy/benches/new_box_zeroed_with_elems_dynamic_padding.x86-64.mca Normal file
View File

@ -0,0 +1,81 @@
Iterations: 100
Instructions: 2100
Total Cycles: 2990
Total uOps: 3000
Dispatch Width: 4
uOps Per Cycle: 1.00
IPC: 0.70
Block RThroughput: 7.5
Instruction Info:
[1]: #uOps
[2]: Latency
[3]: RThroughput
[4]: MayLoad
[5]: MayStore
[6]: HasSideEffects (U)
[1] [2] [3] [4] [5] [6] Instructions:
2 5 1.00 * push r14
2 5 1.00 * push rbx
2 5 1.00 * push rax
1 1 0.33 mov rbx, rdi
1 1 0.33 movabs rax, 3074457345618258598
1 1 0.33 cmp rdi, rax
1 1 1.00 ja .LBB5_1
1 1 0.50 lea r14, [rbx + 2*rbx]
1 1 0.33 or r14, 3
1 1 0.33 add r14, 9
4 7 1.00 * call qword ptr [rip + __rustc::__rust_no_alloc_shim_is_unstable_v2@GOTPCREL]
1 1 0.33 mov esi, 4
1 1 0.33 mov rdi, r14
4 7 1.00 * call qword ptr [rip + __rustc::__rust_alloc_zeroed@GOTPCREL]
1 1 1.00 jmp .LBB5_3
1 0 0.25 xor eax, eax
1 1 0.33 mov rdx, rbx
1 1 0.33 add rsp, 8
1 6 0.50 * pop rbx
1 6 0.50 * pop r14
1 1 1.00 U ret
Resources:
[0] - SBDivider
[1] - SBFPDivider
[2] - SBPort0
[3] - SBPort1
[4] - SBPort4
[5] - SBPort5
[6.0] - SBPort23
[6.1] - SBPort23
Resource pressure per iteration:
[0] [1] [2] [3] [4] [5] [6.0] [6.1]
- - 4.49 4.50 5.00 6.01 4.50 4.50
Resource pressure by instruction:
[0] [1] [2] [3] [4] [5] [6.0] [6.1] Instructions:
- - - - 1.00 - 0.50 0.50 push r14
- - - - 1.00 - 0.50 0.50 push rbx
- - - - 1.00 - 0.50 0.50 push rax
- - 0.49 0.50 - 0.01 - - mov rbx, rdi
- - 0.50 0.50 - - - - movabs rax, 3074457345618258598
- - 0.50 0.50 - - - - cmp rdi, rax
- - - - - 1.00 - - ja .LBB5_1
- - 0.50 0.50 - - - - lea r14, [rbx + 2*rbx]
- - 0.50 0.50 - - - - or r14, 3
- - 0.50 - - 0.50 - - add r14, 9
- - - - 1.00 1.00 1.00 1.00 call qword ptr [rip + __rustc::__rust_no_alloc_shim_is_unstable_v2@GOTPCREL]
- - - 0.50 - 0.50 - - mov esi, 4
- - 0.50 0.50 - - - - mov rdi, r14
- - - - 1.00 1.00 1.00 1.00 call qword ptr [rip + __rustc::__rust_alloc_zeroed@GOTPCREL]
- - - - - 1.00 - - jmp .LBB5_3
- - - - - - - - xor eax, eax
- - 0.51 0.49 - - - - mov rdx, rbx
- - 0.49 0.51 - - - - add rsp, 8
- - - - - - 0.50 0.50 pop rbx
- - - - - - 0.50 0.50 pop r14
- - - - - 1.00 - - ret

9
rust/zerocopy/benches/new_box_zeroed_with_elems_dynamic_size.rs Normal file
View File

@ -0,0 +1,9 @@
use zerocopy::*;
#[path = "formats/coco_dynamic_size.rs"]
mod format;
#[unsafe(no_mangle)]
fn bench_new_box_zeroed_with_elems_dynamic_size(count: usize) -> Option<Box<format::LocoPacket>> {
FromZeros::new_box_zeroed_with_elems(count).ok()
}

22
rust/zerocopy/benches/new_box_zeroed_with_elems_dynamic_size.x86-64 Normal file
View File

@ -0,0 +1,22 @@
bench_new_box_zeroed_with_elems_dynamic_size:
push r14
push rbx
push rax
mov rbx, rdi
movabs rax, 4611686018427387901
cmp rdi, rax
ja .LBB5_1
lea r14, [2*rbx + 4]
call qword ptr [rip + __rustc::__rust_no_alloc_shim_is_unstable_v2@GOTPCREL]
mov esi, 2
mov rdi, r14
call qword ptr [rip + __rustc::__rust_alloc_zeroed@GOTPCREL]
jmp .LBB5_3
.LBB5_1:
xor eax, eax
.LBB5_3:
mov rdx, rbx
add rsp, 8
pop rbx
pop r14
ret

77
rust/zerocopy/benches/new_box_zeroed_with_elems_dynamic_size.x86-64.mca Normal file
View File

@ -0,0 +1,77 @@
Iterations: 100
Instructions: 1900
Total Cycles: 2990
Total uOps: 2800
Dispatch Width: 4
uOps Per Cycle: 0.94
IPC: 0.64
Block RThroughput: 7.0
Instruction Info:
[1]: #uOps
[2]: Latency
[3]: RThroughput
[4]: MayLoad
[5]: MayStore
[6]: HasSideEffects (U)
[1] [2] [3] [4] [5] [6] Instructions:
2 5 1.00 * push r14
2 5 1.00 * push rbx
2 5 1.00 * push rax
1 1 0.33 mov rbx, rdi
1 1 0.33 movabs rax, 4611686018427387901
1 1 0.33 cmp rdi, rax
1 1 1.00 ja .LBB5_1
1 1 0.50 lea r14, [2*rbx + 4]
4 7 1.00 * call qword ptr [rip + __rustc::__rust_no_alloc_shim_is_unstable_v2@GOTPCREL]
1 1 0.33 mov esi, 2
1 1 0.33 mov rdi, r14
4 7 1.00 * call qword ptr [rip + __rustc::__rust_alloc_zeroed@GOTPCREL]
1 1 1.00 jmp .LBB5_3
1 0 0.25 xor eax, eax
1 1 0.33 mov rdx, rbx
1 1 0.33 add rsp, 8
1 6 0.50 * pop rbx
1 6 0.50 * pop r14
1 1 1.00 U ret
Resources:
[0] - SBDivider
[1] - SBFPDivider
[2] - SBPort0
[3] - SBPort1
[4] - SBPort4
[5] - SBPort5
[6.0] - SBPort23
[6.1] - SBPort23
Resource pressure per iteration:
[0] [1] [2] [3] [4] [5] [6.0] [6.1]
- - 3.97 3.98 5.00 5.05 4.50 4.50
Resource pressure by instruction:
[0] [1] [2] [3] [4] [5] [6.0] [6.1] Instructions:
- - - - 1.00 - 0.50 0.50 push r14
- - - - 1.00 - 0.50 0.50 push rbx
- - - - 1.00 - 0.50 0.50 push rax
- - 0.05 0.94 - 0.01 - - mov rbx, rdi
- - 0.94 0.06 - - - - movabs rax, 4611686018427387901
- - 0.06 0.94 - - - - cmp rdi, rax
- - - - - 1.00 - - ja .LBB5_1
- - 0.94 0.06 - - - - lea r14, [2*rbx + 4]
- - - - 1.00 1.00 1.00 1.00 call qword ptr [rip + __rustc::__rust_no_alloc_shim_is_unstable_v2@GOTPCREL]
- - 0.98 0.02 - - - - mov esi, 2
- - 0.02 0.94 - 0.04 - - mov rdi, r14
- - - - 1.00 1.00 1.00 1.00 call qword ptr [rip + __rustc::__rust_alloc_zeroed@GOTPCREL]
- - - - - 1.00 - - jmp .LBB5_3
- - - - - - - - xor eax, eax
- - 0.94 0.06 - - - - mov rdx, rbx
- - 0.04 0.96 - - - - add rsp, 8
- - - - - - 0.50 0.50 pop rbx
- - - - - - 0.50 0.50 pop r14
- - - - - 1.00 - - ret

9
rust/zerocopy/benches/new_vec_zeroed.rs Normal file
View File

@ -0,0 +1,9 @@
use zerocopy::*;
#[path = "formats/coco_static_size.rs"]
mod format;
#[unsafe(no_mangle)]
fn bench_new_vec_zeroed(len: usize) -> Option<Vec<format::LocoPacket>> {
FromZeros::new_vec_zeroed(len).ok()
}

40
rust/zerocopy/benches/new_vec_zeroed.x86-64 Normal file
View File

@ -0,0 +1,40 @@
bench_new_vec_zeroed:
mov rax, rdi
movabs rcx, 1537228672809129301
cmp rsi, rcx
ja .LBB5_5
test rsi, rsi
je .LBB5_2
push r15
push r14
push rbx
lea rcx, [rsi + rsi]
lea rbx, [rcx + 2*rcx]
mov r14, rax
mov r15, rsi
call qword ptr [rip + __rustc::__rust_no_alloc_shim_is_unstable_v2@GOTPCREL]
mov esi, 2
mov rdi, rbx
call qword ptr [rip + __rustc::__rust_alloc_zeroed@GOTPCREL]
mov rsi, r15
mov rcx, rax
mov rax, r14
test rcx, rcx
pop rbx
pop r14
pop r15
je .LBB5_5
mov qword ptr [rax], rsi
mov qword ptr [rax + 8], rcx
mov qword ptr [rax + 16], rsi
ret
.LBB5_5:
movabs rcx, -9223372036854775808
mov qword ptr [rax], rcx
ret
.LBB5_2:
mov ecx, 2
mov qword ptr [rax], rsi
mov qword ptr [rax + 8], rcx
mov qword ptr [rax + 16], rsi
ret

113
rust/zerocopy/benches/new_vec_zeroed.x86-64.mca Normal file
View File

@ -0,0 +1,113 @@
Iterations: 100
Instructions: 3700
Total Cycles: 3486
Total uOps: 4600
Dispatch Width: 4
uOps Per Cycle: 1.32
IPC: 1.06
Block RThroughput: 12.0
Instruction Info:
[1]: #uOps
[2]: Latency
[3]: RThroughput
[4]: MayLoad
[5]: MayStore
[6]: HasSideEffects (U)
[1] [2] [3] [4] [5] [6] Instructions:
1 1 0.33 mov rax, rdi
1 1 0.33 movabs rcx, 1537228672809129301
1 1 0.33 cmp rsi, rcx
1 1 1.00 ja .LBB5_5
1 1 0.33 test rsi, rsi
1 1 1.00 je .LBB5_2
2 5 1.00 * push r15
2 5 1.00 * push r14
2 5 1.00 * push rbx
1 1 0.50 lea rcx, [rsi + rsi]
1 1 0.50 lea rbx, [rcx + 2*rcx]
1 1 0.33 mov r14, rax
1 1 0.33 mov r15, rsi
4 7 1.00 * call qword ptr [rip + __rustc::__rust_no_alloc_shim_is_unstable_v2@GOTPCREL]
1 1 0.33 mov esi, 2
1 1 0.33 mov rdi, rbx
4 7 1.00 * call qword ptr [rip + __rustc::__rust_alloc_zeroed@GOTPCREL]
1 1 0.33 mov rsi, r15
1 1 0.33 mov rcx, rax
1 1 0.33 mov rax, r14
1 1 0.33 test rcx, rcx
1 6 0.50 * pop rbx
1 6 0.50 * pop r14
1 6 0.50 * pop r15
1 1 1.00 je .LBB5_5
1 1 1.00 * mov qword ptr [rax], rsi
1 1 1.00 * mov qword ptr [rax + 8], rcx
1 1 1.00 * mov qword ptr [rax + 16], rsi
1 1 1.00 U ret
1 1 0.33 movabs rcx, -9223372036854775808
1 1 1.00 * mov qword ptr [rax], rcx
1 1 1.00 U ret
1 1 0.33 mov ecx, 2
1 1 1.00 * mov qword ptr [rax], rsi
1 1 1.00 * mov qword ptr [rax + 8], rcx
1 1 1.00 * mov qword ptr [rax + 16], rsi
1 1 1.00 U ret
Resources:
[0] - SBDivider
[1] - SBFPDivider
[2] - SBPort0
[3] - SBPort1
[4] - SBPort4
[5] - SBPort5
[6.0] - SBPort23
[6.1] - SBPort23
Resource pressure per iteration:
[0] [1] [2] [3] [4] [5] [6.0] [6.1]
- - 6.99 6.99 12.00 10.02 8.00 9.00
Resource pressure by instruction:
[0] [1] [2] [3] [4] [5] [6.0] [6.1] Instructions:
- - 0.01 0.98 - 0.01 - - mov rax, rdi
- - 0.98 0.02 - - - - movabs rcx, 1537228672809129301
- - 0.02 0.98 - - - - cmp rsi, rcx
- - - - - 1.00 - - ja .LBB5_5
- - 0.98 - - 0.02 - - test rsi, rsi
- - - - - 1.00 - - je .LBB5_2
- - - - 1.00 - - 1.00 push r15
- - - - 1.00 - 1.00 - push r14
- - - - 1.00 - - 1.00 push rbx
- - - 1.00 - - - - lea rcx, [rsi + rsi]
- - - 1.00 - - - - lea rbx, [rcx + 2*rcx]
- - 1.00 - - - - - mov r14, rax
- - 1.00 - - - - - mov r15, rsi
- - - - 1.00 1.00 2.00 - call qword ptr [rip + __rustc::__rust_no_alloc_shim_is_unstable_v2@GOTPCREL]
- - - 0.01 - 0.99 - - mov esi, 2
- - 0.01 0.99 - - - - mov rdi, rbx
- - - - 1.00 1.00 - 2.00 call qword ptr [rip + __rustc::__rust_alloc_zeroed@GOTPCREL]
- - 0.01 - - 0.99 - - mov rsi, r15
- - 0.99 0.01 - - - - mov rcx, rax
- - - 0.99 - 0.01 - - mov rax, r14
- - 0.99 0.01 - - - - test rcx, rcx
- - - - - - - 1.00 pop rbx
- - - - - - 1.00 - pop r14
- - - - - - - 1.00 pop r15
- - - - - 1.00 - - je .LBB5_5
- - - - 1.00 - 1.00 - mov qword ptr [rax], rsi
- - - - 1.00 - - 1.00 mov qword ptr [rax + 8], rcx
- - - - 1.00 - 1.00 - mov qword ptr [rax + 16], rsi
- - - - - 1.00 - - ret
- - 0.01 0.99 - - - - movabs rcx, -9223372036854775808
- - - - 1.00 - - 1.00 mov qword ptr [rax], rcx
- - - - - 1.00 - - ret
- - 0.99 0.01 - - - - mov ecx, 2
- - - - 1.00 - 1.00 - mov qword ptr [rax], rsi
- - - - 1.00 - - 1.00 mov qword ptr [rax + 8], rcx
- - - - 1.00 - 1.00 - mov qword ptr [rax + 16], rsi
- - - - - 1.00 - - ret

9
rust/zerocopy/benches/new_zeroed.rs Normal file
View File

@ -0,0 +1,9 @@
use zerocopy::*;
#[path = "formats/coco_static_size.rs"]
mod format;
#[unsafe(no_mangle)]
fn bench_new_zeroed() -> format::LocoPacket {
FromZeros::new_zeroed()
}

3
rust/zerocopy/benches/new_zeroed.x86-64 Normal file
View File

@ -0,0 +1,3 @@
bench_new_zeroed:
xor eax, eax
ret

Some files were not shown because too many files have changed in this diff Show More