Use `io_project!` for PTE array and message queues to restore the proper
encapsulation.
The remaining `dma_read!` and `dma_write!` is now only acting on
primitives; thus replace by `io_read!` and `io_write!`.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260706-io_projection-v6-17-72cd5d055d54@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
The GSP SetRegistry command currently stores its registry entries in a
fixed-size array. That makes every additional runtime-dependent registry
object require reshaping the command data structure at the same time as the
feature that needs the new entry.
Keep the existing registry contents unchanged, but store them in a KVec so
SetRegistry can be constructed dynamically. The constructor now returns a
Result to propagate allocation failures while the command payload layout is
still computed from the final entry list.
Cc: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
Link: https://patch.msgid.link/20260701062622.3499033-7-zhiw@nvidia.com
[acourbot: remove orphan comment.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
FSP message handling currently uses FspResponse for the common response
prefix containing the MCTP header, NVDM header and command response
payload.
That name is too broad once other FSP response formats reuse the same
prefix and append protocol-specific payloads. Rename it to
FspResponseHeader so subsequent response structures can embed the common
header without overloading the meaning of FspResponse.
Suggested-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://lore.kernel.org/all/DJMBI9CN2Z67.2T02SR8TAWEC5@nvidia.com/
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
Link: https://patch.msgid.link/20260701062622.3499033-4-zhiw@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
The rust-analyzer generator script recognizes a crate when its
corresponding `.o` appears in the Rust source file's immediate `Makefile`
or `Kbuild` file.
Commit ca524e273c ("gpu: build nova-core and nova-drm from
drivers/gpu/Makefile") moves the build rules for `nova-core` and
`nova-drm` into `drivers/gpu/Makefile`, which results in the generator
script ignoring these crates.
Fix this by naming the crates' module as a comment in their respective
`Makefile`; this is enough for the script to pick them up and restore
`rust-analyzer` functionality on them.
Fixes: ca524e273c ("gpu: build nova-core and nova-drm from drivers/gpu/Makefile")
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Tested-by: Timur Tabi <ttabi@nvidia.com>
Reviewed-by: Onur Özkan <work@onurozkan.dev>
Link: https://patch.msgid.link/20260701-nova-rust-analyzer-v1-1-5209f486f10d@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
This module is now orphaned code, superseded by a kernel-global
implementation, so remove it.
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260701-nova-bitfield-v2-2-2e949bf1836c@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Since commit 5423ef9d4d ("rust: pin-init: internal: suppress
`non_snake_case` lint in `[pin_]init!`"), mere use of the non-snake-case
identifiers would not cause the warning to be generated. Thus remove these
allows.
Signed-off-by: Gary Guo <gary@garyguo.net>
Acked-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Onur Özkan <work@onurozkan.dev>
Link: https://patch.msgid.link/20260612123401.2684025-1-gary@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Replace the unsafe `kernel::transmute::FromBytes` trait implementation
for the `FalconUCodeDescV3`, `PcirStruct`, `BitHeader`, `BitToken`,
`NpdeStruct`, `PciRomHeader`, `PmuLookupTableEntry` and
`PmuLookupTableHeader` structs with the derivable `zerocopy::FromBytes`
trait.
This change eliminates the manual unsafe implementations in favor of a
derivable trait. When this trait is derived, validity checks are
performed at compile time to ensure that the type can safely implement
`FromBytes`.
Suggested-by: Miguel Ojeda <ojeda@kernel.org>
Link: https://github.com/Rust-for-Linux/linux/issues/1241
Signed-off-by: Nicolás Antinori <nico.antinori.7@gmail.com>
Link: https://patch.msgid.link/20260629142007.269873-1-nico.antinori.7@gmail.com
[acourbot: add `vbios:` prefix to commit title.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Export nova-core's Rust symbols so nova-drm can resolve references to it
when loaded as a module.
This is done by generating declarations and EXPORT_SYMBOL_RUST_GPL()
calls for Rust symbols referenced by nova-drm, and compiling them into
the module as `nova_core_exports.o`.
`nova_core_exports.o` declares every Rust symbol as `extern int`.
Running `gendwarfksyms` on it would compute CRCs from those placeholder
types instead of the real Rust ones, so make MODVERSIONS use this shim
only for the export list, and derive CRCs from `nova_core.o`.
This patch is intended to be a workaround until the build system
supports Rust cross-crate dependencies natively.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260622-nova-exports-v5-2-6191773fc977@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
A dependency between nova-core and nova-drm is about to be introduced,
which requires nova-core to be built first. As this is not easily doable
from separate directories, move both build targets to the first common
parent, `drivers/gpu/Makefile`.
Suggested-by: Miguel Ojeda <ojeda@kernel.org>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260622-nova-exports-v5-1-6191773fc977@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQFSBAABCgA8FiEEq68RxlopcLEwq+PEeb4+QwBBGIYFAmpBb4seHHRvcnZhbGRz
QGxpbnV4LWZvdW5kYXRpb24ub3JnAAoJEHm+PkMAQRiGjV8H/iFiGxRTgdNwWJsH
5BQpc9kAmQHBMuzc2yM1HeeKVOm+zfxEmZV7kFabPEGYB1ltV52vspKI5vPUCWhP
9VVqhIQy8ZfALEUIhQOoieJAboRgNDySldnuS8E2MFyfgKC0Tg8dkolmRFuh8Xhu
R88kBeS1+DfpjFrnNLH98iUdtE9HabVt2VGfdrIpTYLC8VqyyuI77dXTEFkkwSCp
NacpX5KYzD9dlDjlH6Hpqy/jqv7Y7JsEynFVUU05IQU9VojV7o1cbMQlKoe1yNeU
dCoDCSwHhc8DUrGx1Tj6lsAS8k0R2it5FIW1eCF7cVKzZ8CbojyhluW6YPCOC0wE
0louZxg=
=5vas
-----END PGP SIGNATURE-----
Merge tag 'v7.2-rc1' into drm-rust-next
Merge v7.2-rc1 into drm-rust-next to start the next cycle.
Devres now requires T: 'static, which conflicts with shmem::Object using
Devres<SGTableMap<T, C>> without that bound. Resolve by adding 'static
as a supertrait to DriverObject and DeviceContext, which does not
restrict any current use cases since DriverObject is always implemented
on owned, refcounted types and DeviceContext is a sealed marker trait
with only unit-type implementors.
If DriverObject (or DeviceContext) ever becomes lifetime-parameterized
(e.g. via a GAT for TTM backed objects), the 'static supertrait can be
relaxed at that point.
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Add usable_fb_regions() to GspStaticConfigInfo to extract the usable FB
regions from GSP's fbRegionInfoParams. Usable regions are those that are
not reserved or protected.
The extracted regions are stored in GetGspStaticInfoReply and exposed
for use by the memory subsystem.
Display the regions and their total size upon device probe.
[acourbot: expose all regions as a KVec, display usable regions and
total usable VRAM.]
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Reviewed-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260617-boot-vram-v3-3-20b9ec5fe9f2@nvidia.com
[acourbot: replace dev_info!() with dev_dbg!().]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
The GSP static information is useful during regular driver runtime;
however it is currently obtained from `Gsp::boot`, with no elegant way
to pass it back to the caller.
Solve this by moving the code acquiring it to a dedicated method of
`Gsp` that can be called as soon as the `Gsp` is booted. This allows us
to obtain and display the static information from the `Gpu` constructor,
and to store the static information for later use.
Its location at the end of `Gsp::boot` was a bit out-of-place anyway:
technically, the GSP is considered booted after we have received the
`GspInitDone` message, so anything that happens afterwards is not part
of the boot sequence anymore.
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260617-boot-vram-v3-2-20b9ec5fe9f2@nvidia.com
[acourbot: add documentation to `get_static_info` method.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
`Gpu` currently owns the state needed to unload the GSP directly. This
means that `unload_bundle` has to be the last initialized field: once GSP
boot succeeds, any later initialization failure would leave `Gpu`
partially initialized, and its `PinnedDrop` implementation would not run.
This prevents adding fallible `Gpu` fields that need to query the GSP
after it has booted.
Move the GSP state and unload bundle into a dedicated pinned
`GspResources` object. Once that subobject has been initialized, its
`PinnedDrop` implementation will run even if initialization of a later
`Gpu` field fails, ensuring that the GSP unload sequence is executed.
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260617-boot-vram-v3-1-20b9ec5fe9f2@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
`FspFirmware` is constructed and immediately passed into `Fsp`. It makes
sense for `Fsp` to ask to load its firmware, so move it there.
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260615-blackwell-fixes-v1-6-f2853e49ff7d@nvidia.com
[acourbot: fix minor merge conflict.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Small cleanup to move these constants which are only used once closer to
their use location.
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Reviewed-by: Alistair Popple <apopple@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260615-blackwell-fixes-v1-4-f2853e49ff7d@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
The GspHal trait methods boot() and post_boot() accept a long list of
individual parameters (dev, bar, chipset, gsp_falcon, sec2_falcon) that
are threaded through the entire GSP boot call chain. This makes the
signatures unwieldy and difficult to extend as new boot-time context
(e.g. vGPU state) is introduced.
Introduce a GspBootContext struct that bundles the common boot
parameters into a single object, and refactor the GspHal trait to accept
&GspBootContext instead of individual arguments. The struct also exposes
a dev() helper with proper lifetime annotation so that HAL
implementations can extract the device reference without reborrowing
constraints.
Update both TU102 and GH100 HAL implementations to extract their
required parameters from the context struct, and simplify the call sites
in Gsp::boot() accordingly.
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
Link: https://patch.msgid.link/20260604114339.1565660-7-zhiw@nvidia.com
[acourbot: pass `GspBootContext` by value to `Gsp::boot`.]
[acourbot: deconstruct `GspBootContext` in `Gsp::boot` to simplify diff.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
To prevent some copy-paste boilerplate doc comments, remove the second copy
of "see [`crate::fb::SysmemFlush`]" from regs.rs.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Link: https://patch.msgid.link/20260611011901.84517-4-jhubbard@nvidia.com
[acourbot: separate unrelated changes into their own patch.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
The Blackwell sysmem flush read helper composed the 64-bit address as
lo | (hi << 32). Write it as (hi << 32) | lo, the order humans expect
to read, matching the bit layout and the new Hopper helper.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Link: https://patch.msgid.link/20260611011901.84517-4-jhubbard@nvidia.com
[acourbot: separate unrelated changes into their own patch.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Hopper has its own FBHUB sysmem flush page registers, but the Hopper
framebuffer HAL delegates to the Ampere NISO path, which encodes the
address with an 8-bit right-shift. That programs the wrong value into
the wrong registers, so the GPU's sysmembar flush targets the wrong
system memory address.
Add Hopper's FBHUB flush registers and program them directly from the
Hopper HAL.
This has not yet been tested on real Hopper hardware (that's true for
nova-core in general).
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Link: https://patch.msgid.link/20260611011901.84517-3-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
The GB20x sysmem flush registers were defined relative to an Fbhub0Base
register window, but there is exactly one FBHUB0 base, so expressing them
as base-plus-offset only adds indirection.
Rename these to FBHUB0 and give them their fixed absolute addresses,
dropping the base struct and its RegisterBase impl.
No functional changes.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Link: https://patch.msgid.link/20260611011901.84517-2-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
- deferred probe:
- Fix race where deferred probe timeout work could be permanently
canceled by using mod_delayed_work()
- Fix missing jiffies conversion in deferred_probe_extend_timeout()
- Guard timeout extension with delayed_work_pending() to prevent
premature firing
- Use system_percpu_wq instead of the deprecated system_wq
- Update deferred_probe_timeout documentation
- device:
- Replace direct struct device bitfield access (can_match, dma_iommu,
dma_skip_sync, dma_ops_bypass, state_synced, dma_coherent,
of_node_reused, offline, offline_disabled) with flag-based
accessors using bit operations
- Reject devices with unregistered buses
- Delete unused DEVICE_ATTR_PREALLOC()
- Add low-level device attribute macros with const show/store
callbacks, allowing device attributes to reside in read-only memory
- Move core device attributes to read-only memory
- Constify group array pointers in driver_add_groups() /
driver_remove_groups(), struct bus_type, and struct device_driver
- device property:
- Fix fwnode reference leak in fwnode_graph_get_endpoint_by_id()
- Initialize all fields of fwnode_handle in fwnode_init()
- Provide swnode_get()/swnode_put() wrappers around kobject_get/put()
- Allow passing struct software_node_ref_args pointers directly to
PROPERTY_ENTRY_REF()
- driver_override:
- Migrate amba, cdx, vmbus, and rpmsg to the generic driver_override
infrastructure, fixing a UAF from unsynchronized access to
driver_override in bus match() callbacks
- Remove the now-unused driver_set_override()
- firmware loader:
- Fix recursive lock deadlock in device_cache_fw_images() when async
work falls back to synchronous execution
- Fix device reference leak in firmware_upload_register()
- platform:
- Pass KBUILD_MODNAME through the platform driver registration macro
to create module symlinks in sysfs for built-in drivers; move
module_kset initialization to a pure_initcall and tegra cbb
registration to core_initcall to ensure correct ordering
- Pass THIS_MODULE implicitly through a coresight_init_driver() macro
- sysfs:
- Upgrade OOB write detection in sysfs_kf_seq_show() from printk to
WARN
- Add return value clamping to sysfs_kf_read()
- Rust:
- ACPI:
- Fix missing match data for PRP0001 by exporting
acpi_of_match_device()
- Auxiliary:
- Replace drvdata() with dedicated registration data on
auxiliary_device. drvdata() exposed the driver's bus device
private data beyond the driver's own scope, creating ordering
constraints and forcing the data to outlive all registrations
that access it. Registration data is instead scoped structurally
to the Registration object, making lifecycle ordering enforced
by construction rather than convention.
- Rust-native device driver lifetimes (HRT):
- Allow Rust device drivers to carry a lifetime parameter on their
bus device private data, tied to the device binding scope -- the
interval during which a bus device is bound to a driver. Device
resources like pci::Bar<'a> and IoMem<'a> can be stored directly
in the driver's bus device private data with a lifetime bounded
by the binding scope, so the compiler enforces at build time that
they do not outlive the binding. This removes Devres indirection
from every access site and eliminates try_access() failure paths
in destructors.
Bus driver traits use a Generic Associated Type (GAT)
Data<'bound> to introduce the lifetime on the private data,
rather than parameterizing the Driver trait itself. Auxiliary
registration data, where the lifetime is not introduced by a
trait callback but must be threaded through Registration, uses
the ForLt trait (a type-level abstraction for types generic over
a lifetime).
- Misc:
- Fix DT overlayed devices not probing by reverting the broken
treewide overlay fix and re-running fw_devlink consumer pickup when
an overlay is applied to a bound device
- Use root_device_register() for faux bus root device; add sanity
check for failed bus init
- Fix dev_has_sync_state() data race with READ_ONCE() and move it to
base.h
- Avoid spurious device_links warning when removing a device while
its supplier is unbinding
- Switch ISA bus to dynamic root device
- Fix suspicious RCU usage in kernfs_put()
- Remove devcoredump exit callback
- Constify devfreq_event_class
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQS2q/xV6QjXAdC7k+1FlHeO1qrKLgUCai1lNQAKCRBFlHeO1qrK
LjX1AP4kEb6SJdvBFlG/IJcG9vyqA7+Cig8OIlx9vEX7e7GvNwEAxL2HltAlAd5Q
cOIJOlu0uM+j1mn0JyuM9b0WJrxDLAA=
=E5mG
-----END PGP SIGNATURE-----
Merge tag 'driver-core-7.2-rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/driver-core/driver-core
Pull driver core updates from Danilo Krummrich:
"Deferred probe:
- Fix race where deferred probe timeout work could be permanently
canceled by using mod_delayed_work()
- Fix missing jiffies conversion in deferred_probe_extend_timeout()
- Guard timeout extension with delayed_work_pending() to prevent
premature firing
- Use system_percpu_wq instead of the deprecated system_wq
- Update deferred_probe_timeout documentation
device:
- Replace direct struct device bitfield access (can_match, dma_iommu,
dma_skip_sync, dma_ops_bypass, state_synced, dma_coherent,
of_node_reused, offline, offline_disabled) with flag-based
accessors using bit operations
- Reject devices with unregistered buses
- Delete unused DEVICE_ATTR_PREALLOC()
- Add low-level device attribute macros with const show/store
callbacks, allowing device attributes to reside in read-only memory
- Move core device attributes to read-only memory
- Constify group array pointers in driver_add_groups() /
driver_remove_groups(), struct bus_type, and struct device_driver
device property:
- Fix fwnode reference leak in fwnode_graph_get_endpoint_by_id()
- Initialize all fields of fwnode_handle in fwnode_init()
- Provide swnode_get()/swnode_put() wrappers around kobject_get/put()
- Allow passing struct software_node_ref_args pointers directly to
PROPERTY_ENTRY_REF()
driver_override:
- Migrate amba, cdx, vmbus, and rpmsg to the generic driver_override
infrastructure, fixing a UAF from unsynchronized access to
driver_override in bus match() callbacks
- Remove the now-unused driver_set_override()
firmware loader:
- Fix recursive lock deadlock in device_cache_fw_images() when async
work falls back to synchronous execution
- Fix device reference leak in firmware_upload_register()
platform:
- Pass KBUILD_MODNAME through the platform driver registration macro
to create module symlinks in sysfs for built-in drivers; move
module_kset initialization to a pure_initcall and tegra cbb
registration to core_initcall to ensure correct ordering
- Pass THIS_MODULE implicitly through a coresight_init_driver() macro
sysfs:
- Upgrade OOB write detection in sysfs_kf_seq_show() from printk to
WARN
- Add return value clamping to sysfs_kf_read()
Rust:
- ACPI:
Fix missing match data for PRP0001 by exporting
acpi_of_match_device()
- Auxiliary:
Replace drvdata() with dedicated registration data on
auxiliary_device. drvdata() exposed the driver's bus device private
data beyond the driver's own scope, creating ordering constraints
and forcing the data to outlive all registrations that access it.
Registration data is instead scoped structurally to the
Registration object, making lifecycle ordering enforced by
construction rather than convention.
- Rust-native device driver lifetimes (HRT):
Allow Rust device drivers to carry a lifetime parameter on their
bus device private data, tied to the device binding scope -- the
interval during which a bus device is bound to a driver. Device
resources like pci::Bar<'a> and IoMem<'a> can be stored directly in
the driver's bus device private data with a lifetime bounded by the
binding scope, so the compiler enforces at build time that they do
not outlive the binding. This removes Devres indirection from every
access site and eliminates try_access() failure paths in
destructors.
Bus driver traits use a Generic Associated Type (GAT) Data<'bound>
to introduce the lifetime on the private data, rather than
parameterizing the Driver trait itself. Auxiliary registration
data, where the lifetime is not introduced by a trait callback but
must be threaded through Registration, uses the ForLt trait (a
type-level abstraction for types generic over a lifetime).
Misc:
- Fix DT overlayed devices not probing by reverting the broken
treewide overlay fix and re-running fw_devlink consumer pickup when
an overlay is applied to a bound device
- Use root_device_register() for faux bus root device; add sanity
check for failed bus init
- Fix dev_has_sync_state() data race with READ_ONCE() and move it to
base.h
- Avoid spurious device_links warning when removing a device while
its supplier is unbinding
- Switch ISA bus to dynamic root device
- Fix suspicious RCU usage in kernfs_put()
- Remove devcoredump exit callback
- Constify devfreq_event_class"
* tag 'driver-core-7.2-rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/driver-core/driver-core: (81 commits)
software node: allow passing reference args to PROPERTY_ENTRY_REF()
driver core: platform: set mod_name in driver registration
coresight: pass THIS_MODULE implicitly through a macro
kernel: param: initialize module_kset in a pure_initcall
soc/tegra: cbb: Move driver registration from pure_initcall to core_initcall
firmware_loader: Fix recursive lock in device_cache_fw_images()
driver core: Use system_percpu_wq instead of system_wq
driver core: remove driver_set_override()
rpmsg: use generic driver_override infrastructure
Drivers: hv: vmbus: use generic driver_override infrastructure
cdx: use generic driver_override infrastructure
amba: use generic driver_override infrastructure
rust: devres: add 'static bound to Devres<T>
samples: rust: rust_driver_auxiliary: showcase lifetime-bound registration data
rust: auxiliary: generalize Registration over ForLt
rust: types: add `ForLt` trait for higher-ranked lifetime support
gpu: nova-core: separate driver type from driver data
samples: rust: rust_driver_pci: use HRT lifetime for Bar
rust: io: make IoMem and ExclusiveIoMem lifetime-parameterized
rust: pci: make Bar lifetime-parameterized
...
Given the macro scoping rules, all macros are rendered twice, in the
module and in the top-level of kernel crate.
Add `#[doc(hidden)]` to the macro definition and `#[doc(inline)]` to the
re-export inside `build_assert` module so the top-level items are hidden.
[ Sadly, because the definition is hidden, `rustdoc` decides to not list
them as re-exports in the `prelude` page anymore, even if we refer to
the not-actually-hidden item.
- Miguel ]
Acked-by: Danilo Krummrich <dakr@kernel.org>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Acked-by: Alexandre Courbot <acourbot@nvidia.com>
Acked-by: FUJITA Tomonori <fujita.tomonori@gmail.com>
Acked-by: Boqun Feng <boqun@kernel.org>
Signed-off-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260609142637.373347-1-gary@kernel.org
[ Kept a single declaration in the prelude, and reworded since they
already had `no_inline`. Removed other imports from `predefine` since
we now use the prelude. - Miguel ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Two comments in the FSP Chain of Trust message setup had drifted from
the code. One referred to a variable name that no longer exists, and
another described the unused sysmem FRTS fields as future work rather
than explaining why they are zero. Update both to describe the code as
it stands.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260603235034.131354-3-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
On Turing and Ampere, resetting the GSP involves running two firmware
images: FWSEC-SB and Booter Unloader. They are independent from one
another, and we should do whatever is possible to restore the GSP's
unloaded state even if a failure occurs along the way.
Thus, keep going and run Booter Unloader even if the execution of
FWSEC-SB failed.
Fixes: adb99ce3cc ("gpu: nova-core: run Booter Unloader and FWSEC-SB upon unbinding")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260529-nova-unload-v7-0-678f39209e00%40nvidia.com?part=3
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260531-nova-unload-fix-v1-1-c8dcdc769b53@nvidia.com
[acourbot: log Booter Unloader errors.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
The module firmware table lists booter_load and booter_unload for every
chipset, but Hopper and Blackwell boot the GSP through FSP and never
load the SEC2 booter. Those modinfo entries point at firmware files that
are not shipped for FSP-based chipsets, so initramfs tooling looks for
images that are never used.
Declare the booter only for chipsets that boot via it, matching how the
FMC and FWSEC bootloader images are already gated on chipset
capabilities.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Danilo Krummrich <dakr@kernel.org>
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Link: https://patch.msgid.link/20260607230539.144382-1-jhubbard@nvidia.com
[acourbot: make comment on FMC/Booter choice a bit more precise.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Change send_sync_fsp() to return the raw response buffer after
validating the common MCTP/NVDM headers and error code. This allows
callers to perform protocol-specific parsing on the response payload,
which is needed for the upcoming PRC protocol support.
For the existing COT caller, the response buffer is unused.
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
Link: https://patch.msgid.link/20260604114339.1565660-4-zhiw@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Extract common MCTP + NVDM headers into FspMessageHeader, rename
FspMessage to FspCotMessage, and update FspResponse to use the shared
header. This prepares for adding new FSP message types.
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
Link: https://patch.msgid.link/20260604114339.1565660-3-zhiw@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Now that we have `zerocopy` support, we can avoid some `unsafe` code.
For instance, for `FalconUCodeDescV2`, we can replace the `unsafe impl
FromBytes` by safely deriving `zerocopy`'s `FromBytes` and then calling
`read_from_prefix`.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260608141439.182634-20-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Use "build" to denote that the index bounds checking here is performed at
build time.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Gary Guo <gary@garyguo.net>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260602-projection-syntax-rework-v2-5-6989470f5440@garyguo.net
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Currently Nova code uses `&'a Bar0` a lot. This is `&'a Mmio`, where `Mmio`
represents an owned MMIO region; this type only exists as a target for
`Deref` so `Bar` and `IoMem` can share code and should be avoided to be
named directly. The upcoming I/O projection series would make `Io` trait
much simpler to implement, and thus the owned MMIO type would be removed
in favour of direct `Io` implementation on `Bar` and `IoMem`.
Add lifetime parameter to `Bar0<'a>` and change it to be alias of `&'a
pci::Bar<'a, ..>`. This also prepares Nova core so that when I/O projection
series land, this could be changed to using a MMIO view type directly which
avoids double indirection.
Signed-off-by: Gary Guo <gary@garyguo.net>
Acked-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260602170416.2268531-1-gary@kernel.org
[ Rebase onto latest drm-rust-next (Blackwell enablement). - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
For non-sec2 it is only required to wait for GSP falcon to halt. This is
because GSP does the main work of unloading on GPUs not using sec2.
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
[ jhubbard: use Result instead of Result<()> in the UnloadBundle impl ]
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Link: https://patch.msgid.link/20260603-b4-blackwell-v13-8-d9f3a06939e0@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
On Hopper and Blackwell, FSP boots GSP with hardware lockdown enabled.
After FSP Chain of Trust completes, the driver must poll for lockdown
release before proceeding with GSP initialization. Add the register
bit and helper functions needed for this polling.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Link: https://patch.msgid.link/20260603-b4-blackwell-v13-7-d9f3a06939e0@nvidia.com
[acourbot: fix `lockdown_released` logic and add explanatory comments.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Build and send the Chain of Trust message to FSP, bundling the
DMA-coherent boot parameters that FSP reads at boot time.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Link: https://patch.msgid.link/20260603-b4-blackwell-v13-6-d9f3a06939e0@nvidia.com
[acourbot: rename `frts_offset` to `frts_vidmem_offset`.]
[acourbot: add note about frts_sysmem_* CoT members.]
Co-developed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
The FSP Chain of Trust handshake is versioned: Hopper speaks version 1
and Blackwell speaks version 2. Provide the version through the FSP HAL
so the boot message carries the value FSP expects, and so chipsets that
do not use FSP need not express a version at all.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260603-b4-blackwell-v13-5-d9f3a06939e0@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
FSP exchanges are request/response: the driver sends an MCTP/NVDM
message and must match the reply against the request before acting on
it. Add the synchronous send-and-wait path that validates the response
transport and message headers and confirms the reply corresponds to the
request that was sent.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260603-b4-blackwell-v13-4-d9f3a06939e0@nvidia.com
[acourbot: make `MessageToFsp` private.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Add the MCTP (Management Component Transport Protocol) and NVDM (NVIDIA
Data Model) wire-format types used for communication between the kernel
driver and GPU firmware processors.
This includes typed MCTP transport headers, NVDM message headers, and
NVDM message type identifiers. Both the FSP boot path and the upcoming
GSP RPC message queue share this protocol layer.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260603-b4-blackwell-v13-3-d9f3a06939e0@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
FSP communication uses a pair of non-circular queues in the FSP
falcon's EMEM, one for messages from the driver to FSP and one for
replies, with the driver polling for response data. Add the queue
registers and the low-level helpers used by the higher-level FSP
message layer.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260603-b4-blackwell-v13-2-d9f3a06939e0@nvidia.com
[acourbot: align register fields names with OpenRM.]
[acourbot: represent registers as arrays of 8 instances, as per OpenRM.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Add external memory (EMEM) read/write operations to the GPU's FSP falcon
engine. These operations use Falcon PIO (Programmed I/O) to communicate
with the FSP through indirect memory access.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260603-b4-blackwell-v13-1-d9f3a06939e0@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Extract the SHA-384 hash, RSA public key, and RSA signature from the
FMC ELF32 firmware sections. FSP Chain of Trust verification needs
these to validate the FMC image during boot.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260602032111.224790-14-jhubbard@nvidia.com
[acourbot: derive `Zeroable` on `FmcSignature` for in-place initialization]
Co-developed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Hopper and Blackwell use FSP instead of SEC2 for secure boot. The
driver must wait for FSP secure boot to complete before continuing
with GSP bring-up. Poll for boot success with a 5-second timeout, and
return the FSP interface only on success so that later Chain of Trust
operations cannot run before FSP is ready. The interface owns the FSP
falcon and the FMC firmware.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260602032111.224790-13-jhubbard@nvidia.com
[acourbot: use `inspect_err` instead of `map_err` and display actual error]
[acourbot: limit visibility of `fsp_hal` to `super``]
Co-developed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
FSP is the Falcon that runs FMC firmware on Hopper and Blackwell.
Load the FMC ELF in two forms: the image section that FSP boots from,
and the full Firmware object for later signature extraction during
Chain of Trust verification. Declare the FMC image in the module's
firmware table so it is bundled for FSP-based chipsets.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260602032111.224790-12-jhubbard@nvidia.com
Co-developed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Add the FSP (Foundation Security Processor) falcon engine type that
will handle secure boot and Chain of Trust operations on Hopper and
Blackwell architectures.
The FSP falcon replaces SEC2's role in the boot sequence for these newer
architectures. This initial stub just defines the falcon type and its
base address.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260602032111.224790-11-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
A firmware image may be either a 32-bit or a 64-bit ELF, and callers
should not have to know which. Detect the ELF class from the image
header at parse time and dispatch to the matching parser, so a single
entry point handles both layouts.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260602032111.224790-10-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Some GPU firmware images are packaged as 32-bit ELF rather than 64-bit.
Add a 32-bit implementation of the shared ELF section-parsing
abstraction so those images can be parsed alongside the existing 64-bit
path.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260602032111.224790-9-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Introduce a single ELF format abstraction that ties each ELF header
type to its matching section-header type. This keeps the shared
section parser ready for upcoming ELF32 support and avoids mixing
32-bit and 64-bit ELF layouts by mistake.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260602032111.224790-8-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Blackwell GPUs moved the sysmem flush page registers away from the
Ampere/Ada location. GB10x routes the flush through a pair of HSHUB0
register sets (primary and egress) that must both be programmed to
the same address. GB20x routes it through FBHUB0.
Define these registers relative to their HSHUB0 and FBHUB0 bases, as
Open RM does, and implement the flush paths in the GB10x and GB20x
framebuffer HALs.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260602032111.224790-7-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
The GSP-RM boot working memory portion of the WPR2 heap must be
larger on Hopper and later GPUs than on Turing, Ampere, and Ada.
Select the larger value for those generations.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260602032111.224790-6-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Hopper and Blackwell need a larger non-WPR heap than the 1 MiB that
earlier architectures use. Hopper and Blackwell GB10x need 2 MiB, while
Blackwell GB20x needs 2 MiB + 128 KiB. These sizes diverge by family,
so give Hopper and each Blackwell family its own framebuffer HAL and
select the non-WPR heap size per chipset family.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260602032111.224790-5-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
GSP boot needs to know how much framebuffer memory is reserved for
the PMU. Compute it per architecture: Blackwell dGPUs reserve a
non-zero amount, earlier architectures leave it at zero, matching
Open RM behavior.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260602032111.224790-4-jhubbard@nvidia.com
Co-developed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Hopper and Blackwell GPUs moved the PCI config space mirror from
0x088000 to 0x092000. Select the correct address per architecture
when building the GSP system info command.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260602032111.224790-3-jhubbard@nvidia.com
Co-developed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Replace the hardcoded 47-bit DMA mask with a GPU HAL method that
provides the correct value for the architecture.
Set the DMA mask in Gpu::new(). Gpu owns all DMA allocations for
the device, so no concurrent allocations can exist while the
constructor is still running.
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260602032111.224790-2-jhubbard@nvidia.com
Co-developed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
If `Gsp::boot` fails, the GSP can be left in a state where boot cannot
be attempted again unless it is reset first.
To avoid this, we want to run the unload bundle whenever `boot` fails to
try and clear the partially-initialized state.
Do this by wrapping the unload bundle into a drop guard up until `boot`
returns. After that, running the unload bundle becomes the
responsibility of the caller.
Reviewed-by: Danilo Krummrich <dakr@kernel.org>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260529-nova-unload-v7-4-678f39209e00@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
When probing the driver, the FWSEC-FRTS firmware creates a WPR2 secure
memory region to store the GSP firmware, and the Booter Loader loads and
starts that firmware into the GSP, making it run in RISC-V mode.
These operations need to be reverted upon unloading, particularly the
WPR2 secure region creation, as its presence prevents the driver from
subsequently probing.
Thus, prepare the Booter Unloader and FWSEC-SB firmware images when
booting the GSP, so they can be executed at unbind time to put the GPU
into a state where it can be probed again.
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Co-developed-by: Eliot Courtney <ecourtney@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Reviewed-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260529-nova-unload-v7-3-678f39209e00@nvidia.com
[acourbot: `Result<()>` -> `Result`]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Currently, the GSP is left running after the driver is unbound. This is
not great for several reasons, notably that it can still access shared
memory areas that the kernel will now reclaim (especially problematic on
setups without an IOMMU).
Fix this by sending the `UNLOADING_GUEST_DRIVER` GSP command when the
`Gpu` is dropped. This stops the GSP and lets us proceed with the rest
of the unbind sequence in a later patch.
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Co-developed-by: Eliot Courtney <ecourtney@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Reviewed-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260529-nova-unload-v7-2-678f39209e00@nvidia.com
[acourbot: `Result<()>` -> `Result`]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Booting the GSP is done differently depending on the architecture. Move
the parts that are chipset-specific under a HAL.
This does not change much at the moment, since the differences between
Turing and Ampere are rather benign, but will become critical to
properly support the FSP boot process used by Hopper and Blackwell.
The Hopper/Blackwell support is not merged yet, so their HAL is a stub
for now.
This patch is intended to be a mechanical code extraction with no
behavioral changes.
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Reviewed-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260529-nova-unload-v7-1-678f39209e00@nvidia.com
[acourbot: `Result<()>` -> `Result`]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Some parts of the GSP boot process are chip-specific actions, whereas
others (like sending the initial post-boot messages) deal directly with
the working GSP.
Reorganize the boot code a bit so the chipset-specific parts are clumped
together, which will make their extraction into a HAL easier.
This has no effect on the GSP boot process.
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260521-nova-unload-v6-5-65f581c812c9@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Move the SEC2 reset/load/boot sequence into a BooterFirmware::run()
method. This is mostly refactoring, with no significant behavior change,
done in preparation for adding an alternative FSP boot path.
Suggested-by: Danilo Krummrich <dakr@kernel.org>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260521-nova-unload-v6-4-65f581c812c9@nvidia.com
[acourbot: fix typo in commit message.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Importing all the firmware commands like we did is a bit confusing, as
the layer of a command type (fw or GSP) cannot be inferred from looking
at its name alone. Furthermore it makes it impossible to create commands
that have the same name as their firmware command.
Thus, stop importing all commands and refer to them from the `fw` module
instead.
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260521-nova-unload-v6-2-65f581c812c9@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
This function was useful before the generic command-queue send methods
got merged, but it is just boilerplate now. Replace it with the correct
sequence to queue the `GetGspStaticInfo` command directly.
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260521-nova-unload-v6-1-65f581c812c9@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Now that SysmemFlush is lifetime-parameterized, the ARef<Device> is
unnecessary -- a plain &'bound Device reference suffices.
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Tested-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260525225838.276108-4-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Now that SysmemFlush can borrow the Bar via HRT lifetime, store a
&'bound Bar0 reference and implement Drop to automatically unregister
the sysmem flush page. This removes the need for manual unregister()
calls and the Gpu::unbind() method.
Reported-by: Eliot Courtney <ecourtney@nvidia.com>
Closes: https://lore.kernel.org/all/20260409-fix-systemflush-v1-1-a1d6c968f17c@nvidia.com/
Fixes: 6554ad65b5 ("gpu: nova-core: register sysmem flush page")
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Tested-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260525225838.276108-3-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Take advantage of the lifetime-parameterized pci::Bar<'bound> to hold
the BAR mapping directly in NovaCore<'bound>, and pass a borrowed
reference to Gpu<'bound>.
This eliminates the Arc<Devres<Bar0>> indirection, removes runtime
revocation checks for BAR access, and simplifies Gpu::unbind().
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Tested-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260525225838.276108-2-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Replace drvdata() with registration data on the auxiliary bus. Private
data is now scoped to the registration object, removing the ordering
constraints and lifetime complications that came with drvdata().
Add Higher-Ranked Lifetime Types (HRT) so driver structs can borrow
device resources like pci::Bar and IoMem directly, tied to the device
binding scope. This removes the need for Devres indirection and
ARef<Device> in most driver code.
This is a stable tag for other trees to merge.
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQS2q/xV6QjXAdC7k+1FlHeO1qrKLgUCahjG2wAKCRBFlHeO1qrK
Lt0SAP9yaQFBx6t8rJdxtj2UgqrEBvW+20Gq5ze06ZRxaogqpAD9Ez0niV2z7Smh
T9YLPmJ5XDaz92QckskoifLmY/oYHQY=
=7fHo
-----END PGP SIGNATURE-----
Merge tag 'dd-lifetimes-7.2-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/driver-core/driver-core into drm-rust-next
Higher-Ranked Lifetime Types for Rust device drivers
Replace drvdata() with registration data on the auxiliary bus. Private
data is now scoped to the registration object, removing the ordering
constraints and lifetime complications that came with drvdata().
Add Higher-Ranked Lifetime Types (HRT) so driver structs can borrow
device resources like pci::Bar and IoMem directly, tied to the device
binding scope. This removes the need for Devres indirection and
ARef<Device> in most driver code.
This is a stable tag for other trees to merge.
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Generalize Registration<T> to Registration<F: ForLt> and
Device::registration_data<F: ForLt>() to return Pin<&F::Of<'_>>.
The stored 'static lifetime is shortened to the borrow lifetime of &self
via ForLt::cast_ref; ForLt's covariance guarantee makes this sound.
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260525202921.124698-24-dakr@kernel.org
[ Use PhantomData<F::Of<'a>> instead of
PhantomData<(fn(&'a ()) -> &'a (), F)>], which also gets us rid of
#[allow(clippy::type_complexity)]. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Introduce NovaCoreDriver as the driver type implementing pci::Driver,
keeping NovaCore as the per-device data type. This prepares for making
NovaCore lifetime-parameterized once auxiliary::Registration requires a
lifetime for the binding scope.
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260525202921.124698-22-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Convert pci::Bar<SIZE> to pci::Bar<'a, SIZE>, storing &'a Device<Bound>
to tie the BAR mapping lifetime to the device.
iomap_region_sized() now returns Result<Bar<'a, SIZE>> directly instead
of impl PinInit<Devres<Bar<SIZE>>, Error>.
Since the lifetime ties the mapping to the device's bound state, callers
no longer need Devres for the common case where the Bar lives in the
driver's private data.
Add Bar::into_devres() to consume the bar and register it as a
device-managed resource, returning Devres<Bar<'static, SIZE>>. The
lifetime is erased to 'static because Devres guarantees the bar does not
actually outlive the device -- access is revoked on unbind.
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260525202921.124698-19-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Add a 'bound lifetime to the associated Data, changing type Data to type
Data<'bound>.
This allows the driver's bus device private data to capture the device /
driver bound lifetime; device resources can be stored directly by
reference rather than requiring Devres.
The probe() and unbind() callbacks thus gain a 'bound lifetime parameter
on the methods themselves; avoiding a global lifetime on the trait impl.
Existing drivers set type Data<'bound> = Self, preserving the current
behavior.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260525202921.124698-13-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Device<Core> references in probe callbacks are scoped to the callback,
not the full binding duration. Add a lifetime parameter to Core and
CoreInternal to accurately represent this in the type system.
Suggested-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260525202921.124698-12-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Add a type Data<'bound> associated type to all bus driver traits,
decoupling the driver's bus device private data type from the driver
struct itself.
In the context of adding a 'bound lifetime, making this an associated
type has the advantage that it allows us to avoid a driver trait global
lifetime and it avoids the need for ForLt for bus device private data;
both of which make the subsequent implementation by buses much simpler.
All existing drivers and doc examples set type Data = Self to preserve
the current behavior.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Link: https://patch.msgid.link/20260525202921.124698-5-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Move constants and functions to be inside the impls of the types they
are related to. This makes it more obvious what each type and value is
for.
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260525-fix-vbios-v5-21-e5e455251537@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
These are unused currently, and it is probably sufficient to just check
the type of BIOS image in the future.
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260525-fix-vbios-v5-19-e5e455251537@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Remove unnecessary fields in PciRomHeader. This allows a simplification
to use `FromBytes` instead of reading fields piecemeal. A lot of these
checks were redundant as well since it checks the size of the `data`
first in `BiosImage`.
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260525-fix-vbios-v5-18-e5e455251537@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Improve readability by moving the success path outside of a nested
branch.
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260525-fix-vbios-v5-17-e5e455251537@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Currently, FWSEC takes the first image and the last image. Treat the
first FWSEC image and all following image data as one logical block
for building the final FWSEC image. This avoids explicitly tracking
two FWSEC images.
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260525-fix-vbios-v5-16-e5e455251537@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Currently, PCI-AT takes the final image if multiple exist. Use the
first one instead, to match nouveau behaviour.
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260525-fix-vbios-v5-15-e5e455251537@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
`FwSecBiosBuilder` now only contains `falcon_ucode_offset` which just
gets passed directly into `FwSecBiosImage`. Remove `FwSecBiosBuilder`
and construct `FwSecBiosImage` directly, as a simplification.
Reviewed-by: Joel Fernandes <joelagnelf@nvidia.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260525-fix-vbios-v5-14-e5e455251537@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
The current code copies the data into a KVec and parses it on demand. We
can simplify the code by storing the parsed entries.
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260525-fix-vbios-v5-13-e5e455251537@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
This simplifies the construction of `PmuLookupTableEntry` and is
allowed now that the driver can assume it is little endian.
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260525-fix-vbios-v5-12-e5e455251537@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
The code first computes `pmu_in_first_fwsec` or adjusts the offset and
then uses it in a branch just once to get the correct source for the PMU
table. This can be simplified to a single branch while also avoiding the
mutation of `offset`. Also, adjust the code after this to keep the
success case non-nested.
Reviewed-by: Joel Fernandes <joelagnelf@nvidia.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260525-fix-vbios-v5-11-e5e455251537@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Push the computation of the falcon data offset into a helper function.
The subtraction to create the offset should be checked, and by doing
this the check can be folded into the existing check in
`falcon_data_ptr`.
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Reviewed-by: Joel Fernandes <joelagnelf@nvidia.com>
Link: https://patch.msgid.link/20260525-fix-vbios-v5-10-e5e455251537@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
This does not need to be stored in `FwSecBiosBuilder` so we can remove
it from there, and just create and use it locally in
`setup_falcon_data`.
Reviewed-by: Joel Fernandes <joelagnelf@nvidia.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260525-fix-vbios-v5-9-e5e455251537@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
This is unused, so we can remove it.
Reviewed-by: Joel Fernandes <joelagnelf@nvidia.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260525-fix-vbios-v5-8-e5e455251537@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Use checked arithmetic for `ucode_offset` in `setup_falcon_data`. This
prevents a malformed firmware from causing a panic.
Fixes: dc70c6ae24 ("gpu: nova-core: vbios: Add support to look up PMU table in FWSEC")
Reviewed-by: Joel Fernandes <joelagnelf@nvidia.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260525-fix-vbios-v5-7-e5e455251537@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Use checked access in `FwSecBiosImage::header` for getting the header
version since the value is firmware derived.
Fixes: 47c4846e43 ("gpu: nova-core: vbios: Add support for FWSEC ucode extraction")
Reviewed-by: Joel Fernandes <joelagnelf@nvidia.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260525-fix-vbios-v5-6-e5e455251537@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Use checked arithmetic and access for extracting the microcode since the
offsets are firmware derived.
Fixes: 47c4846e43 ("gpu: nova-core: vbios: Add support for FWSEC ucode extraction")
Reviewed-by: Joel Fernandes <joelagnelf@nvidia.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260525-fix-vbios-v5-5-e5e455251537@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
If `header.token_size` is smaller than `BitToken`, then we currently can
read past the end of `image.base.data`. Use checked arithmetic for
computing offsets and simplify reading it in using `FromBytes`.
Fixes: dc70c6ae24 ("gpu: nova-core: vbios: Add support to look up PMU table in FWSEC")
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260525-fix-vbios-v5-4-e5e455251537@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Fix bug where `read_more_at_offset` would unnecessarily read more data.
This happens when the window to read has some part cached and some part
not. It would read `len` bytes instead of just the uncached portion,
which could read past `BIOS_MAX_SCAN_LEN`.
Fixes: 6fda04e7f0 ("gpu: nova-core: vbios: Add base support for VBIOS construction and iteration")
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260525-fix-vbios-v5-3-e5e455251537@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
`read_bios_image_at_offset` is called with a length from the VBIOS
header, so we should be more defensive here and use checked arithmetic.
Fixes: 6fda04e7f0 ("gpu: nova-core: vbios: Add base support for VBIOS construction and iteration")
Reviewed-by: Joel Fernandes <joelagnelf@nvidia.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260525-fix-vbios-v5-2-e5e455251537@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Current code lets `current_offset` go to `BIOS_MAX_SCAN_LEN` which is
one byte too far.
Fixes: 6fda04e7f0 ("gpu: nova-core: vbios: Add base support for VBIOS construction and iteration")
Reviewed-by: Joel Fernandes <joelagnelf@nvidia.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://patch.msgid.link/20260525-fix-vbios-v5-1-e5e455251537@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
