github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-05-26 08:02:27 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

Merge tag 'drm-intel-next-2019-10-21' of git://anongit.freedesktop.org/drm/drm-intel into drm-next

Browse Source 
UAPI Changes:

- Introduce a versioning of the i915-perf uapi (Lionel)
- Add support for perf configuration queries (Lionel)

  Allow listing perf configurations with IOCTL in addition
  to sysfs. This is useful in container usecases.

- Allow dynamic reconfiguration of the OA stream (Chris)

  Allows the OA stream to be reconfigured between
  batch buffers, giving greater flexibility in sampling.

- Allow holding preemption on filtered perf ctx

  Allow CAP_ADMIN to block pre-emption of a context
  to query performance counters without disturbances.

  Mesa changes: https://gitlab.freedesktop.org/mesa/mesa/merge_requests/932

Cross-subsystem Changes:

- drm-next backmerge for HDR DP changes
  https://lists.freedesktop.org/archives/dri-devel/2019-September/236453.html

Driver Changes:

- Add DC3CO sleep state for Tigerlake (Anshuman)
- Tigerlake BCS engine support engine relative MMIO (Daniele)
- Simplify the Tigerlake LRC register list for !RCS (Daniele)
- Read SAGV block time from PCODE on Tigerlake (James)
- Add 12 missing Tigerlake workarounds (Mika)
- Enable DDI/Port G for Tigerlake (Khaled)

- Avoid hang in tsg,vfe units by keeping l3 clocks ICL+(Mika)
- Fix Bugzilla #111966: Favor last VBT child device (Ville)
- Fix blue/black screen on boot due to broken gamma (Swati)
- Add support of BT.2020 Colorimetry to DP MSA (Gwan-gyeong)
- Attach colorspace property to DP connector (Gwan-gyeong)
- Attach HDR metadata property to DP connector (Gwan-gyeong)
- Base intel_memory_region support prep for local memory (Matt A)
- Introduce Jasper Lake PCH (Matt R)
- Support multiple GPUs in PMU (Tvrtko)
- Fix MST oops due to MSA changes (Ville)
- Refuse modes with hdisplay==4096 on pre-HSW DP (Ville)
- Correct the PCH type in irq postinstall for JSP (Vivek)
- Save Master transcoder in slave's crtc_state for Transcoder Port Sync (Manasi)
- Enable TRANSCODER PORT SYNC for tiled displays across separate ports (Manasi)
- HW state readout for transcoder port sync config (Manasi)
- Enable master-slaves in trans port sync (Manasi)
- In port sync mode disable slaves first then master (Manasi)
- Fix port checks for MST support on gen >= 11 (Lucas)

- Flush submission tasklet before waiting/retiring (Chris)
- Flush tasklet submission before sleeping on i915_request_wait (Chris)
- Object pin reference counting fixes (Chris, Matt A)
- Clear semaphore immediately upon ELSP promotion (Chris)
- Child device size remains unchanged through VBT 229 (Matt R)
- Restore dropped 'interruptible' flag on retiring requests (Chris)
- Treat a busy timeline as 'active' while waiting (Chris)
- Clean up struct_mutex from perf (Chris)
- Update locking around execlists->active (Chris)
- Mark up expected execlist state during reset (Chris)
- Remove cursor use of properties for coordinates (Maarten)
- Only mark incomplete requests as -EIO on cancelling (Chris)
- Add an rcu_barrier option to i915_drop_caches (Chris)
- Replace perf global wakeref tracking with engine-pm (Chris)
- Prevent merging requests with conflicting flags (Chris)
- Allow for CS OA configs to be created lazily (Lionel)
- Implement active wait for noa configurations (Lionel)
- Execute OA configuration from command stream (Lionel)
- Prefer using the pinned_ctx for emitting delays on config (Chris)
- Port C's hotplug interrupt is associated with TC1 bits (Vivek, Matt R)
- Extend program of VSC Header and DB for Colorimetry Format (Gwan-gyeong)
- Fine-tune timeslicing of contexts (Chris)
- Do initial mocs configuration directly (Chris)
- Fix uninitialized variable on PMU error path (Tvrtko)
- Don't disable interrupts independently of the locking (Sebastian)
- Eliminate struct_mutext from GVT (Chris)

- Move perf types to their own header (Lionel)
- Drop list of perf streams (always size 1) (Lionel)
- Store the perf associated engine of a stream (Lionel)
- Make array hw_engine_mask static (Colin)
- Prefer shortest path to RPM/perf/GT instead of dev_priv (Chris, Tvrtko)
- Virtual request submission fixes (Chris)
- Selftest/CI improvements (Chris)
- Fix Kconfig indentation (Krzysztof)
- Give engine->kernel_context distinct timeline lock classes (Chris)
- Fix null pointer deref on selftest error path (Colin)
- Select DPLL's via mask (Matt R)
- Introduce and use intel_atomic_crtc_state_for_each_plane_state (Maarten)
- Use intel_plane_state in prepare and cleanup plane_fb (Maarten)
- Remove begin/finish_crtc_commit (Maarten)
- Move SAGV block time to dev_priv (James)
- Avoid polluting the i915_oa_config with error pointers (Chris)
- Squelch display kerneldoc warnings (Chris)
- Assert tasklet is locked for process_csb() (Chris)
- Switch to using DP_MSA_MISC_* defines (Ville)
- Stop using drm_atomic_helper_check_planes() (Ville)
- Make .modeset_calc_cdclk() mandatory (Ville)
- Use drm_rect_translate_to()/drm_rect_init() (Ville)
- Refactor timestamping constants update (Ville)
- Switch intel_legacy_cursor_update() to intel_ types (Ville)
- Prepare the connector/encoder mask readout for hw vs. uapi state split (Ville)
- Prepare the mode readout for hw vs. uapi state split (Ville)
- Move swizzle_bit under i915_ggtt (Chris)
- Improve microcontrollers documentation (Daniele)
- Move the cursor rotation handling into intel_cursor_check_surface() (Ville)
- Cleanups to pipe code (Ville)
- Shrink eDRAM ways/sets arrays for code size (Ville)
- Cleanups to HDCP2 timeout code (Ville)
- Restore full symmetry in i915_driver_modeset_probe/remove (Janusz)
- Simplify setting of ddi_io_power_domain (Lucas)
- Add pipe id/name to pipe mismatch logs (Lucas)
- Prettify MST debug message (Lucas)
- Extract GT ring management to separate files (Andi)

Signed-off-by: Dave Airlie <airlied@redhat.com>
From: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20191021180337.GA24338@jlahtine-desk.ger.corp.intel.com
This commit is contained in:
Dave Airlie 2019-10-22 13:51:04 +10:00
commit 89910e6200
134 changed files with 7262 additions and 2799 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

48
Documentation/gpu/i915.rst
View File

@ -415,6 +415,15 @@ Object Tiling IOCTLs
.. kernel-doc:: drivers/gpu/drm/i915/gem/i915_gem_tiling.c
:doc: buffer object tiling
Microcontrollers
================
Starting from gen9, three microcontrollers are available on the HW: the
graphics microcontroller (GuC), the HEVC/H.265 microcontroller (HuC) and the
display microcontroller (DMC). The driver is responsible for loading the
firmwares on the microcontrollers; the GuC and HuC firmwares are transferred
to WOPCM using the DMA engine, while the DMC firmware is written through MMIO.
WOPCM
-----
@ -427,12 +436,24 @@ WOPCM Layout
GuC
---
Firmware Layout
~~~~~~~~~~~~~~~
.. kernel-doc:: drivers/gpu/drm/i915/gt/uc/intel_guc.c
:doc: GuC
GuC Firmware Layout
~~~~~~~~~~~~~~~~~~~
.. kernel-doc:: drivers/gpu/drm/i915/gt/uc/intel_uc_fw_abi.h
:doc: Firmware Layout
GuC Memory Management
~~~~~~~~~~~~~~~~~~~~~
.. kernel-doc:: drivers/gpu/drm/i915/gt/uc/intel_guc.c
:doc: GuC Memory Management
.. kernel-doc:: drivers/gpu/drm/i915/gt/uc/intel_guc.c
:functions: intel_guc_allocate_vma
GuC-specific firmware loader
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@ -448,11 +469,26 @@ GuC-based command submission
.. kernel-doc:: drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
:internal:
GuC Address Space
~~~~~~~~~~~~~~~~~
HuC
---
.. kernel-doc:: drivers/gpu/drm/i915/gt/uc/intel_huc.c
:doc: HuC
.. kernel-doc:: drivers/gpu/drm/i915/gt/uc/intel_huc.c
:functions: intel_huc_auth
.. kernel-doc:: drivers/gpu/drm/i915/gt/uc/intel_guc.c
:doc: GuC Address Space
HuC Memory Management
~~~~~~~~~~~~~~~~~~~~~
.. kernel-doc:: drivers/gpu/drm/i915/gt/uc/intel_huc.c
:doc: HuC Memory Management
HuC Firmware Layout
~~~~~~~~~~~~~~~~~~~
The HuC FW layout is the same as the GuC one, see `GuC Firmware Layout`_
DMC
---
See `CSR firmware support for DMC`_
Tracing
=======

12
drivers/gpu/drm/i915/Kconfig
View File

@ -76,7 +76,7 @@ config DRM_I915_CAPTURE_ERROR
This option enables capturing the GPU state when a hang is detected.
This information is vital for triaging hangs and assists in debugging.
Please report any hang to
https://bugs.freedesktop.org/enter_bug.cgi?product=DRI
https://bugs.freedesktop.org/enter_bug.cgi?product=DRI
for triaging.
If in doubt, say "Y".
@ -105,11 +105,11 @@ config DRM_I915_USERPTR
If in doubt, say "Y".
config DRM_I915_GVT
bool "Enable Intel GVT-g graphics virtualization host support"
depends on DRM_I915
depends on 64BIT
default n
help
bool "Enable Intel GVT-g graphics virtualization host support"
depends on DRM_I915
depends on 64BIT
default n
help
Choose this option if you want to enable Intel GVT-g graphics
virtualization technology host support with integrated graphics.
With GVT-g, it's possible to have one integrated graphics

142
drivers/gpu/drm/i915/Kconfig.debug
View File

@ -1,34 +1,34 @@
# SPDX-License-Identifier: GPL-2.0-only
config DRM_I915_WERROR
bool "Force GCC to throw an error instead of a warning when compiling"
# As this may inadvertently break the build, only allow the user
# to shoot oneself in the foot iff they aim really hard
depends on EXPERT
# We use the dependency on !COMPILE_TEST to not be enabled in
# allmodconfig or allyesconfig configurations
depends on !COMPILE_TEST
bool "Force GCC to throw an error instead of a warning when compiling"
# As this may inadvertently break the build, only allow the user
# to shoot oneself in the foot iff they aim really hard
depends on EXPERT
# We use the dependency on !COMPILE_TEST to not be enabled in
# allmodconfig or allyesconfig configurations
depends on !COMPILE_TEST
select HEADER_TEST
default n
help
Add -Werror to the build flags for (and only for) i915.ko.
Do not enable this unless you are writing code for the i915.ko module.
default n
help
Add -Werror to the build flags for (and only for) i915.ko.
Do not enable this unless you are writing code for the i915.ko module.
Recommended for driver developers only.
Recommended for driver developers only.
If in doubt, say "N".
If in doubt, say "N".
config DRM_I915_DEBUG
bool "Enable additional driver debugging"
depends on DRM_I915
select DEBUG_FS
select PREEMPT_COUNT
select REFCOUNT_FULL
select I2C_CHARDEV
select STACKDEPOT
select DRM_DP_AUX_CHARDEV
select X86_MSR # used by igt/pm_rpm
select DRM_VGEM # used by igt/prime_vgem (dmabuf interop checks)
select DRM_DEBUG_MM if DRM=y
bool "Enable additional driver debugging"
depends on DRM_I915
select DEBUG_FS
select PREEMPT_COUNT
select REFCOUNT_FULL
select I2C_CHARDEV
select STACKDEPOT
select DRM_DP_AUX_CHARDEV
select X86_MSR # used by igt/pm_rpm
select DRM_VGEM # used by igt/prime_vgem (dmabuf interop checks)
select DRM_DEBUG_MM if DRM=y
select DRM_DEBUG_SELFTEST
select DMABUF_SELFTESTS
select SW_SYNC # signaling validation framework (igt/syncobj*)
@ -36,14 +36,14 @@ config DRM_I915_DEBUG
select DRM_I915_SELFTEST
select DRM_I915_DEBUG_RUNTIME_PM
select DRM_I915_DEBUG_MMIO
default n
help
Choose this option to turn on extra driver debugging that may affect
performance but will catch some internal issues.
default n
help
Choose this option to turn on extra driver debugging that may affect
performance but will catch some internal issues.
Recommended for driver developers only.
Recommended for driver developers only.
If in doubt, say "N".
If in doubt, say "N".
config DRM_I915_DEBUG_MMIO
bool "Always insert extra checks around mmio access by default"
@ -59,16 +59,16 @@ config DRM_I915_DEBUG_MMIO
If in doubt, say "N".
config DRM_I915_DEBUG_GEM
bool "Insert extra checks into the GEM internals"
default n
depends on DRM_I915_WERROR
help
Enable extra sanity checks (including BUGs) along the GEM driver
paths that may slow the system down and if hit hang the machine.
bool "Insert extra checks into the GEM internals"
default n
depends on DRM_I915_WERROR
help
Enable extra sanity checks (including BUGs) along the GEM driver
paths that may slow the system down and if hit hang the machine.
Recommended for driver developers only.
Recommended for driver developers only.
If in doubt, say "N".
If in doubt, say "N".
config DRM_I915_ERRLOG_GEM
bool "Insert extra logging (very verbose) for common GEM errors"
@ -111,41 +111,41 @@ config DRM_I915_TRACE_GTT
If in doubt, say "N".
config DRM_I915_SW_FENCE_DEBUG_OBJECTS
bool "Enable additional driver debugging for fence objects"
depends on DRM_I915
select DEBUG_OBJECTS
default n
help
Choose this option to turn on extra driver debugging that may affect
performance but will catch some internal issues.
bool "Enable additional driver debugging for fence objects"
depends on DRM_I915
select DEBUG_OBJECTS
default n
help
Choose this option to turn on extra driver debugging that may affect
performance but will catch some internal issues.
Recommended for driver developers only.
Recommended for driver developers only.
If in doubt, say "N".
If in doubt, say "N".
config DRM_I915_SW_FENCE_CHECK_DAG
bool "Enable additional driver debugging for detecting dependency cycles"
depends on DRM_I915
default n
help
Choose this option to turn on extra driver debugging that may affect
performance but will catch some internal issues.
bool "Enable additional driver debugging for detecting dependency cycles"
depends on DRM_I915
default n
help
Choose this option to turn on extra driver debugging that may affect
performance but will catch some internal issues.
Recommended for driver developers only.
Recommended for driver developers only.
If in doubt, say "N".
If in doubt, say "N".
config DRM_I915_DEBUG_GUC
bool "Enable additional driver debugging for GuC"
depends on DRM_I915
default n
help
Choose this option to turn on extra driver debugging that may affect
performance but will help resolve GuC related issues.
bool "Enable additional driver debugging for GuC"
depends on DRM_I915
default n
help
Choose this option to turn on extra driver debugging that may affect
performance but will help resolve GuC related issues.
Recommended for driver developers only.
Recommended for driver developers only.
If in doubt, say "N".
If in doubt, say "N".
config DRM_I915_SELFTEST
bool "Enable selftests upon driver load"
@ -178,15 +178,15 @@ config DRM_I915_SELFTEST_BROKEN
If in doubt, say "N".
config DRM_I915_LOW_LEVEL_TRACEPOINTS
bool "Enable low level request tracing events"
depends on DRM_I915
default n
help
Choose this option to turn on low level request tracing events.
This provides the ability to precisely monitor engine utilisation
and also analyze the request dependency resolving timeline.
bool "Enable low level request tracing events"
depends on DRM_I915
default n
help
Choose this option to turn on low level request tracing events.
This provides the ability to precisely monitor engine utilisation
and also analyze the request dependency resolving timeline.
If in doubt, say "N".
If in doubt, say "N".
config DRM_I915_DEBUG_VBLANK_EVADE
bool "Enable extra debug warnings for vblank evasion"

3
drivers/gpu/drm/i915/Makefile
View File

@ -51,6 +51,7 @@ i915-y += i915_drv.o \
i915_utils.o \
intel_csr.o \
intel_device_info.o \
intel_memory_region.o \
intel_pch.o \
intel_pm.o \
intel_runtime_pm.o \
@ -86,6 +87,7 @@ gt-y += \
gt/intel_gt_pm_irq.o \
gt/intel_gt_requests.o \
gt/intel_hangcheck.o \
gt/intel_llc.o \
gt/intel_lrc.o \
gt/intel_rc6.o \
gt/intel_renderstate.o \
@ -121,6 +123,7 @@ gem-y += \
gem/i915_gem_pages.o \
gem/i915_gem_phys.o \
gem/i915_gem_pm.o \
gem/i915_gem_region.o \
gem/i915_gem_shmem.o \
gem/i915_gem_shrinker.o \
gem/i915_gem_stolen.o \

10
drivers/gpu/drm/i915/display/intel_atomic_plane.c
View File

@ -195,14 +195,11 @@ get_crtc_from_states(const struct intel_plane_state *old_plane_state,
return NULL;
}
static int intel_plane_atomic_check(struct drm_plane *_plane,
struct drm_plane_state *_new_plane_state)
int intel_plane_atomic_check(struct intel_atomic_state *state,
struct intel_plane *plane)
{
struct intel_plane *plane = to_intel_plane(_plane);
struct intel_atomic_state *state =
to_intel_atomic_state(_new_plane_state->state);
struct intel_plane_state *new_plane_state =
to_intel_plane_state(_new_plane_state);
intel_atomic_get_new_plane_state(state, plane);
const struct intel_plane_state *old_plane_state =
intel_atomic_get_old_plane_state(state, plane);
struct intel_crtc *crtc =
@ -369,5 +366,4 @@ void i9xx_update_planes_on_crtc(struct intel_atomic_state *state,
const struct drm_plane_helper_funcs intel_plane_helper_funcs = {
.prepare_fb = intel_prepare_plane_fb,
.cleanup_fb = intel_cleanup_plane_fb,
.atomic_check = intel_plane_atomic_check,
};

2
drivers/gpu/drm/i915/display/intel_atomic_plane.h
View File

@ -41,6 +41,8 @@ int intel_plane_atomic_check_with_state(const struct intel_crtc_state *old_crtc_
struct intel_crtc_state *crtc_state,
const struct intel_plane_state *old_plane_state,
struct intel_plane_state *intel_state);
int intel_plane_atomic_check(struct intel_atomic_state *state,
struct intel_plane *plane);
int intel_plane_atomic_calc_changes(const struct intel_crtc_state *old_crtc_state,
struct intel_crtc_state *crtc_state,
const struct intel_plane_state *old_plane_state,

28
drivers/gpu/drm/i915/display/intel_bios.c
View File

@ -1270,7 +1270,7 @@ static void sanitize_ddc_pin(struct drm_i915_private *dev_priv,
DRM_DEBUG_KMS("port %c trying to use the same DDC pin (0x%x) as port %c, "
"disabling port %c DVI/HDMI support\n",
port_name(port), info->alternate_ddc_pin,
port_name(p), port_name(port));
port_name(p), port_name(p));
/*
* If we have multiple ports supposedly sharing the
@ -1278,9 +1278,14 @@ static void sanitize_ddc_pin(struct drm_i915_private *dev_priv,
* port. Otherwise they share the same ddc bin and
* system couldn't communicate with them separately.
*
* Give child device order the priority, first come first
* served.
* Give inverse child device order the priority,
* last one wins. Yes, there are real machines
* (eg. Asrock B250M-HDV) where VBT has both
* port A and port E with the same AUX ch and
* we must pick port E :(
*/
info = &dev_priv->vbt.ddi_port_info[p];
info->supports_dvi = false;
info->supports_hdmi = false;
info->alternate_ddc_pin = 0;
@ -1316,7 +1321,7 @@ static void sanitize_aux_ch(struct drm_i915_private *dev_priv,
DRM_DEBUG_KMS("port %c trying to use the same AUX CH (0x%x) as port %c, "
"disabling port %c DP support\n",
port_name(port), info->alternate_aux_channel,
port_name(p), port_name(port));
port_name(p), port_name(p));
/*
* If we have multiple ports supposedlt sharing the
@ -1324,9 +1329,14 @@ static void sanitize_aux_ch(struct drm_i915_private *dev_priv,
* port. Otherwise they share the same aux channel
* and system couldn't communicate with them separately.
*
* Give child device order the priority, first come first
* served.
* Give inverse child device order the priority,
* last one wins. Yes, there are real machines
* (eg. Asrock B250M-HDV) where VBT has both
* port A and port E with the same AUX ch and
* we must pick port E :(
*/
info = &dev_priv->vbt.ddi_port_info[p];
info->supports_dp = false;
info->alternate_aux_channel = 0;
}
@ -1389,6 +1399,7 @@ static enum port dvo_port_to_port(u8 dvo_port)
[PORT_D] = { DVO_PORT_HDMID, DVO_PORT_DPD, -1},
[PORT_E] = { DVO_PORT_CRT, DVO_PORT_HDMIE, DVO_PORT_DPE},
[PORT_F] = { DVO_PORT_HDMIF, DVO_PORT_DPF, -1},
[PORT_G] = { DVO_PORT_HDMIG, DVO_PORT_DPG, -1},
};
enum port port;
int i;
@ -1615,7 +1626,7 @@ parse_general_definitions(struct drm_i915_private *dev_priv,
expected_size = 37;
} else if (bdb->version <= 215) {
expected_size = 38;
} else if (bdb->version <= 216) {
} else if (bdb->version <= 229) {
expected_size = 39;
} else {
expected_size = sizeof(*child);
@ -2248,6 +2259,9 @@ enum aux_ch intel_bios_port_aux_ch(struct drm_i915_private *dev_priv,
case DP_AUX_F:
aux_ch = AUX_CH_F;
break;
case DP_AUX_G:
aux_ch = AUX_CH_G;
break;
default:
MISSING_CASE(info->alternate_aux_channel);
aux_ch = AUX_CH_A;

31
drivers/gpu/drm/i915/display/intel_cdclk.c
View File

@ -1900,9 +1900,11 @@ intel_set_cdclk_post_plane_update(struct drm_i915_private *dev_priv,
intel_set_cdclk(dev_priv, new_state, pipe);
}
static int intel_pixel_rate_to_cdclk(struct drm_i915_private *dev_priv,
int pixel_rate)
static int intel_pixel_rate_to_cdclk(const struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *dev_priv = to_i915(crtc_state->base.crtc->dev);
int pixel_rate = crtc_state->pixel_rate;
if (INTEL_GEN(dev_priv) >= 10 || IS_GEMINILAKE(dev_priv))
return DIV_ROUND_UP(pixel_rate, 2);
else if (IS_GEN(dev_priv, 9) ||
@ -1910,6 +1912,8 @@ static int intel_pixel_rate_to_cdclk(struct drm_i915_private *dev_priv,
return pixel_rate;
else if (IS_CHERRYVIEW(dev_priv))
return DIV_ROUND_UP(pixel_rate * 100, 95);
else if (crtc_state->double_wide)
return DIV_ROUND_UP(pixel_rate * 100, 90 * 2);
else
return DIV_ROUND_UP(pixel_rate * 100, 90);
}
@ -1923,7 +1927,7 @@ int intel_crtc_compute_min_cdclk(const struct intel_crtc_state *crtc_state)
if (!crtc_state->base.enable)
return 0;
min_cdclk = intel_pixel_rate_to_cdclk(dev_priv, crtc_state->pixel_rate);
min_cdclk = intel_pixel_rate_to_cdclk(crtc_state);
/* pixel rate mustn't exceed 95% of cdclk with IPS on BDW */
if (IS_BROADWELL(dev_priv) && hsw_crtc_state_ips_capable(crtc_state))
@ -2277,15 +2281,28 @@ static int intel_modeset_all_pipes(struct intel_atomic_state *state)
return 0;
}
static int fixed_modeset_calc_cdclk(struct intel_atomic_state *state)
{
int min_cdclk;
/*
* We can't change the cdclk frequency, but we still want to
* check that the required minimum frequency doesn't exceed
* the actual cdclk frequency.
*/
min_cdclk = intel_compute_min_cdclk(state);
if (min_cdclk < 0)
return min_cdclk;
return 0;
}
int intel_modeset_calc_cdclk(struct intel_atomic_state *state)
{
struct drm_i915_private *dev_priv = to_i915(state->base.dev);
enum pipe pipe;
int ret;
if (!dev_priv->display.modeset_calc_cdclk)
return 0;
ret = dev_priv->display.modeset_calc_cdclk(state);
if (ret)
return ret;
@ -2596,6 +2613,8 @@ void intel_init_cdclk_hooks(struct drm_i915_private *dev_priv)
} else if (IS_VALLEYVIEW(dev_priv)) {
dev_priv->display.set_cdclk = vlv_set_cdclk;
dev_priv->display.modeset_calc_cdclk = vlv_modeset_calc_cdclk;
} else {
dev_priv->display.modeset_calc_cdclk = fixed_modeset_calc_cdclk;
}
if (INTEL_GEN(dev_priv) >= 10 || IS_GEN9_LP(dev_priv))

33
drivers/gpu/drm/i915/display/intel_color.c
View File

@ -1420,6 +1420,9 @@ static int icl_color_check(struct intel_crtc_state *crtc_state)
static int i9xx_gamma_precision(const struct intel_crtc_state *crtc_state)
{
if (!crtc_state->gamma_enable)
return 0;
switch (crtc_state->gamma_mode) {
case GAMMA_MODE_MODE_8BIT:
return 8;
@ -1433,6 +1436,9 @@ static int i9xx_gamma_precision(const struct intel_crtc_state *crtc_state)
static int ilk_gamma_precision(const struct intel_crtc_state *crtc_state)
{
if (!crtc_state->gamma_enable)
return 0;
if ((crtc_state->csc_mode & CSC_POSITION_BEFORE_GAMMA) == 0)
return 0;
@ -1457,6 +1463,9 @@ static int chv_gamma_precision(const struct intel_crtc_state *crtc_state)
static int glk_gamma_precision(const struct intel_crtc_state *crtc_state)
{
if (!crtc_state->gamma_enable)
return 0;
switch (crtc_state->gamma_mode) {
case GAMMA_MODE_MODE_8BIT:
return 8;
@ -1473,9 +1482,6 @@ int intel_color_get_gamma_bit_precision(const struct intel_crtc_state *crtc_stat
struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
if (!crtc_state->gamma_enable)
return 0;
if (HAS_GMCH(dev_priv)) {
if (IS_CHERRYVIEW(dev_priv))
return chv_gamma_precision(crtc_state);
@ -1613,6 +1619,9 @@ i9xx_read_lut_8(const struct intel_crtc_state *crtc_state)
static void i9xx_read_luts(struct intel_crtc_state *crtc_state)
{
if (!crtc_state->gamma_enable)
return;
crtc_state->base.gamma_lut = i9xx_read_lut_8(crtc_state);
}
@ -1659,6 +1668,9 @@ i965_read_lut_10p6(const struct intel_crtc_state *crtc_state)
static void i965_read_luts(struct intel_crtc_state *crtc_state)
{
if (!crtc_state->gamma_enable)
return;
if (crtc_state->gamma_mode == GAMMA_MODE_MODE_8BIT)
crtc_state->base.gamma_lut = i9xx_read_lut_8(crtc_state);
else
@ -1701,10 +1713,10 @@ chv_read_cgm_lut(const struct intel_crtc_state *crtc_state)
static void chv_read_luts(struct intel_crtc_state *crtc_state)
{
if (crtc_state->gamma_mode == GAMMA_MODE_MODE_8BIT)
crtc_state->base.gamma_lut = i9xx_read_lut_8(crtc_state);
else
if (crtc_state->cgm_mode & CGM_PIPE_MODE_GAMMA)
crtc_state->base.gamma_lut = chv_read_cgm_lut(crtc_state);
else
i965_read_luts(crtc_state);
}
static struct drm_property_blob *
@ -1742,6 +1754,12 @@ ilk_read_lut_10(const struct intel_crtc_state *crtc_state)
static void ilk_read_luts(struct intel_crtc_state *crtc_state)
{
if (!crtc_state->gamma_enable)
return;
if ((crtc_state->csc_mode & CSC_POSITION_BEFORE_GAMMA) == 0)
return;
if (crtc_state->gamma_mode == GAMMA_MODE_MODE_8BIT)
crtc_state->base.gamma_lut = i9xx_read_lut_8(crtc_state);
else
@ -1788,6 +1806,9 @@ glk_read_lut_10(const struct intel_crtc_state *crtc_state, u32 prec_index)
static void glk_read_luts(struct intel_crtc_state *crtc_state)
{
if (!crtc_state->gamma_enable)
return;
if (crtc_state->gamma_mode == GAMMA_MODE_MODE_8BIT)
crtc_state->base.gamma_lut = i9xx_read_lut_8(crtc_state);
else

21
drivers/gpu/drm/i915/display/intel_connector.c
View File

@ -277,7 +277,22 @@ intel_attach_aspect_ratio_property(struct drm_connector *connector)
void
intel_attach_colorspace_property(struct drm_connector *connector)
{
if (!drm_mode_create_hdmi_colorspace_property(connector))
drm_object_attach_property(&connector->base,
connector->colorspace_property, 0);
switch (connector->connector_type) {
case DRM_MODE_CONNECTOR_HDMIA:
case DRM_MODE_CONNECTOR_HDMIB:
if (drm_mode_create_hdmi_colorspace_property(connector))
return;
break;
case DRM_MODE_CONNECTOR_DisplayPort:
case DRM_MODE_CONNECTOR_eDP:
if (drm_mode_create_dp_colorspace_property(connector))
return;
break;
default:
DRM_DEBUG_KMS("Colorspace property not supported\n");
return;
}
drm_object_attach_property(&connector->base,
connector->colorspace_property, 0);
}

178
drivers/gpu/drm/i915/display/intel_ddi.c
View File

@ -45,6 +45,7 @@
#include "intel_lspcon.h"
#include "intel_panel.h"
#include "intel_psr.h"
#include "intel_sprite.h"
#include "intel_tc.h"
#include "intel_vdsc.h"
@ -1740,7 +1741,8 @@ static void intel_ddi_clock_get(struct intel_encoder *encoder,
hsw_ddi_clock_get(encoder, pipe_config);
}
void intel_ddi_set_pipe_settings(const struct intel_crtc_state *crtc_state)
void intel_ddi_set_dp_msa(const struct intel_crtc_state *crtc_state,
const struct drm_connector_state *conn_state)
{
struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
@ -1752,20 +1754,20 @@ void intel_ddi_set_pipe_settings(const struct intel_crtc_state *crtc_state)
WARN_ON(transcoder_is_dsi(cpu_transcoder));
temp = TRANS_MSA_SYNC_CLK;
temp = DP_MSA_MISC_SYNC_CLOCK;
switch (crtc_state->pipe_bpp) {
case 18:
temp |= TRANS_MSA_6_BPC;
temp |= DP_MSA_MISC_6_BPC;
break;
case 24:
temp |= TRANS_MSA_8_BPC;
temp |= DP_MSA_MISC_8_BPC;
break;
case 30:
temp |= TRANS_MSA_10_BPC;
temp |= DP_MSA_MISC_10_BPC;
break;
case 36:
temp |= TRANS_MSA_12_BPC;
temp |= DP_MSA_MISC_12_BPC;
break;
default:
MISSING_CASE(crtc_state->pipe_bpp);
@ -1777,7 +1779,7 @@ void intel_ddi_set_pipe_settings(const struct intel_crtc_state *crtc_state)
crtc_state->output_format != INTEL_OUTPUT_FORMAT_RGB);
if (crtc_state->limited_color_range)
temp |= TRANS_MSA_CEA_RANGE;
temp |= DP_MSA_MISC_COLOR_CEA_RGB;
/*
* As per DP 1.2 spec section 2.3.4.3 while sending
@ -1785,17 +1787,19 @@ void intel_ddi_set_pipe_settings(const struct intel_crtc_state *crtc_state)
* colorspace information.
*/
if (crtc_state->output_format == INTEL_OUTPUT_FORMAT_YCBCR444)
temp |= TRANS_MSA_SAMPLING_444 | TRANS_MSA_CLRSP_YCBCR |
TRANS_MSA_YCBCR_BT709;
temp |= DP_MSA_MISC_COLOR_YCBCR_444_BT709;
/*
* As per DP 1.4a spec section 2.2.4.3 [MSA Field for Indication
* of Color Encoding Format and Content Color Gamut] while sending
* YCBCR 420 signals we should program MSA MISC1 fields which
* indicate VSC SDP for the Pixel Encoding/Colorimetry Format.
* YCBCR 420, HDR BT.2020 signals we should program MSA MISC1 fields
* which indicate VSC SDP for the Pixel Encoding/Colorimetry Format.
*
* FIXME MST doesn't pass in the conn_state
*/
if (crtc_state->output_format == INTEL_OUTPUT_FORMAT_YCBCR420)
temp |= TRANS_MSA_USE_VSC_SDP;
if (conn_state && intel_dp_needs_vsc_sdp(crtc_state, conn_state))
temp |= DP_MSA_MISC_COLOR_VSC_SDP;
I915_WRITE(TRANS_MSA_MISC(cpu_transcoder), temp);
}
@ -3330,6 +3334,86 @@ static void intel_ddi_disable_fec_state(struct intel_encoder *encoder,
POSTING_READ(intel_dp->regs.dp_tp_ctl);
}
static void
tgl_clear_psr2_transcoder_exitline(const struct intel_crtc_state *cstate)
{
struct drm_i915_private *dev_priv = to_i915(cstate->base.crtc->dev);
u32 val;
if (!cstate->dc3co_exitline)
return;
val = I915_READ(EXITLINE(cstate->cpu_transcoder));
val &= ~(EXITLINE_MASK | EXITLINE_ENABLE);
I915_WRITE(EXITLINE(cstate->cpu_transcoder), val);
}
static void
tgl_set_psr2_transcoder_exitline(const struct intel_crtc_state *cstate)
{
u32 val, exit_scanlines;
struct drm_i915_private *dev_priv = to_i915(cstate->base.crtc->dev);
if (!cstate->dc3co_exitline)
return;
exit_scanlines = cstate->dc3co_exitline;
exit_scanlines <<= EXITLINE_SHIFT;
val = I915_READ(EXITLINE(cstate->cpu_transcoder));
val &= ~(EXITLINE_MASK | EXITLINE_ENABLE);
val |= exit_scanlines;
val |= EXITLINE_ENABLE;
I915_WRITE(EXITLINE(cstate->cpu_transcoder), val);
}
static void tgl_dc3co_exitline_compute_config(struct intel_encoder *encoder,
struct intel_crtc_state *cstate)
{
u32 exit_scanlines;
struct drm_i915_private *dev_priv = to_i915(cstate->base.crtc->dev);
u32 crtc_vdisplay = cstate->base.adjusted_mode.crtc_vdisplay;
cstate->dc3co_exitline = 0;
if (!(dev_priv->csr.allowed_dc_mask & DC_STATE_EN_DC3CO))
return;
/* B.Specs:49196 DC3CO only works with pipeA and DDIA.*/
if (to_intel_crtc(cstate->base.crtc)->pipe != PIPE_A ||
encoder->port != PORT_A)
return;
if (!cstate->has_psr2 || !cstate->base.active)
return;
/*
* DC3CO Exit time 200us B.Spec 49196
* PSR2 transcoder Early Exit scanlines = ROUNDUP(200 / line time) + 1
*/
exit_scanlines =
intel_usecs_to_scanlines(&cstate->base.adjusted_mode, 200) + 1;
if (WARN_ON(exit_scanlines > crtc_vdisplay))
return;
cstate->dc3co_exitline = crtc_vdisplay - exit_scanlines;
DRM_DEBUG_KMS("DC3CO exit scanlines %d\n", cstate->dc3co_exitline);
}
static void tgl_dc3co_exitline_get_config(struct intel_crtc_state *crtc_state)
{
u32 val;
struct drm_i915_private *dev_priv = to_i915(crtc_state->base.crtc->dev);
if (INTEL_GEN(dev_priv) < 12)
return;
val = I915_READ(EXITLINE(crtc_state->cpu_transcoder));
if (val & EXITLINE_ENABLE)
crtc_state->dc3co_exitline = val & EXITLINE_MASK;
}
static void tgl_ddi_pre_enable_dp(struct intel_encoder *encoder,
const struct intel_crtc_state *crtc_state,
const struct drm_connector_state *conn_state)
@ -3342,6 +3426,7 @@ static void tgl_ddi_pre_enable_dp(struct intel_encoder *encoder,
int level = intel_ddi_dp_level(intel_dp);
enum transcoder transcoder = crtc_state->cpu_transcoder;
tgl_set_psr2_transcoder_exitline(crtc_state);
intel_dp_set_link_params(intel_dp, crtc_state->port_clock,
crtc_state->lane_count, is_mst);
@ -3415,7 +3500,8 @@ static void tgl_ddi_pre_enable_dp(struct intel_encoder *encoder,
intel_dp_start_link_train(intel_dp);
/* 7.k */
intel_dp_stop_link_train(intel_dp);
if (!is_trans_port_sync_mode(crtc_state))
intel_dp_stop_link_train(intel_dp);
/*
* TODO: enable clock gating
@ -3489,7 +3575,8 @@ static void hsw_ddi_pre_enable_dp(struct intel_encoder *encoder,
true);
intel_dp_sink_set_fec_ready(intel_dp, crtc_state);
intel_dp_start_link_train(intel_dp);
if (port != PORT_A || INTEL_GEN(dev_priv) >= 9)
if ((port != PORT_A || INTEL_GEN(dev_priv) >= 9) &&
!is_trans_port_sync_mode(crtc_state))
intel_dp_stop_link_train(intel_dp);
intel_ddi_enable_fec(encoder, crtc_state);
@ -3512,6 +3599,8 @@ static void intel_ddi_pre_enable_dp(struct intel_encoder *encoder,
tgl_ddi_pre_enable_dp(encoder, crtc_state, conn_state);
else
hsw_ddi_pre_enable_dp(encoder, crtc_state, conn_state);
intel_ddi_set_dp_msa(crtc_state, conn_state);
}
static void intel_ddi_pre_enable_hdmi(struct intel_encoder *encoder,
@ -3666,6 +3755,7 @@ static void intel_ddi_post_disable_dp(struct intel_encoder *encoder,
dig_port->ddi_io_power_domain);
intel_ddi_clk_disable(encoder);
tgl_clear_psr2_transcoder_exitline(old_crtc_state);
}
static void intel_ddi_post_disable_hdmi(struct intel_encoder *encoder,
@ -3768,7 +3858,8 @@ static void intel_enable_ddi_dp(struct intel_encoder *encoder,
intel_edp_backlight_on(crtc_state, conn_state);
intel_psr_enable(intel_dp, crtc_state);
intel_dp_ycbcr_420_enable(intel_dp, crtc_state);
intel_dp_vsc_enable(intel_dp, crtc_state, conn_state);
intel_dp_hdr_metadata_enable(intel_dp, crtc_state, conn_state);
intel_edp_drrs_enable(intel_dp, crtc_state);
if (crtc_state->has_audio)
@ -3926,7 +4017,7 @@ static void intel_ddi_update_pipe_dp(struct intel_encoder *encoder,
{
struct intel_dp *intel_dp = enc_to_intel_dp(&encoder->base);
intel_ddi_set_pipe_settings(crtc_state);
intel_ddi_set_dp_msa(crtc_state, conn_state);
intel_psr_update(intel_dp, crtc_state);
intel_edp_drrs_enable(intel_dp, crtc_state);
@ -4212,6 +4303,9 @@ void intel_ddi_get_config(struct intel_encoder *encoder,
break;
}
if (encoder->type == INTEL_OUTPUT_EDP)
tgl_dc3co_exitline_get_config(pipe_config);
pipe_config->has_audio =
intel_ddi_is_audio_enabled(dev_priv, cpu_transcoder);
@ -4289,10 +4383,13 @@ static int intel_ddi_compute_config(struct intel_encoder *encoder,
if (HAS_TRANSCODER_EDP(dev_priv) && port == PORT_A)
pipe_config->cpu_transcoder = TRANSCODER_EDP;
if (intel_crtc_has_type(pipe_config, INTEL_OUTPUT_HDMI))
if (intel_crtc_has_type(pipe_config, INTEL_OUTPUT_HDMI)) {
ret = intel_hdmi_compute_config(encoder, pipe_config, conn_state);
else
} else {
ret = intel_dp_compute_config(encoder, pipe_config, conn_state);
tgl_dc3co_exitline_compute_config(encoder, pipe_config);
}
if (ret)
return ret;
@ -4661,46 +4758,9 @@ void intel_ddi_init(struct drm_i915_private *dev_priv, enum port port)
intel_encoder->update_complete = intel_ddi_update_complete;
}
switch (port) {
case PORT_A:
intel_dig_port->ddi_io_power_domain =
POWER_DOMAIN_PORT_DDI_A_IO;
break;
case PORT_B:
intel_dig_port->ddi_io_power_domain =
POWER_DOMAIN_PORT_DDI_B_IO;
break;
case PORT_C:
intel_dig_port->ddi_io_power_domain =
POWER_DOMAIN_PORT_DDI_C_IO;
break;
case PORT_D:
intel_dig_port->ddi_io_power_domain =
POWER_DOMAIN_PORT_DDI_D_IO;
break;
case PORT_E:
intel_dig_port->ddi_io_power_domain =
POWER_DOMAIN_PORT_DDI_E_IO;
break;
case PORT_F:
intel_dig_port->ddi_io_power_domain =
POWER_DOMAIN_PORT_DDI_F_IO;
break;
case PORT_G:
intel_dig_port->ddi_io_power_domain =
POWER_DOMAIN_PORT_DDI_G_IO;
break;
case PORT_H:
intel_dig_port->ddi_io_power_domain =
POWER_DOMAIN_PORT_DDI_H_IO;
break;
case PORT_I:
intel_dig_port->ddi_io_power_domain =
POWER_DOMAIN_PORT_DDI_I_IO;
break;
default:
MISSING_CASE(port);
}
WARN_ON(port > PORT_I);
intel_dig_port->ddi_io_power_domain = POWER_DOMAIN_PORT_DDI_A_IO +
port - PORT_A;
if (init_dp) {
if (!intel_ddi_init_dp_connector(intel_dig_port))

3
drivers/gpu/drm/i915/display/intel_ddi.h
View File

@ -30,7 +30,8 @@ void intel_ddi_enable_transcoder_func(const struct intel_crtc_state *crtc_state)
void intel_ddi_disable_transcoder_func(const struct intel_crtc_state *crtc_state);
void intel_ddi_enable_pipe_clock(const struct intel_crtc_state *crtc_state);
void intel_ddi_disable_pipe_clock(const struct intel_crtc_state *crtc_state);
void intel_ddi_set_pipe_settings(const struct intel_crtc_state *crtc_state);
void intel_ddi_set_dp_msa(const struct intel_crtc_state *crtc_state,
const struct drm_connector_state *conn_state);
bool intel_ddi_connector_get_hw_state(struct intel_connector *intel_connector);
void intel_ddi_get_config(struct intel_encoder *encoder,
struct intel_crtc_state *pipe_config);

1023
drivers/gpu/drm/i915/display/intel_display.c
View File

File diff suppressed because it is too large Load Diff

17
drivers/gpu/drm/i915/display/intel_display.h
View File

@ -27,6 +27,7 @@
#include <drm/drm_util.h>
#include <drm/i915_drm.h>
#include "intel_dp_link_training.h"
enum link_m_n_set;
struct dpll;
@ -54,6 +55,7 @@ struct intel_plane;
struct intel_plane_state;
struct intel_remapped_info;
struct intel_rotation_info;
struct intel_crtc_state;
enum i915_gpio {
GPIOA,
@ -93,6 +95,7 @@ enum pipe {
#define pipe_name(p) ((p) + 'A')
enum transcoder {
INVALID_TRANSCODER = -1,
/*
* The following transcoders have a 1:1 transcoder -> pipe mapping,
* keep their values fixed: the code assumes that TRANSCODER_A=0, the
@ -271,6 +274,7 @@ enum aux_ch {
AUX_CH_D,
AUX_CH_E, /* ICL+ */
AUX_CH_F,
AUX_CH_G,
};
#define aux_ch_name(a) ((a) + 'A')
@ -350,7 +354,7 @@ enum phy_fia {
&(dev)->mode_config.plane_list, \
base.head) \
for_each_if((plane_mask) & \
drm_plane_mask(&intel_plane->base)))
drm_plane_mask(&intel_plane->base))
#define for_each_intel_plane_on_crtc(dev, intel_crtc, intel_plane) \
list_for_each_entry(intel_plane, \
@ -440,6 +444,14 @@ enum phy_fia {
(__i)--) \
for_each_if(crtc)
#define intel_atomic_crtc_state_for_each_plane_state( \
plane, plane_state, \
crtc_state) \
for_each_intel_plane_mask(((crtc_state)->base.state->dev), (plane), \
((crtc_state)->base.plane_mask)) \
for_each_if ((plane_state = \
to_intel_plane_state(__drm_atomic_get_current_plane_state((crtc_state)->base.state, &plane->base))))
void intel_link_compute_m_n(u16 bpp, int nlanes,
int pixel_clock, int link_clock,
struct intel_link_m_n *m_n,
@ -453,6 +465,7 @@ enum drm_mode_status
intel_mode_valid_max_plane_size(struct drm_i915_private *dev_priv,
const struct drm_display_mode *mode);
enum phy intel_port_to_phy(struct drm_i915_private *i915, enum port port);
bool is_trans_port_sync_mode(const struct intel_crtc_state *state);
void intel_plane_destroy(struct drm_plane *plane);
void i830_enable_pipe(struct drm_i915_private *dev_priv, enum pipe pipe);
@ -531,8 +544,6 @@ void intel_dp_get_m_n(struct intel_crtc *crtc,
struct intel_crtc_state *pipe_config);
void intel_dp_set_m_n(const struct intel_crtc_state *crtc_state,
enum link_m_n_set m_n);
void intel_dp_ycbcr_420_enable(struct intel_dp *intel_dp,
const struct intel_crtc_state *crtc_state);
int intel_dotclock_calculate(int link_freq, const struct intel_link_m_n *m_n);
bool bxt_find_best_dpll(struct intel_crtc_state *crtc_state,
struct dpll *best_clock);

154
drivers/gpu/drm/i915/display/intel_display_power.c
View File

@ -695,7 +695,11 @@ static u32 gen9_dc_mask(struct drm_i915_private *dev_priv)
u32 mask;
mask = DC_STATE_EN_UPTO_DC5;
if (INTEL_GEN(dev_priv) >= 11)
if (INTEL_GEN(dev_priv) >= 12)
mask |= DC_STATE_EN_DC3CO | DC_STATE_EN_UPTO_DC6
| DC_STATE_EN_DC9;
else if (IS_GEN(dev_priv, 11))
mask |= DC_STATE_EN_UPTO_DC6 | DC_STATE_EN_DC9;
else if (IS_GEN9_LP(dev_priv))
mask |= DC_STATE_EN_DC9;
@ -765,6 +769,52 @@ static void gen9_set_dc_state(struct drm_i915_private *dev_priv, u32 state)
dev_priv->csr.dc_state = val & mask;
}
static u32
sanitize_target_dc_state(struct drm_i915_private *dev_priv,
u32 target_dc_state)
{
u32 states[] = {
DC_STATE_EN_UPTO_DC6,
DC_STATE_EN_UPTO_DC5,
DC_STATE_EN_DC3CO,
DC_STATE_DISABLE,
};
int i;
for (i = 0; i < ARRAY_SIZE(states) - 1; i++) {
if (target_dc_state != states[i])
continue;
if (dev_priv->csr.allowed_dc_mask & target_dc_state)
break;
target_dc_state = states[i + 1];
}
return target_dc_state;
}
static void tgl_enable_dc3co(struct drm_i915_private *dev_priv)
{
DRM_DEBUG_KMS("Enabling DC3CO\n");
gen9_set_dc_state(dev_priv, DC_STATE_EN_DC3CO);
}
static void tgl_disable_dc3co(struct drm_i915_private *dev_priv)
{
u32 val;
DRM_DEBUG_KMS("Disabling DC3CO\n");
val = I915_READ(DC_STATE_EN);
val &= ~DC_STATE_DC3CO_STATUS;
I915_WRITE(DC_STATE_EN, val);
gen9_set_dc_state(dev_priv, DC_STATE_DISABLE);
/*
* Delay of 200us DC3CO Exit time B.Spec 49196
*/
usleep_range(200, 210);
}
static void bxt_enable_dc9(struct drm_i915_private *dev_priv)
{
assert_can_enable_dc9(dev_priv);
@ -820,6 +870,51 @@ lookup_power_well(struct drm_i915_private *dev_priv,
return &dev_priv->power_domains.power_wells[0];
}
/**
* intel_display_power_set_target_dc_state - Set target dc state.
* @dev_priv: i915 device
* @state: state which needs to be set as target_dc_state.
*
* This function set the "DC off" power well target_dc_state,
* based upon this target_dc_stste, "DC off" power well will
* enable desired DC state.
*/
void intel_display_power_set_target_dc_state(struct drm_i915_private *dev_priv,
u32 state)
{
struct i915_power_well *power_well;
bool dc_off_enabled;
struct i915_power_domains *power_domains = &dev_priv->power_domains;
mutex_lock(&power_domains->lock);
power_well = lookup_power_well(dev_priv, SKL_DISP_DC_OFF);
if (WARN_ON(!power_well))
goto unlock;
state = sanitize_target_dc_state(dev_priv, state);
if (state == dev_priv->csr.target_dc_state)
goto unlock;
dc_off_enabled = power_well->desc->ops->is_enabled(dev_priv,
power_well);
/*
* If DC off power well is disabled, need to enable and disable the
* DC off power well to effect target DC state.
*/
if (!dc_off_enabled)
power_well->desc->ops->enable(dev_priv, power_well);
dev_priv->csr.target_dc_state = state;
if (!dc_off_enabled)
power_well->desc->ops->disable(dev_priv, power_well);
unlock:
mutex_unlock(&power_domains->lock);
}
static void assert_can_enable_dc5(struct drm_i915_private *dev_priv)
{
bool pg2_enabled = intel_display_power_well_is_enabled(dev_priv,
@ -932,7 +1027,8 @@ static void bxt_verify_ddi_phy_power_wells(struct drm_i915_private *dev_priv)
static bool gen9_dc_off_power_well_enabled(struct drm_i915_private *dev_priv,
struct i915_power_well *power_well)
{
return (I915_READ(DC_STATE_EN) & DC_STATE_EN_UPTO_DC5_DC6_MASK) == 0;
return ((I915_READ(DC_STATE_EN) & DC_STATE_EN_DC3CO) == 0 &&
(I915_READ(DC_STATE_EN) & DC_STATE_EN_UPTO_DC5_DC6_MASK) == 0);
}
static void gen9_assert_dbuf_enabled(struct drm_i915_private *dev_priv)
@ -948,6 +1044,11 @@ static void gen9_disable_dc_states(struct drm_i915_private *dev_priv)
{
struct intel_cdclk_state cdclk_state = {};
if (dev_priv->csr.target_dc_state == DC_STATE_EN_DC3CO) {
tgl_disable_dc3co(dev_priv);
return;
}
gen9_set_dc_state(dev_priv, DC_STATE_DISABLE);
dev_priv->display.get_cdclk(dev_priv, &cdclk_state);
@ -980,10 +1081,17 @@ static void gen9_dc_off_power_well_disable(struct drm_i915_private *dev_priv,
if (!dev_priv->csr.dmc_payload)
return;
if (dev_priv->csr.allowed_dc_mask & DC_STATE_EN_UPTO_DC6)
switch (dev_priv->csr.target_dc_state) {
case DC_STATE_EN_DC3CO:
tgl_enable_dc3co(dev_priv);
break;
case DC_STATE_EN_UPTO_DC6:
skl_enable_dc6(dev_priv);
else if (dev_priv->csr.allowed_dc_mask & DC_STATE_EN_UPTO_DC5)
break;
case DC_STATE_EN_UPTO_DC5:
gen9_enable_dc5(dev_priv);
break;
}
}
static void i9xx_power_well_sync_hw_noop(struct drm_i915_private *dev_priv,
@ -2931,7 +3039,7 @@ static const struct i915_power_well_desc skl_power_wells[] = {
.name = "DC off",
.domains = SKL_DISPLAY_DC_OFF_POWER_DOMAINS,
.ops = &gen9_dc_off_power_well_ops,
.id = DISP_PW_ID_NONE,
.id = SKL_DISP_DC_OFF,
},
{
.name = "power well 2",
@ -3013,7 +3121,7 @@ static const struct i915_power_well_desc bxt_power_wells[] = {
.name = "DC off",
.domains = BXT_DISPLAY_DC_OFF_POWER_DOMAINS,
.ops = &gen9_dc_off_power_well_ops,
.id = DISP_PW_ID_NONE,
.id = SKL_DISP_DC_OFF,
},
{
.name = "power well 2",
@ -3073,7 +3181,7 @@ static const struct i915_power_well_desc glk_power_wells[] = {
.name = "DC off",
.domains = GLK_DISPLAY_DC_OFF_POWER_DOMAINS,
.ops = &gen9_dc_off_power_well_ops,
.id = DISP_PW_ID_NONE,
.id = SKL_DISP_DC_OFF,
},
{
.name = "power well 2",
@ -3242,7 +3350,7 @@ static const struct i915_power_well_desc cnl_power_wells[] = {
.name = "DC off",
.domains = CNL_DISPLAY_DC_OFF_POWER_DOMAINS,
.ops = &gen9_dc_off_power_well_ops,
.id = DISP_PW_ID_NONE,
.id = SKL_DISP_DC_OFF,
},
{
.name = "power well 2",
@ -3370,7 +3478,7 @@ static const struct i915_power_well_desc icl_power_wells[] = {
.name = "DC off",
.domains = ICL_DISPLAY_DC_OFF_POWER_DOMAINS,
.ops = &gen9_dc_off_power_well_ops,
.id = DISP_PW_ID_NONE,
.id = SKL_DISP_DC_OFF,
},
{
.name = "power well 2",
@ -3603,7 +3711,7 @@ static const struct i915_power_well_desc tgl_power_wells[] = {
.name = "DC off",
.domains = TGL_DISPLAY_DC_OFF_POWER_DOMAINS,
.ops = &gen9_dc_off_power_well_ops,
.id = DISP_PW_ID_NONE,
.id = SKL_DISP_DC_OFF,
},
{
.name = "power well 2",
@ -3924,14 +4032,17 @@ static u32 get_allowed_dc_mask(const struct drm_i915_private *dev_priv,
int requested_dc;
int max_dc;
if (INTEL_GEN(dev_priv) >= 11) {
max_dc = 2;
if (INTEL_GEN(dev_priv) >= 12) {
max_dc = 4;
/*
* DC9 has a separate HW flow from the rest of the DC states,
* not depending on the DMC firmware. It's needed by system
* suspend/resume, so allow it unconditionally.
*/
mask = DC_STATE_EN_DC9;
} else if (IS_GEN(dev_priv, 11)) {
max_dc = 2;
mask = DC_STATE_EN_DC9;
} else if (IS_GEN(dev_priv, 10) || IS_GEN9_BC(dev_priv)) {
max_dc = 2;
mask = 0;
@ -3950,7 +4061,7 @@ static u32 get_allowed_dc_mask(const struct drm_i915_private *dev_priv,
requested_dc = enable_dc;
} else if (enable_dc == -1) {
requested_dc = max_dc;
} else if (enable_dc > max_dc && enable_dc <= 2) {
} else if (enable_dc > max_dc && enable_dc <= 4) {
DRM_DEBUG_KMS("Adjusting requested max DC state (%d->%d)\n",
enable_dc, max_dc);
requested_dc = max_dc;
@ -3959,10 +4070,20 @@ static u32 get_allowed_dc_mask(const struct drm_i915_private *dev_priv,
requested_dc = max_dc;
}
if (requested_dc > 1)
switch (requested_dc) {
case 4:
mask |= DC_STATE_EN_DC3CO | DC_STATE_EN_UPTO_DC6;
break;
case 3:
mask |= DC_STATE_EN_DC3CO | DC_STATE_EN_UPTO_DC5;
break;
case 2:
mask |= DC_STATE_EN_UPTO_DC6;
if (requested_dc > 0)
break;
case 1:
mask |= DC_STATE_EN_UPTO_DC5;
break;
}
DRM_DEBUG_KMS("Allowed DC state mask %02x\n", mask);
@ -4023,6 +4144,9 @@ int intel_power_domains_init(struct drm_i915_private *dev_priv)
dev_priv->csr.allowed_dc_mask =
get_allowed_dc_mask(dev_priv, i915_modparams.enable_dc);
dev_priv->csr.target_dc_state =
sanitize_target_dc_state(dev_priv, DC_STATE_EN_UPTO_DC6);
BUILD_BUG_ON(POWER_DOMAIN_NUM > 64);
mutex_init(&power_domains->lock);

3
drivers/gpu/drm/i915/display/intel_display_power.h
View File

@ -100,6 +100,7 @@ enum i915_power_well_id {
SKL_DISP_PW_MISC_IO,
SKL_DISP_PW_1,
SKL_DISP_PW_2,
SKL_DISP_DC_OFF,
};
#define POWER_DOMAIN_PIPE(pipe) ((pipe) + POWER_DOMAIN_PIPE_A)
@ -256,6 +257,8 @@ void intel_display_power_suspend_late(struct drm_i915_private *i915);
void intel_display_power_resume_early(struct drm_i915_private *i915);
void intel_display_power_suspend(struct drm_i915_private *i915);
void intel_display_power_resume(struct drm_i915_private *i915);
void intel_display_power_set_target_dc_state(struct drm_i915_private *dev_priv,
u32 state);
const char *
intel_display_power_domain_str(enum intel_display_power_domain domain);

7
drivers/gpu/drm/i915/display/intel_display_types.h
View File

@ -870,6 +870,7 @@ struct intel_crtc_state {
bool has_psr;
bool has_psr2;
u32 dc3co_exitline;
/*
* Frequence the dpll for the port should run at. Differs from the
@ -990,6 +991,12 @@ struct intel_crtc_state {
/* Forward Error correction State */
bool fec_enable;
/* Pointer to master transcoder in case of tiled displays */
enum transcoder master_transcoder;
/* Bitmask to indicate slaves attached */
u8 sync_mode_slaves_mask;
};
struct intel_crtc {

235
drivers/gpu/drm/i915/display/intel_dp.c
View File

@ -591,6 +591,25 @@ static u8 intel_dp_dsc_get_slice_count(struct intel_dp *intel_dp,
return 0;
}
static bool intel_dp_hdisplay_bad(struct drm_i915_private *dev_priv,
int hdisplay)
{
/*
* Older platforms don't like hdisplay==4096 with DP.
*
* On ILK/SNB/IVB the pipe seems to be somewhat running (scanline
* and frame counter increment), but we don't get vblank interrupts,
* and the pipe underruns immediately. The link also doesn't seem
* to get trained properly.
*
* On CHV the vblank interrupts don't seem to disappear but
* otherwise the symptoms are similar.
*
* TODO: confirm the behaviour on HSW+
*/
return hdisplay == 4096 && !HAS_DDI(dev_priv);
}
static enum drm_mode_status
intel_dp_mode_valid(struct drm_connector *connector,
struct drm_display_mode *mode)
@ -626,6 +645,9 @@ intel_dp_mode_valid(struct drm_connector *connector,
max_rate = intel_dp_max_data_rate(max_link_clock, max_lanes);
mode_rate = intel_dp_link_required(target_clock, 18);
if (intel_dp_hdisplay_bad(dev_priv, mode->hdisplay))
return MODE_H_ILLEGAL;
/*
* Output bpp is stored in 6.4 format so right shift by 4 to get the
* integer value since we support only integer values of bpp.
@ -1644,6 +1666,7 @@ static i915_reg_t skl_aux_ctl_reg(struct intel_dp *intel_dp)
case AUX_CH_D:
case AUX_CH_E:
case AUX_CH_F:
case AUX_CH_G:
return DP_AUX_CH_CTL(aux_ch);
default:
MISSING_CASE(aux_ch);
@ -1664,6 +1687,7 @@ static i915_reg_t skl_aux_data_reg(struct intel_dp *intel_dp, int index)
case AUX_CH_D:
case AUX_CH_E:
case AUX_CH_F:
case AUX_CH_G:
return DP_AUX_CH_DATA(aux_ch, index);
default:
MISSING_CASE(aux_ch);
@ -2297,6 +2321,7 @@ intel_dp_compute_config(struct intel_encoder *encoder,
pipe_config->has_pch_encoder = true;
pipe_config->output_format = INTEL_OUTPUT_FORMAT_RGB;
if (lspcon->active)
lspcon_ycbcr420_config(&intel_connector->base, pipe_config);
else
@ -2342,6 +2367,9 @@ intel_dp_compute_config(struct intel_encoder *encoder,
if (adjusted_mode->flags & DRM_MODE_FLAG_DBLCLK)
return -EINVAL;
if (intel_dp_hdisplay_bad(dev_priv, adjusted_mode->crtc_hdisplay))
return -EINVAL;
ret = intel_dp_compute_link_config(encoder, pipe_config, conn_state);
if (ret < 0)
return ret;
@ -4450,7 +4478,7 @@ intel_dp_configure_mst(struct intel_dp *intel_dp)
&dp_to_dig_port(intel_dp)->base;
bool sink_can_mst = intel_dp_sink_can_mst(intel_dp);
DRM_DEBUG_KMS("[ENCODER:%d:%s] MST support? port: %s, sink: %s, modparam: %s\n",
DRM_DEBUG_KMS("[ENCODER:%d:%s] MST support: port: %s, sink: %s, modparam: %s\n",
encoder->base.base.id, encoder->base.name,
yesno(intel_dp->can_mst), yesno(sink_can_mst),
yesno(i915_modparams.enable_dp_mst));
@ -4473,9 +4501,36 @@ intel_dp_get_sink_irq_esi(struct intel_dp *intel_dp, u8 *sink_irq_vector)
DP_DPRX_ESI_LEN;
}
bool
intel_dp_needs_vsc_sdp(const struct intel_crtc_state *crtc_state,
const struct drm_connector_state *conn_state)
{
/*
* As per DP 1.4a spec section 2.2.4.3 [MSA Field for Indication
* of Color Encoding Format and Content Color Gamut], in order to
* sending YCBCR 420 or HDR BT.2020 signals we should use DP VSC SDP.
*/
if (crtc_state->output_format == INTEL_OUTPUT_FORMAT_YCBCR420)
return true;
switch (conn_state->colorspace) {
case DRM_MODE_COLORIMETRY_SYCC_601:
case DRM_MODE_COLORIMETRY_OPYCC_601:
case DRM_MODE_COLORIMETRY_BT2020_YCC:
case DRM_MODE_COLORIMETRY_BT2020_RGB:
case DRM_MODE_COLORIMETRY_BT2020_CYCC:
return true;
default:
break;
}
return false;
}
static void
intel_pixel_encoding_setup_vsc(struct intel_dp *intel_dp,
const struct intel_crtc_state *crtc_state)
intel_dp_setup_vsc_sdp(struct intel_dp *intel_dp,
const struct intel_crtc_state *crtc_state,
const struct drm_connector_state *conn_state)
{
struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
struct dp_sdp vsc_sdp = {};
@ -4496,13 +4551,55 @@ intel_pixel_encoding_setup_vsc(struct intel_dp *intel_dp,
*/
vsc_sdp.sdp_header.HB3 = 0x13;
/*
* YCbCr 420 = 3h DB16[7:4] ITU-R BT.601 = 0h, ITU-R BT.709 = 1h
* DB16[3:0] DP 1.4a spec, Table 2-120
*/
vsc_sdp.db[16] = 0x3 << 4; /* 0x3 << 4 , YCbCr 420*/
/* RGB->YCBCR color conversion uses the BT.709 color space. */
vsc_sdp.db[16] |= 0x1; /* 0x1, ITU-R BT.709 */
/* DP 1.4a spec, Table 2-120 */
switch (crtc_state->output_format) {
case INTEL_OUTPUT_FORMAT_YCBCR444:
vsc_sdp.db[16] = 0x1 << 4; /* YCbCr 444 : DB16[7:4] = 1h */
break;
case INTEL_OUTPUT_FORMAT_YCBCR420:
vsc_sdp.db[16] = 0x3 << 4; /* YCbCr 420 : DB16[7:4] = 3h */
break;
case INTEL_OUTPUT_FORMAT_RGB:
default:
/* RGB: DB16[7:4] = 0h */
break;
}
switch (conn_state->colorspace) {
case DRM_MODE_COLORIMETRY_BT709_YCC:
vsc_sdp.db[16] |= 0x1;
break;
case DRM_MODE_COLORIMETRY_XVYCC_601:
vsc_sdp.db[16] |= 0x2;
break;
case DRM_MODE_COLORIMETRY_XVYCC_709:
vsc_sdp.db[16] |= 0x3;
break;
case DRM_MODE_COLORIMETRY_SYCC_601:
vsc_sdp.db[16] |= 0x4;
break;
case DRM_MODE_COLORIMETRY_OPYCC_601:
vsc_sdp.db[16] |= 0x5;
break;
case DRM_MODE_COLORIMETRY_BT2020_CYCC:
case DRM_MODE_COLORIMETRY_BT2020_RGB:
vsc_sdp.db[16] |= 0x6;
break;
case DRM_MODE_COLORIMETRY_BT2020_YCC:
vsc_sdp.db[16] |= 0x7;
break;
case DRM_MODE_COLORIMETRY_DCI_P3_RGB_D65:
case DRM_MODE_COLORIMETRY_DCI_P3_RGB_THEATER:
vsc_sdp.db[16] |= 0x4; /* DCI-P3 (SMPTE RP 431-2) */
break;
default:
/* sRGB (IEC 61966-2-1) / ITU-R BT.601: DB16[0:3] = 0h */
/* RGB->YCBCR color conversion uses the BT.709 color space. */
if (crtc_state->output_format == INTEL_OUTPUT_FORMAT_YCBCR420)
vsc_sdp.db[16] |= 0x1; /* 0x1, ITU-R BT.709 */
break;
}
/*
* For pixel encoding formats YCbCr444, YCbCr422, YCbCr420, and Y Only,
@ -4554,13 +4651,106 @@ intel_pixel_encoding_setup_vsc(struct intel_dp *intel_dp,
crtc_state, DP_SDP_VSC, &vsc_sdp, sizeof(vsc_sdp));
}
void intel_dp_ycbcr_420_enable(struct intel_dp *intel_dp,
const struct intel_crtc_state *crtc_state)
static void
intel_dp_setup_hdr_metadata_infoframe_sdp(struct intel_dp *intel_dp,
const struct intel_crtc_state *crtc_state,
const struct drm_connector_state *conn_state)
{
if (crtc_state->output_format != INTEL_OUTPUT_FORMAT_YCBCR420)
struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
struct dp_sdp infoframe_sdp = {};
struct hdmi_drm_infoframe drm_infoframe = {};
const int infoframe_size = HDMI_INFOFRAME_HEADER_SIZE + HDMI_DRM_INFOFRAME_SIZE;
unsigned char buf[HDMI_INFOFRAME_HEADER_SIZE + HDMI_DRM_INFOFRAME_SIZE];
ssize_t len;
int ret;
ret = drm_hdmi_infoframe_set_hdr_metadata(&drm_infoframe, conn_state);
if (ret) {
DRM_DEBUG_KMS("couldn't set HDR metadata in infoframe\n");
return;
}
len = hdmi_drm_infoframe_pack_only(&drm_infoframe, buf, sizeof(buf));
if (len < 0) {
DRM_DEBUG_KMS("buffer size is smaller than hdr metadata infoframe\n");
return;
}
if (len != infoframe_size) {
DRM_DEBUG_KMS("wrong static hdr metadata size\n");
return;
}
/*
* Set up the infoframe sdp packet for HDR static metadata.
* Prepare VSC Header for SU as per DP 1.4a spec,
* Table 2-100 and Table 2-101
*/
/* Packet ID, 00h for non-Audio INFOFRAME */
infoframe_sdp.sdp_header.HB0 = 0;
/*
* Packet Type 80h + Non-audio INFOFRAME Type value
* HDMI_INFOFRAME_TYPE_DRM: 0x87,
*/
infoframe_sdp.sdp_header.HB1 = drm_infoframe.type;
/*
* Least Significant Eight Bits of (Data Byte Count – 1)
* infoframe_size - 1,
*/
infoframe_sdp.sdp_header.HB2 = 0x1D;
/* INFOFRAME SDP Version Number */
infoframe_sdp.sdp_header.HB3 = (0x13 << 2);
/* CTA Header Byte 2 (INFOFRAME Version Number) */
infoframe_sdp.db[0] = drm_infoframe.version;
/* CTA Header Byte 3 (Length of INFOFRAME): HDMI_DRM_INFOFRAME_SIZE */
infoframe_sdp.db[1] = drm_infoframe.length;
/*
* Copy HDMI_DRM_INFOFRAME_SIZE size from a buffer after
* HDMI_INFOFRAME_HEADER_SIZE
*/
BUILD_BUG_ON(sizeof(infoframe_sdp.db) < HDMI_DRM_INFOFRAME_SIZE + 2);
memcpy(&infoframe_sdp.db[2], &buf[HDMI_INFOFRAME_HEADER_SIZE],
HDMI_DRM_INFOFRAME_SIZE);
/*
* Size of DP infoframe sdp packet for HDR static metadata is consist of
* - DP SDP Header(struct dp_sdp_header): 4 bytes
* - Two Data Blocks: 2 bytes
* CTA Header Byte2 (INFOFRAME Version Number)
* CTA Header Byte3 (Length of INFOFRAME)
* - HDMI_DRM_INFOFRAME_SIZE: 26 bytes
*
* Prior to GEN11's GMP register size is identical to DP HDR static metadata
* infoframe size. But GEN11+ has larger than that size, write_infoframe
* will pad rest of the size.
*/
intel_dig_port->write_infoframe(&intel_dig_port->base, crtc_state,
HDMI_PACKET_TYPE_GAMUT_METADATA,
&infoframe_sdp,
sizeof(struct dp_sdp_header) + 2 + HDMI_DRM_INFOFRAME_SIZE);
}
void intel_dp_vsc_enable(struct intel_dp *intel_dp,
const struct intel_crtc_state *crtc_state,
const struct drm_connector_state *conn_state)
{
if (!intel_dp_needs_vsc_sdp(crtc_state, conn_state))
return;
intel_pixel_encoding_setup_vsc(intel_dp, crtc_state);
intel_dp_setup_vsc_sdp(intel_dp, crtc_state, conn_state);
}
void intel_dp_hdr_metadata_enable(struct intel_dp *intel_dp,
const struct intel_crtc_state *crtc_state,
const struct drm_connector_state *conn_state)
{
if (!conn_state->hdr_output_metadata)
return;
intel_dp_setup_hdr_metadata_infoframe_sdp(intel_dp,
crtc_state,
conn_state);
}
static u8 intel_dp_autotest_link_training(struct intel_dp *intel_dp)
@ -5282,6 +5472,9 @@ static bool icl_combo_port_connected(struct drm_i915_private *dev_priv,
{
enum port port = intel_dig_port->base.port;
if (HAS_PCH_MCC(dev_priv) && port == PORT_C)
return I915_READ(SDEISR) & SDE_TC_HOTPLUG_ICP(PORT_TC1);
return I915_READ(SDEISR) & SDE_DDI_HOTPLUG_ICP(port);
}
@ -6402,6 +6595,13 @@ intel_dp_add_properties(struct intel_dp *intel_dp, struct drm_connector *connect
else if (INTEL_GEN(dev_priv) >= 5)
drm_connector_attach_max_bpc_property(connector, 6, 12);
intel_attach_colorspace_property(connector);
if (IS_GEMINILAKE(dev_priv) || INTEL_GEN(dev_priv) >= 11)
drm_object_attach_property(&connector->base,
connector->dev->mode_config.hdr_output_metadata_property,
0);
if (intel_dp_is_edp(intel_dp)) {
u32 allowed_scalers;
@ -7268,11 +7468,8 @@ intel_dp_init_connector(struct intel_digital_port *intel_dig_port,
intel_connector->get_hw_state = intel_connector_get_hw_state;
/* init MST on ports that can support it */
if (HAS_DP_MST(dev_priv) && !intel_dp_is_edp(intel_dp) &&
(port == PORT_B || port == PORT_C ||
port == PORT_D || port == PORT_F))
intel_dp_mst_encoder_init(intel_dig_port,
intel_connector->base.base.id);
intel_dp_mst_encoder_init(intel_dig_port,
intel_connector->base.base.id);
if (!intel_edp_init_connector(intel_dp, intel_connector)) {
intel_dp_aux_fini(intel_dp);

8
drivers/gpu/drm/i915/display/intel_dp.h
View File

@ -108,6 +108,14 @@ bool intel_dp_read_dpcd(struct intel_dp *intel_dp);
bool intel_dp_get_colorimetry_status(struct intel_dp *intel_dp);
int intel_dp_link_required(int pixel_clock, int bpp);
int intel_dp_max_data_rate(int max_link_clock, int max_lanes);
bool intel_dp_needs_vsc_sdp(const struct intel_crtc_state *crtc_state,
const struct drm_connector_state *conn_state);
void intel_dp_vsc_enable(struct intel_dp *intel_dp,
const struct intel_crtc_state *crtc_state,
const struct drm_connector_state *conn_state);
void intel_dp_hdr_metadata_enable(struct intel_dp *intel_dp,
const struct intel_crtc_state *crtc_state,
const struct drm_connector_state *conn_state);
bool intel_digital_port_connected(struct intel_encoder *encoder);
static inline unsigned int intel_dp_unused_lane_mask(int lane_count)

22
drivers/gpu/drm/i915/display/intel_dp_mst.c
View File

@ -655,21 +655,31 @@ intel_dp_mst_encoder_active_links(struct intel_digital_port *intel_dig_port)
int
intel_dp_mst_encoder_init(struct intel_digital_port *intel_dig_port, int conn_base_id)
{
struct drm_i915_private *i915 = to_i915(intel_dig_port->base.base.dev);
struct intel_dp *intel_dp = &intel_dig_port->dp;
struct drm_device *dev = intel_dig_port->base.base.dev;
enum port port = intel_dig_port->base.port;
int ret;
intel_dp->can_mst = true;
if (!HAS_DP_MST(i915) || intel_dp_is_edp(intel_dp))
return 0;
if (INTEL_GEN(i915) < 12 && port == PORT_A)
return 0;
if (INTEL_GEN(i915) < 11 && port == PORT_E)
return 0;
intel_dp->mst_mgr.cbs = &mst_cbs;
/* create encoders */
intel_dp_create_fake_mst_encoders(intel_dig_port);
ret = drm_dp_mst_topology_mgr_init(&intel_dp->mst_mgr, dev,
ret = drm_dp_mst_topology_mgr_init(&intel_dp->mst_mgr, &i915->drm,
&intel_dp->aux, 16, 3, conn_base_id);
if (ret) {
intel_dp->can_mst = false;
if (ret)
return ret;
}
intel_dp->can_mst = true;
return 0;
}

48
drivers/gpu/drm/i915/display/intel_dpll_mgr.c
View File

@ -247,8 +247,7 @@ static struct intel_shared_dpll *
intel_find_shared_dpll(struct intel_atomic_state *state,
const struct intel_crtc *crtc,
const struct intel_dpll_hw_state *pll_state,
enum intel_dpll_id range_min,
enum intel_dpll_id range_max)
unsigned long dpll_mask)
{
struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
struct intel_shared_dpll *pll, *unused_pll = NULL;
@ -257,7 +256,9 @@ intel_find_shared_dpll(struct intel_atomic_state *state,
shared_dpll = intel_atomic_get_shared_dpll_state(&state->base);
for (i = range_min; i <= range_max; i++) {
WARN_ON(dpll_mask & ~(BIT(I915_NUM_PLLS) - 1));
for_each_set_bit(i, &dpll_mask, I915_NUM_PLLS) {
pll = &dev_priv->shared_dplls[i];
/* Only want to check enabled timings first */
@ -464,8 +465,8 @@ static bool ibx_get_dpll(struct intel_atomic_state *state,
} else {
pll = intel_find_shared_dpll(state, crtc,
&crtc_state->dpll_hw_state,
DPLL_ID_PCH_PLL_A,
DPLL_ID_PCH_PLL_B);
BIT(DPLL_ID_PCH_PLL_B) |
BIT(DPLL_ID_PCH_PLL_A));
}
if (!pll)
@ -814,7 +815,8 @@ hsw_ddi_hdmi_get_dpll(struct intel_atomic_state *state,
pll = intel_find_shared_dpll(state, crtc,
&crtc_state->dpll_hw_state,
DPLL_ID_WRPLL1, DPLL_ID_WRPLL2);
BIT(DPLL_ID_WRPLL2) |
BIT(DPLL_ID_WRPLL1));
if (!pll)
return NULL;
@ -877,7 +879,7 @@ static bool hsw_get_dpll(struct intel_atomic_state *state,
pll = intel_find_shared_dpll(state, crtc,
&crtc_state->dpll_hw_state,
DPLL_ID_SPLL, DPLL_ID_SPLL);
BIT(DPLL_ID_SPLL));
} else {
return false;
}
@ -1447,13 +1449,13 @@ static bool skl_get_dpll(struct intel_atomic_state *state,
if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_EDP))
pll = intel_find_shared_dpll(state, crtc,
&crtc_state->dpll_hw_state,
DPLL_ID_SKL_DPLL0,
DPLL_ID_SKL_DPLL0);
BIT(DPLL_ID_SKL_DPLL0));
else
pll = intel_find_shared_dpll(state, crtc,
&crtc_state->dpll_hw_state,
DPLL_ID_SKL_DPLL1,
DPLL_ID_SKL_DPLL3);
BIT(DPLL_ID_SKL_DPLL3) |
BIT(DPLL_ID_SKL_DPLL2) |
BIT(DPLL_ID_SKL_DPLL1));
if (!pll)
return false;
@ -2401,8 +2403,9 @@ static bool cnl_get_dpll(struct intel_atomic_state *state,
pll = intel_find_shared_dpll(state, crtc,
&crtc_state->dpll_hw_state,
DPLL_ID_SKL_DPLL0,
DPLL_ID_SKL_DPLL2);
BIT(DPLL_ID_SKL_DPLL2) |
BIT(DPLL_ID_SKL_DPLL1) |
BIT(DPLL_ID_SKL_DPLL0));
if (!pll) {
DRM_DEBUG_KMS("No PLL selected\n");
return false;
@ -2975,7 +2978,7 @@ static bool icl_get_combo_phy_dpll(struct intel_atomic_state *state,
&crtc_state->icl_port_dplls[ICL_PORT_DPLL_DEFAULT];
struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
enum port port = encoder->port;
bool has_dpll4 = false;
unsigned long dpll_mask;
if (!icl_calc_dpll_state(crtc_state, encoder, &port_dpll->hw_state)) {
DRM_DEBUG_KMS("Could not calculate combo PHY PLL state.\n");
@ -2984,13 +2987,16 @@ static bool icl_get_combo_phy_dpll(struct intel_atomic_state *state,
}
if (IS_ELKHARTLAKE(dev_priv) && port != PORT_A)
has_dpll4 = true;
dpll_mask =
BIT(DPLL_ID_EHL_DPLL4) |
BIT(DPLL_ID_ICL_DPLL1) |
BIT(DPLL_ID_ICL_DPLL0);
else
dpll_mask = BIT(DPLL_ID_ICL_DPLL1) | BIT(DPLL_ID_ICL_DPLL0);
port_dpll->pll = intel_find_shared_dpll(state, crtc,
&port_dpll->hw_state,
DPLL_ID_ICL_DPLL0,
has_dpll4 ? DPLL_ID_EHL_DPLL4
: DPLL_ID_ICL_DPLL1);
dpll_mask);
if (!port_dpll->pll) {
DRM_DEBUG_KMS("No combo PHY PLL found for [ENCODER:%d:%s]\n",
encoder->base.base.id, encoder->base.name);
@ -3023,8 +3029,7 @@ static bool icl_get_tc_phy_dplls(struct intel_atomic_state *state,
port_dpll->pll = intel_find_shared_dpll(state, crtc,
&port_dpll->hw_state,
DPLL_ID_ICL_TBTPLL,
DPLL_ID_ICL_TBTPLL);
BIT(DPLL_ID_ICL_TBTPLL));
if (!port_dpll->pll) {
DRM_DEBUG_KMS("No TBT-ALT PLL found\n");
return false;
@ -3043,8 +3048,7 @@ static bool icl_get_tc_phy_dplls(struct intel_atomic_state *state,
encoder->port));
port_dpll->pll = intel_find_shared_dpll(state, crtc,
&port_dpll->hw_state,
dpll_id,
dpll_id);
BIT(dpll_id));
if (!port_dpll->pll) {
DRM_DEBUG_KMS("No MG PHY PLL found\n");
goto err_unreference_tbt_pll;

55
drivers/gpu/drm/i915/display/intel_hdmi.c
View File

@ -189,13 +189,19 @@ hsw_dip_data_reg(struct drm_i915_private *dev_priv,
}
}
static int hsw_dip_data_size(unsigned int type)
static int hsw_dip_data_size(struct drm_i915_private *dev_priv,
unsigned int type)
{
switch (type) {
case DP_SDP_VSC:
return VIDEO_DIP_VSC_DATA_SIZE;
case DP_SDP_PPS:
return VIDEO_DIP_PPS_DATA_SIZE;
case HDMI_PACKET_TYPE_GAMUT_METADATA:
if (INTEL_GEN(dev_priv) >= 11)
return VIDEO_DIP_GMP_DATA_SIZE;
else
return VIDEO_DIP_DATA_SIZE;
default:
return VIDEO_DIP_DATA_SIZE;
}
@ -514,7 +520,9 @@ static void hsw_write_infoframe(struct intel_encoder *encoder,
int i;
u32 val = I915_READ(ctl_reg);
data_size = hsw_dip_data_size(type);
data_size = hsw_dip_data_size(dev_priv, type);
WARN_ON(len > data_size);
val &= ~hsw_infoframe_enable(type);
I915_WRITE(ctl_reg, val);
@ -1527,27 +1535,17 @@ bool intel_hdmi_hdcp_check_link(struct intel_digital_port *intel_dig_port)
return true;
}
struct hdcp2_hdmi_msg_data {
struct hdcp2_hdmi_msg_timeout {
u8 msg_id;
u32 timeout;
u32 timeout2;
u16 timeout;
};
static const struct hdcp2_hdmi_msg_data hdcp2_msg_data[] = {
{ HDCP_2_2_AKE_INIT, 0, 0 },
{ HDCP_2_2_AKE_SEND_CERT, HDCP_2_2_CERT_TIMEOUT_MS, 0 },
{ HDCP_2_2_AKE_NO_STORED_KM, 0, 0 },
{ HDCP_2_2_AKE_STORED_KM, 0, 0 },
{ HDCP_2_2_AKE_SEND_HPRIME, HDCP_2_2_HPRIME_PAIRED_TIMEOUT_MS,
HDCP_2_2_HPRIME_NO_PAIRED_TIMEOUT_MS },
{ HDCP_2_2_AKE_SEND_PAIRING_INFO, HDCP_2_2_PAIRING_TIMEOUT_MS, 0 },
{ HDCP_2_2_LC_INIT, 0, 0 },
{ HDCP_2_2_LC_SEND_LPRIME, HDCP_2_2_HDMI_LPRIME_TIMEOUT_MS, 0 },
{ HDCP_2_2_SKE_SEND_EKS, 0, 0 },
{ HDCP_2_2_REP_SEND_RECVID_LIST, HDCP_2_2_RECVID_LIST_TIMEOUT_MS, 0 },
{ HDCP_2_2_REP_SEND_ACK, 0, 0 },
{ HDCP_2_2_REP_STREAM_MANAGE, 0, 0 },
{ HDCP_2_2_REP_STREAM_READY, HDCP_2_2_STREAM_READY_TIMEOUT_MS, 0 },
static const struct hdcp2_hdmi_msg_timeout hdcp2_msg_timeout[] = {
{ HDCP_2_2_AKE_SEND_CERT, HDCP_2_2_CERT_TIMEOUT_MS, },
{ HDCP_2_2_AKE_SEND_PAIRING_INFO, HDCP_2_2_PAIRING_TIMEOUT_MS, },
{ HDCP_2_2_LC_SEND_LPRIME, HDCP_2_2_HDMI_LPRIME_TIMEOUT_MS, },
{ HDCP_2_2_REP_SEND_RECVID_LIST, HDCP_2_2_RECVID_LIST_TIMEOUT_MS, },
{ HDCP_2_2_REP_STREAM_READY, HDCP_2_2_STREAM_READY_TIMEOUT_MS, },
};
static
@ -1564,12 +1562,17 @@ static int get_hdcp2_msg_timeout(u8 msg_id, bool is_paired)
{
int i;
for (i = 0; i < ARRAY_SIZE(hdcp2_msg_data); i++)
if (hdcp2_msg_data[i].msg_id == msg_id &&
(msg_id != HDCP_2_2_AKE_SEND_HPRIME || is_paired))
return hdcp2_msg_data[i].timeout;
else if (hdcp2_msg_data[i].msg_id == msg_id)
return hdcp2_msg_data[i].timeout2;
if (msg_id == HDCP_2_2_AKE_SEND_HPRIME) {
if (is_paired)
return HDCP_2_2_HPRIME_PAIRED_TIMEOUT_MS;
else
return HDCP_2_2_HPRIME_NO_PAIRED_TIMEOUT_MS;
}
for (i = 0; i < ARRAY_SIZE(hdcp2_msg_timeout); i++) {
if (hdcp2_msg_timeout[i].msg_id == msg_id)
return hdcp2_msg_timeout[i].timeout;
}
return -EINVAL;
}

114
drivers/gpu/drm/i915/display/intel_psr.c
View File

@ -534,6 +534,73 @@ transcoder_has_psr2(struct drm_i915_private *dev_priv, enum transcoder trans)
return trans == TRANSCODER_EDP;
}
static u32 intel_get_frame_time_us(const struct intel_crtc_state *cstate)
{
if (!cstate || !cstate->base.active)
return 0;
return DIV_ROUND_UP(1000 * 1000,
drm_mode_vrefresh(&cstate->base.adjusted_mode));
}
static void psr2_program_idle_frames(struct drm_i915_private *dev_priv,
u32 idle_frames)
{
u32 val;
idle_frames <<= EDP_PSR2_IDLE_FRAME_SHIFT;
val = I915_READ(EDP_PSR2_CTL(dev_priv->psr.transcoder));
val &= ~EDP_PSR2_IDLE_FRAME_MASK;
val |= idle_frames;
I915_WRITE(EDP_PSR2_CTL(dev_priv->psr.transcoder), val);
}
static void tgl_psr2_enable_dc3co(struct drm_i915_private *dev_priv)
{
psr2_program_idle_frames(dev_priv, 0);
intel_display_power_set_target_dc_state(dev_priv, DC_STATE_EN_DC3CO);
}
static void tgl_psr2_disable_dc3co(struct drm_i915_private *dev_priv)
{
int idle_frames;
intel_display_power_set_target_dc_state(dev_priv, DC_STATE_EN_UPTO_DC6);
/*
* Restore PSR2 idle frame let's use 6 as the minimum to cover all known
* cases including the off-by-one issue that HW has in some cases.
*/
idle_frames = max(6, dev_priv->vbt.psr.idle_frames);
idle_frames = max(idle_frames, dev_priv->psr.sink_sync_latency + 1);
psr2_program_idle_frames(dev_priv, idle_frames);
}
static void tgl_dc5_idle_thread(struct work_struct *work)
{
struct drm_i915_private *dev_priv =
container_of(work, typeof(*dev_priv), psr.idle_work.work);
mutex_lock(&dev_priv->psr.lock);
/* If delayed work is pending, it is not idle */
if (delayed_work_pending(&dev_priv->psr.idle_work))
goto unlock;
DRM_DEBUG_KMS("DC5/6 idle thread\n");
tgl_psr2_disable_dc3co(dev_priv);
unlock:
mutex_unlock(&dev_priv->psr.lock);
}
static void tgl_disallow_dc3co_on_psr2_exit(struct drm_i915_private *dev_priv)
{
if (!dev_priv->psr.dc3co_enabled)
return;
cancel_delayed_work(&dev_priv->psr.idle_work);
/* Before PSR2 exit disallow dc3co*/
tgl_psr2_disable_dc3co(dev_priv);
}
static bool intel_psr2_config_valid(struct intel_dp *intel_dp,
struct intel_crtc_state *crtc_state)
{
@ -746,6 +813,8 @@ static void intel_psr_enable_locked(struct drm_i915_private *dev_priv,
dev_priv->psr.psr2_enabled = intel_psr2_enabled(dev_priv, crtc_state);
dev_priv->psr.busy_frontbuffer_bits = 0;
dev_priv->psr.pipe = to_intel_crtc(crtc_state->base.crtc)->pipe;
dev_priv->psr.dc3co_enabled = !!crtc_state->dc3co_exitline;
dev_priv->psr.dc3co_exit_delay = intel_get_frame_time_us(crtc_state);
dev_priv->psr.transcoder = crtc_state->cpu_transcoder;
/*
@ -829,6 +898,7 @@ static void intel_psr_exit(struct drm_i915_private *dev_priv)
}
if (dev_priv->psr.psr2_enabled) {
tgl_disallow_dc3co_on_psr2_exit(dev_priv);
val = I915_READ(EDP_PSR2_CTL(dev_priv->psr.transcoder));
WARN_ON(!(val & EDP_PSR2_ENABLE));
val &= ~EDP_PSR2_ENABLE;
@ -901,6 +971,7 @@ void intel_psr_disable(struct intel_dp *intel_dp,
mutex_unlock(&dev_priv->psr.lock);
cancel_work_sync(&dev_priv->psr.work);
cancel_delayed_work_sync(&dev_priv->psr.idle_work);
}
static void psr_force_hw_tracking_exit(struct drm_i915_private *dev_priv)
@ -1208,6 +1279,44 @@ void intel_psr_invalidate(struct drm_i915_private *dev_priv,
mutex_unlock(&dev_priv->psr.lock);
}
/*
* When we will be completely rely on PSR2 S/W tracking in future,
* intel_psr_flush() will invalidate and flush the PSR for ORIGIN_FLIP
* event also therefore tgl_dc3co_flush() require to be changed
* accrodingly in future.
*/
static void
tgl_dc3co_flush(struct drm_i915_private *dev_priv,
unsigned int frontbuffer_bits, enum fb_op_origin origin)
{
u32 delay;
mutex_lock(&dev_priv->psr.lock);
if (!dev_priv->psr.dc3co_enabled)
goto unlock;
if (!dev_priv->psr.psr2_enabled || !dev_priv->psr.active)
goto unlock;
/*
* At every frontbuffer flush flip event modified delay of delayed work,
* when delayed work schedules that means display has been idle.
*/
if (!(frontbuffer_bits &
INTEL_FRONTBUFFER_ALL_MASK(dev_priv->psr.pipe)))
goto unlock;
tgl_psr2_enable_dc3co(dev_priv);
/* DC5/DC6 required idle frames = 6 */
delay = 6 * dev_priv->psr.dc3co_exit_delay;
mod_delayed_work(system_wq, &dev_priv->psr.idle_work,
usecs_to_jiffies(delay));
unlock:
mutex_unlock(&dev_priv->psr.lock);
}
/**
* intel_psr_flush - Flush PSR
* @dev_priv: i915 device
@ -1227,8 +1336,10 @@ void intel_psr_flush(struct drm_i915_private *dev_priv,
if (!CAN_PSR(dev_priv))
return;
if (origin == ORIGIN_FLIP)
if (origin == ORIGIN_FLIP) {
tgl_dc3co_flush(dev_priv, frontbuffer_bits, origin);
return;
}
mutex_lock(&dev_priv->psr.lock);
if (!dev_priv->psr.enabled) {
@ -1284,6 +1395,7 @@ void intel_psr_init(struct drm_i915_private *dev_priv)
dev_priv->psr.link_standby = dev_priv->vbt.psr.full_link;
INIT_WORK(&dev_priv->psr.work, intel_psr_work);
INIT_DELAYED_WORK(&dev_priv->psr.idle_work, tgl_dc5_idle_thread);
mutex_init(&dev_priv->psr.lock);
}

6
drivers/gpu/drm/i915/display/intel_sprite.c
View File

@ -287,10 +287,8 @@ int intel_plane_check_src_coordinates(struct intel_plane_state *plane_state)
src_y = src->y1 >> 16;
src_h = drm_rect_height(src) >> 16;
src->x1 = src_x << 16;
src->x2 = (src_x + src_w) << 16;
src->y1 = src_y << 16;
src->y2 = (src_y + src_h) << 16;
drm_rect_init(src, src_x << 16, src_y << 16,
src_w << 16, src_h << 16);
if (!fb->format->is_yuv)
return 0;

3
drivers/gpu/drm/i915/display/intel_vbt_defs.h
View File

@ -291,6 +291,8 @@ struct bdb_general_features {
#define DVO_PORT_HDMIE 12 /* 193 */
#define DVO_PORT_DPF 13 /* N/A */
#define DVO_PORT_HDMIF 14 /* N/A */
#define DVO_PORT_DPG 15
#define DVO_PORT_HDMIG 16
#define DVO_PORT_MIPIA 21 /* 171 */
#define DVO_PORT_MIPIB 22 /* 171 */
#define DVO_PORT_MIPIC 23 /* 171 */
@ -325,6 +327,7 @@ enum vbt_gmbus_ddi {
#define DP_AUX_D 0x30
#define DP_AUX_E 0x50
#define DP_AUX_F 0x60
#define DP_AUX_G 0x70
#define VBT_DP_MAX_LINK_RATE_HBR3 0
#define VBT_DP_MAX_LINK_RATE_HBR2 1

13
drivers/gpu/drm/i915/gem/i915_gem_context.c
View File

@ -203,15 +203,22 @@ static struct i915_gem_engines *default_engines(struct i915_gem_context *ctx)
for_each_engine(engine, gt, id) {
struct intel_context *ce;
if (engine->legacy_idx == INVALID_ENGINE)
continue;
GEM_BUG_ON(engine->legacy_idx >= I915_NUM_ENGINES);
GEM_BUG_ON(e->engines[engine->legacy_idx]);
ce = intel_context_create(ctx, engine);
if (IS_ERR(ce)) {
__free_engines(e, id);
__free_engines(e, e->num_engines + 1);
return ERR_CAST(ce);
}
e->engines[id] = ce;
e->num_engines = id + 1;
e->engines[engine->legacy_idx] = ce;
e->num_engines = max(e->num_engines, engine->legacy_idx);
}
e->num_engines++;
return e;
}

18
drivers/gpu/drm/i915/gem/i915_gem_context.h
View File

@ -114,6 +114,24 @@ i915_gem_context_clear_user_engines(struct i915_gem_context *ctx)
clear_bit(CONTEXT_USER_ENGINES, &ctx->flags);
}
static inline bool
i915_gem_context_nopreempt(const struct i915_gem_context *ctx)
{
return test_bit(CONTEXT_NOPREEMPT, &ctx->flags);
}
static inline void
i915_gem_context_set_nopreempt(struct i915_gem_context *ctx)
{
set_bit(CONTEXT_NOPREEMPT, &ctx->flags);
}
static inline void
i915_gem_context_clear_nopreempt(struct i915_gem_context *ctx)
{
clear_bit(CONTEXT_NOPREEMPT, &ctx->flags);
}
static inline bool i915_gem_context_is_kernel(struct i915_gem_context *ctx)
{
return !ctx->file_priv;

1
drivers/gpu/drm/i915/gem/i915_gem_context_types.h
View File

@ -146,6 +146,7 @@ struct i915_gem_context {
#define CONTEXT_CLOSED 1
#define CONTEXT_FORCE_SINGLE_SUBMISSION 2
#define CONTEXT_USER_ENGINES 3
#define CONTEXT_NOPREEMPT 4
struct mutex mutex;

3
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
View File

@ -2077,6 +2077,9 @@ static int eb_submit(struct i915_execbuffer *eb)
if (err)
return err;
if (i915_gem_context_nopreempt(eb->gem_context))
eb->request->flags |= I915_REQUEST_NOPREEMPT;
return 0;
}

17
drivers/gpu/drm/i915/gem/i915_gem_internal.c
View File

@ -117,13 +117,6 @@ static int i915_gem_object_get_pages_internal(struct drm_i915_gem_object *obj)
goto err;
}
/* Mark the pages as dontneed whilst they are still pinned. As soon
* as they are unpinned they are allowed to be reaped by the shrinker,
* and the caller is expected to repopulate - the contents of this
* object are only valid whilst active and pinned.
*/
obj->mm.madv = I915_MADV_DONTNEED;
__i915_gem_object_set_pages(obj, st, sg_page_sizes);
return 0;
@ -143,7 +136,6 @@ static void i915_gem_object_put_pages_internal(struct drm_i915_gem_object *obj,
internal_free_pages(pages);
obj->mm.dirty = false;
obj->mm.madv = I915_MADV_WILLNEED;
}
static const struct drm_i915_gem_object_ops i915_gem_object_internal_ops = {
@ -188,6 +180,15 @@ i915_gem_object_create_internal(struct drm_i915_private *i915,
drm_gem_private_object_init(&i915->drm, &obj->base, size);
i915_gem_object_init(obj, &i915_gem_object_internal_ops);
/*
* Mark the object as volatile, such that the pages are marked as
* dontneed whilst they are still pinned. As soon as they are unpinned
* they are allowed to be reaped by the shrinker, and the caller is
* expected to repopulate - the contents of this object are only valid
* whilst active and pinned.
*/
i915_gem_object_set_volatile(obj);
obj->read_domains = I915_GEM_DOMAIN_CPU;
obj->write_domain = I915_GEM_DOMAIN_CPU;

23
drivers/gpu/drm/i915/gem/i915_gem_object.h
View File

@ -25,10 +25,11 @@ void i915_gem_object_free(struct drm_i915_gem_object *obj);
void i915_gem_object_init(struct drm_i915_gem_object *obj,
const struct drm_i915_gem_object_ops *ops);
struct drm_i915_gem_object *
i915_gem_object_create_shmem(struct drm_i915_private *i915, u64 size);
i915_gem_object_create_shmem(struct drm_i915_private *i915,
resource_size_t size);
struct drm_i915_gem_object *
i915_gem_object_create_shmem_from_data(struct drm_i915_private *i915,
const void *data, size_t size);
const void *data, resource_size_t size);
extern const struct drm_i915_gem_object_ops i915_gem_shmem_ops;
void __i915_gem_object_release_shmem(struct drm_i915_gem_object *obj,
@ -139,6 +140,24 @@ i915_gem_object_is_readonly(const struct drm_i915_gem_object *obj)
return obj->base.vma_node.readonly;
}
static inline bool
i915_gem_object_is_contiguous(const struct drm_i915_gem_object *obj)
{
return obj->flags & I915_BO_ALLOC_CONTIGUOUS;
}
static inline bool
i915_gem_object_is_volatile(const struct drm_i915_gem_object *obj)
{
return obj->flags & I915_BO_ALLOC_VOLATILE;
}
static inline void
i915_gem_object_set_volatile(struct drm_i915_gem_object *obj)
{
obj->flags |= I915_BO_ALLOC_VOLATILE;
}
static inline bool
i915_gem_object_type_has(const struct drm_i915_gem_object *obj,
unsigned long flags)

20
drivers/gpu/drm/i915/gem/i915_gem_object_types.h
View File

@ -119,6 +119,11 @@ struct drm_i915_gem_object {
I915_SELFTEST_DECLARE(struct list_head st_link);
unsigned long flags;
#define I915_BO_ALLOC_CONTIGUOUS BIT(0)
#define I915_BO_ALLOC_VOLATILE BIT(1)
#define I915_BO_ALLOC_FLAGS (I915_BO_ALLOC_CONTIGUOUS | I915_BO_ALLOC_VOLATILE)
/*
* Is the object to be mapped as read-only to the GPU
* Only honoured if hardware has relevant pte bit
@ -160,6 +165,21 @@ struct drm_i915_gem_object {
atomic_t pages_pin_count;
atomic_t shrink_pin;
/**
* Memory region for this object.
*/
struct intel_memory_region *region;
/**
* List of memory region blocks allocated for this object.
*/
struct list_head blocks;
/**
* Element within memory_region->objects or region->purgeable
* if the object is marked as DONTNEED. Access is protected by
* region->obj_lock.
*/
struct list_head region_link;
struct sg_table *pages;
void *mapping;

6
drivers/gpu/drm/i915/gem/i915_gem_pages.c
View File

@ -18,6 +18,9 @@ void __i915_gem_object_set_pages(struct drm_i915_gem_object *obj,
lockdep_assert_held(&obj->mm.lock);
if (i915_gem_object_is_volatile(obj))
obj->mm.madv = I915_MADV_DONTNEED;
/* Make the pages coherent with the GPU (flushing any swapin). */
if (obj->cache_dirty) {
obj->write_domain = 0;
@ -160,6 +163,9 @@ __i915_gem_object_unset_pages(struct drm_i915_gem_object *obj)
if (IS_ERR_OR_NULL(pages))
return pages;
if (i915_gem_object_is_volatile(obj))
obj->mm.madv = I915_MADV_WILLNEED;
i915_gem_object_make_unshrinkable(obj);
if (obj->mm.mapping) {

5
drivers/gpu/drm/i915/gem/i915_gem_phys.c
View File

@ -16,6 +16,7 @@
#include "gt/intel_gt.h"
#include "i915_drv.h"
#include "i915_gem_object.h"
#include "i915_gem_region.h"
#include "i915_scatterlist.h"
static int i915_gem_object_get_pages_phys(struct drm_i915_gem_object *obj)
@ -191,8 +192,10 @@ int i915_gem_object_attach_phys(struct drm_i915_gem_object *obj, int align)
/* Perma-pin (until release) the physical set of pages */
__i915_gem_object_pin_pages(obj);
if (!IS_ERR_OR_NULL(pages))
if (!IS_ERR_OR_NULL(pages)) {
i915_gem_shmem_ops.put_pages(obj, pages);
i915_gem_object_release_memory_region(obj);
}
mutex_unlock(&obj->mm.lock);
return 0;

174
drivers/gpu/drm/i915/gem/i915_gem_region.c Normal file
View File

@ -0,0 +1,174 @@
// SPDX-License-Identifier: MIT
/*
* Copyright © 2019 Intel Corporation
*/
#include "intel_memory_region.h"
#include "i915_gem_region.h"
#include "i915_drv.h"
#include "i915_trace.h"
void
i915_gem_object_put_pages_buddy(struct drm_i915_gem_object *obj,
struct sg_table *pages)
{
__intel_memory_region_put_pages_buddy(obj->mm.region, &obj->mm.blocks);
obj->mm.dirty = false;
sg_free_table(pages);
kfree(pages);
}
int
i915_gem_object_get_pages_buddy(struct drm_i915_gem_object *obj)
{
struct intel_memory_region *mem = obj->mm.region;
struct list_head *blocks = &obj->mm.blocks;
resource_size_t size = obj->base.size;
resource_size_t prev_end;
struct i915_buddy_block *block;
unsigned int flags;
struct sg_table *st;
struct scatterlist *sg;
unsigned int sg_page_sizes;
int ret;
st = kmalloc(sizeof(*st), GFP_KERNEL);
if (!st)
return -ENOMEM;
if (sg_alloc_table(st, size >> ilog2(mem->mm.chunk_size), GFP_KERNEL)) {
kfree(st);
return -ENOMEM;
}
flags = I915_ALLOC_MIN_PAGE_SIZE;
if (obj->flags & I915_BO_ALLOC_CONTIGUOUS)
flags |= I915_ALLOC_CONTIGUOUS;
ret = __intel_memory_region_get_pages_buddy(mem, size, flags, blocks);
if (ret)
goto err_free_sg;
GEM_BUG_ON(list_empty(blocks));
sg = st->sgl;
st->nents = 0;
sg_page_sizes = 0;
prev_end = (resource_size_t)-1;
list_for_each_entry(block, blocks, link) {
u64 block_size, offset;
block_size = min_t(u64, size,
i915_buddy_block_size(&mem->mm, block));
offset = i915_buddy_block_offset(block);
GEM_BUG_ON(overflows_type(block_size, sg->length));
if (offset != prev_end ||
add_overflows_t(typeof(sg->length), sg->length, block_size)) {
if (st->nents) {
sg_page_sizes |= sg->length;
sg = __sg_next(sg);
}
sg_dma_address(sg) = mem->region.start + offset;
sg_dma_len(sg) = block_size;
sg->length = block_size;
st->nents++;
} else {
sg->length += block_size;
sg_dma_len(sg) += block_size;
}
prev_end = offset + block_size;
};
sg_page_sizes |= sg->length;
sg_mark_end(sg);
i915_sg_trim(st);
__i915_gem_object_set_pages(obj, st, sg_page_sizes);
return 0;
err_free_sg:
sg_free_table(st);
kfree(st);
return ret;
}
void i915_gem_object_init_memory_region(struct drm_i915_gem_object *obj,
struct intel_memory_region *mem,
unsigned long flags)
{
INIT_LIST_HEAD(&obj->mm.blocks);
obj->mm.region = intel_memory_region_get(mem);
obj->flags |= flags;
mutex_lock(&mem->objects.lock);
if (obj->flags & I915_BO_ALLOC_VOLATILE)
list_add(&obj->mm.region_link, &mem->objects.purgeable);
else
list_add(&obj->mm.region_link, &mem->objects.list);
mutex_unlock(&mem->objects.lock);
}
void i915_gem_object_release_memory_region(struct drm_i915_gem_object *obj)
{
struct intel_memory_region *mem = obj->mm.region;
mutex_lock(&mem->objects.lock);
list_del(&obj->mm.region_link);
mutex_unlock(&mem->objects.lock);
intel_memory_region_put(mem);
}
struct drm_i915_gem_object *
i915_gem_object_create_region(struct intel_memory_region *mem,
resource_size_t size,
unsigned int flags)
{
struct drm_i915_gem_object *obj;
/*
* NB: Our use of resource_size_t for the size stems from using struct
* resource for the mem->region. We might need to revisit this in the
* future.
*/
GEM_BUG_ON(flags & ~I915_BO_ALLOC_FLAGS);
if (!mem)
return ERR_PTR(-ENODEV);
size = round_up(size, mem->min_page_size);
GEM_BUG_ON(!size);
GEM_BUG_ON(!IS_ALIGNED(size, I915_GTT_MIN_ALIGNMENT));
/*
* XXX: There is a prevalence of the assumption that we fit the
* object's page count inside a 32bit _signed_ variable. Let's document
* this and catch if we ever need to fix it. In the meantime, if you do
* spot such a local variable, please consider fixing!
*/
if (size >> PAGE_SHIFT > INT_MAX)
return ERR_PTR(-E2BIG);
if (overflows_type(size, obj->base.size))
return ERR_PTR(-E2BIG);
obj = mem->ops->create_object(mem, size, flags);
if (!IS_ERR(obj))
trace_i915_gem_object_create(obj);
return obj;
}

29
drivers/gpu/drm/i915/gem/i915_gem_region.h Normal file
View File

@ -0,0 +1,29 @@
/* SPDX-License-Identifier: MIT */
/*
* Copyright © 2019 Intel Corporation
*/
#ifndef __I915_GEM_REGION_H__
#define __I915_GEM_REGION_H__
#include <linux/types.h>
struct intel_memory_region;
struct drm_i915_gem_object;
struct sg_table;
int i915_gem_object_get_pages_buddy(struct drm_i915_gem_object *obj);
void i915_gem_object_put_pages_buddy(struct drm_i915_gem_object *obj,
struct sg_table *pages);
void i915_gem_object_init_memory_region(struct drm_i915_gem_object *obj,
struct intel_memory_region *mem,
unsigned long flags);
void i915_gem_object_release_memory_region(struct drm_i915_gem_object *obj);
struct drm_i915_gem_object *
i915_gem_object_create_region(struct intel_memory_region *mem,
resource_size_t size,
unsigned int flags);
#endif

79
drivers/gpu/drm/i915/gem/i915_gem_shmem.c
View File

@ -7,7 +7,9 @@
#include <linux/pagevec.h>
#include <linux/swap.h>
#include "gem/i915_gem_region.h"
#include "i915_drv.h"
#include "i915_gemfs.h"
#include "i915_gem_object.h"
#include "i915_scatterlist.h"
#include "i915_trace.h"
@ -26,6 +28,7 @@ static void check_release_pagevec(struct pagevec *pvec)
static int shmem_get_pages(struct drm_i915_gem_object *obj)
{
struct drm_i915_private *i915 = to_i915(obj->base.dev);
struct intel_memory_region *mem = obj->mm.region;
const unsigned long page_count = obj->base.size / PAGE_SIZE;
unsigned long i;
struct address_space *mapping;
@ -52,7 +55,7 @@ static int shmem_get_pages(struct drm_i915_gem_object *obj)
* If there's no chance of allocating enough pages for the whole
* object, bail early.
*/
if (page_count > totalram_pages())
if (obj->base.size > resource_size(&mem->region))
return -ENOMEM;
st = kmalloc(sizeof(*st), GFP_KERNEL);
@ -417,6 +420,8 @@ shmem_pwrite(struct drm_i915_gem_object *obj,
static void shmem_release(struct drm_i915_gem_object *obj)
{
i915_gem_object_release_memory_region(obj);
fput(obj->base.filp);
}
@ -434,9 +439,9 @@ const struct drm_i915_gem_object_ops i915_gem_shmem_ops = {
.release = shmem_release,
};
static int create_shmem(struct drm_i915_private *i915,
struct drm_gem_object *obj,
size_t size)
static int __create_shmem(struct drm_i915_private *i915,
struct drm_gem_object *obj,
resource_size_t size)
{
unsigned long flags = VM_NORESERVE;
struct file *filp;
@ -455,31 +460,23 @@ static int create_shmem(struct drm_i915_private *i915,
return 0;
}
struct drm_i915_gem_object *
i915_gem_object_create_shmem(struct drm_i915_private *i915, u64 size)
static struct drm_i915_gem_object *
create_shmem(struct intel_memory_region *mem,
resource_size_t size,
unsigned int flags)
{
struct drm_i915_private *i915 = mem->i915;
struct drm_i915_gem_object *obj;
struct address_space *mapping;
unsigned int cache_level;
gfp_t mask;
int ret;
/* There is a prevalence of the assumption that we fit the object's
* page count inside a 32bit _signed_ variable. Let's document this and
* catch if we ever need to fix it. In the meantime, if you do spot
* such a local variable, please consider fixing!
*/
if (size >> PAGE_SHIFT > INT_MAX)
return ERR_PTR(-E2BIG);
if (overflows_type(size, obj->base.size))
return ERR_PTR(-E2BIG);
obj = i915_gem_object_alloc();
if (!obj)
return ERR_PTR(-ENOMEM);
ret = create_shmem(i915, &obj->base, size);
ret = __create_shmem(i915, &obj->base, size);
if (ret)
goto fail;
@ -518,7 +515,7 @@ i915_gem_object_create_shmem(struct drm_i915_private *i915, u64 size)
i915_gem_object_set_cache_coherency(obj, cache_level);
trace_i915_gem_object_create(obj);
i915_gem_object_init_memory_region(obj, mem, 0);
return obj;
@ -527,14 +524,22 @@ i915_gem_object_create_shmem(struct drm_i915_private *i915, u64 size)
return ERR_PTR(ret);
}
struct drm_i915_gem_object *
i915_gem_object_create_shmem(struct drm_i915_private *i915,
resource_size_t size)
{
return i915_gem_object_create_region(i915->mm.regions[INTEL_REGION_SMEM],
size, 0);
}
/* Allocate a new GEM object and fill it with the supplied data */
struct drm_i915_gem_object *
i915_gem_object_create_shmem_from_data(struct drm_i915_private *dev_priv,
const void *data, size_t size)
const void *data, resource_size_t size)
{
struct drm_i915_gem_object *obj;
struct file *file;
size_t offset;
resource_size_t offset;
int err;
obj = i915_gem_object_create_shmem(dev_priv, round_up(size, PAGE_SIZE));
@ -577,3 +582,35 @@ i915_gem_object_create_shmem_from_data(struct drm_i915_private *dev_priv,
i915_gem_object_put(obj);
return ERR_PTR(err);
}
static int init_shmem(struct intel_memory_region *mem)
{
int err;
err = i915_gemfs_init(mem->i915);
if (err) {
DRM_NOTE("Unable to create a private tmpfs mount, hugepage support will be disabled(%d).\n",
err);
}
return 0; /* Don't error, we can simply fallback to the kernel mnt */
}
static void release_shmem(struct intel_memory_region *mem)
{
i915_gemfs_fini(mem->i915);
}
static const struct intel_memory_region_ops shmem_region_ops = {
.init = init_shmem,
.release = release_shmem,
.create_object = create_shmem,
};
struct intel_memory_region *i915_gem_shmem_setup(struct drm_i915_private *i915)
{
return intel_memory_region_create(i915, 0,
totalram_pages() << PAGE_SHIFT,
PAGE_SIZE, 0,
&shmem_region_ops);
}

65
drivers/gpu/drm/i915/gem/i915_gem_stolen.c
View File

@ -10,6 +10,7 @@
#include <drm/drm_mm.h>
#include <drm/i915_drm.h>
#include "gem/i915_gem_region.h"
#include "i915_drv.h"
#include "i915_gem_stolen.h"
@ -150,7 +151,7 @@ static int i915_adjust_stolen(struct drm_i915_private *dev_priv,
return 0;
}
void i915_gem_cleanup_stolen(struct drm_i915_private *dev_priv)
static void i915_gem_cleanup_stolen(struct drm_i915_private *dev_priv)
{
if (!drm_mm_initialized(&dev_priv->mm.stolen))
return;
@ -355,7 +356,7 @@ static void icl_get_stolen_reserved(struct drm_i915_private *i915,
}
}
int i915_gem_init_stolen(struct drm_i915_private *dev_priv)
static int i915_gem_init_stolen(struct drm_i915_private *dev_priv)
{
resource_size_t reserved_base, stolen_top;
resource_size_t reserved_total, reserved_size;
@ -539,6 +540,9 @@ i915_gem_object_release_stolen(struct drm_i915_gem_object *obj)
i915_gem_stolen_remove_node(dev_priv, stolen);
kfree(stolen);
if (obj->mm.region)
i915_gem_object_release_memory_region(obj);
}
static const struct drm_i915_gem_object_ops i915_gem_object_stolen_ops = {
@ -548,8 +552,9 @@ static const struct drm_i915_gem_object_ops i915_gem_object_stolen_ops = {
};
static struct drm_i915_gem_object *
_i915_gem_object_create_stolen(struct drm_i915_private *dev_priv,
struct drm_mm_node *stolen)
__i915_gem_object_create_stolen(struct drm_i915_private *dev_priv,
struct drm_mm_node *stolen,
struct intel_memory_region *mem)
{
struct drm_i915_gem_object *obj;
unsigned int cache_level;
@ -571,6 +576,9 @@ _i915_gem_object_create_stolen(struct drm_i915_private *dev_priv,
if (err)
goto cleanup;
if (mem)
i915_gem_object_init_memory_region(obj, mem, 0);
return obj;
cleanup:
@ -579,10 +587,12 @@ _i915_gem_object_create_stolen(struct drm_i915_private *dev_priv,
return ERR_PTR(err);
}
struct drm_i915_gem_object *
i915_gem_object_create_stolen(struct drm_i915_private *dev_priv,
resource_size_t size)
static struct drm_i915_gem_object *
_i915_gem_object_create_stolen(struct intel_memory_region *mem,
resource_size_t size,
unsigned int flags)
{
struct drm_i915_private *dev_priv = mem->i915;
struct drm_i915_gem_object *obj;
struct drm_mm_node *stolen;
int ret;
@ -603,7 +613,7 @@ i915_gem_object_create_stolen(struct drm_i915_private *dev_priv,
goto err_free;
}
obj = _i915_gem_object_create_stolen(dev_priv, stolen);
obj = __i915_gem_object_create_stolen(dev_priv, stolen, mem);
if (IS_ERR(obj))
goto err_remove;
@ -616,6 +626,43 @@ i915_gem_object_create_stolen(struct drm_i915_private *dev_priv,
return obj;
}
struct drm_i915_gem_object *
i915_gem_object_create_stolen(struct drm_i915_private *dev_priv,
resource_size_t size)
{
return i915_gem_object_create_region(dev_priv->mm.regions[INTEL_REGION_STOLEN],
size, I915_BO_ALLOC_CONTIGUOUS);
}
static int init_stolen(struct intel_memory_region *mem)
{
/*
* Initialise stolen early so that we may reserve preallocated
* objects for the BIOS to KMS transition.
*/
return i915_gem_init_stolen(mem->i915);
}
static void release_stolen(struct intel_memory_region *mem)
{
i915_gem_cleanup_stolen(mem->i915);
}
static const struct intel_memory_region_ops i915_region_stolen_ops = {
.init = init_stolen,
.release = release_stolen,
.create_object = _i915_gem_object_create_stolen,
};
struct intel_memory_region *i915_gem_stolen_setup(struct drm_i915_private *i915)
{
return intel_memory_region_create(i915,
intel_graphics_stolen_res.start,
resource_size(&intel_graphics_stolen_res),
PAGE_SIZE, 0,
&i915_region_stolen_ops);
}
struct drm_i915_gem_object *
i915_gem_object_create_stolen_for_preallocated(struct drm_i915_private *dev_priv,
resource_size_t stolen_offset,
@ -655,7 +702,7 @@ i915_gem_object_create_stolen_for_preallocated(struct drm_i915_private *dev_priv
return ERR_PTR(ret);
}
obj = _i915_gem_object_create_stolen(dev_priv, stolen);
obj = __i915_gem_object_create_stolen(dev_priv, stolen, NULL);
if (IS_ERR(obj)) {
DRM_DEBUG_DRIVER("failed to allocate stolen object\n");
i915_gem_stolen_remove_node(dev_priv, stolen);

3
drivers/gpu/drm/i915/gem/i915_gem_stolen.h
View File

@ -21,8 +21,7 @@ int i915_gem_stolen_insert_node_in_range(struct drm_i915_private *dev_priv,
u64 end);
void i915_gem_stolen_remove_node(struct drm_i915_private *dev_priv,
struct drm_mm_node *node);
int i915_gem_init_stolen(struct drm_i915_private *dev_priv);
void i915_gem_cleanup_stolen(struct drm_i915_private *dev_priv);
struct intel_memory_region *i915_gem_stolen_setup(struct drm_i915_private *i915);
struct drm_i915_gem_object *
i915_gem_object_create_stolen(struct drm_i915_private *dev_priv,
resource_size_t size);

8
drivers/gpu/drm/i915/gem/i915_gem_tiling.c
View File

@ -348,9 +348,9 @@ i915_gem_set_tiling_ioctl(struct drm_device *dev, void *data,
args->stride = 0;
} else {
if (args->tiling_mode == I915_TILING_X)
args->swizzle_mode = to_i915(dev)->mm.bit_6_swizzle_x;
args->swizzle_mode = to_i915(dev)->ggtt.bit_6_swizzle_x;
else
args->swizzle_mode = to_i915(dev)->mm.bit_6_swizzle_y;
args->swizzle_mode = to_i915(dev)->ggtt.bit_6_swizzle_y;
/* Hide bit 17 swizzling from the user. This prevents old Mesa
* from aborting the application on sw fallbacks to bit 17,
@ -421,10 +421,10 @@ i915_gem_get_tiling_ioctl(struct drm_device *dev, void *data,
switch (args->tiling_mode) {
case I915_TILING_X:
args->swizzle_mode = dev_priv->mm.bit_6_swizzle_x;
args->swizzle_mode = dev_priv->ggtt.bit_6_swizzle_x;
break;
case I915_TILING_Y:
args->swizzle_mode = dev_priv->mm.bit_6_swizzle_y;
args->swizzle_mode = dev_priv->ggtt.bit_6_swizzle_y;
break;
default:
case I915_TILING_NONE:

97
drivers/gpu/drm/i915/gem/selftests/huge_pages.c
View File

@ -8,6 +8,7 @@
#include "i915_selftest.h"
#include "gem/i915_gem_region.h"
#include "gem/i915_gem_pm.h"
#include "gt/intel_gt.h"
@ -17,6 +18,7 @@
#include "selftests/mock_drm.h"
#include "selftests/mock_gem_device.h"
#include "selftests/mock_region.h"
#include "selftests/i915_random.h"
static const unsigned int page_sizes[] = {
@ -113,8 +115,6 @@ static int get_huge_pages(struct drm_i915_gem_object *obj)
if (i915_gem_gtt_prepare_pages(obj, st))
goto err;
obj->mm.madv = I915_MADV_DONTNEED;
GEM_BUG_ON(sg_page_sizes != obj->mm.page_mask);
__i915_gem_object_set_pages(obj, st, sg_page_sizes);
@ -135,7 +135,6 @@ static void put_huge_pages(struct drm_i915_gem_object *obj,
huge_pages_free_pages(pages);
obj->mm.dirty = false;
obj->mm.madv = I915_MADV_WILLNEED;
}
static const struct drm_i915_gem_object_ops huge_page_ops = {
@ -168,6 +167,8 @@ huge_pages_object(struct drm_i915_private *i915,
drm_gem_private_object_init(&i915->drm, &obj->base, size);
i915_gem_object_init(obj, &huge_page_ops);
i915_gem_object_set_volatile(obj);
obj->write_domain = I915_GEM_DOMAIN_CPU;
obj->read_domains = I915_GEM_DOMAIN_CPU;
obj->cache_level = I915_CACHE_NONE;
@ -227,8 +228,6 @@ static int fake_get_huge_pages(struct drm_i915_gem_object *obj)
i915_sg_trim(st);
obj->mm.madv = I915_MADV_DONTNEED;
__i915_gem_object_set_pages(obj, st, sg_page_sizes);
return 0;
@ -261,8 +260,6 @@ static int fake_get_huge_pages_single(struct drm_i915_gem_object *obj)
sg_dma_len(sg) = obj->base.size;
sg_dma_address(sg) = page_size;
obj->mm.madv = I915_MADV_DONTNEED;
__i915_gem_object_set_pages(obj, st, sg->length);
return 0;
@ -281,7 +278,6 @@ static void fake_put_huge_pages(struct drm_i915_gem_object *obj,
{
fake_free_huge_pages(obj, pages);
obj->mm.dirty = false;
obj->mm.madv = I915_MADV_WILLNEED;
}
static const struct drm_i915_gem_object_ops fake_ops = {
@ -321,6 +317,8 @@ fake_huge_pages_object(struct drm_i915_private *i915, u64 size, bool single)
else
i915_gem_object_init(obj, &fake_ops);
i915_gem_object_set_volatile(obj);
obj->write_domain = I915_GEM_DOMAIN_CPU;
obj->read_domains = I915_GEM_DOMAIN_CPU;
obj->cache_level = I915_CACHE_NONE;
@ -452,6 +450,88 @@ static int igt_mock_exhaust_device_supported_pages(void *arg)
return err;
}
static int igt_mock_memory_region_huge_pages(void *arg)
{
const unsigned int flags[] = { 0, I915_BO_ALLOC_CONTIGUOUS };
struct i915_ppgtt *ppgtt = arg;
struct drm_i915_private *i915 = ppgtt->vm.i915;
unsigned long supported = INTEL_INFO(i915)->page_sizes;
struct intel_memory_region *mem;
struct drm_i915_gem_object *obj;
struct i915_vma *vma;
int bit;
int err = 0;
mem = mock_region_create(i915, 0, SZ_2G, I915_GTT_PAGE_SIZE_4K, 0);
if (IS_ERR(mem)) {
pr_err("%s failed to create memory region\n", __func__);
return PTR_ERR(mem);
}
for_each_set_bit(bit, &supported, ilog2(I915_GTT_MAX_PAGE_SIZE) + 1) {
unsigned int page_size = BIT(bit);
resource_size_t phys;
int i;
for (i = 0; i < ARRAY_SIZE(flags); ++i) {
obj = i915_gem_object_create_region(mem, page_size,
flags[i]);
if (IS_ERR(obj)) {
err = PTR_ERR(obj);
goto out_region;
}
vma = i915_vma_instance(obj, &ppgtt->vm, NULL);
if (IS_ERR(vma)) {
err = PTR_ERR(vma);
goto out_put;
}
err = i915_vma_pin(vma, 0, 0, PIN_USER);
if (err)
goto out_close;
err = igt_check_page_sizes(vma);
if (err)
goto out_unpin;
phys = i915_gem_object_get_dma_address(obj, 0);
if (!IS_ALIGNED(phys, page_size)) {
pr_err("%s addr misaligned(%pa) page_size=%u\n",
__func__, &phys, page_size);
err = -EINVAL;
goto out_unpin;
}
if (vma->page_sizes.gtt != page_size) {
pr_err("%s page_sizes.gtt=%u, expected=%u\n",
__func__, vma->page_sizes.gtt,
page_size);
err = -EINVAL;
goto out_unpin;
}
i915_vma_unpin(vma);
i915_vma_close(vma);
__i915_gem_object_put_pages(obj, I915_MM_NORMAL);
i915_gem_object_put(obj);
}
}
goto out_region;
out_unpin:
i915_vma_unpin(vma);
out_close:
i915_vma_close(vma);
out_put:
i915_gem_object_put(obj);
out_region:
intel_memory_region_put(mem);
return err;
}
static int igt_mock_ppgtt_misaligned_dma(void *arg)
{
struct i915_ppgtt *ppgtt = arg;
@ -1623,6 +1703,7 @@ int i915_gem_huge_page_mock_selftests(void)
{
static const struct i915_subtest tests[] = {
SUBTEST(igt_mock_exhaust_device_supported_pages),
SUBTEST(igt_mock_memory_region_huge_pages),
SUBTEST(igt_mock_ppgtt_misaligned_dma),
SUBTEST(igt_mock_ppgtt_huge_fill),
SUBTEST(igt_mock_ppgtt_64K),

2
drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c
View File

@ -263,7 +263,7 @@ static int live_parallel_switch(void *arg)
if (!data) {
i915_gem_context_unlock_engines(ctx);
err = -ENOMEM;
goto out;
goto out_file;
}
m = 0; /* Use the first context as our template for the engines */

8
drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c
View File

@ -357,10 +357,10 @@ static int igt_partial_tiling(void *arg)
tile.tiling = tiling;
switch (tiling) {
case I915_TILING_X:
tile.swizzle = i915->mm.bit_6_swizzle_x;
tile.swizzle = i915->ggtt.bit_6_swizzle_x;
break;
case I915_TILING_Y:
tile.swizzle = i915->mm.bit_6_swizzle_y;
tile.swizzle = i915->ggtt.bit_6_swizzle_y;
break;
}
@ -474,10 +474,10 @@ static int igt_smoke_tiling(void *arg)
break;
case I915_TILING_X:
tile.swizzle = i915->mm.bit_6_swizzle_x;
tile.swizzle = i915->ggtt.bit_6_swizzle_x;
break;
case I915_TILING_Y:
tile.swizzle = i915->mm.bit_6_swizzle_y;
tile.swizzle = i915->ggtt.bit_6_swizzle_y;
break;
}

2
drivers/gpu/drm/i915/gt/intel_context.c
View File

@ -62,7 +62,7 @@ int __intel_context_do_pin(struct intel_context *ce)
}
err = 0;
with_intel_runtime_pm(&ce->engine->i915->runtime_pm, wakeref)
with_intel_runtime_pm(ce->engine->uncore->rpm, wakeref)
err = ce->ops->pin(ce);
if (err)
goto err;

17
drivers/gpu/drm/i915/gt/intel_engine.h
View File

@ -136,6 +136,20 @@ execlists_active(const struct intel_engine_execlists *execlists)
return READ_ONCE(*execlists->active);
}
static inline void
execlists_active_lock_bh(struct intel_engine_execlists *execlists)
{
local_bh_disable(); /* prevent local softirq and lock recursion */
tasklet_lock(&execlists->tasklet);
}
static inline void
execlists_active_unlock_bh(struct intel_engine_execlists *execlists)
{
tasklet_unlock(&execlists->tasklet);
local_bh_enable(); /* restore softirq, and kick ksoftirqd! */
}
struct i915_request *
execlists_unwind_incomplete_requests(struct intel_engine_execlists *execlists);
@ -407,8 +421,9 @@ static inline void __intel_engine_reset(struct intel_engine_cs *engine,
engine->serial++; /* contexts lost */
}
bool intel_engine_is_idle(struct intel_engine_cs *engine);
bool intel_engines_are_idle(struct intel_gt *gt);
bool intel_engine_is_idle(struct intel_engine_cs *engine);
void intel_engine_flush_submission(struct intel_engine_cs *engine);
void intel_engines_reset_default_submission(struct intel_gt *gt);

101
drivers/gpu/drm/i915/gt/intel_engine_cs.c
View File

@ -277,6 +277,9 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id)
BUILD_BUG_ON(MAX_ENGINE_CLASS >= BIT(GEN11_ENGINE_CLASS_WIDTH));
BUILD_BUG_ON(MAX_ENGINE_INSTANCE >= BIT(GEN11_ENGINE_INSTANCE_WIDTH));
if (GEM_DEBUG_WARN_ON(id >= ARRAY_SIZE(gt->engine)))
return -EINVAL;
if (GEM_DEBUG_WARN_ON(info->class > MAX_ENGINE_CLASS))
return -EINVAL;
@ -293,6 +296,7 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id)
BUILD_BUG_ON(BITS_PER_TYPE(engine->mask) < I915_NUM_ENGINES);
engine->id = id;
engine->legacy_idx = INVALID_ENGINE;
engine->mask = BIT(id);
engine->i915 = gt->i915;
engine->gt = gt;
@ -328,6 +332,7 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id)
intel_engine_sanitize_mmio(engine);
gt->engine_class[info->class][info->instance] = engine;
gt->engine[id] = engine;
intel_engine_add_user(engine);
gt->i915->engine[id] = engine;
@ -736,6 +741,7 @@ intel_engine_init_active(struct intel_engine_cs *engine, unsigned int subclass)
static struct intel_context *
create_kernel_context(struct intel_engine_cs *engine)
{
static struct lock_class_key kernel;
struct intel_context *ce;
int err;
@ -751,6 +757,14 @@ create_kernel_context(struct intel_engine_cs *engine)
return ERR_PTR(err);
}
/*
* Give our perma-pinned kernel timelines a separate lockdep class,
* so that we can use them from within the normal user timelines
* should we need to inject GPU operations during their request
* construction.
*/
lockdep_set_class(&ce->timeline->mutex, &kernel);
return ce;
}
@ -1040,6 +1054,25 @@ static bool ring_is_idle(struct intel_engine_cs *engine)
return idle;
}
void intel_engine_flush_submission(struct intel_engine_cs *engine)
{
struct tasklet_struct *t = &engine->execlists.tasklet;
if (__tasklet_is_scheduled(t)) {
local_bh_disable();
if (tasklet_trylock(t)) {
/* Must wait for any GPU reset in progress. */
if (__tasklet_is_enabled(t))
t->func(t->data);
tasklet_unlock(t);
}
local_bh_enable();
}
/* Otherwise flush the tasklet if it was running on another cpu */
tasklet_unlock_wait(t);
}
/**
* intel_engine_is_idle() - Report if the engine has finished process all work
* @engine: the intel_engine_cs
@ -1058,21 +1091,9 @@ bool intel_engine_is_idle(struct intel_engine_cs *engine)
/* Waiting to drain ELSP? */
if (execlists_active(&engine->execlists)) {
struct tasklet_struct *t = &engine->execlists.tasklet;
synchronize_hardirq(engine->i915->drm.pdev->irq);
local_bh_disable();
if (tasklet_trylock(t)) {
/* Must wait for any GPU reset in progress. */
if (__tasklet_is_enabled(t))
t->func(t->data);
tasklet_unlock(t);
}
local_bh_enable();
/* Otherwise flush the tasklet if it was on another cpu */
tasklet_unlock_wait(t);
intel_engine_flush_submission(engine);
if (execlists_active(&engine->execlists))
return false;
@ -1102,7 +1123,7 @@ bool intel_engines_are_idle(struct intel_gt *gt)
if (!READ_ONCE(gt->awake))
return true;
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
if (!intel_engine_is_idle(engine))
return false;
}
@ -1115,7 +1136,7 @@ void intel_engines_reset_default_submission(struct intel_gt *gt)
struct intel_engine_cs *engine;
enum intel_engine_id id;
for_each_engine(engine, gt->i915, id)
for_each_engine(engine, gt, id)
engine->set_default_submission(engine);
}
@ -1225,13 +1246,22 @@ static struct intel_timeline *get_timeline(struct i915_request *rq)
return tl;
}
static const char *repr_timer(const struct timer_list *t)
{
if (!READ_ONCE(t->expires))
return "inactive";
if (timer_pending(t))
return "active";
return "expired";
}
static void intel_engine_print_registers(struct intel_engine_cs *engine,
struct drm_printer *m)
{
struct drm_i915_private *dev_priv = engine->i915;
const struct intel_engine_execlists * const execlists =
&engine->execlists;
unsigned long flags;
struct intel_engine_execlists * const execlists = &engine->execlists;
u64 addr;
if (engine->id == RENDER_CLASS && IS_GEN_RANGE(dev_priv, 4, 7))
@ -1288,19 +1318,20 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
unsigned int idx;
u8 read, write;
drm_printf(m, "\tExeclist status: 0x%08x %08x, entries %u\n",
ENGINE_READ(engine, RING_EXECLIST_STATUS_LO),
ENGINE_READ(engine, RING_EXECLIST_STATUS_HI),
num_entries);
drm_printf(m, "\tExeclist tasklet queued? %s (%s), timeslice? %s\n",
yesno(test_bit(TASKLET_STATE_SCHED,
&engine->execlists.tasklet.state)),
enableddisabled(!atomic_read(&engine->execlists.tasklet.count)),
repr_timer(&engine->execlists.timer));
read = execlists->csb_head;
write = READ_ONCE(*execlists->csb_write);
drm_printf(m, "\tExeclist CSB read %d, write %d, tasklet queued? %s (%s)\n",
read, write,
yesno(test_bit(TASKLET_STATE_SCHED,
&engine->execlists.tasklet.state)),
enableddisabled(!atomic_read(&engine->execlists.tasklet.count)));
drm_printf(m, "\tExeclist status: 0x%08x %08x; CSB read:%d, write:%d, entries:%d\n",
ENGINE_READ(engine, RING_EXECLIST_STATUS_LO),
ENGINE_READ(engine, RING_EXECLIST_STATUS_HI),
read, write, num_entries);
if (read >= num_entries)
read = 0;
if (write >= num_entries)
@ -1313,7 +1344,7 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
idx, hws[idx * 2], hws[idx * 2 + 1]);
}
spin_lock_irqsave(&engine->active.lock, flags);
execlists_active_lock_bh(execlists);
for (port = execlists->active; (rq = *port); port++) {
char hdr[80];
int len;
@ -1351,7 +1382,7 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
if (tl)
intel_timeline_put(tl);
}
spin_unlock_irqrestore(&engine->active.lock, flags);
execlists_active_unlock_bh(execlists);
} else if (INTEL_GEN(dev_priv) > 6) {
drm_printf(m, "\tPP_DIR_BASE: 0x%08x\n",
ENGINE_READ(engine, RING_PP_DIR_BASE));
@ -1458,10 +1489,10 @@ void intel_engine_dump(struct intel_engine_cs *engine,
spin_unlock_irqrestore(&engine->active.lock, flags);
drm_printf(m, "\tMMIO base: 0x%08x\n", engine->mmio_base);
wakeref = intel_runtime_pm_get_if_in_use(&engine->i915->runtime_pm);
wakeref = intel_runtime_pm_get_if_in_use(engine->uncore->rpm);
if (wakeref) {
intel_engine_print_registers(engine, m);
intel_runtime_pm_put(&engine->i915->runtime_pm, wakeref);
intel_runtime_pm_put(engine->uncore->rpm, wakeref);
} else {
drm_printf(m, "\tDevice is asleep; skipping register dump\n");
}
@ -1493,8 +1524,8 @@ int intel_enable_engine_stats(struct intel_engine_cs *engine)
if (!intel_engine_supports_stats(engine))
return -ENODEV;
spin_lock_irqsave(&engine->active.lock, flags);
write_seqlock(&engine->stats.lock);
execlists_active_lock_bh(execlists);
write_seqlock_irqsave(&engine->stats.lock, flags);
if (unlikely(engine->stats.enabled == ~0)) {
err = -EBUSY;
@ -1522,8 +1553,8 @@ int intel_enable_engine_stats(struct intel_engine_cs *engine)
}
unlock:
write_sequnlock(&engine->stats.lock);
spin_unlock_irqrestore(&engine->active.lock, flags);
write_sequnlock_irqrestore(&engine->stats.lock, flags);
execlists_active_unlock_bh(execlists);
return err;
}

2
drivers/gpu/drm/i915/gt/intel_engine_pm.c
View File

@ -185,7 +185,7 @@ static const struct intel_wakeref_ops wf_ops = {
void intel_engine_init__pm(struct intel_engine_cs *engine)
{
struct intel_runtime_pm *rpm = &engine->i915->runtime_pm;
struct intel_runtime_pm *rpm = engine->uncore->rpm;
intel_wakeref_init(&engine->wakeref, rpm, &wf_ops);
}

1
drivers/gpu/drm/i915/gt/intel_engine_types.h
View File

@ -148,6 +148,7 @@ enum intel_engine_id {
VECS1,
#define _VECS(n) (VECS0 + (n))
I915_NUM_ENGINES
#define INVALID_ENGINE ((enum intel_engine_id)-1)
};
struct st_preempt_hang {

18
drivers/gpu/drm/i915/gt/intel_engine_user.c
View File

@ -160,10 +160,10 @@ static int legacy_ring_idx(const struct legacy_ring *ring)
};
if (GEM_DEBUG_WARN_ON(ring->class >= ARRAY_SIZE(map)))
return -1;
return INVALID_ENGINE;
if (GEM_DEBUG_WARN_ON(ring->instance >= map[ring->class].max))
return -1;
return INVALID_ENGINE;
return map[ring->class].base + ring->instance;
}
@ -171,23 +171,15 @@ static int legacy_ring_idx(const struct legacy_ring *ring)
static void add_legacy_ring(struct legacy_ring *ring,
struct intel_engine_cs *engine)
{
int idx;
if (engine->gt != ring->gt || engine->class != ring->class) {
ring->gt = engine->gt;
ring->class = engine->class;
ring->instance = 0;
}
idx = legacy_ring_idx(ring);
if (unlikely(idx == -1))
return;
GEM_BUG_ON(idx >= ARRAY_SIZE(ring->gt->engine));
ring->gt->engine[idx] = engine;
ring->instance++;
engine->legacy_idx = idx;
engine->legacy_idx = legacy_ring_idx(ring);
if (engine->legacy_idx != INVALID_ENGINE)
ring->instance++;
}
void intel_engines_driver_register(struct drm_i915_private *i915)

7
drivers/gpu/drm/i915/gt/intel_gpu_commands.h
View File

@ -138,6 +138,7 @@
/* Gen11+. addr = base + (ctx_restore ? offset & GENMASK(12,2) : offset) */
#define MI_LRI_CS_MMIO (1<<19)
#define MI_LRI_FORCE_POSTED (1<<12)
#define MI_LOAD_REGISTER_IMM_MAX_REGS (126)
#define MI_STORE_REGISTER_MEM MI_INSTR(0x24, 1)
#define MI_STORE_REGISTER_MEM_GEN8 MI_INSTR(0x24, 2)
#define MI_SRM_LRM_GLOBAL_GTT (1<<22)
@ -162,7 +163,8 @@
#define MI_BATCH_BUFFER_START MI_INSTR(0x31, 0)
#define MI_BATCH_GTT (2<<6) /* aliased with (1<<7) on gen4 */
#define MI_BATCH_BUFFER_START_GEN8 MI_INSTR(0x31, 1)
#define MI_BATCH_RESOURCE_STREAMER (1<<10)
#define MI_BATCH_RESOURCE_STREAMER REG_BIT(10)
#define MI_BATCH_PREDICATE REG_BIT(15) /* HSW+ on RCS only*/
/*
* 3D instructions used by the kernel
@ -223,6 +225,7 @@
#define PIPE_CONTROL_CS_STALL (1<<20)
#define PIPE_CONTROL_TLB_INVALIDATE (1<<18)
#define PIPE_CONTROL_MEDIA_STATE_CLEAR (1<<16)
#define PIPE_CONTROL_WRITE_TIMESTAMP (3<<14)
#define PIPE_CONTROL_QW_WRITE (1<<14)
#define PIPE_CONTROL_POST_SYNC_OP_MASK (3<<14)
#define PIPE_CONTROL_DEPTH_STALL (1<<13)
@ -230,7 +233,9 @@
#define PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH (1<<12) /* gen6+ */
#define PIPE_CONTROL_INSTRUCTION_CACHE_INVALIDATE (1<<11) /* MBZ on ILK */
#define PIPE_CONTROL_TEXTURE_CACHE_INVALIDATE (1<<10) /* GM45+ only */
#define PIPE_CONTROL_L3_RO_CACHE_INVALIDATE REG_BIT(10) /* gen12 */
#define PIPE_CONTROL_INDIRECT_STATE_DISABLE (1<<9)
#define PIPE_CONTROL_HDC_PIPELINE_FLUSH REG_BIT(9) /* gen12 */
#define PIPE_CONTROL_NOTIFY (1<<8)
#define PIPE_CONTROL_FLUSH_ENABLE (1<<7) /* gen7+ */
#define PIPE_CONTROL_DC_FLUSH_ENABLE (1<<5)

11
drivers/gpu/drm/i915/gt/intel_gt.c
View File

@ -186,7 +186,7 @@ intel_gt_clear_error_registers(struct intel_gt *gt,
struct intel_engine_cs *engine;
enum intel_engine_id id;
for_each_engine_masked(engine, i915, engine_mask, id)
for_each_engine_masked(engine, gt, engine_mask, id)
gen8_clear_engine_error_register(engine);
}
}
@ -197,7 +197,7 @@ static void gen6_check_faults(struct intel_gt *gt)
enum intel_engine_id id;
u32 fault;
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
fault = GEN6_RING_FAULT_REG_READ(engine);
if (fault & RING_FAULT_VALID) {
DRM_DEBUG_DRIVER("Unexpected fault\n"
@ -273,7 +273,7 @@ void intel_gt_check_and_clear_faults(struct intel_gt *gt)
void intel_gt_flush_ggtt_writes(struct intel_gt *gt)
{
struct drm_i915_private *i915 = gt->i915;
struct intel_uncore *uncore = gt->uncore;
intel_wakeref_t wakeref;
/*
@ -297,13 +297,12 @@ void intel_gt_flush_ggtt_writes(struct intel_gt *gt)
wmb();
if (INTEL_INFO(i915)->has_coherent_ggtt)
if (INTEL_INFO(gt->i915)->has_coherent_ggtt)
return;
intel_gt_chipset_flush(gt);
with_intel_runtime_pm(&i915->runtime_pm, wakeref) {
struct intel_uncore *uncore = gt->uncore;
with_intel_runtime_pm(uncore->rpm, wakeref) {
unsigned long flags;
spin_lock_irqsave(&uncore->lock, flags);

21
drivers/gpu/drm/i915/gt/intel_gt_pm.c
View File

@ -94,7 +94,7 @@ static const struct intel_wakeref_ops wf_ops = {
void intel_gt_pm_init_early(struct intel_gt *gt)
{
intel_wakeref_init(&gt->wakeref, &gt->i915->runtime_pm, &wf_ops);
intel_wakeref_init(&gt->wakeref, gt->uncore->rpm, &wf_ops);
BLOCKING_INIT_NOTIFIER_HEAD(&gt->pm_notifications);
}
@ -136,11 +136,18 @@ void intel_gt_sanitize(struct intel_gt *gt, bool force)
intel_uc_sanitize(&gt->uc);
if (!reset_engines(gt) && !force)
return;
for_each_engine(engine, gt, id)
if (engine->reset.prepare)
engine->reset.prepare(engine);
for_each_engine(engine, gt->i915, id)
__intel_engine_reset(engine, false);
if (reset_engines(gt) || force) {
for_each_engine(engine, gt, id)
__intel_engine_reset(engine, false);
}
for_each_engine(engine, gt, id)
if (engine->reset.finish)
engine->reset.finish(engine);
}
void intel_gt_pm_disable(struct intel_gt *gt)
@ -170,7 +177,7 @@ int intel_gt_resume(struct intel_gt *gt)
intel_uncore_forcewake_get(gt->uncore, FORCEWAKE_ALL);
intel_rc6_sanitize(&gt->rc6);
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
struct intel_context *ce;
intel_engine_pm_get(engine);
@ -222,7 +229,7 @@ void intel_gt_suspend(struct intel_gt *gt)
/* We expect to be idle already; but also want to be independent */
wait_for_idle(gt);
with_intel_runtime_pm(&gt->i915->runtime_pm, wakeref)
with_intel_runtime_pm(gt->uncore->rpm, wakeref)
intel_rc6_disable(&gt->rc6);
}

18
drivers/gpu/drm/i915/gt/intel_gt_requests.c
View File

@ -4,6 +4,7 @@
* Copyright © 2019 Intel Corporation
*/
#include "i915_drv.h" /* for_each_engine() */
#include "i915_request.h"
#include "intel_gt.h"
#include "intel_gt_pm.h"
@ -19,6 +20,15 @@ static void retire_requests(struct intel_timeline *tl)
break;
}
static void flush_submission(struct intel_gt *gt)
{
struct intel_engine_cs *engine;
enum intel_engine_id id;
for_each_engine(engine, gt, id)
intel_engine_flush_submission(engine);
}
long intel_gt_retire_requests_timeout(struct intel_gt *gt, long timeout)
{
struct intel_gt_timelines *timelines = &gt->timelines;
@ -32,10 +42,14 @@ long intel_gt_retire_requests_timeout(struct intel_gt *gt, long timeout)
if (unlikely(timeout < 0))
timeout = -timeout, interruptible = false;
flush_submission(gt); /* kick the ksoftirqd tasklets */
spin_lock_irqsave(&timelines->lock, flags);
list_for_each_entry_safe(tl, tn, &timelines->active_list, link) {
if (!mutex_trylock(&tl->mutex))
if (!mutex_trylock(&tl->mutex)) {
active_count++; /* report busy to caller, try again? */
continue;
}
intel_timeline_get(tl);
GEM_BUG_ON(!tl->active_count);
@ -48,7 +62,7 @@ long intel_gt_retire_requests_timeout(struct intel_gt *gt, long timeout)
fence = i915_active_fence_get(&tl->last_request);
if (fence) {
timeout = dma_fence_wait_timeout(fence,
true,
interruptible,
timeout);
dma_fence_put(fence);
}

7
drivers/gpu/drm/i915/gt/intel_gt_types.h
View File

@ -17,6 +17,7 @@
#include "i915_vma.h"
#include "intel_engine_types.h"
#include "intel_llc_types.h"
#include "intel_reset_types.h"
#include "intel_rc6_types.h"
#include "intel_wakeref.h"
@ -79,6 +80,7 @@ struct intel_gt {
*/
intel_wakeref_t awake;
struct intel_llc llc;
struct intel_rc6 rc6;
struct blocking_notifier_head pm_notifications;
@ -109,6 +111,11 @@ enum intel_gt_scratch_field {
/* 8 bytes */
INTEL_GT_SCRATCH_FIELD_COHERENTL3_WA = 256,
/* 6 * 8 bytes */
INTEL_GT_SCRATCH_FIELD_PERF_CS_GPR = 2048,
/* 4 bytes */
INTEL_GT_SCRATCH_FIELD_PERF_PREDICATE_RESULT_1 = 2096,
};
#endif /* __INTEL_GT_TYPES_H__ */

10
drivers/gpu/drm/i915/gt/intel_hangcheck.c
View File

@ -237,7 +237,7 @@ static void hangcheck_declare_hang(struct intel_gt *gt,
hung &= ~stuck;
len = scnprintf(msg, sizeof(msg),
"%s on ", stuck == hung ? "no progress" : "hang");
for_each_engine_masked(engine, gt->i915, hung, tmp)
for_each_engine_masked(engine, gt, hung, tmp)
len += scnprintf(msg + len, sizeof(msg) - len,
"%s, ", engine->name);
msg[len-2] = '\0';
@ -271,7 +271,7 @@ static void hangcheck_elapsed(struct work_struct *work)
if (intel_gt_is_wedged(gt))
return;
wakeref = intel_runtime_pm_get_if_in_use(&gt->i915->runtime_pm);
wakeref = intel_runtime_pm_get_if_in_use(gt->uncore->rpm);
if (!wakeref)
return;
@ -281,7 +281,7 @@ static void hangcheck_elapsed(struct work_struct *work)
*/
intel_uncore_arm_unclaimed_mmio_detection(gt->uncore);
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
struct hangcheck hc;
intel_engine_breadcrumbs_irq(engine);
@ -303,7 +303,7 @@ static void hangcheck_elapsed(struct work_struct *work)
if (GEM_SHOW_DEBUG() && (hung | stuck)) {
struct drm_printer p = drm_debug_printer("hangcheck");
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
if (intel_engine_is_idle(engine))
continue;
@ -322,7 +322,7 @@ static void hangcheck_elapsed(struct work_struct *work)
if (hung)
hangcheck_declare_hang(gt, hung, stuck);
intel_runtime_pm_put(&gt->i915->runtime_pm, wakeref);
intel_runtime_pm_put(gt->uncore->rpm, wakeref);
/* Reset timer in case GPU hangs without another request being added */
intel_gt_queue_hangcheck(gt);

161
drivers/gpu/drm/i915/gt/intel_llc.c Normal file
View File

@ -0,0 +1,161 @@
/*
* SPDX-License-Identifier: MIT
*
* Copyright © 2019 Intel Corporation
*/
#include <linux/cpufreq.h>
#include "i915_drv.h"
#include "intel_gt.h"
#include "intel_llc.h"
#include "intel_sideband.h"
struct ia_constants {
unsigned int min_gpu_freq;
unsigned int max_gpu_freq;
unsigned int min_ring_freq;
unsigned int max_ia_freq;
};
static struct intel_gt *llc_to_gt(struct intel_llc *llc)
{
return container_of(llc, struct intel_gt, llc);
}
static unsigned int cpu_max_MHz(void)
{
struct cpufreq_policy *policy;
unsigned int max_khz;
policy = cpufreq_cpu_get(0);
if (policy) {
max_khz = policy->cpuinfo.max_freq;
cpufreq_cpu_put(policy);
} else {
/*
* Default to measured freq if none found, PCU will ensure we
* don't go over
*/
max_khz = tsc_khz;
}
return max_khz / 1000;
}
static bool get_ia_constants(struct intel_llc *llc,
struct ia_constants *consts)
{
struct drm_i915_private *i915 = llc_to_gt(llc)->i915;
struct intel_rps *rps = &i915->gt_pm.rps;
if (rps->max_freq <= rps->min_freq)
return false;
consts->max_ia_freq = cpu_max_MHz();
consts->min_ring_freq =
intel_uncore_read(llc_to_gt(llc)->uncore, DCLK) & 0xf;
/* convert DDR frequency from units of 266.6MHz to bandwidth */
consts->min_ring_freq = mult_frac(consts->min_ring_freq, 8, 3);
consts->min_gpu_freq = rps->min_freq;
consts->max_gpu_freq = rps->max_freq;
if (INTEL_GEN(i915) >= 9) {
/* Convert GT frequency to 50 HZ units */
consts->min_gpu_freq /= GEN9_FREQ_SCALER;
consts->max_gpu_freq /= GEN9_FREQ_SCALER;
}
return true;
}
static void calc_ia_freq(struct intel_llc *llc,
unsigned int gpu_freq,
const struct ia_constants *consts,
unsigned int *out_ia_freq,
unsigned int *out_ring_freq)
{
struct drm_i915_private *i915 = llc_to_gt(llc)->i915;
const int diff = consts->max_gpu_freq - gpu_freq;
unsigned int ia_freq = 0, ring_freq = 0;
if (INTEL_GEN(i915) >= 9) {
/*
* ring_freq = 2 * GT. ring_freq is in 100MHz units
* No floor required for ring frequency on SKL.
*/
ring_freq = gpu_freq;
} else if (INTEL_GEN(i915) >= 8) {
/* max(2 * GT, DDR). NB: GT is 50MHz units */
ring_freq = max(consts->min_ring_freq, gpu_freq);
} else if (IS_HASWELL(i915)) {
ring_freq = mult_frac(gpu_freq, 5, 4);
ring_freq = max(consts->min_ring_freq, ring_freq);
/* leave ia_freq as the default, chosen by cpufreq */
} else {
const int min_freq = 15;
const int scale = 180;
/*
* On older processors, there is no separate ring
* clock domain, so in order to boost the bandwidth
* of the ring, we need to upclock the CPU (ia_freq).
*
* For GPU frequencies less than 750MHz,
* just use the lowest ring freq.
*/
if (gpu_freq < min_freq)
ia_freq = 800;
else
ia_freq = consts->max_ia_freq - diff * scale / 2;
ia_freq = DIV_ROUND_CLOSEST(ia_freq, 100);
}
*out_ia_freq = ia_freq;
*out_ring_freq = ring_freq;
}
static void gen6_update_ring_freq(struct intel_llc *llc)
{
struct drm_i915_private *i915 = llc_to_gt(llc)->i915;
struct ia_constants consts;
unsigned int gpu_freq;
if (!get_ia_constants(llc, &consts))
return;
/*
* For each potential GPU frequency, load a ring frequency we'd like
* to use for memory access. We do this by specifying the IA frequency
* the PCU should use as a reference to determine the ring frequency.
*/
for (gpu_freq = consts.max_gpu_freq;
gpu_freq >= consts.min_gpu_freq;
gpu_freq--) {
unsigned int ia_freq, ring_freq;
calc_ia_freq(llc, gpu_freq, &consts, &ia_freq, &ring_freq);
sandybridge_pcode_write(i915,
GEN6_PCODE_WRITE_MIN_FREQ_TABLE,
ia_freq << GEN6_PCODE_FREQ_IA_RATIO_SHIFT |
ring_freq << GEN6_PCODE_FREQ_RING_RATIO_SHIFT |
gpu_freq);
}
}
void intel_llc_enable(struct intel_llc *llc)
{
if (HAS_LLC(llc_to_gt(llc)->i915))
gen6_update_ring_freq(llc);
}
void intel_llc_disable(struct intel_llc *llc)
{
/* Currently there is no HW configuration to be done to disable. */
}
#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
#include "selftest_llc.c"
#endif

15
drivers/gpu/drm/i915/gt/intel_llc.h Normal file
View File

@ -0,0 +1,15 @@
/*
* SPDX-License-Identifier: MIT
*
* Copyright © 2019 Intel Corporation
*/
#ifndef INTEL_LLC_H
#define INTEL_LLC_H
struct intel_llc;
void intel_llc_enable(struct intel_llc *llc);
void intel_llc_disable(struct intel_llc *llc);
#endif /* INTEL_LLC_H */

13
drivers/gpu/drm/i915/gt/intel_llc_types.h Normal file
View File

@ -0,0 +1,13 @@
/*
* SPDX-License-Identifier: MIT
*
* Copyright © 2019 Intel Corporation
*/
#ifndef INTEL_LLC_TYPES_H
#define INTEL_LLC_TYPES_H
struct intel_llc {
};
#endif /* INTEL_LLC_TYPES_H */

212
drivers/gpu/drm/i915/gt/intel_lrc.c
View File

@ -247,8 +247,12 @@ static void __context_pin_release(struct intel_context *ce)
static void mark_eio(struct i915_request *rq)
{
if (!i915_request_signaled(rq))
dma_fence_set_error(&rq->fence, -EIO);
if (i915_request_completed(rq))
return;
GEM_BUG_ON(i915_request_signaled(rq));
dma_fence_set_error(&rq->fence, -EIO);
i915_request_mark_complete(rq);
}
@ -348,10 +352,15 @@ static inline bool need_preempt(const struct intel_engine_cs *engine,
* However, the priority hint is a mere hint that we may need to
* preempt. If that hint is stale or we may be trying to preempt
* ourselves, ignore the request.
*
* More naturally we would write
* prio >= max(0, last);
* except that we wish to prevent triggering preemption at the same
* priority level: the task that is running should remain running
* to preserve FIFO ordering of dependencies.
*/
last_prio = effective_prio(rq);
if (!i915_scheduler_need_preempt(engine->execlists.queue_priority_hint,
last_prio))
last_prio = max(effective_prio(rq), I915_PRIORITY_NORMAL - 1);
if (engine->execlists.queue_priority_hint <= last_prio)
return false;
/*
@ -669,64 +678,6 @@ static const u8 gen12_xcs_offsets[] = {
REG16(0x274),
REG16(0x270),
NOP(13),
LRI(2, POSTED),
REG16(0x200),
REG16(0x204),
NOP(11),
LRI(50, POSTED),
REG16(0x588),
REG16(0x588),
REG16(0x588),
REG16(0x588),
REG16(0x588),
REG16(0x588),
REG(0x028),
REG(0x09c),
REG(0x0c0),
REG(0x178),
REG(0x17c),
REG16(0x358),
REG(0x170),
REG(0x150),
REG(0x154),
REG(0x158),
REG16(0x41c),
REG16(0x600),
REG16(0x604),
REG16(0x608),
REG16(0x60c),
REG16(0x610),
REG16(0x614),
REG16(0x618),
REG16(0x61c),
REG16(0x620),
REG16(0x624),
REG16(0x628),
REG16(0x62c),
REG16(0x630),
REG16(0x634),
REG16(0x638),
REG16(0x63c),
REG16(0x640),
REG16(0x644),
REG16(0x648),
REG16(0x64c),
REG16(0x650),
REG16(0x654),
REG16(0x658),
REG16(0x65c),
REG16(0x660),
REG16(0x664),
REG16(0x668),
REG16(0x66c),
REG16(0x670),
REG16(0x674),
REG16(0x678),
REG16(0x67c),
REG(0x068),
END(),
};
@ -857,6 +808,15 @@ static const u8 gen12_rcs_offsets[] = {
static const u8 *reg_offsets(const struct intel_engine_cs *engine)
{
/*
* The gen12+ lists only have the registers we program in the basic
* default state. We rely on the context image using relative
* addressing to automatic fixup the register state between the
* physical engines for virtual engine.
*/
GEM_BUG_ON(INTEL_GEN(engine->i915) >= 12 &&
!intel_engine_has_relative_mmio(engine));
if (engine->class == RENDER_CLASS) {
if (INTEL_GEN(engine->i915) >= 12)
return gen12_rcs_offsets;
@ -892,7 +852,6 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
list_for_each_entry_safe_reverse(rq, rn,
&engine->active.requests,
sched.link) {
struct intel_engine_cs *owner;
if (i915_request_completed(rq))
continue; /* XXX */
@ -907,8 +866,7 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
* engine so that it can be moved across onto another physical
* engine as load dictates.
*/
owner = rq->hw_context->engine;
if (likely(owner == engine)) {
if (likely(rq->execution_mask == engine->mask)) {
GEM_BUG_ON(rq_prio(rq) == I915_PRIORITY_INVALID);
if (rq_prio(rq) != prio) {
prio = rq_prio(rq);
@ -919,6 +877,8 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
list_move(&rq->sched.link, pl);
active = rq;
} else {
struct intel_engine_cs *owner = rq->hw_context->engine;
/*
* Decouple the virtual breadcrumb before moving it
* back to the virtual engine -- we don't want the
@ -928,7 +888,8 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
*/
if (test_bit(DMA_FENCE_FLAG_ENABLE_SIGNAL_BIT,
&rq->fence.flags)) {
spin_lock(&rq->lock);
spin_lock_nested(&rq->lock,
SINGLE_DEPTH_NESTING);
i915_request_cancel_breadcrumb(rq);
spin_unlock(&rq->lock);
}
@ -1092,6 +1053,10 @@ static u64 execlists_update_context(const struct i915_request *rq)
desc = ce->lrc_desc;
ce->lrc_desc &= ~CTX_DESC_FORCE_RESTORE;
/* Wa_1607138340:tgl */
if (IS_TGL_REVID(rq->i915, TGL_REVID_A0, TGL_REVID_A0))
desc |= CTX_DESC_FORCE_RESTORE;
return desc;
}
@ -1137,25 +1102,45 @@ assert_pending_valid(const struct intel_engine_execlists *execlists,
trace_ports(execlists, msg, execlists->pending);
if (!execlists->pending[0])
if (!execlists->pending[0]) {
GEM_TRACE_ERR("Nothing pending for promotion!\n");
return false;
}
if (execlists->pending[execlists_num_ports(execlists)])
if (execlists->pending[execlists_num_ports(execlists)]) {
GEM_TRACE_ERR("Excess pending[%d] for promotion!\n",
execlists_num_ports(execlists));
return false;
}
for (port = execlists->pending; (rq = *port); port++) {
if (ce == rq->hw_context)
if (ce == rq->hw_context) {
GEM_TRACE_ERR("Duplicate context in pending[%zd]\n",
port - execlists->pending);
return false;
}
ce = rq->hw_context;
if (i915_request_completed(rq))
continue;
if (i915_active_is_idle(&ce->active))
if (i915_active_is_idle(&ce->active)) {
GEM_TRACE_ERR("Inactive context in pending[%zd]\n",
port - execlists->pending);
return false;
}
if (!i915_vma_is_pinned(ce->state))
if (!i915_vma_is_pinned(ce->state)) {
GEM_TRACE_ERR("Unpinned context in pending[%zd]\n",
port - execlists->pending);
return false;
}
if (!i915_vma_is_pinned(ce->ring->vma)) {
GEM_TRACE_ERR("Unpinned ringbuffer in pending[%zd]\n",
port - execlists->pending);
return false;
}
}
return ce;
@ -1232,6 +1217,10 @@ static bool can_merge_rq(const struct i915_request *prev,
if (i915_request_completed(next))
return true;
if (unlikely((prev->flags ^ next->flags) &
(I915_REQUEST_NOPREEMPT | I915_REQUEST_SENTINEL)))
return false;
if (!can_merge_ctx(prev->hw_context, next->hw_context))
return false;
@ -1288,7 +1277,7 @@ static void virtual_xfer_breadcrumbs(struct virtual_engine *ve,
static struct i915_request *
last_active(const struct intel_engine_execlists *execlists)
{
struct i915_request * const *last = execlists->active;
struct i915_request * const *last = READ_ONCE(execlists->active);
while (*last && i915_request_completed(*last))
last++;
@ -1489,7 +1478,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
last->hw_context->lrc_desc |= CTX_DESC_FORCE_RESTORE;
last = NULL;
} else if (need_timeslice(engine, last) &&
!timer_pending(&engine->execlists.timer)) {
timer_expired(&engine->execlists.timer)) {
GEM_TRACE("%s: expired last=%llx:%lld, prio=%d, hint=%d\n",
engine->name,
last->fence.context,
@ -1525,8 +1514,17 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
* submission.
*/
if (!list_is_last(&last->sched.link,
&engine->active.requests))
&engine->active.requests)) {
/*
* Even if ELSP[1] is occupied and not worthy
* of timeslices, our queue might be.
*/
if (!execlists->timer.expires &&
need_timeslice(engine, last))
mod_timer(&execlists->timer,
jiffies + 1);
return;
}
/*
* WaIdleLiteRestore:bdw,skl
@ -1680,6 +1678,9 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
if (last->hw_context == rq->hw_context)
goto done;
if (i915_request_has_sentinel(last))
goto done;
/*
* If GVT overrides us we only ever submit
* port[0], leaving port[1] empty. Note that we
@ -1859,6 +1860,13 @@ static void process_csb(struct intel_engine_cs *engine)
const u8 num_entries = execlists->csb_size;
u8 head, tail;
/*
* As we modify our execlists state tracking we require exclusive
* access. Either we are inside the tasklet, or the tasklet is disabled
* and we assume that is only inside the reset paths and so serialised.
*/
GEM_BUG_ON(!tasklet_is_locked(&execlists->tasklet) &&
!reset_in_progress(execlists));
GEM_BUG_ON(USES_GUC_SUBMISSION(engine->i915));
/*
@ -1920,6 +1928,9 @@ static void process_csb(struct intel_engine_cs *engine)
else
promote = gen8_csb_parse(execlists, buf + 2 * head);
if (promote) {
if (!inject_preempt_hang(execlists))
ring_set_paused(engine, 0);
/* cancel old inflight, prepare for switch */
trace_ports(execlists, "preempted", execlists->active);
while (*execlists->active)
@ -1935,9 +1946,8 @@ static void process_csb(struct intel_engine_cs *engine)
if (enable_timeslice(execlists))
mod_timer(&execlists->timer, jiffies + 1);
if (!inject_preempt_hang(execlists))
ring_set_paused(engine, 0);
else
cancel_timer(&execlists->timer);
WRITE_ONCE(execlists->pending[0], NULL);
} else {
@ -1980,8 +1990,11 @@ static void process_csb(struct intel_engine_cs *engine)
static void __execlists_submission_tasklet(struct intel_engine_cs *const engine)
{
lockdep_assert_held(&engine->active.lock);
if (!engine->execlists.pending[0])
if (!engine->execlists.pending[0]) {
rcu_read_lock(); /* protect peeking at execlists->active */
execlists_dequeue(engine);
rcu_read_unlock();
}
}
/*
@ -2773,8 +2786,10 @@ static void __execlists_reset(struct intel_engine_cs *engine, bool stalled)
if (!rq)
goto unwind;
/* We still have requests in-flight; the engine should be active */
GEM_BUG_ON(!intel_engine_pm_is_awake(engine));
ce = rq->hw_context;
GEM_BUG_ON(i915_active_is_idle(&ce->active));
GEM_BUG_ON(!i915_vma_is_pinned(ce->state));
/* Proclaim we have exclusive access to the context image! */
@ -2782,10 +2797,13 @@ static void __execlists_reset(struct intel_engine_cs *engine, bool stalled)
rq = active_request(rq);
if (!rq) {
/* Idle context; tidy up the ring so we can restart afresh */
ce->ring->head = ce->ring->tail;
goto out_replay;
}
/* Context has requests still in-flight; it should not be idle! */
GEM_BUG_ON(i915_active_is_idle(&ce->active));
ce->ring->head = intel_ring_wrap(ce->ring, rq->head);
/*
@ -3205,8 +3223,11 @@ static int gen12_emit_flush_render(struct i915_request *request,
flags |= PIPE_CONTROL_TILE_CACHE_FLUSH;
flags |= PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH;
flags |= PIPE_CONTROL_DEPTH_CACHE_FLUSH;
/* Wa_1409600907:tgl */
flags |= PIPE_CONTROL_DEPTH_STALL;
flags |= PIPE_CONTROL_DC_FLUSH_ENABLE;
flags |= PIPE_CONTROL_FLUSH_ENABLE;
flags |= PIPE_CONTROL_HDC_PIPELINE_FLUSH;
flags |= PIPE_CONTROL_STORE_DATA_INDEX;
flags |= PIPE_CONTROL_QW_WRITE;
@ -3232,6 +3253,7 @@ static int gen12_emit_flush_render(struct i915_request *request,
flags |= PIPE_CONTROL_VF_CACHE_INVALIDATE;
flags |= PIPE_CONTROL_CONST_CACHE_INVALIDATE;
flags |= PIPE_CONTROL_STATE_CACHE_INVALIDATE;
flags |= PIPE_CONTROL_L3_RO_CACHE_INVALIDATE;
flags |= PIPE_CONTROL_STORE_DATA_INDEX;
flags |= PIPE_CONTROL_QW_WRITE;
@ -3253,6 +3275,26 @@ static int gen12_emit_flush_render(struct i915_request *request,
*cs++ = preparser_disable(false);
intel_ring_advance(request, cs);
/*
* Wa_1604544889:tgl
*/
if (IS_TGL_REVID(request->i915, TGL_REVID_A0, TGL_REVID_A0)) {
flags = 0;
flags |= PIPE_CONTROL_CS_STALL;
flags |= PIPE_CONTROL_HDC_PIPELINE_FLUSH;
flags |= PIPE_CONTROL_STORE_DATA_INDEX;
flags |= PIPE_CONTROL_QW_WRITE;
cs = intel_ring_begin(request, 6);
if (IS_ERR(cs))
return PTR_ERR(cs);
cs = gen8_emit_pipe_control(cs, flags,
LRC_PPHWSP_SCRATCH_ADDR);
intel_ring_advance(request, cs);
}
}
return 0;
@ -3415,15 +3457,18 @@ gen12_emit_fini_breadcrumb_rcs(struct i915_request *request, u32 *cs)
PIPE_CONTROL_TILE_CACHE_FLUSH |
PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH |
PIPE_CONTROL_DEPTH_CACHE_FLUSH |
/* Wa_1409600907:tgl */
PIPE_CONTROL_DEPTH_STALL |
PIPE_CONTROL_DC_FLUSH_ENABLE |
PIPE_CONTROL_FLUSH_ENABLE);
PIPE_CONTROL_FLUSH_ENABLE |
PIPE_CONTROL_HDC_PIPELINE_FLUSH);
return gen12_emit_fini_breadcrumb_footer(request, cs);
}
static void execlists_park(struct intel_engine_cs *engine)
{
del_timer(&engine->execlists.timer);
cancel_timer(&engine->execlists.timer);
}
void intel_execlists_set_default_submission(struct intel_engine_cs *engine)
@ -3447,7 +3492,7 @@ void intel_execlists_set_default_submission(struct intel_engine_cs *engine)
engine->flags |= I915_ENGINE_HAS_PREEMPTION;
}
if (engine->class != COPY_ENGINE_CLASS && INTEL_GEN(engine->i915) >= 12)
if (INTEL_GEN(engine->i915) >= 12)
engine->flags |= I915_ENGINE_HAS_RELATIVE_MMIO;
}
@ -4169,6 +4214,7 @@ intel_execlists_create_virtual(struct i915_gem_context *ctx,
ve->base.i915 = ctx->i915;
ve->base.gt = siblings[0]->gt;
ve->base.uncore = siblings[0]->uncore;
ve->base.id = -1;
ve->base.class = OTHER_CLASS;
ve->base.uabi_class = I915_ENGINE_CLASS_INVALID;

318
drivers/gpu/drm/i915/gt/intel_mocs.c
View File

@ -287,10 +287,9 @@ static const struct drm_i915_mocs_entry icelake_mocs_table[] = {
GEN11_MOCS_ENTRIES
};
static bool get_mocs_settings(struct intel_gt *gt,
static bool get_mocs_settings(const struct drm_i915_private *i915,
struct drm_i915_mocs_table *table)
{
struct drm_i915_private *i915 = gt->i915;
bool result = false;
if (INTEL_GEN(i915) >= 12) {
@ -331,9 +330,9 @@ static bool get_mocs_settings(struct intel_gt *gt,
return result;
}
static i915_reg_t mocs_register(enum intel_engine_id engine_id, int index)
static i915_reg_t mocs_register(const struct intel_engine_cs *engine, int index)
{
switch (engine_id) {
switch (engine->id) {
case RCS0:
return GEN9_GFX_MOCS(index);
case VCS0:
@ -347,7 +346,7 @@ static i915_reg_t mocs_register(enum intel_engine_id engine_id, int index)
case VCS2:
return GEN11_MFX2_MOCS(index);
default:
MISSING_CASE(engine_id);
MISSING_CASE(engine->id);
return INVALID_MMIO_REG;
}
}
@ -365,47 +364,95 @@ static u32 get_entry_control(const struct drm_i915_mocs_table *table,
return table->table[I915_MOCS_PTE].control_value;
}
/**
* intel_mocs_init_engine() - emit the mocs control table
* @engine: The engine for whom to emit the registers.
*
* This function simply emits a MI_LOAD_REGISTER_IMM command for the
* given table starting at the given address.
*/
void intel_mocs_init_engine(struct intel_engine_cs *engine)
static void init_mocs_table(struct intel_engine_cs *engine,
const struct drm_i915_mocs_table *table)
{
struct intel_gt *gt = engine->gt;
struct intel_uncore *uncore = gt->uncore;
struct drm_i915_mocs_table table;
unsigned int index;
u32 unused_value;
/* Platforms with global MOCS do not need per-engine initialization. */
if (HAS_GLOBAL_MOCS_REGISTERS(gt->i915))
return;
/* Called under a blanket forcewake */
assert_forcewakes_active(uncore, FORCEWAKE_ALL);
if (!get_mocs_settings(gt, &table))
return;
/* Set unused values to PTE */
unused_value = table.table[I915_MOCS_PTE].control_value;
for (index = 0; index < table.size; index++) {
u32 value = get_entry_control(&table, index);
struct intel_uncore *uncore = engine->uncore;
u32 unused_value = table->table[I915_MOCS_PTE].control_value;
unsigned int i;
for (i = 0; i < table->size; i++)
intel_uncore_write_fw(uncore,
mocs_register(engine->id, index),
value);
mocs_register(engine, i),
get_entry_control(table, i));
/* All remaining entries are unused */
for (; i < table->n_entries; i++)
intel_uncore_write_fw(uncore,
mocs_register(engine, i),
unused_value);
}
/*
* Get l3cc_value from MOCS entry taking into account when it's not used:
* I915_MOCS_PTE's value is returned in this case.
*/
static u16 get_entry_l3cc(const struct drm_i915_mocs_table *table,
unsigned int index)
{
if (table->table[index].used)
return table->table[index].l3cc_value;
return table->table[I915_MOCS_PTE].l3cc_value;
}
static inline u32 l3cc_combine(const struct drm_i915_mocs_table *table,
u16 low,
u16 high)
{
return low | (u32)high << 16;
}
static void init_l3cc_table(struct intel_engine_cs *engine,
const struct drm_i915_mocs_table *table)
{
struct intel_uncore *uncore = engine->uncore;
u16 unused_value = table->table[I915_MOCS_PTE].l3cc_value;
unsigned int i;
for (i = 0; i < table->size / 2; i++) {
u16 low = get_entry_l3cc(table, 2 * i);
u16 high = get_entry_l3cc(table, 2 * i + 1);
intel_uncore_write(uncore,
GEN9_LNCFCMOCS(i),
l3cc_combine(table, low, high));
}
/* Odd table size - 1 left over */
if (table->size & 1) {
u16 low = get_entry_l3cc(table, 2 * i);
intel_uncore_write(uncore,
GEN9_LNCFCMOCS(i),
l3cc_combine(table, low, unused_value));
i++;
}
/* All remaining entries are also unused */
for (; index < table.n_entries; index++)
intel_uncore_write_fw(uncore,
mocs_register(engine->id, index),
unused_value);
for (; i < table->n_entries / 2; i++)
intel_uncore_write(uncore,
GEN9_LNCFCMOCS(i),
l3cc_combine(table, unused_value,
unused_value));
}
void intel_mocs_init_engine(struct intel_engine_cs *engine)
{
struct drm_i915_mocs_table table;
/* Called under a blanket forcewake */
assert_forcewakes_active(engine->uncore, FORCEWAKE_ALL);
if (!get_mocs_settings(engine->i915, &table))
return;
/* Platforms with global MOCS do not need per-engine initialization. */
if (!HAS_GLOBAL_MOCS_REGISTERS(engine->i915))
init_mocs_table(engine, &table);
if (engine->class == RENDER_CLASS)
init_l3cc_table(engine, &table);
}
static void intel_mocs_init_global(struct intel_gt *gt)
@ -416,7 +463,7 @@ static void intel_mocs_init_global(struct intel_gt *gt)
GEM_BUG_ON(!HAS_GLOBAL_MOCS_REGISTERS(gt->i915));
if (!get_mocs_settings(gt, &table))
if (!get_mocs_settings(gt->i915, &table))
return;
if (GEM_DEBUG_WARN_ON(table.size > table.n_entries))
@ -438,197 +485,8 @@ static void intel_mocs_init_global(struct intel_gt *gt)
table.table[0].control_value);
}
static int emit_mocs_control_table(struct i915_request *rq,
const struct drm_i915_mocs_table *table)
{
enum intel_engine_id engine = rq->engine->id;
unsigned int index;
u32 unused_value;
u32 *cs;
if (GEM_WARN_ON(table->size > table->n_entries))
return -ENODEV;
/* Set unused values to PTE */
unused_value = table->table[I915_MOCS_PTE].control_value;
cs = intel_ring_begin(rq, 2 + 2 * table->n_entries);
if (IS_ERR(cs))
return PTR_ERR(cs);
*cs++ = MI_LOAD_REGISTER_IMM(table->n_entries);
for (index = 0; index < table->size; index++) {
u32 value = get_entry_control(table, index);
*cs++ = i915_mmio_reg_offset(mocs_register(engine, index));
*cs++ = value;
}
/* All remaining entries are also unused */
for (; index < table->n_entries; index++) {
*cs++ = i915_mmio_reg_offset(mocs_register(engine, index));
*cs++ = unused_value;
}
*cs++ = MI_NOOP;
intel_ring_advance(rq, cs);
return 0;
}
/*
* Get l3cc_value from MOCS entry taking into account when it's not used:
* I915_MOCS_PTE's value is returned in this case.
*/
static u16 get_entry_l3cc(const struct drm_i915_mocs_table *table,
unsigned int index)
{
if (table->table[index].used)
return table->table[index].l3cc_value;
return table->table[I915_MOCS_PTE].l3cc_value;
}
static inline u32 l3cc_combine(const struct drm_i915_mocs_table *table,
u16 low,
u16 high)
{
return low | high << 16;
}
static int emit_mocs_l3cc_table(struct i915_request *rq,
const struct drm_i915_mocs_table *table)
{
u16 unused_value;
unsigned int i;
u32 *cs;
if (GEM_WARN_ON(table->size > table->n_entries))
return -ENODEV;
/* Set unused values to PTE */
unused_value = table->table[I915_MOCS_PTE].l3cc_value;
cs = intel_ring_begin(rq, 2 + table->n_entries);
if (IS_ERR(cs))
return PTR_ERR(cs);
*cs++ = MI_LOAD_REGISTER_IMM(table->n_entries / 2);
for (i = 0; i < table->size / 2; i++) {
u16 low = get_entry_l3cc(table, 2 * i);
u16 high = get_entry_l3cc(table, 2 * i + 1);
*cs++ = i915_mmio_reg_offset(GEN9_LNCFCMOCS(i));
*cs++ = l3cc_combine(table, low, high);
}
/* Odd table size - 1 left over */
if (table->size & 0x01) {
u16 low = get_entry_l3cc(table, 2 * i);
*cs++ = i915_mmio_reg_offset(GEN9_LNCFCMOCS(i));
*cs++ = l3cc_combine(table, low, unused_value);
i++;
}
/* All remaining entries are also unused */
for (; i < table->n_entries / 2; i++) {
*cs++ = i915_mmio_reg_offset(GEN9_LNCFCMOCS(i));
*cs++ = l3cc_combine(table, unused_value, unused_value);
}
*cs++ = MI_NOOP;
intel_ring_advance(rq, cs);
return 0;
}
static void intel_mocs_init_l3cc_table(struct intel_gt *gt)
{
struct intel_uncore *uncore = gt->uncore;
struct drm_i915_mocs_table table;
unsigned int i;
u16 unused_value;
if (!get_mocs_settings(gt, &table))
return;
/* Set unused values to PTE */
unused_value = table.table[I915_MOCS_PTE].l3cc_value;
for (i = 0; i < table.size / 2; i++) {
u16 low = get_entry_l3cc(&table, 2 * i);
u16 high = get_entry_l3cc(&table, 2 * i + 1);
intel_uncore_write(uncore,
GEN9_LNCFCMOCS(i),
l3cc_combine(&table, low, high));
}
/* Odd table size - 1 left over */
if (table.size & 0x01) {
u16 low = get_entry_l3cc(&table, 2 * i);
intel_uncore_write(uncore,
GEN9_LNCFCMOCS(i),
l3cc_combine(&table, low, unused_value));
i++;
}
/* All remaining entries are also unused */
for (; i < table.n_entries / 2; i++)
intel_uncore_write(uncore,
GEN9_LNCFCMOCS(i),
l3cc_combine(&table, unused_value,
unused_value));
}
/**
* intel_mocs_emit() - program the MOCS register.
* @rq: Request to use to set up the MOCS tables.
*
* This function will emit a batch buffer with the values required for
* programming the MOCS register values for all the currently supported
* rings.
*
* These registers are partially stored in the RCS context, so they are
* emitted at the same time so that when a context is created these registers
* are set up. These registers have to be emitted into the start of the
* context as setting the ELSP will re-init some of these registers back
* to the hw values.
*
* Return: 0 on success, otherwise the error status.
*/
int intel_mocs_emit(struct i915_request *rq)
{
struct drm_i915_mocs_table t;
int ret;
if (HAS_GLOBAL_MOCS_REGISTERS(rq->i915) ||
rq->engine->class != RENDER_CLASS)
return 0;
if (get_mocs_settings(rq->engine->gt, &t)) {
/* Program the RCS control registers */
ret = emit_mocs_control_table(rq, &t);
if (ret)
return ret;
/* Now program the l3cc registers */
ret = emit_mocs_l3cc_table(rq, &t);
if (ret)
return ret;
}
return 0;
}
void intel_mocs_init(struct intel_gt *gt)
{
intel_mocs_init_l3cc_table(gt);
if (HAS_GLOBAL_MOCS_REGISTERS(gt->i915))
intel_mocs_init_global(gt);
}

3
drivers/gpu/drm/i915/gt/intel_mocs.h
View File

@ -49,13 +49,10 @@
* context handling keep the MOCS in step.
*/
struct i915_request;
struct intel_engine_cs;
struct intel_gt;
void intel_mocs_init(struct intel_gt *gt);
void intel_mocs_init_engine(struct intel_engine_cs *engine);
int intel_mocs_emit(struct i915_request *rq);
#endif

12
drivers/gpu/drm/i915/gt/intel_rc6.c
View File

@ -65,7 +65,7 @@ static void gen11_rc6_enable(struct intel_rc6 *rc6)
set(uncore, GEN6_RC_EVALUATION_INTERVAL, 125000); /* 12500 * 1280ns */
set(uncore, GEN6_RC_IDLE_HYSTERSIS, 25); /* 25 * 1280ns */
for_each_engine(engine, rc6_to_gt(rc6)->i915, id)
for_each_engine(engine, rc6_to_gt(rc6), id)
set(uncore, RING_MAX_IDLE(engine->mmio_base), 10);
set(uncore, GUC_MAX_IDLE_COUNT, 0xA);
@ -133,7 +133,7 @@ static void gen9_rc6_enable(struct intel_rc6 *rc6)
set(uncore, GEN6_RC_EVALUATION_INTERVAL, 125000); /* 12500 * 1280ns */
set(uncore, GEN6_RC_IDLE_HYSTERSIS, 25); /* 25 * 1280ns */
for_each_engine(engine, rc6_to_gt(rc6)->i915, id)
for_each_engine(engine, rc6_to_gt(rc6), id)
set(uncore, RING_MAX_IDLE(engine->mmio_base), 10);
set(uncore, GUC_MAX_IDLE_COUNT, 0xA);
@ -192,7 +192,7 @@ static void gen8_rc6_enable(struct intel_rc6 *rc6)
set(uncore, GEN6_RC6_WAKE_RATE_LIMIT, 40 << 16);
set(uncore, GEN6_RC_EVALUATION_INTERVAL, 125000); /* 12500 * 1280ns */
set(uncore, GEN6_RC_IDLE_HYSTERSIS, 25); /* 25 * 1280ns */
for_each_engine(engine, rc6_to_gt(rc6)->i915, id)
for_each_engine(engine, rc6_to_gt(rc6), id)
set(uncore, RING_MAX_IDLE(engine->mmio_base), 10);
set(uncore, GEN6_RC_SLEEP, 0);
set(uncore, GEN6_RC6_THRESHOLD, 625); /* 800us/1.28 for TO */
@ -219,7 +219,7 @@ static void gen6_rc6_enable(struct intel_rc6 *rc6)
set(uncore, GEN6_RC_EVALUATION_INTERVAL, 125000);
set(uncore, GEN6_RC_IDLE_HYSTERSIS, 25);
for_each_engine(engine, i915, id)
for_each_engine(engine, rc6_to_gt(rc6), id)
set(uncore, RING_MAX_IDLE(engine->mmio_base), 10);
set(uncore, GEN6_RC_SLEEP, 0);
@ -344,7 +344,7 @@ static void chv_rc6_enable(struct intel_rc6 *rc6)
set(uncore, GEN6_RC_EVALUATION_INTERVAL, 125000); /* 12500 * 1280ns */
set(uncore, GEN6_RC_IDLE_HYSTERSIS, 25); /* 25 * 1280ns */
for_each_engine(engine, rc6_to_gt(rc6)->i915, id)
for_each_engine(engine, rc6_to_gt(rc6), id)
set(uncore, RING_MAX_IDLE(engine->mmio_base), 10);
set(uncore, GEN6_RC_SLEEP, 0);
@ -371,7 +371,7 @@ static void vlv_rc6_enable(struct intel_rc6 *rc6)
set(uncore, GEN6_RC_EVALUATION_INTERVAL, 125000);
set(uncore, GEN6_RC_IDLE_HYSTERSIS, 25);
for_each_engine(engine, rc6_to_gt(rc6)->i915, id)
for_each_engine(engine, rc6_to_gt(rc6), id)
set(uncore, RING_MAX_IDLE(engine->mmio_base), 10);
set(uncore, GEN6_RC6_THRESHOLD, 0x557);

52
drivers/gpu/drm/i915/gt/intel_reset.c
View File

@ -282,14 +282,14 @@ static int gen6_reset_engines(struct intel_gt *gt,
intel_engine_mask_t engine_mask,
unsigned int retry)
{
struct intel_engine_cs *engine;
const u32 hw_engine_mask[] = {
static const u32 hw_engine_mask[] = {
[RCS0] = GEN6_GRDOM_RENDER,
[BCS0] = GEN6_GRDOM_BLT,
[VCS0] = GEN6_GRDOM_MEDIA,
[VCS1] = GEN8_GRDOM_MEDIA2,
[VECS0] = GEN6_GRDOM_VECS,
};
struct intel_engine_cs *engine;
u32 hw_mask;
if (engine_mask == ALL_ENGINES) {
@ -298,7 +298,7 @@ static int gen6_reset_engines(struct intel_gt *gt,
intel_engine_mask_t tmp;
hw_mask = 0;
for_each_engine_masked(engine, gt->i915, engine_mask, tmp) {
for_each_engine_masked(engine, gt, engine_mask, tmp) {
GEM_BUG_ON(engine->id >= ARRAY_SIZE(hw_engine_mask));
hw_mask |= hw_engine_mask[engine->id];
}
@ -413,7 +413,7 @@ static int gen11_reset_engines(struct intel_gt *gt,
intel_engine_mask_t engine_mask,
unsigned int retry)
{
const u32 hw_engine_mask[] = {
static const u32 hw_engine_mask[] = {
[RCS0] = GEN11_GRDOM_RENDER,
[BCS0] = GEN11_GRDOM_BLT,
[VCS0] = GEN11_GRDOM_MEDIA,
@ -432,7 +432,7 @@ static int gen11_reset_engines(struct intel_gt *gt,
hw_mask = GEN11_GRDOM_FULL;
} else {
hw_mask = 0;
for_each_engine_masked(engine, gt->i915, engine_mask, tmp) {
for_each_engine_masked(engine, gt, engine_mask, tmp) {
GEM_BUG_ON(engine->id >= ARRAY_SIZE(hw_engine_mask));
hw_mask |= hw_engine_mask[engine->id];
ret = gen11_lock_sfc(engine, &hw_mask);
@ -451,7 +451,7 @@ static int gen11_reset_engines(struct intel_gt *gt,
* expiration).
*/
if (engine_mask != ALL_ENGINES)
for_each_engine_masked(engine, gt->i915, engine_mask, tmp)
for_each_engine_masked(engine, gt, engine_mask, tmp)
gen11_unlock_sfc(engine);
return ret;
@ -510,7 +510,7 @@ static int gen8_reset_engines(struct intel_gt *gt,
intel_engine_mask_t tmp;
int ret;
for_each_engine_masked(engine, gt->i915, engine_mask, tmp) {
for_each_engine_masked(engine, gt, engine_mask, tmp) {
ret = gen8_engine_reset_prepare(engine);
if (ret && !reset_non_ready)
goto skip_reset;
@ -536,7 +536,7 @@ static int gen8_reset_engines(struct intel_gt *gt,
ret = gen6_reset_engines(gt, engine_mask, retry);
skip_reset:
for_each_engine_masked(engine, gt->i915, engine_mask, tmp)
for_each_engine_masked(engine, gt, engine_mask, tmp)
gen8_engine_reset_cancel(engine);
return ret;
@ -682,7 +682,7 @@ static intel_engine_mask_t reset_prepare(struct intel_gt *gt)
intel_engine_mask_t awake = 0;
enum intel_engine_id id;
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
if (intel_engine_pm_get_if_awake(engine))
awake |= engine->mask;
reset_prepare_engine(engine);
@ -712,10 +712,10 @@ static int gt_reset(struct intel_gt *gt, intel_engine_mask_t stalled_mask)
if (err)
return err;
for_each_engine(engine, gt->i915, id)
for_each_engine(engine, gt, id)
__intel_engine_reset(engine, stalled_mask & engine->mask);
i915_gem_restore_fences(gt->i915);
i915_gem_restore_fences(gt->ggtt);
return err;
}
@ -733,7 +733,7 @@ static void reset_finish(struct intel_gt *gt, intel_engine_mask_t awake)
struct intel_engine_cs *engine;
enum intel_engine_id id;
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
reset_finish_engine(engine);
if (awake & engine->mask)
intel_engine_pm_put(engine);
@ -769,7 +769,7 @@ static void __intel_gt_set_wedged(struct intel_gt *gt)
if (GEM_SHOW_DEBUG() && !intel_engines_are_idle(gt)) {
struct drm_printer p = drm_debug_printer(__func__);
for_each_engine(engine, gt->i915, id)
for_each_engine(engine, gt, id)
intel_engine_dump(engine, &p, "%s\n", engine->name);
}
@ -786,7 +786,7 @@ static void __intel_gt_set_wedged(struct intel_gt *gt)
if (!INTEL_INFO(gt->i915)->gpu_reset_clobbers_display)
__intel_gt_reset(gt, ALL_ENGINES);
for_each_engine(engine, gt->i915, id)
for_each_engine(engine, gt, id)
engine->submit_request = nop_submit_request;
/*
@ -798,7 +798,7 @@ static void __intel_gt_set_wedged(struct intel_gt *gt)
set_bit(I915_WEDGED, &gt->reset.flags);
/* Mark all executing requests as skipped */
for_each_engine(engine, gt->i915, id)
for_each_engine(engine, gt, id)
engine->cancel_requests(engine);
reset_finish(gt, awake);
@ -811,7 +811,7 @@ void intel_gt_set_wedged(struct intel_gt *gt)
intel_wakeref_t wakeref;
mutex_lock(&gt->reset.mutex);
with_intel_runtime_pm(&gt->i915->runtime_pm, wakeref)
with_intel_runtime_pm(gt->uncore->rpm, wakeref)
__intel_gt_set_wedged(gt);
mutex_unlock(&gt->reset.mutex);
}
@ -872,8 +872,14 @@ static bool __intel_gt_unset_wedged(struct intel_gt *gt)
ok = !HAS_EXECLISTS(gt->i915); /* XXX better agnosticism desired */
if (!INTEL_INFO(gt->i915)->gpu_reset_clobbers_display)
ok = __intel_gt_reset(gt, ALL_ENGINES) == 0;
if (!ok)
if (!ok) {
/*
* Warn CI about the unrecoverable wedged condition.
* Time for a reboot.
*/
add_taint_for_CI(TAINT_WARN);
return false;
}
/*
* Undo nop_submit_request. We prevent all new i915 requests from
@ -928,7 +934,7 @@ static int resume(struct intel_gt *gt)
enum intel_engine_id id;
int ret;
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
ret = engine->resume(engine);
if (ret)
return ret;
@ -1186,7 +1192,7 @@ void intel_gt_handle_error(struct intel_gt *gt,
* isn't the case at least when we get here by doing a
* simulated reset via debugfs, so get an RPM reference.
*/
wakeref = intel_runtime_pm_get(&gt->i915->runtime_pm);
wakeref = intel_runtime_pm_get(gt->uncore->rpm);
engine_mask &= INTEL_INFO(gt->i915)->engine_mask;
@ -1200,7 +1206,7 @@ void intel_gt_handle_error(struct intel_gt *gt,
* single reset fails.
*/
if (intel_has_reset_engine(gt) && !intel_gt_is_wedged(gt)) {
for_each_engine_masked(engine, gt->i915, engine_mask, tmp) {
for_each_engine_masked(engine, gt, engine_mask, tmp) {
BUILD_BUG_ON(I915_RESET_MODESET >= I915_RESET_ENGINE);
if (test_and_set_bit(I915_RESET_ENGINE + engine->id,
&gt->reset.flags))
@ -1228,7 +1234,7 @@ void intel_gt_handle_error(struct intel_gt *gt,
synchronize_rcu_expedited();
/* Prevent any other reset-engine attempt. */
for_each_engine(engine, gt->i915, tmp) {
for_each_engine(engine, gt, tmp) {
while (test_and_set_bit(I915_RESET_ENGINE + engine->id,
&gt->reset.flags))
wait_on_bit(&gt->reset.flags,
@ -1238,7 +1244,7 @@ void intel_gt_handle_error(struct intel_gt *gt,
intel_gt_reset_global(gt, engine_mask, msg);
for_each_engine(engine, gt->i915, tmp)
for_each_engine(engine, gt, tmp)
clear_bit_unlock(I915_RESET_ENGINE + engine->id,
&gt->reset.flags);
clear_bit_unlock(I915_RESET_BACKOFF, &gt->reset.flags);
@ -1246,7 +1252,7 @@ void intel_gt_handle_error(struct intel_gt *gt,
wake_up_all(&gt->reset.queue);
out:
intel_runtime_pm_put(&gt->i915->runtime_pm, wakeref);
intel_runtime_pm_put(gt->uncore->rpm, wakeref);
}
int intel_gt_reset_trylock(struct intel_gt *gt, int *srcu)

6
drivers/gpu/drm/i915/gt/intel_ringbuffer.c
View File

@ -1609,7 +1609,7 @@ static inline int mi_set_context(struct i915_request *rq, u32 flags)
struct intel_engine_cs *signaller;
*cs++ = MI_LOAD_REGISTER_IMM(num_engines);
for_each_engine(signaller, i915, id) {
for_each_engine(signaller, engine->gt, id) {
if (signaller == engine)
continue;
@ -1663,7 +1663,7 @@ static inline int mi_set_context(struct i915_request *rq, u32 flags)
i915_reg_t last_reg = {}; /* keep gcc quiet */
*cs++ = MI_LOAD_REGISTER_IMM(num_engines);
for_each_engine(signaller, i915, id) {
for_each_engine(signaller, engine->gt, id) {
if (signaller == engine)
continue;
@ -1676,7 +1676,7 @@ static inline int mi_set_context(struct i915_request *rq, u32 flags)
/* Insert a delay before the next switch! */
*cs++ = MI_STORE_REGISTER_MEM | MI_SRM_LRM_GLOBAL_GTT;
*cs++ = i915_mmio_reg_offset(last_reg);
*cs++ = intel_gt_scratch_offset(rq->engine->gt,
*cs++ = intel_gt_scratch_offset(engine->gt,
INTEL_GT_SCRATCH_FIELD_DEFAULT);
*cs++ = MI_NOOP;
}

38
drivers/gpu/drm/i915/gt/intel_workarounds.c
View File

@ -567,7 +567,7 @@ static void icl_ctx_workarounds_init(struct intel_engine_cs *engine,
static void tgl_ctx_workarounds_init(struct intel_engine_cs *engine,
struct i915_wa_list *wal)
{
/* Wa_1409142259 */
/* Wa_1409142259:tgl */
WA_SET_BIT_MASKED(GEN11_COMMON_SLICE_CHICKEN3,
GEN12_DISABLE_CPS_AWARE_COLOR_PIPE);
}
@ -892,11 +892,27 @@ icl_gt_workarounds_init(struct drm_i915_private *i915, struct i915_wa_list *wal)
wa_write_or(wal,
GAMT_CHKN_BIT_REG,
GAMT_CHKN_DISABLE_L3_COH_PIPE);
/* Wa_1607087056:icl */
wa_write_or(wal,
SLICE_UNIT_LEVEL_CLKGATE,
L3_CLKGATE_DIS | L3_CR2X_CLKGATE_DIS);
}
static void
tgl_gt_workarounds_init(struct drm_i915_private *i915, struct i915_wa_list *wal)
{
/* Wa_1409420604:tgl */
if (IS_TGL_REVID(i915, TGL_REVID_A0, TGL_REVID_A0))
wa_write_or(wal,
SUBSLICE_UNIT_LEVEL_CLKGATE2,
CPSSUNIT_CLKGATE_DIS);
/* Wa_1409180338:tgl */
if (IS_TGL_REVID(i915, TGL_REVID_A0, TGL_REVID_A0))
wa_write_or(wal,
SLICE_UNIT_LEVEL_CLKGATE,
L3_CLKGATE_DIS | L3_CR2X_CLKGATE_DIS);
}
static void
@ -1260,6 +1276,26 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
{
struct drm_i915_private *i915 = engine->i915;
if (IS_TGL_REVID(i915, TGL_REVID_A0, TGL_REVID_A0)) {
/* Wa_1606700617:tgl */
wa_masked_en(wal,
GEN9_CS_DEBUG_MODE1,
FF_DOP_CLOCK_GATE_DISABLE);
/* Wa_1607138336:tgl */
wa_write_or(wal,
GEN9_CTX_PREEMPT_REG,
GEN12_DISABLE_POSH_BUSY_FF_DOP_CG);
/* Wa_1607030317:tgl */
/* Wa_1607186500:tgl */
/* Wa_1607297627:tgl */
wa_masked_en(wal,
GEN6_RC_SLEEP_PSMI_CONTROL,
GEN12_WAIT_FOR_EVENT_POWER_DOWN_DISABLE |
GEN8_RC_SEMA_IDLE_MSG_DISABLE);
}
if (IS_GEN(i915, 11)) {
/* This is not an Wa. Enable for better image quality */
wa_masked_en(wal,

6
drivers/gpu/drm/i915/gt/mock_engine.c
View File

@ -240,6 +240,7 @@ struct intel_engine_cs *mock_engine(struct drm_i915_private *i915,
struct mock_engine *engine;
GEM_BUG_ON(id >= I915_NUM_ENGINES);
GEM_BUG_ON(!i915->gt.uncore);
engine = kzalloc(sizeof(*engine) + PAGE_SIZE, GFP_KERNEL);
if (!engine)
@ -248,9 +249,11 @@ struct intel_engine_cs *mock_engine(struct drm_i915_private *i915,
/* minimal engine setup for requests */
engine->base.i915 = i915;
engine->base.gt = &i915->gt;
engine->base.uncore = i915->gt.uncore;
snprintf(engine->base.name, sizeof(engine->base.name), "%s", name);
engine->base.id = id;
engine->base.mask = BIT(id);
engine->base.legacy_idx = INVALID_ENGINE;
engine->base.instance = id;
engine->base.status_page.addr = (void *)(engine + 1);
@ -265,6 +268,9 @@ struct intel_engine_cs *mock_engine(struct drm_i915_private *i915,
engine->base.reset.finish = mock_reset_finish;
engine->base.cancel_requests = mock_cancel_requests;
i915->gt.engine[id] = &engine->base;
i915->gt.engine_class[0][id] = &engine->base;
/* fake hw queue */
spin_lock_init(&engine->hw_lock);
timer_setup(&engine->hw_delay, hw_delay_complete, 0);

6
drivers/gpu/drm/i915/gt/selftest_context.c
View File

@ -159,7 +159,7 @@ static int live_context_size(void *arg)
if (IS_ERR(fixme))
return PTR_ERR(fixme);
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
struct {
struct drm_i915_gem_object *state;
void *pinned;
@ -305,7 +305,7 @@ static int live_active_context(void *arg)
goto out_file;
}
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
err = __live_active_context(engine, fixme);
if (err)
break;
@ -415,7 +415,7 @@ static int live_remote_context(void *arg)
goto out_file;
}
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
err = __live_remote_context(engine, fixme);
if (err)
break;

2
drivers/gpu/drm/i915/gt/selftest_engine_pm.c
View File

@ -25,7 +25,7 @@ static int live_engine_pm(void *arg)
}
GEM_BUG_ON(intel_gt_pm_is_awake(gt));
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
const typeof(*igt_atomic_phases) *p;
for (p = igt_atomic_phases; p->name; p++) {

9
drivers/gpu/drm/i915/gt/selftest_gt_pm.c
View File

@ -5,6 +5,8 @@
* Copyright © 2019 Intel Corporation
*/
#include "selftest_llc.h"
static int live_gt_resume(void *arg)
{
struct intel_gt *gt = arg;
@ -32,6 +34,13 @@ static int live_gt_resume(void *arg)
err = -EINVAL;
break;
}
err = st_llc_verify(&gt->llc);
if (err) {
pr_err("llc state not restored upon resume!\n");
intel_gt_set_wedged_on_init(gt);
break;
}
} while (!__igt_timeout(end_time, NULL));
return err;

24
drivers/gpu/drm/i915/gt/selftest_hangcheck.c
View File

@ -323,7 +323,7 @@ static int igt_hang_sanitycheck(void *arg)
if (err)
return err;
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
struct intel_wedge_me w;
long timeout;
@ -400,7 +400,7 @@ static int igt_reset_nop(void *arg)
reset_count = i915_reset_count(global);
count = 0;
do {
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
int i;
for (i = 0; i < 16; i++) {
@ -471,7 +471,7 @@ static int igt_reset_nop_engine(void *arg)
}
i915_gem_context_clear_bannable(ctx);
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
unsigned int reset_count, reset_engine_count;
unsigned int count;
IGT_TIMEOUT(end_time);
@ -560,7 +560,7 @@ static int __igt_reset_engine(struct intel_gt *gt, bool active)
return err;
}
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
unsigned int reset_count, reset_engine_count;
IGT_TIMEOUT(end_time);
@ -782,7 +782,7 @@ static int __igt_reset_engines(struct intel_gt *gt,
h.ctx->sched.priority = 1024;
}
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
struct active_engine threads[I915_NUM_ENGINES] = {};
unsigned long device = i915_reset_count(global);
unsigned long count = 0, reported;
@ -800,7 +800,7 @@ static int __igt_reset_engines(struct intel_gt *gt,
}
memset(threads, 0, sizeof(threads));
for_each_engine(other, gt->i915, tmp) {
for_each_engine(other, gt, tmp) {
struct task_struct *tsk;
threads[tmp].resets =
@ -914,7 +914,7 @@ static int __igt_reset_engines(struct intel_gt *gt,
}
unwind:
for_each_engine(other, gt->i915, tmp) {
for_each_engine(other, gt, tmp) {
int ret;
if (!threads[tmp].task)
@ -1335,7 +1335,7 @@ static int wait_for_others(struct intel_gt *gt,
struct intel_engine_cs *engine;
enum intel_engine_id id;
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
if (engine == exclude)
continue;
@ -1363,7 +1363,7 @@ static int igt_reset_queue(void *arg)
if (err)
goto unlock;
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
struct i915_request *prev;
IGT_TIMEOUT(end_time);
unsigned int count;
@ -1651,7 +1651,7 @@ static int igt_reset_engines_atomic(void *arg)
struct intel_engine_cs *engine;
enum intel_engine_id id;
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
err = igt_atomic_reset_engine(engine, p);
if (err)
goto out;
@ -1695,14 +1695,14 @@ int intel_hangcheck_live_selftests(struct drm_i915_private *i915)
if (intel_gt_is_wedged(gt))
return -EIO; /* we're long past hope of a successful reset */
wakeref = intel_runtime_pm_get(&gt->i915->runtime_pm);
wakeref = intel_runtime_pm_get(gt->uncore->rpm);
saved_hangcheck = fetch_and_zero(&i915_modparams.enable_hangcheck);
drain_delayed_work(&gt->hangcheck.work); /* flush param */
err = intel_gt_live_subtests(tests, gt);
i915_modparams.enable_hangcheck = saved_hangcheck;
intel_runtime_pm_put(&gt->i915->runtime_pm, wakeref);
intel_runtime_pm_put(gt->uncore->rpm, wakeref);
return err;
}

77
drivers/gpu/drm/i915/gt/selftest_llc.c Normal file
View File

@ -0,0 +1,77 @@
/*
* SPDX-License-Identifier: MIT
*
* Copyright © 2019 Intel Corporation
*/
#include "intel_pm.h" /* intel_gpu_freq() */
#include "selftest_llc.h"
static int gen6_verify_ring_freq(struct intel_llc *llc)
{
struct drm_i915_private *i915 = llc_to_gt(llc)->i915;
struct ia_constants consts;
intel_wakeref_t wakeref;
unsigned int gpu_freq;
int err = 0;
wakeref = intel_runtime_pm_get(llc_to_gt(llc)->uncore->rpm);
if (!get_ia_constants(llc, &consts)) {
err = -ENODEV;
goto out_rpm;
}
for (gpu_freq = consts.min_gpu_freq;
gpu_freq <= consts.max_gpu_freq;
gpu_freq++) {
unsigned int ia_freq, ring_freq, found;
u32 val;
calc_ia_freq(llc, gpu_freq, &consts, &ia_freq, &ring_freq);
val = gpu_freq;
if (sandybridge_pcode_read(i915,
GEN6_PCODE_READ_MIN_FREQ_TABLE,
&val, NULL)) {
pr_err("Failed to read freq table[%d], range [%d, %d]\n",
gpu_freq, consts.min_gpu_freq, consts.max_gpu_freq);
err = -ENXIO;
break;
}
found = (val >> 0) & 0xff;
if (found != ia_freq) {
pr_err("Min freq table(%d/[%d, %d]):%dMHz did not match expected CPU freq, found %d, expected %d\n",
gpu_freq, consts.min_gpu_freq, consts.max_gpu_freq,
intel_gpu_freq(i915, gpu_freq * (INTEL_GEN(i915) >= 9 ? GEN9_FREQ_SCALER : 1)),
found, ia_freq);
err = -EINVAL;
break;
}
found = (val >> 8) & 0xff;
if (found != ring_freq) {
pr_err("Min freq table(%d/[%d, %d]):%dMHz did not match expected ring freq, found %d, expected %d\n",
gpu_freq, consts.min_gpu_freq, consts.max_gpu_freq,
intel_gpu_freq(i915, gpu_freq * (INTEL_GEN(i915) >= 9 ? GEN9_FREQ_SCALER : 1)),
found, ring_freq);
err = -EINVAL;
break;
}
}
out_rpm:
intel_runtime_pm_put(llc_to_gt(llc)->uncore->rpm, wakeref);
return err;
}
int st_llc_verify(struct intel_llc *llc)
{
int err = 0;
if (HAS_LLC(llc_to_gt(llc)->i915))
err = gen6_verify_ring_freq(llc);
return err;
}

14
drivers/gpu/drm/i915/gt/selftest_llc.h Normal file
View File

@ -0,0 +1,14 @@
/*
* SPDX-License-Identifier: MIT
*
* Copyright © 2019 Intel Corporation
*/
#ifndef SELFTEST_LLC_H
#define SELFTEST_LLC_H
struct intel_llc;
int st_llc_verify(struct intel_llc *llc);
#endif /* SELFTEST_LLC_H */

912
drivers/gpu/drm/i915/gt/selftest_lrc.c
View File

File diff suppressed because it is too large Load Diff

10
drivers/gpu/drm/i915/gt/selftest_reset.c
View File

@ -17,7 +17,7 @@ static int igt_global_reset(void *arg)
/* Check that we can issue a global GPU reset */
igt_global_reset_lock(gt);
wakeref = intel_runtime_pm_get(&gt->i915->runtime_pm);
wakeref = intel_runtime_pm_get(gt->uncore->rpm);
reset_count = i915_reset_count(&gt->i915->gpu_error);
@ -28,7 +28,7 @@ static int igt_global_reset(void *arg)
err = -EINVAL;
}
intel_runtime_pm_put(&gt->i915->runtime_pm, wakeref);
intel_runtime_pm_put(gt->uncore->rpm, wakeref);
igt_global_reset_unlock(gt);
if (intel_gt_is_wedged(gt))
@ -45,14 +45,14 @@ static int igt_wedged_reset(void *arg)
/* Check that we can recover a wedged device with a GPU reset */
igt_global_reset_lock(gt);
wakeref = intel_runtime_pm_get(&gt->i915->runtime_pm);
wakeref = intel_runtime_pm_get(gt->uncore->rpm);
intel_gt_set_wedged(gt);
GEM_BUG_ON(!intel_gt_is_wedged(gt));
intel_gt_reset(gt, ALL_ENGINES, NULL);
intel_runtime_pm_put(&gt->i915->runtime_pm, wakeref);
intel_runtime_pm_put(gt->uncore->rpm, wakeref);
igt_global_reset_unlock(gt);
return intel_gt_is_wedged(gt) ? -EIO : 0;
@ -125,7 +125,7 @@ static int igt_atomic_engine_reset(void *arg)
if (!igt_force_reset(gt))
goto out_unlock;
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
tasklet_disable_nosync(&engine->execlists.tasklet);
intel_engine_pm_get(engine);

50
drivers/gpu/drm/i915/gt/selftest_timeline.c
View File

@ -35,7 +35,7 @@ static unsigned long hwsp_cacheline(struct intel_timeline *tl)
#define CACHELINES_PER_PAGE (PAGE_SIZE / CACHELINE_BYTES)
struct mock_hwsp_freelist {
struct drm_i915_private *i915;
struct intel_gt *gt;
struct radix_tree_root cachelines;
struct intel_timeline **history;
unsigned long count, max;
@ -68,7 +68,7 @@ static int __mock_hwsp_timeline(struct mock_hwsp_freelist *state,
unsigned long cacheline;
int err;
tl = intel_timeline_create(&state->i915->gt, NULL);
tl = intel_timeline_create(state->gt, NULL);
if (IS_ERR(tl))
return PTR_ERR(tl);
@ -106,6 +106,7 @@ static int __mock_hwsp_timeline(struct mock_hwsp_freelist *state,
static int mock_hwsp_freelist(void *arg)
{
struct mock_hwsp_freelist state;
struct drm_i915_private *i915;
const struct {
const char *name;
unsigned int flags;
@ -117,12 +118,14 @@ static int mock_hwsp_freelist(void *arg)
unsigned int na;
int err = 0;
i915 = mock_gem_device();
if (!i915)
return -ENOMEM;
INIT_RADIX_TREE(&state.cachelines, GFP_KERNEL);
state.prng = I915_RND_STATE_INITIALIZER(i915_selftest.random_seed);
state.i915 = mock_gem_device();
if (!state.i915)
return -ENOMEM;
state.gt = &i915->gt;
/*
* Create a bunch of timelines and check that their HWSP do not overlap.
@ -151,7 +154,7 @@ static int mock_hwsp_freelist(void *arg)
__mock_hwsp_record(&state, na, NULL);
kfree(state.history);
err_put:
drm_dev_put(&state.i915->drm);
drm_dev_put(&i915->drm);
return err;
}
@ -476,11 +479,11 @@ tl_write(struct intel_timeline *tl, struct intel_engine_cs *engine, u32 value)
}
static struct intel_timeline *
checked_intel_timeline_create(struct drm_i915_private *i915)
checked_intel_timeline_create(struct intel_gt *gt)
{
struct intel_timeline *tl;
tl = intel_timeline_create(&i915->gt, NULL);
tl = intel_timeline_create(gt, NULL);
if (IS_ERR(tl))
return tl;
@ -497,7 +500,7 @@ checked_intel_timeline_create(struct drm_i915_private *i915)
static int live_hwsp_engine(void *arg)
{
#define NUM_TIMELINES 4096
struct drm_i915_private *i915 = arg;
struct intel_gt *gt = arg;
struct intel_timeline **timelines;
struct intel_engine_cs *engine;
enum intel_engine_id id;
@ -516,7 +519,7 @@ static int live_hwsp_engine(void *arg)
return -ENOMEM;
count = 0;
for_each_engine(engine, i915, id) {
for_each_engine(engine, gt, id) {
if (!intel_engine_can_store_dword(engine))
continue;
@ -526,7 +529,7 @@ static int live_hwsp_engine(void *arg)
struct intel_timeline *tl;
struct i915_request *rq;
tl = checked_intel_timeline_create(i915);
tl = checked_intel_timeline_create(gt);
if (IS_ERR(tl)) {
err = PTR_ERR(tl);
break;
@ -548,7 +551,7 @@ static int live_hwsp_engine(void *arg)
break;
}
if (igt_flush_test(i915))
if (igt_flush_test(gt->i915))
err = -EIO;
for (n = 0; n < count; n++) {
@ -570,7 +573,7 @@ static int live_hwsp_engine(void *arg)
static int live_hwsp_alternate(void *arg)
{
#define NUM_TIMELINES 4096
struct drm_i915_private *i915 = arg;
struct intel_gt *gt = arg;
struct intel_timeline **timelines;
struct intel_engine_cs *engine;
enum intel_engine_id id;
@ -591,14 +594,14 @@ static int live_hwsp_alternate(void *arg)
count = 0;
for (n = 0; n < NUM_TIMELINES; n++) {
for_each_engine(engine, i915, id) {
for_each_engine(engine, gt, id) {
struct intel_timeline *tl;
struct i915_request *rq;
if (!intel_engine_can_store_dword(engine))
continue;
tl = checked_intel_timeline_create(i915);
tl = checked_intel_timeline_create(gt);
if (IS_ERR(tl)) {
intel_engine_pm_put(engine);
err = PTR_ERR(tl);
@ -620,7 +623,7 @@ static int live_hwsp_alternate(void *arg)
}
out:
if (igt_flush_test(i915))
if (igt_flush_test(gt->i915))
err = -EIO;
for (n = 0; n < count; n++) {
@ -641,8 +644,7 @@ static int live_hwsp_alternate(void *arg)
static int live_hwsp_wrap(void *arg)
{
struct drm_i915_private *i915 = arg;
struct intel_gt *gt = &i915->gt;
struct intel_gt *gt = arg;
struct intel_engine_cs *engine;
struct intel_timeline *tl;
enum intel_engine_id id;
@ -664,7 +666,7 @@ static int live_hwsp_wrap(void *arg)
if (err)
goto out_free;
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
const u32 *hwsp_seqno[2];
struct i915_request *rq;
u32 seqno[2];
@ -740,7 +742,7 @@ static int live_hwsp_wrap(void *arg)
}
out:
if (igt_flush_test(i915))
if (igt_flush_test(gt->i915))
err = -EIO;
intel_timeline_unpin(tl);
@ -751,7 +753,7 @@ static int live_hwsp_wrap(void *arg)
static int live_hwsp_recycle(void *arg)
{
struct drm_i915_private *i915 = arg;
struct intel_gt *gt = arg;
struct intel_engine_cs *engine;
enum intel_engine_id id;
unsigned long count;
@ -764,7 +766,7 @@ static int live_hwsp_recycle(void *arg)
*/
count = 0;
for_each_engine(engine, i915, id) {
for_each_engine(engine, gt, id) {
IGT_TIMEOUT(end_time);
if (!intel_engine_can_store_dword(engine))
@ -776,7 +778,7 @@ static int live_hwsp_recycle(void *arg)
struct intel_timeline *tl;
struct i915_request *rq;
tl = checked_intel_timeline_create(i915);
tl = checked_intel_timeline_create(gt);
if (IS_ERR(tl)) {
err = PTR_ERR(tl);
break;
@ -831,5 +833,5 @@ int intel_timeline_live_selftests(struct drm_i915_private *i915)
if (intel_gt_is_wedged(&i915->gt))
return 0;
return i915_live_subtests(tests, i915);
return intel_gt_live_subtests(tests, &i915->gt);
}

211
drivers/gpu/drm/i915/gt/selftest_workarounds.c
View File

@ -33,8 +33,32 @@ struct wa_lists {
} engine[I915_NUM_ENGINES];
};
static int request_add_sync(struct i915_request *rq, int err)
{
i915_request_get(rq);
i915_request_add(rq);
if (i915_request_wait(rq, 0, HZ / 5) < 0)
err = -EIO;
i915_request_put(rq);
return err;
}
static int request_add_spin(struct i915_request *rq, struct igt_spinner *spin)
{
int err = 0;
i915_request_get(rq);
i915_request_add(rq);
if (spin && !igt_wait_for_spinner(spin, rq))
err = -ETIMEDOUT;
i915_request_put(rq);
return err;
}
static void
reference_lists_init(struct drm_i915_private *i915, struct wa_lists *lists)
reference_lists_init(struct intel_gt *gt, struct wa_lists *lists)
{
struct intel_engine_cs *engine;
enum intel_engine_id id;
@ -42,10 +66,10 @@ reference_lists_init(struct drm_i915_private *i915, struct wa_lists *lists)
memset(lists, 0, sizeof(*lists));
wa_init_start(&lists->gt_wa_list, "GT_REF", "global");
gt_init_workarounds(i915, &lists->gt_wa_list);
gt_init_workarounds(gt->i915, &lists->gt_wa_list);
wa_init_finish(&lists->gt_wa_list);
for_each_engine(engine, i915, id) {
for_each_engine(engine, gt, id) {
struct i915_wa_list *wal = &lists->engine[id].wa_list;
wa_init_start(wal, "REF", engine->name);
@ -59,12 +83,12 @@ reference_lists_init(struct drm_i915_private *i915, struct wa_lists *lists)
}
static void
reference_lists_fini(struct drm_i915_private *i915, struct wa_lists *lists)
reference_lists_fini(struct intel_gt *gt, struct wa_lists *lists)
{
struct intel_engine_cs *engine;
enum intel_engine_id id;
for_each_engine(engine, i915, id)
for_each_engine(engine, gt, id)
intel_wa_list_free(&lists->engine[id].wa_list);
intel_wa_list_free(&lists->gt_wa_list);
@ -191,10 +215,10 @@ static int check_whitelist(struct i915_gem_context *ctx,
err = 0;
i915_gem_object_lock(results);
intel_wedge_on_timeout(&wedge, &ctx->i915->gt, HZ / 5) /* safety net! */
intel_wedge_on_timeout(&wedge, engine->gt, HZ / 5) /* safety net! */
err = i915_gem_object_set_to_cpu_domain(results, false);
i915_gem_object_unlock(results);
if (intel_gt_is_wedged(&ctx->i915->gt))
if (intel_gt_is_wedged(engine->gt))
err = -EIO;
if (err)
goto out_put;
@ -243,7 +267,6 @@ switch_to_scratch_context(struct intel_engine_cs *engine,
struct i915_gem_context *ctx;
struct intel_context *ce;
struct i915_request *rq;
intel_wakeref_t wakeref;
int err = 0;
ctx = kernel_context(engine->i915);
@ -255,9 +278,7 @@ switch_to_scratch_context(struct intel_engine_cs *engine,
ce = i915_gem_context_get_engine(ctx, engine->legacy_idx);
GEM_BUG_ON(IS_ERR(ce));
rq = ERR_PTR(-ENODEV);
with_intel_runtime_pm(&engine->i915->runtime_pm, wakeref)
rq = igt_spinner_create_request(spin, ce, MI_NOOP);
rq = igt_spinner_create_request(spin, ce, MI_NOOP);
intel_context_put(ce);
@ -267,13 +288,7 @@ switch_to_scratch_context(struct intel_engine_cs *engine,
goto err;
}
i915_request_add(rq);
if (spin && !igt_wait_for_spinner(spin, rq)) {
pr_err("Spinner failed to start\n");
err = -ETIMEDOUT;
}
err = request_add_spin(rq, spin);
err:
if (err && spin)
igt_spinner_end(spin);
@ -313,7 +328,7 @@ static int check_whitelist_across_reset(struct intel_engine_cs *engine,
if (err)
goto out_spin;
with_intel_runtime_pm(&i915->runtime_pm, wakeref)
with_intel_runtime_pm(engine->uncore->rpm, wakeref)
err = reset(engine);
igt_spinner_end(&spin);
@ -586,15 +601,11 @@ static int check_dirty_whitelist(struct i915_gem_context *ctx,
goto err_request;
err_request:
i915_request_add(rq);
if (err)
goto out_batch;
if (i915_request_wait(rq, 0, HZ / 5) < 0) {
err = request_add_sync(rq, err);
if (err) {
pr_err("%s: Futzing %x timedout; cancelling test\n",
engine->name, reg);
intel_gt_set_wedged(&ctx->i915->gt);
err = -EIO;
intel_gt_set_wedged(engine->gt);
goto out_batch;
}
@ -693,34 +704,29 @@ static int check_dirty_whitelist(struct i915_gem_context *ctx,
static int live_dirty_whitelist(void *arg)
{
struct drm_i915_private *i915 = arg;
struct intel_gt *gt = arg;
struct intel_engine_cs *engine;
struct i915_gem_context *ctx;
enum intel_engine_id id;
intel_wakeref_t wakeref;
struct drm_file *file;
int err = 0;
/* Can the user write to the whitelisted registers? */
if (INTEL_GEN(i915) < 7) /* minimum requirement for LRI, SRM, LRM */
if (INTEL_GEN(gt->i915) < 7) /* minimum requirement for LRI, SRM, LRM */
return 0;
wakeref = intel_runtime_pm_get(&i915->runtime_pm);
file = mock_file(gt->i915);
if (IS_ERR(file))
return PTR_ERR(file);
file = mock_file(i915);
if (IS_ERR(file)) {
err = PTR_ERR(file);
goto out_rpm;
}
ctx = live_context(i915, file);
ctx = live_context(gt->i915, file);
if (IS_ERR(ctx)) {
err = PTR_ERR(ctx);
goto out_file;
}
for_each_engine(engine, i915, id) {
for_each_engine(engine, gt, id) {
if (engine->whitelist.count == 0)
continue;
@ -730,43 +736,43 @@ static int live_dirty_whitelist(void *arg)
}
out_file:
mock_file_free(i915, file);
out_rpm:
intel_runtime_pm_put(&i915->runtime_pm, wakeref);
mock_file_free(gt->i915, file);
return err;
}
static int live_reset_whitelist(void *arg)
{
struct drm_i915_private *i915 = arg;
struct intel_engine_cs *engine = i915->engine[RCS0];
struct intel_gt *gt = arg;
struct intel_engine_cs *engine;
enum intel_engine_id id;
int err = 0;
/* If we reset the gpu, we should not lose the RING_NONPRIV */
igt_global_reset_lock(gt);
if (!engine || engine->whitelist.count == 0)
return 0;
for_each_engine(engine, gt, id) {
if (engine->whitelist.count == 0)
continue;
igt_global_reset_lock(&i915->gt);
if (intel_has_reset_engine(gt)) {
err = check_whitelist_across_reset(engine,
do_engine_reset,
"engine");
if (err)
goto out;
}
if (intel_has_reset_engine(&i915->gt)) {
err = check_whitelist_across_reset(engine,
do_engine_reset,
"engine");
if (err)
goto out;
}
if (intel_has_gpu_reset(&i915->gt)) {
err = check_whitelist_across_reset(engine,
do_device_reset,
"device");
if (err)
goto out;
if (intel_has_gpu_reset(gt)) {
err = check_whitelist_across_reset(engine,
do_device_reset,
"device");
if (err)
goto out;
}
}
out:
igt_global_reset_unlock(&i915->gt);
igt_global_reset_unlock(gt);
return err;
}
@ -782,6 +788,14 @@ static int read_whitelisted_registers(struct i915_gem_context *ctx,
if (IS_ERR(rq))
return PTR_ERR(rq);
i915_vma_lock(results);
err = i915_request_await_object(rq, results->obj, true);
if (err == 0)
err = i915_vma_move_to_active(results, rq, EXEC_OBJECT_WRITE);
i915_vma_unlock(results);
if (err)
goto err_req;
srm = MI_STORE_REGISTER_MEM;
if (INTEL_GEN(ctx->i915) >= 8)
srm++;
@ -807,12 +821,7 @@ static int read_whitelisted_registers(struct i915_gem_context *ctx,
intel_ring_advance(rq, cs);
err_req:
i915_request_add(rq);
if (i915_request_wait(rq, 0, HZ / 5) < 0)
err = -EIO;
return err;
return request_add_sync(rq, err);
}
static int scrub_whitelisted_registers(struct i915_gem_context *ctx,
@ -872,9 +881,7 @@ static int scrub_whitelisted_registers(struct i915_gem_context *ctx,
err = engine->emit_bb_start(rq, batch->node.start, 0, 0);
err_request:
i915_request_add(rq);
if (i915_request_wait(rq, 0, HZ / 5) < 0)
err = -EIO;
err = request_add_sync(rq, err);
err_unpin:
i915_gem_object_unpin_map(batch->obj);
@ -991,7 +998,7 @@ check_whitelisted_registers(struct intel_engine_cs *engine,
static int live_isolated_whitelist(void *arg)
{
struct drm_i915_private *i915 = arg;
struct intel_gt *gt = arg;
struct {
struct i915_gem_context *ctx;
struct i915_vma *scratch[2];
@ -1005,17 +1012,14 @@ static int live_isolated_whitelist(void *arg)
* invisible to a second context.
*/
if (!intel_engines_has_context_isolation(i915))
return 0;
if (!i915->kernel_context->vm)
if (!intel_engines_has_context_isolation(gt->i915))
return 0;
for (i = 0; i < ARRAY_SIZE(client); i++) {
struct i915_address_space *vm;
struct i915_gem_context *c;
c = kernel_context(i915);
c = kernel_context(gt->i915);
if (IS_ERR(c)) {
err = PTR_ERR(c);
goto err;
@ -1044,7 +1048,10 @@ static int live_isolated_whitelist(void *arg)
i915_vm_put(vm);
}
for_each_engine(engine, i915, id) {
for_each_engine(engine, gt, id) {
if (!engine->kernel_context->vm)
continue;
if (!whitelist_writable_count(engine))
continue;
@ -1098,7 +1105,7 @@ static int live_isolated_whitelist(void *arg)
kernel_context_close(client[i].ctx);
}
if (igt_flush_test(i915))
if (igt_flush_test(gt->i915))
err = -EIO;
return err;
@ -1133,16 +1140,16 @@ verify_wa_lists(struct i915_gem_context *ctx, struct wa_lists *lists,
static int
live_gpu_reset_workarounds(void *arg)
{
struct drm_i915_private *i915 = arg;
struct intel_gt *gt = arg;
struct i915_gem_context *ctx;
intel_wakeref_t wakeref;
struct wa_lists lists;
bool ok;
if (!intel_has_gpu_reset(&i915->gt))
if (!intel_has_gpu_reset(gt))
return 0;
ctx = kernel_context(i915);
ctx = kernel_context(gt->i915);
if (IS_ERR(ctx))
return PTR_ERR(ctx);
@ -1150,25 +1157,25 @@ live_gpu_reset_workarounds(void *arg)
pr_info("Verifying after GPU reset...\n");
igt_global_reset_lock(&i915->gt);
wakeref = intel_runtime_pm_get(&i915->runtime_pm);
igt_global_reset_lock(gt);
wakeref = intel_runtime_pm_get(gt->uncore->rpm);
reference_lists_init(i915, &lists);
reference_lists_init(gt, &lists);
ok = verify_wa_lists(ctx, &lists, "before reset");
if (!ok)
goto out;
intel_gt_reset(&i915->gt, ALL_ENGINES, "live_workarounds");
intel_gt_reset(gt, ALL_ENGINES, "live_workarounds");
ok = verify_wa_lists(ctx, &lists, "after reset");
out:
i915_gem_context_unlock_engines(ctx);
kernel_context_close(ctx);
reference_lists_fini(i915, &lists);
intel_runtime_pm_put(&i915->runtime_pm, wakeref);
igt_global_reset_unlock(&i915->gt);
reference_lists_fini(gt, &lists);
intel_runtime_pm_put(gt->uncore->rpm, wakeref);
igt_global_reset_unlock(gt);
return ok ? 0 : -ESRCH;
}
@ -1176,7 +1183,7 @@ live_gpu_reset_workarounds(void *arg)
static int
live_engine_reset_workarounds(void *arg)
{
struct drm_i915_private *i915 = arg;
struct intel_gt *gt = arg;
struct i915_gem_engines_iter it;
struct i915_gem_context *ctx;
struct intel_context *ce;
@ -1186,17 +1193,17 @@ live_engine_reset_workarounds(void *arg)
struct wa_lists lists;
int ret = 0;
if (!intel_has_reset_engine(&i915->gt))
if (!intel_has_reset_engine(gt))
return 0;
ctx = kernel_context(i915);
ctx = kernel_context(gt->i915);
if (IS_ERR(ctx))
return PTR_ERR(ctx);
igt_global_reset_lock(&i915->gt);
wakeref = intel_runtime_pm_get(&i915->runtime_pm);
igt_global_reset_lock(gt);
wakeref = intel_runtime_pm_get(gt->uncore->rpm);
reference_lists_init(i915, &lists);
reference_lists_init(gt, &lists);
for_each_gem_engine(ce, i915_gem_context_lock_engines(ctx), it) {
struct intel_engine_cs *engine = ce->engine;
@ -1229,12 +1236,10 @@ live_engine_reset_workarounds(void *arg)
goto err;
}
i915_request_add(rq);
if (!igt_wait_for_spinner(&spin, rq)) {
ret = request_add_spin(rq, &spin);
if (ret) {
pr_err("Spinner failed to start\n");
igt_spinner_fini(&spin);
ret = -ETIMEDOUT;
goto err;
}
@ -1251,12 +1256,12 @@ live_engine_reset_workarounds(void *arg)
}
err:
i915_gem_context_unlock_engines(ctx);
reference_lists_fini(i915, &lists);
intel_runtime_pm_put(&i915->runtime_pm, wakeref);
igt_global_reset_unlock(&i915->gt);
reference_lists_fini(gt, &lists);
intel_runtime_pm_put(gt->uncore->rpm, wakeref);
igt_global_reset_unlock(gt);
kernel_context_close(ctx);
igt_flush_test(i915);
igt_flush_test(gt->i915);
return ret;
}
@ -1274,5 +1279,5 @@ int intel_workarounds_live_selftests(struct drm_i915_private *i915)
if (intel_gt_is_wedged(&i915->gt))
return 0;
return i915_subtests(tests, i915);
return intel_gt_live_subtests(tests, &i915->gt);
}

31
drivers/gpu/drm/i915/gt/uc/intel_guc.c
View File

@ -9,6 +9,27 @@
#include "intel_guc_submission.h"
#include "i915_drv.h"
/**
* DOC: GuC
*
* The GuC is a microcontroller inside the GT HW, introduced in gen9. The GuC is
* designed to offload some of the functionality usually performed by the host
* driver; currently the main operations it can take care of are:
*
* - Authentication of the HuC, which is required to fully enable HuC usage.
* - Low latency graphics context scheduling (a.k.a. GuC submission).
* - GT Power management.
*
* The enable_guc module parameter can be used to select which of those
* operations to enable within GuC. Note that not all the operations are
* supported on all gen9+ platforms.
*
* Enabling the GuC is not mandatory and therefore the firmware is only loaded
* if at least one of the operations is selected. However, not loading the GuC
* might result in the loss of some features that do require the GuC (currently
* just the HuC, but more are expected to land in the future).
*/
static void gen8_guc_raise_irq(struct intel_guc *guc)
{
struct intel_gt *gt = guc_to_gt(guc);
@ -548,9 +569,15 @@ int intel_guc_resume(struct intel_guc *guc)
}
/**
* DOC: GuC Address Space
* DOC: GuC Memory Management
*
* The layout of GuC address space is shown below:
* GuC can't allocate any memory for its own usage, so all the allocations must
* be handled by the host driver. GuC accesses the memory via the GGTT, with the
* exception of the top and bottom parts of the 4GB address space, which are
* instead re-mapped by the GuC HW to memory location of the FW itself (WOPCM)
* or other parts of the HW. The driver must take care not to place objects that
* the GuC is going to access in these reserved ranges. The layout of the GuC
* address space is shown below:
*
* ::
*

3
drivers/gpu/drm/i915/gt/uc/intel_guc_log.c
View File

@ -607,7 +607,6 @@ int intel_guc_log_relay_open(struct intel_guc_log *log)
void intel_guc_log_relay_flush(struct intel_guc_log *log)
{
struct intel_guc *guc = log_to_guc(log);
struct drm_i915_private *i915 = guc_to_gt(guc)->i915;
intel_wakeref_t wakeref;
/*
@ -616,7 +615,7 @@ void intel_guc_log_relay_flush(struct intel_guc_log *log)
*/
flush_work(&log->relay.flush_work);
with_intel_runtime_pm(&i915->runtime_pm, wakeref)
with_intel_runtime_pm(guc_to_gt(guc)->uncore->rpm, wakeref)
guc_action_flush_log(guc);
/* GuC would have updated log buffer by now, so capture it */

12
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
View File

@ -29,6 +29,12 @@ enum {
/**
* DOC: GuC-based command submission
*
* IMPORTANT NOTE: GuC submission is currently not supported in i915. The GuC
* firmware is moving to an updated submission interface and we plan to
* turn submission back on when that lands. The below documentation (and related
* code) matches the old submission model and will be updated as part of the
* upgrade to the new flow.
*
* GuC client:
* A intel_guc_client refers to a submission path through GuC. Currently, there
* is only one client, which is charged with all submissions to the GuC. This
@ -1014,7 +1020,7 @@ static void guc_interrupts_capture(struct intel_gt *gt)
* to GuC
*/
irqs = _MASKED_BIT_ENABLE(GFX_INTERRUPT_STEERING);
for_each_engine(engine, gt->i915, id)
for_each_engine(engine, gt, id)
ENGINE_WRITE(engine, RING_MODE_GEN7, irqs);
/* route USER_INTERRUPT to Host, all others are sent to GuC. */
@ -1062,7 +1068,7 @@ static void guc_interrupts_release(struct intel_gt *gt)
*/
irqs = _MASKED_FIELD(GFX_FORWARD_VBLANK_MASK, GFX_FORWARD_VBLANK_NEVER);
irqs |= _MASKED_BIT_DISABLE(GFX_INTERRUPT_STEERING);
for_each_engine(engine, gt->i915, id)
for_each_engine(engine, gt, id)
ENGINE_WRITE(engine, RING_MODE_GEN7, irqs);
/* route all GT interrupts to the host */
@ -1145,7 +1151,7 @@ int intel_guc_submission_enable(struct intel_guc *guc)
/* Take over from manual control of ELSP (execlists) */
guc_interrupts_capture(gt);
for_each_engine(engine, gt->i915, id) {
for_each_engine(engine, gt, id) {
engine->set_default_submission = guc_set_default_submission;
engine->set_default_submission(engine);
}

37
drivers/gpu/drm/i915/gt/uc/intel_huc.c
View File

@ -9,6 +9,34 @@
#include "intel_huc.h"
#include "i915_drv.h"
/**
* DOC: HuC
*
* The HuC is a dedicated microcontroller for usage in media HEVC (High
* Efficiency Video Coding) operations. Userspace can directly use the firmware
* capabilities by adding HuC specific commands to batch buffers.
*
* The kernel driver is only responsible for loading the HuC firmware and
* triggering its security authentication, which is performed by the GuC. For
* The GuC to correctly perform the authentication, the HuC binary must be
* loaded before the GuC one. Loading the HuC is optional; however, not using
* the HuC might negatively impact power usage and/or performance of media
* workloads, depending on the use-cases.
*
* See https://github.com/intel/media-driver for the latest details on HuC
* functionality.
*/
/**
* DOC: HuC Memory Management
*
* Similarly to the GuC, the HuC can't do any memory allocations on its own,
* with the difference being that the allocations for HuC usage are handled by
* the userspace driver instead of the kernel one. The HuC accesses the memory
* via the PPGTT belonging to the context loaded on the VCS executing the
* HuC-specific commands.
*/
void intel_huc_init_early(struct intel_huc *huc)
{
struct drm_i915_private *i915 = huc_to_gt(huc)->i915;
@ -118,10 +146,9 @@ void intel_huc_fini(struct intel_huc *huc)
*
* Called after HuC and GuC firmware loading during intel_uc_init_hw().
*
* This function pins HuC firmware image object into GGTT.
* Then it invokes GuC action to authenticate passing the offset to RSA
* signature through intel_guc_auth_huc(). It then waits for 50ms for
* firmware verification ACK and unpins the object.
* This function invokes the GuC action to authenticate the HuC firmware,
* passing the offset of the RSA signature to intel_guc_auth_huc(). It then
* waits for up to 50ms for firmware verification ACK.
*/
int intel_huc_auth(struct intel_huc *huc)
{
@ -185,7 +212,7 @@ int intel_huc_check_status(struct intel_huc *huc)
if (!intel_huc_is_supported(huc))
return -ENODEV;
with_intel_runtime_pm(&gt->i915->runtime_pm, wakeref)
with_intel_runtime_pm(gt->uncore->rpm, wakeref)
status = intel_uncore_read(gt->uncore, huc->status.reg);
return (status & huc->status.mask) == huc->status.value;

15
drivers/gpu/drm/i915/gt/uc/intel_huc_fw.c
View File

@ -7,21 +7,6 @@
#include "intel_huc_fw.h"
#include "i915_drv.h"
/**
* DOC: HuC Firmware
*
* Motivation:
* GEN9 introduces a new dedicated firmware for usage in media HEVC (High
* Efficiency Video Coding) operations. Userspace can use the firmware
* capabilities by adding HuC specific commands to batch buffers.
*
* Implementation:
* The same firmware loader is used as the GuC. However, the actual
* loading to HW is deferred until GEM initialization is done.
*
* Note that HuC firmware loading must be done before GuC loading.
*/
/**
* intel_huc_fw_init_early() - initializes HuC firmware struct
* @huc: intel_huc struct

2
drivers/gpu/drm/i915/gt/uc/intel_uc.c
View File

@ -587,7 +587,7 @@ void intel_uc_suspend(struct intel_uc *uc)
if (!intel_guc_is_running(guc))
return;
with_intel_runtime_pm(&uc_to_gt(uc)->i915->runtime_pm, wakeref)
with_intel_runtime_pm(uc_to_gt(uc)->uncore->rpm, wakeref)
intel_uc_runtime_suspend(uc);
}

3
drivers/gpu/drm/i915/gt/uc/intel_uc_fw_abi.h
View File

@ -39,9 +39,6 @@
* 3. Length info of each component can be found in header, in dwords.
* 4. Modulus and exponent key are not required by driver. They may not appear
* in fw. So driver will load a truncated firmware in this case.
*
* The only difference between GuC and HuC firmwares is how the version
* information is saved.
*/
struct uc_css_header {

42
drivers/gpu/drm/i915/gt/uc/selftest_guc.c
View File

@ -108,22 +108,15 @@ static bool client_doorbell_in_sync(struct intel_guc_client *client)
* validating that the doorbells status expected by the driver matches what the
* GuC/HW have.
*/
static int igt_guc_clients(void *args)
static int igt_guc_clients(void *arg)
{
struct drm_i915_private *dev_priv = args;
struct intel_gt *gt = arg;
struct intel_guc *guc = &gt->uc.guc;
intel_wakeref_t wakeref;
struct intel_guc *guc;
int err = 0;
GEM_BUG_ON(!HAS_GT_UC(dev_priv));
wakeref = intel_runtime_pm_get(&dev_priv->runtime_pm);
guc = &dev_priv->gt.uc.guc;
if (!guc) {
pr_err("No guc object!\n");
err = -EINVAL;
goto unlock;
}
GEM_BUG_ON(!HAS_GT_UC(gt->i915));
wakeref = intel_runtime_pm_get(gt->uncore->rpm);
err = check_all_doorbells(guc);
if (err)
@ -188,7 +181,7 @@ static int igt_guc_clients(void *args)
guc_clients_create(guc);
guc_clients_enable(guc);
unlock:
intel_runtime_pm_put(&dev_priv->runtime_pm, wakeref);
intel_runtime_pm_put(gt->uncore->rpm, wakeref);
return err;
}
@ -199,21 +192,14 @@ static int igt_guc_clients(void *args)
*/
static int igt_guc_doorbells(void *arg)
{
struct drm_i915_private *dev_priv = arg;
struct intel_gt *gt = arg;
struct intel_guc *guc = &gt->uc.guc;
intel_wakeref_t wakeref;
struct intel_guc *guc;
int i, err = 0;
u16 db_id;
GEM_BUG_ON(!HAS_GT_UC(dev_priv));
wakeref = intel_runtime_pm_get(&dev_priv->runtime_pm);
guc = &dev_priv->gt.uc.guc;
if (!guc) {
pr_err("No guc object!\n");
err = -EINVAL;
goto unlock;
}
GEM_BUG_ON(!HAS_GT_UC(gt->i915));
wakeref = intel_runtime_pm_get(gt->uncore->rpm);
err = check_all_doorbells(guc);
if (err)
@ -295,19 +281,19 @@ static int igt_guc_doorbells(void *arg)
guc_client_free(clients[i]);
}
unlock:
intel_runtime_pm_put(&dev_priv->runtime_pm, wakeref);
intel_runtime_pm_put(gt->uncore->rpm, wakeref);
return err;
}
int intel_guc_live_selftest(struct drm_i915_private *dev_priv)
int intel_guc_live_selftest(struct drm_i915_private *i915)
{
static const struct i915_subtest tests[] = {
SUBTEST(igt_guc_clients),
SUBTEST(igt_guc_doorbells),
};
if (!USES_GUC_SUBMISSION(dev_priv))
if (!USES_GUC_SUBMISSION(i915))
return 0;
return i915_subtests(tests, dev_priv);
return intel_gt_live_subtests(tests, &i915->gt);
}

2
drivers/gpu/drm/i915/gvt/aperture_gm.c
View File

@ -198,7 +198,7 @@ static int alloc_vgpu_fence(struct intel_vgpu *vgpu)
mutex_lock(&dev_priv->ggtt.vm.mutex);
for (i = 0; i < vgpu_fence_sz(vgpu); i++) {
reg = i915_reserve_fence(dev_priv);
reg = i915_reserve_fence(&dev_priv->ggtt);
if (IS_ERR(reg))
goto out_free_fence;

4
drivers/gpu/drm/i915/gvt/execlist.c
View File

@ -534,7 +534,7 @@ static void clean_execlist(struct intel_vgpu *vgpu,
struct intel_vgpu_submission *s = &vgpu->submission;
intel_engine_mask_t tmp;
for_each_engine_masked(engine, dev_priv, engine_mask, tmp) {
for_each_engine_masked(engine, &dev_priv->gt, engine_mask, tmp) {
kfree(s->ring_scan_buffer[engine->id]);
s->ring_scan_buffer[engine->id] = NULL;
s->ring_scan_buffer_size[engine->id] = 0;
@ -548,7 +548,7 @@ static void reset_execlist(struct intel_vgpu *vgpu,
struct intel_engine_cs *engine;
intel_engine_mask_t tmp;
for_each_engine_masked(engine, dev_priv, engine_mask, tmp)
for_each_engine_masked(engine, &dev_priv->gt, engine_mask, tmp)
init_vgpu_execlist(vgpu, engine->id);
}

18
drivers/gpu/drm/i915/gvt/scheduler.c
View File

@ -415,10 +415,9 @@ int intel_gvt_scan_and_shadow_workload(struct intel_vgpu_workload *workload)
{
struct intel_vgpu *vgpu = workload->vgpu;
struct intel_vgpu_submission *s = &vgpu->submission;
struct drm_i915_private *dev_priv = vgpu->gvt->dev_priv;
int ret;
lockdep_assert_held(&dev_priv->drm.struct_mutex);
lockdep_assert_held(&vgpu->vgpu_lock);
if (workload->shadow)
return 0;
@ -580,8 +579,6 @@ static void update_vreg_in_ctx(struct intel_vgpu_workload *workload)
static void release_shadow_batch_buffer(struct intel_vgpu_workload *workload)
{
struct intel_vgpu *vgpu = workload->vgpu;
struct drm_i915_private *dev_priv = vgpu->gvt->dev_priv;
struct intel_vgpu_shadow_bb *bb, *pos;
if (list_empty(&workload->shadow_bb))
@ -590,8 +587,6 @@ static void release_shadow_batch_buffer(struct intel_vgpu_workload *workload)
bb = list_first_entry(&workload->shadow_bb,
struct intel_vgpu_shadow_bb, list);
mutex_lock(&dev_priv->drm.struct_mutex);
list_for_each_entry_safe(bb, pos, &workload->shadow_bb, list) {
if (bb->obj) {
if (bb->accessing)
@ -609,8 +604,6 @@ static void release_shadow_batch_buffer(struct intel_vgpu_workload *workload)
list_del(&bb->list);
kfree(bb);
}
mutex_unlock(&dev_priv->drm.struct_mutex);
}
static int prepare_workload(struct intel_vgpu_workload *workload)
@ -685,7 +678,6 @@ static int prepare_workload(struct intel_vgpu_workload *workload)
static int dispatch_workload(struct intel_vgpu_workload *workload)
{
struct intel_vgpu *vgpu = workload->vgpu;
struct drm_i915_private *dev_priv = vgpu->gvt->dev_priv;
struct i915_request *rq;
int ring_id = workload->ring_id;
int ret;
@ -694,7 +686,6 @@ static int dispatch_workload(struct intel_vgpu_workload *workload)
ring_id, workload);
mutex_lock(&vgpu->vgpu_lock);
mutex_lock(&dev_priv->drm.struct_mutex);
ret = intel_gvt_workload_req_alloc(workload);
if (ret)
@ -729,7 +720,6 @@ static int dispatch_workload(struct intel_vgpu_workload *workload)
err_req:
if (ret)
workload->status = ret;
mutex_unlock(&dev_priv->drm.struct_mutex);
mutex_unlock(&vgpu->vgpu_lock);
return ret;
}
@ -887,7 +877,7 @@ void intel_vgpu_clean_workloads(struct intel_vgpu *vgpu,
intel_engine_mask_t tmp;
/* free the unsubmited workloads in the queues. */
for_each_engine_masked(engine, dev_priv, engine_mask, tmp) {
for_each_engine_masked(engine, &dev_priv->gt, engine_mask, tmp) {
list_for_each_entry_safe(pos, n,
&s->workload_q_head[engine->id], list) {
list_del_init(&pos->list);
@ -1594,9 +1584,9 @@ intel_vgpu_create_workload(struct intel_vgpu *vgpu, int ring_id,
*/
if (list_empty(workload_q_head(vgpu, ring_id))) {
intel_runtime_pm_get(&dev_priv->runtime_pm);
mutex_lock(&dev_priv->drm.struct_mutex);
mutex_lock(&vgpu->vgpu_lock);
ret = intel_gvt_scan_and_shadow_workload(workload);
mutex_unlock(&dev_priv->drm.struct_mutex);
mutex_unlock(&vgpu->vgpu_lock);
intel_runtime_pm_put_unchecked(&dev_priv->runtime_pm);
}

4
drivers/gpu/drm/i915/i915_active.c
View File

@ -590,8 +590,8 @@ static struct active_node *reuse_idle_barrier(struct i915_active *ref, u64 idx)
int i915_active_acquire_preallocate_barrier(struct i915_active *ref,
struct intel_engine_cs *engine)
{
struct drm_i915_private *i915 = engine->i915;
intel_engine_mask_t tmp, mask = engine->mask;
struct intel_gt *gt = engine->gt;
struct llist_node *pos, *next;
int err;
@ -603,7 +603,7 @@ int i915_active_acquire_preallocate_barrier(struct i915_active *ref,
* We can then use the preallocated nodes in
* i915_active_acquire_barrier()
*/
for_each_engine_masked(engine, i915, mask, tmp) {
for_each_engine_masked(engine, gt, mask, tmp) {
u64 idx = engine->kernel_context->timeline->fence_context;
struct active_node *node;

50
drivers/gpu/drm/i915/i915_debugfs.c
View File

@ -1660,9 +1660,9 @@ static int i915_swizzle_info(struct seq_file *m, void *data)
wakeref = intel_runtime_pm_get(&dev_priv->runtime_pm);
seq_printf(m, "bit6 swizzle for X-tiling = %s\n",
swizzle_string(dev_priv->mm.bit_6_swizzle_x));
swizzle_string(dev_priv->ggtt.bit_6_swizzle_x));
seq_printf(m, "bit6 swizzle for Y-tiling = %s\n",
swizzle_string(dev_priv->mm.bit_6_swizzle_y));
swizzle_string(dev_priv->ggtt.bit_6_swizzle_y));
if (IS_GEN_RANGE(dev_priv, 3, 4)) {
seq_printf(m, "DDC = 0x%08x\n",
@ -2405,6 +2405,13 @@ static int i915_dmc_info(struct seq_file *m, void *unused)
if (INTEL_GEN(dev_priv) >= 12) {
dc5_reg = TGL_DMC_DEBUG_DC5_COUNT;
dc6_reg = TGL_DMC_DEBUG_DC6_COUNT;
/*
* NOTE: DMC_DEBUG3 is a general purpose reg.
* According to B.Specs:49196 DMC f/w reuses DC5/6 counter
* reg for DC3CO debugging and validation,
* but TGL DMC f/w is using DMC_DEBUG3 reg for DC3CO counter.
*/
seq_printf(m, "DC3CO count: %d\n", I915_READ(DMC_DEBUG3));
} else {
dc5_reg = IS_BROXTON(dev_priv) ? BXT_CSR_DC3_DC5_COUNT :
SKL_CSR_DC3_DC5_COUNT;
@ -3583,6 +3590,37 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_wedged_fops,
i915_wedged_get, i915_wedged_set,
"%llu\n");
static int
i915_perf_noa_delay_set(void *data, u64 val)
{
struct drm_i915_private *i915 = data;
const u32 clk = RUNTIME_INFO(i915)->cs_timestamp_frequency_khz;
/*
* This would lead to infinite waits as we're doing timestamp
* difference on the CS with only 32bits.
*/
if (val > mul_u32_u32(U32_MAX, clk))
return -EINVAL;
atomic64_set(&i915->perf.noa_programming_delay, val);
return 0;
}
static int
i915_perf_noa_delay_get(void *data, u64 *val)
{
struct drm_i915_private *i915 = data;
*val = atomic64_read(&i915->perf.noa_programming_delay);
return 0;
}
DEFINE_SIMPLE_ATTRIBUTE(i915_perf_noa_delay_fops,
i915_perf_noa_delay_get,
i915_perf_noa_delay_set,
"%llu\n");
#define DROP_UNBOUND BIT(0)
#define DROP_BOUND BIT(1)
#define DROP_RETIRE BIT(2)
@ -3592,6 +3630,7 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_wedged_fops,
#define DROP_IDLE BIT(6)
#define DROP_RESET_ACTIVE BIT(7)
#define DROP_RESET_SEQNO BIT(8)
#define DROP_RCU BIT(9)
#define DROP_ALL (DROP_UNBOUND | \
DROP_BOUND | \
DROP_RETIRE | \
@ -3600,7 +3639,8 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_wedged_fops,
DROP_SHRINK_ALL |\
DROP_IDLE | \
DROP_RESET_ACTIVE | \
DROP_RESET_SEQNO)
DROP_RESET_SEQNO | \
DROP_RCU)
static int
i915_drop_caches_get(void *data, u64 *val)
{
@ -3652,6 +3692,9 @@ i915_drop_caches_set(void *data, u64 val)
i915_gem_shrink_all(i915);
fs_reclaim_release(GFP_KERNEL);
if (val & DROP_RCU)
rcu_barrier();
if (val & DROP_FREED)
i915_gem_drain_freed_objects(i915);
@ -4333,6 +4376,7 @@ static const struct i915_debugfs_files {
const char *name;
const struct file_operations *fops;
} i915_debugfs_files[] = {
{"i915_perf_noa_delay", &i915_perf_noa_delay_fops},
{"i915_wedged", &i915_wedged_fops},
{"i915_cache_sharing", &i915_cache_sharing_fops},
{"i915_gem_drop_caches", &i915_drop_caches_fops},

12
drivers/gpu/drm/i915/i915_drv.c
View File

@ -354,6 +354,8 @@ static void i915_driver_modeset_remove(struct drm_i915_private *i915)
{
intel_modeset_driver_remove(i915);
intel_irq_uninstall(i915);
intel_bios_driver_remove(i915);
i915_switcheroo_unregister(i915);
@ -1073,8 +1075,8 @@ intel_get_dram_info(struct drm_i915_private *dev_priv)
static u32 gen9_edram_size_mb(struct drm_i915_private *dev_priv, u32 cap)
{
const unsigned int ways[8] = { 4, 8, 12, 16, 16, 16, 16, 16 };
const unsigned int sets[4] = { 1, 1, 2, 2 };
static const u8 ways[8] = { 4, 8, 12, 16, 16, 16, 16, 16 };
static const u8 sets[4] = { 1, 1, 2, 2 };
return EDRAM_NUM_BANKS(cap) *
ways[EDRAM_WAYS_IDX(cap)] *
@ -1804,7 +1806,7 @@ static int i915_drm_resume(struct drm_device *dev)
DRM_ERROR("failed to re-enable GGTT\n");
i915_gem_restore_gtt_mappings(dev_priv);
i915_gem_restore_fences(dev_priv);
i915_gem_restore_fences(&dev_priv->ggtt);
intel_csr_ucode_resume(dev_priv);
@ -2502,7 +2504,7 @@ static int intel_runtime_suspend(struct device *kdev)
intel_gt_runtime_resume(&dev_priv->gt);
i915_gem_restore_fences(dev_priv);
i915_gem_restore_fences(&dev_priv->ggtt);
enable_rpm_wakeref_asserts(rpm);
@ -2582,7 +2584,7 @@ static int intel_runtime_resume(struct device *kdev)
* we can do is to hope that things will still work (and disable RPM).
*/
intel_gt_runtime_resume(&dev_priv->gt);
i915_gem_restore_fences(dev_priv);
i915_gem_restore_fences(&dev_priv->ggtt);
/*
* On VLV/CHV display interrupts are part of the display

401
drivers/gpu/drm/i915/i915_drv.h
View File

@ -85,6 +85,7 @@
#include "intel_device_info.h"
#include "intel_pch.h"
#include "intel_runtime_pm.h"
#include "intel_memory_region.h"
#include "intel_uncore.h"
#include "intel_wakeref.h"
#include "intel_wopcm.h"
@ -93,6 +94,7 @@
#include "i915_gem_fence_reg.h"
#include "i915_gem_gtt.h"
#include "i915_gpu_error.h"
#include "i915_perf_types.h"
#include "i915_request.h"
#include "i915_scheduler.h"
#include "gt/intel_timeline.h"
@ -106,8 +108,8 @@
#define DRIVER_NAME "i915"
#define DRIVER_DESC "Intel Graphics"
#define DRIVER_DATE "20191007"
#define DRIVER_TIMESTAMP 1570451087
#define DRIVER_DATE "20191021"
#define DRIVER_TIMESTAMP 1571651766
struct drm_i915_gem_object;
@ -338,6 +340,7 @@ struct intel_csr {
i915_reg_t mmioaddr[20];
u32 mmiodata[20];
u32 dc_state;
u32 target_dc_state;
u32 allowed_dc_mask;
intel_wakeref_t wakeref;
};
@ -500,6 +503,9 @@ struct i915_psr {
bool sink_not_reliable;
bool irq_aux_error;
u16 su_x_granularity;
bool dc3co_enabled;
u32 dc3co_exit_delay;
struct delayed_work idle_work;
};
#define QUIRK_LVDS_SSC_DISABLE (1<<1)
@ -599,13 +605,8 @@ struct intel_rps {
struct intel_rps_ei ei;
};
struct intel_llc_pstate {
bool enabled;
};
struct intel_gen6_power_mgmt {
struct intel_rps rps;
struct intel_llc_pstate llc_pstate;
};
/* defined intel_pm.c */
@ -678,6 +679,8 @@ struct i915_gem_mm {
*/
struct vfsmount *gemfs;
struct intel_memory_region *regions[INTEL_REGION_UNKNOWN];
struct notifier_block oom_notifier;
struct notifier_block vmap_notifier;
struct shrinker shrinker;
@ -689,11 +692,6 @@ struct i915_gem_mm {
*/
struct workqueue_struct *userptr_wq;
/** Bit 6 swizzling required for X tiling */
u32 bit_6_swizzle_x;
/** Bit 6 swizzling required for Y tiling */
u32 bit_6_swizzle_y;
/* shrinker accounting, also useful for userland debugging */
u64 shrink_memory;
u32 shrink_count;
@ -974,305 +972,6 @@ struct intel_wm_config {
bool sprites_scaled;
};
struct i915_oa_format {
u32 format;
int size;
};
struct i915_oa_reg {
i915_reg_t addr;
u32 value;
};
struct i915_oa_config {
char uuid[UUID_STRING_LEN + 1];
int id;
const struct i915_oa_reg *mux_regs;
u32 mux_regs_len;
const struct i915_oa_reg *b_counter_regs;
u32 b_counter_regs_len;
const struct i915_oa_reg *flex_regs;
u32 flex_regs_len;
struct attribute_group sysfs_metric;
struct attribute *attrs[2];
struct device_attribute sysfs_metric_id;
atomic_t ref_count;
};
struct i915_perf_stream;
/**
* struct i915_perf_stream_ops - the OPs to support a specific stream type
*/
struct i915_perf_stream_ops {
/**
* @enable: Enables the collection of HW samples, either in response to
* `I915_PERF_IOCTL_ENABLE` or implicitly called when stream is opened
* without `I915_PERF_FLAG_DISABLED`.
*/
void (*enable)(struct i915_perf_stream *stream);
/**
* @disable: Disables the collection of HW samples, either in response
* to `I915_PERF_IOCTL_DISABLE` or implicitly called before destroying
* the stream.
*/
void (*disable)(struct i915_perf_stream *stream);
/**
* @poll_wait: Call poll_wait, passing a wait queue that will be woken
* once there is something ready to read() for the stream
*/
void (*poll_wait)(struct i915_perf_stream *stream,
struct file *file,
poll_table *wait);
/**
* @wait_unlocked: For handling a blocking read, wait until there is
* something to ready to read() for the stream. E.g. wait on the same
* wait queue that would be passed to poll_wait().
*/
int (*wait_unlocked)(struct i915_perf_stream *stream);
/**
* @read: Copy buffered metrics as records to userspace
* **buf**: the userspace, destination buffer
* **count**: the number of bytes to copy, requested by userspace
* **offset**: zero at the start of the read, updated as the read
* proceeds, it represents how many bytes have been copied so far and
* the buffer offset for copying the next record.
*
* Copy as many buffered i915 perf samples and records for this stream
* to userspace as will fit in the given buffer.
*
* Only write complete records; returning -%ENOSPC if there isn't room
* for a complete record.
*
* Return any error condition that results in a short read such as
* -%ENOSPC or -%EFAULT, even though these may be squashed before
* returning to userspace.
*/
int (*read)(struct i915_perf_stream *stream,
char __user *buf,
size_t count,
size_t *offset);
/**
* @destroy: Cleanup any stream specific resources.
*
* The stream will always be disabled before this is called.
*/
void (*destroy)(struct i915_perf_stream *stream);
};
/**
* struct i915_perf_stream - state for a single open stream FD
*/
struct i915_perf_stream {
/**
* @dev_priv: i915 drm device
*/
struct drm_i915_private *dev_priv;
/**
* @link: Links the stream into ``&drm_i915_private->streams``
*/
struct list_head link;
/**
* @wakeref: As we keep the device awake while the perf stream is
* active, we track our runtime pm reference for later release.
*/
intel_wakeref_t wakeref;
/**
* @sample_flags: Flags representing the `DRM_I915_PERF_PROP_SAMPLE_*`
* properties given when opening a stream, representing the contents
* of a single sample as read() by userspace.
*/
u32 sample_flags;
/**
* @sample_size: Considering the configured contents of a sample
* combined with the required header size, this is the total size
* of a single sample record.
*/
int sample_size;
/**
* @ctx: %NULL if measuring system-wide across all contexts or a
* specific context that is being monitored.
*/
struct i915_gem_context *ctx;
/**
* @enabled: Whether the stream is currently enabled, considering
* whether the stream was opened in a disabled state and based
* on `I915_PERF_IOCTL_ENABLE` and `I915_PERF_IOCTL_DISABLE` calls.
*/
bool enabled;
/**
* @ops: The callbacks providing the implementation of this specific
* type of configured stream.
*/
const struct i915_perf_stream_ops *ops;
/**
* @oa_config: The OA configuration used by the stream.
*/
struct i915_oa_config *oa_config;
/**
* @pinned_ctx: The OA context specific information.
*/
struct intel_context *pinned_ctx;
u32 specific_ctx_id;
u32 specific_ctx_id_mask;
struct hrtimer poll_check_timer;
wait_queue_head_t poll_wq;
bool pollin;
bool periodic;
int period_exponent;
/**
* @oa_buffer: State of the OA buffer.
*/
struct {
struct i915_vma *vma;
u8 *vaddr;
u32 last_ctx_id;
int format;
int format_size;
int size_exponent;
/**
* @ptr_lock: Locks reads and writes to all head/tail state
*
* Consider: the head and tail pointer state needs to be read
* consistently from a hrtimer callback (atomic context) and
* read() fop (user context) with tail pointer updates happening
* in atomic context and head updates in user context and the
* (unlikely) possibility of read() errors needing to reset all
* head/tail state.
*
* Note: Contention/performance aren't currently a significant
* concern here considering the relatively low frequency of
* hrtimer callbacks (5ms period) and that reads typically only
* happen in response to a hrtimer event and likely complete
* before the next callback.
*
* Note: This lock is not held *while* reading and copying data
* to userspace so the value of head observed in htrimer
* callbacks won't represent any partial consumption of data.
*/
spinlock_t ptr_lock;
/**
* @tails: One 'aging' tail pointer and one 'aged' tail pointer ready to
* used for reading.
*
* Initial values of 0xffffffff are invalid and imply that an
* update is required (and should be ignored by an attempted
* read)
*/
struct {
u32 offset;
} tails[2];
/**
* @aged_tail_idx: Index for the aged tail ready to read() data up to.
*/
unsigned int aged_tail_idx;
/**
* @aging_timestamp: A monotonic timestamp for when the current aging tail pointer
* was read; used to determine when it is old enough to trust.
*/
u64 aging_timestamp;
/**
* @head: Although we can always read back the head pointer register,
* we prefer to avoid trusting the HW state, just to avoid any
* risk that some hardware condition could * somehow bump the
* head pointer unpredictably and cause us to forward the wrong
* OA buffer data to userspace.
*/
u32 head;
} oa_buffer;
};
/**
* struct i915_oa_ops - Gen specific implementation of an OA unit stream
*/
struct i915_oa_ops {
/**
* @is_valid_b_counter_reg: Validates register's address for
* programming boolean counters for a particular platform.
*/
bool (*is_valid_b_counter_reg)(struct drm_i915_private *dev_priv,
u32 addr);
/**
* @is_valid_mux_reg: Validates register's address for programming mux
* for a particular platform.
*/
bool (*is_valid_mux_reg)(struct drm_i915_private *dev_priv, u32 addr);
/**
* @is_valid_flex_reg: Validates register's address for programming
* flex EU filtering for a particular platform.
*/
bool (*is_valid_flex_reg)(struct drm_i915_private *dev_priv, u32 addr);
/**
* @enable_metric_set: Selects and applies any MUX configuration to set
* up the Boolean and Custom (B/C) counters that are part of the
* counter reports being sampled. May apply system constraints such as
* disabling EU clock gating as required.
*/
int (*enable_metric_set)(struct i915_perf_stream *stream);
/**
* @disable_metric_set: Remove system constraints associated with using
* the OA unit.
*/
void (*disable_metric_set)(struct i915_perf_stream *stream);
/**
* @oa_enable: Enable periodic sampling
*/
void (*oa_enable)(struct i915_perf_stream *stream);
/**
* @oa_disable: Disable periodic sampling
*/
void (*oa_disable)(struct i915_perf_stream *stream);
/**
* @read: Copy data from the circular OA buffer into a given userspace
* buffer.
*/
int (*read)(struct i915_perf_stream *stream,
char __user *buf,
size_t count,
size_t *offset);
/**
* @oa_hw_tail_read: read the OA tail pointer register
*
* In particular this enables us to share all the fiddly code for
* handling the OA unit tail pointer race that affects multiple
* generations.
*/
u32 (*oa_hw_tail_read)(struct i915_perf_stream *stream);
};
struct intel_cdclk_state {
unsigned int cdclk, vco, ref, bypass;
u8 voltage_level;
@ -1560,6 +1259,8 @@ struct drm_i915_private {
I915_SAGV_NOT_CONTROLLED
} sagv_status;
u32 sagv_block_time_us;
struct {
/*
* Raw watermark latency values:
@ -1630,61 +1331,7 @@ struct drm_i915_private {
struct intel_runtime_pm runtime_pm;
struct {
bool initialized;
struct kobject *metrics_kobj;
struct ctl_table_header *sysctl_header;
/*
* Lock associated with adding/modifying/removing OA configs
* in dev_priv->perf.metrics_idr.
*/
struct mutex metrics_lock;
/*
* List of dynamic configurations, you need to hold
* dev_priv->perf.metrics_lock to access it.
*/
struct idr metrics_idr;
/*
* Lock associated with anything below within this structure
* except exclusive_stream.
*/
struct mutex lock;
struct list_head streams;
/*
* The stream currently using the OA unit. If accessed
* outside a syscall associated to its file
* descriptor, you need to hold
* dev_priv->drm.struct_mutex.
*/
struct i915_perf_stream *exclusive_stream;
/**
* For rate limiting any notifications of spurious
* invalid OA reports
*/
struct ratelimit_state spurious_report_rs;
struct i915_oa_config test_config;
u32 gen7_latched_oastatus1;
u32 ctx_oactxctrl_offset;
u32 ctx_flexeu0_offset;
/**
* The RPT_ID/reason field for Gen8+ includes a bit
* to determine if the CTX ID in the report is valid
* but the specific bit differs between Gen 8 and 9
*/
u32 gen8_valid_ctx_bit;
struct i915_oa_ops ops;
const struct i915_oa_format *oa_formats;
} perf;
struct i915_perf perf;
/* Abstract the submission mechanism (legacy ringbuffer or execlists) away */
struct intel_gt gt;
@ -1765,10 +1412,10 @@ static inline struct drm_i915_private *pdev_to_i915(struct pci_dev *pdev)
for_each_if ((engine__) = (dev_priv__)->engine[(id__)])
/* Iterator over subset of engines selected by mask */
#define for_each_engine_masked(engine__, dev_priv__, mask__, tmp__) \
for ((tmp__) = (mask__) & INTEL_INFO(dev_priv__)->engine_mask; \
#define for_each_engine_masked(engine__, gt__, mask__, tmp__) \
for ((tmp__) = (mask__) & INTEL_INFO((gt__)->i915)->engine_mask; \
(tmp__) ? \
((engine__) = (dev_priv__)->engine[__mask_next_bit(tmp__)]), 1 : \
((engine__) = (gt__)->engine[__mask_next_bit(tmp__)]), 1 : \
0;)
#define rb_to_uabi_engine(rb) \
@ -2031,6 +1678,11 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
#define IS_ICL_REVID(p, since, until) \
(IS_ICELAKE(p) && IS_REVID(p, since, until))
#define TGL_REVID_A0 0x0
#define IS_TGL_REVID(p, since, until) \
(IS_TIGERLAKE(p) && IS_REVID(p, since, until))
#define IS_LP(dev_priv) (INTEL_INFO(dev_priv)->is_lp)
#define IS_GEN9_LP(dev_priv) (IS_GEN(dev_priv, 9) && IS_LP(dev_priv))
#define IS_GEN9_BC(dev_priv) (IS_GEN(dev_priv, 9) && !IS_LP(dev_priv))
@ -2128,6 +1780,8 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
#define HAS_IPC(dev_priv) (INTEL_INFO(dev_priv)->display.has_ipc)
#define HAS_REGION(i915, i) (INTEL_INFO(i915)->memory_regions & (i))
#define HAS_GT_UC(dev_priv) (INTEL_INFO(dev_priv)->has_gt_uc)
/* Having GuC is not the same as using GuC */
@ -2217,6 +1871,8 @@ void i915_gem_cleanup_early(struct drm_i915_private *dev_priv);
int i915_gem_freeze(struct drm_i915_private *dev_priv);
int i915_gem_freeze_late(struct drm_i915_private *dev_priv);
struct intel_memory_region *i915_gem_shmem_setup(struct drm_i915_private *i915);
static inline void i915_gem_drain_freed_objects(struct drm_i915_private *i915)
{
/*
@ -2346,6 +2002,9 @@ int __must_check i915_gem_evict_for_node(struct i915_address_space *vm,
unsigned int flags);
int i915_gem_evict_vm(struct i915_address_space *vm);
void i915_gem_cleanup_memory_regions(struct drm_i915_private *i915);
int i915_gem_init_memory_regions(struct drm_i915_private *i915);
/* i915_gem_internal.c */
struct drm_i915_gem_object *
i915_gem_object_create_internal(struct drm_i915_private *dev_priv,
@ -2354,9 +2013,9 @@ i915_gem_object_create_internal(struct drm_i915_private *dev_priv,
/* i915_gem_tiling.c */
static inline bool i915_gem_object_needs_bit17_swizzle(struct drm_i915_gem_object *obj)
{
struct drm_i915_private *dev_priv = to_i915(obj->base.dev);
struct drm_i915_private *i915 = to_i915(obj->base.dev);
return dev_priv->mm.bit_6_swizzle_x == I915_BIT_6_SWIZZLE_9_10_17 &&
return i915->ggtt.bit_6_swizzle_x == I915_BIT_6_SWIZZLE_9_10_17 &&
i915_gem_object_is_tiled(obj);
}

20
drivers/gpu/drm/i915/i915_gem.c
View File

@ -45,7 +45,6 @@
#include "gem/i915_gem_context.h"
#include "gem/i915_gem_ioctls.h"
#include "gem/i915_gem_pm.h"
#include "gem/i915_gemfs.h"
#include "gt/intel_engine_user.h"
#include "gt/intel_gt.h"
#include "gt/intel_gt_pm.h"
@ -1111,15 +1110,6 @@ static int __intel_engines_record_defaults(struct drm_i915_private *i915)
if (err)
goto err_rq;
/*
* Failing to program the MOCS is non-fatal.The system will not
* run at peak performance. So warn the user and carry on.
*/
err = intel_mocs_emit(rq);
if (err)
dev_notice(i915->drm.dev,
"Failed to program MOCS registers; expect performance issues.\n");
err = intel_renderstate_emit(rq);
if (err)
goto err_rq;
@ -1369,7 +1359,7 @@ int i915_gem_init(struct drm_i915_private *dev_priv)
/* Minimal basic recovery for KMS */
ret = i915_ggtt_enable_hw(dev_priv);
i915_gem_restore_gtt_mappings(dev_priv);
i915_gem_restore_fences(dev_priv);
i915_gem_restore_fences(&dev_priv->ggtt);
intel_init_clock_gating(dev_priv);
}
@ -1441,16 +1431,10 @@ static void i915_gem_init__mm(struct drm_i915_private *i915)
void i915_gem_init_early(struct drm_i915_private *dev_priv)
{
int err;
i915_gem_init__mm(dev_priv);
i915_gem_init__pm(dev_priv);
spin_lock_init(&dev_priv->fb_tracking.lock);
err = i915_gemfs_init(dev_priv);
if (err)
DRM_NOTE("Unable to create a private tmpfs mount, hugepage support will be disabled(%d).\n", err);
}
void i915_gem_cleanup_early(struct drm_i915_private *dev_priv)
@ -1459,8 +1443,6 @@ void i915_gem_cleanup_early(struct drm_i915_private *dev_priv)
GEM_BUG_ON(!llist_empty(&dev_priv->mm.free_list));
GEM_BUG_ON(atomic_read(&dev_priv->mm.free_count));
WARN_ON(dev_priv->mm.shrink_count);
i915_gemfs_fini(dev_priv);
}
int i915_gem_freeze(struct drm_i915_private *dev_priv)

36
drivers/gpu/drm/i915/i915_gem.h
View File

@ -37,10 +37,8 @@ struct drm_i915_private;
#define GEM_SHOW_DEBUG() (drm_debug & DRM_UT_DRIVER)
#define GEM_BUG_ON(condition) do { if (unlikely((condition))) { \
pr_err("%s:%d GEM_BUG_ON(%s)\n", \
__func__, __LINE__, __stringify(condition)); \
GEM_TRACE("%s:%d GEM_BUG_ON(%s)\n", \
__func__, __LINE__, __stringify(condition)); \
GEM_TRACE_ERR("%s:%d GEM_BUG_ON(%s)\n", \
__func__, __LINE__, __stringify(condition)); \
BUG(); \
} \
} while(0)
@ -66,17 +64,33 @@ struct drm_i915_private;
#if IS_ENABLED(CONFIG_DRM_I915_TRACE_GEM)
#define GEM_TRACE(...) trace_printk(__VA_ARGS__)
#define GEM_TRACE_ERR(...) do { \
pr_err(__VA_ARGS__); \
trace_printk(__VA_ARGS__); \
} while (0)
#define GEM_TRACE_DUMP() ftrace_dump(DUMP_ALL)
#define GEM_TRACE_DUMP_ON(expr) \
do { if (expr) ftrace_dump(DUMP_ALL); } while (0)
#else
#define GEM_TRACE(...) do { } while (0)
#define GEM_TRACE_ERR(...) do { } while (0)
#define GEM_TRACE_DUMP() do { } while (0)
#define GEM_TRACE_DUMP_ON(expr) BUILD_BUG_ON_INVALID(expr)
#endif
#define I915_GEM_IDLE_TIMEOUT (HZ / 5)
static inline void tasklet_lock(struct tasklet_struct *t)
{
while (!tasklet_trylock(t))
cpu_relax();
}
static inline bool tasklet_is_locked(const struct tasklet_struct *t)
{
return test_bit(TASKLET_STATE_RUN, &t->state);
}
static inline void __tasklet_disable_sync_once(struct tasklet_struct *t)
{
if (!atomic_fetch_inc(&t->count))
@ -98,4 +112,18 @@ static inline bool __tasklet_is_scheduled(struct tasklet_struct *t)
return test_bit(TASKLET_STATE_SCHED, &t->state);
}
static inline void cancel_timer(struct timer_list *t)
{
if (!READ_ONCE(t->expires))
return;
del_timer(t);
WRITE_ONCE(t->expires, 0);
}
static inline bool timer_expired(const struct timer_list *t)
{
return READ_ONCE(t->expires) && !timer_pending(t);
}
#endif /* __I915_GEM_H__ */

89
drivers/gpu/drm/i915/i915_gem_fence_reg.c
View File

@ -59,6 +59,16 @@
#define pipelined 0
static struct drm_i915_private *fence_to_i915(struct i915_fence_reg *fence)
{
return fence->ggtt->vm.i915;
}
static struct intel_uncore *fence_to_uncore(struct i915_fence_reg *fence)
{
return fence->ggtt->vm.gt->uncore;
}
static void i965_write_fence_reg(struct i915_fence_reg *fence,
struct i915_vma *vma)
{
@ -66,7 +76,7 @@ static void i965_write_fence_reg(struct i915_fence_reg *fence,
int fence_pitch_shift;
u64 val;
if (INTEL_GEN(fence->i915) >= 6) {
if (INTEL_GEN(fence_to_i915(fence)) >= 6) {
fence_reg_lo = FENCE_REG_GEN6_LO(fence->id);
fence_reg_hi = FENCE_REG_GEN6_HI(fence->id);
fence_pitch_shift = GEN6_FENCE_PITCH_SHIFT;
@ -95,7 +105,7 @@ static void i965_write_fence_reg(struct i915_fence_reg *fence,
}
if (!pipelined) {
struct intel_uncore *uncore = &fence->i915->uncore;
struct intel_uncore *uncore = fence_to_uncore(fence);
/*
* To w/a incoherency with non-atomic 64-bit register updates,
@ -132,7 +142,7 @@ static void i915_write_fence_reg(struct i915_fence_reg *fence,
GEM_BUG_ON(!is_power_of_2(vma->fence_size));
GEM_BUG_ON(!IS_ALIGNED(vma->node.start, vma->fence_size));
if (is_y_tiled && HAS_128_BYTE_Y_TILING(fence->i915))
if (is_y_tiled && HAS_128_BYTE_Y_TILING(fence_to_i915(fence)))
stride /= 128;
else
stride /= 512;
@ -148,7 +158,7 @@ static void i915_write_fence_reg(struct i915_fence_reg *fence,
}
if (!pipelined) {
struct intel_uncore *uncore = &fence->i915->uncore;
struct intel_uncore *uncore = fence_to_uncore(fence);
i915_reg_t reg = FENCE_REG(fence->id);
intel_uncore_write_fw(uncore, reg, val);
@ -180,7 +190,7 @@ static void i830_write_fence_reg(struct i915_fence_reg *fence,
}
if (!pipelined) {
struct intel_uncore *uncore = &fence->i915->uncore;
struct intel_uncore *uncore = fence_to_uncore(fence);
i915_reg_t reg = FENCE_REG(fence->id);
intel_uncore_write_fw(uncore, reg, val);
@ -191,15 +201,17 @@ static void i830_write_fence_reg(struct i915_fence_reg *fence,
static void fence_write(struct i915_fence_reg *fence,
struct i915_vma *vma)
{
struct drm_i915_private *i915 = fence_to_i915(fence);
/*
* Previous access through the fence register is marshalled by
* the mb() inside the fault handlers (i915_gem_release_mmaps)
* and explicitly managed for internal users.
*/
if (IS_GEN(fence->i915, 2))
if (IS_GEN(i915, 2))
i830_write_fence_reg(fence, vma);
else if (IS_GEN(fence->i915, 3))
else if (IS_GEN(i915, 3))
i915_write_fence_reg(fence, vma);
else
i965_write_fence_reg(fence, vma);
@ -215,6 +227,8 @@ static void fence_write(struct i915_fence_reg *fence,
static int fence_update(struct i915_fence_reg *fence,
struct i915_vma *vma)
{
struct i915_ggtt *ggtt = fence->ggtt;
struct intel_uncore *uncore = fence_to_uncore(fence);
intel_wakeref_t wakeref;
struct i915_vma *old;
int ret;
@ -256,7 +270,7 @@ static int fence_update(struct i915_fence_reg *fence,
old->fence = NULL;
}
list_move(&fence->link, &fence->i915->ggtt.fence_list);
list_move(&fence->link, &ggtt->fence_list);
}
/*
@ -269,7 +283,7 @@ static int fence_update(struct i915_fence_reg *fence,
* be cleared before we can use any other fences to ensure that
* the new fences do not overlap the elided clears, confusing HW.
*/
wakeref = intel_runtime_pm_get_if_in_use(&fence->i915->runtime_pm);
wakeref = intel_runtime_pm_get_if_in_use(uncore->rpm);
if (!wakeref) {
GEM_BUG_ON(vma);
return 0;
@ -280,10 +294,10 @@ static int fence_update(struct i915_fence_reg *fence,
if (vma) {
vma->fence = fence;
list_move_tail(&fence->link, &fence->i915->ggtt.fence_list);
list_move_tail(&fence->link, &ggtt->fence_list);
}
intel_runtime_pm_put(&fence->i915->runtime_pm, wakeref);
intel_runtime_pm_put(uncore->rpm, wakeref);
return 0;
}
@ -312,11 +326,11 @@ int i915_vma_revoke_fence(struct i915_vma *vma)
return fence_update(fence, NULL);
}
static struct i915_fence_reg *fence_find(struct drm_i915_private *i915)
static struct i915_fence_reg *fence_find(struct i915_ggtt *ggtt)
{
struct i915_fence_reg *fence;
list_for_each_entry(fence, &i915->ggtt.fence_list, link) {
list_for_each_entry(fence, &ggtt->fence_list, link) {
GEM_BUG_ON(fence->vma && fence->vma->fence != fence);
if (atomic_read(&fence->pin_count))
@ -326,7 +340,7 @@ static struct i915_fence_reg *fence_find(struct drm_i915_private *i915)
}
/* Wait for completion of pending flips which consume fences */
if (intel_has_pending_fb_unpin(i915))
if (intel_has_pending_fb_unpin(ggtt->vm.i915))
return ERR_PTR(-EAGAIN);
return ERR_PTR(-EDEADLK);
@ -351,7 +365,7 @@ int __i915_vma_pin_fence(struct i915_vma *vma)
return 0;
}
} else if (set) {
fence = fence_find(vma->vm->i915);
fence = fence_find(ggtt);
if (IS_ERR(fence))
return PTR_ERR(fence);
@ -402,7 +416,7 @@ int i915_vma_pin_fence(struct i915_vma *vma)
* Note that we revoke fences on runtime suspend. Therefore the user
* must keep the device awake whilst using the fence.
*/
assert_rpm_wakelock_held(&vma->vm->i915->runtime_pm);
assert_rpm_wakelock_held(vma->vm->gt->uncore->rpm);
GEM_BUG_ON(!i915_vma_is_pinned(vma));
GEM_BUG_ON(!i915_vma_is_ggtt(vma));
@ -418,14 +432,13 @@ int i915_vma_pin_fence(struct i915_vma *vma)
/**
* i915_reserve_fence - Reserve a fence for vGPU
* @i915: i915 device private
* @ggtt: Global GTT
*
* This function walks the fence regs looking for a free one and remove
* it from the fence_list. It is used to reserve fence for vGPU to use.
*/
struct i915_fence_reg *i915_reserve_fence(struct drm_i915_private *i915)
struct i915_fence_reg *i915_reserve_fence(struct i915_ggtt *ggtt)
{
struct i915_ggtt *ggtt = &i915->ggtt;
struct i915_fence_reg *fence;
int count;
int ret;
@ -439,7 +452,7 @@ struct i915_fence_reg *i915_reserve_fence(struct drm_i915_private *i915)
if (count <= 1)
return ERR_PTR(-ENOSPC);
fence = fence_find(i915);
fence = fence_find(ggtt);
if (IS_ERR(fence))
return fence;
@ -463,7 +476,7 @@ struct i915_fence_reg *i915_reserve_fence(struct drm_i915_private *i915)
*/
void i915_unreserve_fence(struct i915_fence_reg *fence)
{
struct i915_ggtt *ggtt = &fence->i915->ggtt;
struct i915_ggtt *ggtt = fence->ggtt;
lockdep_assert_held(&ggtt->vm.mutex);
@ -472,19 +485,19 @@ void i915_unreserve_fence(struct i915_fence_reg *fence)
/**
* i915_gem_restore_fences - restore fence state
* @i915: i915 device private
* @ggtt: Global GTT
*
* Restore the hw fence state to match the software tracking again, to be called
* after a gpu reset and on resume. Note that on runtime suspend we only cancel
* the fences, to be reacquired by the user later.
*/
void i915_gem_restore_fences(struct drm_i915_private *i915)
void i915_gem_restore_fences(struct i915_ggtt *ggtt)
{
int i;
rcu_read_lock(); /* keep obj alive as we dereference */
for (i = 0; i < i915->ggtt.num_fences; i++) {
struct i915_fence_reg *reg = &i915->ggtt.fence_regs[i];
for (i = 0; i < ggtt->num_fences; i++) {
struct i915_fence_reg *reg = &ggtt->fence_regs[i];
struct i915_vma *vma = READ_ONCE(reg->vma);
GEM_BUG_ON(vma && vma->fence != reg);
@ -550,15 +563,16 @@ void i915_gem_restore_fences(struct drm_i915_private *i915)
*/
/**
* i915_gem_detect_bit_6_swizzle - detect bit 6 swizzling pattern
* @i915: i915 device private
* detect_bit_6_swizzle - detect bit 6 swizzling pattern
* @ggtt: Global GGTT
*
* Detects bit 6 swizzling of address lookup between IGD access and CPU
* access through main memory.
*/
static void detect_bit_6_swizzle(struct drm_i915_private *i915)
static void detect_bit_6_swizzle(struct i915_ggtt *ggtt)
{
struct intel_uncore *uncore = &i915->uncore;
struct intel_uncore *uncore = ggtt->vm.gt->uncore;
struct drm_i915_private *i915 = ggtt->vm.i915;
u32 swizzle_x = I915_BIT_6_SWIZZLE_UNKNOWN;
u32 swizzle_y = I915_BIT_6_SWIZZLE_UNKNOWN;
@ -720,8 +734,8 @@ static void detect_bit_6_swizzle(struct drm_i915_private *i915)
swizzle_y = I915_BIT_6_SWIZZLE_NONE;
}
i915->mm.bit_6_swizzle_x = swizzle_x;
i915->mm.bit_6_swizzle_y = swizzle_y;
i915->ggtt.bit_6_swizzle_x = swizzle_x;
i915->ggtt.bit_6_swizzle_y = swizzle_y;
}
/*
@ -822,14 +836,15 @@ i915_gem_object_save_bit_17_swizzle(struct drm_i915_gem_object *obj,
void i915_ggtt_init_fences(struct i915_ggtt *ggtt)
{
struct drm_i915_private *i915 = ggtt->vm.i915;
struct intel_uncore *uncore = ggtt->vm.gt->uncore;
int num_fences;
int i;
INIT_LIST_HEAD(&ggtt->fence_list);
INIT_LIST_HEAD(&ggtt->userfault_list);
intel_wakeref_auto_init(&ggtt->userfault_wakeref, &i915->runtime_pm);
intel_wakeref_auto_init(&ggtt->userfault_wakeref, uncore->rpm);
detect_bit_6_swizzle(i915);
detect_bit_6_swizzle(ggtt);
if (INTEL_GEN(i915) >= 7 &&
!(IS_VALLEYVIEW(i915) || IS_CHERRYVIEW(i915)))
@ -842,20 +857,20 @@ void i915_ggtt_init_fences(struct i915_ggtt *ggtt)
num_fences = 8;
if (intel_vgpu_active(i915))
num_fences = intel_uncore_read(&i915->uncore,
num_fences = intel_uncore_read(uncore,
vgtif_reg(avail_rs.fence_num));
/* Initialize fence registers to zero */
for (i = 0; i < num_fences; i++) {
struct i915_fence_reg *fence = &ggtt->fence_regs[i];
fence->i915 = i915;
fence->ggtt = ggtt;
fence->id = i;
list_add_tail(&fence->link, &ggtt->fence_list);
}
ggtt->num_fences = num_fences;
i915_gem_restore_fences(i915);
i915_gem_restore_fences(ggtt);
}
void intel_gt_init_swizzling(struct intel_gt *gt)
@ -864,7 +879,7 @@ void intel_gt_init_swizzling(struct intel_gt *gt)
struct intel_uncore *uncore = gt->uncore;
if (INTEL_GEN(i915) < 5 ||
i915->mm.bit_6_swizzle_x == I915_BIT_6_SWIZZLE_NONE)
i915->ggtt.bit_6_swizzle_x == I915_BIT_6_SWIZZLE_NONE)
return;
intel_uncore_rmw(uncore, DISP_ARB_CTL, 0, DISP_TILE_SURFACE_SWIZZLING);

7
drivers/gpu/drm/i915/i915_gem_fence_reg.h
View File

@ -29,7 +29,6 @@
#include <linux/types.h>
struct drm_i915_gem_object;
struct drm_i915_private;
struct i915_ggtt;
struct i915_vma;
struct intel_gt;
@ -39,7 +38,7 @@ struct sg_table;
struct i915_fence_reg {
struct list_head link;
struct drm_i915_private *i915;
struct i915_ggtt *ggtt;
struct i915_vma *vma;
atomic_t pin_count;
int id;
@ -55,10 +54,10 @@ struct i915_fence_reg {
};
/* i915_gem_fence_reg.c */
struct i915_fence_reg *i915_reserve_fence(struct drm_i915_private *i915);
struct i915_fence_reg *i915_reserve_fence(struct i915_ggtt *ggtt);
void i915_unreserve_fence(struct i915_fence_reg *fence);
void i915_gem_restore_fences(struct drm_i915_private *i915);
void i915_gem_restore_fences(struct i915_ggtt *ggtt);
void i915_gem_object_do_bit_17_swizzle(struct drm_i915_gem_object *obj,
struct sg_table *pages);

65
drivers/gpu/drm/i915/i915_gem_gtt.c
View File

@ -1569,7 +1569,7 @@ static void gen7_ppgtt_enable(struct intel_gt *gt)
}
intel_uncore_write(uncore, GAM_ECOCHK, ecochk);
for_each_engine(engine, i915, id) {
for_each_engine(engine, gt, id) {
/* GFX_MODE is per-ring on gen7+ */
ENGINE_WRITE(engine,
RING_MODE_GEN7,
@ -2744,6 +2744,59 @@ int i915_init_ggtt(struct drm_i915_private *i915)
return 0;
}
void i915_gem_cleanup_memory_regions(struct drm_i915_private *i915)
{
int i;
for (i = 0; i < INTEL_REGION_UNKNOWN; i++) {
struct intel_memory_region *region = i915->mm.regions[i];
if (region)
intel_memory_region_put(region);
}
}
int i915_gem_init_memory_regions(struct drm_i915_private *i915)
{
int err, i;
for (i = 0; i < INTEL_REGION_UNKNOWN; i++) {
struct intel_memory_region *mem = ERR_PTR(-ENODEV);
u32 type;
if (!HAS_REGION(i915, BIT(i)))
continue;
type = MEMORY_TYPE_FROM_REGION(intel_region_map[i]);
switch (type) {
case INTEL_MEMORY_SYSTEM:
mem = i915_gem_shmem_setup(i915);
break;
case INTEL_MEMORY_STOLEN:
mem = i915_gem_stolen_setup(i915);
break;
}
if (IS_ERR(mem)) {
err = PTR_ERR(mem);
DRM_ERROR("Failed to setup region(%d) type=%d\n", err, type);
goto out_cleanup;
}
mem->id = intel_region_map[i];
mem->type = type;
mem->instance = MEMORY_INSTANCE_FROM_REGION(intel_region_map[i]);
i915->mm.regions[i] = mem;
}
return 0;
out_cleanup:
i915_gem_cleanup_memory_regions(i915);
return err;
}
static void ggtt_cleanup_hw(struct i915_ggtt *ggtt)
{
struct i915_vma *vma, *vn;
@ -2781,6 +2834,8 @@ void i915_ggtt_driver_release(struct drm_i915_private *i915)
{
struct pagevec *pvec;
i915_gem_cleanup_memory_regions(i915);
fini_aliasing_ppgtt(&i915->ggtt);
ggtt_cleanup_hw(&i915->ggtt);
@ -2790,8 +2845,6 @@ void i915_ggtt_driver_release(struct drm_i915_private *i915)
set_pages_array_wb(pvec->pages, pvec->nr);
__pagevec_release(pvec);
}
i915_gem_cleanup_stolen(i915);
}
static unsigned int gen6_get_total_gtt_size(u16 snb_gmch_ctl)
@ -3240,11 +3293,7 @@ int i915_ggtt_init_hw(struct drm_i915_private *dev_priv)
if (ret)
return ret;
/*
* Initialise stolen early so that we may reserve preallocated
* objects for the BIOS to KMS transition.
*/
ret = i915_gem_init_stolen(dev_priv);
ret = i915_gem_init_memory_regions(dev_priv);
if (ret)
goto out_gtt_cleanup;

5
drivers/gpu/drm/i915/i915_gem_gtt.h
View File

@ -411,6 +411,11 @@ struct i915_ggtt {
int mtrr;
/** Bit 6 swizzling required for X tiling */
u32 bit_6_swizzle_x;
/** Bit 6 swizzling required for Y tiling */
u32 bit_6_swizzle_y;
u32 pin_bias;
unsigned int num_fences;

4
drivers/gpu/drm/i915/i915_getparam.c
View File

@ -5,6 +5,7 @@
#include "gt/intel_engine_user.h"
#include "i915_drv.h"
#include "i915_perf.h"
int i915_getparam_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv)
@ -156,6 +157,9 @@ int i915_getparam_ioctl(struct drm_device *dev, void *data,
case I915_PARAM_MMAP_GTT_COHERENT:
value = INTEL_INFO(i915)->has_coherent_ggtt;
break;
case I915_PARAM_PERF_REVISION:
value = i915_perf_ioctl_version();
break;
default:
DRM_DEBUG("Unknown parameter %d\n", param->param);
return -EINVAL;

48
drivers/gpu/drm/i915/i915_irq.c
View File

@ -412,7 +412,7 @@ void gen9_reset_guc_interrupts(struct intel_guc *guc)
{
struct intel_gt *gt = guc_to_gt(guc);
assert_rpm_wakelock_held(&gt->i915->runtime_pm);
assert_rpm_wakelock_held(gt->uncore->rpm);
spin_lock_irq(&gt->irq_lock);
gen6_gt_pm_reset_iir(gt, gt->pm_guc_events);
@ -423,7 +423,7 @@ void gen9_enable_guc_interrupts(struct intel_guc *guc)
{
struct intel_gt *gt = guc_to_gt(guc);
assert_rpm_wakelock_held(&gt->i915->runtime_pm);
assert_rpm_wakelock_held(gt->uncore->rpm);
spin_lock_irq(&gt->irq_lock);
if (!guc->interrupts.enabled) {
@ -440,7 +440,7 @@ void gen9_disable_guc_interrupts(struct intel_guc *guc)
{
struct intel_gt *gt = guc_to_gt(guc);
assert_rpm_wakelock_held(&gt->i915->runtime_pm);
assert_rpm_wakelock_held(gt->uncore->rpm);
spin_lock_irq(&gt->irq_lock);
guc->interrupts.enabled = false;
@ -2248,11 +2248,19 @@ static void icp_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir)
tc_hotplug_trigger = pch_iir & SDE_TC_MASK_TGP;
tc_port_hotplug_long_detect = tgp_tc_port_hotplug_long_detect;
pins = hpd_tgp;
} else if (HAS_PCH_MCC(dev_priv)) {
} else if (HAS_PCH_JSP(dev_priv)) {
ddi_hotplug_trigger = pch_iir & SDE_DDI_MASK_TGP;
tc_hotplug_trigger = 0;
pins = hpd_tgp;
} else if (HAS_PCH_MCC(dev_priv)) {
ddi_hotplug_trigger = pch_iir & SDE_DDI_MASK_ICP;
tc_hotplug_trigger = pch_iir & SDE_TC_HOTPLUG_ICP(PORT_TC1);
tc_port_hotplug_long_detect = icp_tc_port_hotplug_long_detect;
pins = hpd_icp;
} else {
WARN(!HAS_PCH_ICP(dev_priv),
"Unrecognized PCH type 0x%x\n", INTEL_PCH_TYPE(dev_priv));
ddi_hotplug_trigger = pch_iir & SDE_DDI_MASK_ICP;
tc_hotplug_trigger = pch_iir & SDE_TC_MASK_ICP;
tc_port_hotplug_long_detect = icp_tc_port_hotplug_long_detect;
@ -3375,11 +3383,24 @@ static void icp_hpd_irq_setup(struct drm_i915_private *dev_priv,
* equivalent of SDE.
*/
static void mcc_hpd_irq_setup(struct drm_i915_private *dev_priv)
{
icp_hpd_irq_setup(dev_priv,
SDE_DDI_MASK_ICP, SDE_TC_HOTPLUG_ICP(PORT_TC1),
ICP_DDI_HPD_ENABLE_MASK, ICP_TC_HPD_ENABLE(PORT_TC1),
hpd_icp);
}
/*
* JSP behaves exactly the same as MCC above except that port C is mapped to
* the DDI-C pins instead of the TC1 pins. This means we should follow TGP's
* masks & tables rather than ICP's masks & tables.
*/
static void jsp_hpd_irq_setup(struct drm_i915_private *dev_priv)
{
icp_hpd_irq_setup(dev_priv,
SDE_DDI_MASK_TGP, 0,
TGP_DDI_HPD_ENABLE_MASK, 0,
hpd_icp);
hpd_tgp);
}
static void gen11_hpd_detection_setup(struct drm_i915_private *dev_priv)
@ -3782,8 +3803,11 @@ static void icp_irq_postinstall(struct drm_i915_private *dev_priv)
if (HAS_PCH_TGP(dev_priv))
icp_hpd_detection_setup(dev_priv, TGP_DDI_HPD_ENABLE_MASK,
TGP_TC_HPD_ENABLE_MASK);
else if (HAS_PCH_MCC(dev_priv))
else if (HAS_PCH_JSP(dev_priv))
icp_hpd_detection_setup(dev_priv, TGP_DDI_HPD_ENABLE_MASK, 0);
else if (HAS_PCH_MCC(dev_priv))
icp_hpd_detection_setup(dev_priv, ICP_DDI_HPD_ENABLE_MASK,
ICP_TC_HPD_ENABLE(PORT_TC1));
else
icp_hpd_detection_setup(dev_priv, ICP_DDI_HPD_ENABLE_MASK,
ICP_TC_HPD_ENABLE_MASK);
@ -4313,7 +4337,9 @@ void intel_irq_init(struct drm_i915_private *dev_priv)
if (I915_HAS_HOTPLUG(dev_priv))
dev_priv->display.hpd_irq_setup = i915_hpd_irq_setup;
} else {
if (HAS_PCH_MCC(dev_priv))
if (HAS_PCH_JSP(dev_priv))
dev_priv->display.hpd_irq_setup = jsp_hpd_irq_setup;
else if (HAS_PCH_MCC(dev_priv))
dev_priv->display.hpd_irq_setup = mcc_hpd_irq_setup;
else if (INTEL_GEN(dev_priv) >= 11)
dev_priv->display.hpd_irq_setup = gen11_hpd_irq_setup;
@ -4460,10 +4486,10 @@ void intel_irq_uninstall(struct drm_i915_private *dev_priv)
int irq = dev_priv->drm.pdev->irq;
/*
* FIXME we can get called twice during driver load
* error handling due to intel_modeset_cleanup()
* calling us out of sequence. Would be nice if
* it didn't do that...
* FIXME we can get called twice during driver probe
* error handling as well as during driver remove due to
* intel_modeset_driver_remove() calling us out of sequence.
* Would be nice if it didn't do that...
*/
if (!dev_priv->drm.irq_enabled)
return;

3
drivers/gpu/drm/i915/i915_params.c
View File

@ -46,7 +46,8 @@ i915_param_named(modeset, int, 0400,
i915_param_named_unsafe(enable_dc, int, 0400,
"Enable power-saving display C-states. "
"(-1=auto [default]; 0=disable; 1=up to DC5; 2=up to DC6)");
"(-1=auto [default]; 0=disable; 1=up to DC5; 2=up to DC6; "
"3=up to DC5 with DC3CO; 4=up to DC6 with DC3CO)");
i915_param_named_unsafe(enable_fbc, int, 0600,
"Enable frame buffer compression for power savings "

Some files were not shown because too many files have changed in this diff Show More