Suspend/resume not working after adding AMD dGPU

Question

I've added a discrete GPU (AMD RX 6900XT) to my setup. Works great, except for suspend/resume not working anymore. Previously, I was running on the Intel iGPU and that worked fine, so the GPU must be throwing a spanner in the works.

lspci -k shows:

01:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Upstream Port of PCI Express Switch (rev c0)
    Kernel driver in use: pcieport
02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Downstream Port of PCI Express Switch
    Kernel driver in use: pcieport
03:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 [Radeon RX 6800/6800 XT / 6900 XT] (rev c0)
    Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Radeon RX 6900 XT
    Kernel driver in use: amdgpu
    Kernel modules: amdgpu
03:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 HDMI Audio [Radeon RX 6800/6800 XT / 6900 XT]
    Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 HDMI Audio [Radeon RX 6800/6800 XT / 6900 XT]
    Kernel driver in use: snd_hda_intel
    Kernel modules: snd_hda_intel
03:00.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] Device 73a6
    Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Device 73a6
    Kernel driver in use: xhci_hcd
    Kernel modules: xhci_pci
03:00.3 Serial bus controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 USB
    Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Device 0408
    Kernel driver in use: i2c-designware-pci

lsmod | grep -ie 'amd' shows:

amdgpu               8601600  18
drm_ttm_helper         16384  1 amdgpu
ttm                    86016  3 amdgpu,drm_ttm_helper,i915
gpu_sched              53248  1 amdgpu

xrandr --listproviders shows:

Providers: number : 2
Provider 0: id: 0x58 cap: 0xf, Source Output, Sink Output, Source Offload, Sink Offload crtcs: 6 outputs: 4 associated providers: 1 name:AMD Radeon RX 6900 XT @ pci:0000:03:00.0
Provider 1: id: 0x6f7 cap: 0xf, Source Output, Sink Output, Source Offload, Sink Offload crtcs: 4 outputs: 3 associated providers: 1 name:modesetting

Things I've done:

• Created a Xorg.conf with X -configure
• Added amdgpu to Kernel modules (in mkinitcpio.conf)
• Tried various BIOS settings, like enable ASPM, disable iGPU
• Installed xf86-video-amdgpu package (besides the obvious meta package and Vulkan packages)
• Blacklisted module i915 (blacklist.conf in /etc/modprobe.d) in case the iGPU was still involved
• Blacklisted ucsi_ccg (was showing an error)
• Installed acpid package (logs showed some warnings)
• Tried amdgpu.dc=0, amdgpu.reset_method=2, amdgpu.runpm=0, nomodeset, pcie_aspm=force, acpi_osi=Linux, acpi_osi=Windows kernel parameters
• Downgraded kernel from 5.17.1 to 5.16.6, 5.16.3, 5.15.13
• Tried the experimental mesa-git package (and related packages)
• Reinstalled Mesa
• Reinstalled linux-headers
• Tried solutions from https://gitlab.freedesktop.org/drm/amd/-/issues/1797, which seems a similar issue

I'm using Arch Linux (5.17.1) with GNOME/Xorg. Hardware specs: Intel 12th gen 12700 on a GIGABYTE AERO G motherboard with 32GB DDR5 G.SKILL memory. I have the monitor (Alienware AW3420DW) hooked up with a DisplayPort cable.

If I look at the logs (journal) from the last suspend attempt, all it logs from suspending is:

apr 02 11:43:43 arch NetworkManager[764]: <info>  [1648892623.2585] manager: sleep: sleep requested (sleeping: no  enabled: yes)
apr 02 11:43:43 arch NetworkManager[764]: <info>  [1648892623.2589] device (wlo1): state change: unavailable -> unmanaged (reason 'sleeping', sys-iface-state: 'managed')
apr 02 11:43:43 arch NetworkManager[764]: <info>  [1648892623.2602] device (wlo1): set-hw-addr: reset MAC address to F0:9E:4A:8A:9B:02 (unmanage)
apr 02 11:43:43 arch NetworkManager[764]: <info>  [1648892623.2623] manager: NetworkManager state is now ASLEEP
apr 02 11:43:43 arch gnome-shell[1091]: Timelines with detached actors are not supported
apr 02 11:43:47 arch systemd[1]: Reached target Sleep.
apr 02 11:43:47 arch systemd[1]: Starting System Suspend...

The system seems to suspend normally. When I press a key, the system turns back on (power LED lights up, fans spin up) but the screen stays black. What is interesting, is there is no log whatsoever of the resume attempt. I've dealt with suspend/resume issues with a NVIDIA card on another computer, but that always logged something about resume being initiated (and then typically the GPU could not be found due to modesetting issues).

I've also looked at dmesg and Xorg logs, but nothing of interest there. At least, no obvious errors or warnings.

I'm pretty much out of ideas at this point, so I hope someone can share some tips or point me to something I can investigate further.

Answer 1

It seems I managed to solve it.

With all the testing, kernel tweaking, building kernels from source and many hours of bisecting not leading to any solutions, I thought there might've been a BIOS update in the meantime. Indeed there was: GIGABYTE released the 'F6' version for my Z690 AERO G motherboard, up from 'F5'.

The new BIOS version instantly solved the issue, making suspend/resume work without fault on the current 5.19 kernel!

Unfortunately, with BIOS code being proprietary, the release notes for updates are brief and quite vague. They list added support for new CPU's and better support for DDR5, along with 'performance and stability updates'. Nothing directly relating to my issue, but one could speculate that the BIOS code had some particular issues causing it to not interpret sleep/resume signals correctly (from the logs I had noted i2c-designware-pci shows up from 5.12 onward and that might be causing a stumble in the process).

If you face a similar issue and a BIOS update does not solve it, refer here for further ideas and troubleshooting: https://gitlab.freedesktop.org/drm/amd/-/issues/1963