VGA device assignment
From KVM
Assigning physical VGA adapters
It would appear that assigning modern PCI/PCIe VGA adapters should not be any different from passing through normal PCI devices. But there are additional challenges to solve, specifically related to legacy VGA requirements and proprietary mechanisms to relocate I/O regions of the adapters. Moreover, some QEMU and SeaBIOS deficits contribute to the current situation that things rarely work out of the box.
Open issues
- Insufficient PCI window size
This is a fixable SeaBIOS limitation. Patches are currently under development by Gerd Hoffmann. - Non-standard I/O region remapping
Some NVIDIA Quadro adapters are known to be affected by this, but probably only during early boot. A workaround could be identity mapping, i.e. locating the memory BARs in the guest at the same locations as on the host. Of course, this breaks if the guest OS or guest drivers decide to remap them. - Legacy VGA support
In order to get boot messages of the guest BIOS, boot loader, or early OS stages, access to the legacy VGA PIO and MMIO regions need to be forwarded to the passed-through adapter. Moreover, the VBIOS of the adapter must be bootable inside the guest. As a result, the following aspects will need to be addressed:- Legacy VGA pass-through
QEMU requires patches to forward guest access to legacy VGA via the assigned adapter. - PAM/SRAM QEMU breakage
Current QEMU contains broken code that prevents access to legacy VGA even if the adapter is emulated but differently initialized (test case: -vga none -device VGA) - Legacy VGA arbitration
As the legacy IO resources cannot be remapped, access needs to be switched between guests and the host on demand. The Linux kernel provides a VGA arbiter for this purpose, but it is not used consistently by all involved parties. Moreover, QEMU also requires additional patches to support this. - VBIOS ROM access
To re-run the POST procedures of the assigned adapter inside the guest, the proper VBIOS ROM image has to be used. However, when passing through the primary adapter of the host, Linux provides only access to the shadowed version of the VBIOS which may differ from the pre-POST version (due to modifications applied during POST). This has been observed with NVIDIA Quadro adapters. A workaround is to retrieve the VBIOS from the adapter while it is in secondary mode and use this saved image (-device pci-assign,...,romfile=...). But even that may fail, either due to problems of the host chipset or BIOS (e.g. host kernel complains about an unmappable ROM BAR).
- Legacy VGA pass-through
Potential tweaks & tricks
- Some adapters may advertise MSI support even if that feature does not work properly. In that case, KVM will use MSI on the host side unconditionally unless instructed to follow the guest driver usage via -device pci-assign,...,prefer_msi=off.
Success stories
Despite all existing problems, some users have already succeeded in utilizing pass-through functionality for various VGA adapters: