VGA device assignment
From KVM
Assigning physical VGA adapters
On first glance, assigning modern PCI/PCIe VGA adapters should not be different from passing through normal PCI devices. But there are additional challenges to solve, specifically related to legacy VGA requirement and proprietary mechanisms to relocate I/O regions of the adapters. Moreover, some QEMU and SeaBIOS deficits contribute to the current situation that things rarely work out of the box.
Open issues
- Insufficient PCI window size
This is a fixable SeaBIOS limitations. Patches are under development, e.g. by Gerd Hoffmann - Non-standard I/O region remapping
At least NVIDIA adapters of the Quadro series a known to be affected by this, but probably only during early boot. A workaround could be identity mapping, i.e. locating the memory BARs in the guest at the same locations as on the host. Of course, this breaks if guest OS or guest driver decided to remap them for some reason. - Legacy VGA support
In order to get boot messages of the guest BIOS, boot loader, or early OS stages, access to the legacy VGA PIO and MMIO regions need to be forwarded to the passed-through adapter. Moreover, the VBIOS of the adapter must be bootable inside the guest. In details, the following aspects need to be addressed:- Legacy VGA pass-through
QEMU requires patches to forward guest access to legacy VGA to the assigned adapter. - PAM/SRAM QEMU breakage
Current QEMU contains broken code that prevents access to legacy VGA even if the adapter is emulated but differently initialized (test case: -vga none -device VGA) - Legacy VGA arbitration
As the legacy IO resources cannot be remapped, access need to be switched between guests and host on demand. The Linux kernel provides a VGA arbiter for this purpose, but it is not used consistently by all involved parties. Moreover, QEMU requires additional patches to support this. - VBIOS ROM access
To re-run the POST procedures of the assigned adapter inside the guest, the proper VBIOS ROM image has to be used. However, when passing through the primary adapter of the host, Linux provides only access to the shadowed version of the VBIOS which may differ from the pre-POST version (due to modification applied during POST). This has be been observed with NVDIA Quadro adapters. A workaround is to retrieve the VBIOS from the adapter while it is in secondary mode and use this saved image (-device pci-assign,...,romfile=...). But even that may fail, either due to problems of the host chipset or BIOS (host kernel complains about unmappable ROM BAR).
- Legacy VGA pass-through
Potential tweaks & tricks
- Some adapters may advertise MSI support even if that feature does not work properly. In that case, KVM will use MSI on the host side unconditionally unless instructed to follow the guest driver usage via -device pci-assign,...,prefer_msi=off.
Success stories
Despite all existing problems, some users already succeeded passing through VGA adapters: