KVM Autotest Refactor page
KVM autotest refactor
Suggested approach
1) Move code that has negligible impact [see table below] from 'client/tests/kvm' to 'client' ASAP
+ This would immediatelly benefit xen autotest, since it is frequently lagging behing features and fixes. One example is newer syntax in kvm_config.py that was not available in the xen autotest counterpart.
2) Synchronize that code (step 1) usage between kvm and xen autotest + Rewrite current tests, changing imports of modules from 'kvm_*' and 'xen_' to a common prefix ('virt_*' ?). + This can be done by an automated script and verified by the unit tests.
3) Work gradually on code that would generate more impact, implementing abstractions and other mechanisms + The config file and thus the params passed to tests already add a lot of flexibility [see example snippet #1] + Improve monitor functionality, introducing methods that wrap the hypervisor instead of sending specific commands [see example snippet #2, #3]. + This would eventually become a rather large API. libvirt API should be evaluated and if considered sound and appropriate for autotest, could be made similar. + Modify tests so that they make use of improved Monitor/VM API
4) Publish resulting API and best practices documentation so that new tests writers may submit
Other notes
- Regarding the prefix (virt_* ?), some functionality is not even tied to virtualization (such as the configuration format/parser) and could benefit other complex test modules;
- kvm_vm:
- There are significant differences in (KVM) VM class and (xen) XendDomain. Methods such as create() are quite different in parameters in implementation - (KVM) VM uses a 'monitor' to interact with qemu, whether (xen) XenDomain executes 'xm' commands - the 'monitor' abstraction is valid, but should have clear boundaries - xen autotest should also get a 'monitor' implementation - this even paves the way to a libvirt based monitor implementation - probably a capabilities mechanism would be necessary to acomodate diferences between virt technologies (see kvm_monitor below)
- kvm_monitor
- There are methods and parameters which do not map completely to Xen, specially PV guests (screendump, migrate with full disk copy, mouse move, etc) - Again, a capabilities mechanism could be used
Example #1 (pci_hotplug.py)
Parameters provide a lot of flexibility and diminish the need to implement a interface on a variant that does not need or cannot provide it. This code snippet:
26 # Modprobe the module if specified in config file
27 module = params.get("modprobe_module")
28 if module:
29 session.cmd("modprobe %s" % module)
could be kept exactly like that, loading 'acpiphp' as it does now for RHEL guests onon kvm and nothing for RHEL xen domU kernels (with XEN_PCIDEV_FRONTEND=y).
If someone wants to test a xen domU kernel with modular XEN_PCIDEV_FRONTEND, only this parameter would have to be adjusted.
Example #2 (pci_hotplug.py)
Only operations from the monitor side (vm.monitor.*) would have to be implemented for each virtualization technology. On the guest side (session.cmd_output() and the like), things could go unchanged, helped by parameters when necessary.
Curent code:
31 # Get output of command 'info pci' as reference
32 info_pci_ref = vm.monitor.info("pci")
33
34 # Get output of command as reference
35 reference = session.cmd_output(params.get("reference_cmd"))
Suggested code:
31 # Get output of command 'info pci' as reference
32 info_pci_ref = vm.monitor.get_pci_info()
33
34 # Get output of command as reference
35 reference = session.cmd_output(params.get("reference_cmd"))
Example #3 (pci_hotplug.py)
Current test code not only interacts directly with qemu but also deals with different versions and syntaxes:
41 # Probe qemu to verify what is the supported syntax for PCI hotplug
42 cmd_output = vm.monitor.cmd("?")
43 if len(re.findall("\ndevice_add", cmd_output)) > 0:
44 cmd_type = "device_add"
45 elif len(re.findall("\npci_add", cmd_output)) > 0:
46 cmd_type = "pci_add"
47 else:
48 raise error.TestError("Unknow version of qemu")
Again, this should be abstracted in the monitor implementation, and consumed by the test code (this is even commented in the current code):
59 # Execute pci_add (should be replaced by a proper monitor method call)
60 add_output = vm.monitor.cmd(pci_add_cmd)
61 if not "OK domain" in add_output:
62 raise error.TestFail("Add PCI device failed. "
63 "Monitor command is: %s, Output: %r" %
64 (pci_add_cmd, add_output))
65 after_add = vm.monitor.info("pci")
This should become:
59 # Execute pci_add
60 if not vm.monitor_pci_add(model):
61 raise error.TestFail("Add PCI device failed.")
62 after_add = vm.monitor.get_pci_info()
Library code under client/tests/kvm
The componentes are sorted by depencies, so that working down the list would solve all depencies, except when circular dependencies exist.
Filename Impact Dependencies Spec Tied Xen -------- ------ ------------ ---- ---- --- kvm_config.py Negligible kvm_utils (logging) No No Yes kvm_subprocess.py Negligible none No No Yes scan_results.py Negligible none No No Yes cd_hash.py Negligible kvm_utils (logging) No No No rss_file_transfer.py Negligible none No No No
kvm_monitor.py High kvm_utils (random_str) No Yes No[1] kvm_vm.py High kvm_utils No Yes Yes
kvm_subprocess kvm_monitor rss_file_transfer
kvm_scheduler.py Medium kvm_utils No[2] Yes[3] No kvm_preprocessing.py Medium kvm_vm No Yes[4] Yes
kvm_utils kvm_subprocess kvm_monitor
kvm_utils.py High kvm_subprocess No Yes[5] Yes
kvm_test_utils.py High kvm_utils No Yes Yes
kvm_vm kvm_subprocess scan_results
Caption:
- Spec: Specific for KVM (that is, not applicable to other virt technologies) - Tied: Currently tied to KVM in its current implementation - Xen: Currently used by xen autotest or has equivalent/derived implementation
[1] - Xen autotest currently does not implement a monitor, but this analysis recommends one to be implemented [2] - Currently xen autotest does not have a control file that uses parallel tests, so it's low priority but could be useful [3] - Tied only to kvm.VM and vm.monitor, used as abstractions [4] - Tied only to kvm.VM and vm.monitor, used as abstractions [5] - There's both kvm specific code and code that should be shared between xen and kvm autotest