KVM Autotest Refactor page
KVM autotest refactor
Suggested approach
1) Move code that has negligible impact [see table below] from 'client/tests/kvm' to 'client' ASAP
+ This would immediatelly benefit xen autotest, since it is frequently lagging behing features and fixes. One example is newer syntax in kvm_config.py that was not available in the xen autotest counterpart.
2) Synchronize that code (step 1) usage between kvm and xen autotest
+ Rewrite current tests, changing imports of modules from 'kvm_*' and 'xen_' to a common prefix ('virt_*' ?).
+ This can be done by an automated script and verified by the unit tests.
3) Work gradually on code that would generate more impact, implementing abstractions and other mechanisms
+ The config file and thus the params passed to tests already add a lot of flexibility [see example snippet #1]
+ Improve monitor functionality, introducing methods that wrap the hypervisor instead of sending specific commands [see example snippet #2, #3].
+ This would eventually become a rather large API. libvirt API should be evaluated and if considered sound and appropriate for autotest, could be made similar.
+ Modify tests so that they make use of improved Monitor/VM API
4) Publish resulting API and best practices documentation so that new tests writers may submit
Other notes
* Regarding the prefix (virt_* ?), some functionality is not even tied to virtualization (such as the configuration format/parser) and could benefit other complex test modules;
* kvm_vm: - There are significant differences in (KVM) VM class and (xen) XendDomain. Methods such as create() are quite different in parameters in implementation - (KVM) VM uses a 'monitor' to interact with qemu, whether (xen) XenDomain executes 'xm' commands - the 'monitor' abstraction is valid, but should have clear boundaries - xen autotest should also get a 'monitor' implementation - this even paves the way to a libvirt based monitor implementation - probably a capabilities mechanism would be necessary to acomodate diferences between virt technologies (see kvm_monitor below)
* kvm_monitor - There are methods and parameters which do not map completely to Xen, specially PV guests (screendump, migrate with full disk copy, mouse move, etc) - Again, a capabilities mechanism could be used
Example #1 (pci_hotplug.py)
Parameters provide a lot of flexibility and diminish the need to implement a interface on a variant that does not need or cannot provide it. This code snippet:
26 # Modprobe the module if specified in config file 27 module = params.get("modprobe_module") 28 if module: 29 session.cmd("modprobe %s" % module)
could be kept exactly like that, loading 'acpiphp' as it does now for RHEL guests onon kvm and nothing for RHEL xen domU kernels (with XEN_PCIDEV_FRONTEND=y).
If someone wants to test a xen domU kernel with modular XEN_PCIDEV_FRONTEND, only this parameter would have to be adjusted.
Example #2 (pci_hotplug.py)
Only operations from the monitor side (vm.monitor.*) would have to be implemented for each virtualization technology. On the guest side (session.cmd_output() and the like), things could go unchanged, helped by parameters when necessary.
Curent code:
31 # Get output of command 'info pci' as reference 32 info_pci_ref = vm.monitor.info("pci") 33 34 # Get output of command as reference 35 reference = session.cmd_output(params.get("reference_cmd"))
Suggested code:
31 # Get output of command 'info pci' as reference 32 info_pci_ref = vm.monitor.get_pci_info() 33 34 # Get output of command as reference 35 reference = session.cmd_output(params.get("reference_cmd"))
Example #3 (pci_hotplug.py)
Current test code not only interacts directly with qemu but also deals with different versions and syntaxes:
41 # Probe qemu to verify what is the supported syntax for PCI hotplug 42 cmd_output = vm.monitor.cmd("?") 43 if len(re.findall("\ndevice_add", cmd_output)) > 0: 44 cmd_type = "device_add" 45 elif len(re.findall("\npci_add", cmd_output)) > 0: 46 cmd_type = "pci_add" 47 else: 48 raise error.TestError("Unknow version of qemu")
Again, this should be abstracted in the monitor implementation, and consumed by the test code (this is even commented in the current code):
59 # Execute pci_add (should be replaced by a proper monitor method call) 60 add_output = vm.monitor.cmd(pci_add_cmd) 61 if not "OK domain" in add_output: 62 raise error.TestFail("Add PCI device failed. " 63 "Monitor command is: %s, Output: %r" % 64 (pci_add_cmd, add_output)) 65 after_add = vm.monitor.info("pci")
This should become:
59 # Execute pci_add 60 if not vm.monitor_pci_add(model): 61 raise error.TestFail("Add PCI device failed.") 62 after_add = vm.monitor.get_pci_info()
Library code under client/tests/kvm
The componentes are sorted by depencies, so that working down the list would solve all depencies, except when circular dependencies exist.
Filename Impact Dependencies Spec Tied Xen
------ ------------ ---- ---- ---
kmv_config.py Negligible kvm_utils (logging) No No Yes kvm_subprocess.py Negligible none No No Yes scan_results.py Negligible none No No Yes cd_hash.py Negligible kvm_utils (logging) No No No rss_file_transfer.py Negligible none No No No
kvm_monitor.py High kvm_utils (random_str) No Yes No[1] kvm_vm.py High kvm_utils No Yes Yes kvm_subprocess kvm_monitor rss_file_transfer
kvm_scheduler.py Medium kvm_utils No[2] Yes[3] No kvm_preprocessing.py Medium kvm_vm No Yes[4] Yes kvm_utils kvm_subprocess kvm_monitor kvm_utils.py High kvm_subprocess No Yes[5] Yes
kvm_test_utils.py High kvm_utils No Yes Yes kvm_vm kvm_subprocess scan_results
Legend:
- Spec: Specific for KVM (that is, not applicable to other virt technologies) - Tied: Currently tied to KVM in its current implementation - Xen: Currently used by xen autotest or has equivalent/derived implementation
[1] - Xen autotest currently does not implement a monitor, but this analysis recommends one to be implemented [2] - Currently xen autotest does not have a control file that uses parallel tests, so it's low priority but could be useful [3] - Tied only to kvm.VM and vm.monitor, used as abstractions [4] - Tied only to kvm.VM and vm.monitor, used as abstractions [5] - There's both kvm specific code and code that should be shared between xen and kvm autotest