KVM-Autotest-Docs

Introduction

Recent autotest supports passing test parameters to control file through --args options. This is also been introduced in kvm-autotest which can let user do some basic tests configurations (variants limitation/excluding, parameters configuration) through command line.

You should have some basic understanding of the test parameters and configuration file format before using the command line. If not, you are suggested to read the test configuration file format and test parameters before continuing.

How to use

It's very simple for you to use the command line, suppose you are in the directory of kvm subtest, you can pass test parameters through:

../../bin/autotest control --args="key1=value1 key2=value2 ...... keyN=valueN"

KVM-Autotest supports two kinds of parameters:

1. keywords "only" and "no": they serve the same function as in configuration file, limit and exclude variants.
   1. For each "only=x", the control file would generate a "only x" and pass it to the configuration file analyzer
   2. For each "no=y", the control would generate a "no y" and pass it to the configuration file analyzer
2. test parameters: all other are passed to the configuration analyzer untouched, that is to say, you could specify any test parameters by this way.

Limitations

The command line parameters interface can not provide the same flexibility as the configuration file and have the following limitations:

1. The generation of the final test matrix still depends on the configuration file, you still need it as a base to do the test.
2. As all test parameters passed from the command line and analyzed after the configuration file, any test parameters passed from cmdline would override ALL previous ones defined in configuration file. So the command line interface is better to be used to specify the frequently changed parameters, for the parameter which is not changed frequently, it's suggested to put them as a customized variant in configuration file.
3. It can not be used to adding variants, if you want them, you must edit the configuration file.

Examples

Suppose you use the default tests.cfg.sample as the tests.cfg. The first step is comment the last line as it has greatly reduced the test matrix which would block our configuration.

As the default test matrix too huge to run, so the second thing is to know the eight kinds of variants which could be used to reduce the matrix.

1. Operating Systems: We must specify the type of operating system of the image through which KVM-Autoest can decide the os-dependent parameters such as the type of shell client and others. You can take a look at test_base.cfg.sample to get the accurate names.
2. Test cases: The test case you want to run, see test_base.cfg.sample for details
3. Type of virtual network card: For example, you can use "only=e1000" in cmdline to test e1000 only. For expert, this can be overrided by "nic_model=XXX"
   1. variant name: rtl8139: Realtek 8139C
   2. variant name: virtio_net: Virtio Network Adapter
   3. variant name: e1000: Inter e1000 Adapter
4. Type of block device: For example, you can use "only=e1000" in cmdline to test e1000 only. For expert, this can be overrided by "nic_model=XXX"
   1. variant name: ide: IDE drive
   2. variant name: scsi: SCSI driver
   3. variant name: virtio_blk: Virtio Block Drive
5. Type of image format: For example, you can use "only=qcow2" to test qcow2 only, and if you want to test a pre-installed image XXX.qcow2, you should use "image_name=XXX only=qcow2". For expert, you can use "image_format=XXX" to override.
   1. variant name: qcow2: Qemu Copy-On-Write Version 2 image format
   2. variant name: raw: RAW format
   3. variant name: vmdk: VMWare Virtual Machine DisK image format
6. Num of vcpus, For example, you can use "only=smp2" to test 2 vcpus, and if you want to test more vcpus you can use "smp=$num_of_cpus"
   1. variant name: up: Only use one vcpu
   2. variant name: smp2: Use two vcpus
7. PCI Assignable Status, For example, you can use "only=no_pci_assignable" to test ordinary scenario. For expert, you can override this through "pci_assignable=XX"
   1. variant name: no_pci_assignalbe: Do not use device assignment in the test
   2. variant name: pf_assignable: Assign a pf to guest
   3. variant name: vf_assignable: Assign a vf to guest
8. Whether use hugepages
   1. variant name: smallpages: Do not use hugepages
   2. variant name: hugepages: Use hugepages

Only after the you've chose the above eight variants (either through configuration file or cmdline or both) can you get one test. For the variants which is not modified frequently, you'd better to put them in the configuration file.

If I have a pre-installed image (RHEL.5.5.x86_64 locate at /tmp/x.qcow2 with password XYZ, and I want to run a reboot test on it, how can I do this through command line?

../../bin/autotest control --verbose --args="only=full qemu_binary=/usr/bin/qemu-kvm only=boot only=rtl8139 only=smallpages only=ide image_name=/tmp/x only=qcow2 only=smp2 only=no_pci_assignable only=RHEL.5.5.x86_64 password=XYZ"

params	meaning
only=full	Use the variant full in tests.cfg which contains the biggest test matrix, this would let us to choose our desired configuration at will. You can put the parameters/variant which is not changed frequently in a customized variant and just use it through cmdline.
qemu_binary=/usr/bin/qemu-kvm	Specify the path of the qemu-kvm binary. If this path is fixed or you do not want to test other version of qemu, it's better to put this in you customized variants.
only=boot	Choose the testcase, only boot is tested
only=smallpages	Do not allocate hugepages for the guest.
only=ide	Choose the disk type, only use IDE drive.
image_name=/tmp/x only=qcow2	Specify the image name, use /tmp/x.qcow2 as the disk of guest
only=no_pci_assignable	Run guest without any assigned devices.
only=RHEL.5.5.x86_64	Choose the type of guest, 64bit RHEL5.5 is used.
password=XYZ	Override the default password.

TODO

1. support parameters surrounded by quotes
2. cli support for build?
3. auto completion?