KVM-unit-tests

kvm-unit-tests

git://git.kernel.org/pub/scm/virt/kvm/kvm-unit-tests.git

Introduction

kvm-unit-tests is a project as old as KVM. As its name suggests, it's purpose is to provide unit tests for KVM. The unit tests are tiny guest operating systems that generally execute only tens of lines of C and assembler test code in order to obtain its PASS/FAIL result. Unit tests provide KVM and virt hardware functional testing by targeting the features through minimal implementations of their use per the hardware specification. The simplicity of unit tests make them easy to verify they are correct, easy to maintain, and easy to use in timing measurements. Unit tests are also often used for quick and dirty bug reproducers. The reproducers may then be kept as regression tests. It's strongly encouraged that patches implementing new KVM features are submitted with accompanying unit tests.

While a single unit test is focused on a single feature, all unit tests share the minimal system initialization and setup code. There are also several functions made shareable across all unit tests, comprising a unit test API. The setup code and API implementation make up the kvm-unit-tests framework, which is briefly described in the next section. We then describe testdevs, which are extensions to KVM's userspace that provide special support for unit tests, in the "Testdevs" section. Section "API" lists the subsystems, e.g. MMU, SMP, that the API covers, along with a few descriptions of what the API supports. It specifically avoids listing any actual function declarations though, as those may change (use the source Luke!). The section "Running tests" gives all the details necessary to build and run tests, and the final section "Adding a test" provides an example of adding a test.

Framework

The kvm-unit-tests framework has the following components 1) test building support, 2) shared code for test setup and API, and 3) test running support. The framework also supports multiple architectures; currently supporting i386, x86_64, armv7 (arm), and armv8 (arm64). Support for ppc64[le] is in progress. Test building support is done through makefiles and some supporting bash scripts. Test setup code includes, for example, early system init, MMU enablement, and UART init. The API provides some common libc functions, e.g. strcpy, atol, malloc, printf, as well as some low-level helper functions commonly seen in kernel code, e.g. irq_enable/disable(), barrier(), and some kvm-unit-tests specific API for, e.g., installing exception handlers and reporting test success/failure. The test running support is provided with a few bash scripts, using a unit tests configuration file as input. Generally tests are run from within the source root directory using the supporting scripts, but tests may optionally be built as standalone tests as well. More information about the standalone building and running is in the section "Running tests".

Testdevs

Like all guests, a kvm-unit-test unit test (a mini guest) is run not only with KVM, but also with KVM's userspace. It's useful for unit tests to be able to open a test specific communication channel to KVM's userspace, allowing it to send commands for further setup. In particular this channel is useful for initiating an exit, i.e. to quit the unit test. Testdevs fill these roles. The following are testdevs currently in QEMU

isa-debug-exit

x86 device that opens an I/O port. When the I/O port is written it induces an exit, using the value written to form the exit code. Note, the exit code written is modified with "(code << 1) | 1". Thus, a unit test exiting with zero for success induces an exit from QEMU with 1.

pc-testdev

x86 device that opens several I/O ports, where each port provides an interface to a helper function for the unit tests. One such function is interrupt injection.

pci-testdev

A PCI "device" that when read and written helps test PCI accesses.

testdev

An architecture neutral testdev that takes its commands in postfix notation over a serial channel. Unit tests add an additional serial channel to their unit test guest config (the first being for test output), and then bind this device to it. kvm-unit-tests has minimal support for virtio in order to allow the additional serial channel to be an instance of virtio-serial. Currently testdev only supports the command "codeq", which works exactly like the isa-debug-exit testdev.

Running tests

Here are a few examples of building and running tests

Run all tests on the current host

$ git clone git://git.kernel.org/pub/scm/virt/kvm/kvm-unit-tests.git
$ cd kvm-unit-tests/
$ ./configure
$ make
$ ./run_tests.sh

Cross-compile and run with a specific QEMU

$ ./configure --arch=arm64 --cross-prefix=aarch64-linux-gnu-
$ make
$ export QEMU=/path/to/qemu-system-aarch64
$ ./run_tests.sh

Run a single test, passing additional QEMU command line options

$ ./arm-run arm/selftest.flat -smp 4 -append smp

Note1: run_tests.sh runs each test in $TEST_DIR/unittests.cfg (TEST_DIR, along with some other variables, is defined in config.mak after running configure)
Note2: When a unit test is run separately, all output goes to stdout. When unit tests are run through run_tests.sh, all output is saved to test.log

Building and running standalone tests

$ make standalone
$ tests/hypercall # example run of a standalone test

Tests may be installed with 'make install', which copies the standalone version of each test to /usr/local/share/kvm-unit-tests/

Adding a test

1. Create the new unit test's main code file

$ cat > arm/new-unit-test.c
#include <libcflat.h>

int main(int ac, char **av)
{
    report("hello!", true);
    return report_summary();
}

2. Add a line like the following to the appropriate config/ .mak file

$(TEST_DIR)/new-unit-test.elf: $(cstart.o) $(TEST_DIR)/new-unit-test.o

And also add $(TEST_DIR)/new-unit-test.flat to the tests-common makefile variable. Note, the tests-common variable identifies tests shared between common architectures, i.e. i386, x86_64 and arm, arm64. Use the tests makefile variable of a specific architecture's makefile to build the test specifically for that architecture.

3. You can now build and run the test

$ make
$ arm/run arm/new-unit-test.flat 

Contributing

To contribute new tests and enhancements and fixes to the framework, please submit patches to the KVM mailing list with the additional tag 'kvm-unit-tests' in the subject, i.e. [kvm-unit-tests PATCH]