Site Home Page
The UML Wiki
UML Community Site
The UML roadmap
What it's good for
Case Studies
Kernel Capabilities
Downloading it
Running it
Skas Mode
Incremental Patches
Test Suite
Host memory use
Building filesystems
User Contributions
Related Links
The HOWTO (html)
The HOWTO (text)
Host file access
Device inputs
Sharing filesystems
Creating filesystems
Resizing filesystems
Virtual Networking
Management Console
Kernel Debugging
UML Honeypots
gprof and gcov
Running X
Diagnosing problems
Installing Slackware
Porting UML
IO memory emulation
UML on 2G/2G hosts
Adding a UML system call
Running nested UMLs
How you can help
Kernel projects
A virtual network
An X session
A login session
A debugging session
Slackware installation
Kernel switches
Slackware README
ALS 2000 paper (html)
ALS 2000 paper (TeX)
ALS 2000 slides
LCA 2001 slides
OLS 2001 paper (html)
OLS 2001 paper (TeX)
ALS 2001 paper (html)
ALS 2001 paper (TeX)
UML security (html)
LCA 2002 (html)
WVU 2002 (html)
Security Roundtable (html)
OLS 2002 slides
LWE 2005 slides
Fun and Games
Kernel Hangman
Disaster of the Month

The UML test suite

The UML test suite is a set of Perl scripts which I use as a regression test on new UML patches. It comprises a test harness, which organizes and runs the suite, and the suite itself, which is a directory hierarchy which groups similar tests.

The tests to be done during a particular test run are specified by the parent directory, if you want all of the tests in that directory to run, an individual test, or any combination of directories and files.

The tests are able to specify the properties of the UML in which they are to run. The test harness compiles and runs UMLs according to the test requirements, trying to minimize the number of builds and boots required to satisfy all of the tests' requirements.

The tests vary widely in what they test and how they work. The simplest ones run a command (which they may have compiled themselves) and check the return value. Others run gdb and make it step through a test program to check that breakpoints work correctly. These use an already-booted UML, and don't need any special setup of the UML.

In contrast, there are tests which have requirements of the UML. For example, the boot_filesystems test checks that a user-specified set of filesystems boots correctly. So, it requests an unbooted UML for its use, but puts no more requirements on it. Others test various configurable options, and request a UML with certain options enabled, which they boot.

Finally, there are tests which stress the kernel generally in order to make sure that it is stable under a hard workload. The kbuild test, which runs a kernel build, is an example of this.
Why you should run it

The main reason would be to do a sanity-check on UML when compiled locally, with whatever unusual config options you favor, and run on your hardware. UML is somewhat sensitive to toolchain changes, and, to a less extent, to the host hardware. Users regularly shake out bugs that don't appear during development, and these are often exposed by different toolchains and hardware.

Secondly, if UML fails mysteriously somehow, a test may exercise the same bug. In this case, the test is likely to be much more specific than your workload, and the search for the bug much more constrained.

It would also be a help to the UML project in general to have the tests run as often as possible, and on as a great a variety of hosts as possible. This will expose bugs, and get them fixed, more quickly.

How to run it
First, download the latest version from the test suite section of the UML download page.
host% wget
Uncompress and extract it
host% bunzip2 uml_tests_20040726.tar.bz2
host% tar xf uml_tests_20040726.tar
Set up a ~/.umltest, which is the config file needed to tell the harness about the external resources, such as filesystems, and other information it will need in order to run. The file contains a perl hash, with different types of information under different keys. If you don't know perl, don't panic, because the syntax is simple enough. Mine looks like this:
    kernel_pool => "/home/jdike/linux/2.4/um",
    uml => {
        ubd => { 0 => "/home/jdike/roots/debian_22" },
        mem => "64M",
        cow_ubds => 1,
        prompts => { "/home/jdike/roots/debian_22" => "usermode:.*#" },
    tests => { "filesystems/boot_filesystems" => 
               { filesystems => [ "/home/jdike/roots/debian_22" ],
                 time => 0
               "stress/kbuild" => 
               { kernel_pool => "/home/jdike/roots/kernel",
                 time => 0 },

The kernel_pool tells the harness the location of a UML source pool. This will be where it builds whatever UMLs it needs.

The uml section specifies the default UML command. This will be used for any tests which don't have special requirements. The ubd hash within that is a mapping of device numbers to files, and this one will turn into "ubd0=/home/jdike/roots/debian_22" on the command line. The prompts hash specifies a regular expression which matches the shell prompts for the filesystems that will be booted.

The tests section provides test-specific information. The keys here are the filenames of the tests under the tests/ directory, and without the ".pl" extension. They can ask for arbitrary information here. I specify one filesystem for the boot_filesystems test to boot, and an image of a filesystem containing a kernel source tree for the kernel build test.

Now, the tests can be run as follows

host% perl
This will run the full test suite, reporting an overall status when it's done. A full log of the run will appear in tests.log. This will contain all output from the UMLs, including boot logs and anything the tests do.

Specific tests can be run by adding them to the command line

perl stress/kbuild
Adding a directory to the command line will cause all of the tests within the that directory to be run. So,
perl functional
will run all the tests under functional/, but nothing else.
How to add tests
The tests are all under the "tests" directory, each in a separate perl file. Here is a very simple test, commented:
# Virtually all tests will need these two modules.  The first defines
# the Test object, an instance of which is returned from here.  The
# second is a library of useful utilities which essentially all of the
# tests use.
use UML::Test;
use UML::Testlib;

# strict is good
use strict;

# This function is the actual test.  It will be run inside a booted
# UML, at the shell.  run_exit_0 is from TestLib.  It runs the dd
# command within the UML object given to the test, returning if the
# command had an exit status of 0, and dying if not.  In this case,
# the harness will trap that and report the failure.
sub run {
    my $test = shift;
    my $uml = shift;

    run_exit_0($uml, "dd if=/dev/mem of=/dev/null");

# This constructs the Test object and returns it.  This specifies that
# this test be run in a booted UML (state => "up") and that the test
# is contained in the 'run' function.
UML::Test->new(state => "up", run => \&run);

This illustrates the two essential features of a test
  • The function which executes the actual test
  • The construction of the test object, which specifies the test function and any requirements that the test makes on the UML.
The test can also request an unbooted UML with 'state => "up"', if the purpose of the test is to check that UML boots. It can also request a UML with special configuration options. The initrd test does this like so:
UML::Test->new(state => "up", 
               run => \&run, 
               kconfig => { MMAPPER => "y" },
               config => { "uml/iomem" => "mmapper,$file" } );

The kconfig option requests that it be given a UML with CONFIG_MMAPPER set to "y", and the config option adds 'iomem=mmapper,$file' to the UML command line.
Hosted at SourceForge Logo