User-mode Linux – A virtual OS

Not a virtual machine – no hardware emulation

although it is currently used solely as a VM

UML is intrinsically a guest OS

no attempt is made to boot an unmodified or lightly modified guest

Advantage – UML can be used to construct a virtualization toolkit

What is a virtualization toolkit

The user chooses

what to virtualize

where to virtualize it

What to virtualize?

scheduler

VM system

file system

entire OS

Example : virtual scheduler

Separate the scheduler from the rest of UML

Load it into the host kernel

UML

Host

Scheduler

Scheduler

Example : virtual scheduler

Guest scheduler is a process to the host scheduler

Processes loaded into guest scheduler

Group competes as a single process for host CPU

Partition processor by tying guest scheduler to it

Or not

Host

Scheduler

Virtualizing everything else

Guest VM system

Assign memory to it

Assign processes to it

Confined processes compete with each other for that memory

Guest network stack

Separate network node

Has its own ports, IP addresses, filtering, etc

In Short - Resource Control/Jailing

Resource control implemented by rearranging existing code

The key is that the virtualization has already been done, in the course of implementing UML

The next step is simply exposing this latent functionality

Userspace – Captive UMLs

Captive (or embedded) UML – a UML embedded into another application

Export application internals to the UML

control the application from the inside

export application data

import data from outside

Evolution with a captive UML

Evolution

Calendar

Tasks

Contacts

UML

/evolution/calendar

/evolution/tasks

/evolution/contacts

Scripts

Servers

Clients

email

bug

tracker

Virtualization Toolkit Chart

UML differs from other virtualization technologies in being a virtual OS rather than a virtual machine. The difference is that there is no hardware virtualization layer. Instead, the guest OS talks directly to the host OS, and is thus a “modified” guest.

The fact that UML knows how to be a guest enables it to be used as the basis of a virtualization toolkit.

A virtualization toolkit is a virtualization technology which allows you to choose what to virtualize and where to virtualize it in order to best fit your application.

Here, the units of virtualization are kernel subsystems, such as the scheduler, VM system, the file system. Any combination of these, including the entire kernel, should be available, and should work.

Here is an example, using the scheduler. We separate the (virtualized) scheduler from the rest of UML. This amounts to the scheduler, plus sleep and wakeup primitives. This, when loaded into the host kernel, will look like a single process, which competes for CPU time on an equal basis with the other processes on the system, things like priority and scheduling policy being the same.

Some host processes will then be moved from the host scheduler to the guest scheduler. They will then compete with each other for the slices of the host CPU that the group gets from the host scheduler.

You can then partition a single CPU by tying a group of guest schedulers to it, or partition the full host CPU resources by loading a number of guest schedulers and not tying them to processors.

Here's what it means to virtualize some other subsystems. A guest VM system would have some amount of memory assigned to it. Processes loaded into it would compete with each other for this memory, and their total memory usage would be limited to this pool.

The combination of a guest filesystem plus a guest VM system would limit the file caching to this pool, as well, making it a good compartment for something like updatdb.

A guest network stack would be a separate, independent network node. Processes confined to this would have access to a separate set of ports and its own IP address. They would share whatever filtering, QOS, TC, etc applied to the virtual interface in the host kernel.

Overall, this is a resource control and jailing mechanism. It has the advantage that there is nearly no new code that needs to be written. There need to be interfaces to loading guest subsystems and jailing processes within them, but the actual jails are already implemented. They were implemented in the course of writing UML. The subsystems need to be split apart, which possibly requires some code rearrangement, and ported into the kernel, which is in the works.

These steps will simply expose the functionality which is already latent.

Going back to userspace virtualization, we can ask what we can do there. The big idea here is captive UMLs, which are UML instances which are embedded in other applications. The value of this is that the UML would have access to the application's internal data, since it's in the application's address space. Since the UML is a full-blown Linux instance, potentially with access to the network, you can do things like control the application from the inside, export the data arbitrarily to the outside, and arbitrarily import data from the outside.

Here is an example, using evolution as the application into which UML has hypothetically been embedded. There is a /proc-like filesystem used to import evolution internal data into the UML. The calendar, tasklist, and contact list are imported into the UML as subdirectories /evolution/calendar, ../tasks, and ../contacts, respectively. So, each calendar entry would have a directory or file under ../calendar, and similarly for tasks and contacts under ../tasks and ../contacts. New entries or changes made through the GUI will appear immediately in this directory.

Within the UML, we can then run any sort of tools we want in order to look at and manipulate these entries. For example, by putting this UML on the network and running a mail server inside it would allow us to automatically convert incoming email (sent to tasks@evolution.mydomain.com) into tasks. Or we could use some scripts to tie the task list to a bug tracker. New bugs would be sent to the embedded UML and converted into tasks. Conversely, finished tasks would be sent back to the bug tracker as closed bugs. Another example would be running a web server inside the UML in order to make the application Web-accessible.

These sorts of integration and network- and group-enabling require only scripting inside the UML, and require no changes to the application itself.

This is a chart of the various switches that I am talking about being able to flip, and the applications that result.

With UML in the kernel, we have the various sorts of jailing, just as limited memory pools, partitioned CPUs, another chroot implementation, and a separate in-kernel network node. With a full UML loaded into the host kernel, you get full jailing of all resources, and a full virtual machine running inside the host kernel.

In userspace, you can run UML standalone or embedded in another application. A full UML is a captive UML with the capabilities I described in the evolution example. You can also embed subsystems, in which case you provide the application with the capabilities of the subsystem. The VM system provides a memory allocator with interesting properties. The scheduler would make a threads library. The network stack would make the application is separate network node, allowing it to participate on the network as a peer with the physical machines on the net. A full, standalone UML is the current, standard UML. I haven't talked about isolated, userspace subsystems, so I don't have these filled in, but I can see applications here, as well.

2005-02-13T12:51:25