Porting Linux KVM to FreeBSD (SoC 2007 Project)


Linux KVM is a Virtual Machine Monitor, part of the Linux kernel, that uses Intel VT-x or AMD-V extensions for x86 processors to create a full virtualization environment. This project will consist in porting Linux KVM to the FreeBSD kernel.

Since Linux KVM has a structure similar to that of a device driver (actually, it is a device driver, from many points of view,) core kernel changes will not be required, and the final product of this project will be an external loadable kernel module, exporting an interface based on ioctl() calls to a device descriptor. Part of the project will be also the porting of the userspace client for that interface, a modified qemu that uses KVM for guest execution.

Here you can find the complete submitted proposal.

Here you can find the patches submitted to Google for the Summer of Code evaluation process, while the full sources for the project at the end of the SoC (August 20th) are available here.

After the end of the SoC 2007 the development continued, and its webpage moved here.

Project Goals

First and foremost, this work will make it possible to use FreeBSD as a Virtual Machine Monitor for fully virtualized guests, and, in the future, para-virtualized ones.

There is also an additional benefit. In order to reduce the complexity of the work, we will try to import as much as possible from the existing Linux kernel KVM code, and enhance the linux-kmod-compat code (in /usr/ports/devel/) to implement the features required by the KVM code. As a consequence, the linux-kmod-compat improvements will be useful for other ports as well.

Project Status

In this section we list the main milestones of the project. A more detailed description can be found on this page, that contains links to the actual source code and full commit logs.

The SoC final result is somehow working, under restrictive conditions: you have to run an AMD processor (only SVM has been tested for lack of hardware and time) with an amd64 installation; only one client at time will (hopefully) work. The code is available via the git soc07-end tag or the submitted snapshot, anyway please consider that it is highly experimental and it may and probably will crash your system, it is provided only for documentation of the SoC work.

You should really use updated versions of the code, available on this page.

  • May 12th. Imported original Linux KVM sources and linux-kmod-compat code. This is the starting point of our work.
    From now on we will be trying to make the whole thing compile and link. The choice has been to stick with a KVM version (the one shipped within Linux 2.6.21) at least until we have something working.
    The final result will consist in three modules: the first one, lkvm, will contain architecture independent support, the other ones, mutually exclusive, vmx and svm, will contain architecture specific code.
  • May 13th. Compiled the whole thing as a single module.
    We had to take into account minor changes in CURRENT that slightly outdated linux-kmod-compat, and we added to the latter stubs for the Linux calls used in KVM. The KVM code also needed some kind of adaptation, mainly for C99 issues or conflicting symbol definitions between FreeBSD and Linux. Modules are still not splitted because this requires a little bit of support from linux-kmod-compat.
    So the next thing to work on will be module splitting and loading.
  • May 29th. Succeeded in loading the separate modules.
    To split the modules we had to use DECLARE_MODULE explicitly inside Linux source code and to exclude some USB glue code from compilation. Now the modules can be loaded, but of course they do nothing more than just register themselves.
    Now the funny part begins, since we have to fill in the stubs added to linux-kmod-compat trying to fool the KVM code about the system it is compiled for...
    The order we'll follow will be given by the code itself, working on functions in the order they are called in the normal usage patterns of KVM under Linux (i.e., first load lkvm, that implies executing its registration function and filling the stubs to let it work, then load one of the architectural modules and do the same, and so on.) This will be one of the biggest tasks of the project, so it will be splitted into subtasks.
  • June 26th. Imported the userspace client and started filling the module calls in the order they're actually used. To reach this point all the compatibility functions in the codepath from the module registration to the device creation and opening have been implemented in linux-kmod-compat; the work has been mostly on page allocations, MSRs and cpuid support, and SMP basic calls. Also miscdevice support has been added.
    WRT the userspace client we applied the patches of the qemu port to let it compile on FreeBSD, but we could not reuse the code of the port itself, since mainline qemu still does not contain the KVM code.
    From now on we'll just fill in the calls the client makes when loading a guest.
  • August 20th. Finally succeeded in having something working... We dealt mainly with differences in the way ioctl() parameters are handled in Linux and FreeBSD (Linux leaves the copyin/copyout to the implementor, while on FreeBSD this is done by the core kernel,) with some SMP/locking issues, such as mapping of Linux locking to FreeBSD and handling preemption enabling/disabling and with memory management. We had to extend the KVM interface to work around the fact that we could not hook into mmap() for the file descriptor we use to provide operations on virtual cpus.
    At this point SMP hosts crash unless the KVM client is bound to run only on one cpu.
    Anyway, many changes went on, and the final result is that a FreeBSD 7.0 guest has been booted on an amd64 UP SVM guest. All the work from now on will not be considered part of the Summer of Code 2007.

Last updated October 4th, 2007