Advogato: Blog for etbe

2 May 2008 etbe » (Master)

The Future of Xen

I’m currently in Xen hell. My Thinkpad (which I won’t replace any time soon) has a Pentium-M CPU without PAE support. I think that Debian might re-introduce Xen support for CPUs without PAE in Lenny, but at the moment I have the choice of running without Xen or running an ancient kernel on my laptop. Due to this I’ve removed Xen from my laptop (I’m doing most of my development which needs Xen on servers anyway).

Now I’ve just replaced my main home server. It was a Pentium-D 2.8GHz machine with 1.5G of RAM and a couple of 300G SATA disks in a RAID-1. Now it’s a Pentium E2160 1.8Ghz machine with 3G of RAM with the same disks. Incidentally Intel suck badly, they are producing CPUs with names that have no meaning, and most of their chipsets don’t support more than 4G of physical address space [1]. I wanted 4G of RAM but the machine I was offered only supported addressing 4G and 700M of that was used for PCI devices. For computation tasks it’s about the same speed as the old Pentium-D, but it has faster RAM access, more RAM, uses less power, and makes less noise. If I was going to a shop to buy something I probably would have chosen something different to get support for more than 4G of RAM, but as I got the replacement machine for free as a favor I’m not complaining!

I expected that I could just install the new server and have things just work. There were some minor issues such as configuring X for the different video hardware (and installing the 915resolution package (which is only needed in Etch) to get the desired 1650×1400 resolution. But for the core server tasks I expected that I could just move the hard drives across and have it work.

After the initial install the system crashed whenever I did any serious hard drive access from Dom0, the Dom0 kernel Oopsed and network access was cut off from the DomU’s (I’m not sure whether the DomU’s died but without any way of accessing them it doesn’t really matter much). As a test I installed the version of the Xen hypervisor from Unstable and it worked. But the Xen hypervisor from Unstable required the Xen tools from Unstable which also required the latest libc6, and therefore the entire Dom0 had to be upgraded. Then in an unfortunate accident unrelated to Xen (cryptsetup in Debian/Unstable warns you if you try to use a non-LUKS option on a device which has been used for LUKS and would have saved me) I lost the root filesystem before I finished the upgrade.

So I did a fresh install of Debian/Unstable, this time it didn’t crash on heavy disk IO, instead it would lock up randomly when under no load.

I’ve now booted a non-Xen kernel and it’s working well. But this situation is not acceptable long-term, a large part of the purpose of the machine is to run virtualisation so that I can test various programs under multiple distributions. I think that I will have to try some other virtualisation technologies. The idea of running KVM on real servers (ones that serve data to the Internet) doesn’t thrill me, Tavis Ormandy’s paper about potential ways of exploiting virtual machine technologies [2] is a compelling argument for para-virtualisation. Fortunately however my old Pentium-3 machines running Xen seem quite reliable (replacing both software and hardware is a lot of pain that I don’t want).

In the near future I will rename the Xen category on my blog to Virtualisation. For older machines Xen is still working reasonably well, but for all new machines I expect that I will have to use something else - and I’ll be blogging about the new machines not the old. I expect that an increasing number of people will be moving away from Xen in the near future. It doesn’t seem to have the potential to give systems that are reliable when running on common hardware.

Ulrich Drepper doesn’t have a high opinion of Xen [3], the more I learn about it the more I agree with Ulrich.