29 Oct 2008 jnewbigin   » (Apprentice)

How to run a 64 bit kernel on 32 bit CentOS 4

Imagine that you have CentOS-4 i686 installed. Your CPU can run x86_64 and you decide you would like to use the 64 bit kernel.

The reason I wanted to try this was that I have a server with 6 Gig of ram and it was suffereing from lowmem starvation wich in turn was triggering the OOM killer.

Is this supported by Red Hat/CentOS/Anyone?
No. This is just my own experiment.

They said you can’t do this!
I am trying to prove that (wrong it seems).

Why would you do this anyway?
The 64 bit linux kernel can run 32 bit linux applications.
Even though 32 bit applications are still restricted to 3Gig of RAM each, the kernel will be free from the 1Gig limit which applies to the 32bit kernel.

The main problem I found was running 32bit programs from initrd. I don’t know why this is a problem but I do know how to fix it.

This process seems to work for me. It is just an experiment and should not break your 32bit setup. You should however do a backup before you start!

The process
Step 1: Download the following packages (or the latest release of each package):
kernel-2.6.9-78.0.5.EL.x86_64.rpm (This is the most important one)
mkinitrd-4.2.1.13-1.x86_64.rpm (for nash)
lvm2-2.02.37-3.el4.x86_64.rpm (for lvm)
udev-039-10.22.el4.x86_64.rpm (for udev)
module-init-tools-3.1-0.pre5.3.10.x86_64.rpm (for insmod)
iptables-1.2.11-3.1.RHEL4.x86_64.rpm (for iptables)
glibc-2.3.4-2.41.x86_64.rpm (to run iptables)

Step 2: Install the kernel
rpm -i kernel-2.6.9-78.0.5.EL.x86_64.rpm –ignorearch
Ignore the errors from the strip command. We will fix that soon.

Step 3: Extract tools for the initrd
cd /tmp
mkdir initrd
for i in mkinitrd-4.2.1.13-1.x86_64.rpm udev-039-10.22.el4.x86_64.rpm lvm2-2.02.37-3.el4.x86_64.rpm module-init-tools-3.1-0.pre5.3.10.x86_64.rpm; do
(cd initrd ; rpm2cpio $i | cpio -idv )
done

Note: I am using /tmp because this is just a test. If you want these files to stay you should put them somewhere else.

Step 4: Edit /tmp/initrd/sbin/mkinitrd:
Change “inst /sbin/nash” to “inst /tmp/initrd/sbin/nash”
Change “inst /sbin/insmod.static” to “inst /tmp/initrd/sbin/insmod.static”
Change “inst /sbin/udev.static” to “inst /tmp/initrd/sbin/udev.static”
Change “inst /sbin/lvm.static” to “inst /tmp/initrd/sbin/lvm.static”
Change “-x /usr/bin/strip” to “! -x /usr/bin/strip”

Step 5: Make the new initrd
/tmp/initrd/sbin/mkinitrd -v -f /boot/initrd-2.6.9-78.0.5.EL.img 2.6.9-78.0.5.EL

Step 6: Reboot
You should now be able to boot into the 64 bit kernel and load up to a more or less working system. Once you get this far there are a few more things you might want to tweak.

Step 7: haldaemon
haldaemon was printing some errors. I don’t want to use it so I turned it off
chkconfig haldaemon off

Step 8: glibc
In order to run some 64bit applications we need the 64bit libs.
cd /tmp/initrd
rpm2cpio glibc-2.3.4-2.41.x86_64.rpm | cpio -idv
cd /
ln -s /tmp/initrd/lib64 .

Step 9: Magic script
What instructions would be complete without some magic? I created a wrapper script which will invoke the correct binary based on the kernel arch.
make a command called /sbin/archrun
#!/bin/bash
exec $0.$(/bin/uname -m) “$@”

chmod a+x /sbin/archrun

Step 10: Fix modprobe
mv /sbin/modprobe /sbin/modprobe.i686
ln -s /sbin/archrun /sbin/modprobe
cp /tmp/initrd/sbin/modprobe /sbin/modprobe.x86_64

Step 11: Fix iptables
mv /sbin/iptables /sbin/iptables.i686
ln -s /sbin/archrun /sbin/iptables
cp /tmp/initrd/sbin/iptables /sbin/iptables.x86_64
mv /sbin/iptables-restore /sbin/iptables-restore.i686
ln -s /sbin/archrun /sbin/iptables-restore
cp /tmp/initrd/sbin/iptables-restore /sbin/iptables-restore.x86_64

Check with:
service iptables start

Step 12: Quota
If you use quota, you need the 64 bit tools. I don’t use quota so I didn’t do this.

There may be other steps depending on what you run on your system. The steps for fixing them will probably be the same as iptables/modprobe.

This procedure is not automatically repeatable when updates are released. It changes things without the knowledge of RPM which is not very good. It does not use rpm to track the 64 bit applications. They are saved on /tmp etc. These issues would probably need to be addressed if you were going to think about running the 64bit kernel long term.

So how did it work out?
Well I would say that it was a resounding success. I can now run vmware and LVM2 snapshots without the lowmem starvation.

I have not done any performance tests bit it does not seem to be any different.

Syndicated 2008-10-29 03:53:00 from John's test blog

Latest blog entries     Older blog entries

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!