I had some communication with Blizzard about difficulties OLPC was having when booting off USB. A large part of it is down to usb-storage steadily getting worse and worse, as far as bootstrapping is concerned, because of improvements elsewhere in it.
The original, main problem is how the scanning for available devices is decoupled from the module initialization (for a reason). Thus, modprobe or a static initialization return before the scanning has completed or even started. All sorts of early userland adapted to this by waiting an arbitrary amount of time.
Over time we started to delay the scanning more and more. The khubd, SCSI EH, worker thread, and the ad-hoc scanning thread each add their own delays. Every new delay blows up the old userland. Worse, it only happens sometimes, when something goes wrong and SCSI recovery kicks in. I have to deal with this to support IBM Bladecenter, and I thought I would end with a heart disease. Now the OLPC folks are in the same boat with me.
I hinted Blizzard that it might make sense just to switch to ub for OLPC and thus solve these problems, but he ignored me. I looked at public information about OLPC closer, and I think I know why. A USB stack comes into play twice: first time it's run by firmware, and second time it's run by OS. Neither is likely to use ub.
At the second stage, OLPC people won't deconfigure SCSI completely, in case it's needed for some unspecified use in the future. Although there's no sensible reason now, maybe some tinpot dictator in the Middle East would strip his kids from OLPC kits, make a huge Beowulf cluster for calculating his nukes' performance, and boot them off iSCSI over IPv6. Hey, it can happen. So they want to keep SCSI. But if both ub and usb-storage are in use, there's an additional complexity, which is addressed clumsily by the current Linux, so they won't do it.
Looking at the first stage, I thought that at least LinuxBIOS was a perfect fit, but no, it's not the case. As it turns out, LinuxBIOS actually does not boot squat in case of USB (or so it seems from the mailing lists). Instead, it loads standalone booloaders, which actually boot things. I saw some console traces which showed usb-storage being used, but that code seems to be just a transplant, and a temporary one. The OLPC Hardware Spec page says: "LinuxBIOS is our BIOS for production units; Open Firmware is used as the bootloader."
Now that I wrote this down, I'm going to forget it all ASAP and go hack on Xen.