uberbugs.

Every release, there's at least one bug that affects a significant number of users. Most of the time, we get lucky, and the users can get the system installed, and we can fix the problem in an update. The worse-case scenario however is the case where a bug prevents installation, or prevents booting post-install.

F10 had a bug just like this.

It manifested in several different ways.
For some people, it result in a 10 second pause during boot up. For others, it failed to find the root filesystem, even after waiting.

What happened here?

The initrd has code to wait for the kernel to finish enumerating devices. This is an important thing to do so we don't try anything silly like trying to mount the root filesystem before the driver for the disk controller has found the disks plugged into it.

In F9 and earlier, we would emit this code when the initrd included the scsi_mod module.

Late in the F10 development, we moved a bunch of things to be built-ins, to tweak the boot-up speed.
This turned out to be a really bad idea, as it meant the 'stabilization' code didn't get emitted. Ouch.

How come it didn't affect everyone? Different controllers behave differently. Some initialise so quickly that there's no reason to wait any longer.

Why didn't it get caught during testing? We did have a large number of reports, but the correlation wasn't made until post-release.

Should it have held up the release? Possibly, if we knew it would affect so many people.

FTR: None of the hardware I personally tested was affected by this problem, which made it a real pain to track down.

The good news is that in the hopefully not to distant future, all this code is going the way of the dodo anyway.

Syndicated 2008-12-16 20:15:18 from Dave Jones recollection of stuff that happened.