Anatomy of a Fedora 17 ISO image
My spare time has been limited lately, so progress on producing Mac-bootable Fedora install images has been a bit slow. Thankfully it looks like everything's gong to land in time for Fedora 17[1] so hurrah for that - all that's missing right now are the last couple of patches that make sure the boot picker displays useful labels on the drives.
So how is this accomplished? Here's a hex dump of the first few K of the image.
00000000 45 52 08 00 00 00 90 90 00 00 00 00 00 00 00 00 |ER..............|
00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000020 33 ed fa 8e d5 bc 00 7c fb fc 66 31 db 66 31 c9 |3......|..f1.f1.|
00000030 66 53 66 51 06 57 8e dd 8e c5 52 be 00 7c bf 00 |fSfQ.W....R..|..|
00000040 06 b9 00 01 f3 a5 ea 4b 06 00 00 52 b4 41 bb aa |.......K...R.A..|
00000050 55 31 c9 30 f6 f9 cd 13 72 16 81 fb 55 aa 75 10 |U1.0....r...U.u.|
00000060 83 e1 01 74 0b 66 c7 06 f1 06 b4 42 eb 15 eb 00 |...t.f.....B....|
00000070 5a 51 b4 08 cd 13 83 e1 3f 5b 51 0f b6 c6 40 50 |ZQ......?[Q...@P|
00000080 f7 e1 53 52 50 bb 00 7c b9 04 00 66 a1 b0 07 e8 |..SRP..|...f....|
00000090 44 00 0f 82 80 00 66 40 80 c7 02 e2 f2 66 81 3e |D.....f@.....f.>|
000000a0 40 7c fb c0 78 70 75 09 fa bc ec 7b ea 44 7c 00 |@|..xpu....{.D|.|
000000b0 00 e8 83 00 69 73 6f 6c 69 6e 75 78 2e 62 69 6e |....isolinux.bin|
000000c0 20 6d 69 73 73 69 6e 67 20 6f 72 20 63 6f 72 72 | missing or corr|
000000d0 75 70 74 2e 0d 0a 66 60 66 31 d2 66 03 06 f8 7b |upt...f`f1.f...{|
000000e0 66 13 16 fc 7b 66 52 66 50 06 53 6a 01 6a 10 89 |f...{fRfP.Sj.j..|
000000f0 e6 66 f7 36 e8 7b c0 e4 06 88 e1 88 c5 92 f6 36 |.f.6.{.........6|
00000100 ee 7b 88 c6 08 e1 41 b8 01 02 8a 16 f2 7b cd 13 |.{....A......{..|
00000110 8d 64 10 66 61 c3 e8 1e 00 4f 70 65 72 61 74 69 |.d.fa....Operati|
00000120 6e 67 20 73 79 73 74 65 6d 20 6c 6f 61 64 20 65 |ng system load e|
00000130 72 72 6f 72 2e 0d 0a 5e ac b4 0e 8a 3e 62 04 b3 |rror...^....>b..|
00000140 07 cd 10 3c 0a 75 f1 cd 18 f4 eb fd 00 00 00 00 |...<.u..........|
00000150 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
This is the boot sector. One of the fun things about ISO9660 is that it doesn't have a boot sector in the usual x86 sense - firmwares are expected to read enough of the filesystem that they can find the bootloader in it and run it directly. x86 isn't really smart enough to manage that, so it uses something called El Torito[2]. In any case, this is usually empty space on an x86 CD. But with an increasing number of systems shipping without optical drives, real CD installs are decreasing in popularity. We use a piece of software called
isohybrid that allows the building of ISO9660 images that can be written directly onto a USB stick. Asking a machine to boot off them will execute the boot sector located here, which then loads a real bootloader that's capable of reading ISO and booting the OS. It'll just be ignored when it's on a real CD.
There's one difference here, though. The isohybrid bootsector has been modified so that the first 32 bytes are just noops, and this has been inserted:
00000000 45 52 08 00 00 00 90 90 00 00 00 00 00 00 00 00 |ER..............|
45 52 ("ER") indicates the presence of an Apple partition map. This will become important later. 0800 indicates that it's using 2048-byte sectors. The 9090 is a lie to convince the firmware that it's not a zero-sized disk without representing dangerous code. We need an Apple partition map because some older Macs won't boot off CD unless the CD has one. But it's sitting right at the start of the boot sector, and this code is going to be executed. Thankfully, the entire Apple header decodes to
00000000 45 inc bp
00000001 52 push dx
00000002 0800 or [bx+si],al
00000004 0000 add [bx+si],al
00000006 90 nop
00000007 90 nop
00000008 0000 add [bx+si],al
0000000A 0000 add [bx+si],al
0000000C 0000 add [bx+si],al
0000000E 0000 add [bx+si],al
00000010 0000 add [bx+si],al
00000012 0000 add [bx+si],al
00000014 0000 add [bx+si],al
00000016 0000 add [bx+si],al
00000018 0000 add [bx+si],al
0000001A 0000 add [bx+si],al
0000001C 0000 add [bx+si],al
0000001E 0000 add [bx+si],al
which is completely harmless - we can execute all these instructions without any interesting changes in state. They'll all be undone by the rest of the boot sector.
000001b0 14 05 00 00 00 00 00 00 4c 1e ed 76 00 00 80 00 |........L..v....|
000001c0 01 00 00 3f a0 89 00 00 00 00 00 50 14 00 00 fe |...?.......P....|
000001d0 ff ff ef fe ff ff a4 00 00 00 70 04 00 00 00 fe |..........p.....|
000001e0 ff ff 00 fe ff ff 44 05 00 00 c0 08 00 00 00 00 |......D.........|
000001f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 aa |..............U.|
This is the MBR partition table. The aim here was to produce media that would boot via BIOS just as well as it does via EFI, and it turns out that there are some systems that refuse to BIOS-boot off GPT media. So we need an MBR partition map. The first entry covers the entire disk and is of type 0. This is important, because some EFI implementations are strict about MBR parsing - if there's an MBR with overlapping partitions, the entire MBR will be ignored. Inconvenient. Fortunately, we've got the source code to this code, and it turns out that partitions of type 0 are ignored when performing this check. So, there's a partition of type 0.
Why does it cover the entire disk? Once we've booted the OS, we need to be able to read the ISO image contained on the USB stick. That means the kernel needs to be able to mount it, which means we need a partition entry that covers the entire disk. Linux is perfectly happy mounting a filesystem from a partition of type 0.
So, the next partition. This is an EFI boot partition that points at the embedded VFAT El-Torito image. Some firmware will decide that we've got an MBR and so will only boot off a partition that exists in it. So, here's an MBR partition to keep them happy.
The final partition covers the embedded HFS+ image. It's there purely for convenience - it means we can access it under Linux in order to update its contents for testing purposes.
00000200 45 46 49 20 50 41 52 54 00 00 01 00 5c 00 00 00 |EFI PART....\...|
00000210 09 27 93 7e 00 00 00 00 01 00 00 00 00 00 00 00 |.'.~............|
00000220 fe 4f 14 00 00 00 00 00 30 00 00 00 00 00 00 00 |.O......0.......|
00000230 de 4f 14 00 00 00 00 00 2a 26 0f 37 23 c9 7f 4f |.O......*&.7#..O|
00000240 90 df 5e fe 0a 60 1c b0 10 00 00 00 00 00 00 00 |..^..`..........|
00000250 80 00 00 00 80 00 00 00 ec 70 63 c1 00 00 00 00 |.........pc.....|
00000260 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
And here we have the GPT header. Nothing complicated here, other than it indicates that the first partition descriptor is at LBA 0x10, ie byte 0x2000 - that's further into the disk than normal. That leaves us enough space to fit the Apple partition entries.
Why have a GPT at all? The EFI spec says that machines should boot fine from an MBR partition. Sadly, not all seem to. It also lets us represent the HFS+ partition in a way that makes the Mac firmware happy.
00000800 50 4d 00 00 00 00 00 03 00 00 00 01 00 00 00 10 |PM..............|
00000810 41 70 70 6c 65 00 00 00 00 00 00 00 00 00 00 00 |Apple...........|
00000820 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000830 41 70 70 6c 65 5f 70 61 72 74 69 74 69 6f 6e 5f |Apple_partition_|
00000840 6d 61 70 00 00 00 00 00 00 00 00 00 00 00 00 00 |map.............|
00000850 00 00 00 00 00 00 00 0a 00 00 00 03 00 00 00 00 |................|
00000860 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
2K into the disk, here's an Apple partition map. It's 2K into the disk because we set the sector size to 2K in the partition map header. Why did we do that? Because the default is 512 bytes, and if we'd put the partition map at 512 bytes it would have been on top of the GPT header. Why 2K rather than 1K? A couple of reasons. First, the Apple partition map is only used when we boot off an actual CD, and the CD sector size is genuinely 2K. Secondly, because CDs have a sector size of 2K, 2K is a value that's actually been tested in the real world. It's nice to avoid tempting fate.
00001000 50 4d 00 00 00 00 00 03 00 00 00 29 00 00 04 70 |PM.........)...p|
00001010 45 46 49 00 00 00 00 00 00 00 00 00 00 00 00 00 |EFI.............|
00001020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00001030 41 70 70 6c 65 5f 48 46 53 00 00 00 00 00 00 00 |Apple_HFS.......|
00001040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00001050 00 00 00 00 00 00 04 70 00 00 00 33 00 00 00 00 |.......p...3....|
00001060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00001800 50 4d 00 00 00 00 00 03 00 00 01 51 00 00 08 c0 |PM.........Q....|
00001810 45 46 49 00 00 00 00 00 00 00 00 00 00 00 00 00 |EFI.............|
00001820 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00001830 41 70 70 6c 65 5f 48 46 53 00 00 00 00 00 00 00 |Apple_HFS.......|
00001840 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00001850 00 00 00 00 00 00 08 c0 00 00 00 33 00 00 00 00 |...........3....|
00001860 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
The Apple partition map entries. One partition covers the entire disk, the other the embedded HFS+ filesystem. Why do this? Because not all Macs understand EFI El Torito, so won't EFI boot a CD unless there's an Apple partition map. Why have an HFS+ partition at all? Because the same Macs won't boot off FAT.
00002000 a2 a0 d0 eb e5 b9 33 44 87 c0 68 b6 b7 26 99 c7 |......3D..h..&..|
00002010 d0 84 a5 d4 aa 89 e0 40 81 f9 fc e4 04 e9 c2 99 |.......@........|
00002020 00 00 00 00 00 00 00 00 10 4b 14 00 00 00 00 00 |.........K......|
00002030 00 00 00 00 00 00 00 00 49 53 4f 48 79 62 72 69 |........ISOHybri|
00002040 64 20 49 53 4f 00 49 53 4f 48 79 62 72 69 64 00 |d ISO.ISOHybrid.|
00002050 41 70 70 6c 00 00 00 00 00 00 00 00 00 00 00 00 |Appl............|
00002060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00002080 a2 a0 d0 eb e5 b9 33 44 87 c0 68 b6 b7 26 99 c7 |......3D..h..&..|
00002090 68 d5 e0 27 72 83 c6 4c a7 ae 92 dd ed 6c 89 71 |h..'r..L.....l.q|
000020a0 a4 00 00 00 00 00 00 00 13 05 00 00 00 00 00 00 |................|
000020b0 00 00 00 00 00 00 00 00 49 53 4f 48 79 62 72 69 |........ISOHybri|
000020c0 64 00 41 70 70 6c 65 00 41 70 70 6c 00 00 00 00 |d.Apple.Appl....|
000020d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00002100 00 53 46 48 00 00 aa 11 aa 11 00 30 65 43 ec ac |.SFH.......0eC..|
00002110 68 d5 e0 27 72 83 c6 4c a7 ae 92 dd ed 6c 89 71 |h..'r..L.....l.q|
00002120 44 05 00 00 00 00 00 00 03 0e 00 00 00 00 00 00 |D...............|
00002130 00 00 00 00 00 00 00 00 49 53 4f 48 79 62 72 69 |........ISOHybri|
00002140 64 00 41 70 70 6c 65 00 41 70 70 6c 00 00 00 00 |d.Apple.Appl....|
00002150 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
Three GPT entries - one covers the entire CD, one covers the embedded FAT filesystem, one covers the embedded EFI filesystem. That ensures that they're all accessible to the firmware.
00008000 01 43 44 30 30 31 01 00 4c 49 4e 55 58 20 20 20 |.CD001..LINUX |
00008010 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 | |
00008020 20 20 20 20 20 20 20 20 46 65 64 6f 72 61 2d 4c | Fedora-L|
00008030 69 76 65 43 44 20 20 20 20 20 20 20 20 20 20 20 |iveCD |
00008040 20 20 20 20 20 20 20 20 00 00 00 00 00 00 00 00 | ........|
00008050 c4 12 05 00 00 05 12 c4 00 00 00 00 00 00 00 00 |................|
00008060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00008070 00 00 00 00 00 00 00 00 01 00 00 01 01 00 00 01 |................|
00008080 00 08 08 00 40 00 00 00 00 00 00 40 15 00 00 00 |....@......@....|
00008090 00 00 00 00 00 00 00 17 00 00 00 00 22 00 1d 00 |............"...|
000080a0 00 00 00 00 00 1d 00 08 00 00 00 00 08 00 70 05 |..............p.|
000080b0 01 09 30 36 f0 02 00 00 01 00 00 01 01 00 20 20 |..06.......... |
000080c0 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 | |
The ISO9660 superblock. This is completely standard. From here on there's nothing terribly surprising - the CD filesystem itself is entirely normal. The only slight oddity is that we have three embedded El Torito images. The first of these is a BIOS-bootable disk image. That'll be executed whenever the CD is put in a non-EFI machine. The second is a VFAT EFI boot image. That's used for the majority of EFI systems. The third is an HFS+ EFI boot iamge. There's no strict requirement for the HFS+ one to be an El Torito image - CD booting on Macs is already handled by the Apple partition map. Storing it as an El Torito is purely for convenience, since it guarantees correct alignment and avoids us having to parse the entire ISO9660 image to find its location when generating the partition maps.
In summary: Three partition maps, three bootable images, support for BIOS, UEFI and Mac platforms, works whether it's burned to a CD or written directly to a USB stick. I'm pretty pleased with that.
[1] Other than an awkward bug where something goes horribly wrong during Radeon graphics init, so Radeon-based Macs aren't working too well at the moment. We're working on it.
[2] The world decided that writing a real ISO9660 driver for BIOS was hard, so came up with El Torito. El Torito is a spec for embedding disk images inside ISO9660. There's a pointer to the disk image in the Cd header, so the BIOS simply reads a few blocks off CD, finds that pointer and then sets up a bunch of disk i/o interrupts to point at the embedded filesystem rather than a real disk. From that point on, the El Torito image simply behaves like a floppy or hard drive. This was a reasonable compromise given how resource limited older BIOS implementations were.
For reasons that aren't entirely clear, UEFI doesn't mandate that implementations support ISO9660. So even though our firmware is now sufficiently capable that the only thing standing between it and emacs is someone being sufficiently bored, we still use El Torito. Fortunately the spec allows multiple El Torito images to be embedded, so we can include one that's bootable by BIOS systems and another that's bootable by UEFI systems.
comments
Syndicated 2012-05-02 13:52:20 from Matthew Garrett