Advogato blog for jhb

24 Jul 2001

Tue, 24 Jul 2001 23:45:22 GMT

Gee, been a long time since I added a diary entry here. Such is life I suppose. My last entry noted that I was starting at BSDi/WC in the following week. Since then things have changed slightly since BSDi's software side was acquired by WindRiver after a somewhat bumpy ride, and today I type this entry while people run around packing up the last of their things in preparation for the move to WRS's Alameda facility tomorrow. Most of the engineering group from the Concord office has already moved, but a few of us held out until tomorrow's move. Personally, I wanted to keep my 2 minute reverse commute for as long as possible.

As far as my work in FreeBSD, it has changed rather dramatically in the past year or so. Last year I was a documentation committer sneaking into the coding side of the tree by way of the x86 bootstrap code and scheduled to work on the ia64 bootstrap. A few weeks after I arrived at Concord, most of the engineering project left for the big SMP meeting to talk about FreeBSD's SMP next generation project to multithread our kernel. I didn't attend the meeting, read the summary e-mail on the list with little interest, and didn't bother to read any of the slides posted.

About three months later, I was stuck working on a relatively boring contract job when Greg Lehey wandered into the BSD coding IRC channel asking for help with some problems he was having getting interrupt threads stable on a UP x86 kernel. Once I finally got the contract job done (somewhat time-crunched due to working on the SMP stuff every once in a while) I turned to the SMPng project full time and ported the new ithreads to SMP x86 and to the alpha. Since then I have been working on the SMPng project full time in various areas of the kernel. Rest assured that I am not doing this alone however (thank goodness). I suppose one's diary can give a false exaggeration of one's own accomplishments relative to other people's. Anyways, several things have changed in the FreeBSD kernel since the SMP project have changed, and several more changes are slated to come. However, there is the "small" roadblock of 5.0 release scheduled for late November. Couple this with the fact that despite all the progress that has been made, the kernel is still about 95+% under the Giant lock, and it seems that 5.0 is scheduled for a performance nightmare. While people seem to accept that 5.0 will suck, they really want 5.1 to not suck quite so badly, and for 5.2 to be both stable and at least as decent performancewise as 4.x. While I can agree that these are reasonable requests from our customers point of view, I am afraid that there is just not enough manpower and time to get there. However, those release dates are still quite a ways off, and we might be able to pull it off yet.

For those curious about the current work on SMPng, I'm presently working on making the kernel fully preemptive. That is, we will preempt to a process in setrunqueue if it is a higher priority process. This is rather different from the standard behavior of the Unix kernel, but this is the direction most (if not all) successful multithreaded Unix kernels have gone. It also greatly simplifies some pieces of code with respect to synchronization primitives and removes several opportunities for programmer error.

As far as non-code work, I'm planning on giving a paper at BSDCon that serves as a tutorial/guide on how to do data locking to help educate the developerbase about the new world order in the FreeBSD kernel. I will also probably use this paper to flesh out the Locking Notes chapter in the FreeBSD Developer's Handbook. Also, I'm planning on giving a talk about the Witness lock order verifier at the Usenix ATC in 2002. I may attempt to port it to userland as a wrapper around pthreads, but I'm not sure about that yet.

24 May 2000

Wed, 24 May 2000 12:43:55 GMT

Well, it has been a while since I've added an entry, so I'll start off with several updates.

First, I did get the USB install stuff finished and committed to -current. I hope to MFC it in time for 4.1 release. Currently, we have problems with USB keyboards during install as follows:
- USB keyboards on the x86 arch can't be used in installation because the kernel userconfig only works with AT keyboards. The solution is to make userconfig MI and move it into the loader, which is now a WIP.
- USB keyboards on the alpha arch should work, except that I haven't enabled USB in the GENERIC kernel yet, and SRM doesn't support a USB keyboard.
PXE support is now done and committed to both -current and -stable. We used a polling method of reading packets, removing the need for ISR hooks in BTX, so the support for ISR's in BTX was dropped.
Automagical LBA/EDD support in the bootstrap is still on the todo list.
I graduated May 13 (11 days ago).

Now that school is over, it's time to start work. I've got a job with BSDi out at the WC office (former home of WC CDROM) and will be starting that in a couple of weeks. I'll be spending most of next week driving across the country to get there. Once I do get there, I still have to find an apartment, and will then turn around and head down to Usenix with the rest of the gang.

In other FBSD news, we now have the latest and greatest version of binutils, including a gas(1) that actually groks 16-bit x86 assembly. As such, I've been on a rampage recently going through and cleaning up all the old 16-bit code in the bootstrap to be actual assembly now w/o faking the arguments or using M4 macros to work around deficiencies.

28 Mar 2000

Tue, 28 Mar 2000 04:52:40 GMT

Well, I finally got my final, cleaned-up version of the PXE boot loader for FreeBSD tested. With that out of the way, I committed all my current patches to the boot loader, cleaning up some things, and adding PXE support. Next up on the todo list for the loader is to add support in the BTX environment for hooking hardware interrupts. Client apps will register an ISR that I call at PL3 like the rest of the application. This way the ISR can be written in C and use the same vm86 entry points as the rest of the loader. I also have another set of patches to look at to detect the int 13 EDD extensions and turn on LBA support automatically in the loader. I plan on merging similar functionality into the other stages of the boot process so that LBA support will no longer have to be manually enabled after installation.

In other news, my latest USB-install release is broken due to silliness on my part. The end result is a kernel that has no console device and thus can't perform any I/O when it tries to boot. Now that I have tracked this silly typo down, I am waiting for the recent world breakge to be fixed so that I can roll another test release. Thankfully, almost all of my changes to enable USB support in the installation floppies can be merged easily into the main installation floppies. However, USB keyboards do not work with the kernel's userconfig utility. Thus, the INSTRO_USERCONFIG option cannot be compiled into the USB boot floppy's kernel. As a result, it looks like we will have to have a seperate kern.flp image for systems with USB keyboards until userconfig can be moved out of the kernel and into the loader.

47 days to graduation now; I can't wait.