Advogato blog for jct

19 Feb 2001

Mon, 19 Feb 2001 17:50:49 GMT

ODS version 0.9 is almost ready to release! I had planned to get it out this weekend, but had my plans changed for me. Here's the story, as written up for a non-technical friend:

The weekend got off to a great start. I finally figured out the problem with the new network and had it functioning as well as could be expected. The parts had arrived earlier in the week: a new 100 megabit network hub, and three network cards to go in the PCs. I had put the new cards in the computers, connected the cables to the new hub... but it wouldn't work... at least not at the 100 megabit speed, though it would still work at the slower 10 megabit speed. At the higher speed, I couldn't even ping one machine from the other.

Friday night I had finally figured out the problem: it was my homemade network cables. I had been crimping the connectors onto the cables the wrong way. Once I had this figured out, I got out my knife and crimping tool, snipped the old connectors off all the cables, and crimped new connectors on them all. I didn't have that many to recrimp, but it took me most of the night because I'm slow. I was especially glad when the cable I had run through the attic, connecting the server in my study to the PC in the play room, finally worked correctly. Pulling that cable through the attic had been a real headache, and I wasn't looking forward to having to replace it.

I was up until the wee hours Friday, but when I was done, the network was working well. All the cables had been tucked neatly away. The test cables, including the temporary that had been laying stretched on the floor between my study and the playroom, were tidily coiled and stowed away for future use. And the new network hub was sitting in its new home on the shelf, warm and cozy, its "100" lights all lit up and blinking happily as the computers talked. I went to bed sometime around 3:00 AM, feeling good.

During the night, disaster struck.

I got up Saturday morning expecting finish writing the manual for my camera software so I could finally release it to the world. But my server was in an odd state: system load was extremely high, and it was unresponsively slow. It appeared to respond to mouse movement, but I couldn't bring up a process list to see who was hogging the CPU. Finally, I switched to the console window and saw an endless list of disk errors spewing out the error log. Uh-oh.

Trying hard not to panic, I attempted to shut down the system, but with the disk not working, I couldn't shut it down normally. I had to just turn the power off. Now, this is a bad thing to do on a Windows box, and it's even worse on a linux machine, but I had no choice. Things weren't looking good.

They got even worse when I tried to restart the machine. One of the disk drives wouldn't respond at all. It was a 9 Gigabyte Ultra-2 SCSI drive: relatively new, the fastest disk in my system. I tried repeatedly to get the drive online, and as I did, I began to get that sick feeling in the pit of my stomach. The drive wouldn't respond.

Eventually I experienced a brief moment of hope as I got the drive to talk to the SCSI controller again. It wasn't dead, it was just badly injured! If I could get the drive online again, even for just a little bit, I could dump its contents onto tape and rescue my information before the drive died complelety. I fumbled through my tape bin looking for some empties to dump onto. But my efforts were in vain: the drive wouldn't go back online. It would talk to the SCSI controller, but could only say that it had an internal error. Its injuries had left it brain-dead, and I would get nothing more out of it.

At this point, panic began to look more and more like a viable option. Trouble was, I wasn't exactly sure what to panic over. On Windows system, you have C: and D: drives, etc. These drive letters usually correspond one-to-one to the separate pieces of hardware installed in your machine. Your A: drive is your floppy. If you have one hard drive, it's your C: drive. If you have a second hard drive, it's your D: drive. If you have a CDROM, it'll then be E: drive.

On any type of unix system, these distinctions are hidden from you; all the different drives are merged seamlessly into one huge filesystem. You don't see the boundaries between one disk and the next, You don't think in terms of which "drive" your information is on, you just think about its directory -- its location in the hierarchy. Eventually you tend to forget about the fact that all this information is really stored on separate pieces of hardware; it's just one large happy collection of information.

This is fine for day-to-day work, but it's an incredible hindrance when you want to panic. I couldn't remember exactly what had been stored on that 9 GB disk drive. I wanted to run about the house screaming "oh my god I've lost all my ________", but I wasn't sure whether to fill in the blank with "e-mail" or "software" or "snapshots" or "porn" or "secret Nixon audio tapes" or what!

Eventually, I began to settle down and think again. I booted my linux rescue CD, and so I could look at the remaining good disk drive and figure out, through process of elimination, what I had lost. I got the good drive mounted, looked through its contents, and found my home directory. So I hadn't lost any e-mail. I had lost my operating system and my archive of digital photos. Brain beginning to function again, I remembered that I had a tape machine, so I probably had backups. Checking the shelf, I found backup tapes. I was finally beginning to see a way out of all this mess.

I won't talk you through the rest of it step by step, but suffice to say that I spent most of the next 24 hours just bootstrapping a running system. I found two unused old disk drives, and I had to take my system almost completely apart to extract the old drive and find places for the two replacements. I was able to recover a functional operating system from tape; my most recent backups were from July of last year, but not much had changed since then, so my system is almost fully back to normal.

That was Saturday. In getting the new system running again, I'd needed access to the innards of the system, and the best way was just to lay the drives out on the floor, connect their cables, and let them run that way. When I finally got my system up and running again, having worked through the night, it was 7AM and I was too tired to reassamble it. Sunday I spent reassembling my system and cleaning up the mess.

The most critical item on the dead drive is the lost collection of digital photos because they're irreplaceable. The backup tape is from July of last year. Fortunately, I haven't taken many photos since then, and what photos I have taken are still in the camera. So it looks like I might not have lost much... assuming my backup tape is good. I'll try to recover those sometime this week.

18 Dec 2000

Mon, 18 Dec 2000 07:45:49 GMT

Vacation! Three glorious weeks of nothing to do but sit around the house and work on personal projects. No emulator bugs to fix, no new emulator features to think about. Actually, it's now two weeks of vacation; I spent the first week trying to finally sleep off this nagging cold (it's still with me) and generally being lazy.

Like I do for most vacations, I have made myself a long list of things that urgently need doing and, like most vacations, the majority of them will go undone. I've started on a couple of the projects: the update of jimthompson.org and the release of ODS. I expect to get ODS out before Christmas; likewise, my web site should soon be updated. The other major home project is to lay Pergo in my study - it's the only room of the house to escape the recent reflooring - and Wendy is not going to let it go undone. We'll be buying the materials next week, and will probably lay the new floor on Friday.

* * *

As I've been updating my web site, I've been using a variety of browsers to look at the new code (HTML4/CSS2, as generated by PHP3) to see how the different browsers render it. The list I've used so far is:

Netscape 4.x
Mozilla M17
Mozilla M18
Netscape 6.0
Internet Explorer 4.5
Internet Explorer 5.x
Opera 4.0 Beta
Lynx 2.8.2

All of the above, except for the IE browsers, I run under linux. I test my pages, both my HTML and my CSS, against the validators run by the W3C, so I know that both my HTML and my style sheets are legal, and should produce similar results, at least among the more recent browsers. Here are some things I have learned about each browser:

Netscape 4: It's not surprising that this venerable old browser does the worst layout job on my pages. But I suspect that, at least among the linux/unix crowd, it's still a widely used browser so it's good to know how my pages look in it. Not that Netscape 4 does a bad job, but it ignores some of the margin properties in the CSS, which tend to make its layout look very different than most of the other browsers. This layout can be corrected by using Netscape-specific attributes in the <body> tag, but then the HTML doesn't pass the W3C validator. I'd rather be standards-complient than pretty.

Mozilla: These three browsers (M17, M18, and Netscape 6) seem to do as good a job as any on my pages. Best as I can tell, layout is exactly as it should be and except for variations in font size, layout seems to be exactly as in the MSIE browsers (this is a good thing).

I like Netscape 6 very much; it seems to be the best yet of the Mozilla series. I would throw away my old Netscape 4 were it not for a very troubling problems. The most annoying is with printing; for some reason, the Mozilla print engine has real trouble laying out the page correctly when preparing PostScript for the printer. It seems to have the most trouble on pages that use both tables and graphics (in other words, most every page). I've printing several articles off the net in which the first page of the printed copy consists of nothing but a single banner ad, with the text following on subsequent pages. Why is layout for a printer so different than layout for the screen? Netscape 6 also seems to have some form-related problems with Advogato. Together these problems mean I won't be uninstalling Netscape 4 any time soon.

Internet Explorer: I'm no fan of Microsoft, but I have to say that its MSIE browsers are some of the best around. Both 4.x and 5.x seem to be fast and easy to use (if not exactly stable). I had heard that one or the other of these browsers had "broken" CSS support, but I haven't seen any sign of breakage. Both MSIE browsers render my pages exactly right so far as I can tell.

Opera 4.0 Beta: I probably wouldn't be testing with Opera except that I had heard such good things about it, so when a new beta version was recently announced on Slashdot or Freshmeat, I downloaded a copy. What a huge disappointment! It doesn't handle my CSS correctly, specifically in margin values and fonts. I specified my font family as "helvetica, ariel, sans-serif" which pretty much covers all worlds, but Opera insists on rendering my pages in a serif font, probably Times.

I also don't like its cluttered UI (all the browser makers need to learn that sometimes less is more). I especially loathe its windowing system in which documents float within a virtual desktop internal to Opera, in much the same way that documents float around within a Microsoft product like Word. Note to Opera: the application shouldn't be doing window managament - that's the Window Manager's job.

Needless to say, I'll be uninstalling my copy of Opera's browser the minute its 30-day eval license expires

Lynx: Good old depenable lynx. I don't use it on a day-to-day basis, but it somehow makes me feel good to know it's out there, still alive and healty and being kept up to date. My pages won't look beautiful in lynx, but they will at least be usable.

- = -

7 Dec 2000

Thu, 7 Dec 2000 15:20:13 GMT

Finally! It looks like I may have fixed the emulator so that it will correctly run the OI29 software, in particular the GPCF stuff available through the 70-series SPEC pages. SPEC 71 is the one that was causing the most trouble, but it now appears to run correctly.

We found the problem yesterday, with the help of Roscoe and Paul from Flight Software. When we found the problem, though, it was late in the day so I left the fix for this morning. The fix turned out to be related to my earlier "fix", in which I realized that the intermediate addresses for the indirect addressing mode need to be expanded from 16 bits to 19 bits just like the final addresses are expanded.

When I added the code to expand these intermediate addresses, I made a real dumb mistake: I casted the 19-bit expanded address back down to an unsigned short, thus throwing away 3 bits of address. D'oh! Anyway, SPEC 71 no longer hoses the emulation, and I'm going to look at other places we expand address to see if I have any other address-truncation problems.

-=-

7 Dec 2000

Thu, 7 Dec 2000 01:24:30 GMT

Spent the last couple of days tracking down a bug in the emulator; today I even had a couple of flight software guys helping me out. Their insight was invaluable. Without knowing how the low-level I/O routines work, I never would have figured it out. The problem turns out to be with the same addressing mode as the first I fixed. One of the AP101/S addressing modes with indexed, indirect, and its own flowchart to show how it works. This mode is so complicated that I might never get it completely right; maybe when we get the microcode listings I'll be able to do better.

-=-

5 Dec 2000

Tue, 5 Dec 2000 09:40:53 GMT

Note to jules, who observed that Solaris users seem to dislike bash because it's not installed by default:

I discovered, much to my surprise, that bash is installed by default on Solaris 2.8. I was mucking around with my PATH the other day, and I took all the GNU stuff out (why, I don't remember), thinking that at my next login my shell would fall back to csh. But it didn't. I then discovered that bash resides in /usr/bin/bash. This was on a Sparc Ultra 10, right out of the box, default OS install, and still had that "new computer" smell.

Once I worked with a fellow who would have used bash as his shell, but the OS, via chsh(1), wouldn't allow it as a logon shell. He was a sharp guy, but he didn't try very hard to solve this one. It's pretty easy to make bash your default shell even on an uncooperative OS like Solaris using a bit of script like the following in your login script:

  export SHELL=/usr/local/bin/bash
  if [ -x $SHELL ]; then
    exec env SHELL=$SHELL $SHELL -login
  fi

The "env" in there is hanging around from when I used this snippet with csh. I changed the "if" syntax, but left the env. In ksh, I guess this would be simpler as "SHELL=$SHELL exec $SHELL -login".

I digress. To get back to the point, bash is available on newer Sun systems. Don't remember about less, but I do seem to remember gzip being on my new sun too. Even if bash weren't installed by default, what's the problem with building it? I've never had a problem with bash in six or seven years of use; "./configure; make" has always worked just fine for me.

-=-

4 Dec 2000

Mon, 4 Dec 2000 23:53:18 GMT

My beloved Auburn Tigers played in the SEC championship game in Atlanta Saturday where they fell again to Florida. But just getting the chance to play there was quite an accomplishment for coach Tuberville in only his second year. We'll be expecting even bigger things over the next couple of years, Coach.

Next we play the Michigan Wolverines in the Florida Citrus bowl; I was in school at Auburn the last (and, I think, only) time we played Michigan. That was in the Sugar Bowl with Pat Dye's first SEC champion ship team, in his third year as coach. It was Bo Jackson's sophomore year, but the game hero was senior Al Del Greco, who kicked a field goal on the last play of the game to give Auburn a 9-7 win. (Yes, that's the same Al Del Greco who kicked the winning field goal for the Tennessee Titans yesterday against the Philadelphia Eagles in Philly.)

* * *

Why is it so hard to get a gcc/gdb combination that works correctly on Solaris 2.8? I spent nearly two days trying to get a compiler that will produce code and a debugger that will report correct line numbers.

I'm no rookie at this; I've been building gcc for as long as I remember, but I've never had this kind of trouble. I tried gcc versions 2.7, 2.80, 2.90, and 2.95 (minor versions omitted), and gdb 4.15, 4.16, 4.17, 4.18, and 5.0. I also tried building gcc with and without the GNU binutils, versions 2.9 and 2.10. What eventually worked for me was binutils 2.9, gcc 2.95, and gdb 4.18. I know I must have been doing something stupid, but I never figured out exactly what.

Just as troubling is the problem on IRIX. The commercial, proprietary debugger on that system, for which we pay a metric buttload of money, barely works. It can attach to the process and look through memory, but it can't single-step through code. What a useless piece of junk! Unfortunately, gdb won't work with pthreads programs on IRIX, so it can't help us on that platform.

* * *

We have problems running the emulator on the new OI29 software. Last week, I found and fixed an addressing-mode problem, but there's at least one other problem that keeps us from running SPEC 71, the new GPCF screen. I'll keep plugging at it, now that I've got my debugger up and running again.

* * *

The ODS release is overdue. I have most of the code functioning, but I haven't finished writing the new man page, or the release notes. I hope to have time before the end of the year.

-=-

3 Dec 2000

Sun, 3 Dec 2000 03:03:43 GMT

Do not adjust your set. This is a Streaming Freedom video bulletin. The cable hack will last exactly sixty seconds; it cannot be traced. It cannot be stopped. And it is the only free voice left in this city.

So goes the tagline of a rogue journalist in the TV series Dark Angel. The character represents the voice of freedom (free as in "speech") in the struggle against a corrupt, dysfunctional society. It's an interesting parallel: his fiction and our reality in the OSS struggle against proprietary software. The resemblance is only skin deep, but compelling nevertheless.

* * *

I discovered Advogato some time ago, but have neglected to register until now. I have my own online diary software: some PHP3 code I threw together some time ago. My software is modeled after sites like everything.org and wiki-wiki-web in that it automatically generates hyerlinks between entries (a feature that might be cool to see here at Advogato). But my software is also immature, so I'm going to give Advogato a try.

I used to attempt keeping notes in a file in my home directory: kind of like a changelog with some simple formatting convetions. I used a small Emacs mode I wrote to help automate the editing. It worked very well until about five years ago when I started using linux. Until then I only had one account on one unix system, which made keeping track of my log file no problem at all - it was always there when I needed it in the same file in the same directory.

When I started using linux on my home PC, I used the same software to keep the same kind of log on my home system. At first, this worked: work-related notes on the system at work, personal notes on the personal system at home. Pretty quickly, though, the system broke down: I often think about work while I'm at home, and vice-versa. (Yeah, I'm not supposed to think about personal stuff on the clock, but the thoughts come unbidden. Try not to think about purple elephants, and you'll see what I mean.) My notes started to cross-pollinate between the two log files, and pretty soon it was hard to tell one from the other.

To make matters worse, I changed jobs to one that requires me to work on several different systems. And just last year, I bought a laptop, one more system to throw into the mix. My notes, thoughts, to-do items were now scattered across several files on several systems. I could still record my thoughts anytime I needed, but going back and finding them later became a real problem - often the entry I was seeking wasn't on the same system I was using, and then I had to go into hunt mode.

I suppose I'm probably preaching to the converted here: who hasn't had a similar problem? I thought my Palm organizer might help matters - it solved a similar problem I had keeping track of phone numbers. But great though the Palm is, it just isn't great at writing notes onto.

Now that the Internet is everywhere, it's clearly a great potential candidate to solve this kind of problem: put the the notes online. Store them in a database. Enable me to retrieve them, add new ones, edit old ones, etc., from any computer connected to the net.

I've been using my own PHP journal pages, but they're not as mature yet as the Advogato diary system. The Advogato system is obviously very simple, and not very feature-burdened, but sometimes less is more. I really like its ease of use. I like its fast-loading design. I like its clean, uncluttered look. Some new features would be nice, and who knows, maybe I'll take the Advogato code and add my own features. That is how the whole OSS thing works, right?

Stay tuned. More about ODS, exif-tools, and linux digital photography still to come.