Older blog entries for ciphergoth (starting at number 1)

8 Jul 2002 (updated 9 Jul 2002 at 13:42 UTC) »
This diary

While I keep a LiveJournal diary, I don't want to fill it with geek content since it doesn't mostly have geek readers. So I thought I'd move that sort of thing to here.

Backups

I've been trying to find a way to back up my system onto multiple CD-Rs. Several people have basically told me that every solution out there is woefully inadequate, no-one has suggested anything good, and my searches haven't found anything good. I got quite interested in DAR for a while, but in practice its multi-archive handling is too far from the Good Thing to be worth the bother.

I've been thinking about what the Right Thing should look like. One thing it should be able to do is write each "slice" to a pipe rather than a file. This is more general than calling a hook after a slice is complete; you catch the "beginning-of-slice" as well as "end-of-slice" events, you can do arbitrary processing on the content, and you get to stop and start the archive writing process with simple flow control.

Reading the "afio" manual page, many of the options could be removed if this were possible. I sent some mail about this to the DAR maintainer; here's my discussion about how this feature could help you do backups if you've only the disk space to store a little over one slice at a time

Isn't using flow control much simpler? It would be very easy to write another general tool that takes a filename on the command line, and writes stdin to that file, and when a write fails because the disk is full, it waits a little while and tries again. dar (or dar_xform) would write to this process. If the disk filled up, dar would simply block during a write to the pipe, because pipes are flow controlled.

While dar is blocked, another process might be busy taking a completed slice file and writing it to CD, deleting it once it's written and verified.

Once the old slice has been deleted, space will appear, and the writing of the new slice will start to work again. dar won't even notice that there was a pause, and no signals-related complexity is needed.

And all of this happens outside dar itself. In accord with the UNIX philosophy, a whole lot of tools that do one thing well come together to perform a more complex task in a very configurable way. All of this is only possible with the "pipes" approach.

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!