25 Apr 2005 jpick   » (Master)

Hey Stuart, good luck with the cancer thing.

I've been toying around with lots of neat stuff lately.

I finally tried out Scratchbox. It's sort of an interesting way of making a cross-compiler look like a native-compiler by running it inside a custom build environment. So it's a little different than building with a cross compiler, and a bit different than building natively on the target machine. I maintain a cross-compiler at work (cross-building Fedora RPMs), so I'm interested in this type of stuff. I wanted to play with cross-compiling to PowerPC, but they only had ARM ones - so I'd have to build my own.

Everybody seems to be writing a revision control system / SCM nowadays. Even Linus is getting into the act. Matt Mackall at work just made one that he's calling Mercurial. I tried it out, and it seems pretty cool, even though it's in the early days. I like some of his ideas.

I stumbled across the OceanStore distributed storage thing yesterday. This really has me excited. I highly recommend reading the paper. I've been excited about distributed hash tables since I was playing with Freenet and Frost about two years ago. Freenet is all about anonymity and non-censorability - it's great at that, but the performance sucks. Oceanstore is about performance and being able to do filesystem/database things.

Anyways, the OceanStore source code has been released on SourceForge under a BSD license, it's written in Java, and it's got some very interesting things in it. It's built around using Matt Welsh's SEDA programming model (Sandstorm is the core). You might know Matt Welsh's work from NBIO (the predecessor to NIO), or his "Running Linux" book from O'Reilly (my first Linux book).

One application is a global-scale distributed NFS implementation (ulnfs). This is something I could use right now. Theoretically I should be able to set up a node at home, and a node at work, and use nfs to read/write to it and get good performance in either place. A side-effect of using a DHT is you also can do time-travel on the data. If this works, it would be great for providing nfs-mounted /home directories on geographically distributed Xen sessions, or for mounting the whole root partition, and using it for hot-spares or clustering. And you get massive redundancy and bittorrent-like distribution for free. It sounds pretty fast - in the paper, they claim they got 4.6x faster than NFS for reading, and 7.3x slower than NFS for writing.

There are also some other cool looking applications, like an IMAP server, a webmail client, and a Palm-pilot synchronizer, with all the storage done in the DHT cloud. Very cool.

Some of the cool related projects are the Bamboo DHT, and OpenDHT. OpenDHT is cool - if you need some place to stuff some bits for a week, you could put it on their 200-300 servers scattered around the planet (for research, of course)

Now all I have to do is figure out how to get it all working. The code is all dumped out there, but you have to assemble the parts yourself, and there are a lot of them. I've now got enough Xen sessions and machines scattered all over the place that I can probably really test this stuff out.

Latest blog entries     Older blog entries

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!