Older blog entries for itamar (starting at number 23)

My first article in a series on protocol design went up. In between contract work, free software (Python VoIP phone, woo), and other stuff, I am also looking for a full time job - read my resume.

Just got back from Usenix 2003. The paper glyph and I wrote is available: Networking For The Rest Of Us, as are the slides. I'm still not sure we got the point across well, but the points I realized I wanted to add (configuration as scripting, ability to encode common configurations in a deployment generator, unified authentication backends) wouldn't have fit in a 30 minute talk anyway. Some of this arose from the time Glyph spent presenting Twisted to various sysadmins, and our plans to reimplement the authentication infrastructure.

The conference itself, while small, was extremely interesting and educational.

Our power grows!

Mystery Robot is written with Twisted. Mystery Robot is an implementation of Nambers, a way of making IP addresses easy to remember.

Entrance of the Central Scrutinizer, from Frank Zappa's "Joe's Garage" album:

This is the CENTRAL SCRUTINIZER...it is my responsibility to enforce all the laws that haven't been passed yet. It is also my responsibility to alert each and every one of you to the potential consequences of various ordinary everyday activities you might be performing which could eventually lead to *The Death Penalty* (or affect your parents' credit rating). Our criminal institutions are full of little creeps like you who do wrong things...and many of them were driven to these crimes by a horrible force called MUSIC!

Our studies have shown that this horrible force is so dangerous to society at large that laws are being drawn up at this very moment to stop it forever! Cruel and inhuman punishments are being carefully described in tiny paragraphs so they won't conflict with the Constitution (which, itself, is being modified in order to accommodate THE FUTURE).

Hack of the day - Chopshop. Rips Shoutcast radio MP3 streams into separate MP3 files on your disk. It requires Twisted CVS (or 1.0.2 once released).

There are already existing programs that do this (Streamripper sounds nice, gotta steal that mp3 silence trick), but Chopshop is probably the shortest one - I wrote about 230 lines of code, half of it added to Twisted as generic Shoutcast protocol support.

Thanks to shlomif for the link to Neo-Tech, a hysterically funny cult. I highly recommend reading through the site (think of a cross between Heinlein's opinions as interpreted by a 16 year old, traditional rich right-wing American's political and economic agenda, and plain endearing wackiness.)

They call Bush a "closet secular humanist"! They explain how "The Zon/string-theory speculates that by orchestrating multi-dimensional quantum vibrations or gravity-unit ripples, technologically-advanced people could not only birth universes, but could design their initial conditions for optimum efficiency in developing invaluable conscious life yielding maximum returns on investment." They call Libertarians "left wing". They even explain what those mysterious Illuminati are up to.

In the immortal words of Pokey the Penguin: HOORJ!

I'm the first result of searching for "ontological proof" on Google.

The only way to deal with "logical proofs" of God's existence is VICIOUS MOCKERY!

Two weeks where I used computer a lot less, due to pain in wrists and hands. It's still not completely cleared up. I did feel well enough to finally release Cog version 0.3.

Cog is the Checkpointed Object Graph object database, providing semi-transparent persistence for large sets of interrelated Python objects. It handles automatic loading of objects on reference, and saving of modified objects back to disk. Reference counting is used to automatically remove no longer referenced objects from storage, and objects will be automatically be attached to the database if a persistent object references them.

It was originally written by Oliver Jowett, but I am now maintaining it.

I just got back from the week long Zope3 coding sprintathon in Rotterdam. Lots of fun, very educational, and now I have even more obligations of code I need to write for free software projects :)

The sprint was an impressive commentary on the success of free software as a commercial enterprise, with around 25-30 developers from 15 different organizations. Unlike Zope2, which was in general developed by a single company - Zope Corporation - Zope3 is being developed in a series of coding sprints in addition to the individual ongoing contribution of developers. The sprints and development are done, in general, by programmers at companies whose business is based on Zope2.

There are 95 companies listed as Zope Solution Proviers on zope.org, but in practice there are probably many more, as well as even more companies who sites run on Zope. No doubt almost all of them are interested in Zope3, and quite a few are putting in some resources (e.g. giving workers leave for a few days to go to a sprint) to make it happen.

On the technical side, it's interesting to compare Twisted to Zope3. While they have different purposes - Twisted is a general purpose networking framework and libraries, Zope3 is a web application server, they are both fairly large Python frameworks.

Someone called the Twisted developers "abstraction astronauts", but compared to Zope3 Twisted is so concrete you could build bomb shelters with it. This has a price of course, Zope3 is quite slow and only does 1 request a second on my laptop, but it has not been optimized at all. Knowing the people involved I don't doubt their claims that they have clear paths for optimization, and I'm sure the final release will have decent speed.

In any case, the abstraction is a good thing. Zope3 is an inherently complex application, and the new design is quite superior to the kludge known as Zope2. And Zope2 is the best content-management oriented web framework I know of (the next person to point to their favorite PHP project gets a $10 voucher for Toys R Us where they can buy some more toys.)

Twisted has been running applications for most of its existence. For example, http://twistedmatrix.com has been running using Twisted's web server for about 2 years. Since this also includes user homepages and the #python wiki (MoinMoin CGI), this provides some real world testing under real conditions. In addition, a number of commercial and open source projects are using Twisted, some of which have already been deployed. Zope3 on the other hand has not been used for any real world usage, other than developing itself - not even a test server for the development project. This will soon be changing as a couple of projects will probably start using it once the alpha is out.

I am confident in the Zope3 source code quality however, beyond the design itself. The use of interfaces means there's a clean separation between the requirements and the implementation, which does of course suck in some places. In addition, there are a *lot* of tests. Twisted has 75,000 lines of code, Zope3 has 150,000. Twisted has 350 automated tests and about 15 tests that need to be checked by users. Zope3 has 2700 automated tests! This will mean refactoring to deal with real world requirements will be much easier.

All in all, I'm really looking forward to Zope3. I'd probably be jumping up and down going "GIMME GIMME GIMME I WANT IT NOW" except I'm trying to get out of doing content management website jobs.

Spending my time getting the Cog object database up to speed for use in a project I'm working on. I've gotten it to the point that developers only need to mark their classes as persistent and then mark them as dirty when necessary, and it Does The Right Thing.

Next task is rewriting the storage backend, probably using the Python bindings to bsddb. Long term features I want to add are a backup script, GC script, hooks for indexing stored objects, maybe rollback support, but I'll do those after I finish the rest of the contract.

As usual in Python, you need to work a lot less than you might expect - Python's pickle library already has all the hooks necessary for serialization to an object database. So Cog can concentrate on doing all the other parts. It's quite a nice package even as is, other than a few rough spots.

14 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!