Recent blog entries for Artimage

Game Theory
I have finished the first chapter of Game Theory for Applied Economists. So far my biggest problem is that I don't know economics terminology, but that is a small hurdle. I am hoping to use game theory to work out the payoffs for file leaching. Then to setup a system where one gets the best payoff for acting honestly. I have taken a stab at a protocol to enforce this using ripped coins, but I need to go over it at least once more before I will be actually ready to believe in it. (Never mind show it to others.)

So far game theory has been interesting, and surprisingly, not wholly intuitive. Probably part of that is its assumption that all players are rational, or that one understands the payoff of another person. Both of those assumptions often turn out to be false. The most trivial example comes from the prisoner's dilemma, probably the best known example of game theory. Most analysis only deals with the payoff in terms of jail time, where as most people would think about many other factors such as loyalty, and their future safety. (The link actually talks a bit about this, but most treatments I have read skip this.)

I finished my lookup code that uses a distributed hash table, Khashmir. The only problem lies in the fact that Khashmir now uses sqlite, and it does not want to work on my system. I tested sqlite, and it seems to be ok, but whenever I run the unit tests for Khashmir I get an error about too many open files, or "pysqlite_exceptions.OperationalError: database is locked". Maybe its a thread safety issue? But it works for other people, and it never works for me. I will have to keep playing with it.

1 Nov 2002 (updated 1 Nov 2002 at 15:11 UTC) »
I spent most of the day fighting with Symbol's 1 mbit Spectrum card. These are pre 802.11b frequency hopping wireless cards. There is a driver for Linux on SourceForge, so that seemed like a good place to start. But I hit some snags since the driver wasn't really meant for the 2.4 kernel. So I tried turning of the PCMCIA kernel functionality, but that didn't work. Finally I figured out that the driver didn't want to work with PCMCIA tools package 3.2.1. So I switched to 3.2.0, and that made some headway. I also leaned that <linux/malloc.h> has been deprecated for <linux/slab.h>... That caused some confusion as well. But now I have a card that is recognized, so tomorrow the real fun begins. I am going to add some functionality to the driver so that I can change the MAC address, and then I will put it into promiscuous mode. Then all I need to do is tie it in with Kismet, a wireless sniffer, and I am good to go.

At home I installed the Netgear MR314 wireless router. That was actually pretty easy and pleasant. Now I can hack from my bed.

I just read Doucer's paper on Sybil attacks. The gist of his paper is that you must have some central authority if you want to prevent an attacker from assuming multiple identities. I like his example, hashing an IP address, since the central authority is somewhat hidden but still there. Zooko believes that he is missing the point since trust networks can get around this problem, but then again trust networks will someday fix all our problems and probably become the sentient AI that send robots back in time to kill us. Anyway, this has got me thinking about ways to choose an arbiter that is truly unbiased. I have been thinking about a protocol that is verifiable by both parties and gives either side very small chance to cheat. (I don't think it is possible to eliminate that chance completely since there is a chance that your random arbiter is either a second instance of one party, or a friend.) More to come on this once I have it all worked out, hopefully with some numbers and properties of resistance.

itamar* came over on Sunday for Mnet hack day, it was really a lot of fun to have someone to code with. I am helping him port EGTP to Twisted, and he is teaching me the zen of the Twisted reactor. We did end up getting a bit sidetracked though, talking about digital cash and anonimity. On a seperate note, I am still fighting with my dev environment and it is really starting to annoy me. The cygwin bash shell does not play well with Python's os.path.join(). This has led me to explore a tcsh shell not written by cygwin. I hope it works better because it is hard to test code when you can't get the packages installed. (This wasn't a problem before, but now that we are using distutils I am stuck.)

*It is an unfortunate side effect of advogato that one's name is case sensitive, hence my impoliteness at not capitalizing some people's names.

icepick is going to be in town on Sunday the 24th of November, so we are going to try to have a CryptoBrunch. There is a local coffee shop that has WiFi, and we intend to pipe in others over the net. I am also going to try to gather some of the NYC crowd (Itamar is alread in). If you live in NYC and you would like to talk about digital cash, crypto anarchy, p2p systems, or A-Team re-runs drop me an email.

Well, I am a year older today. :) Actually, for the first time I actually feel younger than last year. Being 25 is cool. Last night Heather and I went to Daniel for dinner, for those not in NYC it is highly rated French restaurant. We had a five course tasting menu, with matched wines, that was really amazing. My favorite was the saffron soup with muscles. I haven't yet opened my presents with one exception. I ordered myself a Netgear wireless AP from Amazon, so when an Amazon package arrived I opened it. Turned out it was a present from my little sister. Real Genius on DVD which is my favorite movie. I couldn't be happier.

Hacking for a living is still kind of fun. I coded my first test buffer overflow yesterday. It was pretty satisfying really. I want to take a look at some p2p programs for overflows now. :) My new project may be to write a wireless sniffer for a precursor to 802.11b cards. There is already a Linux driver out there, so the work should be pretty easy. But it looks like I only have 8 days to do it.

We are having a party on Saturday to celebrate the earth completing a trip around the sun without snuffing me out in the process. My best friend is coming in from Boston, which makes me very happy. And most of my friends, those not traveling through Thailand, will be there. All in all, its a good life.

I still want to add Khashmir support to EGTP. I just haven't had time in the bast couple of weeks. Too much Capoeira. And this weekend is full too. But maybe I can steal some time one evening and at least get the unit tests written.

I just finished Cronoliths by Robert Charles Wilson. It was pretty good, and definitely made me think. But I was not thrilled with the ending. Overall, I would rate it 4/5 or a B+. I am about to start A New Kind of Science by Wolfram. Hack and I are reading it together, so maybe we will get something out of it. :) I may also read The Man Who Loved Only Numbers: The Story of Paul Erdos and the Search for Mathematical Truth by Paul Hoffman for the subway since I would need a pack mule to carry ANKOS around with me.


It has been a few years since I played around with java so it is nice to be back. I am trying to write a plug-in for WebProxy that will automate cookie collection and analysis. During this I found a place where I wanted to copy an object, so being from a C++ school of thought, I figured a copy constructor. I did a google search and found an interesting article arguing that in Java instead of a copy constructor one should implement the cloneable interface. It turns out that is interface is supported by Object, so that all one has to do is make a public method that calls the super.clone(). Wham, you have a copy. Simple, elegant, and easy. Now if the the rest of the API was only clean.

The Bunker

So my friend Lock now has the server that we will ship off to The Bunker. It is a dual proc 1U with two SCSI 3 drives. I believe he set it up to be a ~76G RAID 0 array. As of today it has FreeBSD and all the packages we need, so hopefully we can ship it across the pond by the end of the week. :) Today will be spent locking it down so that when it finally gets net access we won't get hacked in the first 30 seconds. We need to install tripwire before we ship it too.. Can't forget that. I am quite excited to finally have a secure box in a secure location. As soon as its up, we can put a remailer on it. And maybe a Tarzan node if they release the source.


I read the Tarzan white paper over the past few days. Since Def Con I have been thinking about this stuff a lot. Roger Dingledine has made me really pessimistic about anonymity, of course he has chosen the toughest possible threat model. It seems to me that what Tarzan offers us is a leap ahead of everything else, but still won't satisfy Roger. The basic problem is that traffic analysis is still possible, and over time, it will work. Having said that, this system seems to do well against many forms of attacks. It assumes that every node has global knowledge of the node network, which allows them to do some interesting things. I still think that active attacks using DOS will be possible. But this system is a lot better than anything I can run at the moment. So that makes me happy. Still, it would be nice to get provable security. (Though that may be impossible.) Maybe using Palladium to ensure nodes can not be malicious will give us a boost. But even if you can guarantee that active attacks by nodes are impossible there is still traffic analysis. It seems to be that traffic analysis is like brute force, it always wins.


At lunch I am going to go down the street and see if the local T-Mobil store has the new Sidekick for sale. This is simply the Danger hip-top renamed. I like the fact that it has unlimited data sending, and since my company pays for my cell phone now, I figure I might as well try this thing out. So I am going to go and see if its really as cool as it looks, if it is, expect me to post a review.


Wired has a really good article on Lessig's background. I recommend it. I am still unsure of what Lessig really wants when he asks geeks to fight. He does have a good point about not spending money that helps Hollywood and MS if you are fighting them. Instead of buying a DVD give the money to the EFF. etc... But that only goes so far. A planned boycott would be much more effective than me simply not buying an Xbox. Then there is the theory that we should work on projects that try to ensure freedom through technical means. Lessig doesn't think this is a good approach, becuase laws can make technology illegal. Ok, fair enough, but I am not going to become a lawyer either. So I need a bit more guidance as to how I can help. I write letters and send faxes, but that really doesn't feel substantial. Writing code at least ends with a tangeble product, though my job has cut into my time to work on cool projects. :(


I am now officially getting paid to hack. Its really very satisfying, in a junior high sort of way. It has been so long since I did this kind of thing that I was afraid I wouldn't be able to do it. But it turns out that not much has really changed in 4 years. There are a few new attacks, the tools are better, but really its same stuff I was doing back in Purdue's Coast Lab (now renamed Cerias). Its nice to work with really smart people too. :)


Well, v0.5.1 Stable was just released on sourceforge. Thats cool. I have been slacking due to my new job. Plus I am having serious issues with Cygwin and Python. I am not using Cygwin's python, just its shell. We have a .sh script to set up the env for us, but due to windows using \ and UNIX wanting / it has been having problems. Actually, I now believe that the slashes are not the problem. My current theory is that /cygdrive/d is not being interpreted correctly by python. Today I will try to set the path by hand and see if that fixes it. Anyway, long story short, until I figure this out, I can't test the changes I made. :(


We now have a 1u box. We need to put BSD on it, and then send it off to TheBunker. Once that is up, I will look into setting up an anonymous remailer.


I am just completing Rudy Rucker's non-fiction collection Seekwhich is broken up into 3 parts: Science, Life, and Art. I could have done without the Life sections since its mostly travelogues, having said that, I have really enjoyed it. The science section mostly deals with Cellular Automata, which is particularly relavant due to Wolfram's new opus. The Art section is short, but it talks more about cyberpunk, which is why I like Rudy in the first place. I really miss Mondo 2k (I also miss the cypherpunks). I miss the optimism that we had when we believed the net would really change the way society worked. I was talking with an ex-Kozmo employee a few days ago, and he said something that really summed it up for me: "For a while we were living in the future." I loved ordering things online and getting them an hour later, it really was instantanous gratification. And while the current crypto p2p movement is nice, it lacks the pithy slogans that made cypher/cyber-punks so cool. :)

Obligatory statement: It has been a while since my last diary entry.


We are dangerously close to releasing 0.5.1-Stable. I did some one line Perl chicanery to parse the CVS logs and make a changefile. Basically this needs to be released so we can start in on the fun stuff. v0.6 has a totally new block fetcher that is far superior and it will use distutils. If those two major advantages don't turn you on then how about Zooko's new hack that will allow people to hide their IP behind a relay. In theory this gives you the same anonymity you get using something like anonimizer. (In truth this is not true because the MetaTracking system we use, but it is a pretty good start.) As for me, I am re-working the bootpage loader code, not very sexy, but it will help me get a better feel for EGTP, Mnet, and the MetaTracker system.

I also took a look at khashmir which is a distributed hash table library of the Kademlia flavor implemented in Python (Definition stolen from SF page.) It looks like Drue, its creator, is going to port it to Twisted reactor async io core. If this works out then, like EGTP, it should be able to route past Firewalls and NAT's using relays in a dynamic fashion. That is pretty dope. Its big advantage is that it is completely decentralized, unlike EGTP which uses a semi-centralized MetaTracker system.

The Bunker

Not much to tell really. It looks like Lock, my friend who works at Dell, has tracked us down a 1u server. I believe it will get shipped out to England some time next week. Woot!


So I am flying off to Boston again on Tuesday, this time sign the contracts making me part of the working world again. Though I am really looking forward to starting to work again, not to mention being paid, I will miss having unlimited time to code my own projects.


So EGTP gives this warning under windows: This OS needs a better source of entropy (or something similar). So I thought hey, this is something I should look into. Google, Oracle for all things geeky, of course shown a light on the path of wisdom. This post to a mailing list seemed to have a cookie cutter answer for what I was looking for. (Side note: this should work on all windows with IE3 or later, or 95 OEMsp2 or later.)

So I quickly got it working, and spewing out nice random output. But I wondered how good the output was. Which led to David Wagner's page on randomness. What a treasure trove this is. So I grabbed Diehard, a series of tests for RNG's (Random Number Generators), and read up on how to use it. Thirty minutes later I have generated an eleven meg data file for it, and it is churning away. (Small side note, due to an error in the DOS version I was forced to enter the filename each time.)

This turned out to be disappointing, though not surprising. Since I was really using code to grab a seed for an RNG, it turned out the data wasn't passing the tests. Which means what? I am not sure. I know that on linux /dev/random will run out if you if you don't give it time to replenish. The cookbook answer didn't tell me how to check to make sure my data was good. In the end, this is still better than the way we were doing it. (Using time and mouse position. This short post by Don Davis tells why mouse position is not a good source.)

With all of this done, I talked to the other EGTP hackers about where to put my code. First reaction, add a patch to crypto++, Wei Dai's C++ crypto package that we use. Now, if you have been following my links, you will notice that the first recipe I found was also posted by Mr. Dai. Wouldn't he have used that snippet in his own library you ask? Oh, yes, he did. But when the code was written it either wasn't there, or didn't get researched. Wei has a great class called AutoSeededRandomPool that plays nicely with /dev/random (Linux), /dev/urandom (*BSD), and Windows.

I talked with Zooko, and it turns out that embedding C++ code into python isn't the easiest thing for a Python novice to be doing. It will be much simpler to just create a small file that simply does our entropy gathering. So that is what I will do now, and we can come back and do a major overhaul later.

Anyway, I have got the code compiling and working in Python using distutils. There is still some testing ahead, but it feels good to have gotten it working and to have a better understanding of distutils and the C/Python API.

Job hunting

I went to Boston on an interview yesterday for a consulting firm. I have never thought that consulting was up my alley, but after talking to them I must say that I am quite excited by the prospect. They all seem smart and the work seems interesting, I have a good feeling about it.


I didn't get to code at all yesterday, and today I have just been futzing around. I wrote my first python class, a dummy verify function that always returns true, and used it to fix the calls in TristeroLookup. Of course it's really trivial, but it seems to work. Now if I can just figure out why the local tests are failing....

14 Aug 2002 (updated 14 Aug 2002 at 16:28 UTC) »

So this is my first entry here. I was very dubious of this entire blog thing until I started reading them. Now I must admit, I am hooked.


Two days ago I finally got EGTP to compile on Windows. There were a bunch of little issues with Makefiles and directories, as well as one 'bug'. It turns out that you need the b flag (for binary) when dealing with windows but not with Unix. So the code worked on Unix, but under windoze it broke. The lesson here is to never leave Unix. :)

I am now trying to get the tests to work. This is a bit hard since I really don't understand ... well.. there are a lot of things.

  • Python - I am trying to learn Python. It still surprises me when it doesn't act like Java or Perl.
  • P2P concepts - Distributed programming is not the same as client server. I keep rediscovering this fact.
  • EGTP/MNet - There isn't a ton of documentation on how things are supposed to work or why they are designed the way they are.
  • Windows - I miss Unix.

As you can see, I am in a little deep. But the people on #mnet have been really helpful.

The Bunker

Hopefully we will have our 1u shipped to The Bunker early next week. I am looking forward to moving to a more reliable mail host. I am also going to put up a projects page there for myself.


Last week I re-engineered the Amazon Wish List parsing Perl module and sent the diffs to the maintainer. He had been using a regexp that was just huge. I decided to use the Perl package HTML::TokeParser. It is smaller, and each regexp is much easier to deal with. Now I need to start saving the data to XML. In the end I want to be able to track prices in Amazon. When a price drops by more than 10% I will send an email. Kinda like a stock watcher.


I talked with Len Sassaman at Def Con. He wants me to put a remailer in The Bunker. He actually knows one of the founders and is talking to him about it. (I know the other founder.)

During our chat he brought up the fact that most remailers are run either in the US or in Germany. I find this to be very interesting. Apparently, because of the Nazi's and their Gestapo Germans are very careful about privacy. The argument often heard here in the US: "You only have to fear the government if you are doing something illegal" just doesn't wash in Germany. I mentioned this to a friend who pointed out that we had McCarthy and his communist witch-hunt. Why have we forgotten this lesson so soon? And what about Nixon and Watergate? John Ashcroft scares me more than Iraq.

Anyway, back to tech. This led me to look into where remailers are. I found The Remailer Geographical Mapping Project, but they don't have an actual map. So I did a bit of looking, and there are some Perl packages that will do just fine. One that will generate a map with points given lat/longitude and the other that will return a lat/longitude given a postal address. So I want to put that up on the bunker as well.

Wow, this has gotten pretty long. I guess I should go code now. :)

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!