30 Sep 2005 mazurek   » (Observer)

Unwanted WWW Robots (bots) I recently woke up to the staggering level of abuse occurring on my web site. This is old news to some, but we all wake up at different times. I'm talking, of course, about automated robots and spiders. They come at all hours, they take as much as they can, and they leave me with the (bandwidth) bill. They do so without respecting the Robot Exclusion Standard (robots.txt), now almost 10 years old. Some come to gather email addresses, which are then sold to spammers; some come to steal images or other content, and republish it without my consent; some come to spy on me and sell information to their clients about perceived violations of copyright, trademark, or some nebulous concept of brand identity.

Latest blog entries     Older blog entries

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!