Advogato blog for bradfitz

Perl on App Engine

Wed, 23 Jul 2008 04:09:47 GMT

Fellow Perl hackers,

I'm happy to announce that the Google App Engine team has given me permission to talk about a 20% project inside Google to to add Perl support to App Engine. To be clear: I'm not a member of the App Engine team and the App Engine team is not promising to add Perl support. They're just saying that I (along with other Perl hackers here at Google) are now allowed to work on this 20% project of ours out in the open where other Perl hackers can help us out, should you be so inclined.

As background, I've been writing Perl code for almost 15 years now and quite fond of the language. (I'm "bradfitz" on CPAN.) Here at Google, though, it's not one of our big languages so I don't get to write as much Perl as I used to. I'd still like to run my personal web apps on App Engine, though, and I'd like to write them in Perl. And I'm definitely not alone, looking at how many people have starred the wishlist bug. Some of you have already started talking about it. We'd like to join the discussion, and start hacking out in the public.

In the process we can build the start of an open source App Engine server clone that's suitable for many purposes: initially just for regression testing & local development (like the "dev_appserver" that comes with the App Engine Python SDK), but perhaps in the future (once Hypertable/Hbase/etc are ready) a full stack to give to ISPs to let them run App Engine apps on their own.

Before I get into my proposed roadmap, let me describe what's publicly known about the App Engine architecture. In a nutshell, it looks like this:

The App runs in a multi-layer hardened environment, one layer of which will need to be a hardened Perl interpreter.

Basically, we need a hardened Perl runtime which can:

open & read files
NOT write files
NOT open sockets
NOT fork
NOT do any other system functionality that's not strictly needed for a web app

Basically we need a Perl interpreter that's very tame and isn't allowed to do anything other than read web requests and write out responses. Any privileged operations (like Datastore access, fetching URLs, etc) need to be done via a trusted XS Perl module (the "apiproxy") that takes a service request parameter and returns a service response. The request and response are both encoded as Protocol Buffers, which were recently open sourced by Google.

Perl on App Engine then would involve the following steps (in no particular order):

Hardened Perl Interpreter: basically, we'll be statically linking in a hardened, customized libperl to a C++ application, disabling all Perl dynamic loading. Only vetted, security-audited XS modules will be allowed. Only safe Perl opcodes will be allowed. (No sockets, no ioctl, no fork, etc, etc.) To get a preview for what this'll feel like restriction-wise, check out the newly written Sys::Protect which Artur and I wrote this evening and will be continuing to develop for people's dev environments (not production).
Protocol Buffers for Perl: we need support for Protocol Buffers for Perl. I've started on this project internally and will open source the code soon, once I have a few free minutes.
Server: we need to write an App Engine server for testing, local development, and potentially production deployment. (Replace Bigtable with MySQL, Hypertable, Hbase, Couch DB, etc.)
Libraries: Perl client libraries for Datastore, URLFetch, etc services. Including docs.

Not included is the Google-internal side of things, gluing the hardened Perl interpreter into the GAE world. That needs to be done by a Googler and not open source.

If you'd like to discuss this and/or help out, join the perl-appengine mailing list. We'll be submitting code to the appengine-perl project on Google Code hosting. For more information about this, see the Perl-on-AppEngine FAQ.

Brad & the other Perl Googlers

IPv6

Tue, 15 Jul 2008 21:11:52 GMT

I saw that this was open sourced today:

http://code.google.com/p/stubl/

I followed our internal instructions for using it and now I have IPv6 on my desktop at work. Any good IPv6 sites to hit? (besides the Great Experiment, which isn't quite SWF)

My Daily Tweets

Mon, 14 Jul 2008 07:05:24 GMT

09:49 Google Earth with 3D polarized glasses and huge LCD = f'ing awesome #

(posted using LoudTwitter)

My Daily Tweets

Sat, 12 Jul 2008 07:07:12 GMT

01:03 Playing with twitterific on new iPhone software. Yay new shiny! #

(posted using LoudTwitter)

Я ищу друзей на одноклассниках!

Fri, 11 Jul 2008 07:07:32 GMT

http://www.odnoklassniki.ru/user/219970653299

(Have to try out all the social networking sites, ya know...)

Facebook QR Code "me"-link

Thu, 10 Jul 2008 17:06:34 GMT

Facebook's public profile pages don't let you include any links (notably: no XFN "me" links), so you can't setup a bi-direction proof that you own your Facebook account from another page.

I got around that:

http://www.facebook.com/people/Brad_Fitzpatrick/500033387

Check out my QR code in my profile photo. That QR code says "http://bradfitz.com/".

:-)

Update 12:42pm: I made it with Google's new Chart API feature. Make a URL like this:

http://chart.apis.google.com/chart?chs=150x150&cht=qr&chl=http://bradfitz.com/

You can read them with this: http://code.google.com/p/zxing/

Car Crash

Wed, 9 Jul 2008 06:06:24 GMT

The Audi's fucked up...
http://picasaweb.google.com/bradley.j.fitzpatrick/CarCrash

Good thing everybody's okay and the two propane tanks in the back seat (seat-belted in) which I was heading out to refill didn't blow up.

Couldn't find non-emergency number quickly, called 911, they said just exchange contact info and insurance. Did that. Police did show up and took down info and mostly just stood around and watched as me and the other dude had it almost wrapped up by then. Other guy needed to be towed (to his friend's body-shop) but wasn't sure how he was going to pay (no credit card?) so I paid the $200 to the tow-truck driver just to get things moving. Hope that isn't an implicit admission of guilt. Tried to call State Farm to report it but no answer. I drove home, despite the left side doors not opening. Car seems fine.

But I got new propane tanks afterwards on the way home ... who wants to come celebrate a delayed 4th of July tomorrow night at my place? Burgers and hot dogs!

yay protocol buffers!

Mon, 7 Jul 2008 23:10:33 GMT

Yay!
http://google-opensource.blogspot.com/2008/07/protocol-buffers-googles-data.html

(and don't worry, I'm working on Perl support... :-))

Update: Join the Perl group if you're interested:
http://groups.google.com/group/protobuf-perl

My Daily Tweets

Sat, 28 Jun 2008 07:07:00 GMT

23:53 Love the long days! I seek drinking options now. #
19:26 7:25pm: new meme: all tweets to contain self-written timestamps so your friends know when twitter sucking. #

(posted using LoudTwitter)

My Daily Tweets

Sun, 22 Jun 2008 07:07:52 GMT

23:37 Travelodge is so cheap they double up on the el. I'd never noticed! (... or stayed at one) #
07:20 It's rafting time. #

(posted using LoudTwitter)