Advogato blog for jtauber

On Intelligence and the HTM Workshop

Sun, 6 Jul 2008 09:08:56 GMT

Like a lot of geeks, I've been interested in how the brain works for most of my life. Artificial Intelligence was always one of my interests within computing (and part of what got me interested in linguistics at a very early age).

Within my linguistics research, I've always been interested in models that are biologically plausible so it was a huge delight to read Jeff Hawkins' On Intelligence back in early 2005 and find a theory that was biologically-based and believable from a linguistics point of view. One prominent psycholinguist told me in 2006 that it was one of the most promising theories he'd ever read.

After reading the book, I promptly went out and built a library (as I am wont to do) of about 20 books on general neuroscience, computational neuroscience and the relationship between the brain and language. I started thinking about how to implement the ideas and, after reading some of Jeff's and Dileep George's early papers, augmented the library further with books on Bayesian networks, belief propagation, etc.

When Jeff and Dileep started Numenta and eventually released an early version of their Hierarchical Temporal Memory (HTM) platform in Python, I was particularly excited to try it out, in particular applying it to linguistics. I started the htm-ling mailing list to gather other people interested in applying HTM to models of language. It turned out to be hard to get word out to other people interested in HTM and linguistics, however.

I never got very far with Numenta's code, mostly because there were just too many other things I was working on.

But then a couple of months ago, I found out Numenta was running a workshop / conference. I thought it would be an excellent opportunity for me to (a) get back up to speed with what Numenta was doing and how to use their NuPIC platform; (b) meet other people interested in applying HTM to linguistics.

So a couple of weeks ago, I attended the first Numenta HTM Workshop. I had a great time. It was great to meet Jeff and the rest of the team. Dileep's talk on the algorithms in NuPIC was particularly helpful to me in understanding how things work.

There were a number of people who expressed an interest in the application to linguistics so in the evening I ran a BOF. None of the attendees (as far as I could tell) were linguists by training so I didn't really get to talk too technically from a linguistics perspective. The boost to the mailing list membership hasn't created any more discussion yet either.

But I am still hopeful that an HTM-like approach (whether in the form of NuPIC or some other implementation) might be useful in building biologically-plausible models of language processing.

Changes to Google Maps Satellite Images

Sat, 5 Jul 2008 03:06:17 GMT

It used to be obvious in Google Maps where the boundaries of different satellite images were. Each image had different brightness, contrast, colour, etc which gave away the stitching.

I always wondered whether there were techniques to normalize that.

I guess there are: Today I noticed the satellite images are stitched together seamlessly.

I also noticed some level-of-detail differences between land and ocean and that is also done pretty seamlessly.

It actually makes navigating around the satellite view a little eerie.

Anyone know when the change was made?

UPDATE: Actually it depends on the zoom level. Compare this to this. And notice the image credits are different. Interestingly, my home town of Perth looks fully normalized at all scales, even though the image sources are still TerraMetrics for the large scale and DigitalGlobe/GeoEye for the small scale.

The Annotated Turing

Fri, 4 Jul 2008 04:08:30 GMT

Some books entertain, some inform; some confirm what you already knew, some make you change your mind about something. But then there are some books that just make you think "wow! I wish I'd written that".

For me, Charles Petzold's The Annotated Turing falls into that last category. It's a book worth reading not only for the topic itself but the way it's presented.

He provides the necessary background before working through Turing's famous 1936 paper "On computable numbers, with an application to the Entscheidungsproblem" with rich annotations at every stage, including biographical details.

If you are interested in the foundations of mathematics, computability, Turing's work, or even just ways of explaining mathematics in a historical context, I highly recommend this book.

Pinax Progress III

Thu, 3 Jul 2008 04:06:28 GMT

In the six weeks that it's been around (that's not six weeks since launch, that's six weeks since coding started), the Pinax platform and community has grown in ways I never expected.

Since I last blogged about it, we've added:

localization into Brazilian Portuguese and Hebrew
timezone localization
external blog aggregation
wikis
threaded discussions
bookmarks with voting
contact import
blogs with tagging and threaded comments

It is becoming clear that what was originally intended to be a demo site is a useful site in its own right, irrespective of whether you care or even know about the Pinax platform underlying it. So it will be moving over to a new site with a new identity soon.

Back to Blogging

Wed, 2 Jul 2008 15:07:17 GMT

Last month was my worst blogging drought ever.

It happened for a number of reasons: I was travelling almost 3 weeks out of the month; free time was spent on other projects; Twitter replaced a lot of my drive to blog (and subsequently, the django-hotclub IRC channel replaced a lot of my drive to Twitter).

I'm going to try to blog a lot more regularly this month. I have a long list of things to blog about.

Two Podcast Interviews

Wed, 4 Jun 2008 15:09:58 GMT

In the last week two podcasts have come out which I appear on.

The first is the Google Summer of Code podcast where Titus Brown and I were interviewed about the Python Software Foundation's participation in both GSoC and GHOP.

http://google-opensource.blogspot.com/2008/05/podcast-with-pythonistas.html

The second is This Week in Django where I talk a little about myself and (more importantly) Pinax and the Hot Club of France.

http://blog.michaeltrier.com/2008/6/2/this-week-in-django-24-2008-06-01

I sound much better in the second one because I recorded my track locally with a large diaphragm condenser mic and sent the audio file for mixing. In the first one, it's just my laptop mic going over Skype.

They were both fun although I babble too much, especially on the TWiD podcast.

Question-Driven Commenting

Wed, 28 May 2008 20:11:40 GMT

I've always loved Titus Brown's notion of stupidity driven testing. It's okay to make a mistake but you write a test to make sure you don't do it again.

With Pinax, I've been adopting the sister practice of "question-driven commenting". When someone asks me what a function does or how to implement a particular extension, that's when I go and add comments to the code.

Of course, the person asking the question could be me :-)

Pinax Progress II

Tue, 27 May 2008 03:16:15 GMT

Yesterday I reported that we'd added the following to Pinax in the last 24 hours:

user profile pages
gravatars
user-to-user messages (via django-messages)
announcements (a new app, django-announcements)
OpenID support
invitations to join

Well, in the last 24 hours we've added to that:

translations into German, Spanish and Swedish
a new design / logo
auto-completion on message recipient field
a basic Twitter clone
OEmbed support in tweets
the beginnings of tribes (i.e. groups)

I could not have hoped for a more productive weekend!

Earlier in the day I did a rough count purely based on file-size and estimated that Pinax is:

63% re-usable apps;
10% local app;
13% templates;
8% localization;
5% media;
1% util

which bodes well for the original goal of Pinax: to jump-start some reusable Django apps.

Pinax Progress

Sun, 25 May 2008 15:15:35 GMT

Here is what the team of brosner, leidel, floguy and myself have added to Pinax in the last 24 hours:

user profiles pages
gravatars
user-to-user messages (via django-messages)
announcements (a new app, django-announcements)
OpenID support
invitations to join

Check it out at http://pinax.hotcluboffrance.com/

Programming as Jazz

Fri, 23 May 2008 09:10:52 GMT

I'm starting to appreciate that programming has more similarities to Jazz than just the project naming conventions encouraged by Django.

When I started the Hot Club of France mailing list (named for Django Reinhardt's Quintet du Hot Club de France) I explicitly mentioned it was about writing reusable apps that "jam" well together. But I'm realising that it's as much about the developers jamming as the code itself.

Ever since the inception of the Hot Club project, I've thought about cloning Web 2.0 websites using reusable Django projects. I noticed at the time that this is sort of like the "contrafact" approach of early Jazz where you would improvise new songs using the chord progressions of existing songs. The analogy isn't perfect but I do think the term "contrafact" is a great one to use for the programming practice.

Things are really starting to come together with Pinax and the conversations I've been having recently with other Django developers keen to jam with me on it reminds me a lot of stories I've read of the bebop years. You check out someone's work, think it will work well with your own style and start doing some improv together. Awesome stuff!

Things are really starting to come together at http://pinax.hotcluboffrance.com/

Just call me yardbird :-)