What's the fuss about, and what's pyjamas, and what the hell's
python?
How can i put this best... pyjamas is possibly the most significant free
software applications development tool that i've ever encountered.
On my radar, which is pretty sensitive to 'Significant, Archaic, Esoteric,
Obscure and/or Elegant and/or Beautiful Technology', it's like glowing
and overloading circuits.
Pyjamas (and GWT) is, basically, a compiler tool. You take python
(or JAVA) as the input, and you compile it, and out pops pure HTML
and Javascript. The neat thing is that if you want to access data,
you do so from your application using a JSON plugin (or any other plugin
that you care to create, to be honest). And the JSON library is, again,
implemented in Javascript (actually, AJAX).
To give you an example: i struggled to do my
old site for ten days
trying to research all of the necessary techniques and skills that are
required
to place web content in the layout that i wanted. What i wanted was
this: seven blocks in three rows (2-3-2), everything in the middle of
the screen, and if you resize your browser, it reconfigures so that you
don't need to go onto the horizontal scrollbar, as i consider that to
be extremely lame. The new site
looks
exactly the same: it took me about 40 minutes to convert to
pyjamas, most of which was
spent going 'errr' as i cut/paste bits of the kitchen sink example
and the warm words of my old site into one joyous beautiful over-colourful
site. and hooray! now i can do those popups that i wanted to do on
each colour block, without having that sinking feeling!
python, as you know, is one of the most powerful, flexible, compact,
elegant programming languages ever designed. there are more
elegant
languages; there are more powerful ones; there are more flexible ones,
but no programming language that i know of has all those
attributes,
and a lot more besides.
modula-3 for example - but who the hell uses modula-3.
sqlobject and the tantalising prospect of seamless desktop / web
some time in 2002/2003, i encountered SQLobject, one of the first
python free software object-orientated 'meta-sql' building blocks,
which pissed all over my 'DPyDB' code - a 2nd generation relational
database sql-building toolkit. ian bicking's SQLobject, a 3rd generation
object-orientated sql-building toolkit, has morphed into the most powerful
database tool i know - but, it must use meta-classes which
aren't supported in python 2.1, which is a pity.
oh - and by the way: those guys who are doing sqlalchemy are bloody
idiots, because they didn't bother to check whether ian had written
sqlobject. and they're basically reimplementing sqlobject, and
doing a
piss-poor job of it. they've no idea why ian added in object
cacheing
and transaction rollbacks, for example, stupid idiots. all they
really
had to do was look at putting an Object-Orientated syntax on top of
SQLbuilder (utilising python syntax, the key 'feature' of sqlalchemy).
anyway. ian was talking with andy dustman, the author of python-mysql,
and andy was developing a classes system which had two modes:
1) output HTML
2) run on the desktop
basically, from the same code, which would be from building blocks
that matched nearly 100% in classes like Table, Input, Form, all of the
attributes of HTML, you could move the HTML library out the way,
and
the web server, and the web browser, substitute a different
library,
and end up with a desktop application (!)
the code even generated CSS stylesheets (remember - no AJAX at the
time, at least, not by anyone but the most esoteric and brilliant of
web programmers: it generated straight HTML with CSS) in 'html' mode.
and i never found out whether it was a TCL/TK or a QT3 or whatever thing
in 'desktop' mode.
now, he had an agreement with the people he was working for that within
one year he would be able to release all this code as free software.
within six months, the company was bought up. and the new owners refused
to honour the agreement.
so, rather than lose the contract and the possibility of work over a
bun-fight, he backed down.
the code never made it out of the clutches of those stupid assholes.
good ideas come around again... five years later in this case...
however - the idea has stuck with me ever since: have an application
that runs on the desktop and can run as a web application
without
having to friggin rewrite and redesign the bloody thing.
that's just stupid.
imagine my delight when i discovered gwt last year. a tool that
auto-generates html! and does AJAX! and would therefore take away all
the pain because the expertise is all 'hidden' yet focussed on and
refined by lots of applications writers debugging it on many many
different browsers.
and ... very quickly i was bitterly disappointed because it uses java -
and, i'm sorry to have to say this, but java really is shit.
people who love it - it's great. but if you know lots of
programming
languages, you'll know that it's too close to c++ to be worthwhile bothering
with (you might as well use c++), and as a compiled language, it's too
far away from object-orientated scripting languages to be worthwhile
bothering with (you might as well use python, or *shudder* perl or
ruby or even lisp would be better). anyway - end of rant, i
think...
so. when i encountered pyjamas, i went NUTS. because, i made an
attempt, last year, to develop a similar framework. and, i got about
25% to 30% of the way there (and then the project went all weird or more
specifically the business people i was working with went paranoid due to
lack of experience and i stopped).
the framework that i developed, written as a mod_python site, had a
couple of requirements: 1) do AJAX 2) do straight < noscript > HTML
and i was trying to design it so that i could do a 'desktop'
version
later - but that was eating a bit too much of my time.
you can maybe start to see why i'm so excited about pyjamas...
anyway.
the 'specification' files - the python code that you pass to the pyjamas
compiler - is actually very similar to code for python-qt, python-tk
etc. (and even superkaramba, or KDE 4's in-development 'plasma' !)
it really would not be difficult to actually put in a framework
where
that code gets executed and you end up with a desktop application.
the only bit that appears to be hard is the 'event' loop - the bit where
you normally have to access a 'web' page which returns you JSON data.
and, that's where my work with the mod_python-based site i developed
last year comes in.
the holy grail of supporting all browsers (and desktops!)
the basic principle of mod_python - in fact any web site framework -
is that you specify a web page by name (which represents a function) and
you give it a query string (which represents the function parameters).
it's just that in mod_python, and zope, and in fact in all of the decent
web development environments, the relationship between the HTTP POST
or GET and the function which generates the HTML response is that
much clearer. (this is one of the fundamental reasons why i despise PHP,
PSP and other web scripting languages so much, because it's so easy to
be ignorant of all of the things that make good, stable and maintainable
web sites - but i digress...)
what i did was, instead of specifying the web page as a string (but,
to be honest, you could still leave it as a string and parse it
like
you would in a cgi script) i specified it as a dictionary of lists
(before being passed through cgi.urlargs basically) plus a 'module'
name - the name of a function which was executed by mod_python.
the HTML page (or, more often as not, the HTML fragment as returned by
AJAX) is returned as a parameter, which would then be substituted as text
back into the function ... you can see where this is going, right?
basically, in the 'AJAX' mode (the point which is equivalent to where
in pyjamas you go off and do JSON things) you construct a div tag with
a unique id and match it with a small amount of javascript which, when
executed, does an AJAX call (this is a standard AJAX trick, by the way -
i'm describing nothing new here). when this HTML fragment is returned
from mod_python to the web browser, it instructs the user's browser to
do an AJAX call (with the parameters to the web page that i mentioned
at the beginning of the paragraph above!) which ends up
substituting
that div tag with the HTML that comes back from the new web page
(and there's a really simple example dlink that i found on someone's
site, linked further down on this page, that demonstrates this principle.
use babelfish to translate the japanese!).
in this way, the web page is constructed layer by layer by repeated
calls to the web server, and the nice thing is that you can use this to
avoid duplicating code or duplicating entire sections of your web site,
server-side, because each 'function' has its own screen area where the
HTML... you get the idea :)
effectively, it modularises a web page design, allowing you to
reuse
and combine components, even hierarchically. and because you can pass
in parameters (and cookies - the equivalent of global variables),
you can get those components to dynamically and completely
reconfigure
themselves. for example, on the left hand side you can have menu options
which tell the centre portion to 'redraw' to display email (menu option 1)
or instant messages (menu option 2) etc.
and in the 'plain HTML' mode, what happens in the mod_python code is
that that AJAX call i mentioned is cut out completely from the loop,
and instead my code calls the function DIRECTLY, and actually returns
the HTML code fragment DIRECTLY!!!
so you end up hierarchically constructing the same web page, to be
returned only once as a complete page, that, in the 'AJAX' mode is
constructed by repeated calls back-and-forth by the user's web browser
to exactly the same functions!!! as a result, the page looks EXACTLY
the same in noscript mode as it does if you have a javascript-capable
web browser. FROM THE SAME WEB SITE. WRITTEN ONLY ONCE! :)
these two techniques are the two missing techniques from the
GWT and
the Pyjamas code of the three (mainly three) ways in which you can do
web programming:
1) pure AJAX. everything's on the user's browser, already. the only
thing that's loaded is data. other than loading the HTML and the
javascript, the web server is never contacted again, other than via
JSON, to get data which the html+javascript 'application' needs.
2) hybrid AJAX. HTML is loaded in tiny fragments, and substituted into
div tags through repeated calls. it requires that you do a lot of
frickin about to make the HTML executable if it contains < script >
sections with further AJAX in it. but - after a lot of pissing about
and a lot of research, i found some hack techniques. oh, and also some
techniques for making CSS stylesheets actually work, too, when they are
downloaded via AJAX.
3) plain HTML. no AJAX. everything is generated server-side. we know
aalllll about this one, and have done for years, so i won't go into
it in detail, other than to say this: this is the one where you can
support people who have text-only browsers, and also it's the one where
you can solve the problem of google ads not being able to ad-walk your
site properly because it's an AJAX application! but - remember, that
still doesn't help you because the stupid terms and conditions of the
stupid google ad system only allow you to 'cut and paste' a stupid bit
of javascript, and the techniques described in here go WAY beyond the
impracticality of manually cutting-and-pasting stupid bits of javascript
into stupid unmaintainable and effectively useless static web pages.
i'll leave it at that - because it's beyond the scope of this article.
4) plain HTML using frames or iframes to construct the hierarchy of
the site. this is quite neat, as it is a variation of the hybrid AJAX
with all the advantages of 2) with the added advantage of full support
for non-javascript-capable web browsers from 3).
the hybrid approach 2) is a nice one, because it's the best of both worlds.
the user's browser does minimal work (only doing multiple AJAX calls),
and the server does minimal work (only serving up bits of page that need
changing - not the whole damn lot).
the hybrid approach: it's FAST! i mean - REALLY fast!
there's no significant delay downloading a larrrrrge initial html +
javascript application, and, also, quite often, the javascript
application is so big and unwieldy that it makes the user's entire
computer run like a dog.
gmail is a good example of this.
my 1.2ghz 512mb TravelMate C110 runs so slowly and so
unreliably
with firefox 1.5 under linux that, whenever i run gmail, i have one
chance to skip to the 'plain html' mode, whereupon everything not only
suddenly runs 5 times faster, but also it actually... well... works.
if i even attempt to use the gmail application, it falls over in a
heap.
anyway - that's another story, plus my setup is very weird, and always
has been, and i've found bugs and problems that nobody else seems to be
able to replicate... *sigh*....
where was i...
GWT and pyjamas are examples of 1)
my mod_python web framework is a duality-mode thing which does both 2)
and 3)
and i considered doing 4) and wished i had time to do 5)
andy dustman's code was a duality-mode thing which did 3) and 5) - 5)
being a desktop application.
the bottom line is: with very little extra coding effort, it's possible
to make pyjamas cover ALL FIVE CASES. thereby making it a truly unified
application development environment.
oh yes: one other thing...
the refresh 'Thing' of ajax applications.
the most annoying thing about AJAX web applications is that the history of
'where you were' when you do a 'page refresh' is completely
forgotten.
i fixed that.
basically what i did was: you remember where that function call returns
some AJAX to get the browser to come back and reload that HTML fragment?
well, part of my mod_python framework's job is to record that hierarchical
information in a cookie! the only thing you have to watch is: in a big
site, that would be a big damn cookie.
also, what i found was that when clicking through the site rapidly,
sometimes the same area would get overwritten (think of menu options on the
left hand side updating the main screen area by setting one of the
'parameters' of the web function to say 'the purpose of this area is now
to display email. return HTML containing email!')
but, because it's a cookie, and because when you click on that area it's
AJAX, what happens is that two asynchronous browser threads get set
up, and the slower one will 'win'. this gets very odd, very quickly -
but i am not entirely sure what to do about it...
but, basically: that cookie contains all of the information that is
required to reconstruct the web page. the alternative approach would
be to have a session cookie which refers to some information stored in a
database - but that is left as an exercise to the reader. the neat thing
is that this complements the GWT 'history cache' concept.
remember:
an AJAX application breaks nastily when you do 'Back' button, and the
GWT 'cache' stores a copy of the DOM model of the web page, and captures
the forward/back button and 'replays' the requested 'cached' content.
but - that still doesn't help you if you hit 'refresh', and that's
what my little trick of storing the hierarchy of the last displayed
page-fragments in a cookie solves.
the vision of the holy grail of applications environment
so: concept-for-concept, we have a near perfect match between
recognised and well-known desktop development environments and their web
equivalents, thanks to google web kit and pyjamas, as seen through my rosy
farscape-tinted glasses. for completeness, let's enumerate them here:
1) global variables. your application needs global variables or at
least per-session information: in web-speak, these are cookies.
2) event-loop notifications and callbacks. every desktop environment has
these: onResize, onShow etc. GWT and pyjamas makes these easy to use,
as if you were developing a desktop application.
3) widgets. plain widgets and extendible widgets! in the GWT examples,
there is even a 2D drawing package, which spews forth stuff onto a
'canvas'! it's fantastic!
4) separating the look-and-feel from the data-crunching hard work.
pyjamas and GWT, due to limitations, of the framework and the fact
that it comes from a web background, forces the applications writer
to separate out the layout portion from the back-end which supplies
the data. this can only be a good thing to encourage, and there is
enough scope to put in a stand-alone JSON (python-simplejson package)
threaded server which can seamlessly be accessed by the 'desktop version'
of the JSON proxy.
5) providing other communications methods / plugins to fetch data.
your alternative to JSON only need to implement two callbacks (remember,
this is client/server Remote Procedure Call architecture, not plain
vanilla desktop architecture). one: 'did it succeed, gimme all the data'
and two: 'it failed miserably, let the poor buggers know'.
the wishlist, always the wishlist...
one final thing: i have a wishlist item for pyjamas (and if it makes it
into GWT, then that'd be great, too)
treeview menus. you need a dynamic treeview menu widget. why?
because some databases contain many thousands of entries, and are four
to five levels of hierarchy deep, and you sure as hell ain't gonna
load that into javascript in one go, which is what you get in
pyjamas
and GWT. yes it's fast to display, but there's no way it can cope with
2,500 database entries - the javascript would take FOREVER to load up.
now, i've had to do this already (in straight python / ajax) and so i
know what's needed, and it doesn't take long to write, and it's only
about... *thinks*... 200 lines of python, and probably a lot less AJAX
than i'm using because what i'm using is old-school manual AJAX crud.
the original dynamic-treeview-menu example that i used was this:
http://jsgt.org/ajax/ref/test/dlink/test1.htm
and i absolutely love that example because it's so trivial. and it
was what i based the 'hybrid' stuff on that i mentioned above.
in the treeview case, you need:
a get-me-children-by-parent function (to get the next level)
a count-my-children function (to tell you when to stop recursing!)
a get-my-details-by-id function (to obtain the item to display)
a 'display-my-details' function (to actually display the
item)
and a few others (minor but very important things!) which i forget
off the top of my head. you need to pass in a plus-or-minus option.
just like you can see in the dlink example (ignore all the japanese crud)
the desktop becomes the web, or vice-versa? who cares, it's all the
same!
the future for pyjamas and GWT as i envisage it provides enough for
developers to write extremely powerful applications ... like... you
could write a desktop app for browsing filesystems (any number of files /
subdirectories, any levels deep...) that then oh look, the files all got
moved onto the backup server, and oh look, there's a web application
that looks exactly like my desktop app and .... err.... where does
browser and where does deskop actually begin and end, and what, really,
is the damn difference any more???
and... remind me again: why the bloody hell am i running a 200 watt
computer with a 3Ghz P4 that costs me $150 per year at domestic
rates and in some countries over fives times that in business rates,
in power consumption alone??
i'll leave you with something that'll fry your brains for good.
with pyjamas, and KDE 4's plasma, there's the tantalising possibility
to turn the entire thing on its head:
remove qt3 and put in a drop-in replacement that communicates with a
web browser, to run a KDE desktop or any other qt application on a web
browser - in AJAX and HTML!!!
now that's gotta flip people out and make steve ballmer turn in his
grave oh damn he's not dead yet, somebody gimme a gun and get me
diplomatic immunity, for all our sake's.