Posted 28 Sep 2008 at 14:03 UTC (updated 28 Sep 2008 at 16:33 UTC) by lkcl
pyv8 is an experimental project to combine two-way python bindings to v8
performance increase of python code converted and executed as
fair, cython gives a 100 times performance increase).
sloisel has created two-way python bindings to v8: pyv8 download
a version with a Makefile for linux users is here:
pyv8 with Makefile
compiler (called pyjs.py)
together with python bindings to v8, and,
due to v8 being able to call back out, it's also been possible to bind
back in to standard c python modules and third party c-based python
the simple test, test-pyjs.py, containing a deliberately inefficient
fibonacci series algorithm, gives a ten times performance increase
compared to running python fib.py. the output is shown via a
python-tk popup window, to demonstrate that it's possible to call
out to c-based python modules.
this is an experimental project. a much better version of pyv8 would
do away with the need to use python to perform the conversion of its
based on the v8 "shell"
example rather than using python-boost to
wrap libv8. under these circumstances, python-boost would only really
be required to perform the bootstrap-compilation process, required
for pyv8 developers and maintainers only: standard _use_ of pyv8 as
a drop-in replacement for /usr/bin/python would use the precompiled
so whilst pyv8 is a way off from being a formal release, the
of the performance gain, the benefits of being able to bind to standard
c-based python modules. the outright simplicity of pyjs.py (only 1200
lines) makes it worthwhile letting people know that the experiment was a
definitely benefit from using v8.
First, to lkcl, to clarify: two-way bindings means that the runtime supports both calls out
from the compiled code and calls back? It's impressive, then, that the whole python runtime
can be ported to the browser through light hackery, but I guess that this must be non-
comprehensive survey of all such notable bridges, with careful attention made to the space
of design choices made, but since that document does not exist, it might help to assemble a
list. I'll start with six languages chosen by theme:
- Lkcl's compiler pyjamas;
- pyv8 is a compiler together with a fixed bridge: is this article currently the goto
document on this?
attempts to model common lisp idioms;
- Compilers for other languages: Ruby seems the most active, and rb2js seems to be the most important;
- Language interpreters inside js: eg. Oberon Script; this LtU story has some interesting further
I have researched only a few of the above in any depth, so doubtless there are mistakes; I'm
more interested in hearing of more projects, and what makes them distinctive.
Mistake #1, posted 30 Sep 2008 at 08:58 UTC by chalst »
I described Oberon Script as an interpreter; it is, in fact, a compiler; I was actually thinking of jsscheme; an interpreter for a near approximation of R5RS scheme.
OberonScript is an interesting ->js compiler, in that Oberon paradigmatically takes a runtime more like C than the scripting languages, such as Ruby, usually compiled to js.
clarification, posted 30 Sep 2008 at 09:07 UTC by lkcl »
First, to lkcl, to clarify: two-way bindings means that the runtime supports both calls out from the compiled code and calls back?
e.g. python-numeric, or python-tkinter.
It's impressive, then, that the whole python runtime can be ported to the browser through light hackery, but I guess that this must be non- portable?
this has nothing to do with browsers. absolutely nothing.
so, sloisel's experiment is a trifle oversimplistic for real-world usage, because he didn't include any of the builtin libraries from pyjamas. if anyone wants to progress this further they will need to look at build.py, note the way that includes are done by extending the library path, and make sure that sprintf.js, builtins.py and math.py are all in the path.
regarding portability: portability is actually restricted not by pyjs.py but by google's v8 engine. v8 supports 32-bit x86 and arm processors only.
# Lkcl's compiler pyjamas;
pyv8 is a compiler together with a fixed bridge: is this article currently the goto document on this?
yep. that and pyjamas-dev. pyv8 is definitely an early-days experiment (with a startlingly impressive empirical result).
Thanks for the answers. Some minor points:
- I meant js engine when I said browser, but there is no reason I can think of why one
couldn't create a custom Chrome browser that supports pyv8's extensions. When I said non-
portable, the point is that only such a modified browser could handle js that makes calls out
to the non-js runtime.
To be more concrete, a call to python_tkinter, say, is certainly code that will need specialist
support in either the js engine or browser;
- Credit where credit is due:
- Who started the pyjamas project?
- I'm right that you are behind Pyjamas Desktop?
- I've at least got a handle for the developer of pyv8. But who is sloisel? Google won't tell
- I guess my list of projects would benefit from talking about what they make of DOM.
:), posted 30 Sep 2008 at 13:02 UTC by lkcl »
1) james tauber.
2) pyjd.org. yes. and the glib bindings to webkit, without which pyjamas-desktop doesn't happen.
pyv8 0.1 release notice on pyjamas-dev.
ok so let's ask the original question as intended :)
the neat thing about the v8 callback bindings is that there is far more available to pyv8 developers than there is for pyjamas developers, because pyv8 can do access to c-based python modules (and pyjamas apps are restricted to running in web browsers).
# I meant js engine when I said browser, but there is no reason I can think of why one couldn't create a custom Chrome browser that supports pyv8's extensions. When I said non- portable, the point is that only such a modified browser could handle js that makes calls out to the non-js runtime.
so... what you're saying is that (leaving aside the fact that pyv8 demonstrates that you can call out to python from command-line, demonstrating that standard python apps can be speeded up by a factor of ten) pyv8 shows that it would be possible to access standard c-based python modules, amongst other things, from inside a browser that used v8 (e.g. google chrome).
i can see that it would be incredibly powerful and useful to allow access to other languages. i kinda worry somewhat at the security risks associated with doing so, though. particularly as the addition of __class__.__new__ (or something like that) was added to python at about python 2.2 which totally broke rexec.py (an escape-route, via the optimised c implementation, was implemented in such a way that the standard restricted python module was irrevocably broken).
i do think it says a lot about the design of v8 that it allows this kind of mix-and-matching, though. it's fantastic.
google's v8 engine is a stand-alone library, implemented in c++, where _one_ usage of v8 _happens_ to be in google chrome's browser.
ultimately, with quite a lot of wizardy, pyv8 could become a drop-in replacement for /usr/bin/python (or python.exe if you use windows).
pyv8's callback mechanism benefits somewhat in the extreme from the existence of python-boost: i wouldn't know where to begin to suggest to other language writers how they should go about doing external callbacks, but the more of the language that is written _in_ that language (unlike python, which has a lot of c-based modules) the better.
Thanks..., posted 30 Sep 2008 at 15:48 UTC by chalst »
...for your patient explanations. You wrote, of the idea of the custom Chrome browser — let's call it pychrome— i can see that it would be incredibly powerful and useful to allow access to other languages. i kinda worry somewhat at the security risks associated with doing so.
Well, quite. It would need some sort of security model to be anything other than a dangerous toy.
questions, posted 30 Sep 2008 at 16:33 UTC by lkcl »
...for your patient explanations.
no problem - thanks for voicing the questions: if you weren't to ask, then there would be other people - who didn't ask - getting a mistaken impression.
there must be other compilers (or JITs) faster than V8 that can serve as
the intermediates? You mentioned "cython", but can Java or C# or Lisp
choices as well?
because it's there, posted 30 Sep 2008 at 18:52 UTC by lkcl »
have you ever seen that visual studio tool which does language conversion? i wouldn't believe it if i hadn't seen it: it's a plugin for visual studio, and it literally converts any language to any language. my friend demonstrated conversion of java to c sharp to b to ruby - the only significant language missing was python. from what i can gather, it's as simple as converting to-and-from CLR.
connect the two chains together.
cython is a different starting point: it's not _quite_ "real python", it's a subset of python, with extensions. for a fair comparison, pyjs.py definitely isn't perfect: there are bugs in the use of the sprintf.js library that stop you from having more than one % substitution in the specifier string - "%s %d" % ("hello", 5) doesn't work, but "%s " % hello + "%d" % 5 does - and there are little niggling incompatibilities that make it really awkward to work with (which is why i did pyjamas-desktop, so that you could actually develop your application with the "real" python interpreter).
the cython compiler therefore is kinda cheating, as this really good cython tutorial demonstrates. it shows you how you should manually convert your initial python source over to cython, what sorts of things you should look for; avoid calling out to the standard python libraries (those that cython supports) in inner loops, that sort of thing. many people simply don't want to go to that kind of effort, which is why in many cases they simply plump for a c-based module for the inner loops - or just avoid python altogether.
so yes, there are definitely a lot of options - they've just been joined by one more.
RJS, posted 12 Oct 2008 at 11:31 UTC by lkcl »
not sure if RJS is the same as rb2js - see RJS, here
I thought of hooking python to v8 as soon as I saw the v8 announcement,
too. Glad that sloisel actually did it!
I think going from Python bytecode is the way to go long-term: that way
you don't have to worry about parsing python, and the python bytecode is
actually pretty small and straightforward.
fit to Python's, and v8 was explicitly written to take advantage of
this. As a basic example: in python I can add methods and fields to an
object "on the fly" after it has been created. You can't do this in
(say) Java. A JVM is built to assume that method dispatch is fixed at
object creation time, and targeting Python towards such a virtual
machine is complicated and inefficient. v8 was explicitly built to
allow these types of object mutation, and to still use efficient method
and slot dispatch: basically, the type of the object is invisibly
changed by v8 as you mutate the object, so that you always get efficient
method dispatch, even for dynamically-added methods. So taking
advantage of this feature of v8 allows you to make a really efficient