Advogato blog for gary

Synchronization

Thu, 24 Jul 2008 10:19:07 GMT

I figured out the deal with getting the number of monitors from typeflow. I was confused because it only tells you the number of monitors at the start of each block, but what I didn’t realise is that it inserts block boundaries after every monitorenter and monitorexit, so the number of monitors remains constant for entire block. A quick scan through a method’s blocks will find the one with the most monitors, and that’s the maximum number of monitors you need for your method. I wrote the code to allocate them yesterday; today I write the code to actually lock and unlock things with them.

As an aside, I’ve seen a few people refer to zero as a VM in its own right, a replacement for HotSpot, but this isn’t the case at all: zero is a port of HotSpot. HotSpot is a largely generic VM with a small number of CPU- and OS-specific files, and when people refer to a port of HotSpot to some processor or operating system, they’re talking about these files. If you look inside an OpenJDK tarball, you’ll find the HotSpot sources in something like these directories:

openjdk/hotspot/src/share
openjdk/hotspot/src/cpu/x86
openjdk/hotspot/src/cpu/sparc
openjdk/hotspot/src/os/linux
openjdk/hotspot/src/os/solaris
openjdk/hotspot/src/os/win32
openjdk/hotspot/src/os_cpu/linux_x86
openjdk/hotspot/src/os_cpu/linux_sparc
openjdk/hotspot/src/os_cpu/solaris_x86
openjdk/hotspot/src/os_cpu/solaris_sparc
openjdk/hotspot/src/os_cpu/win32_x86

Zero adds the following directories to that list:

openjdk/hotspot/src/cpu/zero
openjdk/hotspot/src/os_cpu/linux_zero

So when you build HotSpot with zero, you’re using the following code:

openjdk/hotspot/src/share              444,621 lines
openjdk/hotspot/src/cpu/zero             5,300 lines
openjdk/hotspot/src/os/linux             7,865 lines
openjdk/hotspot/src/os_cpu/linux_zero    1,085 lines

That’s 98.6% pure, unadulterated HotSpot.

(Shark isn’t a VM either: it’s a optional JIT compiler for the zero port of HotSpot.)

IcedTea is served on ARM

Thu, 24 Jul 2008 09:05:59 GMT

Matthias Klose got the first ever “IcedTea is served” on ARM today.

Shark’s framewalker interface

Wed, 23 Jul 2008 10:17:29 GMT

Yesterday I finished my zero bugs, so now I’m back on Shark. Until a couple of weeks ago I’d been implementing bytecodes one by one as they came up, but during some team meetings I had the idea to cause unimplemented bytecodes to abort only that particular compilation. Previously this would abort the entire VM, but now that I have a fair few of the most common ones implemented already this change would mean Shark was usable even in it’s unfinished state, both for general use and for a little bit of benchmarking. Getting day-to-day grind of implementing bytecodes out of the way means I now need to figure out some of the bigger design issues, however.

One that’s been lurking in the background since pretty much day one is the framewalker stuff. There are all kinds of different types of stack frames in HotSpot, but the relevant ones here are interpreted frames and native frames. Each method in the stack will have one or the other of these*, and it’s one of the cpu-specific code’s jobs to provide accessors into the ABI stack frames so that HotSpot can poke around in them. In the very first days of Shark I tried to make Shark frames appear as native frames, but doing that required all kinds of things I couldn’t provide: PCs and such like. I put in some pretty nasty hacks to make Shark frames look like interpreter frames and got on with the rest of it.

Fast forward a couple of months and this approach is showing the strain. I’m at a bit where the garbage collector is interrogating frames, and the GC needs a lot more than just a method pointer and a BCI. I’m faced with the decision of either trying to store everything into the Shark frames that the C++ interpreter stores (in which case I may as well make them identical, and do away with the distinction) or have another go at implementing Shark frames as native frames.

My gut feeling is to have another go with native frames. Making them interpreter frames is hacky enough, and trying to mimic the C++ interpreter perfectly feels too horrible to contemplate. What I may do first, however, is implement synchronization. It should be possible to handle it without having to resize frames, but it’s not obvious how to extract the number of monitors from the typeflow information so I’ve been avoiding thinking about it. Once I have that done, everything that Shark needs to store will be stored, and I can add the extra stuff to mimic HotSpot’s interpreter/native frames at my leisure.

* This is a simplification, but it’s certainly true of zero as it stands today.

General update

Mon, 21 Jul 2008 16:08:16 GMT

It’s been a while. I’ve been doing loads of little things lately, and that tends to stop me blogging. If I’m working on only one big thing then I tend to write about it as a break, often at the end of the day when I don’t want to start anything else, but hopping from task to task means there’s not one obvious thing to either a) write about or b) take breaks from. Ah well…

Probably the single biggest thing is that I started the process of getting an official OpenJDK porters project for zero. Zero’s been pretty much stable since the end of March so it’s high time I did this, but I’ve been procrastinating about it because I’m not sure how I’ll be able to develop in that environment. I want my local tree to be as close as possible to whatever upstream I’m using, which is currently the ecj-bootstrap side of IcedTea, but I won’t be able to use ecj to build straight from a project repository without either applying and committing the icedtea-ecj.patch (bad) or applying it locally and having a non-buildable repository (also bad). I may have to bite the bullet and use IcedTea to build it, which isn’t as bad as it initially seems since the target I use for rebuilds contains no Java. The initial build will be painful but I’ll just have to do it and be more careful not to blow away my tree!

Of course, the increasing interest in zero means I’m getting more bug reports. A few people have mentioned builds failing with java.lang.IllegalArgumentException: disparate values. I assumed they were all the same, so I investigated (and fixed) it on ia64, but that particular issue was little-endian specific and so doesn’t explain that people have been seeing it on ppc EPEL. It seems that this is the first place in the build where floating point values are used and checked, so this message is not one specific error but a catch-all for generic floating point bugs. Anyway, if you’re seeing this error then please let me know.

I’ve been thinking about TCK runs too. I didn’t fancy the idea of setting up an entire TCK environment on ppc, so I filed it in my head under “Future work”, but it occurred to me that zero works (or should do) on amd64 too, so it shouldn’t be too difficult to drop it into our nightly tester. I’ve not done more than thinking about this as yet since there’s not much point in doing it without copious free time to fix the thousands of bugs it’ll doubtless find — which I don’t have!

All this zero stuff has left Shark on the back-burner for a bit, but work there is progressing too. Lately I’ve been working on making unimplemented stuff abort only that compilation (not the entire VM) so that a) people can use it in it’s unfinished state, and b) I can run some benchmarks and get some idea of the progress I’m making.

Shark 0.01 released

Tue, 24 Jun 2008 09:12:37 GMT

I just committed what I have of Shark to icedtea6 hg. Its version number should be taken as an indication of its level of completeness. To build it you need to use ./configure --enable-shark with a very recent LLVM installed on your system. I’m using 52487…

20 Jun 2008

Fri, 20 Jun 2008 09:14:26 GMT

I spent quite a lot of time tracking down a miscompilation in LLVM lately but I’m pretty much back on track now. One fun thing is that I figured out how to add diagnostic options to OpenJDK, so when I eventually release Shark you’ll be able to debug it with the following exciting options:

-XX:SharkStartAt=N: Start compiling only after N compilation requests. (Use in conjunction with -XX:+PrintCompilation.)
-XX:SharkStopAfter=N: Stop compiling after N compilation requests.
-XX:SharkDumpModuleAfter=N: Dump all generated LLVM bitcode (to hotspot.bc) after N compilation requests.
-XX:SharkPrintTypeflowAfter=N: Print the results of the typeflow pass of the Nth compilation request.
-XX:+SharkTraceBytecodes: Print the name and bci of each bytecode compiled (Handy for crashes).

I tend to use them in pairs, eg -XX:Shark{StartAt,DumpModuleAfter}=17.

This list will probably get real old real quick, but they’ll all be listed in ports/hotspot/src/share/vm/shark/shark_globals.hpp.

3 Jun 2008

Tue, 3 Jun 2008 16:14:33 GMT

Keeping the (Java) stack and locals in registers is all very well, but it makes method calls and safepoints tricky as previously everything was on the (Zero) stack where it needed to be, and now it isn’t.

28 May 2008

Wed, 28 May 2008 12:27:08 GMT

So here it is, the first method from the all new Shark, String.hashCode(). And the same method from old Shark for comparison. Some highlights:

The inner loop for the actual calculation is 18 instructions (down from 53)
A complete pass through the method when fetching a cached hashcode is 39 instructions (down from 66)

All in all I’m pretty pleased with myself.

27 May 2008

Tue, 27 May 2008 10:24:36 GMT

I nearly have String.hashCode() compiling with Shark II and the code is so short it’s unbelievable!

21 May 2008

Wed, 21 May 2008 23:16:49 GMT

I started tearing Shark apart yesterday. I had a chat with Tom Tromey on Saturday and he mentioned some properties of Java bytecode of which I was unaware — like, you can compile it without using a stack pointer. I was trying to figure out how to modify my first pass to generate some of this stuff when a reply to a seemingly unrelated question made me realise that not only does the server JIT have a pass that does all this and more, but it’s been abstracted out for easy reuse! So stay tuned for the Shark that keeps its stack and locals in registers :)