johnm is currently certified at Journeyer level.

Name: John Marshall
Member since: 2000-04-21 03:42:14
Last Login: N/A

FOAF RDF Share This


A while back I was planning to be doing little shareware sorts of Palm OS applications, but a desire to do this in C++ sidetracked me into working on prc-tools, which is basically GCC targeting Palm OS, along with a few extra utilities. Now all my application ideas have fallen by the wayside and I just work on the toolchain and sometimes contribute to other Palm OS developer tools.

I worked at the Palm OS mothership for a couple of years, both on free software (prc-tools, small POSE tweaks) and on proprietary things like the Palm OS ROMs and SDKs.

However, my Silicon Valley adventure has come to an end; I now live in Norway and work at, where we make development tools for Palm OS programmers. Thus I still get paid to work on free software and I no longer have to wrestle with the insoluble compatibility issues that arise when you try to improve a broken SDK. Life is good.

Public key: 1024D/AC686552
Fingerprint: 2032 EEEB 38E2 C0E6 AC83  8BDA 9A70 3E71 AC68 6552


Recent blog entries by johnm

Syndication: RSS 2.0

Ob-it's been a really really really long time...

For today, this is just a wee note about my previous entry, in which I had found an inadequacy in ccache's hash. It had returned a cached object with incorrect file paths in its debug information, which the linker was then printing out in warnings and error messages.

Other people must have been noticing that too, because soon thereafter ccache acquired a CCACHE_HASHDIR option to put the whole path into the hash calculation.

That wouldn't help me much, of course, because with my build directory varying daily it would be the same thing as just disabling the cache entirely! Instead, I wound up using a small sed script to canonicalise the logs before I looked at them.

Pretty stone age, but effective.

Extreme panic!

Each time I do a full prc-tools build, I get a 1.3 Mb build log and sometimes I even look at it. (Building prc-tools means building two binutilses, two GCCs, a GDB, and a few other bits and pieces. Along the way you build four BFDs and six libiberties. So that log builds up pretty quickly!)

At the moment, due to popular demand (three FreeBSD whiners :-)), I'm working on making --enable-nls work properly. To be sure, there's a bug here in the current top-level prc-tools, and I'm glad to be finally looking into it.

So I'm comparing the logs from builds with and without --with-included-gettext. I look at the differences between successive builds regularly (all hail tkdiff!), but these two (of course) differ from each other in different places from the usual. And there's many more differences to look at than usual, alas.

This means I'm studying different parts of the logs from usual. Here's a few lines from tonight's builds:

gcc -W -Wall -Wstrict-prototypes -Wmissing-prototypes -g -O2 -o size size.o bucomm.o version.o filemode.o ../bfd/.libs/libbfd.a ../libiberty/libiberty.a
bucomm.o: In function `make_tempname':
/home/johnm/work/rpmbuild/BUILD/prc-tools-20020831/m68k-palmos/ binutils/binutils/../../.././binutils/binutils/bucomm.c:236: the use of `mktemp' is dangerous, better use `mkstemp'

Huh? What's that path from the end of August doing there? That build tree is long since blown away -- each one is over half a gigabyte, so I nuke them pretty quickly. So I look at the build logs from the last week, and they all say 20020831, and sure enough that directory certainly hasn't existed for a long time.

What's going on? Has ext3 let me down? Is my hard drive failing again? Is the end nigh?

Panic stations! Single user mode, fsck... no, nothing.

Have you guessed yet? It was ccache.

I last really compiled bucomm.c back in August when I switched ccache on. One of ccache's big features over compilercache is that it doesn't give up in the face of warnings. So that 20020831 directory doesn't really exist; it's just that I'm getting the old stderr log from August. It's hard to see how ccache could correct this without rerunning the compilation or parsing the basic error format and guessing, so it's understandable; on the other hand, the Web page does say

[ccache provides] exactly the same object files and exactly the same compiler warnings that would be produced if you use the real compiler. The only way you should be able to tell that you are using ccache is the speed.

Sure, the speed is great -- ten minutes instead of 40 with my machine on this project -- but I've just learnt one grain of salt to apply to the warning guarantees :-).

Sure, there's other lessons to be learnt here: like don't vary your paths when you're trying to do repeatable builds (but some form of date convention is a useful and commonly used way to stay sane) and don't use absolute paths (but sometimes your tools will make them absolute whether you want it or not).

Anyway. Onward to the next disaster...

8 Sep 2002 (updated 8 Sep 2002 at 17:53 UTC) »

My fascinating question for today: What should malloc(0) return?

Composing a critique of somebody else's crappy half-baked stdlib implementation today caused me to take another look at my own crappy half-baked stdlib implementation. In part, my critique puts the memory allocation functions malloc(), free(), calloc(), and realloc() under the microscope. One of the four is easy: calloc() is really just malloc() in disguise -- the only wrinkle is to remember to call memset() afterwards, which the somebody else in question forgot :-( --, but the other three engage in an interesting dance.

The C99 standard has the following to say about malloc(0) and friends, and the text in C90 is similar:

If the size of the space requested is zero, the behavior is implementation-defined: either a null pointer is returned, or the behavior is as if the size were some nonzero value, except that the returned pointer shall not be used to access an object.

I've always preferred the latter behaviour for malloc(): it avoids overloading the meaning of a NULL return, so, especially if your size was a variable rather than a constant, you can say "if malloc returned NULL, then abort due to out-of-memory" without having to fudge about and check that it wasn't really a returned-NULL-due-to-zero-size situation. (Of course, you could say that people who actually call malloc() with a size of zero deserve what they get!)

That (the latter) is also the behaviour required of a C++ allocation function. So if you're writing operator new() in terms of malloc(), you have to know that your malloc() has the same preference as I do, or take extra care instead of just calling malloc().

Now let's look at some of the other requirements that C90 and C99 place on these functions:

   free (NULL)    = ({})
realloc (NULL, n) = malloc (n)
realloc (p, 0)    = ({ free (p); return NULL; })    when p != NULL

The first is just saying that free(NULL) does nothing, which is, of course, very convenient -- you get to avoid some tedious checking. The other two are parts of the definition of realloc(). They are spelt out in the C90 text; the C99 text as pertains to the last one is quite different, but it can still be derived from the text.

Now, the "when p != NULL" condition on the last rule is not very pleasant: it makes for an ugly algebraic rule. Futhermore, I would contend that programmers who actually write code that's literally "realloc (p, 0)", with p variable but a constant 0, expect realloc() to Do The Right Thing even when p is NULL, just as free() does. Thus I would contend that programmers actually expect that they are using a stronger set of rules, in which the third rule holds regardless of the value of p. (In particular, this is what the Linux man page says realloc() does; yes, the man page is way stronger here than the C Standard!)

In that case, we can calculate the value of malloc(0):

malloc (0) = realloc (NULL, 0)
           = ({ free (NULL); return NULL; })
           = NULL

So that means that the choice is really between:

  • either a realloc() that Does The Right Thing for size=0
  • or a malloc() that does what I've always preferred for size=0
You can't have both! So I've changed my mind: being able to write realloc(p, 0) and have it Just Work, like free(p) does, is more important to me than having malloc(0), which I never actually use, be easily distinguishable from memory exhaustion.

In retrospect, this was all obvious. The specification of implementation-defined behaviour that I quoted at the beginning applies to both malloc() and realloc(). Wanting realloc(p, 0) to be useful in cleanup functions (by just freeing through p, and not also allocating some dummy "zero-sized" object that the caller will ignore and will become garbage) means that you've already made the choice: realloc() given a size of zero will return a null pointer. Either that means you've also already made the choice for malloc() too; or you've decided to make the choice differently for the two functions, which would be really ugly and I'm not even sure is allowed by the Standard.

But playing with the algebra was fun while it lasted!

(Looks like I'm going to have to put that "malloc (size? size : 1)" stuff back in my library's default C++ allocation functions.)

Ilan writes
If Jango Fett was the genetic template for the storm troopers (which is what I assume they were implying), why do they have American accents in the first three movies? Were English accents considered a genetically undesirable trait and removed from the cloning process?

That's a New Zealand accent, mate! (Oh, and the answer to your question is no, English and other non-USA accents are not genetically undesirable :-).)

The kid's (Boba's) accent was pretty strong too. I was laughing during their conversation while flying through the asteroid field: it's not every day I get to hear a Kiwi accent out here in Norway.

ObLongTimeNoDiaryEntry:   I've been submerged with work for ages. I decided a while back that I was going to ignore everything else until I got the new release of prc-tools out the door. It took at least six weeks longer than it should have, alas, and I finally released 2.1 on Monday. Phew! So now I can get back to having a life, working on more interesting parts of prc-tools, and replying to all the friends' emails I've been ignoring (sorry Graham!).

I've been back home from New Zealand and PalmSource for a couple of weeks now. It's great to be back, especially since we've had a nice sprinkling of snow over the last few days: the frozen lake near my house is looking really really good. It's a beautiful place for a (very flat) walk at the moment.

Palm tools stuff

A compatibility bug in PilRC's parser has been bugging me for almost a year:

INTEGER 1000 10
got unilaterally changed to
back then. There was some value in this new VALUE keyword because it was a way of specifying negative numbers, but it should have been made optional so as not to break people's old source code.

Aaron is back motivated to make releases again (yay!), and on Friday I made a fix to make the VALUE optional that was better than my original fix that hadn't been integrated. It was a nightmare btw: every time I do anything in the PilRC source code I swear that I'm never going to touch it ever again.

So there's now a 2.9p2 PilRC release, and I can finally put new binaries up on the Sourceforge site. (I refused to update them until the source code compatibility bug was fixed.) I've put up x86 Linux RPMs tonight, and will make packages for the Cygwin users sometime over the next few days.

I did something a bit naughty in the RPMs: I created a new Group for them, instead of using one of the standard ones from Red Hat's RPM groups file. I'm in two minds about this; I've previously used Development/Tools for my prc-tools RPMs and resisted creating a new group. But the fact is everybody else does it, and it's helpful to the users if they can find the Palm OS-related tools in one place rather than having to hunt through the 800 other packages in Development/Tools.

After the next time rpmfind crawls through Sourceforge, hopefully people will be able to find Palm OS-related tools in

Development/Palm OS
Pretty soon I'll be making POSE RPMs and putting them there, and the RPMs of the next prc-tools release (any month now :-(, really) will be moving there from Development/Tools too.

raph's rebar

Describing builds as a lazy functional program. Wow. That is just a gorgeous idea!

skipping the compilation if it's already been done (in a previous invocation)
Do you mean in a previous compiler invocation (in a different part of the calculation) during the same build, or possibly a (memoized :-)) compile of the same thing during a previous full build? (Or maybe the question is meaningless in a functional context. :-))

The latter is similar to compilercache, which was recently featured on sweetcode, and which I've just started using. It caches object files based on the compiler flags and preprocessed source used to produce them, and just spits back the cached object instead of calling the compiler if sees that the exact same compilation has already been done. It's working wonders on testing POSE RPM builds: each full build takes a few minutes instead of twenty or more.

20 older entries...


johnm certified others as follows:

  • johnm certified hacker as Journeyer

Others have certified johnm as follows:

  • pretzelgod certified johnm as Journeyer
  • jooon certified johnm as Journeyer
  • lerdsuwa certified johnm as Journeyer
  • abg certified johnm as Journeyer
  • ardiri certified johnm as Journeyer
  • gibreel certified johnm as Journeyer
  • remle certified johnm as Journeyer
  • ladypine certified johnm as Journeyer

[ Certification disabled because you're not logged in. ]

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

Share this page