Older blog entries for pphaneuf (starting at number 7)

Forgot to mention something in my previous diary entry...

I hope that timers/timeouts will be integrated in that API.

Maybe they could be added afterwards as file descriptors objects? That could be even cleaner.

Asynchronous I/O

Linus Torvalds is a freakin' genius. I find some problem with his proposed API because of the fact that the events are edge-triggered, and I like being able to read or write only a limited amount and have the event trigger again to continue (so that all connections are given a fair chance).

Actually, that's what I thought upon first reading this, but you see, the guy is really a freakin' genius!

It's not a perfect proposal (means many system calls for binding file descriptors, if you want to rebind after every event), but it's a really good idea. The bind_event system call could be made user-space and its functionality rolled into the get_events system call, so that the next time get_events is called, it gets the events to bind/rebind, and then returns what's new.

I know there are racy problems with this last paragraph, but they could be fixed, I just don't know how (did I mention Linus is a freakin' genius?)...

I think I'm getting the point of Christian (after a few more e-mails): he's using automake, he likes it, and that's the way it is. Even if each of my makefile subparts are about as big as a Makefile.am. I'm pretty sure that if I could hack a version of automake that would produce non-recursive makefiles, he'd use it.

Anyone who's a firm believer of recursive makefiles should hack on Mozilla a bit. On my Pentium 225, an empty make (one where there is nothing to build, like if you do make twice) takes upward of 15-20 minutes! And it's nothing close to "exact dependencies".

Maybe I should do just that, hack automake to produce a single non-recursive makefile?

But I also have to hack so many other things...

Movies

I went to see Meet the Parents yesterday with my girlfriend. It's okay, but then, I loved 10 Things I Hate About You, so I guess I'm some kind of twisted, crazy moviegoer. The Whole Nine Yards is also very good, plus you get to see Montréal!

Woke up around 8 AM today, was weird waking up this early. Now, will I be able to stop hacking on my girlfriend computer to go to work before noon? That remains to be seen. :-)

Non-recursive makefiles again...

I e-mailed Christian Rainiger (of LDGC and late PenguinPlay fame) to tell him that I finally had a non-recursive makefile setup worthy of looking at (in Quadra). We argued about this in the past, but Quadra was proprietary at the time and XPLC was (and still is) too small for fair comparison.

He still don't believe in it. I just can't believe he doesn't sees this, this is one of the top 5 things that improved my hacking productivity in the recent years (along with using X Window and XEmacs, upgrading my 486 to a Pentium and a few other things)...

Just think of it: no "make clean ; make" just to be sure everything is rebuilt correctly! "make clean" is only used when I change the compilation options for example. Also, an empty make (the second make in "make && make") is under 2 seconds.

Maybe I'll just hack a set of makefiles to some common project to provide a clear comparison (in my Copious Tree Time (tm)).

Mozilla

I worked on Mozilla a while ago and I was curious to see how it was going these days, so I downloaded Netscape 6 PR3 to get a feel of what Netscape was doing with it.

Everybody's saying that Mozilla is smooth, but does that also apply to Netscape 6? I know, there's those buy-this icons and stuff, but the slowness is incredible. I love the design and how things are laid out and the architecture, but seeing dialog box paint themselves bit by bit is not fun at all!

Now, I still have my CVS account there, should I pull the latest version and build with some of the "configure" script optimization options or is PR3 the fastest it can be?

BTW, I have a Pentium 225 (overclocked 166 MMX, with a 75 MHz bus) with 96 megs of RAM, Red Hat Linux 7.0 (with a very simple Window Maker setup, no memory-eating GNOME or KDE) and Netscape 4.75 is just fine. I find my box quite crisp most of the time (for its age, it's pretty good), but when I start Netscape 6 PR3, it makes me think of my 486 again.

Argh, gcc is so braindead at times... According to the man page:

-MD    Like `-M' but the dependency information is written
       to files with names made  by  replacing  `.o'  with
       `.d'  at the end of the output file names.  This is
       in   addition   to   compiling    the    file    as
       specified--`-MD' does not inhibit ordinary compila­
       tion the way `-M' does.

Except one detail: it doesn't do that. I use "-o foo/bar.o", so my output file is foo/bar.o. Replacing '.o' with '.d' should yield foo/bar.d, but no, bar.d comes out, putting all of the dependency files in the top level directory.

Nice.

The DEPENDENCIES_OUTPUT environment variable seems promising, but it can only take a single file as the target of the rule (I would like to make the dependency file itself a dependent, so that it is automatically updated).

Other than that, the nearly perfect dependency checking and amazing speed of non-recursive makefiles is incredible. If more people would try that, they'd ditch automake pretty quickly (or automake would be updated to create non-recursive makefiles, which would be even better!).

I say "nearly", because I still use a single dependency file, which I obviously don't rebuild every time, so it could get wrong. But it is very easy to get a perfect dependency build by simply removing the file to force the recomputation of dependencies.

apm did a nice write-up of ExtremeProgramming. About courage in XP: it's also about making changes that would seem dangerous in a more conservative development methodology, relying on the extensive test scaffolding that XP requires.

Note that there already is a OpenSourceExtremeProgramming page on Wiki, for those interested. Link in that page, like ExtremeHacking, are also of interest.

I am trying to develop XPLC in such a OSS/XP mix. For example, unit tests comes with every parts of the system and it is very easy for a developer to simply run a "make test" before doing a CVS commit. Of course, since I'm alone on that project, it's rather hard to follow-up on the "pair programming" part!

I wouldn't totally agree with his claim that "most OSS projects are pretty good at adhering to coding standards". The largest ones sure do (they would collapse otherwise), but I would say that you see a lot of variation in this department.

I hacked autoconf into Quadra today. I'm a bit afraid that we'll have to bite the bullet and modularize the thing (as in dynamically loaded shared objects), to reduce runtime dependencies on crazy optional libraries (like SVGAlib). Damn, I wish XPLC was ready, it would just be perfect for the job...

There is so many things I would like to do with Quadra, like using ZIP files for resources (like Quake III Arena does), turning the underlying game library into nice XPLC components, etc...

If anyone wants to help, go ahead! :-)

10 Nov 2000 (updated 10 Nov 2000 at 23:55 UTC) »

I am trying to see how one is supposed to bootstrap himself with this certification system. I think it is a very cool idea, but it's hard "getting a voice".

I would have liked being able to comment on the Autobook article today, to say that RecursiveMakeConsideredHarmful, seeing how Automake encourages recursive makefiles.

Nice, the diary being Wiki-enabled! PierrePhaneuf loves Wiki!

:-)

Thought about threads: as I wrote about on my home page, I dislike threads and their pervasive overuse in modern software (both closed and open source). One of the big problem where I often have to resolve to threads (or fork) is when one of your inputs is time.

Most other forms of inputs can be turned into a file descriptor that can be fed to some select()-based main loop (like Xt or the one in glib), like signal handlers writing a byte into a pipe. A file descriptor is very handy, and can easily brige arbitrary libraries. For example, you can get the ConnectionNumber() of an Xlib display, and register it with glib main loop.

Time can be handled in such loops (with the select() timeout), and libraries like glib and Xt have hooks to register time-based events, but the problem is that a library that want to register a time event has to know the main loop library itself.

This problem came up trying to think of a way to do asynchronous DNS nicely without resorting to threads. For (the rarely used) TCP DNS requests, you can just select() that, but for the (much more common) UDP requests, you might never get anything, and the library has to timeout at some point.

I could implement that async DNS library as a component of my main loop system, but that is ugly and has no doing there. I want someone to be able to use my library with Xt, glib and other main loop libraries!

It would be nice if timers could exist as file descriptors. I think they do on Win32 (as handles usable in WaitForMultipleObjects or somesuch).

Yes, I know of SIGALRM, but you can only have one of these at once. I could spawn a thread or fork something to do this, but I could as well do the whole lookup in there.

Oh well. That's life I guess!

P.S.: dank has good ideas about threads and select() and all of this. Kudos for his C10K web page.

10 Nov 2000 (updated 26 Jul 2006 at 12:14 UTC) »

I was reviewing a library on SourceForge (ZooLib), and again, I was just shocked at the threading requirements. I still have trouble thinking that my ICQ client starts a gazillion threads and that my girlfriend (who uses GNOME) panel uses more memory (not counting applets!) than my fancy Window Maker process.

Check out the threading rant on my web site to read more about this.

I really am chagrined to see how little time I have, and how many interesting projects, all worth of my attention, I am involved with. A brief list:

  • Quadra
  • XPLC
  • A web environment for my employer, which is very promising and is not merely a web site. I am building an engine which is a cross between Zope and the Ars Digita Community System. I'll try to have this open sourced, of course!
  • Various smaller systems research projects, like high performance network servers, efficient integration of multiple programs in a single process and others.
  • Last but not least: my life, my sanity and my girlfriend.

This is Hard.

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!