Older blog entries for jum (starting at number 28)

I had the plan to look into packaging today, with rpm first on the list and then the MacOS X packaging tool. Unfortunately someone did check in changes at a central library and left everything unbuildable. I hate it if one checks in changes without trying it first.

Found out why the Tru64 build was broken for quite a while. We recently switched compiler -D switches to _XOPEN_SOURCE=500. From looking at sys/socket.t this looked like it implies -D_SOCKADDR_LEN, so our main machine.h file defined HAS_SALEN for this architecture. The HAS_SALEN define means that this machine uses variable length struct sockaddr's, and thus one has to do extra voodoo to handle the struct ifreq array returned by SIOCGIFCONF. As I just found out the sys/ioctl.h file does not bother looking at _XOPEN_SOURCE and thus defined SIOCGIFCONF to the old version with fixed length structures. The sa_len member of each structure remains set in this case, and this made my code stumble across unaligned accesses. Turning on -D_SOCKADDR_LEN on the command line fixes that and all runs fine again.

Indeed smswebde works again with gocr. It is pretty funny that one has to read numbers from a gif file to make that work.

On the development side I implemented a plug-in class for our license mechanism to allow for multiple ways to determine the unique machine ID used to license the software. Traditionally this has been the unique hostid or some ethernet address, but on some architectures we will extract this number from a custom made USB dongle. Due to the machine ID mechanism put out as shared library instead of being linked statically into all programs it is surely easier to crack, but it also much more easy to support new dongle types. People fully determined to crack our license mechansim probably found other means already, so the additional insecurity is probably offset by the flexibility.

As usual the COM clone I did two years ago makes these kind of plug-ins very easy to implement. The Microsoft COM mechanism is pretty nicely designed, and the clone I did away with a few things I did not like. For one thing I did not use UUID's but Java style reverse domain names for interface ID's and factory ID's, this makes the source a bit easier to read. I also did not use the concept of globally registering objects in the Registry as Microsoft did it but in a per directory cache file. Applications then specify which directories the COM subsystem should look at and all objects in that directories are available. The only thing that can get a bit difficult with COM is properly keeping all the reference counts under control.

I have contacted the smswebde.pl author and indeed ocr will be the only way to get it working. Appearently he already had some successes with gocr and will post an update soon. I will not believe it until I have seen it working. :-)

Well, I have been using a script named smswebde.pl to send new mail notifications to my cell phone via sms for quite some time. This uses my web.de freemail account (which I did create just for that purpose). Recently this script stopped working and today I took the time to check why. To my surprise the freemail provider now requires to enter an additional numeric code into the send form to authenticate the sms send. The code itself is on the web page, but as gif image! Well done web.de folks, that one really surprised me. But not that I have given up, I think I will have to look at the encoded hidden fields if I can find anything that helps me to find that authentication number. I do not think that OCR on a gif file is practical in this situation.

For quite some time we have used a system named Preferences to consolidate all the configuartion information we have into one place. It was initially used only by the OPI subsystem and over the years other programs migrated to Preferences as well. Currently we are progressing for the next version to have nearly no ASCII readable configuration any more, everything is stored in the hierarchical Preferences data base (which happens to be some HFS deriative for those who know what HFS is).

So this decision will probably make the Unix old timers cringe as one of the hallmarks of Unix is the abundance of text config files. It also smells in particular like Windows Registry, which will not necessarily attract Unix wizards. But it really makes quite a few things easier. For example setting a single parameter is just a single command:


prefvalue -k Programs/atalkd/if -t strlist "hme0,le0"

Similarily it is easy do delete one parameter by using a command like this:

prefvalue -k Programs/atalkd/debug -d

All this can be entered on the command line without needing an editor, hunting down for the proper line and editing the line while not messing up the syntax. And internally the API is really easy to extract these parameters or iterate over the tree. The tree form also makes some more advanced usage possible, like having the same tree structure rooted at various points in the tree needs just one parameter changed in the API.

One of major complaints against binary configuration files is the inability to fix the file if something goes wrong. We added two commands named prefdump and prefrestore to help with that, prefdump dumps all the Preferences into an ASCII file and prefrestore restores it. In particular on system start we always save one prefdump away. On the modification side we do always completely rewrite the file under a new name and rename it after it is complete to avoid partial updates.

Today I got haunted by a rather old bug that surfaced on changing the size of a basic data type. At first it was a bit strange as the program in question did run fine on one machine and not on the other, both of which are SPARCstations running Solaris 7 with the same patches installed.

One one machine the program did bus error shortly after connecting from a Mac client. As it turned out one machine was running with one crucial library compiled in debug mode without optimization. The bug was that in this library the structure alignment requirements did change due to changing off_t to 64 bits as the result of the large file compile environment.

The library did place these structures into a shared memory segment using it's own allocator, but did align only a 32 bit boundary. The C compiler without optimizer did pad all structures to at least 64 bit boundaries and thus the problem did not appear. With the optimizer turned on not all of the structures placed into shared memory are aligned to 64 bits and thus at some point an off_t struct member was not properly aligned for the SPARC processor.

As one can see again a programming error (failing to provide the proper alignment) can go unnoticed for a long time and a change in the environment can make it pop up in code believed to be rock solid.

In my ongoing effort to make perl/apache/modperl work well together under AIX I did follow the lead of the apache developers to switch over to the AIX native dlopen instead of using my proven emulation using load() and loadbind()if the AIX version is 4.3 or above. Actually I did have some indication that using the native dlopen in apache and the emulation in modperl did produce conflicts. I thus patched perl (and submitted the patch) to use the native dlopen and it worked fine in my configuration.

Unfortunately it does not work if modperl itself is a DSO to apache, as the native dlopen has no sane way to backreference symbols in the parent, it can only easily backreference to the main part, e. g. Apache. Both apache and perl use dlopen with the RTLD_GLOBAL option to add symbols of loaded plug-ins to the global group, and all other dlopen implementations to resolve references to the global group upon loading further shared objects. The AIX dlopen cannot do that, it can only reference symbols in the real main part, so only the apache symbols are available and not the perl ones. Sigh.

I am attempting to get things organized for my vacation, I will be leaving Friday morning for two weeks worth of sailing in the caribbean. Mostly this means delegating work to someone else, most of the low level library changes are progressing well now. There are still substantial things left to do, especially at the application level to cope with all the changes in the directory structure. What is still not decided if we will include perl as part of the base install to make scripting and installation easier.

Did clean up RPC code today. As we do send stat structures across the wire there are all kinds of custom types like ino_t and dev_t that are different across the platforms. I have settled to send most of this stuff as int64 instead of uint32, which is in most modern RPC implementations converted using xdr_hyper. The exception appears to be MacOS X and DG/UX, which do not have xdr_hyper, so I will have to make my own. Beyond the fact the ONC/RPC works quite ok it has a badly designed type system in my opinion.

19 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!