9 Jul 2004 bluefoxicy   » (Observer)

I'd like to thank all those who have certified me oh so long ago, back before the time where I forgot about this account.

I'm starting up an article, called Bloat, about how bloated and clunky software is. I suppose I'll make diary posts and work on SVTK in the mean time. Please excuse the unkempt, unproofread nature of this entry.

I'm tired of the bloat these days. Look at Firefox. Look at Dillo. Do you think 100M more code and about 80-500M resident size increase will come from adding in JavaScript, client side XML, and a plug-in to translate from Netscape plug-ins? I doubt it. Maybe the system requirements will go up to a 12Ghz cpu as well?

While I ponder this, I'm working on an object oriented framework in Objective-C, SVTK. This framework will be pretty bloated itself I guess; but I'll still make it well made. We're talking about 44 inherant bytes in the base SVObject; plus two pthreads_key_t and two pthread_mutex_t locks gotten with malloc(), however big those are.

NOTE: All numbers ahead are approximate and qualitative; do not take them as actual statistics. I gathered approximations with a few testcases and `time`.

The need for this mass comes from my need to support fault recovery. The overhead from this should be low (very low). Let's tally:

-retain -lock swizzle lock (8) -lock retain lock (6) -incriment retain value (1/2) -unlock retain lock (6) if(in_threaded_mode) (2) -read specific key (4) -incriment value (1/2) -unlock swizzle lock (8) Total cost: 35 Thread safety costs: 12 Swizzle safety costs: 16 Fault tolerance costs: 6.5

We'll call (j = k) temporally 1, so the numbers above are in relation to how long it takes to do (j = k), approximately.

Outside threaded mode, locks are cheap. Also, with swizzling disabled permenantly on an object, the swizzle locking is cheapened due to lack of need. The new calculations are as follows:

-retain -lock swizzle lock (2) -lock retain lock (2) -incriment retain value (1/2) -unlock retain lock (2) if(in_threaded_mode) (2) -read specific key (0) (not executed) -incriment value (0) (not executed) -unlock swizzle lock (2) Total cost: 10.5 Thread safety costs: 4 Swizzle safety costs: 4 Fault tolerance costs: 2

These numbers aren't achievable in a threaded environment; because real locks are used when threaded, the thread safety cost becomes 12; fault tolerance becomes 7.5; total becomes 25.

In truth, I'm going to have to sacrifice the equivalence of 12 lines of (j = k) to be thread safe, 16 if you want to be able to swizzle an object, and 6.5 to clean up after faults in the program. This is a total of 34.5 simple assignments; 22.5 in members that don't need to do special locking (this will be most members).

This isn't including message passing overhead. It's also based around simple, single-assignment lines of code; the line, if (my_field == anArg->field), is probably ~5-7 on this scale. Also, these estimates are extremely arbitrary anyway, and give no accurate measure of how much overhead I'll be adding to do this.

This is complex shite. I supply this all to make the programmer's life simpler. The actual swizzling process that these facilitate is a major operation with a lot of overhead, which would likely be impossible to actually code above the framework.

Let's finish up with fault tolerance. The fault tolerance stuff is there to make sure that if a thread is killed off--or whenever a thread ends really--it will attempt to release retained objects and unlock held locks. It costs a menial level of overhead; but it does one magical thing: It prevents memory leaks and deadlocking.

Deadlocking is when a program encounters a blocking lock which is held by a thread which is either itself waiting for that lock to unblock, or which has ceased to run. In either of these situations, the body of code having locked the lock is no longer executing; and thus, the lock can never be unlocked. Any code encountering this lock will deadlock, and never continue.

My threading object will take a void*(*function)(void*) function pointer and the void* argument as arguments to create a thread. It will encase these in a structure, and pass that structure and a pointer to the condestructor for threading to the pthread_create() function (or whatever native equivalent there is).

The condestructor will de-encapsulate the structure it is passed and execute the initializer with the argument. Upon return, it will release all local autorelease pools and then return whatever the initializer function returned.

In the event of a memory leak or deadlock scenario, such as when an exception terminates a thread, the condestructor will be either returned to during this state OR called with NULL as the argument. In the latter case, it will simply proceed through to releasing the autorelease pools and then return an error.

Once the autorelease pools are free, we can proceed to release retained objects and unlock held locks in any order we please. This will have the pleasant effect of ensuring that excessive releasing will not occur from releasing objects and then letting autorelease pools re-release them; and at the same time releasing objects that are retained by the thread. It will also unlock held locks, allowing other threads waiting on those locks to continue.

I am pondering adding an sv_trash() to put data in a thread-local "trash bin" which upon thread termination is emptied. This would facilitate avoidance of memory leaking from regular malloc()ed data.

Ah well. I should shut up now. This post is poorly written and covers much of just me talking to myself. :)

Latest blog entries     Older blog entries

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!