right to self-defense
Cool, gitmo is located in the free world after all.
[ Home | Articles | Account | People | Projects | FAQ ] right to self-defense
Cool, gitmo is located in the free world after all.
Dysfunctional
A kid? That must be good, I guess, if possible, yeah…
ait and tuna
Work is supposed to take most of your time, right? Survival, mouths to feed, all that stuff… But it can be fun, even if not _directly_ kernel related. Sure, there is life outside the kernel, hey, “kernel”… I keep listening to the head-honchos (heck, I had to use that term, it looks like spanish, the language of our capital, Buenos Aires!): “forget about the kernel, the action is somewhere else, in (l)userland!”
So here I am, in userland, playing with GUI stuff, drag and drop! Python! GTK! Wow! Sounds boring? No, I don’t think so. Discovery time is never boring. Its fun to try for hours to grok some new semantic domain. Even if you, in another life, wrote a DOS GUI system out of reading a marketing insertion on Unix Review in 1988 
So what have I been doing in this strange land? Well, when you work you have to show the numbers, and explain them, and remember what happened when you switched that knob or applied that patch, no?
Damn, alchemy doesn’t requires you remember all that stuff, just taste the new stuff, if you think its not poisonous, that is.
But if you need to remember… There is something cooking for this year OLS, or to the first conference after it if it thinks I’m too bollocksish 
And to _show you the code_, full of python newbie mistakes… but hey, I even dared to write python bindings for such supposedly interesting stuff as ethtool and schedutils.
And finally to what uses it: AIT, because short, meaningless names are en vogue. Anyway, try tuna, a tuning application that has a cool name, one that I unfortunately didn’t came up with but fortunately was near the genius that though about it, thanks!
And thanks to whoever did the right thing and brought Evgeniy to kernelplanet, he is da guy from Russia! Get healthy and in crazy coding frenzy mode again!
Dwarving
Cool stuff in the dwarves front. They are growing. One new feature I’ve been working on is to be able to change the word-size.
One can now use the DWARF info produced for an architecture with a X word-size and see how it looks like on another architecture with word-size Y.
More work is needed to cope with exceptions to a general set of alignment rules and then we’ll be able to ask pahole for a the best possible member organization that will work across a series of architectures.
The mailing list is picking up steam, with different suggestions being made and improvement ensuing, with more patches trickling in.
One example of recent changes that were discussed in the mailing list was the inline improvements in codiff, the binary diff tool, discussed with Ilpo Järvinen, that he right away used in debloating the Linux kernel network infrastructure.
Soon I’ll be hard pressed to do some further improvements, with database, probably sqlite, being used to cache the results using build-id, that is becoming the norm in the -debuginfo packages made by distros, used as a key to previous results.
Database caching, once implemented, will pave the way for historical data collection, when we’ll be able to see how data structures evolve over time, like when you keep drinking and see your belly go round 
At that point creating a script that checks out tag after tag (changeset even, since we have plenty unused CPU power and disk space) and builds a software project, creating database entries, this time SCM changeset/tag tagged.
Wouldn’t it be lovely to know exactly when that extra belly wrinkle fat was generated? 
dwarves mailing list
I invite everybody interested in the continuing development of pahole and the dwarves to join us subscribing the dwarves mailing list by sending a message to majordomo@vger.kernel.org with “subscribe dwarves” on the body of the message. There are people already working on packaging the dwarves for several Linux distributions and even working on man pages!
Quickie: dwarves 1.3 released, bitfields edition
F8 RPMS for x86_64 here, will build for other arches soon. If you saw a bitfield related BRAIN FART ALERT! please try again with this release.
ctracer on this release already generates systemtap scripts, here is a callgraph generated with it. Nevermind the timestamps, its using a too expensive routine.
pahole on debian!
I was just thinking about (finally) learning how to write debian packages when I came across this. Now I can continue procrastinating… Thank you whoever you are!
vee-tables
So now we have basic support for exposing the vtables information in the DWARF info for C++ objects, class__fprintf will just print something like this (from a struct in the CERN ATLAS project):
/* vtable has 7 entries: {
[5] = setProperty(_ZN9IProperty11setPropertyERK8Property),
[6] = setProperty(_ZN9IProperty11setPropertyERKSs),
[7] = setProperty(_ZN9IProperty11setPropertyERKSsS1_),
[8] = getProperty(_ZNK9IProperty11getPropertyEP8Property),
[9] = getProperty(_ZNK9IProperty11getPropertyERKSs),
[10] = getProperty(_ZNK9IProperty11getPropertyERKSsRSs),
[11] = getProperties(_ZNK9IProperty13getPropertiesEv),
} */
We still have to support multiple vtables, but its a good start, and by looking at the linkage_name (C++ mangle-o-rama) we can get an idea where the vtable entries are from anyway.
Break on thru, to the other side
I spent most of my early life as a contributor to free/open source software as a packager, coming from a life as a software developer in the dark dungeons. Now I’m just happy as a project I finally managed to make go to the 1st version, the funny little peoples one, get on gentoo, mandriva and now in one of the ones I respected the most while a packager: PLD. Thank you guys, its good to be packaged. Too late for a cl package tho 
Back Home
The dwarves presentation at OLS went pretty well, people even seem to have liked it. The paper is now available, use it as the documentation.
Implemented –expand_pointers, that unfolds the pointer types in the same way that –expand_types expand non-pointer types. Should be useful in getting a bigger picture of a project data structure maze of relationships.
It should be a good first step on helping with checking ABI breakage, just use it in the old and new binary and use plain old diff to see what changed, perhaps something down three pointer levels. abichk will probably be a combination of this and what codiff does.
Here is an example:
Lets look at struct request_list in the Linux kernel:
$ pahole -C request_list fs/super.o
struct request_list {
int count[2]; /* 0 8 */
int starved[2]; /* 8 8 */
int elvpriv; /* 16 4 */
/* XXX 4 bytes hole, try to pack */
mempool_t * rq_pool; /* 24 8 */
wait_queue_head_t wait[2]; /* 32 48 */
/* --- cacheline 1 boundary (64 bytes) was 16 bytes ago --- */
/* size: 80, cachelines: 2 */
/* sum members: 76, holes: 1, sum holes: 4 */
/* last cacheline: 16 bytes */
};
Now lets expand its pointers, just one, mempool_t, and one that doesn’t have lots of pointers, to fit into this blog entry:
$ pahole --expand_pointers -C request_list fs/super.o
struct request_list {
int count[2]; /* 0 8 */
int starved[2]; /* 8 8 */
int elvpriv; /* 16 4 */
/* XXX 4 bytes hole, try to pack */
/* typedef mempool_t */ struct mempool_s {
spinlock_t lock;
int min_nr;
int curr_nr;
void * *elements;
void * pool_data;
/* typedef mempool_alloc_t */ void * (*alloc)(gfp_t, void *);
/* typedef mempool_free_t */ void (*free)(void *, void *);
wait_queue_head_t wait;
/* --- cacheline 1 boundary (64 bytes) was 8 bytes ago --- */
} *rq_pool; /* 24 8 */
wait_queue_head_t wait[2]; /* 32 48 */
/* --- cacheline 1 boundary (64 bytes) was 16 bytes ago --- */
/* size: 80, cachelines: 2 */
/* sum members: 76, holes: 1, sum holes: 4 */
/* last cacheline: 16 bytes */
};
It even seems to naturally avoid expanding opaque types, i.e. struct forward declarations, etc, so if header file with structs in the ABI is well written it just doesn’t goes down the non-ABI type rabbit hole, which seems the right thing to do.
Use it with –expand_types and the complete picture can be seen. And in the Linux kernel it is, humm, big:
acme@mica build]$ pahole --quiet --expand_pointers -C inode fs/super.o | wc -l 1345 [acme@mica build]$ pahole --quiet --expand_pointers --expand_types -C inode fs/super.o | wc -l 121922 [acme@mica build]$
One explanation for this last example is needed tho: –expand_types expands a type possibly many times, as many as there are members of its type, –expand_pointers, on the other hand, only expands any given type once, for the first member that is a pointer to this type.
This is because –expand_types was implemented to help in finding what was the field at some offset from complex data structures with a deep hierarchy, –expand_pointers, on the other hand, used most of the –expand_types code, but was implemented to help in finding ABI breakage deep inside the type.
I’ll eventually implement a config option to tell that only the first type should be expanded in the –expand_types case, make that a shell script that takes as parameters two files, call pahole for the two, do a diff and show where the ABI stopped being a virgin 
New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.
Keep up with the latest Advogato features by reading the Advogato status blog.
If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!