Recent blog entries for acme

Cross platform perf.data analysis support


There are still some problems related to loading vmlinux files, but those are
unrelated to the feature implemented in this patch, so will get fixed in the
next patches, but here are some results:

1. collect perf.data file on a Fedora 12 machine, x86_64, 64-bit userland

2. transfer it to a Debian Testing machine, PARISC64, 32-bit userland

acme@parisc:~/git/linux-2.6-tip$ perf buildid-list | head -5
74f9930ee94475b6b3238caf3725a50d59cb994b [kernel.kallsyms]
55fdd56670453ea66c011158c4b9d30179c1d049 /lib/modules/2.6.33-rc4-tip+/kernel/net/ipv4/netfilter/ipt_MASQUERADE.ko
41adff63c730890480980d5d8ba513f1c216a858 /lib/modules/2.6.33-rc4-tip+/kernel/net/ipv4/netfilter/iptable_nat.ko
90a33def1077bb8e97b8a78546dc96c2de62df46 /lib/modules/2.6.33-rc4-tip+/kernel/net/ipv4/netfilter/nf_nat.ko
984c7bea90ce1376d5c8e7ef43a781801286e62d /lib/modules/2.6.33-rc4-tip+/kernel/drivers/net/tun.ko

acme@parisc:~/git/linux-2.6-tip$ perf buildid-list | tail -5
22492f3753c6a67de5c7ccbd6b863390c92c0723 /usr/lib64/libXt.so.6.0.0
353802bb7e1b895ba43507cc678f951e778e4c6f /usr/lib64/libMagickCore.so.2.0.0
d10c2897558595efe7be8b0584cf7e6398bc776c /usr/lib64/libfprint.so.0.0.0
a83ecfb519a788774a84d5ddde633c9ba56c03ab /home/acme/bin/perf
d3ca765a8ecf257d263801d7ad8c49c189082317 /usr/lib64/libdwarf.so.0.0
acme@parisc:~/git/linux-2.6-tip$

acme@parisc:~/git/linux-2.6-tip$ perf report –sort comm
The file [kernel.kallsyms] cannot be used, trying to use /proc/kallsyms…

^^^^ The problem related to vmlinux handling, it shouldn’t be trying this
^^^^ rather alien /proc/kallsyms at all…

/lib64/libpthread-2.10.2.so with build id 5c68f7afeb33309c78037e374b0deee84dd441f6 not found, continuing without symbols
/lib64/libc-2.10.2.so with build id eb4ec8fa8b2a5eb18cad173c92f27ed8887ed1c1 not found, continuing without symbols
/home/acme/bin/perf with build id a83ecfb519a788774a84d5ddde633c9ba56c03ab not found, continuing without symbols
/usr/sbin/openvpn with build id f2037a091ef36b591187a858d75e203690ea9409 not found, continuing without symbols
Failed to open /lib/modules/2.6.33-rc4-tip+/kernel/drivers/net/e1000e/e1000e.ko, continuing without symbols
Failed to open /lib/modules/2.6.33-rc4-tip+/kernel/drivers/net/wireless/iwlwifi/iwlcore.ko, continuing without symbols
<SNIP more complaints about not finding the right build-ids,
those will have to wait for ‘perf archive’ or plain
copying what was collected by ‘perf record’ on the x86_64,
source machine, see further below for an example of this>

# Samples: 293085637
#
# Overhead          Command
# ……..  ……………
#
61.70%             find
23.50%             perf
5.86%          swapper
3.12%             sshd
2.39%             init
0.87%             bash
0.86%            sleep
0.59%      dbus-daemon
0.25%             hald
0.24%   NetworkManager
0.19%  hald-addon-rfki
0.15%          openvpn
0.07%             phy0
0.07%         events/0
0.05%          iwl3945
0.05%         events/1
0.03%      kondemand/0
acme@parisc:~/git/linux-2.6-tip$

Which matches what we get when running the same command for the same perf.data
file on the F12, x86_64, source machine:

[root@doppio linux-2.6-tip]# perf report –sort comm
# Samples: 293085637
#
# Overhead          Command
# ……..  ……………
#
61.70%             find
23.50%             perf
5.86%          swapper
3.12%             sshd
2.39%             init
0.87%             bash
0.86%            sleep
0.59%      dbus-daemon
0.25%             hald
0.24%   NetworkManager
0.19%  hald-addon-rfki
0.15%          openvpn
0.07%             phy0
0.07%         events/0
0.05%          iwl3945
0.05%         events/1
0.03%      kondemand/0
[root@doppio linux-2.6-tip]#

The other modes work as well, modulo the problem with vmlinux:

acme@parisc:~/git/linux-2.6-tip$ perf report –sort comm,dso 2> /dev/null | head -15
# Samples: 293085637
#
# Overhead          Command                      Shared Object
# ……..  ……………  ……………………………
#
35.11%             find                   ffffffff81002b5a
18.25%             perf                   ffffffff8102235f
16.17%             find  libc-2.10.2.so
9.07%             find  find
5.80%          swapper                   ffffffff8102235f
3.95%             perf  libc-2.10.2.so
2.33%             init                   ffffffff810091b9
1.65%             sshd  libcrypto.so.0.9.8k
1.35%             find  [e1000e]
0.68%            sleep  libc-2.10.2.so
acme@parisc:~/git/linux-2.6-tip$

And the lack of the right buildids:

acme@parisc:~/git/linux-2.6-tip$ perf report –sort comm,dso,symbol 2> /dev/null | head -15
# Samples: 293085637
#
# Overhead          Command                      Shared Object  Symbol
# ……..  ……………  ……………………………  ……
#
35.11%             find                   ffffffff81002b5a  [k] 0xffffffff81002b5a
18.25%             perf                   ffffffff8102235f  [k] 0xffffffff8102235f
16.17%             find  libc-2.10.2.so                     [.] 0×00000000045782
9.07%             find  find                               [.] 0×0000000000fb0e
5.80%          swapper                   ffffffff8102235f  [k] 0xffffffff8102235f
3.95%             perf  libc-2.10.2.so                     [.] 0×0000000007f398
2.33%             init                   ffffffff810091b9  [k] 0xffffffff810091b9
1.65%             sshd  libcrypto.so.0.9.8k                [.] 0×00000000105440
1.35%             find  [e1000e]                           [k] 0×00000000010948
0.68%            sleep  libc-2.10.2.so                     [.] 0×0000000011ad5b
acme@parisc:~/git/linux-2.6-tip$

But if we:

acme@parisc:~/git/linux-2.6-tip$ ls ~/.debug
ls: cannot access /home/acme/.debug: No such file or directory
acme@parisc:~/git/linux-2.6-tip$ mkdir -p ~/.debug/lib64/libc-2.10.2.so/
acme@parisc:~/git/linux-2.6-tip$ scp doppio:.debug/lib64/libc-2.10.2.so/* ~/.debug/lib64/libc-2.10.2.so/
acme@doppio’s password:
eb4ec8fa8b2a5eb18cad173c92f27ed8887ed1c1                   100% 1783KB 714.7KB/s   00:02
acme@parisc:~/git/linux-2.6-tip$ mkdir -p ~/.debug/.build-id/eb
acme@parisc:~/git/linux-2.6-tip$ ln -s ../../lib64/libc-2.10.2.so/eb4ec8fa8b2a5eb18cad173c92f27ed8887ed1c1 ~/.debug/.build-id/eb/4ec8fa8b2a5eb18cad173c92f27ed8887ed1c1
acme@parisc:~/git/linux-2.6-tip$ perf report –dsos libc-2.10.2.so 2> /dev/null

# dso: libc-2.10.2.so
# Samples: 64281170
#
# Overhead          Command  Symbol
# ……..  ……………  ……
#
14.98%             perf  [.] __GI_strcmp
12.30%             find  [.] __GI_memmove
9.25%             find  [.] _int_malloc
7.60%             find  [.] _IO_vfprintf_internal
6.10%             find  [.] _IO_new_file_xsputn
6.02%             find  [.] __GI_close
3.08%             find  [.] _IO_file_overflow_internal
3.08%             find  [.] malloc_consolidate
3.08%             find  [.] _int_free
3.08%             find  [.] __strchrnul
3.08%             find  [.] __getdents64
3.08%             find  [.] __write_nocancel
3.08%            sleep  [.] __GI__dl_addr
3.08%             sshd  [.] __libc_select
3.08%             find  [.] _IO_new_file_write
3.07%             find  [.] _IO_new_do_write
3.06%             find  [.] __GI___errno_location
3.05%             find  [.] __GI___libc_malloc
3.04%             perf  [.] __GI_memcpy
1.71%             find  [.] __fprintf_chk
1.29%             bash  [.] __gconv_transform_utf8_internal
0.79%      dbus-daemon  [.] __GI_strlen
#
# (For a higher level overview, try: perf report –sort comm,dso)
#
acme@parisc:~/git/linux-2.6-tip$

Which matches what we get on the source, F12, x86_64 machine:

[root@doppio linux-2.6-tip]# perf report –dsos libc-2.10.2.so
# dso: libc-2.10.2.so
# Samples: 64281170
#
# Overhead          Command  Symbol
# ……..  ……………  ……
#

14.98%             perf  [.] __GI_strcmp
12.30%             find  [.] __GI_memmove
9.25%             find  [.] _int_malloc
7.60%             find  [.] _IO_vfprintf_internal
6.10%             find  [.] _IO_new_file_xsputn
6.02%             find  [.] __GI_close
3.08%             find  [.] _IO_file_overflow_internal
3.08%             find  [.] malloc_consolidate
3.08%             find  [.] _int_free
3.08%             find  [.] __strchrnul
3.08%             find  [.] __getdents64
3.08%             find  [.] __write_nocancel
3.08%            sleep  [.] __GI__dl_addr
3.08%             sshd  [.] __libc_select
3.08%             find  [.] _IO_new_file_write
3.07%             find  [.] _IO_new_do_write
3.06%             find  [.] __GI___errno_location
3.05%             find  [.] __GI___libc_malloc
3.04%             perf  [.] __GI_memcpy
1.71%             find  [.] __fprintf_chk
1.29%             bash  [.] __gconv_transform_utf8_internal
0.79%      dbus-daemon  [.] __GI_strlen
#
# (For a higher level overview, try: perf report –sort comm,dso)
#
[root@doppio linux-2.6-tip]#

So I think this is really, really nice in that it demonstrates the portability
of perf.data files and the use of build-ids across such aliens worlds :-)

There are some things to fix tho, like the bitmap on the header, but things are
looking good.

Syndicated 2010-01-14 14:55:38 from Arnaldo's Ramblings

perf on parisc64


It built, after removing -fstack-protector-all that is not available for that target and suppressing the libelf-dev tests.

Transferred a perf.data file created in a i386 machine and… it fails. Endianness issues, I guess. Will fully investigate this in the next days.

Syndicated 2010-01-08 21:28:38 from Arnaldo's Ramblings

Modules encoded


Ended up encoding modules as PERF_RECORD_MMAP events details at: http://lkml.org/lkml/2010/1/7/370. Lets see how people react.

Syndicated 2010-01-07 22:08:26 from Arnaldo's Ramblings

Recording where modules were loaded in perf.data


While trying to fix the build-id generation so as not to produce duplicates, I noticed another problem that needs to be solved before we can introduce perf archive and be able to analyse a perf.data file recorded in one machine on another, possibly with a different architecture and OS.

The problem is similar to the relocatable kernel problem solved today: we need to have perf events that state where kernel modules were loaded, right now we are using the current /proc/modules to get that information, but it can no longer have some modules, unloaded after perf record and before perf report.

To properly fix that we need the kernel infrastructure to emit PERF_MODULE_LOAD/PERF_MODULE_UNLOAD events just like it does when DSOs get loaded by means of executable mmap, when it emits PERF_MMAP/PERF_MUNMAP events, so that there are no races and we can support long running perf record sessions where modules get loaded/unloaded.

Tomorrow I’ll work on synthesizing such events in perf record and then when all works we can do the kernel bits and stop synthesizing then in user space.

Syndicated 2010-01-06 00:50:35 from Arnaldo's Ramblings

dwarves on the spotlight

Recently I saw somebody boasting that Linus now uses Fedora and I don’t know why I found it silly. But then when Ingo Molnar even mentioned the name of the package where codiff is available I found that I can be silly too ;-)

Syndicated 2009-02-06 02:07:18 from Arnaldo's Ramblings

Thank you Jeff & Aristeu

Someplace -> My home: Aristeu Rozanski
My home -> usefulness: Jeff Garzik (best guess)

[root@doppio tb]# mount | grep tera
/dev/mapper/teravg1-teralv1 on /media/tb type ext3 (rw)
[root@doppio tb]#
[root@doppio tb]# lspci | grep SATA
00:1f.2 SATA controller: Intel Corporation 82801GBM/GHM (ICH7 Family) SATA AHCI Controller (rev 02)
16:00.0 Mass storage controller: Silicon Image, Inc. SiI 3512 [SATALink/SATARaid] Serial ATA Controller (rev 01)
[root@doppio tb]#
root@doppio tb]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol01
16251816 14685780 727176 96% /
/dev/mapper/VolGroup00-LogVol02
74116232 24936588 45355968 36% /home
/dev/sda1 2030736 163560 1762356 9% /boot
tmpfs 1025588 2460 1023128 1% /dev/shm
/dev/sdc1 57685532 43834840 10920440 81% /media/disk-1
/dev/mapper/teravg1-teralv1
961428808 428881904 483709068 47% /media/tb
[root@doppio tb]#

Syndicated 2008-12-13 00:58:08 from Arnaldo's Ramblings

tuna & the oscilloscope

Since I haven’t blogged about these toys, please take a look at Carstes’s article at OSADL and perhaps at my OLS 2008 paper too.

Syndicated 2008-07-30 13:57:58 from Arnaldo's Ramblings

right to self-defense

Cool, gitmo is located in the free world after all.

Syndicated 2008-06-13 02:40:47 from Arnaldo's Ramblings

Dysfunctional

A kid? That must be good, I guess, if possible, yeah…

Syndicated 2008-03-17 23:06:39 from Arnaldo's Ramblings

ait and tuna

Work is supposed to take most of your time, right? Survival, mouths to feed, all that stuff… But it can be fun, even if not _directly_ kernel related. Sure, there is life outside the kernel, hey, “kernel”… I keep listening to the head-honchos (heck, I had to use that term, it looks like spanish, the language of our capital, Buenos Aires!): “forget about the kernel, the action is somewhere else, in (l)userland!”

So here I am, in userland, playing with GUI stuff, drag and drop! Python! GTK! Wow! Sounds boring? No, I don’t think so. Discovery time is never boring. Its fun to try for hours to grok some new semantic domain. Even if you, in another life, wrote a DOS GUI system out of reading a marketing insertion on Unix Review in 1988 :-P

So what have I been doing in this strange land? Well, when you work you have to show the numbers, and explain them, and remember what happened when you switched that knob or applied that patch, no?

Damn, alchemy doesn’t requires you remember all that stuff, just taste the new stuff, if you think its not poisonous, that is.

But if you need to remember… There is something cooking for this year OLS, or to the first conference after it if it thinks I’m too bollocksish :-)

And to _show you the code_, full of python newbie mistakes… but hey, I even dared to write python bindings for such supposedly interesting stuff as ethtool and schedutils.

And finally to what uses it: AIT, because short, meaningless names are en vogue. Anyway, try tuna, a tuning application that has a cool name, one that I unfortunately didn’t came up with but fortunately was near the genius that though about it, thanks!

And thanks to whoever did the right thing and brought Evgeniy to kernelplanet, he is da guy from Russia! Get healthy and in crazy coding frenzy mode again!

Syndicated 2008-02-23 00:19:35 from Arnaldo's Ramblings

166 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!