Older blog entries for ensonic (starting at number 102)

I will give a talk about "Writing audio applications using gstreamer" at the Linux Audio Conference 2010. Naturally quite some time went for writing the article. Now that it has been accepted, I need to start working on the slides and demos.

In buzztard I've tried to make the life of people that use jack or pulse a bit easier. The problem here is that the player is only visible in jack/pulse when it is actually playing. I have now some code that keeps the sink alive (in GST_STATE_READY). This allows e.g. changing the volume in pulse or routing the output in QJackCtrl even if the app is not playing. Unfortunately it only works after the sink was in playing once. I need to confirm my theory of what wrong still.

I recently brought a netbook (MSI L1300) and thus wanted to try buzztard on this machine. I have installed moblin 2.1 on it to play with moblin. As usual, installing own software on a new linux distro/machine always brings up something. After a few rounds of configure maintenance and dependency trimming I am set and having it running. Performance seems to be okay even for complex songs. Yay!

Besides that, I fixed a few more bugs and wrote more tests.

gtk-doc, xslt and profiling

As part of the endless journey of getting gtk-doc to spit out the docs faster I managed to get xsltproc from libxslt to output full callgraphs. The output is compatibe with oprofile and thus the same tools to rendering can be used (e.g. gprof2dot). Bit sadly me profiles hit a python assertion in the above code which I have to disable to run it, buug filed but dormant since 1 month. Same for the xsltproc patch, the patch and an example image is here but no reviews so far.

Now the way to make gtk-doc faster would be to make the xslt faster and as I can see from the profile, this could be done, by preprocessing the xsl with some variables set. Right now I have not too high hopes on this as libxslt seems quite dormant and this is not so straight forward :/

As if February isn't short enough already, a nasty stomach infection send me to the couch (and bathroom) for a couple of days.

Still managed to do a couple of things. Some refactoring in the editor allowed me to harmonize context menus, e.g. one can open machine windows also from pattern and sequence view.

GStreamer has support for accelerated and decelerated playback, forward and backward. I started to overhaul the seek handling a bit and added UI for fast- forward and fast-backward. Fast-forward is working with some limitations, but reverse playback needs some fixes in plugins still. As a side effect one can now scrub with the mouse over the timeline.

I also wrote a standalone test app for my level meter. I was still not convinced that is was always drawing correctly. Indeed there was a bug in the resize behaviour, that now is fixed too.

As gtk+3 comes closer I did several rounds of adapting the the new style to use accessor functions everywhere.

44 files changed, 1087 insertions(+), 317 deletions(-)

I couldn't really concentrate one a single task this months. Thus I ended up with lots of small changes here and there. First obvious change is that buzztard got an over-engineered tip-of-the-day dialog. I takes extra care to present tips in random order, without repeating tips before you have seen them all. I also played a bit with parasite - the cool gtk ui introspector and noticed that my dialogs are inconsistently layouted. Well, fixed now.

Besides I did lots of small cleanups and refactorings. GStreamer pipelines load faster as I now link pads directly (not elements). This also gives me better diagnostics if it goes wrong. I could get rid of some dead code, removed a lot of double checking and generally lowered the amount of gobject runtime type checks. I also turned some more objects into proper gobject singletons. This way I don't need to pass them around anymore.

Finally I wrote some test code in preparation of adding device selection to the audio settings (like selecting the alsa device). Need to make a plan of how to store the selection for multiple sinks.

173 files changed, 6014 insertions(+), 5939 deletions(-)

I worked most of the December time on gtk-doc. After more that a year I released 1.12 and 1.13 on the same day:). Most notable features are probably that now one can have syntax highlighted and xreffed examples and the generated indexes have a prefix detection. That means the index has proper a,b,c sections and not everything under 'g' for for gnome libraries. The best thing - you don't need to do anything, just update your gtk-doc version. Now I hope that someone kindly installs 'highlight' (or 'gnu-highlight') on the server that runs library.gnome.org (or do we need them on build.gnome.org?) so that we get the highlighted docs there too.

Now if you want to do something, have a look at these two commits and do the same in your package. It helps maintenance and it builds faster. If you want even more speedups go for gtk-doc from git head. I managed to tweak the xslt to save a third of build time. I mostly replaced some templates to avoid i18n which the api-docs don't use anyway, but also cached some nodesets.

buzztard

On buzztard I also spent a little time, improving the docs and pushing the test coverage further. I will attack the bullets on the 0.6 roadmap next. Happy new year and stay tuned.

I went over pending bugs and fixed a couple more parser errors. The testsuite is definitely helping. The I also had a stab on one feature request - print warnings for failed xrefs. This would be helpful to fix typos in the doc comments that were intended as links, but don't become ones. It was easy to add the warnings, but unfortunately it produced lots of false positives. This is mainly because gtk-doc also creates implicit links. Almost one week later I had most of them under control and now the amount of false positives is quite low. I pushed my ohloh score by fixing the found issues already in gstreamer core/base and in buzztard. You can do the same in other libs. If noone complains any further I intend to release gtk-doc 1.12 this year still :)

buzztard

I made a nice change to the pipeline graph builder - It now uses a fixed internal format (float 32bit). This allows us to avoid plugging audioconvert elements in wires and machines. Now we plug them only if the machine cannot support the internal format. This gives 7 -> 1 sec speedup for load & play of e.g. Aenathron.bmx.

In general also plain to wav rendering is now quite fast. On my AMD64 @ 1.8GHz the 11:45 min. song renders in 45 sec. and on my Intel Core 2 Duo @ 2.0GHz it takes 28 sec..

The rest of the month I spent a lot of time on project maintenance, improving the api-docs and writing more test. Regarding the latter, I got the test coverage improved a lot. Writing automated tests for some areas is quite tough though.

Then I also made a small change in the UI. All context menus can now be invoked via a popup menu button in the toolbar above (machine, pattern and sequence view), this makes it more usable on touchscreen devices.

I started the month with upstream bugs work. First I looked into #581873, made a fix and two days later realized that my git checkout was on a stale branch and its already fixed in gtk+-2.18. Next I looked at the mime matching issue that is already plaguing me for a while (#541236). I also have a patch for that and hope that I get green light for it or a review how it should be fixed instead.

buzztard

In buzztard I implemented copy and paste in sequence. I also found our what I did wrong in my clipboard handling code (see my previous post). Dunno what wrong with the old code, but this change made it work:

GtkClipboard *cb
-cb=gtk_clipboard_get_for_display(gdk_display_get_default(),GDK_SELECTION_CLIPBOARD);
+cb=gtk_widget_get_clipboard(widget,GDK_SELECTION_CLIPBOARD);

Next I finally got rid of the gnomevfs hard dependency. It is only needed if you have a quite old gtk to get help working. While testing I noticed that some translations disappear during run time. I made some i18n handling fixes regarding to initialisation and libraries.

I am probably the last one to figure that one has to draw with the 0.5 px offset in cairo to get non blurry gfx. After Matthias post I looked at my vu-meters and voila, I was doing it wrong. Fixes are in svn together with some small optimizations.

After a release it is always not so easy for me to pick what I will work on next. I decided to look into copy'n'paste as this is crippling usability right now. This sounds like a trivial thing to implement, but its not. Its partially difficult because the gtk documentation on that matter leaves a lot to answer. With the help of the sources I got it implemented for patterns. One thing that worries me right now is that I use GDK_SELECTION_SECONDARY instead of GDK_SELECTION_CLIPBOARD (which seems to be the recommended one). In my case I have a custom widget to edit structured data. I serialize the data in a textual form, but use an own mime-type for copying, as its kind of pointless to paste it into e.g. your editor. The mime type thing works. To debug it, I can also provide the data as text and then I can paste it in some text editors. What puzzles me is that if GDK_SELECTION_CLIPBOARD is used, gtk+ seemingly randomly post a GDK_SELECTION_CLEAR in gtk_widget_event_internal() and that clear my copy before I have a chance to paste the data. It has something to do with a "selection-clear-event", but again that's only partially documented and the other related events are not documented at all.

Besides a few bug fixes, I also did more optimisations on song-loading. There are still a few candidates for more optimizations, but some of them require quite substantial refactorings. Some of the improvements I already did were on the gstreamer side and they are now in the official tree (some already in 0.10.25 release and some in the next one).

The buzztard team has released version 0.5.0 "crown of thorns" of its buzz-alike music composer. All modules got extensive improvements over the last release from almost a year ago. Give it a try, join hacking and report bugs.

[project-page] [screenshots] [downloads]

After all the thinking last month regarding the saves and handling external files, I finally found time at GUADEC to finish the implementation. I can do a quick copy from the old file to the new one on save and create wav files on the fly when e.g.. saving a buzz song in buzztards own format.

During the month I did a lot of real world testing. I recorded children songs as I had scores from my sons kindergarden :), but also just played around. I had the app running under gdb with G_DEBUG="fatal_warnings". Obviously I did not got far with making music on the first evenings. A couple of days later we have a lot bugs less.

My talk at GCDS about "multimedia in your pocket" went fine. I showed short demos of running pitivi, jokosher and buzztard on a omap3 device using maemo 5. I hope the slides will be available soon. For the demo I did some more performance tuning. Using GTK_DEBUG=updates was very helpful to improve the UI to do less redraws.

I also spend some more time on the user manual again. Much less FIXME paragraphs left :)

gtk-doc

Also did some hacking on gtk-doc. I can haz pdfs now! Try the git version and report please. I should make a release. I also hacked a bit on the migratetmpl script. It's now trying to patch tmpl docs into sources. I uses ctags to locate the positions. It needs more work still, but I see that as the only way to get e.g. gtk+ converted. If you want to try it talk to me on #gtkdoc, don't run it on a dirty work tree!

May passed quickly. I was testing a lot and as usual this brings bugs to day- light. I noticed one particular problem, that I still have to solve. Buzztard songs are actually zip file containing the songs.XML file and references files like samples. When the song is save the externals are copied from its original location into the zip. First there is a chance that the file becomes unavailable between loading it and saving the song (e.g. if the files where loaded some a sample CD) and then of course if those songs are given to someone then the zip is all there is. I see two ways to handle it - a) if the file is not available, save the raw-pcm as a wave-file, b) when loading new files samples, copy these somewhere and keep the file open and when loading a song, copy this somewhere and keep it open (somewhere = local file system). One thing I already have implemented is save saving, that is rename the old file, save and in the case of failure roll back.

There were also a couple of bugfixes in the UI. I had a long session to fix object destruction and detaching related signal handlers - too bad that the limitations in g_signal_connect_object() still render it pretty useless.

In the beginning of the month I continued with the work on the buzzmachines module. I rewrote one effect from scratch and its pretty compatible. I also added more presets and demo-songs.

While testing more buzz songs I realized two long standing issues. On the buzz song loader I got one detail wrong from the reverse-engineering - I was loading polyphonic pattern with voices and time swapped. That was easy to fix and make many songs sound proper. I made a few other compatibility fixes and improvements on the song load too. The other problem was actually much more work - the buzzmachine emulation had a big flaw. When I instantiated a plugin I was loading the dll/so once for the class and created the machine instance at the same time. The emulation API only had a init(). That means for multiple instances I was "reusing" the class wide instance. It worked surprisingly well :) Now I did a big refactoring there. In class init the dll/so is loaded and for each gobject instance I create a instance of the wrapped machine.

Finally I got around implementing one feature in the GStreamer side. There are a couple of wrapper plugins in gstreamer that act as a bridge between different plugin APIs. The way this was done had quite a drawback. Whenever the plugin was loaded, it was re-registering all the element types. In order to do that it needed plugin specific metadata. So it was loading each of the bridged plugins and querying the features. It needed two patches to get that changed. Now gstreamer does not ref each element class of a loaded plugin during loading and the registry can cache plugin specific metadata. What does it mean practically? A typical buzz song now loads in a quarter of the time.