31 Dec 2012 dangermaus   » (Journeyer)

"You just keep pushing. You just keep pushing. I made every possible mistake that could be made. But I just kept pushing!" René Descartes 1596-1650

This is a report for the last months spent at the Open Source front and marked by several failures including the loss of the last server for the Global Processing Unit project. Hidden around the corner there was some success, too.


deltasql was essentially improved in two ways: copy&paste support via browser was added using the ZeroClipboard library. It is not necessary anymore to manually select the SQL script before copying it. With another library called OpenFlashChart (by John Glazebrook), we added graphs and pie charts to show how the complexity of a deltasql server increases over time. The number of scripts and of synchronizations are shown in a line chart, the size of modules and projects in pie charts. The latter improvement was picked up by Linux Magazine and deltasql 1.5.0 did a short headline also in the printed edition of the magazine.

Global Processing Unit

The change which gave me most pleasure was adding variable daylength duration depending on the day of the year and latitude to the Simclimate frontend. The effect is given by the inclination of the Earth rotation axis in respect to the ecliptic plane (the plane which contains both Earth and Sun). Earth inclination is called declination, too. Declination gives the typical bell shape to the sunlight projected onto the 360x180 degree matrix of the world's surface. I documented for the first time Simclimate here.

I finally documented the Orbit Reconstrunction and Analysis extension as well. The discovery of the likely cause for the Pioneer Anomaly puts an end to the development of this extension. The cause of the tiny anomalous acceleration toward the Sun experienced by the Pioneer space probes is probably due to the radioactive heat of the power source reflected by the parabolic antenna dish of the space probe, and not due to new Physics. For the rest, the domain www.gpu-grid.net is down, GPU clients do not autoupdate nor connect automatically to each other anymore.

Adventures on hacker.org

Our soliders fought epic battles totalling 229 challenges solved. Solving some of them gave me a kind of Zen flash. Some ROT13 hints on my favourite challenges follow:

Sbe 'Nalobql Bhg Gurer', V erprvirq na vzcbegnag gjrrg sebz rsr. Qvwxfgen urycrq zr jvgu 'Pnyphyngbe'. Urephyrf vf qrsvavgryl abg gur evtug rzhyngbe sbe 'Znvasenzr' orpnhfr vg vf gbb qvssvphyg gb frg hc. N oyhrvfu tybj ba lbhe snpr jvyy yrnq gb gur fbyhgvba bs 'Cbxr Zr'. Fbyivat 'Vafpehgnoyr' erdhverq zr gb tb onpx gb gur svefg punyyratr bs gur frevrf gb ercrng gur onfvpf. Va 'Wrhk qh Fbeg' V vzcyrzragrq gur gurbergvpnyyl snfgrfg nytbevguz, nygubhtu vg ghearq bhg gb or pbzcyrgryl birexvyy. Sbe 'Pnir Znfgre', qba'g ehfu ohg nqwhfg lbhe fcrrq. Naq fghqlvat Znqber'f fvgr urycrq zr jvgu 'Dhvar'.

I also wrote a Bash tool named discochal to compare challenge solvers. You can compare any two players and see which challenges where solved only by one player. Sometimes, this information is useful to infer additional information on difficult challenges (if you study a bit strengths and weaknesses of other players). The tool can be checked out from the GPU git repository following the instructions on the forum.

One of the challenges brought me to study the maths behind stereograms. The article "Bewegte Autostereogramme" in c't Programmieren 3/2012 has an interesting explanation on how to create stereograms using the Kinect sensor!


Tapeworm is the fourth puzzle on hacker.org I am attacking (Runaway Robot, Crossflip and OneOfus being the first three), and here I miserably bit the dust. My automatic solver uses simple Depth First Search using hashes to detect already known boards and computing weighted shortest paths as simple heuristics, and barely solves level 40 because this puzzle is crazy deep. My beloved Virus managed to manually solve levels up to level 104. I had at least one good idea: to debug the code I inject into a html level template of Tapeworm my current board as javascript variable, and then look at the result in a browser as it would be a level on hacker.org. This probably works with other puzzles as well, but i never tried it before.

Firewall Analysis

I kindly received a configuration file of a midsize company to analyze. I discovered some interesting things, especially looking at the routing table. Many invalid IP subranges are all rerouted to the same server. Maybe a Nagios server (or one of its sensors) which is able to detect local port scans? On the DMZ there is a suspect IP address (with a SMTP rerouting for email notification), which might be a honeypot? An external security company reserved 20 IPs, specifying them one by one and called it HyperVM. Probably, they are monitoring the network from outside with 20 virtual machines! Amazing :-)

The LISP way

The challenge 'Cons Car' first, the book '7 languages in 7 weeks' later with Clojure, brought me to LISP and the LISP way of thinking. Although I am still looking for my own project to be developed in LISP, I experimented with the online book Practical Common LISP. I also read through the series on LISP by Douglas Hofstadter on old Scientific American issues. Looking at ways to replace the computational engine of the GPU project, I developed an interest in Computer Algebra Systems and looked at both Maxima and REDUCE which have a core in LISP. LISP was intentionally developed with the manipulation of algebraic equations in mind. One of the first conferences on LISP of its creator John McCarthy was about symbolic differentiation. The file sin.lisp of Maxima source package implements symbolic integration and references in the source code the original typewritten PhD Thesis of Joel Moses. LISP was the first language which introduced recursion and garbage collection, ideas later picked up by other programming languages.
The most powerful idea, which was never introduced in other languages, because if they would, they would look like LISP and simply become a LISP dialect, is that data and code are both expressed as lists and this gives to the language a particular plasticity. By using so called LISP macros (C/C++ macros express a different less powerful concept) it is very easy to extend the language; for example, try in any other language to implement a change that causes the programming language itself to be parsed in reverse order. In LISP, simply write

(defmacro backwards (expr) (reverse expr))

Now, you can sum two numbers with (+ 1 1) or more originally with (backwards (1 1 +))

And speaking of numbers: Common LISP natively supports big numbers, complex numbers (!) and conversion between numbers in an unrivalled way!
Another thing discovered by McCarthy is that in LISP, it is very easy to write an interpreter of LISP, no more than thirty lines are required to do it.

Other parallellization ideas

With LISP, I am looking to improve my understanding of (semi) functional languages. Stateless functions are easy to parallelize. One idea that popped up was running the Global Processing Unit on clients which would run MultiLisp and on top of them some Computer Algebra system. A Microsoft research paper, Gordon et al, Uniqueness and Reference Immutability for Safe Parallelism, points into another direction: variable qualifiers gives important hints to the compiler to compile code on parallel architectures. By the way, it is time to check the specifications of your graphics card: your dual core laptop might hide up to 100 CUDA cores which can run fragment shaders or vertex shaders in parallel! The magazine c't Programmieren 3/2012 has lot of source code to leverage this CUDA power!

Online book about Cryptography

The Codebreakers book features a comprehensive and interesting history of Cryptography, it just misses the developments at Bletchley Park because it was published before the documents were declassified. By the way, some more papers on Enigma by Alan Turing were declassified.

Broken Tire Story

When trying to reach the parent's house of Virus, the overloaded car hit a little stone. In the cockpit, I heard a loud bump and four minutes later we were at the house, greeted by the 'pffff' of the broken tire loosing air. I replaced the tire with a reserve one and then dedicated my attention to the sharp little stone down the street in the forest. Using shovel and pickaxe to remove the earth round it, I discovered that it was a big one, like the iceberg is just the tip of a huge block of ice. I thought I will never remove it, but being in holiday and not knowing how to use TNT, and where to buy it, I decided to give it a try. All of a sudden, I was able to move the stone a tiny bit, carefully leveraging with the pickaxe. Then another bit, I filled holes first with sand, then with stones of increasing size. About one hour later, I managed to remove the big stone and learned my daily lesson, namely that things which look impossible might turn manageable if worked out without pressure.

Wasp War and their Revenge

We found two yellowjacket nests hidden on the curtain of our balcony. Armed with a special spray (Gesal Protect), we decided to kill them all. I carefully read instructions on the spray can, but unfortunately missed to read the last sentence.
I dressed myself with jacket, scarf around my neck and gokart helmet to avoid wasp stings. It was a calm evening with no wind.
Then with an amazing jet coming from the spray at high pressure, I killed many wasps (counted the next day in daylight, they were 27). Doing the operation in the evening on the bare light coming from the apartment, I completely missed the tiny drops of insecticide which were in suspension at mid air. I breathed them, unaware that they were existing.
I felt like in Kafka's short story (the Metamorphosis) when I discovered with horror that the last sentence on the instructions was about avoiding to breath the insecticide (kind of obviuos, but you know... ehm)! Through the night, my lungs burned (the Prallethrin which stops air holes of the wasp?). The next day I almost had to stop working, I had dizziness and glows swirling in front of my eyes(the Cyphenothrin burning the yellowjacket brain?). Fortunately, the latter effect lasted only for twenty minutes, and the dizziness disappeared at midday and since then I am back human. The daily lesson thaught me this time that insecticides are really something to leave hands off, to be used with _extreme_ care, and I wonder how role they play in the scaring pollinator population decline.

Casino Adventure

I had the chance to visit a Casino: they gave us first a drink and then an introduction to the Russian Roulette and the Black Jack game. As premise let me mention that I never was lucky at games.
They gave us fifteen coins to test our understanding of Roulette rules. So, I put them all on Black, but the ball finished its trajectory to a Red cell. I lost all coins in one shot! The croupier and someone in our group laughed at me; oh man, someone who is so stupid, why not carefully playing complex combinations on corners! After three rounds with test coins on which I could not play because I already lost everything at the first, they gave us three real coins, each one worth five Swiss Francs.
Again, I put all three coins on colour (I think it was Red, honestly I do not remember) and won doubling them to six. I then put all them on Even and won again. I had now twelve coins and decided it was time to play Black Jack. Believe it or not, simply by playing Black Jack golden rule which was even written on the green table and saying 'Dealer must draw on 16 or less and stand on all 17s', I was able to increase my capital up to 27 coins. Fortunately, my group was now leaving for dinner and therefore I was forced to stop. I went to the bank to exchange the coins. They gave me 120 Swiss Francs but said, they can't turn the first three coins into cash because they are a gift from the Casino anyway. So, I have to play them or keep them as souvenir. I then looked myself around and found an old woman and gave the three coins to her. She looked wise to me and did not look them like a gift of destiny. So many people ruin their life and loose their wealth in Casino games because they do not understand that the randomness of the game is constrained by the rules with the ruler as ultimate winner (the Casino is the ruler).

Sal is my Khan

Peter Collingbridge published a tool to see personal statistics on Khan Academy. Freaks will like the feature to see how many minutes are needed to get the next Listener badge. I bought the book of Sal "The One World Schoolhouse" and found some interesting ideas. The first was kind of obvious for people actively using the academy, and it is about filling holes in education with videos. Then, there is the idea of mixed ages classes where older children can teach younger, and younger can have their superheros. Finally, I did not know that Sal managed to get three MIT degrees in parallel, avoiding lessons and just learning with peers on a bet on whom could do more courses at once!

Two incredible landings

We were all astonished, looking at the landing of Mars Rover Curiosity with sky crane and at the jump of Felix Baumgartner from almost outer space. After Space Shuttle's retirement there is new hope for advancements in space technology! Instead of expensive Soyuz vessels to return from the International Space Station, we could simply step out and jump from there :-) An incredible jump for a man, a huge leap for mankind!

Happy New Year, and many apologies for my English :-)

Latest blog entries     Older blog entries

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!