Older blog entries for dangermaus (starting at number 58)

23 Sep 2011 (updated 23 Sep 2011 at 08:49 UTC) »
"Force has no place where there is need of skill." - Herodotus

Kahn's wisdom

Khan Academy published new exercices and this revived my interest in the platform: I solved some more exercices and watched through several videos. While watching "How Earth's tilt causes Seasons" I realised a conceptual mistake in simclimate (released with GPU): I consider the angle for sunlight but not day duration of sunlight... I also watched the series on Milancovitch Precession and Obliquity, Orbit Excentricity wobbling and Perihelion precession. I had the Python introduction, courses about Human Evolution and Earth Formation, and I then decided to start a complete refresh of Linear Algebra, as the course attended at university was taken 13 years ago. And even at work, from time to time, linear algebra is lurking. (Current status: exercices: 176/188, videos: 187/2259)

Journey in Hackerland

The quests are getting definitively more difficult to solve. This month I could solve only eight challenges. Revision 54 of my Brainfuck interpreter written in Superhack language finally conquered the fortified castle. For "Spiral bits" my prototypes aren't good enough yet, so that one complete area of Hackerland remains off range. "Execution Style" and several didactic cipher challenges were solved. Also the Lawsonomy secret is unveiled. But speaking of didactic cipher challenges: brute force works only up to 4 billion keys, which can be tested in roughly a quarter hour of computations assuming printable ASCII characters. If the key space is bigger, this quote is definitely not valid: "If force does not work, you are not using enough force". For "Branches" I painfully worked out a solution, but the number I get is not accepted as solution, meaning that probably I cut off too much code or that I oversimplified the recursion. Who knows. For "Snake Arithmetic", I understood how the nominator is composed, but I still could not telescope the denominator part of the Python equation. For "Anybody Out There" I bought the book "Fourier Transform for pedestrians", but I am still waiting for it as it is out of print and the new edition should come this month. For this alien challenge, I also asked help to an old keen friend of mine who studied physics.

Through my journey, I discovered plenty of tools worth of mention: IntelliJ IDEA, Netbeans and ImageMagick. My favourite pseudorandom generator ISAAC has a companion in form of the RadioGatun hash function.
(Current status: rank 212, quests 165/277)


deltasql 1.4.1

deltasql 1.4.1 is a maintenance release. I discovered a stability problem in deltaclient (for which a dedicated patch of 1.4.0 was done), and tried to minimize the impact of undefined index errors when using strict error_reporting in php.ini. Some unused columns were dropped and the synchronization INSERT statement was simplified. This release really qualifies for the maintenance tag.

My next goal is to try to develop a plugin for the Squirrel SQL client, which is phantastic client which is able to connect to multiple database types (as it is based among other drivers on JDBC ones, a technology I am familiar with). Squirrel SQL is useful at work as well, as there I do not have administrator rights, and I am therefore prevented to install the full Oracle driver.

Linux Magazines

I am a fan of Linux Magazine. I read many articles there, including one presenting System Dynamics tools used in simulations and one about the Hercules Mainframe emulator. And all the cloud computing stuff is more than a hype. Virtualbox is definitely running on my computers. Plenty of images for Virtualbox are here.

Cables

I downloaded Wikileaks cables from Cryptome. For the moment, I decided to split them with split -b 4096k cables.csv cableparts and to search through them with fgrep -i tanzania cableparts* > tanzania.txt. Probably not the best approach, though.
deltasql with new Freepascal/Lazarus client

I am quite proud of the new deltasql release 1.4.0, as it features a client completely written in Freepascal/Lazarus. It contains some of the libraries written for the GPU II platform, and some more stuff I will use to write an applaunch plugin for GPU II. I will test this plugin together with the anagramas plugin, developed as technical fallback from code written for hacker.org.

Echolink on Android

I tested the Echolink app on the Android phone of virus (HTC Legend). I connected to node HB3YGE-L two kilometers from my house. When I spoke the QSO handshake sequence,
it came out of the Yaesu FT60 with a delay of approximately two seconds. It worked also the other way round, speaking into the Yaesu FT60 echoed on the Android phone. I then connected to a node in New Zealand and heard ham radios talking, unfortunately I could not understand their English.

There is something which I miss on the Echolink app: I'd like to have the node number information along with the node name, so that I can connect to the nodes with the Yaesu FT60 by using the Echolink DTMF commands. I list them here as a reminder, the text is from the previous link.


Info Request: By pressing "*" will ask the Node to play an ID message.
Connect: Enter the Node ID number of the station you want to connect to.
Random Node: By pressing "00" the system will connect you to any available Node.
Random Link: By pressing "01" the system will connect you to any available Link or Repeater.
Disconnect: Press "#" to disconnect.


Virtual Academies

Life is definitely worth living :-), in the sense that the same named challenge on Hacker.org finally got solved, after destroying our best equipment. "Countdown Calc", "Patience" and "Blackbox" were interesting as they spaced on the decompiling domain. "Number Theory" and "Primal Pi" brushed up my aging C++ knowledge. "Filtration Residue", "Blood, Toil, Tear and Sweat" and "Dejà vu" taught us a lot. "Right Key Left", "My Chemical Romance", "Hereditary" and "Yoga Message" were pure fun. New tools discovered are Boomerang Decompiler, Hercules Emulator, Scilab and the NTL library. The challenge on which I hope to learn most is "Anybody Out There", although my mathematical skills are quite poor when it comes to signal analysis.

Problem is now that there are two castles, one protected by "Superbrainfuck", the other protected by "Spiral Bits". We have prototypes to solve the two challenges, but they still do not work :-(. And no way to execute "Execution Style", nor "One Function". Is this the end of our journey on hackerland?

For the record, our current ranking on Hacker.org is 239 with 157 challenges solved. Also Khan Academy published some more exercises (137/149)... Virtual Academies are the best escape path for Alice in Dilbertland!
The hacker explores the intersection of art and science in an insatiable quest to understand and shape the world around him.
We guide you on this journey.
From www.hacker.org

Yesterday evening we achieved the 128 Challenges solved mark :-) Best challenges in this journey from the last Blog entry were 'Maeda Path', 'Tic Tac Blah', 'Secure Room', 'Broken keys', 'Immortal' and 'Train Wreck'.

The challenge which keeps resisting us is 'A life worth living'. The spartacusii server burnt while computing the Game of Life for this challenge. Or maybe it died due too a lightning tempest which occurred two days ago. Or it did too much BOINC computations. Also the router is playing strange: I get a red light on the Internet... It was a second hand computer anyway. Basically, to solve more quests, we are risking the best of our gears. Now, I will need to reenable the pico server which I did not operate to spare current (and money). I also have another old server ready for operations.

We are now ranked 516 on the hacker.org site and almost in the top 500. This evening, me and virus will go out for dinner to party this incredible achievement in our life :-)
of mice and men

we just solved the Cavern challenge on the hacker academy. our bash warrior fought and died several times deep down into the dungeon, but a final modification into his mind brought him in about 16 hours down to the diabolic beast at the bottom. our warrior slaught the beast in two moves :-) we did not think someone could create such a deep dungeon!
dungeon's challenges are our entry to the more complicated challenges that require games to be solved by automation.

digging deeper

our "challenges solved" counter scores 102, but now the next challenges require us to invest much more time. we solved the first 90 challenges in two weeks, and then it took one week to solve another ten. this week we solved only one or two... also, we need to dig deeper into our toolbox. in particular, we start performing statistical analysis on ciphertexts (like done on one of my previous jobs)... i also printed out the Python code of the HackVM to learn its inners. and we need to decompose cellular automata into its components to know the number of cells at the 10th billion generation...

a life worth living

the Conway's game of life challenges are interesting, too. and make me feel younger, as my first Life implementation in BASIC for C64 was around 1989, after reading some old articles in Scientific American. i still remember the flash experience i had; my life simulation was not behaving as expected: only when i introduced a copy of the lifeboard for the previous generation, things started working out :-). with my father, we did another implementation in Turbo Pascal for Windows 3.1 back in 1992. the third implementation i did was just some days ago in Freepascal to solve the 'life skills' challenge. but in all my implementations were on bounded universes.

with please, i then noticed the Golly project, a simulator for Cellular Automata including Life. Golly has an unbounded universe, still it is very fast when the lifeboard size is small and can adapt lifeboard size if cells grow out of the box. and it is amazing feeling, as there are plenty of patterns in Golly and even a complete Turing machine working on the Life universe. it is amazing because i remember reading in Scientific American about it and how it theoretically should be possible to construct it in Life, as some NAND patterns were already working when simulated on a big supercomputer. 20 years later, i can run a complete Turing machine on the Life universe on my old laptop :-D.

in Scientific American, i read also about Life in 3 dimensions, and i wonder if there were progress on this as well. but probably not, as we 3D beings we think best in 2D. or at least me...

thinking different

all this time spent on the hacker academy is producing some chemical changes in my mind: i start looking at the detail, giving importance to an end of line or looking at ranges were a function is presumed to work. errors get more important than all the rest! and evidence just becomes obvious background noise... too much evidence can become even a misleading trail.
numbers previously thought big get small like a quarter hour or like four billion keys XORed on a ciphertext. numbers never considered before (because greater than the universe's atom count) are solution to challenges...
writing use-once code and combining tools and operating systems... burning CPU time, network bandwidth and hard disks... in short, hacker.org is wonderful!
fall to the dark side

as Khan stopped feeding me with exercises, I fell into the dark side of the power, and my wife virus too. to date, we solved together 92 quests. Some highlights until now were: we disassembled Java to retrieve passphrases, compiled ADA, uncompressed strange files and brought the JVM to throw exceptions were nobody could think of it. we wrote crawlers to get answer to the One Minute Man oracle (two lines in bash cronjobbed) and wrote a Conway's game of life in Freepascal to compute the maximum number of cells for a particular cell pattern. we heard pictures and morse code, each time extracting the signal out of an amazing amount of noise. we digged into Pi until we found the patterns we wanted. virus solved, among others, the CipherQuest D exercise, which is a substitute cipher with errors and at least 15 meaningless characters. exercices vary in difficulty: we found messages in newspapers as the Simpsons did and we took the natural logarithm of the 150 000 000 000 Fibonacci number... but still, we face another 178 challenges
and we do not know if we are up to it... we feel like a newbie climber in front of the Everest...

deltasql 1.3.7

at least, the fall to the dark side produced something powerful in form of a new release of deltasql: it features salted MD5 hashed passwords with a password reset mechanism. Sure enough for most people around... but not for them.

I learned bookkeeping on Khan Academy with the two types of accounting: accrual basis and cash accounting. Assets, liabilities and equity words entered my head, currently busy with some Khan brain teasers.

And I managed to solve all currently published exercises on Khan Academy (129/129) :-) Kudos to my TI-89 which helped me out so many times! However, the unknown and legendary Black Hole badges aren't yet in my journeyer backpack... And to earn them, it is currently impossible because... the answer is ... hidden in Kahn's source code. Check badges/point_badges.py and exercise_completion_count_badges.py to unveil the secret...

Fabian Bellard's scientific web calculator is very powerful! I discovered it today while reading an advogato entry on him!

My academic career was a failure, but fortunately my khanacademic career is skyroketing :-). These days, I watched videos on Thermodynamics (in Chemistry section). I had a deep look at the Carnot Cycle, at adjabatic and isothermic processes and at the entropy definition, and the Maxwell's demon was lurking all the time.

I watched videos in history from French revolution, through Napoleon's rise and until his fall, and I felt sympathetic with people in the Haitian revolution and was scared by the dishonest debt they had to pay until 1947! In my previous history courses I had a more theoretical approach to the French revolution: we discussed in deep all Illuministic theories, but I completeley missed the stuff going on in Europe in 1789-1815. In the end, the Swiss constitution was enforced by Napoleon... I wonder how I could visit London and Paris without knowing what I do now... I practiced with exercises on the Academy site, too. To date, I watched 124 videos and solved 104 exercices. My goal is to earn at least one of the two unknown Black Hole badges!

Dangertube

As my one and one half year old daughter is learning by imitation, I decided to do the same! I equipped myself with an Aiptek 10000U (for about 130 Swiss francs) and bought Screenvideorecorder for another 40 bucks. I then created a tutorial video on how to compute the gamma factor of special relativity. English, audio and overall handling are experimental! (what else would you expect from dangermouse? ;-)

I also created 7 videos on how deltasql works and put an old introduction to the GPU project on Youtube.

Joomla

I am very happy on how Joomla manages the site of our local boy scout group. People keep sending me articles and in a matter of minutes I get them online.

Excel SUMIFS function

It adds the cells in a range (range: Two or more cells on a sheet. The cells in a range can be adjacent or nonadjacent.) that meet multiple criteria. For example, if you want to sum the numbers in the range A1:A20 only if the corresponding numbers in B1:B20 are greater than zero (0) and the corresponding numbers in C1:C20 are less than 10, you can use the following formula:


=SUMIFS(A1:A20, B1:B20, ">0", C1:C20, "<10")

I recently saw SUMIFS used as an extended VLOOKUP with more than one cell which needed to match.

There is an interplanetary internet, and there may exist an internet between civilizations running at tachyonic speed. This time I ventured into

Another Internet

It starts with configuring the DNS servers with ones published at the OpenNIC project. You need to configure both primary and secondary DNS. Configuring only the primary DNS does not work, as the algorithm always chooses the DNS entry with lower network latency. After configuration, you still are able to surf on the usual Internet, but you also opened a little door to domains like .geek, .bbs and .telnet.

On the wiki of OpenNiC there are very few links, and in the beginning I was disappointed of such few links. Then I learnt of the specular search engine http://grep.geek. By searching for keywords like Blog or Chat you get to view some of the most accurate blogs and interesting chatrooms, especially for people who like to code.

Pioneer Anomaly solved with heat model

Pioneer space probes are the farthest human made objects from the Earth. In a previous post I explained how to connect to the JPL server via telnet and get their current distance from Earth.

Pioneer space probes feel an additional force towards the sun in the order of magnitude of about a_pioneer ~= H*c, which is also the one predicted by Modified Newton Dynamics theories to empirically explain the flat shape of radial velocity curves observed in galaxies. Currently, the the best theories to explain the galactic radial velocity anomaly are ones that look at the Universe at as inhomogeneous sponge. I hope the dark matters theories will be then finally wiped out, as to me they look as a revival of the theories on ether used before Einstein to explain the Michelson-Morley experiment. Also to have so much dark and heavy things passing through my body without interfering with it does not make me feel confortable at all. Springtime is time to give a compete cleanup to the Universe, it's needed once in a while :-)

The GPU project was looking for a solution of the Pioneer Anomaly. It ported source code from the Orsa Project made by Pasquale Tricarico from C to Delphi and experimented a bit with special physics without further success. It was not possible to replicate the linear shift of the signal received from the Jet Propulsion Laboratory dued to the pioneer anomalous acceleration towards the sun.

There is now an article here, that explains how the anomaly can be explained with 1970s computer graphics (basically a more advanced heat model as the one presented in the original Anderson et al. paper about the anomaly). The paper with the heat model explanation is on Arxiv and awaits confirmation from other scientific teams.

How seismic waves tell us about the inner Earth core

The GPU project was also experimenting with seismic sensors. Nanobit built one (picture) and I also built one with a 3D accelerometer on top of a coil. This series of 6 videos on seismic waves by Khan Academy explain very well the types of seismic waves, why Secondary waves travel only into solids (and in pudding :-), the curved path of refracted waves due to the density differences, the S-Wave shadow zone beginning from 105 degrees, and the P-Wave shadow zones. The seismic waves indirectly tell us how the inner Earth core is made, even if none of us digged that deep. Humankind only scratched the surface of Earth!

Learning the periodic table of the elements

The Fukushima accident made me courious of chemistry, and I sorted out an old periodic table of the chemical elements I was using in school. It has a list of the radioctive elements and their half decay time. I then realized that I did not understand anymore why the table was organized in this way. To be honest, I never understood it! I then took time to watch the introductory course on Chemistry by Khan Academy. The first ten videos explained me much more in detail how the periodic table works. In particular, I can do electronic configurations which was never taught in my chemistry course, and I understand a bit why metals are placed in the 'd' configuration. Metals and salts I already knew from before. And the f configuration at the bottom is for the heavier elements, which in principle should be moved between earth alcali and transitive metals, for the electronic configuration process.

IRC chat primer

I use on Gentoo weechat-curses as IRC chat terminal (emerge -av weechat-curses to install it).

  • Connecting to an IRC server: /connect irc.oftc.net
  • Connecting to an IRC server without certificate: /connect pari.para.net:9999 -ssl -ssl_verify=false -password=****
  • List available chat channels: /msg ChatServ list *
  • Join a channel: /join Debian
  • Leave a channel: /leave
  • Change nickname: /nick dangermouse
  • Register nickname: /msg NickServ register yourpasswd your@email.com then follow instructions on email you receive on how to authenticate
  • Bind nickname once registered: /msg NickServ identify yourpasswd
  • Exit weechat-curses: /leave or /exit
28 Mar 2011 (updated 29 Mar 2011 at 13:06 UTC) »

Some topics in maths I did not knew or refreshed with Khan Academy are: Laplace Transforms, Convolution Theorem, Dirac Delta function, Taylor Theorem, Implicit differentation, insights in Chain rule, Conic Sections (foci of Ellipse, Hyperbola, and Parabola with Directrix), partial fraction expansion, l'Hôpital rule, surface integrals... I also learned a bit finance on forward contracts, futures and the margin mechanism I can use on the job. Sal, you are great!

Useful git hints

These git hints are taken from the Linux Magazine, article "Git in der Praxis".

  • Initial configuration:
    
    git config --global user.name Danger Mouse
    git config --global user.email dangermaus@users.sf.net
    
  • Coloured output:
    
    git config --global color.diff auto
    git config --global color.status auto
    git config --global color.branch auto
    
  • New repository:
    
    cd myproject
    git init
    git add .
    git commit
    
  • Ignoring files: put your entries in a file .gitignore
  • Clone Linus kernel's repository:
    git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git linux-2.6
  • Push changes to repository:
    
    git commit -a -m "these are the changes"
    git push
    
  • Retrieve changes from repository with git pull
  • List existing branches: git branch
  • List existing tags: git tag
  • Create new branch:
    
    git branch Refactor
    #switch to branch
    git checkout Refactor
    #back to master
    git checkout master
    
  • Create new branch and switch into branch:
    
    git checkout -b Refactor
    
  • Merge from branch Refactor into master:
    
    git checkout master
    git merge Refactor
    
  • Create a tag:
    
    git tag -a "Tag_1.3.6" -m "new tag at version 1.3.6"
    
  • Retrieve uncommitted changes with git status
  • List latest commits with git log
  • Create a patch:
    
    # on branch Refactor
    git commit
    git checkout master
    git pull
    git checkout Refactor
    git rebase master
    # repair conflicts
    git commit
    git format-patch master --stdout > mybugfix-patch.diff
    

I am also trying to understand a bit of this book:

Indra's pearls

The book is centered around Möbius transformations and Schottky circles. Möbius transformations


f(z) := (az+b)/(cz+d)
are visualized as compositons of affine projections (az+b) and of projections on the Riemann's sphere 1/(cz+d) of the complex plane. They are at the ground of Beltrami disk (on page 379) and represent a non-euclidean geometry for which the parallel postulate is not valid. Schottky circles are generated by two Möbius maps which project one circle a into a circle A and another circle b into a circle B. With a breadth-first or depth first algorithm all set of reduced words (e.g AbBb reduces to Ab) are generated, and interesting pictures come out. Like the glowing gasket by Apollonius, invented in 220 BC, which is also the best way to pack as many tangent circles as possible (apollonian packing). The most famous fractal (Mandelbrot's apple) shortly appears at page 291.

Debian Squeeze

When installing Debian Squeeze, the partition with Windows XP was not recognized. Normally I adjust /boot/grub/menu.lst but this time the file was missing.


os-prober
grub-update
did it....

As "scientist of information", you might find these tricks with M$ tools useful:

Creating SQL statements out of an Excel table with data
An insert:
=CONCATENATE("INSERT INTO tbdata(id, name, address) VALUES(";A1;"'";A2;"'";"'";A3;"'";");");
Or an update:
=CONCATENATE("UPDATE tbdata SET name='";A1;"');");

Lookup table in Excel


=VLOOKUP(C3;$L2:$O$25;3;FALSE);
This formula looks up for the value in C3 in the table $L$2:$O$25 and returns values in the 3 column where C3 was found. It's a vertical lookup.

Importing Excel tables into access With File->Import->LInk External Table it is possible to import Excel tables into Access and then with SQL queries to extract data from them. However, I learnt that there is a limitation in the complexity of queries you can launch. I always switch from 'Design view' into 'SQL view' when creating queries.

For the interoperability of the Deltasql project, I setup a MS SQL server 2008 instance...

Installing SQL server

Downloading and installing it is easy, but at the end you need to assign a port to it, else the thing won't work. Might sound simple, but I assumed this step was automated by the install procedure. At best, you install it without touching defaults and as last step tweak it to open say port 2345 on TCP/IP protocol. Only after this step, you'll be able to connect with SQL Server Management studio client.

I still think that Oracle and Postgres are the high end of the database offer. However, I see once again how Microsoft is eating more and more of the lower segment. I know many little companies (10-30 employees) which now use some accounting solution based on SQL server. Still, sometimes at work I am puzzled and wonder if the transaction integrity of SQL server is always guaranteed, when under heavy load...

Tips to install Gentoo

I used an old CD to reinstall the latest Gentoo on andromeda, but failed due to some kernel issue with udev after several tries. I then decide to install Debian Squeeze, as I did not want to recompile everything on this old toaster (1GHz, 512 RAM) :-) As side effect I learned that it is always best to use the latest image of Gentoo install CD to get it running 8-)

Another update on Fukushima nuclear accident by Barry Brook is here. Also look at the site made by the students of the Department of Nuclear Science and Engineering at MIT.

49 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!