The motivation for re-writing code in multiple languages

Posted 2 Feb 2003 at 00:54 UTC by Stevey Share This

 This piece was inspired by a recent poll upon Slashdot, asking people to vote upon their favourite programming languages.

 In my time I've written multiple projects, using multiple languages - it's only recently that I've rewritten the same program in several languages.

 This is how, and why it happened...

Introducing Edna

 Once I'd setup my home network (three Linux machines) I wanted a simple solution for streaming them from a central location. After looking around for a while I came across Edna which is a small media server written in Python.

 Edna sufficed my needs for a very long time. Although it appears that all practical development has ceased it serves it's purpose extremely well.

Edna's deficiencies

 After a few months use I found that I was missing several things which would have made the server a lot nicer - Such as the ability to search the archive, and view statistics.

 Unfortunately my Python knowledge is minimal. I hacked around with a for a little while but then decided to recode the server in Perl, I did this because it seemed sensible to move from one scripting language to another.

The Perl Server

 The perl server whas quite honestly a mess. I only used it for a short while before thinking of recoding it in C.

 This was mostly due to me attempting to rewrite the Python code in Perl, something that was a mistake; picking a more appropriate design would probably have resulted in a great server right from the start.

The C server

 As an excuse to play with writing a server in C with threading support I began to recode the server one rainy weekend. Although it wasn't very complete, in 2 days I'd managed to build a nice simple server which did the basic listing and streaming of MP3's. This sufficed my needs for a short while.

 The C code was released to Freshmeat.net on the following Monday.

 I received around 20 emails that week from users who liked it. To my intense suprise I actually started to recieve patches for various things too.

 Like many other developers I'd released software before but this was the first time I'd received patches from multiple users in such a short amount of time. (I put this down to the fact that there aren't many simple streaming servers, and the server has the magic three letters 'MP3' in its name ;).

 Development continued with regular bugfix releases and feature updates.

The Move To C++

 After a few releases it became apparent that there were several problems with the codebase that weren't going to be simple to fix. (Mostly because of complications with the multi-threaded nature of the server)

 The flaws in the codebase were primarily concerned with string handling and memory management. (Two areas where C is traditionaly prone to misuse and abuse).

 I've previously written a lot of code in Java but I didn't want to alienate any users who were without a Java runtime environment. This meant that C++ was a natural choice for the new implementation as it led to a natural reuse of the existing code.

 (Looking back on it maybe I would have been better of going for Java; it has built in support for both threading and networking - who can tell what the server would look like if I'd decided to go with Java instead?)

 Migrating the code to C++ took a couple of days. Really making the code object orientated took slightly longer - but eventually it was done.

 There were a few problems with the first C++ release, but they were fixed fairly quickly and know I made the correct choice in doing the re-write. More Users

 After a while I started getting curious as to how many users I had - so I started running websearches - this is when I discovered it had made it into Gentoo Linux, and the FreeBSD ports collection.

 Many people were beginning to complain that the server was too hard to modify; which I thought was a shame, especially since I'd put such a lot of effort into the plugin system.

 Two other problems were starting to show up:

  • Buggy versions of the standard C++ library causing random crashes
  • The difficulty of integrating database support.

 Database support was something that originally I'd resisted - as it would increase the dependancies of the code. But with an active userbase of people with large collections of music I was coming round to the idea of supporting this.

 Around this time I decided that the idea of using a scripting language with good database interoperability would be a good thing.

 Perl was the obvious candidate for me, Ruby was an interesting contender but a little too new for me.

 Python I'd already moved away from, and it's still a language I don't know.

The Future?

 In the past I had considered the version 1 release to be feature complete; it did everything I'd ever wanted it to do. Streaming, searching, stability, and extensibility were all there.

 But now with the added power of Perl <sup>tm</sup> interesting new possibilities are presenting themselfs - the ability to rate songs, and choose to listen only to songs that have been rated positively, the ability for the server to track your music and suggest similar things, both these ideas and more will be possible in the future.

 After that .. who knows?

 I guess it will come down to people making interesting suggestions.


Learn, posted 2 Feb 2003 at 02:35 UTC by ncm » (Master)

Not knowing Python is a very poor reason not to use it in a project. You can learn enough to do useful work in a day or two, and by the time you finish, you have a new and valuable skill. If you are doing network coding, be sure to look into the Twisted framework.

But your remark about an imagined "added power of Perl" makes me wonder if you are serious. On that note, there's no such word as "orientated". It's "oriented". (Likewise, there's no such word as "administrate"; it's "administer".)

Languages, posted 2 Feb 2003 at 03:49 UTC by mslicker » (Journeyer)

Languages are not skills, skills are independent of language. At best a language can open up new more productive ways of thinking about computation, but computation is independent of language.

With minor differences in syntax and semantics, the languages mentioned are mostly equivalent. It seems the author is simply shopping around for the library support available in each language. Really, I don't know what can be gained from such an approach.

the languages mentioned are not equivalent, posted 2 Feb 2003 at 07:14 UTC by johnnyb » (Journeyer)

The fact that all the languages are turing-capable does not make them equivalent. Expressive power is just as important. Perl and Python stand out from the others because of their ability to do closures (Perl more so than Python). Python stands out from all of them with the ability to do generators and bound functions (which are really just another method of doing closures, but it can make some coding patterns a cinch). C++ stands out with it's ability to do templates, and a huge amount of compile-time work. Perl has probably the richest set of libraries. C is a great language for its predictability, and for it's binary compatibility. Perl and Python have garbage collection. You can add GC to C and C++, but it's not standard. If you need to do regular expressions, perl has the best and easiest support for that. Python has the best and cleanest implementation of OO concepts. Scheme wasn't mentioned, but scheme has the ability to do continuations, which can simplify a lot of things which are normally hacked around. Lisp has an amazing macro system which puts C++ templates to shame. You can basically write full-featured program generators in Lisp. Interestingly, as pointed out by Andrei Alexandrescu in Modern C++ Design, C++'s compile-time programming system can only be done in a purely functional style. Interestingly also, in Lisp, you can also do procedural compile-time programming. Odd, eh? Perl is a great language for mixing OO and procedural style, and is especially good when moving a project from its initial stages into a more complicated sttage (i.e. - makes refactoring a lot easier). It also has a great documentation system.

Anyyway, all that to say that there are mounds and mounds of differences between these languages mentioned. It goes way beyond syntax. The lack or availability of any single one of these features can change the design of the program dramatically for good or for ill. Trying out a program in multiple languages is a great idea for learning how languages influence design decisions. It would be meaningless to do so on a trivial program. The fact that he did it on a real-life program means that he will learn a ton.

Many programmers have forgotten how much you learn just by playing around and doing silly things like this.

Thanks for the feedback., posted 2 Feb 2003 at 13:58 UTC by Stevey » (Master)

 True I think the correct solution would have been to get to grips with Python right at the start. Had I done so I would have learned interesting things, and become capable of programming in another language. But by not going that route I learned different interesting things.

 In the C incarnation the code just grew huge, without any overriding design; as an attempt to change this I made frequent use of Design Patterns in the C++ evolution, and this combined with the doc++ system to allow for stunningly simple, and effective code documentation. (Combined with some overview material)

 Part of the reason for switching from C++ was to avoid buggy std libraries, and issues with different compiler versions. Had this not been necessary I would have probably hunted around for a decent database interface library, rather than going with Perl.

 Still perl has a lot to be said for it in a networking server application, you get the choice of fork()/select() type servers, along with a great deal of file/text manipulating functions thrown in. Alongside that there's the knowledge that if you ever need to interface with anything somebody has likely written a module for it already. <hr />

 (And for the record I've programmed commercially and for fun in Lisp, x86 asm, Java, perl, C, and C++. Each language has it's place.)

Misunderstanding, posted 2 Feb 2003 at 17:13 UTC by mslicker » (Journeyer)

johnnyb, Perhaps my comments need some explanation. When I say the languages mentioned are mostly equivalent, I'm not talking about turing completeness, I'm talking of the way programs are expressed. Essenially programs in these languages are data + procedures. The differences are in functionalities, traditionally available as a libraries, emodied directly in the syntax. To me this is a frivolous excuse to write a new language. Equally frivolous is rewriting a program based such minor differences in languages. Though I conceed on one point, garbage collection will have major influence on the development of certain programs. However, this should be entirely obvious to begin with, you shouldn't need to rewrite a program to know the effect of this feature.

Now, you somehow attribute my statement to saying Lisp is equivalent to the mentioned languages. Lisp devserves distintion from this group for sure. In Lisp programs = data, the Lisp model of computation is the lambda calculus. I must say though, it is likely this particular program would not greatly change rewriten in Lisp. Application programs often have a imperative nature to them that will show similar patterns in just about any language. Hence my statement that computation is independent of language. What then should guide language choice? Economy of expression. Given a precise idea of what you want to achieve, how easily can this be expressed in a given language. This should not be confused the superficial embedding of library features into syntax that seems to have so much sway in current language choice.

As far language influencing design, I hope you mean on level of how you would express the program, not the actual design of the program. Design should independent of language. Where programming and design mix (object oriented programming), is where you see much unnecessary work expended. Code that serves no actual purpose -- mere scafolding which surounds the real program.

python is worth learning, posted 2 Feb 2003 at 17:29 UTC by mkc » (Journeyer)

As an exercise, I think it's probably very useful to recode this software in a number of different languages.

Learning Python would also be very useful, even if only as an exercise. I suspect that if you applied yourself, you could know more of Python in a couple days than you do of Perl now. Python is unsurpassed for syntactic clarity and has some interesting features (e.g., comprehensions) not present in other mainstream languages.

Cheers, --Mike

Learning vs Knowing, posted 2 Feb 2003 at 18:13 UTC by Stevey » (Master)

 mkc you're right that I could probably learn the basics of Python quickly.

 But there's a different between understanding a language sufficiently well to hack it, and knowing it well enough to use it "properly".

 When I say I don't know python what I really mean that I can read it, and I can modify existing code, but the whole 'set' thing doesn't sit well with me - I would be unlikey to use them with appopriately - so any Python I write from scracth is just Pythonesque C, or Pythonesque perl, not Pythonesque Python.

 As an excercise I'd love to encourage people to do something similar, for a sufficiently sized program.

 I'd just be wary of suggesting people use languages completely new to them - after the first implementation the problem/project would probably be understood fairly well; but the implementation wouldn't be as good as it could have been from a person who understands the nuances and idiomatic styles of a coder who knew the language.

Ruby, posted 10 Feb 2003 at 01:29 UTC by Guillaume » (Master)

johnnyb : Following up your comments on Perl and Python, I suggest you look at Ruby :-)

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

X
Share this page