Developing with Open Source

Posted 26 Mar 2000 at 14:57 UTC by darkewolf Share This

The Open Source phenomena threatens to overtake us all. IBM is supporting it strongly, Sun is doing it and even the bane of existence, Microsoft has an interest in it. But where does that leave developers and potential developers that have found themselves called to this clandestine world?

Seasoned Developers

People that have been developing software for the commercial market for some time will have some idea about how you go about developing software. Migrating to a new market and possibly a new tool-set won't cause them too much trouble. In fact many ex-commercial developers have contributed useful tools that mimic the closed-source equivalents. The developer can create the tool they are used to and then add the feature they wished it had. The best example that comes to mind is cgvg, an almost-clone of cscope. Cscope lets a programmer dig through source code and find out where something is defined and also where it is used, cgvg replicates this functionality using Perl.

The seasoned professional might need a little pointer toward the particular tools that are available. But once they are headed in the right direction, the process of development should not be a hassle.

The only obstacle for ex-commercial developers when it comes to developing Open Source software is the justification. No longer is is the connection between code and money obvious. You give away the thing you have spent hours writing.

Neophyte Developers

But what about the enthused youngster that just installed GNU Linux for the first time or the mathematics professor that is watching FreeBSD seamlessly build. Their minds are full of ideas, they know their code but their experience from the university only provided them with vi and gcc while they learnt. They don't know anything about the tools that are available, nor do they understand the design methodologies that large projects require.

Unfortunately the neophyte will often assume the tools are the most important aspect of developing their wondrous idea. Where more often than not the approach to the development is where their energies should be invested. Those five extra minutes per session documenting changes could make the world of difference when someone reports a bug three weeks down the track.


A good place to start finding out about the tools available are places like freshmeat, searcha engines and of course its always good to ask people that do development themselves (IRC is a great way to do this). It is important to remember that not all tools are directly related to code. How are you going to track the time spent on each part of the project (doing this is great for working out how to distribute the workload latter)? How is it going to be advertised? What format do you want documentation in?

The Approach

Tools are only part of the picture. How the project development is handled is the crux of the matter. The mathematics professor is a one man team, should she just load the editor and start hacking away at the keyboard. Definitely not. As any university course on software design and engineering teaches, and as any software house will say, all software should be designed first: State what its function will be. How will it do this (the algorithm)? How will it store its data? From there you have an outline to start coding from.

"But I am just one person!" you call. Maybe so, but if the program gains interest and has potential, it could be the next Apache or even the next GNU Linux. Why spend time developing it from one person's point of view, only to have to restructure the whole thing sometime down the line to allow for five developers, or five hundred. Play the many roles necessary for development. Have an account for documentation, code development, PR (web page design). With today's UN*X-like operating systems, making new accounts at home is easy. Treat the roles as if they are different people, wear a different hat each time you perform each role if that helps you remember. By separating the roles, they can be easily assigned to other people if the need comes up.

Be very strict about the project. Write guidelines on how the code is going to be formated. Write guidelines about the extent of documentation required both in the code and externally. And then, stick to your guidelines. This way you'll understand the code in two months time, but more importantly if new developers join the project, there will be some consistency and cohesion to it.

Peer Review

Peer review is an important factor of the Open Source movement. You are provided with world wide audience that are quite willing to look at your creation and critique it for you. The good and the bad points will be explored in great detail (usually in a mature and sensible mannner, but with anything there are a few bad eggs). If your project is even slightly useful to at least one other person you'll get bug fixes and feature suggestions, making the work load easier for you.

It has been said that Peer Review is what drives the Open Source movement, and makes Open Source a community rather than just a licensing issue.


Why would you want to develop Open Source software? Because you can. There is nothing to stop you writing the perfect text editor, or the best accounting package possible. Why not write for the ego of it? If you write something that is impressive enough (either through eye-candy or rock solid design) people will talk about it, your name will be mentioned. Why not be the next Linus, Larry Wall or even the next RMS. Write a program for the pure love of it, and watch your program make thousands of people's lives just that little bit easier because your word processor adapts to their needs as they go.

With free tools, free help and an enormous community spirit, even the most grandiose idea can be accomplished. Think of it as programming the minds of your users and potential future developers.. Load them with code in the hope they'll execute it ... code for their machine, code for their mind. in the same way your software provides the logic to perform a machine function, the documentation provides the user/developer with the logic to do the same. Share the code, spread the word!

Ugh, posted 26 Mar 2000 at 17:03 UTC by xach » (Master)

This article is rambling and fluffy. It may have a point, but it's hard to find. Maybe you could trim it down to 10% of its size and make a clear point concisely and directly.

hmm,, posted 26 Mar 2000 at 19:53 UTC by mwimer » (Journeyer)

Its obvoiusly written to 'Seasoned Developers' of the (ex-)commerial type and to 'Neophythe Developers', not to someone of your L33T code hax0ring skillz, Xach.

sun?, posted 26 Mar 2000 at 20:27 UTC by graydon » (Master)

I think it's a pretty severe warping-of-truth to say that sun is "doing" free software. "botching" or "abusing" might be more accurate.

Take two, posted 26 Mar 2000 at 20:28 UTC by xach » (Master)

Ok, perhaps I should expand a bit.

The article starts off by stating that "the Open Source phenomena threatens to overtake us all." That would seem to indicate that this article is for everyone. But my eyes started to glaze over when I read about "the bane of existence, Microsoft."

It goes on to pose the question, "Where does that leave developers and potential developers who have found themselves called to this clandestine world?" I've read through the article several times, but I can't seem to find the answer to that question.

The article presents a sequence of paragraphs that are tied together only weakly. They're not well-written. They don't clearly illustrate a common theme (or perhaps I'm just not in tune with some current popular groupthink that would make this article seem worthwhile).

I would enjoy it more, I suppose, if it clearly said who it was intended for, justified some of its assumptions (e.g. Microsoft is satan), and was more focused on a particular theme. As it's currently written, I really can't figure out why anyone would find it useful.

Call me Old Man Grumpus, I guess.

Hm., posted 27 Mar 2000 at 01:02 UTC by kelly » (Master)

This article is an exact replica of the author's editorial on freshmeat. Frankly, I find myself agreeing with Xach -- this article meanders without any real sense of direction or theme, which might be fine for a diary entry, but I expect better for lead articles. This article should either be about 70% shorter or 300% longer.

whoops, posted 27 Mar 2000 at 11:50 UTC by darkewolf » (Journeyer)

I actually didn't mean it to be posted in the sense it was, it was meant to go as a diary entry. However, instead of getting some sleep while suffering a migraine I decide to post various news services.

Much apologizing to those whose time i wasted.


Old Man Grumpus, posted 27 Mar 2000 at 15:16 UTC by scottyo » (Apprentice)

I think you were too harsh. I think it's fairly clear that the article is aimed at the neophyte developer. In most universities, students can learn programming languages, theory, software design, and perhaps some related mathematics, but there are not nearly as many universities where students can learn details about "open source" development. At the universities where I attended and taught, there were no classes that even touched on what are such basic tools as CVS or autoconf. Most of even the simplest open-source projects I have built and installed are more complex than any project I completed as a student or taught as a teacher. I am in the process of designing my own project which I intend to release as open source, and so I for one appreciate some tips and pointers, wherever I can get them.

In addition, darkewolf has only been certified as an apprentice on advogato. If you, being a journeyer, read his article expecting to learn something new, and didn't, then you should have taken note of that fact. Where the article might have been weak in your eyes, then offer your critical insight to improve an apprentice's (and others') understanding instead of just belittling the effort. If on the other hand you aim is to drive away budding open-source programmers from advogato or to keep them from posting, then you are pursuing the right tactics. OK, you are an Old Man Grumpus.

who is this written for, anyways?, posted 27 Mar 2000 at 15:39 UTC by cmm » (Journeyer)

I was under an (mistaken?) impression that Advogato is the least probable place where I could find an editorial with words like "seasoned developer" (who?), etc.

when a person calls me a "developer", I assume he is a "developer" too. I prefer to define myself as a programmer, dunno about other Advogato members.

disclaimer: the above comment may or may not be fair, as my English is by no means perfect.


Ability to write advocacy != ability to develop software, posted 27 Mar 2000 at 18:13 UTC by kelly » (Master)

The problem isn't that the article expressed misunderstandings about developing free software. My complaint, and apparently Xach's, is that the article does a poor job of expressing what it's trying to express. It's the form, and not necessarily the content, that needs work.

Advogato certification levels aren't supposed to be predicated on the ability of the individual to put her thoughts into words.

Aaaaack!, posted 27 Mar 2000 at 20:04 UTC by advogato » (Master)

This site already has problems with people being too intimidated to post. And now this! After this kind of treatment, I'm surprised that anyone would post (aside, of course, from my thick-skinned and thick-furred self).

One of the goals of this site is to give developers practice in improving their ability to communicate in writing. So this piece wasn't as earth-shattering as a Knuth interview or as well written as one of my editorialls (meow!). It's still a piece reasonably on-topic for the site and deserving of constructive criticism.

To that end, I'm going to outline a few tips for writing the near-perfect Advogato essay.

What are you trying to say?

In English classes, essay writers are encouraged to come up with a thesis statement, which belongs in the first paragraph of the essay. You might think of this as overly rigorous, but in my opinion the discipline is worthwhile. It seems particularly difficult to do when writing about software, probably because of the inherent difficulty and complexity of many of the interesting concepts.

Who are you writing for?

The Advogato audience consists of people who are actively developing free software. I think it can be helpful to look at the thesis statement in this context. The thesis statement of this essay appears to be "writing Open Source software is a good thing." Well, that thesis statement runs pretty close to preaching to the choir. A really interesting thesis statement would be "there are some cases when it's really better to develop proprietary software" provided you can give examples and really back up your arguments. But consider, in a Microsoft developers chat forum, it would be the other way around.

Why are you writing it?

There are a lot of good reasons to write an essay - to argue your views on a controversial subject, to inform people of news (implying it's not already well known), to share insight and experiences gained from working on a project, or simply to explore the frontiers of knowledge, if only in a small way. But in any of these cases, it's probably a good idea to be aware of your reasons.

Rewrite, rewrite, rewrite

The secret to really good writing is revision. Steven Pinker (himself an excellent writer) argues this point convincingly in The Language Instinct, and it is of course a basic tenet of The Elements of Style.

Even better is when you show the drafts to a few people to get their comments. Having another set of eyes on the text helps you avoid basic mistakes, and good reviewers can add insight and perspective to the piece. I very often distribute a draft of my Advogato's Number editorials to the kind folks of #gnome before posting to the front page. Note the analogies here to Open Source itself :)

I realize it can be difficult to find reviewers for drafts. Thus, a planned feature here is an explicit queue for drafts, with a mechanism for people to submit comments, and for the original poster to go through revision cycles before submitting to the front page. But it's certainly possible to get feedback and make revisions even without such support from the website.

So please, let's have more content here, not less, and let's work together to make the content the best we can.

Old Man Grumpus applauds, posted 27 Mar 2000 at 20:15 UTC by xach » (Master)

Good call, advogato. I'm spoiled by the great quality of articles by you, that Raph Levien guy, Raphael Quinet, and David Monneaux. After seeing an article so much weaker than those, I wanted to register my displeasure.

A draft system would be nice, and it hopefully would have made this article strong enough so that it wouldn't be so drastically different from some of the better Advogato articles. But until a draft system is in place, do you (as the advogato overseer) think that the main article system should be the place for article-polishing?

(Should this whole discussion be moved into a new article?)

An ethic of sharing and helping?, posted 27 Mar 2000 at 20:21 UTC by Ankh » (Master)

I sympathise with darkewolf, I posted a diary entry under "personal details" once. There are some usability questions there; some are common to HTML-based Web user interfaces today, and some are Advogato-specific. For example, using different screen background colours for post diary, post article might help non-Lynx users.

scottyo raises some other questions, and some of them have been bugging me for a while. An "elitist" meritocracy based on who you know and the whims of those by whom you are known is an environment where politics and back-biting are to be expected. In fact, at least on the surface, there seems to be little of that. Yes, I noticed someone "demoted" from Master to Journeyer post a rant, and then retract it quietly not long after. And it was odd to see someone I'd never heard of "certify" me as an apprentice, too.

There's a social behaviour that I'm used to associating (for example) with EFnet, where it's considered acceptable to flame the newbie; in MIT jargon, the non-programmer user was called a luser (you have to pronounce it in an American way, to sound like loser), and was, by implication, unimportant. It's odd to live and work in an industry that considers its customer to be unimportant and pathetic, and where social reward comes from becoming so elevated a wizard that you can be brusque to those less elevated.

In my own writing, and when I have taught or mentored programmers, I have tried to instill an attitude of helpfulness and cooperation, of respect for the needs of others. So perhaps I'm oversensitive. I've seen (and written (ulp!)) far more severe flames, but xach, kelly, cmm, let's offer positive suggestions. And if you don't have positive suggestions, offer a clear statement of a problem in a neutral tone, so that others can respond with suggestions.

I remember talking about my C coding style to someone, and explaining that it was designed to be robust in the face of other people modifying it, when they might be less experienced or in a hurry, and his response (half-joking) was "well, if you're coding for dummies!" No, I was coding for intelligent people who happened not to be experienced at C, so I avoided "tricky" features. I do the same in Perl, for the sake of people who have to modify my code later.

The true guru does not eschew the company of the beginner, but is always aware of where those first footsteps may lead.

Shoud we set up a collaborative authoring server so

apology, sorta, posted 27 Mar 2000 at 21:23 UTC by kelly » (Master)

I don't feel that my time was wasted in reading (or replying to) this article. Nor do I want to think that I'm trying to discourage posting. What I am trying to do (and probably badly) is encourage posters to apply a slightly higher standard of editorial care than they might when posting to something like, say, Slashdot.

I've been specifically trained in legal writing; while legal writing is a rather odd form of writing, many of the same principles apply to standard persuasive writing, which is what most editorials are. I see Advogato has already offered some general advice on writing, all of which I agree with. If darkewolf wants to contact me privately (kelly at, I'll be glad to discuss in more detail how he might improve his essay. I'd rather not do a detailed dissection in public, though.

I do like the idea of a draft review queue, although I also think use of it should be optional.

And we probably should move this discussion to another thread.

On "being strict", posted 28 Mar 2000 at 17:41 UTC by apenwarr » (Master)

Darkewolfe says: Why spend time developing it from one person's point of view, only to have to restructure the whole thing sometime down the line to allow for five developers, or five hundred.

I know people say things like this in the most well-meaning way, but in general it's just not good advice. As far as I can tell, the major difference between successful and unsuccessful open source projects is that the successful ones start simple. The Linux kernel was originally a terminal emulator; apache was a bunch of performance patches on top of NCSA httpd. Debian never used to have a constitution or voting mechanism.

Grandiose plans work (sometimes) in the corporate world, because the only way to get any kind of investment is to sound impressive up front. Open source is different. Your project has to work before anyone will care.

Besides, evolution is a natural part of programming. For a long time, I believed that once I got good enough, or gained enough experience, or looked at enough people's code, or just got more mature personally, that I would be able to design things right the first time. After all, I thought, that's how it's done in the real world.

Yeah, right. I've certainly gotten a lot better, and because of my experience I can pull of bigger projects without failing miserably, but I've learned this through my own experience and by observing lots of brilliant (and not-so-brilliant) programmers in action: it will never be right the first time. The bigger your first try is, the more work it'll be when you have to rewrite it.

So start small :)

And organizing people is a lot like organizing programs: you'll screw that up the first time too. Doing things by yourself vs. with 5 people vs. with 500 are totally different things. Restructuring for 500 people will be hard, but there'll be 500 people to help when the time comes. Trying to work with 5 people as if there are 500 people will just overrun you with paperwork :)

Have fun...

Apache..., posted 30 Mar 2000 at 02:01 UTC by mbp » (Master)

Here is a clear explanation of how a medium-large project, Apache, works.

Thanks, posted 30 Mar 2000 at 14:22 UTC by darkewolf » (Journeyer)

As much as criticism hurts, stepping back and listening to it without taking any barbs (intentional or not) seriously, it has done something useful for me. It has shown me some holes in my ideas. It has certainly shown me places in my document where ideas should have been much clearer.

Brushing up on my english skills would be a good idea. Although its considered my native language, its fallen to the wayside compared to perl and C.


anal-retentive grammar cop chimes in, posted 3 Apr 2000 at 09:24 UTC by branden » (Master)

singular -> plural

phenomenon -> phenomena

criterion -> criteria

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

Share this page