Got Stable?

Posted 24 Jun 2002 at 14:50 UTC by SteveMallett Share This

Are there readily available indicators, that are more than judgement calls, to you and your users that a program has reached `stable' in the development process?

Theoretically, a `stable' software project means that it is production ready. But, what does that mean? Right now programmers make a judgement about whether their software is stable in the development process. It goes from planning to alpha to beta to stable to mature. Sometimes in one giant leap and sometimes painstakingly over years depending on the nature of the project.

There are stable releases of beta code too. However it seems obvious to us that these `stable' releases have the quality of just not being totally screwy. And that is good enough for software in development. The stable release won't screw up and the unstable release probably will or cause you problems. Fair enough.

But in the overall development cycle what does `stable' mean to you as a developer? And is it the same as what stable means to your users? What do you think are the cold facts of a project that indicates that it, not a particular release, is stable? Are there indicators, that are more than judgement calls, to you and your users that a program has reached `stable'?

My View on Stability, posted 24 Jun 2002 at 15:15 UTC by shlomif » (Master)

A stable release, as far as I can tell, means that you used it extensively and it did not break on you, and that all the features that you wanted to put it in on the development branch are already implemented in it.

Note that it is a good idea to use a Linux Kernel-like versioning mechanism, so you'll be able to announce both stable releases and development ones without breaking synchronization.

How much a stable release should be verified to be bug-free depends on the functionality of the program. For example, a CD-playing front-end need not be as secure as an IRC client, which in itself can be less secure than a program that may run in superuser priviliges. If security is the issue, it is a good idea to have code reviewers and auditors.

Just my 20 agoroth...

depends..., posted 24 Jun 2002 at 21:29 UTC by xtifr » (Journeyer)

It's really going to depend on how you run your project. In practice, a project is "stable" when the person or persons in charge of labelling it decide to label it "stable". Generally (with sane people), this is going to mean that all of the planned features are implemented (or have been officially postponed to a later release) and work (for some reasonable definition of "work").

From this fairly pedestrian point of view, the term "stable beta" may not seem to make much sense, but, in fact, most "stable beta" releases I've seen are missing planned features, but the features that are implemented work. So that fits.

Bottom line, if you actually have a plan and a design, you shouldn't have to ask "is this stable?" And if you don't, you probably have much bigger problems than worrying about what to label the thing.

test suites and goals, posted 25 Jun 2002 at 02:20 UTC by splork » (Master)

xtifr hit the nail on the head. unless your project has defined goals in terms of functionality that must be there and bugs that must be not you shouldn't call it stable.

One highly important thing to do in any development process is write effective tests. Both unit tests and overall test suites. Nothing should be called stable if there is not a consistent way of testing that its functionality reaches some defined desired state. Read up on XP (extreme programming) for more good ideas along these lines.

you mean project stability?, posted 25 Jun 2002 at 05:13 UTC by bdodson » (Journeyer)

The notion of a "stable beta" refers to application stability. The app is tested enough that you don't think it will crash very often under normal use. That kind of stability is quite different from project stability. When you're talking about the stability of a project (or of a branch) you are saying that it isn't going to change much, except for bug fixes and minor incremental improvements.

If your product includes an API or a scripting interface, developers can be confident when using your "stable project" that upgrades within that branch will maintain compatibility. Likewise, if your app is a productivity application, end users like to have some confidence that the document file format will not change in a way that breaks compatibility. Therein lies the marketing value of labeling your app as stable.

Keep in mind that working on a stable project is boring. If it is not a widely known, popular project, it can also be thankless - the hacker community respects acts of altruism, but only if they notice. In any case it does not have the same instant gratification that comes from implementing something cool and new. (Exception: the bug that has been bugging you for years, and now you've eradicated it.) If you are going to tap into the power of the commons, you would do well to have a faster moving branch to hold developers' creative interest. "Feed my id."

I'm dealing with this right now, posted 25 Jun 2002 at 05:26 UTC by Bram » (Master)

My own project BitTorrent is currently nearing maturity. It currently does an version check to a single server every time it starts up, and once it reaches maturity I'm going to get rid of that behavior.

Because there's a protocol involved, getting rid of the version check amounts to much more than drawing a line in the sand. Once installations which contain no version check are widespread, changing the protocol will be difficult, if not impossible.

As Splork said, the key to being able to declare maturity is keeping the scope contained - right now it's very tempting for me to try to tackle streaming, or try to add features which might help with that later, but that could easily put off the first mature release for another year.

I'm going to push out a release with no version check once all of the following criteria are satisfied -

  • The list of critical todos is empty
  • Nothing new has been added to the list of critical todos for a week
  • No new bugs have shown up for a week
  • A large-scale deployment has been done without a hitch

I've been approaching this goal slowly (sometimes it seems asymptotically). All the old todos have now been finished, and only two new ones are left. A bad bug was squashed a few weeks ago, but since then there have been no indications of ill behavior caused by a bug whatsoever. There was a recent deployment which went almost smoothly.

Alarmingly, the rate at which new tasks have been appearing has been pretty much constant, although it's been slower than the rate at which tasks have been completed, and I expect to soon just plain run out of things which can possibly be improved.

One final safeguard is that I intend to make the first version with no version check on startup be identical to the previous one except for the lack of version check. That is, after all, the point when it becomes very hard to fix something quickly and without much pain.

Thanks :), posted 25 Jun 2002 at 19:21 UTC by Denny » (Journeyer)

Just wanted to say 'thanks' to the people who posted comments above this - I'm looking at adding a stable branch to my pet project YAWNS soon, and it's been very helpful to read the points you each made - particularly the distinction to be drawn between 'interface' stability, and 'bug free' stability.

Looks like I'm holding off for mostly the right reasons so far - nothing major, just tidying up stuff before I call it tidy I guess. I think a lot of open source projects are probably quite neat in that respect before they launch a stable branch, because it's so embarassing when people send you really simple fixes to your hideously hacked together algorithms :)

Anyway, this kind of article and feedback is why I read this site, so thanks again...


New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

Share this page