Ask the Advogatos: why do Free Software projects fail?
Posted 20 Jul 2000 at 17:08 UTC by lalo
Over at Hack&Roll I'm
come up with a tool for inception, design and tracking of Free Software
projects, and an
matching formal programming process designed to address the most common
reasons Free Software community-based projects fail. I'd like your
opinion on what these reasons are...
I have been in a lot of projects which succeeded and a lot of projects
which failed. I have my own opinion about why projects fail, and some
measures that, despite being unrelated to these reasons, could avoid
that (like a formal process, to begin with).
But of course, I may be wrong ;-)
One of the major points of my process is getting comunity feedback all
along - after all, that's one of the most important things that
distinguish community-based Free Software development form other models.
So, before I say "this process and site are finished and ready for use",
I'd like to hear from you possible users.
Please don't post feedback as stories in the Farm; either post comments
to this Advogato story, or in the "Feedback" area of the CodeFarm Project's
Too many OSS hackers have no formal training/background in
*engineering*. Many hate to do boring but necessary stuff like
documentation, design, and regression testing. Also, they are too
green (as a rule) to know how to *invent* things; they can only copy
what they've seen done a thousand times before -- witness the legions
of FTP, IRC, and news clients. Add to that the dangerous ego-bloat
prevalent in the Linux community, and you've got real problems.
This is the true "dark side" of Open Source: not all hackers are
created equal. Many programmers are simply *bad*, and need much more
training before they become adequate. Yet much lousy code makes it
into Linux because no one feels empowered to "just say no".
IMO, the Linux Standard Base would be a good place to implement
refereeing: vetting code for correctness and cleanness before it is
allowed into the "standard" distro. (This would also eliminate a lot
of stupid security exploits.) But you can bet that a lot of hackers
would protest when their crapware is kicked out of the distros and say
that the initiative is "fascist".
It's a hard thing, but it badly needs doing.
Revision tracking, posted 20 Jul 2000 at 18:18 UTC by prash »
I think something like FreeBSD's revision tracking system might help
some projects, large ones especially. People can see when changes were
made, what was changed, and why. So if "lousy code" does make it in to a
project, it would be a little easier to see what was intended and
hopefully fix things.
This might not be as useful for a project with just a handful or
developers, but if lots of developers contribute, then it might make
things easier. Also, if the project takes a turn that a lot of people
don't like, they can go and get the source of whatever version they want
and work from there. This would prevent people from having to start from
scratch if the goals of a project they were contributing to changed
It's difficult and time consuming to bring a piece
of software to a "finished" state. It's a lot
more work than most anyone is willing to admit.
Seems to me it is pretty obvious why Free Software
or any software projects fail. It's hard work. Or
am I just kooky?
And if the sheer workload isnt enough, though in
personal problems, political problems, leadership
problems, and the mother and root of them all: financial,
corporate, and monetary issues. fun aint it?
If you want long endless eye glazing semantic
jousting about licensing, development methodolgy,
personailty traits of developors, paranoid
delusional conspiracy theories about any given
vendor, or the moral superiority of one
or more user communities, I'm sure someone
else on advogato can meet those needs.
failure, posted 20 Jul 2000 at 21:43 UTC by higb »
What does failure mean in this context? I'm tempted to take
the view that failure means a project that benefitted no one. I
don't think that happens very often. Even the ghost towns are there,
if nothing else, to teach the danger of false expectations.
I consider my mpEDIT project to be dead, but it got to be one
chapter in somebody's book, was used in a college course, and was
picked up briefly by the NCSA. Along the way it helped people learn
programming. Even if it never got to be a widely used tool, I count it
as a measured success.
- There was a GPL project once to create a Mathematica style
numerical programme - It never even got beyond the design stage.
When I looked through the mailing lists there were tons of people
all giving their opinions on how the system should be designed,
but no-one ever took the inititative to start writing
something. As a result everyone eventually lost interest
in a project that was clearly going nowhere.
- I have often considered contributing to a project, but the
code was so poorly written (Read: poorly documented/commented,
long functions, meaningless or unintelligible variable names, and so on
ad nauseum) that it just wasn't worth my time to decipher someone
else's crappy code to make the small change I wanted to.
- When starting a project you need to actually write something
before announcing it to discourage the hangers on who have no
intention/ability to contribute from flooding your mailing list with
- Everyone should be using Knuth's Web system of
literate programming to lower the barier for those who want to
contribute to your project.
Rob Pike, in his presentation, "Systems Software
Research is Irrelevant," encourages work on component-based
applications, but I don't think that he would agree at all with Miguel's
fawning over MS COM. Pike writes:
There has been much talk about component architectures but only one true
success: UNIX pipes. It should be possibile to build interactive and
distributed applications from pice parts.
(I think he considers the "plumbing"
architecture in Plan 9 to be something like the next stage of UNIX
oops..., posted 20 Jul 2000 at 22:41 UTC by jaz »
Wrong article... (How did I do that?)
practical issue, posted 20 Jul 2000 at 23:37 UTC by higb »
As a practical issue, beyond my what is failure folderol, I
agree with mettw
that code is the bottom line. I've seen projects get off to the wrong
foot, with an empasis on who is managing what, rather than who
is coding what. A tool that emphasizes the code base, and
spotlights code contributions, might help things stay productive.
I feel that a great fault with many Open Source projects is that they
aren't managed well. The initiator of a project doesn't need to
necessarily know how to code him or herself, but he or she does need
know how to gather, and organize people to accomplish a common goal.
Fortunately, SourceForge is
helping out a lot by having hosting for Open Source projects.
This gives developers the resources to create a community forum for
discussing the project, and keeping it's development under scrutiny.
Personally, the projects I've been in have succeeded, I feel, because
of two reasons:
First, I am the sole developer; this means I organize, code, and
document the project; unfortunately, if I tried to delegate out the
responsibility, I would more likely fail, because I may not have the
greatest management skills.
The second reasons my projects have succeeded is likely that they are
small, easily-graspable projects. The goals I set are accomplishable,
and I can forsee from the beginning how the project could come together.
Helpful in this is the fact that I use a high-level language, Perl; it
allows me to develop rapidly, keeping everything in my head at once.
management vs. code, posted 21 Jul 2000 at 03:00 UTC by higb »
I don't know ftobin, maybe you can
show some examples
of sites that got into trouble having code but no management. I had the
opposite experience. People would email me and offer to suck my 40
thousand lines of code into their project so they could manage me. At
one point I joked that I could lay those guys from here (California) to
New York. I'm tempted to point to one such site, still on the web ...
but I'll leave it anonymous. From that page:
Establish the feature set for the editor.
input, so please feel free to view the features we have considered and
send me your comments. We also need to recruit programmers, create a
design, write code.
IIRC, it has been that way since sometime in late 1998.
Concerning the quote that higb gave us, I would say
that there was definite lack of management going on in such a scenario.
It appears that the project in question was poorly managed in the sense
that they tried to create a project without any codebase; or they bit
off more than they could chew. This is what a good manager needs to
make sure doesn't happen.
Managing projects means thinking things through, and making sure the
work can and does get done. It is not necessarily in conflict with or
exclusive from creating code; rather, making sure the code is created is
part of being a good manager.
mettw has the right of this. Most projects fail on startup or near enough to that time through
marketing style dross. This app will be incredible, super this, super that, a powerful burst of
synergestic energy will sweep away the shadows and we will march into the new era.
Seeing as we've got 4 lines of code somewhere along the line someone must write the rest, I
know, the other free software hackers will do it.
You got to be focused, and you got to take things one very small step at a time, grandious
goals are doomed to failure, did linus hop out and say "ta-da heres a plan for a fully
featured kernel with diagrams and neat annotated graphs of future progress, I havn't written
a thing yet, but hey lets do it", far too many of these aspirational projects seem to have taken
the plot of some sort of movies where the main character calls his friends or sends out a call
and this vast army of misfits/ordinary folk/ewoks and succeed in overthrowing the
system/baddies/empire because their hearts are pure and they have a half baked plan.
No code, no success. Keep it simple, create informal milestones, "I got this vague semiworking
kernel, neat eh check it out", the next small achievable thing to do would be to ...
One achievable step at a time and some serious motivation by the person who started it, you has to
want this thing. You have to write it, not your army of willing minions, sure you might get a few patches
here and there, and even accrete a group of hackers whose contribution outnumbers your code, but
going into the project with that attitude is inane. The Gimp authors did not head into their project with
"We're going to write a supreme photoshop competitor and along the while write a Motif killer as
a sideeffect", you increment towards it. Proving that your tiny group has the wherewithall to actually
achieve something constructive, and then you go on from there.
Firstly, a definition. I believe that a project has failed if it never
garners any users or a useable version of the project is never
released. A project that is finished but could be improved and has more
than one user is successful in this context.
I believe that projects like Berlin or GGI failed because of a lack of
political nouse on how to handle the interaction between the
established power base - and before they had a single line of code. A
project I'm working on, reiserfs, may fail for this reason as well. You
must exist with your neighbours and bring them along for the ride -
both Sun Tzu and Machiavelli both agreed on this point. Berlin, GGI and
reiserfs all fill a void that must be filled, but maybe not by these
particular projects because they offended the sensibilities of the
Some projects never even get off the drawing board simply because it is
too hard to write them without either a dedicated team of clever people
with some form of an idea of what they are going to achieve. This is
why the Gimp succeeded (and Koffice will) and there's still no open
source Visio or project management software out there, even though I
and lots of others need both programs. These sort of projects need
either lots of people or they need a few core employed people with
innate or learnt project management and software engineering skills.
Some projects choose inadvisable or just plain wrong tools. Some
projects are impossible to maintain, some projects fill voids required
a few years ago and are filled in by technology advancement (gopher
servers anyone?). Some projects lose interest by their members once
they become stable (pnm2ppa is one of these - it's worked good enough
for me for the last six months, which is why there's been no patches
since then by me, even though it's been finished by others in the
Good or heavily used projects reach critical mass and are nearly self-
perputated - see NetBSD or OpenBSD for a project or two of the living
dead. The people I know who are involved in NetBSD (including one core
member, who's my best friend) are really really good. Great software
engineers, but their visibility in the open source world is near zero.
Is this a failed project? I think "maybe" is the answer. There's plenty
of NetBSD users, but they have no real visibility in the open source
world, particularly at a commercial level. NetBSD are about to get
their first commercial distribution (Wasabi), some 8 or so years after
Linux. I think that this may have been a contributing factor.
But basically it comes down to a lack of time (most of us are
volunteers), a lack of interest, and lastly a lack of skills -
including political skills which causes project failure.
NetBSD is already a complete operating system. It doesn't have that huge hole called "everything but the kernel" that's missing in Linux.
Maybe that's something a successful software project needs. Clearly defined areas of doubt and uncertainty.
Or on the other hand, NetBSD is part of the BSD development world. The BSD "commercial distributions" are FreeBSD and BSDI.
Like the Apache "commercial distributions" are things like Netscape's server.
you claim there are no open source Visio or Project Management tools for
Linux. If you read this ajv, I have to inform you that you are badly mistaked, there is Dia
which is a very high featured application with Visio like functionality. For project
management there is Toutdoux
. Both projects are highly usable today.
This brings me to what I think is one of the biggest problem for many free software projects that has the potential for greatness, it
is not lack of management or coders, but obscurity. The 'obscurity/lack of fame' many good free software apps suffer from leads to few
new users -> few new developers -> disencouraged origianal author.
Another problem is lack of knowledge by the coders on a program other systems that perfom the same task etc, which means
lot of work goes into reinventing the wheel for each project. The lead developer or project manager probably should spend a little more
before starting to code, looking around to see what is already available.
I will agree that obscurity is a major problem. One of the major causes
of obscurity is that there is so much crappy software that people are
unwilling or unable to seperate the wheat from the chaff. Something
needs to be done to make good free software available to people.
In theory freshmeat.net helps make
free software available. In reality
it makes it available for
mockery. By ephasizing recentness it creates a
push for people to get every minor release
out on freshmeat. Maybe we
need a goodmeat.net, where only good pieces of free software are listed,
categorized and with much comentary.
The gnome.org software map
could be this, but it is not. It, like many
linux distributions, suffers from including everything submitted so as
not to offend. And it still doesn't have a search button.
Point: If such a site does not already exist, there needs to be
someplace where my mother can go and get the software she
having to worry that she will get crappy software.
Not that I am going to do anything about it. I have too many projects of
Goodmeat, posted 26 Jul 2000 at 12:14 UTC by mettw »
A `goodmeat.net' system could be set up on advogato. All you
would need is to allow the projects to be rated like people are.