How many Linux Distributions do we need?

In the spirit of Linux, the whole point is about creating your own distrobution. It's open source so that you can modify it how you will, and the GPL allows you to modify distrobutions however you will. Companies offer quick pre-made solutions, but this doesn't mean you should use it straight out of the box. This also doesn't mean you should make www.mylinuxdistro.org for every dist out there. Variety is the spice of life, as they say.

How many Linux distros do we need? However many it takes for each user to be completely comfortable on his/her own system (I'm not saying someone should go out and write three thousand distros to cover many user bases, but if the individual user is exerienced enough he/she should customize his/her distro however they want to).

I think the problem with infinite distros is not the fact that there are many but the fact that we make such a big hoopla over which distro we run. Just run whatever makes you happy. You don't have to run around shouting from the rooftops what you run, though.

My theory for at least some of this is that a lot of the people that put together Linux distributions are not programmers. So instead of using their novice skills to make yet another IRC program, they are putting together new tiny distributions, or a major distribution with some updated packages. They can then feel like they are contributing to the community by releasing it back for general consumption. Just as with the countless other duplication, hopefully they will learn something from the experience and then contribute ideas (either directly or by someone noticing they did something cool) to the major distributions.

The bad thing is the newbies have to wade through pages of distributions without really knowing which are the best to use.

Sure, there's about 80 ICQ client, 70 IRC clients and who knows how many distributions. But point is all of them offer different features and they are all free for download, code modification and use. Point is hack on / create whatever you would like to. This is free software with freedom of creativity. However, reinventing the wheel is not that great of an idea, but most of the time the applications / distributions all provide unique features of their own. Take window manager's for example. Sure, they "manage the windows" but compare Window Maker, Sawfish, Twm and Aewm and you'll see that the wheel has not been reinvented.

One. Debian.

:)

It's easy to empathize with anyone who has not been fully satisfied with their linux distribution, and who wants to make it better. Linux distributions are big, complex, not particularly well-thought-out or integrated, and they all have lots of room for improvement. And by improving them, you have the potential to make things better for a lot of people.

From there, it's really not too hard to detemine why someone would fork a distribution. Why does any free software project fork? Because two groups of people cannot get along and agree on how something should be done, for the most part. Unfortunatly, in the linux distribution space, getting the changes you want into a distribution is often very difficult[1]. Much harder than getting a change into a peice of free software. Many of the major distributions -- the distributions that appeal to many people -- are controlled by companies. Changing something in one of these distributions is well-near impossible for an outsider. So instead, some people try to fork them.

I've had a similar experience. About 5 years ago, I was a happy red hat user, and I was becoming involved in packaging software for red hat, and I also had ideas about ways red hat could be improved. So I was playing around, uploading packages to red hat's contrib section, hanging out on their lists, etc. I suppose I was exactly in the place scandal describes above: not much of a coder, still wanting to feel that I was giving something back.

Until one day it hit me: I wasn't able to actually get anything in red hat changed; I wasn't able to actually contribute to red hat. As I recall, the trigger to this epiphany was a flamewar about where contrib rpm packages should unpack to, according to the FHS. Several people were holding that they were not a part of the distribution and should go in /usr/local. I lost my arguments with those people (although I don't see too many contrib rpms that go in /usr/local today :-P), and that's when I realized I wasn't really contributing to red hat at all.

I actually thought about trying to get a job at red hat so I could contribute to the distribution for real. And I might have applied, and probably at that point (when redhat was a small company, and I had 5 years less experience), been turned down. If I had gone down that road, I very well might have tried to fork red hat to some degree, producing a distribution with my enhancements in it.

Instead, I actually became a debian developer. I joined the debian project, and was a developer within the same week (things were a lot more relaxed back then). Back then, there were many things about Debian that annoyed me and needed to be improved. I worked on improving them. All it took was convincing the rest of the developers that each improvement was a good idea, and doing some of the work.

This article is really very apropos for me, as just this morning I was thinking about the things that still need to be fixed in Debian, which includes one or two items that have annoyed me from the very beginning, and wondering if/when they would finally be resolved. At this point, though, I am very close to having a distribution that makes me completely comfortable on [my] own system.

The points I want to draw from this ramble are these:

• If everyone has to follow my path to find/make a distribution that is fully comfortable for them, there must be a lot of unsatisfied people out there right now. It's a pretty long path. I expect a lot of people don't really care as much about this area as I do.
• In the long term, unless you have terribly focused needs (as some of the people making one floppy distributions do have), maintaining a full forked distribution just isn't practical for one or two people. It's a full-time job and then some. I expect to see many forked distributions die off over time. If I had forked red hat in 1995, my fork would be dead now. :-)
• The alternative, for people who really feel the need to be involved in making a distribution their own, where they can fix all the things that annoy them, seems to be to somehow get in a position to work on an existing distribution. This is hard to do if the distribution is run by a company, with its own goals.
• The other option is to get involved with a distribution like stampede or debian, that is run by volenteers. This isn't particularly easy either -- it'd be a PITA to become a part of debian if you don't agree with our stance on free software -- but at least you don't have to leave your current job and move to become a part of it..
• Both of these options are really too hard right now.
• For every person who has went off and made their own distribution, there is probably another person who joined stampede, debian, or some other open distribution, and is working on making it better. The former people are just much more visible.

[1] I should note that the type of changes I'm talking about here are distribution integration type things, not enhancements to software already in the distribution. I'm not trying to imply that the authors of that code have a hard time getting updates into distributions, as we clearly do not.

This is not unrelated to the Quality vs Quantity discussion from last week.

As someone who's had a go at it, it's a great learning experience to build your own distribution from scratch. Kernel hacking is fun, but it's also fun to learn what else you need to add to a kernel to make a working operating system. Not that I would distribute such a toy...

What annoys me is that very few of these distributions are anything genuinely different. We've already got a hundred instantiations of GNU/Linux. Why can't we see a BSD/Linux? Simon Cozens in TPJ 18 probably did the most impressive distribution hack, which was a Linux distribution with only two platform-specific executables: a Linux kernel and a Perl interpreter. (Sorry, I don't have a working URL at the moment.)

So I guess I'm saying: bring 'em on! The more distributions the merrier, but only if they contribute something new.

I don't think the fork taboo applies to projects that are explicitly driven by commercial or other specific objectives. It only applies to projects that leave project direction open to influence by anyone who contributes.

For example, mozilla.org is explicitly open to community participation in its direction, rather than letting Netscape call all the shots, so forking Mozilla would be frowned upon. Similarly for Debian or the kernel. But if you aren't really allowed to become an equal-footing developer in the project, then forking is viewed as acceptable.

This seems reasonable to me. After all, the justification for any fork is that the existing project doesn't meet your needs; and if the existing project can't be made to meet your needs for technical or nontechnical reasons, a fork is a fine thing. The taboo is against forking for no good reason, rather than against forking period.

Though even for strictly closed projects, I can see at least attempting to get what you need into the main sources (to save yourself trouble if nothing else), and there are certainly gray area projects that aren't clearly open or closed.

I think that a lot of those distributions spring out of one's personal quest to become a master of something. For many, the final journey to being a master can be to make your own. The problem is that some people tend to muddy the waters when they probably shouldn't. But I imagine it would be pretty hard to convince people them not to publically release something that they've worked so hard(or what they think was hard work).

What we all seem to need is for some great leader in the open source community to come along and convince us to share with others these needs to create and get credit for ideas. People are just to damn selfish right now(especially me, help!) to make the whole open source thing work. We need to set examples for others by giving credit to the small guys as well as the big guys in these projects, then I think that maybe people would join together a bit more. Ok, rant is over.

I don't want to add to the noise but I had to add to what Pseudonym was saying. I maintained my own distribution based very loosely on Slackware for a while, from about '96-'98. Eventually I stopped bothering, because I just had too much work to do. But it is a good way to learn how a GNU/Linux-style system goes together, and just how much work it is to put one together. I might go so far as to say that everyone who is really interested in learning about Linux is to try to build an operating system out of it. :-)

I also put together ('97) a (mostly) BSD/Linux on a floppy. This was actually a big win, because it let me build a one-disk rescue floppy without actually ever having seen one before, and the organization which I was working for needed exactly what I came up with. I think that this is an idea which has a lot of potential, and might actually benefit more than just the BSD partisans who wish they could have the best of both worlds. It also impressed me how little code I had to write to compile things straight out of the BSD source tree (and how much slimmer the binaries were than their GNU counterparts...tsk tsk tsk!)

Neither of these systems was ever ready for distribution per se, beyond the three or four machines they ended up running on. But they served their purposes well enough, and I highly recommend the experience.

I must second what jdube said about not making a big hoopla about what you run, as well.

I rewrote the article 3 times and I'm still not really statisfied with what I wrote. If anyone has been offended my main reason was wondering.

joey's reply was very enligthing. I didn't really see the two points about how hard it could be to get the comercial distribution changed (I too used to Debian) and the non-coders way of trying to contribute to the community (even though I'm probally rightfully in that position).

Diversity is good. My problem is probally that by looking at the Linux Distribution List I see very little diversity. Much of the discussion about this could probally just be copied fro the Quality vs. Quantity-discussion which I should have followed better. And the problem is again: Should we stall people making their own distribution until they have a clear idears what new idears they are trying to prove? (And if so, how do we do it?)

I was just thinking about forking this morning - specifically, the license forking that's been going on in recent years. And I noticed something - many of these license forks are beginning to mend. Qt is now under the QPL/GPL. Mozilla will be under the MPL/GPL. StarOffice - from one of the worst offenders with their SCSL monstrosity - will be under the SISSL/GPL. The world seems to be rapidly converging on a state where all free software is either 1) GPL'd, 2) GPL compatible (eg modified-BSD, X11...) or 3) GPL+SomeOtherLicense.

Why does this happen? Because a fork isn't in anyone's interests, in the long run. And I think (to get back on topic) the same thing applies to micro-distribution forks. Anyone who has produced a 7337 linux distribution for their own consumption is going to find very quickly that they can't keep up with repackaging every program as it comes out with new versions, the world is changing around them and none of the 3rd party rpms (or debs) work on their system any more. Oops. So they go back and find that - hey, Debian/Mandrake/RedHat/SuSe/Caldera isn't actually that bad after all! Maybe the problem they had with it has been fixed in the meantime. Maybe they learnt from their experience that it actually wasn't that good of an idea. Maybe they found Debian ;) Maybe they learnt enough to usefully contribute patches to their distribution and have them accepted.

I have to wonder how many of these hundreds of distributions are actually maintained - or even still used by their creators.

I have to wonder how many of these hundreds of distributions are actually maintained - or even still used by their creators.

Would it be break some rule trying to investigate this?

Try to contact the maintainer of some of the non-comercial distribution and ask them about how maintained their distribution is, if they use the distributions and what makes their distribution special.

If it is marked clearly that it is purly out of curiousity nobody could be offended by that?

I don't think the fork taboo applies to projects that are explicitly driven by commercial or other specific objectives. It only applies to projects that leave project direction open to influence by anyone who contributes.

hp, yours is a telling point in light of my own recent experience. paul ashton and myself started the NT Domains for Unix project back in august 97. part of that project involved Samba. unfortunately, the aims of that project clashed with the objectives and general ethos of the other primary samba developers.

your point is a telling one because it emphasises, to me, that the direction of the samba project is most definitely not "open to influence by anyone who contributes". it is only open to those people who defer to the remaining primary maintainers.

i considered a code split. i considered revoking all source code i have ever written from the GPL license. i considered offering an opportunity for the remaining primary maintainers to ... how do i put this ... be less restrictive. i chose the last of these options, and they failed to take it. in all probability, this is as much my failing and deliberate calculation as theirs.

so why am i mentioning this? as a lesson that i hope people, including myself, will and are learning from.

From the Debian recipe book: integrate all your packages to interoperate perfectly. Have a policy and follow it. Don't hype. Use apt-get.

If you want to have automated package upgrading a la apt-get, use something better than RPM. From the user's point of view, doing repeated upgrades with RPM is a PITA mainly due to configuration file management -- simply using .rpmsave files won't be enough. From the developer's point of view, sorting out all the file dependency, bad packaging habits and lack of pre- and post-configuration issues is a PITA. Just look at the myriad of apt-get wannabes that popped up recently -- and in this case, the fork taboo seems to be valid: we have lots of clones out there, and very few (if any) forks. This isn't random bashing: RPM is certanly well designed in many aspects, but support to auto-upgrading isn't one of them (hint, hint! to the RPM maintainers: add some switch to allow interactive configuration file updating, with use new, keep old, diff, shell escape ;))

(But it's not impossible, of course. A friend of a friend of mine has the apt-get working with RPM for a few weeks now, and it's almost at production quality.)

Also don't make /bin/sh depend on curses. Don't laugh! I've seen this happening.

ESR notwithstanding, there's no taboo on forks. Most people with good skills have serious exposure to the issues and know it's difficult to maintain a project. These people have the best chance of carrying a fork through. Their reluctance to fork is an important constant in the equation.

Others are intimidated by the name recognition the parent project has and avoid forking because they think people would ignore their fork. Others know they have no idea how to manage a project.

Still others go ahead and fork, many incompetently. Certainly LinuxOne at its very best represented a terminally incompetent fork of Red Hat. It died on the vine because of public ridicule, though for all I know they're still out there trying. And many projects don't fork because the number of people who would be interested in project management for that code tree is < 2.0. Or because the project manager of the original project is receptive to patches when the alternative is a fork. Or because the project manager was receptive to patches from the start.

Forking doesn't happen that much, but it's not because it can't. It's because there is a certain amount of stability to an existing project, for a variety of reasons. But forking remains an excellent threat to the reputation of participants in the forked project, which makes it one credible threat against insular project behavior.