A hard problem worth solving

There's an ongoing debate about whether a free/open source project needs to be "organic" to be worthwhile, where "organic" is (arguably) defined as a project which the first release included source, and is generally characterized as by a distributed development team with no single company truly in control, and "inorganic" is generally code that started off life as a proprietary effort. I'd like to argue that making "inorganic" open source work is a big challenge worth tackling.

Now that Firefox is such a big success, it's easy to forget the long years that the Mozilla folks spent building the community and figuring out how to transition from proprietary to open development. Jamie Zawinski was the initial leader of this effort, and burned out after the first year. He wrote a scathing essay upon his resignation from AOL/Netscape. In that essay, he notes: "There exist counterexamples to this, but in general, great things are accomplished by small groups of people who are driven, who have unity of purpose. The more people involved, the slower and stupider their union is." His assessment is brutally frank (probably too brutal), but any of us who do a similar job would be lying if we said we had no idea what he's talking about.

Brian Aker pointed out in a blog discussion with Theodore Ts'o that there's not really a successful blueprint for taking projects open source. There are at least some "do's" and "don'ts", and an ability to analyze the many projects that have followed in the decade following the initial Mozilla release. But no one has had anything approaching a home run when it comes to both community and commercial success when opening up proprietary software. Some base hits, sacrifice flies, and strikeouts, but no home runs that I can think of.

Corporate-originated open source (arguably "inorganic" software) has a much, much more difficult time keeping the core team aligned around open principles, because there's a much weaker selection bias keeping out open source skeptics in the same way that organic projects generally keep them out. Participating in "organic" open source projects is often more appealing, since they generally start from day one with public tools for everything. It's much more comfortable as a community member to have the full visibility of a peer from the first day you take an interest in a project, even if you probably won't be treated as one until you really prove your mettle.

There are many company-driven open source projects that drag their feet at getting that level of transparency. Many potential participants in these fledgling communities rightfully get frustrated, because they imagine how much more they could potentially help with a peer's level access to the tools and information. They are probably correct in thinking that companies that don't do it as quickly as possible are squandering a big opportunity, and get tired of trying to hit companies upside the head with the cluebat. They retreat back to the organic alternatives, because, well, they just aren't getting paid enough for that....stuff.

I think ushering companies into the open source development model is a hard but extremely worthwhile problem to solve. That's why I took the job I have today. I have a pretty unique situation at Linden Lab. When we launched, I was helping a company open the source code for a very commercially successful product from a market-leading position (rather than as a scrappy underdog or a has-been). It's easier than trying to do it while trying to start or revive a business, but presents the different problem that I was (and still am) worried about killing the goose that laid the golden egg. We've taken a gradual approach, in part based on my recommendation, and in part because the development cycles just weren't there for anything more radical than what we did. I'm not going to argue that I've done the best job at this (in fact, I know I've made several textbook errors), or that Linden Lab has been without fault. But I think we've done pretty well with our approach, and as of this writing, our business is still doing very well, and we're getting a lot of code contributions and bug reports.

While that gradual approach has been good for our business, it has frustrated many people in our community, and it's clear we're not the only company in that position. It's really difficult when you have to untangle processes and habits that have formed over years. Regardless of whether or not those are the best practices, they are the current practices, and the difficulty of process change is often underestimated by those with the best of intentions. Compound that with the fact that the open source culture is a very email-centric (or at least "text-centric") culture, and how well that works for actually gaging goodwill, and you have a situation where too many people draw their cluebats too soon.

The success of Firefox shows us why it's worthwhile to persevere with corporate-origin open source. We can kid ourselves and say that that was a fluke, and that the rest of the software we need is going to come from building it all from scratch. We can read too much into Eric Raymond's statement that 90% of professionally-created software is written by the same companies that use the software, and not for resale. Raymond's 90% is the long tail, and that general purpose software is still largely written for resale, and most often, isn't open (yet).

We've got an uphill battle to fight to get companies writing proprietary software to instead direct those resources to writing free software. Savio Rodriguez points out:

IDC estimates OSS software revenue growing from $1.8B in 2006 to $5.8B in 2011. [...] Now take that $5.8B as a percentage of the IDC estimated total software market and you get a grand total of 1.8%. [...] This is the future; 1.8% won’t convince vendors to abandon the business model driving the other 98.2% of market spending on software. [..] It’s completely possible that IDC estimates will be proven wildly wrong. Maybe the actual number will be 4.9% :-)

The point I'm making here is that there's going to be a lot of vendors chasing the (projected) 95-98% of the software market associated with proprietary software. That's where the investment is going to be for the foreseeable future. I have a hard time believing that all of that software is junk that we shouldn't need to care about. Let's take the best case scenario and say there's a sea change coming. That means there's going to be a lot of formerly proprietary software (and associated software professionals) that we're going to need to get better at ushering into the broader community. It's not merely about teaching those companies "how things are done", but probably changing those norms in obvious and not-so-obvious ways.

We have a hard problem to solve, but I think it's worth solving. I'm anxious to have much more software that doesn't suck.

This is a slightly edited version of an article posted on my personal blog