A hard problem worth solving

Posted 16 Jul 2008 at 06:01 UTC by robla Share This

There's an ongoing debate about whether a free/open source project needs to be "organic" to be worthwhile, where "organic" is (arguably) defined as a project which the first release included source, and is generally characterized as by a distributed development team with no single company truly in control, and "inorganic" is generally code that started off life as a proprietary effort. I'd like to argue that making "inorganic" open source work is a big challenge worth tackling.

Now that Firefox is such a big success, it's easy to forget the long years that the Mozilla folks spent building the community and figuring out how to transition from proprietary to open development. Jamie Zawinski was the initial leader of this effort, and burned out after the first year. He wrote a scathing essay upon his resignation from AOL/Netscape. In that essay, he notes: "There exist counterexamples to this, but in general, great things are accomplished by small groups of people who are driven, who have unity of purpose. The more people involved, the slower and stupider their union is." His assessment is brutally frank (probably too brutal), but any of us who do a similar job would be lying if we said we had no idea what he's talking about.

Brian Aker pointed out in a blog discussion with Theodore Ts'o that there's not really a successful blueprint for taking projects open source. There are at least some "do's" and "don'ts", and an ability to analyze the many projects that have followed in the decade following the initial Mozilla release. But no one has had anything approaching a home run when it comes to both community and commercial success when opening up proprietary software. Some base hits, sacrifice flies, and strikeouts, but no home runs that I can think of.

Corporate-originated open source (arguably "inorganic" software) has a much, much more difficult time keeping the core team aligned around open principles, because there's a much weaker selection bias keeping out open source skeptics in the same way that organic projects generally keep them out. Participating in "organic" open source projects is often more appealing, since they generally start from day one with public tools for everything. It's much more comfortable as a community member to have the full visibility of a peer from the first day you take an interest in a project, even if you probably won't be treated as one until you really prove your mettle.

There are many company-driven open source projects that drag their feet at getting that level of transparency. Many potential participants in these fledgling communities rightfully get frustrated, because they imagine how much more they could potentially help with a peer's level access to the tools and information. They are probably correct in thinking that companies that don't do it as quickly as possible are squandering a big opportunity, and get tired of trying to hit companies upside the head with the cluebat. They retreat back to the organic alternatives, because, well, they just aren't getting paid enough for that....stuff.

I think ushering companies into the open source development model is a hard but extremely worthwhile problem to solve. That's why I took the job I have today. I have a pretty unique situation at Linden Lab. When we launched, I was helping a company open the source code for a very commercially successful product from a market-leading position (rather than as a scrappy underdog or a has-been). It's easier than trying to do it while trying to start or revive a business, but presents the different problem that I was (and still am) worried about killing the goose that laid the golden egg. We've taken a gradual approach, in part based on my recommendation, and in part because the development cycles just weren't there for anything more radical than what we did. I'm not going to argue that I've done the best job at this (in fact, I know I've made several textbook errors), or that Linden Lab has been without fault. But I think we've done pretty well with our approach, and as of this writing, our business is still doing very well, and we're getting a lot of code contributions and bug reports.

While that gradual approach has been good for our business, it has frustrated many people in our community, and it's clear we're not the only company in that position. It's really difficult when you have to untangle processes and habits that have formed over years. Regardless of whether or not those are the best practices, they are the current practices, and the difficulty of process change is often underestimated by those with the best of intentions. Compound that with the fact that the open source culture is a very email-centric (or at least "text-centric") culture, and how well that works for actually gaging goodwill, and you have a situation where too many people draw their cluebats too soon.

The success of Firefox shows us why it's worthwhile to persevere with corporate-origin open source. We can kid ourselves and say that that was a fluke, and that the rest of the software we need is going to come from building it all from scratch. We can read too much into Eric Raymond's statement that 90% of professionally-created software is written by the same companies that use the software, and not for resale. Raymond's 90% is the long tail, and that general purpose software is still largely written for resale, and most often, isn't open (yet).

We've got an uphill battle to fight to get companies writing proprietary software to instead direct those resources to writing free software. Savio Rodriguez points out:

IDC estimates OSS software revenue growing from $1.8B in 2006 to $5.8B in 2011. [...] Now take that $5.8B as a percentage of the IDC estimated total software market and you get a grand total of 1.8%. [...] This is the future; 1.8% won’t convince vendors to abandon the business model driving the other 98.2% of market spending on software. [..] It’s completely possible that IDC estimates will be proven wildly wrong. Maybe the actual number will be 4.9% :-)

The point I'm making here is that there's going to be a lot of vendors chasing the (projected) 95-98% of the software market associated with proprietary software. That's where the investment is going to be for the foreseeable future. I have a hard time believing that all of that software is junk that we shouldn't need to care about. Let's take the best case scenario and say there's a sea change coming. That means there's going to be a lot of formerly proprietary software (and associated software professionals) that we're going to need to get better at ushering into the broader community. It's not merely about teaching those companies "how things are done", but probably changing those norms in obvious and not-so-obvious ways.

We have a hard problem to solve, but I think it's worth solving. I'm anxious to have much more software that doesn't suck.

This is a slightly edited version of an article posted on my personal blog

WebKit, posted 16 Jul 2008 at 12:15 UTC by chalst » (Master)

Excellent article. Another related phenomena is coping with semi-proprietary adoption of free software, the most high profile example being Apple's adoption of KDE's khtml code, which they used as the basis for the highly successful WebKit framework. It's fair to describe this story as being marked by friction; without getting into the ins and outs, better social norms for dealing with such situations might increase the number of commercial software shops willing to invest in developing free software.

[OT] - 3D desktop integration, posted 18 Jul 2008 at 22:22 UTC by lkcl » (Master)

ah! excellent! i was absolutely delighted to see 2ndlife being free software - i wondered whodunnit :) now for a biggie: have you read snowcrash? of course you have. ng, the bithead, is overkill, but i'd settle for something where, using XComposite, i could actually view someone else's document _in_ second life... because i was invited into their 2ndlife "office" and that "automatically" granted me the right to collaborate and/or view their documents...

... which come up in _my_ 3D 2nd life rendition, using openoffice / XComposite...

it's not so hard to do (he said... :)

free software: a more honest approach, posted 19 Jul 2008 at 11:14 UTC by lkcl » (Master)

what is the key difference between free and proprietary software?

is it that the average person can modify and enhance the software?

absolutely not.

the average person doesn't give a flying xxxx about the source code. the average person cannot program. the average person doesn't care about "freedom of source code" because it has absolutely zero value to them: it's an alien language, well beyond their capabilities.

what's much more important to them is "zero cost". even oprah winfrey is promoting skype as wonderful technology to connect people across the world, because "it's free".

our laudable lofty aims of "opening up the source code" are utterly meaningless to the "average person" - and, the bigger the source code base, the less "meaningful" the freedom of the source code is to even EXPERIENCED programmers.

is it that the process underpinning free software is much more honest?

absolutely damn right, it is. proprietary software has users "locked in". if the company dies, so does the software (except cases like blender).

the key fundamental thing is that the business model for proprietary software is typically "boxed product" whereas free software is a "service model".

free software is like the "guilds" - crafts where a guildsman's skills were recognised, valued and appreciated. you didn't BUY the guildsman - you bought his SERVICES.

proprietary software development is based on "slavery". slavery of intelligence (even in the name "intellectual property" is the up-front in-your-face god-given-right to OWN intelligence).

a true transition from proprietary to non-proprietary software development is therefore very, very profound.

by "true" i don't mean "MySQL", and i don't mean "TrollTech". the purpose of MySQL and Trolltech's "open-ness" was to "be bought". consequently, MySQL has been sold several times and TrollTech just the once (so far). the "intellectual property" behind MySQL and TrollTech is ENTIRELY owned by those companies, and is released as a "sop" to the free software community as a "one-way push" rather than a community engagement effort. "here. have the crumbs from under our table, because we're better than you: we make money. we owwwwn our code. now get lost".

in moving to a community-based TRUE free software effort, you're declaring absolutely up-front that your business model is truly a service-based one, rather than the dishonest "put it in a box, make sure it breaks so that people will come back and buy another one" model.

you are declaring "we are so confident that our business model is service based that look - we can even release the source code of the software for free, so that anyone can use it and help out with it".

is it fantastic that linden labs have released 2nd life as free software?

absolutely damn right it is. i'm dead impressed.

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

Share this page