Extreme Programming and Open Source Software

xp is just another overhyped set of best practice type things, preached as though it's the silver bullet everyone's looking for, flogged to death in as many books as they can possibly crank out on the subject while it's still a popular buzzword and plenty of people still want to get in on it first, or if not first, at least before plenty of other people do, until it's been milked for all it's worth, but by then it doesn't matter anymore, cuz they've made enough money to spend the rest of their days on a yacht.

oss? no thanks, i prefer the free software perspective.

Bazaar development (as opposed to Open Source, which is a superset; there are plenty of open source projects which do not employ bazaar development) is a natural for Extreme Programming. Remember that for all of the rules, principles and guidelines in XP, there's really only one rule: If something is worth doing, it's worth doing to extremes.

For example, KISS is good. So you should always "do the simplest thing that could possibly work". Design is good. So "refactor mercilessly". And so forth and so on.

Some of the principles designed for in-house development don't carry across well (e.g. pair programming), but XP allows you to drop some principles if they don't work in your situation. With some exceptions. For example, it's not extreme if you "do the simplest thing that could possibly work" and don't "refactor mercilessly". On the other hand there is no "forty hour week" on a volunteer project.

The project I'm currently working on, Mojo Nation has a number of principles taken from (or at least voted for by) Extreme Programming. Most notably we probably wouldn't be doing pass/fail unit tests if I hadn't toyed with JUnit a bit. Simple design, quick releases, and scheduling based on dates of release rather than features to be done, while not advocated by Extreme Programming alone, were all ideas I understood much better and believed in much more after reading Extreme Programming Explained.

That said, Extreme Programming isn't the methodology - each project must in practice be carried out in it's own slightly different way. The Extreme Programming crowd are rather pedantic about the way, but they do have some good ideas.

I advocate developing a consistent methodology by codifying what you wind up doing in practice. There is no need to invent something a priori. My experience has been that if an activity feels like a waste of time it probably is a waste of time.

XP has been chosen as software development methodology at my work place. Most the ideas seem familiar.

The one I still have to get used to is pair programming.

This is a tighter coupling of developers than the usual "n developers in a room together" scheme, while I would rather switch to a looser coupling over email and common web sites, the one used in open source development, if I could. :-)

Not only because I am rather short-sighted (in the sense of myoptic :-) but because I believe I can concentrate better on written stuff and can answer better in written form. And that communication 's inherent delay helps too.

Contrast that to two folks sitting in front of a box, both having to work at same speed, concentrated on the same issue.

Right now, in this early stage of experience, I am not convinced yet. Possibly due to the present shortage of man power too - I simply don't buy that two developer's combined work is always as good or better than two developers working seperately.

But possibly it might turn useful for certain situations. Who knows.

I've been looking at XP methodologies for a while (and I know several people who are very keen on the idea). Some ideas are definitely good -- I know I should write many more test cases than I currently do. But I'm worried that the `Do the simplest thing which works' aspect of it can be taken as an excuse for not having any design at all.

When I'm coding, I always try to make clear distinctions between interfaces and implementations (I guess that having Java as my current primary language influences this a bit). I don't have any problem with writing throw-away implementations when they are expedient, but when a new public interface is added, this is always something to be pondered (and debated with the rest of the development group, if it's a collaberative project). Implementations can (and should!) come and go, but I've always seen a good, stable, set of interfaces as the key to making that work.

As for pair programming, I guess it's got it's ups and downs. Extra eyes certainly find bugs -- I often `code swap' with office-mates to track down irritating problems. But I have trouble being tied to someone else for more than a few minutes at a time. I'm not a terribly focused coder -- unless I'm writing something that's algorithmically difficult, I'll quite often sit with an editor buffer on one half of the screen and a mail reader or web-browser on the other and flick between the two. Probably not the kind of practice which is likely to get incorporated into a formal coding methodology any time soon, but it works for me, and keeps my concentration much higher than if I were staring at the same block of code for hours on end.

Actually, I was just about to post on this (XP + open source). I'm glad other people are looking at it.

I think Extreme Programming is a great idea. This doesn't necessarily mean it fits in every situation (no idea does), but inside of it's problem domain, I think it's excellent. Perhaps the reason there's a lot of hype about it is simply because the conditions which XP tries to attack (small teams of programmers developing software where the requirements are loosely defined and/or change rapidly) is becoming more the mode (which I think is true of most small software companies) these days.

Practical software design (whereby you produce a functional requirements doc, design, and then implement) is pretty rigid, but in large software projects or even in small projects where there are clearly defined constraints and goals that don't change (perhaps a la JPL), that rigidity can be good, so XP isn't for everyone. I think the XP chorus is accepting of that and has no problem with it -- use what works.

Also, XP grew out of highly object-oriented software development projects and the use of certain languages/environments in those projects (Smalltalk, mostly), so while I certainly wouldn't say that XP doesn't work for any non-OO language, I would say that the use of a complete OO language makes XP much easier (I can't imagine XP on assembly code being necessarily all that more useful over regular design proceedures, for instance). By the same token, what I find interesting is that a number of XP practices are things I've been doing all along in functional programming (I'm a big Lisp person, and a functional programming adovcate in general). Regular design proceedures tend to constrain people who use functional langauges, since the natural evolution of software in functional languages is (usually) much different than that of proceedural languges, which I especially notice when doing things in Common Lisp (I like to take advantage of things like macros, the MOP, and full reflective-ness).

I'm a big fan of XP, but I'm not convinced that conventional XP will work in the normal distributed OSS project.

Firstly, XP relies on always doing the simplest thing to get something working, and continual refactoring of other components. These two habits seem to be at odds with the idea of a distributed group of people working of separate modules of code. This is tied into the XP idea of group ownership of code - no one is responsible for a single part of code. In most OSS projects, it is the total opposite - a single person is responsible for driving the development of their module.

The Pair Programming part of XP obviously does not occur in a distributed OSS environment, but in some cases this if offset by the many eyeballs factor - while you may program alone, many people look at your work and find bugs. This depends on the size of your development community, though, of course.

The XP emphasis on testing is a good thing, and some OSS projects do similar things already.

I think where most OSS programmers would have a problem with XP is the do the simplest thing philosophy. That is totally against the huge plans most OSS projects make (the BDUF - Big Design Up Front - philosophy).

This difference in philosopy may be caused by the areas that XP and OSS specialise in. Most XP proponants write business applications for non-software companies (ie, in a corporate environment). This means they have control over the release procedure, so they can put out incompaitible releases - they know there will never be any old versions around. In the OSS environement (with no real control over who uses which release), compatibility is always important, so it is sometimes better to put stuff in at the start of the project, then fix it later - that way there won't be so many incompatibilities - just bugs and fixes.

I think we could use one twist on the Pair Programming concept, something like Serial Pair Programming as opposed to Parallel Pair Programming. There are many quality foundations of free software available where the original developers would be glad to mentor new programmers but no longer have the time to proactively extend and maintain what they started. Not every creator of open source can indefinitely continue to work on a project at full tilt . But any quality piece of software needs constant maintenance and extension to remain viable to its users.

So we see many older software projects wither as the life circumstances of the original developers change, and amazingly similar yet disconnected new projects get started where an enormous effort is required to replicate the work of others. With Serial Pair Programming, the knowledge-rich time-poor original programmers could assist the time-rich apprentices as they get up to speed. Theory would say that these collaborations would take us all a lot farther than the preponderance of born-yesterday vanity projects you see at places like sourceforge.

I have strong caveats to what I just wrote. I believe in diversity and freedom as much as I believe in continuity and continual evolution. If you think you have a better idea, or you just want to do your own thing, please do. But if you believe you might have something to learn from someone else's software or programming experience, why not find a source package you like and apprentice yourself?

Considering the first time I saw the book eXtreme programming the first thing that popped into my head was someone with a laptop base jumping while hacking Perl, this has opened me up a little bit to what it really is. While it does seem a little, um, buzz-wordy, the ideals you posted here sound nice. I admit, I will miss the chance to be able to hang out with my friends in a coding session and leap up at random times shouting EXTREME!!! WHOO!! now that I know what it is.

A lot of people react badly to "Do The Simplest Thing That Works" because of a gut feeling on the importance of design. Well, so do your design. What XP tells you is to not waste lots of hours in design trying to predict the future, because you WILL be wrong. Design for today's needs, don't add a lot of hooks for tomorrow. Then when you need to change the interface tomorrow, if there is already third-party software using this interface, refactor in a backwards-compatible way.

Granted, a lot of XP advocates will tell you not to design at all. But most of the serious ones do some design. In special, I'd point you to Fowler's work on XP, and some of the stuff by Cockburn.

Extreme Programming reminds me of the "Infinite Monkeys" idea -- i.e., that if you put an infinite number of monkeys with an infinite number of typewriters into a room, eventually they will produce the collected works of Shakespeare. Each monkey does the simplest thing -- bang on a typewriter key -- and somehow all these simple events will lead up to the greatest work of literature the world has ever known.

The only problem is, statistically, this wouldn't happen before the heat-death of the universe. I suspect the same is true of XP.

Yet another ill-conceived fad meant to line the pockets of IT consultants. God save us all!

Mats Henricson pointed out that Refactoring also has a website:

www.refactoring.com

Thanks Mats.

for those that have been reading my diary here, you'll know that i have a new Free software project starting soon that will be sold commercially as well. it also happens that i recently read the first XP book, and found it really interesting - if somewhat lacking in the "how-to" area. last week i stumbled across the new, unpublished XP book on the web. it is available here (ftp) or here (http). it is an excellent read, even if some of the parts of the book are in their final draft - so good in fact that we are going to use XP for our project to see what it is like in practise. each chapter focuses on a different practical aspect of how to implement XP.

even though no development methodology is perfect, i think XP will be able help us create a better product in a shorter amount of time.

lilo, mrorganic: A lot of people misunderstand the "do the simplest thing that could possibly work" maxim. It doesn't work unless you also "refactor mercilessly". The idea is that the design of your software evolves. You mutate it to add a new feature or fix a bug, then you refactor to improve the design of what you have. And you do it mercilessly. The point is not to avoid design. Far from it. Design is a great thing! So in XP you do it to extremes: you design ALL the time.

BTW, the subtitle of the refactoring book is "Improving the design of existing software".

Both new XP books are now out:

Extreme Programming Installed - Ron Jeffries, Ann Anderson, Chet Hendrickson

Planning Extreme Programming - Kent Beck and Martin Fowler

(Addison-Wesley)

Check out Martin Fowler's keynote from the XP2000 conference here. The speech is entitled "is design dead?", and is quite a good read.

Saw one .signature once that had a nice answer to the Infinity Monkey scenario:
"The internet proves otherwise" :-)

So I like to think I was doing a number if not all of these things before I had heard of XP. I disagree with the comments made above about design, however.

Simple design does not mean no design. Simple designs don't preclude creating complex systems; in fact, they facilitate them. Things that are not designed well end up having interfaces that either not well suited to the problem at hand or are unstable because they are continuously changing since the requirements were not managed. Trying to put things together that don't have interfaces that work together is hard.

Someone--I forget whom--said that UNIX pipes were the only component model that ever worked. I think the underlying statement is that interfaces that are more complicated than UNIX files (forget ioctl and sockets for now; I think these are things that were hacked on later that don't fit the original model, and are a symptom of the problem mentioned above) are too complicated to be successful. I think this is a bit of an exaggeration, but it is not orders of magnitude off.

There is also a Wiki page on this subject: CombiningOpenSourceAndXp