This week, Advogato gets up on his soapbox and dispenses advice to free software neophytes.
This week, Advogato gets up on his soapbox and dispenses advice to free software neophytes.
I'll take the risk of demonstrating my advanced age and give some advice for new free software developers. With luck, this post will get some discussion rolling. What are some of the worst mistakes you've made as a free software developer. What has worked well for you?
1. Learn real computer science
Traditionally, "real computer science" has been one of the great strengths of free software, I think largely because so many of its practitioners are computer science graduate students. Many software projects can be tackled with little or no knowledge of algorithms, etc., but there are a few reasons why learning computer science can be helpful.
Computer science is very much like a shared language binding the community together. As an example, I can tell my friends that my implementation of Cascading Style Sheets selectors is based on a lazy NFA to DFA construction, and most of them immediately know what I mean, even though it's a fairly complex and subtle concept.
Many areas of computer science show the way to avoiding mistakes. For example, the unreliability of the Internet's e-mail infrastructure is largely due to the fact that e-mail delivery isn't designed to follow transaction semantics. I see a lot of new network protocols being designed that make the same mistakes over and over again.
Even if you're never going to prove a program correct, it helps to know how, so you can design for correctness. A program written as if you were going to prove it correct has a lot better chance of working well than one hacked together.
Lastly, sometimes the best solution to a problem is to use a nontrivial algorithm or data structure. If you don't learn them, you're limiting your scope.
Of course, there's a lot more to software than computer science, much of which is in the exciting area of cyberbitchology, the study of why computers are such a pain to work with. Academic computer science barely (if at all) recognizes the existence of cyberbitchology, but there are a number of free software projects dedicated to improving things, including RPM and the autoconf suite.
2. Keep your sights on what you want
There are a lot of reasons to do free software. As Linus eloquently states, one of the most important is to have fun. For me, and many others I suspect, learning new stuff is key. Another thing that drives me is creating tools for artistic expression, the way I like them.
Perhaps, though, your main interest is to help Linux become the dominant operating system on the desktop, or to help the GNU project reach its goals.
Whatever your goal, the important thing is to stick to it. Obviously, your interests will determine your choice of project, and how you go about doing it.
A case in point is "instant messaging" clients (IRC and ICQ). It's often bemoaned that there are so many of these floating around, and that the energy put into them would be better invested somewhere else. This is certainly true if you look at Linux as a business. However, my guess is that the people writing instant messaging clients are doing it primarily as a learning experience. Indeed, the protocols are simple enough, and the results so immediate, that it's an excellent way to learn networking. There are probably thousands of parsers for toy languages as a result of compiler classes. Should we be worried about those too?
On the other hand, if your goal is to help Linux and the GNU project, a new instant messaging client is not a good choice. Friction can arise when you're dealing with people whose goals don't closely match your own.
A case in point is when Raph was working on Gzilla, his Gtk+ web browser. He got a lot of flak from people to merge the project with another one of the many free browsers under development. But he stuck to his guns, as he felt he had a different vision (namely to create a fast, simple, small browser) than most other people. Merging with a different project would have been difficult and probably not a lot of fun.
3. Chase the dream, not the competition
The motto of the Nemesis air racing team is "chase the dream, not the competition," and it's widely adopted by free software authors.
Staying with the Gzilla example, Raph more or less abandoned the project when Netscape announced that they'd be open sourcing their browser. There were other good reasons for laying the project down (the work that Raph is doing now on Gnome graphics is more fun, and more specific to his skills), but looking back, the announcement of Mozilla wasn't one of them. It's two years later, and Mozilla is just at the cusp of alpha. Two years ago, Gzilla was actually usable for browsing the web, if a bit spartan in features. If Raph had wanted to be lead developer of a popular browser, it seems likely he could have stuck with it.
Free software is usually way behind proprietary software in terms of polish and UI. However, much free software is in some important ways better. For example, TeX has had typographic features (such as whole-paragraph optimization) that are only now beginning to appear in proprietary page-layout programs. How did this happen? Certainly not because Knuth was trying to clone the (rather primitive) typesetting systems of his day. He was chasing the dream, not the competition.
4. Be a part of the community
One of the strongest features of free software is the community that has developed around it. The world of proprietary software is tied up in NDA's, and companies wary of competition. The open nature of free software allows a community, the Internet makes it happen, and many free software authors take full advantage of it. I'm fortunate to be a member of the Gnome project, which has a very strong community around it.
There are a few technical reasons why a community helps. For one, these days integration issues are generally more important than algorithms. They can only be addressed in a collaboration between a number of people. Further, some projects are simply too ambitious for a single person to take on.
However, I've found that the most gratifying aspect of the free software community has been the human interaction. I delight in being around people who share many of the same goals, passions, and values. I've also been surprised by how those in the Gnome community have come to care for one another as people.
One of the major goals of Advogato is to bring the free software community online. However, as exciting as computer-mediated communications are, they can't quite compare to face-to-face contact. Look up some of the free software developers near you, and make it to one of the good free software conferences if you possibly can.
5. Read lots of code
There is an incredible body of code out there, just waiting for you to read it. Hidden among this code are brilliant ideas, examples of virtuoso technique, and lots of hard-won practical knowledge about how to solve problems. There is an incredible diversity of programming styles. In some code, the personal quirks of the author shine through clearly. Other code is equally clearly the product of a collaboration, and you can see either the accumulation of cruft by committee, or a collaborative process to refine and polish the code. Some code is almost crystalline in its clarity, while some is cryptic to such an extreme that it can be an entertaining puzzle to figure out what the code does. By Sturgeon's Law, there's also a lot of really boring, tedious, and in many cases downright awful code out there.
Our training for musicians includes listening to lots of music, for writers reading lots of books, and for architects studying a lot of architecture. Yet, the standard computer science curriculum consists mostly of a steady diet of creating toy programs to satisfy homework assignments.
And of course by reading code of similar projects to your own, you'll see a number of different ways of approaching the same problems. Whether you decide to adapt the best one, take them all as a guide of what not to do, or just steal the best bits of each, it will help guide your own decisions.
6. Code for communication, not cleverness
With all the soft, touchy-feely advice given above, I thought I'd close with some technical advice.
Many programmers grow up learning how to create clever solutions for problems. While this can be fun, it's often bad news in a large, complex piece of software. Often, it's better to think in terms of communicating the logic of your code as effectively as possible. For example, if you're implementing a standard, it might be good to structure the code to follow the text of the standard, or at least illuminate the connections.
Nowhere is this more important than in optimization. After all, as Knuth says, "Premature optimization is the root of all evil in programming." It's not that optimization is unimportant - in fact, I consider most of today's computer systems to be appallingly slow, especially considering the speed of the hardware. But the most common sources of performance problems are memory consumption and having the stack of abstractions be too tall. Yes, do optimize inner loops that are shown to be performance bottlenecks by profiling, but only after you determine that crunching that loop is really necessary for the logic of the program. What's left is tiny fraction of the code, but it sure can be fun when it happens!
Gnome practice is to document the purpose and calling convention of each function, and I recommend following it generally. Comments inside functions should not be necessary - if they are, it might be a sign to simplify the function, perhaps by splitting it up.
Ok, these are a few things I think can be helpful to beginning free software developers. Feel free to share your experiences as well.
Well, I am a "free software neophyte" I suppose. I definitly agree about the computer science thing. I read about algorithms and whatnot a lot, but there's also the whole math part to it that I havn't yet had the chance to do anything with which probably holds me back in many ways. The "reading code" thing seems like a "well, duh" sort of statement, but not really. I'm in a C++ class now (don't get me started.... I rant enough about it in my diary entries) that is exactly like the one described here: teaching kids to write little annoying programs that do nothing (and with a god-awful style too, check here to see the style he wants us to use if you want to see what I mean. Unfortunatly he's pretty obstinate (as I believe most teachers are) not to change his ways, so in essence he is spoiling a learning experience for new programmers by teaching them a style that will get them fired or laughed out of any professional job of programming they have. Blah blah blah I'm done talking. Sorry, not good at ending comments which can sometimes drawn it on and on and.... well, nevermind.
Overall, I do agree with all the points made. However, point one is one that I never did follow. My background is in chemical engineering, something (far?) removed from the "pure CS" world. I don't muck around in the world of OS design, but in the world of polymers and statistical mechanics...but I think that I've managed to understand a lot and (hopefully) contributed a great deal to the free software community.
So, for the non-CS people out there who want to get started in this (all 5 of you ;) ), here's my helpful tips of advice...
1) History is a great teacher
This kind of goes back to the point of "read other people's code," but I like to take it a step further. One of the things I have found in practice, both in programming and when I was working as a chem. engr. for a major corporation, is that if you know WHY something was done the way it was, you will understand it better. You will also get the added insight of HOW to avoid problems as you continue your work.
2) Find a problem solving strategy you can live with, and STICK TO IT
In a nutshell, I find programming to be a way to solve a "problem," whether the problem be solving a reaction mechanism on a catalytic surface, to displaying moving pictures on an X display. I have a pretty set way to solve a problem (I state the problem, write down what I have to work with, what I need to solve, think about how to get from point A to B with the tools I have, execute, debug, etc.)
For larger problems, I break it down into subtasks, and repeate above schema. You might have a different method, and if it works WELL for you, stick with it.
If you DON'T have a methodology, you'll find working on larger projects difficult. You should always keep an idea of how the forest is layed out, instead of focusing on one tree specifically...or wandering aimlessly.
3) Learn to deal with people WELL...even people you don't want to deal with
I learned this one the hard way. I was working on a project to make a sound editor for Gnome. It flopped....HARDCORE. Why? I gave up on the project because of one specific person who just rubbed me the wrong way. Mine and his persona just clashed and clashed and clashed... I will admit I dealt with it in an unprofessional manner. In the end, it pretty much spelled the death of the project. Which leads to point 4...
4) If you head up a project, you better have some clear goals.
Point 4 ties in points 2 and 3. You better know where you're going, and what you want to do, or else the project will die, due to lack of work on it...or some nasty infighting about what should be done...
5) Have fun
If you don't enjoy working on something, you won't want to do it. Worse, if you want to LEARN something from it, and you're not having fun, not only will you not learn a thing, but you may be put off learning whatever it is you were trying to learn (be it GTK, C, whatnot).
6) Step out of the box
Sometimes, you will run up against a problem that you'll have NO IDEA how to solve. Don't beat yourself up trying to work out the problem yourself, look for help. Sometimes the help will come from someone who knows the problem. However, don't underestimate the solution-solving power of an English major or brick wall. I've been able to work out some of my hariest problems while shooting pool (ok, so blame me if I see the pool balls as hard-sphere molecules...)
7) Date an engineer
It's just a good idea in general... ;)
Good luck in your future endeavors.
Well, I was looking over book reviews at ACCU and then browsing over Advogato's Number and then taking a nap and then I thought:
It is good to read code. But there's a lot of code out there — probably more than I could read in a lifetime. Some of the code is likely better for reading than other. Some of it's hacked together by clueless knobs, some of it is substandard by the author's own admissions, and some of it is excessively opaque. Likewise, there are probably some brilliantly written quite lucid samples out there.
So where does one start?
It all depends what you want to do. Want an mp3 player? read mpg123
code. Want a browser? Read Mozilla code. If you're just getting into
programming, a couple good books you may want to look at are:
Practical C Programming
The Knuth book
MUCHO here but I liked that book when I was getting into it.
The Strousrup book (not sure of the title)
Much more here as well
For (my personal favorite) Perl:
Learning Perl (Never read it, heard it was good)
Programming Perl (Learned off this book, definitly a great thing)
The Perl Cookbook (definitly not for learning beginning Perl but good for a continuation)
Well, those are the books I reccomend. Also, decide what kind of thing you want to do, and look at the code for similar programs. It'll help.
New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.
Keep up with the latest Advogato features by reading the Advogato status blog.
If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!