Priority: Documentation, Subject: Patterns, Language: Perl, Status: Failure

Posted 15 Nov 2002 at 00:08 UTC by scrottie Share This

Projects I work on are universally doomed to failure. Every company I work for goes out of business (note: shortsell Motorola). Failure doesn't happen in the mode one would ever suspect, which is what makes this interesting. The code is always written in a timely manner and debugged. Perhaps I'm being premature, and I'm certainly venting, but I think its curious to see how a free documentation effort can fail, joining the ranks of commercial software development efforts and every other sort of project. Deaths of past projects and companies are a different story and one less relevant to Free Software development. This is about failing to write documentation.

Priority: Documentation, Subject: Patterns, Language: Perl, Status: Failure

The last half of the work always takes twice as long. Apply rule recursively. Being unemployed for 1.5 years in one of the worst job markets in the US, I've had a lot of time on my hands. For 0.5 years of that, my primary project has been "Perl Design Patterns". I value my time here on Earth above all else, but if I said it wasn't fraught with frustration I'd be lying.

"Perl Design Patterns" is free, online book, forum, and GNU Savanna project.

Another quasi-official mission rant is:

Exploring, and writing about Design Patterns as they apply to the Perl language. Yes, that's right, Perl. Everyone knows that Perl programmers lack discipline and bondage. Perl does offer many OO features, as long as you aren't looking for compile time checking. Don't get me wrong - Java is great. I highly recommend Java for large or collaborative development efforts. Java addresses most shortcomings of C++, and creates a consistent, likeable language. Perl won't go away, despite many people's sincere desires, so I'm attempting to retrofit the bondage beating stick with a carrot. If Design Patterns can exploit the Perl programmer's natural love of fun and neat tricks, we may be able to convert some of the unwashed masses - right? Having OO, lambdas, lexical scoping, tied data, operator overloading, symbol table manipulation, runtime inheritance tree manipulation, multiple inheritance, weak typing, and access to the bytecode tree certainly makes for adventures to be had. I apologize - I know that adventuring can be fun, rescuing people who got themselves up a creek isn't. Documenting useful interaction between these diverse ideas is uncharted territory, perhaps akin to a polar region, waiting to be explored.

  • Sourceforge and most of its ilk don't accept documentation projects. GNU Savanna does, and thank goodness. In this day and age, only the terminally bored take more than one hop off of a well known, established website.
  • Perl sites are too numerous and well rooted to be fragmented, and I don't want to fragment anyway.
  • Users wander in, manage to track down 3 or 4 of the pages that are placeholders while completely missing all of the content, then close the browser window. I've placed dire warnings on the main page to download the entire site as a single page, but this is universally unheeded.
  • My "friends" will give me feedback on code, point me at resources, pick me up from the side of the road if I hypothetically had a car and it hypothetically broke down. One thing people, even friends, will not do is read sample chapters and give you any sort of feedback. The net result is I have no idea how readable it is, how interesting, how coherent, and so on. Mind you, these self-same people frequently read books, but an unfinished book is somehow dirty, worse than any alpha code could ever be.
  • Publishers don't respond to proposals. Once again, GNU/FSF was the only exception, expressing interest, reaffirming that documentation is important, and asking to be kept up to date on my progress. Other free online Perl rags have been as incommunicative as major publishers.
  • Beat to the punch. Exchanging email with many people, asking permission to quote them or adapt something they've published online, I was eventually warned that a publisher was working with someone who had a name in the Perl community to publish something. This will be my deathblow.

    Sites like Slashdot, which make heavy use of aggregation and linking to tie sites together, are an interesting breed. Your typical subject-specific website only links off to another website when you pay them money, and then in the form of a short lived obnoxious banner, or in the form of something like Link Exchange or a web ring. Linking is seen as a quick way to loose traffic. Sourceforge is another example of a sort of online community that links heavily and readily and gains value itself because of it. Of the many Perl websites, most of them are of very high quality - Perl Monks,, O'Reilly's Perl Network, and others. None of them are in the business of linking off-site. Perl Monks might run an article doing so, but it is more interested in specific code fragments, rather than projects. Advogato, no excuse. Should have done that first, though I only recently recovered my lost password - doh. WikiWikiWeb, also known as the Portland Pattern Repository, is an interesting concept and a bandwagon I've eagerly jumped on, in which offsite links definitely do happen. Its readers have little to no interest in Perl patterns, as their readership is the Java/C++ crowd, but it generates a large number of hits to the page downloads of my own homebrewed version of Wiki.

    See? You mention software and people perk up. Running software is a zero-attention-cost thrill. It installs itself, does something amusing, and requires no time or attention on your part. As it turns out, documentation is like high priced software: people weigh the decision of which documentation to use very heavily in that it requires a serious investment in time and energy, both of which have value to the possessor.

    WikiWiki is an interesting beast. By nature, it is very simple. Simplicity translates to genericity. Anyone can edit anything, links happen automagically, and following links creates the page they refer to if it doesn't already exist. A documentation project done in WikiWiki could in theory take off on its own, as its readers began contributing to it.

    After 150 pages had been written, I started to send proposals to publishers. I'm not doing this project to get it done, to do a summary of other texts, or rough translation of a certain popular one - I'm doing it because I feel I have real experiences to share and some cool code, harvest from dozens of projects and countless Perl Mongers meetings, as well as research. That being said, it would be nice if even one of the dozens publishers would return an email in the number of weeks their website promised. Writing proposals for books is time consuming, taking about 4-6 hours per publisher. Each has a huge number of questions and their own custom template. Many of the question talk about how it would fit in with their lineup, compete against other publishers, and other questions whose answers cannot be directly reused from one to the next.

    When something is published, I fully expect it to be a regurgitation of the standard ilk that has been regurgitated so many times since the knock off of the original that started it. I fully expect it to fling my last six months of work into obscurity, as nearly universally people prefer a stolen copy of a poor commercial product over a quality free one. Somewhere, someone, in secret, under an NDA, is preparing a cathedral style monolith to be crowned as the definitive work. My many past failures hang heavily over my head. The long solitude isn't coming to an end. There is so much that I want to do, and I have so little evidence that anyone would ever care if I did. Feedback is nonexistent and the guest book is unsigned.

  • I've actually read this site..., posted 15 Nov 2002 at 01:17 UTC by mtearle » (Journeyer)

    ... and thought it was pretty neat. I'd just finished reading Design Patterns Explained (which is a good book BTW). Personally, I felt I didn't know enough about patterns and Perl OO silliness to contribute effectively.

    That said, if you want a proof reader for grammar at least, drop me an email.

    Blatant feedback, posted 15 Nov 2002 at 04:35 UTC by tk » (Observer)

    Some knee-jerk comments:

    • If people just read placeholder pages and then go away, it means there are way too many placeholder pages, and they're in just the right places (or the wrong places, depending on the point of view).
    • Besides, it's not too reasonable to expect 56Kbps dial-up users to download a multi-Mbyte book.
    • Perhaps the bulk of the problem lies in the introduction, which actually consists of several sections? My guess is that someone who will actually read a book titled "Design Patterns in Perl" will actually know enough about Perl and design patterns to already know most of what's covered in the introduction.
    • The main text also seems to waste too much time explaining simple things. For example, the "instance variable" idea is already covered in a different way, right inside the perltoot(1) man page.

    Apologies if the above sounds harsh...

    Feedback graciously stolen, posted 15 Nov 2002 at 15:47 UTC by scrottie » (Journeyer)

    mtearle, thanks for your comments. Re: feedback, don't worry about grammar - thats manual labor and my responsibility. Feedback on readability of the text and how interesting or uninteresting the subject is would be much more helpful. You don't seem to have an email address. You should do what tk did and provide an email address in the form of a Unix shell command ;-)

    tk, heck ya! "You're doing a few stupid things" is a lot better than "You can't write and its a stupid idea anyway". Like the old GI Joe cartoons say, "knowing is half the battle". I've incorporated your suggestions (thank you), but I'm still encouraging people to download rather than view online because I can't solve how to give a Wiki any sort of visible structure other than lining up everything up sequentially on one big page an lots of anchors. Perhaps some sort of distance metrics from root to compute probable "up", "left", "right" relationships... The download is only about a meg. My post was intentionally a bit minipulative and I apologize for that.

    Anyone else with the cahoonas to tell it like it is?


    As you said yourself..., posted 15 Nov 2002 at 16:06 UTC by glyph » (Master)

    "Perl programmers lack discipline [...]". You're writing a book for an audience that, based on their technology choices, explicitly doesn't care about what you're writing. There may be some weird intersection audience that cares both about maintainable code and about perl, but they're likely already using another language while they're waiting for Perl 6 to rescue them for the disaster of Perl 5 OO.

    Some of your more general points about documentation would be interesting, such as Sourceforge not accepting documentation projects, but only if they were true. Also that "in secret [...] a cathedral style monolith to be crowned as the definitive work.": actually it's been done already and it's called the Perl Cookbook.

    The utility of design patterns is mainly in that languages like Java and C++ are hobbled in particular ways, so it is not possible to implement such examples as a library. In higher-level languages, since those "patterns" can actually be implemented generally, it is important to discuss more advanced techniques, like recipies.

    As to your friends: when I've written documentation and asked for feedback, my peers have eagerly (and harshly ;-)) criticized it. Maybe you need better friends; you'd be lucky to meet any good ones with this style of advocacy. If you think what you're doing is interesting but nobody is noticing, don't start off with "so, about my most recent failure...".

    If you're thinking of starting a documentation project and you want it to be popular, you have to follow the same rules as any other project: make something that a lot of people will want. "Perl Design Patterns" is something that the target audience is likely to already understand. The perl standard library is exhaustively documented. What about "Perl for Financial Applications" or "Perl for Automating Embedded Development"? Choose something that's really a documentation weakness that people ask about a lot. Better yet, find a project written in Perl whose documentation is weak and add a manual for it. Then you can piggyback the "free thrill" effect of the software to get people interested in reading your stuff, and maybe get your name established as a writer.

    All the best books write themselves, posted 18 Nov 2002 at 17:26 UTC by jds » (Journeyer)

    You're working too hard. Relax. When the book is ready, it will come tumbling out so fluidly that you won't fret over whether it is readable or not: you'll be carried on the tide of readability.

    We don't need more books which are produced by people fretting over whether they're readable or not. We need more books written with such devoted passion that all the research is focused, all the writing is compelling, and all the readers are satisfied, want more, and help create more of the same, being inspired.

    You're right; grammar is a mechanical labor that can be fixed rather routinely. It's the passion you're looking for. If it ain't passionately composed, then why write it in the first place, except maybe you want the back-pat of 'having written a book,' which produces most of the worst books on the planet.

    Inspire me. Readability will work itself out.

    All the best books look <i>as if</i> they wrote themselves, posted 19 Nov 2002 at 14:15 UTC by dan » (Master)

    Usenet is a convincing demonstration that devoted passion is not sufficient by itself for writing a book that anyone else would want to read.

    How I see it ...., posted 19 Nov 2002 at 19:50 UTC by forrest » (Journeyer)

    I have to vehemently disagree with glyph about this statement:

    There may be some weird intersection audience that cares both about maintainable code and about perl [... but most perlers don't care ... ]

    Every serious programmer cares about maintainability. Just because TIMTOWTDI and there are a lot of examples of cute, clever WTDI on the 'net doesn't mean that perl programmers want to write code that way for real production purposes.

    I do think that the way GoF is cited as some kind of "bible" by adherents to the java religion is a turn-off to perlers. What you need to do is to present this as a "bag of tricks" for maintainable OO code. Perlers like bags of tricks!

    (BTW, I notice you're using my graphic. That's cool. Note my change of address.)

    New Advogato Features

    New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

    Keep up with the latest Advogato features by reading the Advogato status blog.

    If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

    Share this page