I think Advogato is (now was, alas) a Really Cool Thing.
I always wondered, was the source code behind this website freely available?
Hopefully it will be. Or, if not, passed on to someone else.
Long live Advogato.
I think Advogato is (now was, alas) a Really Cool Thing.
I always wondered, was the source code behind this website freely available?
Hopefully it will be. Or, if not, passed on to someone else.
Long live Advogato.
Never in a million, jillion years will I understand software engineers.
Everyday I struggle with this archaic, 20 year old, inefficient, bothersome, idiotic, so-called Menu System that Windows -- and Linux -- uses.
That horrible, terrible, rotten, "pull-down, scroll-by-mouse" menu is so bad.
Here is what I mean. You can check for yourself, right now. Go on and click your web-browser's "Bookmarks" menu (or "Favorites" if you go IE). ("Oooo, shiny ones!" *)
A list of "links" comes over the browser's web-page -- this web-page -- and some "links" bring up side lists of more "links" which can include more side lists of "links" which...
Yeah, so? you say. Well, perhaps it is me. But guess what a hand and rotating pointing device likes to do?
It likes to go in circles doesn't it? Circles and angles; angles about, what, 45 or 135 degrees? Yeah. Circles and angles. But maybe it's just me.
I click on "Bookmarks" and I get a pull-down list. I click on a link that brings up a side list of more links, and guess what I am forced to do in order to make further clicking work? That's right. I have to move the pointing device exactly 90 degrees to the right (and sometimes left). Exactly 90 degrees.
Okay, not exactly if and only if I move the pointing device really, really fast over into the sub list. But I keep forgetting to move fast enough, or I move too far, or too shallow, or I move too slow and then it acts as if I paused and if that happens of another sub-link another side menu pops up. Menus come and go like mad, appearing here, there, I go to click, but another menu takes over, and I go back and miss again... ARRRRRRRRRRGGGGHHHHHH!!!!!!!
I'm sorry, but I feel that I have to swear. Crap! (Well, I ain't much of a curser.)
Adobe, as an example of the worst offender, has the most horrible menu structure ever designed. Ever. It is so bad, I waste hours a day. Hours, just messing around trying to get it right.
Oh, there's keyboard short cuts, you say. You mean, like "Control-P"? "Alt-F-S"? "Control-Alt-Shift-S"? If only my memory were not so bad.
Here is a simple fix
Get rid of the "pause" to open a menu or sub-menu. Make them appear only by clicking. So, I click, the list comes down, I click again and a sub-list comes out -- and here's the fix part -- stays out until I click again! Never goes away until I click again! Problem solved!
But, perhaps it's just me.
* This reference, "Oooo, shiny ones!", is from a character in the PC Video Game, Baldur's Gate. I love that game, and not just for it's D&D game play. I love that game for it's INTERFACE! With one exception, it's interface is intuitive and painless for me to use. Fuck I wish software engineers would actually treat the high-resolution graphics capable Operating Environments computers in the 21st Century has as high-resolution graphics capable Operating Environments!
I manage a small IT department, and part of the work involves helping users deal with SPAM. Since we have hosted e-mail service that uses OpenWebMail, I basically just enable Blocklist Filtering and SpamNabber, which gets most, but not all, SPAM out of the user's way. But also, those avenues sometimes mark legitimate e-mail as SPAM.
So, to check for legitimate e-mail being directed as SPAM, I periodically manually review SPAM flagged e-mail. It is easy enough to see the legitimates among the SPAM in a list showing From and Subject. Occasionally I have to actually read the e-mail.
And I have an idea of a service that would submit e-mail addresses to a database of existing Opt-out services.
What started me out on this was the influx of many SPAMs referring to similar "targeted dating services." Odd, I thought, until I noticed that they were all from a related UBE-er, http://www.revenueuplink.com/. These UBE SCUM SPAM on other's behalf. In this case, several dating services.
Interestingly, following the obvious Opt-out link in the SPAM led to being removed from the third party's services and not the SCUM UBE's services. I had to do a little extra work through a small maze of links at http://media-uplink.com/ and http://cool-uplink.com/ before finding http://www.revenueuplink.com/.
Also, these Revenue Uplink scum are scum because they harvested the e-mail address they spammed.
Anyway, I finally got to the Opt-out link, and did so, and was driven to ads in an IFRAME along the way too! Most of these UBE SCUM actually do supply a Website form to submit and address to Opt-out.
For grins, that day I followed several more Opt-out processes. About half of them provide a small maze of re-directing passages all different too.
The Proposal For a Global UBE Opt-Out Database
I would like to offer a service that would compile a list of UBE-ers that support Opting-out. It would just be a list of those UBE-ers to whom I can submit an e-mail address to. It would have a database of the UBE-er and how to submit an e-mail address for removal.
For example, a person would go to http://www.optmeout.com/ (not a real Website) and then submit their e-mail address. The website would then use it's database of UBE-ers' with an Opt-out mechanism, and then submit the e-mail address to each and every one of those UBE-ers.
Boom. That's it.
This Almost Exists
I am looking around for something like this, but found nothing yet, only references to "How to Opt-out" instructions.
There is the FTC's "How To" (http://www.ftc.gov/bcp/conline/pubs/alerts/optoutalrt.htm) and there is a website that sort of look like it may be of help but is poorly designed at Center for Democracy and Technology (http://opt-out.cdt.org/).
And Spamhaus has a reference to Opt-out (http://www.spamhaus.org/mailinglists.html).
And there is a paper mailing "How To" from Privacy Rights Clearinghouse (http://www.privacyrights.org/fs/fs24a-OptOutAddresses.htm).
But I have not seen anything that would opt-out for me.
[the rest of this entry is just informational]
SPAM generally comes in two flavours, outright fraud and UCE. Fraudulent SPAM tries real hard to hide its source, uses misspelled words, stupid subjects, absurd e-mail addresses, sometimes has no body except for embedded images, etc. Fraudulent SPAM never has "remove" or "unsubscribe" references. Most of these are scams and sellers of fraudulent services and/or products, or just promote some obscure stock.
UBE is unusually not about fraudulent services or products. About half of UBE are by companies that just think sending UBE is okay. The rest are UBE services hired by companies to advertise on their behalf. These latter are basically defrauding their clients by pretending to send to Opt-in databases. Almost all of these UBEs have "remove" or "unsubscribe" references.
About From: Fields
I have noticed a few things about the From header. E-mail with a From field that has a name that is not reflected in the address is always suspicious, such as:
"Andres Hardin" <email@example.com>
As long as newsletters and Opt-in advertisers use a properly formatted From address, I can spot 99% of SPAM vs. legitimate e-mail from the From fields alone.
Here are some examples of properly formatted From fields:
DSL from Yahoo! <firstname.lastname@example.org>
Staples Rewards <email@example.com>
"BizQuest Newsletter" <firstname.lastname@example.org>
But, then some fraudulent SPAM uses okay looking addresses, like:
"Daisy Haas" <DaisyHaas@mail.ru>
And then there are idiot e-mail hosting companies such as AOL, Comcast and Yahoo, that do not enforce proper address formats and allow simply email@example.com. They could at least enforce an addr-spec address enclosed in angle brackets!
This entry attempts to document a programming paradigm I call Anal Programming (AP). AP represents nothing "new" in the way of programming design or theory, introduces no new buzzwords or acronyms (beyond AP), uses no fancy procedures or documentation, nor requires any adherence to any particular "style". AP is simply writing code in as straight forward a manner as possible with an emphasis on the combination of three universal aspects of programming:
The first thing to note about AP is that the three aspects must be taken (and used) together, in that code that is efficient but not comprehensible is not Anal. Code that is very efficient and extremely comprehensible would be an abject failure if it is not also versatile (as these terms are defined within the aspect of Anal Programming).
The Definitions of the three "Universalities"
Efficiency -- in the case of AP -- is not to be confused with minimumality, i.e. it is not simply least number of machine cycles, characters or number of lines. Efficiency, here, is all encompassing -- which is why, but not solely, it comes first in the list. Efficiency is the sum of the code taken in "block" size pieces and is a measure of code quality in the context of versatility and comprehensability. Efficiency is recursive but does not require recursion.
Versatility is a measure of how useful the code is as well as how easy it is to use the code and as well how easy it is for the code to be used. This last aspect -- "for the code to be used" -- is not a trite euphemism, but a requirement of Anal Programming. This triad of use -- Usefulness, Usage and Use -- will require more definition. Usefulness is self-explanatory; code that does something measurably useful to you. Usage is a measure of access, installation, configuration, modification, editing, and requirements. Use is the interface, the API, or even just the function list; think of this use as the programmer's use of the code. In larger abstract terms, one can think of PERL and PHP as Versatile programming languages in this regard as they generally meet all three elements of these definitions of "use". (PERL: "There is more than one way to do it." AP: "There should be enough ways to do it.")
Comprehension is last in the list because one might almost think of the formula Efficient + Versatile = Comprehsible. However, if one spends all his/her time and effort on efficiency (as the least amount of characters and expressions and blocks) and versatility (as complex classes, function overloading, etc) one can easy produce non-comprehensible code. In fact, it is all to easy to produce non-comprehensible code. Ideal comprehension means understanding at a glance. Though it can be difficult to achieve, it is hardly impossible. By working with E + V = C in mind one will find that producing Comprehension is achievable when Efficiency is combined with Versatility.
Where Anal Programming Fits with Regard to other Programming Paradigms
Sometimes someone needs to dash off some code to "scratch and itch". In that case, that someone should do what ever he or she wants to do to accomplish the goal. One would not use AP just to fix something.
Sometimes someone needs to adhere to an existing paradigm -- a driver, an extension of a class or library, and extension of a language, an add-on (or add-in) to an existing program. In all those cases one would not use AP.
AP is a discipline. AP is a discipline based upon practicality (and perhaps even pragmatism).
One would not use AP with OOP.
UML is the antithesis of AP.
The Drawbacks of Anal Programming
Before I get into actual coding and programming practices I must mention that achieving all three of the aspects defined above comes at a price: a lot of work. One must be willing, for AP requires it, to constantly re-think, re-examine and re-write his or her code. In fact, it might be that early purveyors of AP will be re-writing most of the time as old programming practices will get in the way (more on this later in "Actual AP Coding and Programming Practices").
Within the AP paradigm one would not hesitate to completely re-implement something -- the format of a configuration file for example -- right in the middle of a project, if in doing so would decrease complexity and increase either efficiency or versatility. One would do so even if the reduction would be just one global variable.**
Within the AP paradigm one would not simply choose one database (unless one had to for reasons mentioned in "Where Anal Programming Fits with Regard to other Programming Paradigms"), but would code so that ANY database could be used.***
Simply put, AP is a lot of work.
Actual AP Coding and Programming Practices
Before starting out on a road to Anal Programming one would have to throw away, forget, vanquish, utterly annihilate all high-falutin concepts such as "Reusability" and "Oriented" and "Elements" and "Patterns". One would never write "anchored exception declarations to solve problems with checked exceptions and reusable software elements" within the AP paradigm. One must remove one's "pre-programming" when implementing as AP. There is only one thing of utmost importance in AP - efficient, versatile and comprehensible code.
Here is a small example of an AP program. PHP Form Handler. (It is, as most AP projects, not finished and parts will be re-implemented at a later date.)
[END PART ONE -- MORE HERE LATER -- NOT FINISHED]
* In this preliminary post I leave out the critical examination of the example. A critical examination and analysis will, of course, be required. Perhaps I can finish this essay with a few months.
** I will concede already that AP would never be adopted in a corporate setting.
*** Within the AP paradigm one would not hesitate to have an "encapsulation of an encapsulation" to achieve this.
In this entry I outline a new Web technology. It may already exist. I just have not seen it done.
It is my dislike of poorly designed technology that drives me to write this rant.
I dislike virtual brickwalls. Like trying to fix or to understand a programming problem only to find that access to the people most likely able to help you are behind technological barriers.
Are Internet based technical forums really so "vandalized" by anonymous users that in order to participate one must always have to Register?
I registered to places like Advogato and Freshmeat to be a member of a community.
But oh so many times I want to enter into an online forum simply to ask a question. Always doing so after searching the Web (and sometimes books and magazines as well) turns up no answer. Usually these forums are hardware or software related, like a Dell computer, or Adobe software; something for which I do not want to "join up" because I am a "Dell Groupie" or an "Adobe Fan", but to simply ask something of someone closer to the technology than I have been able to get to. (More on this later.)
And this is the typical forum register process:
I have gone through this process several times. An many times only in order to ask my one or two questions, read the feedback, offer resulting feedback, never to appear again.
I do not like this process. It is so... archaic. We have the technology to automate this entire process. That is what computers are for!
So, the Globally Recognized User Environment would be similar to the Globally Recognized Avatar.
Now, I am someone who has trouble understanding most RFCs, so I can't write one myself, but I can outline the technology:
The Web Browser would have a setting for your GRUE (you see why I picked the name ;-), i.e. you would enter your E-mail Address and Password -- that is it. (There would probably be more, optional data, like User Name.) The password would then be stored as a one-way hash.
When you go to a GRUE enabled Web Site, you would hit a link like: "Register with GRUE Account".
Then, and this is the good part, you would only have to make sure cookies are enabled, the Web Site would:
It would then verify you by comparing your typed in password with your hashed password; if OK it would allow you to post.
Obviously, the transmission of the password is in the clear, and this may be a gaping security hole -- I'm not experienced enough to know if this is something that can not be overcome other than HTTPS -- but most Websites already do this anyway.
This, GRUE - Globally Recognized User Environment thingy is just an idea. And ideas are always free to be used. (If this is a valid idea I'd love to see it pursued by someone in the Free Software Community.)
I will further elaborate at times future. See also, perhaps: GRUE.
** The amount of data required to be input varies. Most technical sites require simply ID and Password; some more generic sites require many pieces of information.
*** Some Web Sites have horrible data verification processes. I've encounter one that had two pages of forms before checking for existing user names. Obviously, e-mail addresses are guaranteed to be unique and most sites use it.
Adobe's PDF Software sucks. Their Business Model sucks. And their so-called Portable Document Format is a fraud in that it is not portable at all except when using their horrible software.
But I have to deal with Adobe. Everybody uses PDF documents.
Perhaps there are some other ways around what I am going to be complaining about here but I am complaining because I have not been able to find a solution other than buying Adobe SDK for thousands of dollars and writing my own C++ application. And I have spent months looking at what is available for download; I have investigated everything.
What Was Wanted
I had several pages of PAPER FORMS that our company's clients fill out. They were created in Microsoft Words years ago -- text with underlines and little square boxes.
We want interactive online fillable PDFs of these forms.
Adobe Acrobat is -- IMHO -- absolutely the stupidest way of creating PDF forms. In fact, using a GUI at all to create PDF forms is stupid.
Say I want a typical name and address form. This is what you do with a GUI to create the name field using Adobe Acrobat:
We now have now a the Name field. Repeat these steps for all fields (and buttons and checkboxes etc.). With six or seven fields no big deal. But I left something out.
You can not, in Adobe Acrobat, place simple text. I.e. we want a form like:
Name: ____________________ Address: _________________
Since you can not place the "Name:" text with Acrobat you must use a word processor -- a compatible word processor -- to PRE-create the form with general text that goes around all the form fields. THEN you create a basic PDF and edit THAT to ADD the form fields. Again, no big deal. Or is it?
What if you have ten forms? With twenty, thirty fields per page?
What if you have hundreds? THOUSANDS?
That is a lot of people manually editing PDF files.
And guess what? All those fields you used the latest pretty GUI to click and drag? You have to manually place and size each and every one of them! Sure, there are tricks like copy and paste, nudge, align horizontally, etc. But still it is all such... drudgery! Computers are supposed to free us from work. Instead computers (software) degrade us by simply replacing pencil and paper with mouse and screen.
There Must Be A Better Way
There is a better way. Adobe's Form Designer software is a step in a better direction but it too is still the GUI paradigm -- clicking and dragging pixels on a screen. Form Designer's GUI helps with some of the drudgery only with "advanced property dialog boxes" which is hardly a solution.
But, one thing Form Designer does is to save the document data not in a proprietary format but in XML!
Where a PDF is 500K the same in XDP is 50K. Not only that, I can use any text editor to modify the XML. I can use any scripting language to process the XML. XML is that cool. What Perl did to programming XML is doing (has done?) to documents.
More on why XML is so nice later, but now I need to make a side step...
But I still need PDFs for our company's clients. So, still learning Adobe Form Designer I "saved as" a PDF. And it turns out that -- right back to the sucky Adobe Business Model -- form fields in Adobe Form Designer created PDFs can not be edited in Adobe Acrobat!
Why is this a big deal?
Form Designer does a really stupid thing with it's form identifiers. I name it "name" and it names it -- when converted to a PDF -- "F.P1.name".
Why is that a big deal?
Well, for me, our office needs a way to get an online filled-in PDF from our website to out office. Since our webserver is LAMP server, I had designed a rather simple and easy to use PHP script to gather the submitted PDF forms -- as an FDF file -- and then to e-mail the FDF file to the office where I can save the file in the clients "folder". Now all submitted forms are stored as an FDF which when opened by an office minion using Windows will see the form exactly as the client filled it out online. She can then print it, input it into the database, respond to it, etc.
But Form Designer's form-sub-form "F.P1" garbage breaks my PHP form handling system. And to accommodate it I not only have to re-design it all I will have to delve into the inner -- and undocumented -- working of Form Designer's new format. I already went through that process learning PDF and FDF. Now I have to start from scratch again!
Also, since the later versions of Adobe software are so damn expensive, we do not want to convert all our staff to the new versions -- the old versions are just fine for other documents. And, of course, newer PDFs are not editable by older Adobe software.
What I Did
That Form Designer used XML was a boon because I was fairly quickly able to figure out what they were doing. So I came up with my own file format to describe and layout my forms (by converting the Microsoft Word document to text and some more text editing) -- this is all a very old way of doing things.
I then wrote a Perl script to read my form file, providing some basic input parameters like page size and margins, line height, etc., which converted it to Form Designer's XML. And I did a pretty good job for a quick hack. I had to use Form Designer to tweak the pages a bit, but I bypassed 99% of the click 'n drag drudgery!
I then created the PDFs and I used the command-line program called PDFTK to convert the PDFs into an earlier version of PDF which were much more Perl friendly (less binary data and more line-feeds).
I then created another Perl script to rename the field identifiers back to normal names (also converting them from UNICODE).
I now have PDFs that remain compatible with all Adobe versions, my PHP form handling software, and I can create PDFs from easily editable text templates.
With my Perl scripts processing my data and templates I bypass those terrible Adobe GUIs! I save so much time!
I think that the best thing that could happen to the "Office Paradigm" is the death of the GUI.
Yeah sure, you want to do some things with the mouse, but to click 'n drag hundreds of tiny little boxes all around the screen for hours on end to create a simple form is absurd when all one needs to do is to enter a few columns of data and then have the computer do the rest!
Use the GUI to design the template, the images, the "look and feel". But start with a text file or a spreadsheet and then process the data and merge that data with a template and THEN create the resulting PDF or what have you.
Keeping all your text and images only in the end result is such a waste.
I will be a good day when all I have is text and images and rules and templates to create my PDFs, my brochures, my advertising, my letters and my websites.
All I need is a Perl script and I can change the world.
This is a horror story. One that makes me cringe to think about it. I have wasted so much time, have been made so angry over this, that I feel a need to vent a little.
First, some background.
Microsoft's Terminal Services Licensing is difficult to understand fully (one need just read the Usenet newsgroups dedicated to Terminal Services to see), however, there is one rather easy to understand aspect: Windows 2000 Server can give temporary licenses to Windows 98 and XP Home clients. These temporary licenses expire after 90 days. To continue using the temporary license one can, in Microsoft's words, "transfer a Terminal Services (TS) client access license (CAL) internally from one client computer to another in the same company within Terminal Services Licensing."
Although this "license transfer" is allowable by EULA, one has to call Microsoft's Terminal Services Licensing Customer Service Center. You can complete this reissue process only by using the telephone for a charge of $245.00USD. *
But -- and this is the one easy to understand part -- one can "reset" the 90 day license just by deleting a Registry Key on the client computer. I can live with that. Every 90 days I just reset my access counter. Very simple. Get this: Delete a registry key. Very simple.
But, ah, no... We have remote users... So we now enter the...
Microsoft Madness Zone! With a side road called AOL Hell.
One fine day a remote user contacts me with the usual e-mail stating that she "Can't connect." Subsequent e-mails resulted in my learning that indeed it was that the TS license had expired. ("Oh, yes. I did get the warning. But I didn't understand it.")
"Just run REGEDIT and... Click the start button. Click run. Enter R E G ..."
"But this is so difficult!" she moaned. "I can't do this!" she groaned. "Fine," I say. "I'll send you a file to do it for you." ** So I e-mail her a file which, when she "double-clicks" it, will delete the registry key. Of course, the route is through AOL. Turns out, AOL does not know what to do with a file name that ends in ".reg" ***
"So, just save the file... I mean attachment, and double click it," I say, on the phone now for she could not understand the e-mail. "Okay. I think I saved it. Now what to I do?" And it got worse. She could not find and ".reg" file. (Slapping forehead... grumble about hide extensions...) "Oh. Let me try and explain about file extensions. You see, Microsoft... [snip] So look for a file with an icon of little blue-green squares..." Very confused now (both of us), I had her search for the base filename. Eventually she found something with a "MIM" extension she said. "MIM?" I asked? "Yes, M-I-M." **** Totally confused now (and frustrated). I decided to try something else.
I then created a HTML page with a link to the REG file and put them both on a website. I sent her the link to that page (not wanting AOL anywhere near it) I told her to open the address in Internet Explorer -- which she did. She clicks on the link to the file and she says something odd about just seeing "M Store". (Turns out that for .REG files, Internet Explorer does not offer to save but simply views them; "M store" was her referring to some of the file's content.)
Getting her to right-click to save -- and making sure she notes where the file was saved -- she finally got to double-click on the file. "Click yes and then finish," I say.
That she was really happy when she got the remote access to work did not satisfy me in any way -- I know that I will have to repeat this process in about 90 days.
It gets worse
Later, when this happened to another user, I thought that I would again e-mail the .REG file to the person -- he uses Microsoft Outlook. Well, this time I was sending with Microsoft Outlook, and it warned that sending a .REG file might now work. Well, I'll send it to myself first -- it does not work because Outlook knows of the security risk and has blocked the file.
Okay, Microsoft wants to protect users from running programs that complete strangers send them, fine. But what about friends? Is there some "I Know What I'm Doing" Outlook setting? Perhaps there is a registry setting for it?
Fine. I'll just rename the file to .TXT and provide instructions to rename the file back. I send it as a test to myself. When I try to save the file Outlook tells me that it "cannot save the URL to a file," that it "Can't find this file," and that I should "Make sure that the path and file name are correct." *****
A Google search of that error message points me to the Microsoft Knowledge Base KB810646 which states: This problem occurs when you install Microsoft Office 2000 Service Pack 3 (SP-3). ****** Well, I guess I never saves a file attachment since installing SP3 because this is the first I heard of it.
Since it is certain that the recipient will have SP3 installed I again will refer him to the webpage I created, providing detailed instructions about how to save the file.
Is this really the State of the Art in 21st Century Desktop Computing?
This is all a complete nightmare as I see it.
A complete and utter FAILURE on almost all levels of Microsoft software.
Madness... Complete and Utter Madness... Microsoft Madness
* Via some MS voodoo chant or something? Why can't an Administrator administrator his/her own computer? Is there some secret access code that must be given verbally and which can not be written down? I don't know. I'd love to see some brave soul anonymously post to the 'Net what goes on during this process.
** For those not experienced in Windows, one can automate registry hacking by using .REG files. Windows associates .REG files with its REGEDIT program.
*** Because in the Microsoft Madness Zone files are not really files but "things" such as programs or pictures or recordings. And these "things" verified solely by their "cover" -- the ending letters of their name, i.e., ends in ".exe" run it! Ends in ".gif" view it!
**** It turns out that AOL, when is encounters a file with an extension that it does not handle, will save the file in its MIME state, changing the extension to ".mim".
***** Microsoft's award winning error messages make so much sense! And what makes this "windows paradigm" so dumb, is that when an error "window" pops up, there is no way of getting further help -- no help button, no log files, no nothing -- one can not even use the mouse or keyboard to copy the text of the message!!!!!
****** Microsoft goes on to say: To resolve this problem immediately, contact Microsoft Product Support Services to obtain the fix. For a complete list of Microsoft Product Support Services phone numbers and information about support costs, visit the following Microsoft Web site:. What the fuck?!?!?
Microsoft's MSDN a lesson in how NOT to present important information for your customers.
Go here: http://msdn.microsoft.com/library/default.asp and you try and find information regarding, say, VBSCRIPT's ERR object, and you tell me how easy it is. These people at Microsoft are INSANE. That is the only conclusion I can find.
At first glance one would think that the overall HTML layout is a pretty good choice: a header across the top, a narrow left side "navigation" frame, and a large right side frame for the information you want. However, Microsoft manages to completely ruin it.
First thing you will find is that the navigation frame (their "TOC") is extremely large, extremely redundant HTML (well, XML generated, but HTML in its final presentation to a Browser); HTML that at first is not very large but its size increases as you start "navigating". You will find that initial navigation by clicking in the navigation frame is easy and does not generate much HTTP traffic as at this point the entire TOC has not been loaded (assuming you have started viewing the MSDN Library for the first time). And the Browser caches things nicely.
For example, I navigated the TOC frame and after several ten's of clicks got the tree down to the VBSCRIPT ERR object.
The description of the Err object is a nice overview of the object but lacks completeness. I know it's an object; I want to see it's properties and methods. A couple are mentioned in the text. The text is great if you already have knowledge of this object! But for someone wanting to get a complete reference to the object the information about this object is entirely inadequate. Okay -- so we just have to click another level in the TOC or a link in the description page: Err Object Properties and Methods .
Clicking in the TOC brings up the properties and methods page in the description frame. All well and good as the saying goes. Things get worse from here on in as the saying goes.
At this point the TOC tree has ended and there is a list of links for the properties and methods of the Err object. No descriptions of them, just links. Now clicking on a property link results in a curious thing, we get to the description of the property and then the entire TOC reloads and we suddenly find ourselves very far down in the TOC. So, there is a Book analogy of sorts, we were at a page reading a list of object properties and each property had a reference to a page further down the book to get an actual description of the property. Okay fine. In this case the page was only say, 20 pages away. And the TOC that the browser had to reload had only several more branches expanded*. Well, not too bad. The Browser does have a "Back" button.
So now I read the description of the property I want to learn. And I click several more times to learn about the rest of them.
At this point I (starting form the point where I got to the base Err Object page) I had to click a dozen times to learn about the 5 properties and 2 methods of this object, for the property descriptions are linked together, they are not linked to the methods. All the while for every click the TOC reloads each time. And how Microsoft has implemented the TOC it is not cached**.
What Microsoft has done, is to list consecutively, all objects, then all properties, then all methods (and keywords, functions etc.) for not only the entire VBSCRIPT reference but their entire MSDN reference as a whole.
Instead of listing each object as AN OBJECT! with all of it's properties and methods, for each object, they have linearized their entire database!
And what makes things worse is that each time you want to read the description of something you must load just that one something (property, method, etc.) and at the same time the entire TOC!***.
There is no "next" button in the content of an object, i.e. go to the next description of the next property, you can not navigate a single object as a single object! You can only "move" from next to next for all properties or all methods. This lineararity works within a single context only; all statements, all constants, all events, and all properties of a object for each object. It is horrible that they linearize on all properties for all objects. Compounded by the fact that they have a single page for each property, method, etc.
Now, VBSCRIPT has five objects with a total of 11 properties and 7 methods. So it actually ain't too bad. However, the references for Excel.Application or Word.Application objects are far more complex with hundreds of properties and methods.
It takes hours to navigate MSDN. Hours. Hours.
Fully, 90% of my time programming for Microsoft Office is spent navigating MSDN (and Googling). The tops of their reference tree for all of their objects are slight overviews only, with simplistic, useless examples that show only one or two properties or methods. There is no complete set of links all in one place for any one Office Object.
MSDN is a horrible, horrible quagmire of twisty little passages all alike. There is no wonder whatsoever as to why people say that Microsoft Sucks. MSDN is an example of either design by compartmentalized committees, planned obsolescence, or stupidity. I actually believe that it is a combination of all three, with most of the problem based on the sickness of "getting people to buy CDROMS and BOOKS" to try to get better programming references. However, have you ever read a popular book about Microsoft Programming? They too are horrible. Every book about Microsoft Programming that I have ever purchased or seen -- with the exception of O'reilly's VBSCRIPT reference -- has been as poorly presented as MSDN with the addition of being never complete and having lame examples.
I have come to the conclusion over the last several years that Microsoft's programming languages, VBS, ASP, along with COM and the DOM, are actually VERY GOOD THINGS. PHP and Perl also integrate well into this model. HOWEVER. The Documentation SUCKS COMPLETELY!
Steps to Programming Success
In this document I outline the steps for successfully developing a programming project. By programming project I mean one or more pieces of software that makes up something like a library, an application or a set of utilities, for either a personal computer or an embedded system. By success I mean the result of ending up with bug free, working, quality software.
The intended audience is that of a lead programmer or architect in a small groups of developers, or a programmer who is a member in a group of developers, working at a small software company. As I wrote this open source projects kept coming to mind so I think that my thoughts here will work for both commercial and open source software.
The following steps should be generally useful when starting a new, not too large, programming project. But I outline overall programming practices that should scale quite a bit. Please keep in mind that I do not provide an exact recipe for success, but provide an outline which can be applied to the process of software development.
I have developed these ideas over many years of writing code, creating programs small and large, and interacting with people one-on-one, through e-mail and as a member of various programming teams.
I will not say that what I have written here is the one best way. However, I will say that I have had my share of both failures and successes so that I feel that I am not completely ignorant in these matters. I also have nothing to gain from what I say here -- I am not an author and am doing this solely for my own personal gratification. (This is basically a compilation of notes I have written down in an old notebook over the summer, so do not expect a great piece of literature.)
We start with zero, or, more importantly, before step one.
0. Have a clear goal.
There always must be a clear goal. Without one there can be nothing to succeed in. Success is reaching your goal. You cannot have success without a goal.
Obviously, wistful goals are not goals. A goal of "success" or "fame" or "fortune" is not a goal. A goal of "the best Web application" is unrealistic and unreachable. Goals cannot be vague, they must be specific.
A goal to learn a second spoken language is not unlike a goal to write a computer program -- as long as you want to learn a specific language and write a specific program.
Scratching an itch is just a vulgar way of saying to write something that does not exist (or that you do not have), to accomplish something that you need or want, but the relief of that itch is the goal in that the result is a specific program that you want to use for a specific reason.
For open source software, a project will not succeed by just opening doors and expecting people to come. It takes a clear goal with a clear result as the reward to attract people.
1. Design your design.
The first thing to do once you have a goal is to document a design. As with a goal, specifics are needed. It will not be enough to simply say "fast," "flexible," "versatile," or "massively multiplayer." Software cannot be "powerful" -- it cannot push your car up a hill. Software is limited to manipulating data and computer peripherals.
A design is more than just an idea or a list of ideas. As a filmmaker uses a storyboard, a writer an outline, and a builder a blueprint, a programmer needs a design.
The design should be in proportion to the size of the project. Small programs that can be (and are) written by a single programmer can be designed "on the fly" in the programmer's head as he or she writes the code. Larger programs may need a design from a page or two in a notebook to several documents including many diagrams. The documentation is not directly proportional to size but to complexity. A program that only interacts locally -- keyboard, screen, file system -- may be smaller than the same program doing the same thing interacting with multiple connections over the Internet, but it probably will not require a larger design or more documentation.
The design may also be influenced by hardware or software interfaces such as device drivers. Designing the UI may be sufficient for some programs.
But a design should never be too detailed. Do not over document! If you try to document beyond data structures and APIs, and cover how to code down to for loops vs. while loops or other code aspects, you can wind up documented into a corner, losing the creativity and inspirations of the coders. When a code comes up with a more efficient way of doing something you will be faced with either adhering to inefficient code, having to rewrite the documentation or having inaccurate documentation.
Once you have the basics down the code will follow.
There is no need for a detailed design of the implementation down to the level of each function call before coding can begin. If you have to document how you are going to implement the code you are then essentially writing the code twice -- the "how I am going to implement the code" document and then code itself. You will then be in the situation of having to maintain two code bases, the implementation documentation and the code itself.
So, what is a design? How would one know what is sufficient? Well, perhaps this will be seen as a cop out, but answering those questions will require another essay of itself. Let me at least try to offer some suggestions. A design could range from a list of features the software is to accomplish in the case of a service or to something that would be similar to the software manual in the case of a an application. A design should answer the question, "What does the software do?"
2. Keep a strong lead.
One experienced person needs to be in charge, to oversee things, to have final say in disputes. The design needs to be communal, but the overall control needs to be top down. A strong lead needs to be experienced in knowing that no one will always be right. The lead must know when a change to the design is necessary, and when to let others have their way when they are right.
A leader who demands total control over the design risks alienating the coders, slowly killing a project. A leader that demands that the design strictly follow a textbook, rather than treating textbooks as guidelines, will quickly kill a project.
But too many people all trying to design, and arguing over details, can slow a project down, risking making the project difficult to manage.
Adding a little Cathedral to the Bazaar can be a good thing. One person designing everything, instructing everyone, will probably not work well unless that person is extremely good. How many of us are that good? But one person, keeping the team together, getting the team to design together, delegating, asking for input and ideas, making the "tie-breaking" decisions, will work and work well. How many of us can do that? Perhaps more than we think if they understand the process.
In the Bazaar model, the "community" of developers spread across the Net are not all directly involved in the design; in actuality, one person or a few people (the small "core" group), does most of the work, while fielding suggestions, bug fixes and patches from the rest of the "community" (taking the best solutions, rejecting the rest).
3. Let the coders implement to the design.
Once the design is done, unknowns marked down, and the coders start coding, the team spirit must be maintained but only on the large scale. The individual coders must be able to act and program like individual coders; they must be able to converse and consult among themselves and their peers when making implementation decisions.
A weekly meeting to see how things are going, to see if the work is progressing within the time frame, to see if anyone needs anything, is quite adequate. The leader's role is important, needing to be supportive and not intrusive. A leader working one-on-one with each coder in separate little meetings is a recipe for disaster.
The reason why the design should not go into excessive detail will become apparent during implementation. Very often, as the coding starts, a better idea comes along and a different algorithm takes over an original idea -- this should never be prevented. Trying to justify a new algorithm to an inexperienced manager who had you over-design in the first place is not fun, to say the least.
Programmers have many resources at their disposal: books, magazines, the World Wide Web, Usenet and colleagues. A good programmer will takes advantage of all of these. So there is a little of the Bazaar in the Cathedral too as programmers working in a closed source environment take advantage of everyone else's published works. With today's Web, all "in house" closed source projects are helped indirectly by a world wide community of developers. All free software is available to all.
(In looking at the competing aspects of closed source vs. open source, for closed source projects there can be less of a sustained programmer base for programmers as closed source companies are subject to programmer turnover, as a result there may be little or limited peer review (of code and coding practices).)
4. Relaxation and reflection are important.
Sometimes there is an end to a project, more often so for closed source (when a delivery is made, or a release is published). But while developing one needs to have some sort of closure from time to time -- a period to rest, idle time to think about and clean-up the code, perhaps polishing it a bit here and there. (Remember that algorithm that you know can be more efficient?)
If there is no rest or room for reflection, especially if there is feature creep, chances are that the code will remain in a "just finished" state, or "just good enough," and it will never get cleaned-up, let alone polished. Debugging statements may get left in, test code may be left in but commented out, comments may remain wrong or misleading.
Time needs to be set aside, during development, for relaxing and reflecting on the code -- allowing the coders to clean-up and polish.
I realize that such an R & R process during development will be quite controversial and many people will immediately call me crazy. And I do understand the importance of scheduling and of delivering on time. But there is importance in quality code and in quality time.
If a project has a tight schedule or is running late there will be no time for pauses in the development will be one argument. Another will be that if everything is designed properly there will be no need to re-code anything in the middle of development.
Late projects will always, perhaps, be an exception. But scheduling can easily accommodate a few days a month toward this. Since the entire code base can not be designed fully and must be implemented in sections there may always be room for improvement. And I am certainly not advocating any sort of basic re-design every once in a while -- code clean-up and polishing should consist of comment clean up, the improvement of a conditional branch here or the re-organization of a for-loop there. Along with the ever important process of catching bugs.
As for re-implementing an algorithm based solely on its efficiency, this is where a strong lead and weekly meetings shows its most important role. If, during development, someone thinks of a way of re-implementing something non-trivial, she can bring it to the table where it can be discussed. If re-implementation would effect the schedule there would be no need to carry it out.
This need not be thought of R & R if you do not like that term. Thinking of it as a code review process may help in understanding this. Improving quality and preventing bugs is extremely important. Why leave code review and clean-up to the end of a project?
Sometimes I think this is all right on the money, other times... it seems like a bunch of hooey... I don't know. My programming projects are getting smaller and smaller these days; PERL, PHP and a few scripting languages are all I use these days. Well, perhaps there is something good here. If not... Well, no harm done. (I hope. ;-)
The following books have influenced me or I feel that they having something important to say on this subject.
1. The Art of Programming, Donald Knuth
Although the works have not effected much of my experiences of the management process, the three volumes that make up The Art of Programming has had by far the most influence on my coding style.
2. The C Programming Language, Brian Kernighan and Dennis Ritchie
The second most influential book on my coding style.
3. Software Tools, Brian Kernighan and P. J. Plauger
4. The Practice of Programming, Brian Kernighan and Rob Pike
Two more books explaining the basis of quality programming and program development.
5. Literate Programming, Donald Knuth
6. The Cathedral and the Bazaar, Eric Raymond
7. The Psychology of Computer Programming, Gerald Weinberg
8. The Mythical Man-Month, Frederick Brooks
I am reminded of an old saying frequently heard on BBSs:
Open mouth, insert foot, echo internationally.
The older I get it seems the more my brain does not work.
New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.
Keep up with the latest Advogato features by reading the Advogato status blog.
If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!