Older blog entries for MichaelCrawford (starting at number 34)

5 Dec 2002 (updated 5 Dec 2002 at 07:09 UTC) »

Updated: softkid, yes, implementing the performance-critical part of my product in assembly is one of the things I'm doing. But it's not completely obvious what the best assembly is. So one of my tasks this week is to experiment with different alternatives in assembly code, and understand the performance of each that I get.

One problem is that I haven't done assembly in a long time, although I used to do a lot. Another is that I haven't ever programmed the assembly for this instruction set architecture before. I've done well at learning it so far, but of course I don't know all the clever tricks yet. I used to write some really odd 68000 assembly code, stuff that no one could figure out just by reading it, but that was significantly faster than the best I could ever get in C.

There are some tricks to what I'm doing that turn out to speed it up quite a bit, that I was unaware of when I wrote the proof of concept demo. I had known that there were some well-known optimizations, but didn't look into them because I didn't realize they would be necessary even for a proof of concept. I had no idea that the straightforward, obvious way to implement this would be so slow. But I am aware of some of the optimizations now, and am researching others.

I'm working on a new implementation in C that should be a lot faster even that what I had in my first attempt at assembly code. When I get that working well, I will have another go at assembly and probably should be able to improve a little on the C.

Again, I'm sorry to be so oblique about what my product is. But it's something I think you will all be happy is available. It won't be expensive, and maybe some of you will want to get one. Maybe I can even ship it in time for Christmas.

Happy News!

My client has decided to fund continued development of the project I'm working on for him. As I previously reported he was testing my proof of concept demo before deciding whether to go further, but the performance was unsatisfactory, not fast enough to make it a marketable product..

I spent several days explaining to my client how I intended to speed it up, but some of those ways will be quite difficult and would take some time, and things are so tight financially that I could not afford to just implement those changes and prove it to him by showing him working code. He based his decision on his faith that my plans would work out.

Ordinarily I enjoy doing optimization. I think it is the best part of programming. I have some tricks for optimization that some people think are quite twisted, but have worked well in the past.

In this coming week I face one of the most significant challenges of my entire career. I have to make my code go so much faster than it does now that I really honestly don't know if I can succeed. There are some tricks I could use if I were running on a desktop PC but I am programming an embedded device with very little RAM or ROM.

This week will get funded, but whether funding for the product continues to bring it to market depends on the results I am able to achieve.

It would be fun if there weren't so much riding on it.

Sometimes I am able to forget about the peril we are in and just enjoy coding. The whole experience has been very hard on my poor wife, who asks how I can be in such good spirits during a time of such turmoil. I tell her that is is like programming with the Sword of Damocles hanging over my head, and my only hope for survival is to write good code. But I cannot write good code while I am frightened, so I do whatever is necessary to put it out of my mind.

Embedded Systems Programming

My client is testing the test version of the firmware I delivered for him as I write this. I'm very anxious to hear his reaction, as he will base the decision of whether to fund bringing the product to market on how his testing of this proof of concept goes.

He had some trouble with his testing at first, but I was able to help him get the tests working after talking to him on the phone for a little bit. But I had to let him go work on his own because I had written up a complicated test procedure to go with some test tools that I gave him tonight, that would prove to him that the product actually did what it was intended to. I needed to let him study my test procedure and try it out.

It would really be a problem for me if he doesn't want to continue with the development. I have other work in the pipeline, but it would be a while before I could see a paycheck from any of the other projects.

29 Nov 2002 (updated 29 Nov 2002 at 22:09 UTC) »
Embedded Systems Programming

Recall my previous diary entry where I said how difficult my project has been, and how I have been facing some severe performance problems.

It took me quite a lot of work, but I have implemented the optimization I was contemplating. It was really hard to get the code working correctly with this optimization in place.

However, it seems to be working correctly now.

It meant working Thanksgiving night. I ended up staying up all night and worked through the day today. I had a bad bug today.

I'm running a long test to check the performance. It looks faster, but not as fast as I'd hoped. I still have some debugging code compiled in, which would slow it down some.

I told my client just now that I would run some more tests on a release build once I was sure I had it working correctly, then send him whatever I had. I'm not going to try to optimize it anymore for this delivery. It doesn't have to be fast - it just has to be fast enough to make my client believe I can get it fast enough in the final product. It at least had to be usable, even if unpleasantly slow. I'm just afraid that not I'm sure if it's usable yet.

Posted again so it stays in recentlog a little while:

29 Nov 2002 (updated 29 Nov 2002 at 14:18 UTC) »

Updated: Why I Write

New at GoingWare's Bag of Programming Tricks:

How to Promote Your Business on the Internet

Getting a Google search for software consultant to list my resume on its first page of search results didn't require me to pay a lot of money to a search engine placement service. I didn't have to do anything dishonest, like build a link farm. It didn't cost a cent. It did take time and a lot of hard work, but my method is so simple that anyone could do it - and if you did, the Web would be a better place.

Does Advogato save text that's being edited?

I had a crash in Mozilla while the above text was in a diary entry window. The crash was while emailing the above announcement to some friends. When I went to re-enter my diary entry, which I had previewed but hadn't posted, I found that my text was restored. Neat. Or is that a Mozilla feature?

A feature I have always wanted in a browser is a confirmation dialog: "Post form contents before closing window?"

27 Nov 2002 (updated 27 Nov 2002 at 20:53 UTC) »
movement, I just posted a comment to my bugzilla report about how unpleasant mozilla is for my wife and reopened it.

I ask that you leave the bug open and encourage the mozilla developers to discuss the important issues I raise in the bug report.

You give some hairsplitting technical reasons for why it's an invalid bug report, but I assert that the report is completely valid. You have a serious organizational problem. You are not going to succeed in your mission unless you are able to convince people who try your product to take it seriously.

My wife is willing to give it a second chance, but I don't think she will give it a third. I'm hesitant to even let her try out 1.2. Most people wouldn't give you a second chance. That's one of the drawbacks of free software - people don't have so much investment in the product and aren't so willing to work with the vendor to get it fixed.

Most people would say that what I should do is file bug reports and submit patches. I do that when I can, but my time is limited. I feel that I can have a greater, positive effect on the community as a whole through the kind of quality advocacy role that I play with LinuxQuality.

Your turn.

PS. I don't think Kameleon works on windows. However, Internet Explorer works just fine.

Updated: movement said:

If you want to change things, pay for it, code it yourself, or get hired by Netscape. Don't whine.

There's something like 50,000 open source projects hosted at sourceforge. There's something like 8000 packages in debian woody, many of which contain more than one program. I regularly interact with dozens of different open source programs on Windows, Linux and Mac OS.

But you say that if I have a problem with any of them, I should pay for it, code it myself, or get hired by the publisher? Where will I start? Bill Gates himself doesn't have the cash to pay for all the bug fixes we need. I contribute actively to two projects, and I don't have enough time to do a decent job on those. And I'm happy with the job I have. In any case, each of those would only allow an incremental change to one application.

What I feel is the best thing to do is to help everyone to do better. I don't feel that enough attention is paid to quality in open source. That's why I write articles like these and and this and this one..

To some extent the poor quality of many open source packages works out OK because so many of the users are committed because of their ideology. But what about people who have no such emotional commitment, who want to get work done and get on with their lives? People who don't know how to program, are frightened by talkback, and find bugzilla unfathomable? Are you suggesting that their answer is they shouldn't bother, and use proprietary software like Internet Explorer?

27 Nov 2002 (updated 27 Nov 2002 at 14:47 UTC) »

Updated: Please Understand That This is Not a Troll

Despite the fact that it got modded as off-topic, Please Understand Why My Wife Can't Stand Mozilla was meant in all seriousness to help the Mozilla developers succeed in their effort to win the browser wars.

I'd like to suggest that anyone writing software meant for regular users (as opposed to developers) would benefit from reading it.

(Updated yet again: filed this in bugzilla as #182221.

Embedded Systems Programming

... is certainly challenging. Recall my earlier diary entry where I said that despite kroah's warning I thought it would be really cool:

Personally, I would be really thrilled if I could get paid for six months in order to get a beeper not to warble. I've been in that position before, and I loved it. I miss it terribly.

Well, now I'm spending six months trying to get a beeper not to warble. Sometimes it's hard to remember that I'm supposed to be thrilled.

I proposed my current project to my client. I came up with the idea myself, but he has the hardware my idea would need and is in a position to market it.

He was skeptical whether it could be gotten to work, and suggested that before he committed to funding the product, that I write a "proof of concept" that would demonstrate the core features of the product without actually going to the effort of completing a finished product.

"Sure," I said, "no problem. That should take me about a week." So he agreed to pay the amount I'd normally want for a week for the proof of concept to be demonstrated.

Well, I'm going on week six now for my "proof of concept". There have been many late nights. My wife has been very distressed at it all, seeing economic disaster on the horizon. I have to say I've been pretty distressed too, but I've mostly I kept that to myself.

I had thought I could deliver it two nights ago, but once I got the software "feature complete" (for just the features required in this early version) the performance was so slow as to make the product completely unusable. I had planned to leave any optimization for the next phase of development, but I feared that showing what I had to my client would make him decide to cancel the project.

I stayed up round the clock up until late last night hand-optimizing my code, and although I got it to go much faster, it is still too slow to even be presentable, let alone to make a product we're going to charge money for.

Late last night I thought of a different approach I could take to my optimization, and I'm working on that now. I'm pretty sure I can get it to the point that my client will believe in its potential when he sees it. But there will be significant challenges remaining before it can be brought to market.

One thing I have to say, I'm much better at embedded systems programming than I was to start with. That's been part of my problem, not having too much of a clue how to debug complex problems when the target system doesn't have enough ram to support a debugger.

I had spent some time studying how to do embedded programming before I got actual work in it, but I found long ago that I learn most effectively (at least about programming) when I'm being paid to do what I'm trying to learn.

For one thing, I can afford to put real time into the study. An importantly, the sense of desperation that ensues when I don't understand how to do something that I will only get paid if I accomplish it helps me to develop a mental focus that I'm not normally able to accomplish. If I have to understand something difficult in order to continue eating, I'll do whatever it takes to come to that understanding. It's hard to develop that kind of motivation when I'm just studying something out of idle curiousity.

But it's been really painful. I had two individual bugs that took me a week apiece to fix, and one bug that took me two weeks. I had to spend a lot of time reassuring my panicked wife that my business wasn't going to fail, we weren't about to become homeless, because fixing those bugs was "right around the corner" - even when I wasn't so sure myself.

And getting everything done that I had planned to do for my "proof of concept", and getting it all working reliably, only to find it so horribly slow, was simply devastating.

Part of what keeps me going is the simple fact that I just don't have anything better to do right now. The alternative would be to just take some time off and do nothing. I am in discussions with several potental new clients, but setting up new work takes time. So despite all the unpleasantness, I might as well continue the work and do the best I can. At the very least, I'm learning a lot, and there is still the potential that it might pay off well in the end.

I'm sorry to be so oblique about what my product actually is. I know it's not at all in the Free Software spirit, but I want to get the product to market before anyone can start working on a competing product. I will be sure to announce it here once my client announces it on his website, and I'll put a link to his announcement. My current development plans call for the product to be announced in about two more weeks :-/

23 Nov 2002 (updated 24 Nov 2002 at 03:25 UTC) »

Update: Writing for Ransom?

To pick up again on my previous entry where I wondered whether I should copyleft one of my articles, I just read this interested /. piece on Ransoming code. Basically the idea is that rights in software remain restricted until a certain amount of money has been raised to pay for releasing it as open source.

Aside from the advertising benefit of reserving an article only to my own website, another reason I am reluctant to copyleft them is that they take a lot of work to write, and require a significant amount of time off from doing billable work for my clients.

In terms of time I could have spent consulting, Pointers, References and Values cost me about $5000 to write. Most of my articles don't take nearly as much work, but one or two thousand dollars worth of time is typical.

I don't think I would require the full amount of that in "ransom" to copyleft an article, as the publicity the articles would continue to generate would be worth a lot, but it would take quite a bit of money if that were all that was put forward to motivate me to copyleft one of my articles.

Let me ask you then, what would you pay me to copyleft any of my writing? Have a look at some of my articles, and email me at crawford@goingware.com to tell me what you think you'd pay to get me to copyleft an article. Would you pay up-front, or at least commit to paying, for articles I haven't written yet, based on my past performance?

You can find my proprietary articles here and here. I also have some copylefted articles here (so you can see I'm not completely mercenary about it).

Something to keep in mind is that, while most of my articles are proprietary, I've been posting everything I write on the web where anyone can read them for free. The copylefting would just allow them to be distributed more widely, allow modifications (usually) and facilitate translation into foreign languages.

Also, I write like this any chance I get. The problem is that I have a hard time finding the time to write. My goal for the last year has been to write one new article each month, but I have never achieved that because I've been so busy just making a living. Getting paid to write would make it much easier to make time away from my development work. I'd do some writing anyway, but I'd do more if I could get paid for it.

I have also wondered about getting grants to write articles (not from individuals, as in the Ransom model, but from charitable foundations or corporations).

I do plan to try writing for magazines, where I would be paid outright, but it would be pretty unusual for the magazines to allow the articles to be copylefted.

Shameless Plug

My wife Bonita has started a new web design business. Her first project for her first clients went online Thursday evening. Check out the folk-rock duo The Divine Maggees at www.divinemaggees.com.

MP3's aren't available yet but will be soon. You could always order their new CD, "Way Back When".

They will be playing an east coast (usa) tour early next year.

Help Me Decide Whether to Copyleft My C++ Article

I have written a number of technical articles which are copylefted. But I have written even more which aren't.

One reason I don't copyleft everything is that some of my pieces are almost pure opinion pieces, and other articles are very popular, and draw traffic to my website, which helps to advertise my consulting business.

A discussion started today on a Linux Documentation Project mailing list about what some describe as poor quality in The C++ Programming HOWTO. Some feel it should be removed until it can be rewritten, and some have suggested that it has little content that can be salvaged at all.

Take a look in particular at Pointers are Problems. He just says not to use pointers if you can possibly avoid them, and to use references instead. I admit I hadn't read the document before it came up today, but the document looks like it was written by a Java fan who is trying to dress up C++ to look like Java, which I think is really inappropriate. There are strengths and weaknesses to both languages - if you don't like C++ and prefer Java, you should just program in Java rather than trying to make C++ something it is not.

In particular, pointers are required in C++. What looks like a reference in Java is really a pointer, it just has restrictions placed on it to avoid common problems. You can rebind a "reference" in Java, but you can't in C++, so you have to use pointers. But the author of the C++ HOWTO says "Pointers are not required for general purpose programming.".

So it occurred to me today that I have a wonderful opportunity to step in and do some good, if I were to relicense Pointers, References and Values under the GFDL and contribute it to the Linux Documentation Project. But I'm very hesitant about it. The reason is that the document is one of the most-visited web pages on my whole site, and everyone who reads it gets to see my banner ad at the top of every page.

Last month, the first page of Pointers, References and Values got 736 hits. So far this month, it's gotten 526. That's quite a lot of traffic for a website for a one-man consulting company. When I published a condensed version at Kuro5hin traffic to my whole site doubled for a month. Lots of people link to it too, guaranteeing me future visitors and helping the page to place highly in search engine rankings (the first fellow programmer I met in the state of maine found me by searching Google for C++ help).

I know that I could help a lot more people if I copylefted my article, but I fear that allowing people to read the article elsewhere would reduce the visibility of my consulting business. I'm sorry to say that, I know we are all here for Free Software, and I support it as best I can (and I think I can do more good with my writing than with any software I might write), but I worked long and hard to write the article and the dot-com crash has been particularly difficult for consultants.

Here's how I could be convinced to copyleft "Pointers, References and Values": tell me how copylefting it would benefit my consulting business more than it would to have the current proprietary copyright where people have to visit my website to read it.

Probably if the Linux Documentation Project is to carry my article they would want me to remove this section at the end.

If you don't want to post your response here you can email me at crawford@goingware.com.

Embedded systems programming

Well I think I am over the difficult obstacle I was having with my project, which involves customizing the firmware for an embedded firewire device. However, it does not appear to work reliably, I get I/O errors when communicating any really large amount of data with a host. But modest tests seem to work fine. Before I couldn't get it to work at all so I'm happy for the time being.

What I will do is write some test tools that exercise the device more extensively than what I had been doing, and then try to debug failures of the more rigorous tests.

However, I had hoped to deliver this milestone tonight. That's disappointing, it will take me a couple more days at least because I'm taking off tomorrow for the Embedded Systems Conference.

Here's how to do an assert in an embedded program. Suppose you have a function called LightLED that lights up an LED on your board, and you need to call ResetWatchdog periodically to avoid having the watchdog timer reset the CPU:

if ( !ConditionThatMustBeTrue() ){
    LightLED();
    while ( 1 ){
        ResetWatchdog();
    }
}

so if your assertion fails, the LED will light up and the board will stop processing.

It's Snowing!

About an inch has fallen so far this evening. I just put the car in the garage.

It's not the first snow of the season, we had a little snow a couple weeks ago, but it didn't stick.

It's coming down pretty hard, we might get quite a bit.

In other news, a nasty bug I've been stuck on for almost a couple weeks in my embedded project has been fixed. I should be able to finish my milestone by tomorrow evening.

Monday I'm going to the embedded systems conference in Boston, to pass out my business card to the vendors there and hit them up for consulting business.

25 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!