Older blog entries for MichaelCrawford (starting at number 38)

Embedded Systems Programming

I worked through the night, night before last, to speed up my code, and hit what I believed at the time was a brick wall. The kind of brick wall that would make it impossible to get my product fast enough to bring to market.

But after I finished up for the day and spent a few hours just relaxing I thought of a possible solution. I don't know if it will work, but it could work, and maybe if I can get my solution working at all some further work will make it fast enough.

I was eager to implement my idea yesterday, but I hadn't slept and was exhausted. It was very hard to bring myself to go to bed. I managed to, but I woke up at 2:30 am today and got up to go to work. Even though I had gone to bed in the late afternoon, it's not like me to wake up so early and go straight to work. That's how badly I want to succeed.

This embedded project has been a Trial by Fire. But it still has the potential to pay off well, and the painful experience I'm gaining should make future embedded projects a lot easier.

Writing

Zach Frey very helpfully wrote to me about my last diary entry where I discussed the tension between my struggle to make a living and my desire to do things that I felt were more important like writing and political activism.

He referred me to a speech C.S. Lewis gave to some British University students at the outbreak of World War II, in 1939. The speech is called Learning in War-time.

A Google search for "Learning in War-time" doesn't find the complete text online but there are a number of quotes available that I have found helpful:

The war creates no absolutely new situation: it simply aggravates the permanent human situation so that we can no longer ignore it,..

Human life has always been lived on the edge of a precipice. Human culture has always had to exist under the shadow of something infinitely more important than itself. If men had postponed the search for knowledge and beauty until they were secure, the search would never have begun...

We are mistaken when we compare war with 'normal life'. Life has never been normal.

I'm going to go look for the full text of the speech in the library and a couple bookstores some time in the next few days.

Zach also wrote about the importance of what I do in the way of making a living, that this advances the cause of Freedom - working independently as a consultant, and making what contributions I can to Free Software.

My long-sufferring wife understands why what I do is so important to me, and she supports and encourages me as best she can, but the way my other activities sometimes threaten to bring us closer to the precipice often frighten her.

I wish I didn't cause her so much pain when I take off to do something that I'm not getting paid for. I wish the way I do make my living were much more secure, so that when I am working billable hours, there would be more certainty in our lives. Usually I make enough money from my work that I can afford to take take off now and then to write, but it's never certain at the time I do it that my consulting is going to work out well.

Very proud to have done my small part

So I was hanging out on Slashdot for quite a while during the discussion of Because only terrorists can use 802.11 because I felt it was an important opportunity to get the message out about working together to oppose the terrible course the U.S. has been taking since September 11.

But my wife was distressed because things are very difficult financially and I need to get a bunch of hard work done in order to get ourselves out of this jam.

The more upset she got that I should be working instead of raising people's consciousness on the Internet, the more upset I got that she was trying to prevent me from doing something that I felt was of signficantly more importance than spending the night hacking some code.

We had a long talk about this eventually. It cleared the air quite a bit.

My wife is a very apolitical person. She prefers not to read the news because it just gets her down. She doesn't feel that the individual has much power to make a real difference. And she is very frightened about how we are going to pay all these bills that are overdue.

I, on the other hand, read the news pretty obsessively and am very frightened about what's going to happen a year or two down the road if Bush, Cheney, Ashcroft & Company succeed in what they are up to.

I worry that saying stuff like this could get me imprisoned someday - or worse.

Many of you might feel that that's an absurd thing to say, but I say that it's only relatively recently in history that many people at all have been able to speak their minds without fear of reprisal, and many, many people even today live in places where speaking out can mean a sudden disappearance.

I feel the only thing that I can really do to prevent that from happening here is to continue to speak out.

After reading some of the comments posted in reaction to posting the essay at K5, I feel that it's even more important that I continue to speak out. A couple of people raised the point that the Left is rather lacking in people able to speak eloquently. Or even coherently.

I feel that my ability to reason with people through my writing is a gift that I should be using for the common good.

The main reason I am a Green and no longer a Democrat is that the Democratic Party has shown itself to be completely spineless as far as mounting a credible opposition.

If my writing is able to convince even one person to stop what they're doing, to speak out, and to take action, then I think it is very important that I write. I think writing "Is This the America I Love?" is one of the best things I have ever done. I feel I need to do more of it.

But then there's the problem of all those bills. I need to keep food on the table and a roof over our heads. If I don't keep producing as a programmer, I won't have to worry about bootheels at our door coming to arrest me, but about the bank coming to repossess our home.

But I told my wife that if I had to give up doing the things that I felt were the most important to me just to ensure our survival, that there was just no point to even continuing, I might as well give up.

Our life has been a constant struggle the last several years. But if the only thing I could do was to struggle just to survive, and not to pursue the finer things, then it would break my spirit.

To give up my writing, and my activism, would be to lose the struggle. It would be worse than to be hungry or homeless. It would be to have no reason to live anymore.

There has to be something better than just struggling to get by. There has to be a reason for what I do beyond mere survival.

For me, that reason is to write.

But I wish it weren't so hard on poor Bonita. I really do. It breaks my heart to see her so frightened when I take a night or sometimes a few days off to write. No one pays me to write. Writing doesn't put any food on our table and it doesn't pay the mortgage - but I think it makes the struggle to get food on the table worthwhile.

Thank you for your attention.

I have gotten my embedded product working quite a bit faster than it was. Recall that the demo I gave my client was way too slow and it took some convincing to get him to continue funding the product, but he did eventually decide to continue paying me for development.

Unfortunately, it is still too slow to be marketable. However, it has become suprisingly usable even for the performance that it does have. Before it wasn't usable at all - I could demonstrate that it worked correctly, and measure its miserable speed, but that's about it.

I can't claim that much credit for the speedup. It turned out that there was a more efficient algorithm for what I'm trying to do, it gets the same results but is much faster. However there was the challenge of getting it to actually work in my firmware, which took a couple of days. I was very relieved when I got the new algorithm functioning and found that a significant improvement had happened (a speedup of about 20 times from what I had).

But I need to speed it up by another factor of 10 before my client will consider marketing it. And at this point he's only committed to paying me to improve the speed, more work will be required to bring the product to market and I have to hit a minimum performance threshhold for that to happen.

I have always been a big fan of tweaking code for performance. I have yet to find an optimizing compiler that I cannot beat by writing better C. And I am always able to beat my best C performance by writing in assembler.

Everyone says that you should only try to improve performance by selecting a better algorithm, but what if you are already using the best known algorithm and it's not fast enough. That's when you need to tweak your code.

Also some algorithms are only more efficient when they are used on large data sets, because they are expensive to set up. If you have many independent and small data sets, what would ordinarily be considered the best algorithm may really be a poor choice. In that case an algorithm that scales poorly but can be coded tightly may be a better choice.

I'm afraid that because I considered performance not to be a real concern for the proof-of-concept demo, I didn't even try to look for better algorithms when I wrote my demo, because it simply didn't occur to me that the straightforward and obvious way could have had such disastrous results. I've done some research now and I'm pretty sure that the algorithm I have now is the best available.

I don't think it will be too hard to write it in assembly and there are a number of ways that I can see that this will help. I should be able to get some performance gain easily. Getting enough may be difficult but I still have several days left of the time I have funded. I could take more time if I needed it but it would make things unpleasant financially again.

Again I'm sorry I won't say what exactly it is I'm working on but hopefully it will only be about a week and a half until the product is announced.

What Kuro5hin members are saying about my essay Is This the America I Love?:

Alright, who let this intelligent, well written, reasonable person into the Left?
-- wiredog

Michael better be careful, or people are going to start thinking that Greens are capable of rational thought.
-- wiredog

very well written and articulated, profound and thought provoking. Plenty of information and examples backing your opinion.
-- theforlorneone

Thank you for your attention.

5 Dec 2002 (updated 5 Dec 2002 at 07:09 UTC) »

Updated: softkid, yes, implementing the performance-critical part of my product in assembly is one of the things I'm doing. But it's not completely obvious what the best assembly is. So one of my tasks this week is to experiment with different alternatives in assembly code, and understand the performance of each that I get.

One problem is that I haven't done assembly in a long time, although I used to do a lot. Another is that I haven't ever programmed the assembly for this instruction set architecture before. I've done well at learning it so far, but of course I don't know all the clever tricks yet. I used to write some really odd 68000 assembly code, stuff that no one could figure out just by reading it, but that was significantly faster than the best I could ever get in C.

There are some tricks to what I'm doing that turn out to speed it up quite a bit, that I was unaware of when I wrote the proof of concept demo. I had known that there were some well-known optimizations, but didn't look into them because I didn't realize they would be necessary even for a proof of concept. I had no idea that the straightforward, obvious way to implement this would be so slow. But I am aware of some of the optimizations now, and am researching others.

I'm working on a new implementation in C that should be a lot faster even that what I had in my first attempt at assembly code. When I get that working well, I will have another go at assembly and probably should be able to improve a little on the C.

Again, I'm sorry to be so oblique about what my product is. But it's something I think you will all be happy is available. It won't be expensive, and maybe some of you will want to get one. Maybe I can even ship it in time for Christmas.

Happy News!

My client has decided to fund continued development of the project I'm working on for him. As I previously reported he was testing my proof of concept demo before deciding whether to go further, but the performance was unsatisfactory, not fast enough to make it a marketable product..

I spent several days explaining to my client how I intended to speed it up, but some of those ways will be quite difficult and would take some time, and things are so tight financially that I could not afford to just implement those changes and prove it to him by showing him working code. He based his decision on his faith that my plans would work out.

Ordinarily I enjoy doing optimization. I think it is the best part of programming. I have some tricks for optimization that some people think are quite twisted, but have worked well in the past.

In this coming week I face one of the most significant challenges of my entire career. I have to make my code go so much faster than it does now that I really honestly don't know if I can succeed. There are some tricks I could use if I were running on a desktop PC but I am programming an embedded device with very little RAM or ROM.

This week will get funded, but whether funding for the product continues to bring it to market depends on the results I am able to achieve.

It would be fun if there weren't so much riding on it.

Sometimes I am able to forget about the peril we are in and just enjoy coding. The whole experience has been very hard on my poor wife, who asks how I can be in such good spirits during a time of such turmoil. I tell her that is is like programming with the Sword of Damocles hanging over my head, and my only hope for survival is to write good code. But I cannot write good code while I am frightened, so I do whatever is necessary to put it out of my mind.

Embedded Systems Programming

My client is testing the test version of the firmware I delivered for him as I write this. I'm very anxious to hear his reaction, as he will base the decision of whether to fund bringing the product to market on how his testing of this proof of concept goes.

He had some trouble with his testing at first, but I was able to help him get the tests working after talking to him on the phone for a little bit. But I had to let him go work on his own because I had written up a complicated test procedure to go with some test tools that I gave him tonight, that would prove to him that the product actually did what it was intended to. I needed to let him study my test procedure and try it out.

It would really be a problem for me if he doesn't want to continue with the development. I have other work in the pipeline, but it would be a while before I could see a paycheck from any of the other projects.

29 Nov 2002 (updated 29 Nov 2002 at 22:09 UTC) »
Embedded Systems Programming

Recall my previous diary entry where I said how difficult my project has been, and how I have been facing some severe performance problems.

It took me quite a lot of work, but I have implemented the optimization I was contemplating. It was really hard to get the code working correctly with this optimization in place.

However, it seems to be working correctly now.

It meant working Thanksgiving night. I ended up staying up all night and worked through the day today. I had a bad bug today.

I'm running a long test to check the performance. It looks faster, but not as fast as I'd hoped. I still have some debugging code compiled in, which would slow it down some.

I told my client just now that I would run some more tests on a release build once I was sure I had it working correctly, then send him whatever I had. I'm not going to try to optimize it anymore for this delivery. It doesn't have to be fast - it just has to be fast enough to make my client believe I can get it fast enough in the final product. It at least had to be usable, even if unpleasantly slow. I'm just afraid that not I'm sure if it's usable yet.

Posted again so it stays in recentlog a little while:

29 Nov 2002 (updated 29 Nov 2002 at 14:18 UTC) »

Updated: Why I Write

New at GoingWare's Bag of Programming Tricks:

How to Promote Your Business on the Internet

Getting a Google search for software consultant to list my resume on its first page of search results didn't require me to pay a lot of money to a search engine placement service. I didn't have to do anything dishonest, like build a link farm. It didn't cost a cent. It did take time and a lot of hard work, but my method is so simple that anyone could do it - and if you did, the Web would be a better place.

Does Advogato save text that's being edited?

I had a crash in Mozilla while the above text was in a diary entry window. The crash was while emailing the above announcement to some friends. When I went to re-enter my diary entry, which I had previewed but hadn't posted, I found that my text was restored. Neat. Or is that a Mozilla feature?

A feature I have always wanted in a browser is a confirmation dialog: "Post form contents before closing window?"

27 Nov 2002 (updated 27 Nov 2002 at 20:53 UTC) »
movement, I just posted a comment to my bugzilla report about how unpleasant mozilla is for my wife and reopened it.

I ask that you leave the bug open and encourage the mozilla developers to discuss the important issues I raise in the bug report.

You give some hairsplitting technical reasons for why it's an invalid bug report, but I assert that the report is completely valid. You have a serious organizational problem. You are not going to succeed in your mission unless you are able to convince people who try your product to take it seriously.

My wife is willing to give it a second chance, but I don't think she will give it a third. I'm hesitant to even let her try out 1.2. Most people wouldn't give you a second chance. That's one of the drawbacks of free software - people don't have so much investment in the product and aren't so willing to work with the vendor to get it fixed.

Most people would say that what I should do is file bug reports and submit patches. I do that when I can, but my time is limited. I feel that I can have a greater, positive effect on the community as a whole through the kind of quality advocacy role that I play with LinuxQuality.

Your turn.

PS. I don't think Kameleon works on windows. However, Internet Explorer works just fine.

Updated: movement said:

If you want to change things, pay for it, code it yourself, or get hired by Netscape. Don't whine.

There's something like 50,000 open source projects hosted at sourceforge. There's something like 8000 packages in debian woody, many of which contain more than one program. I regularly interact with dozens of different open source programs on Windows, Linux and Mac OS.

But you say that if I have a problem with any of them, I should pay for it, code it myself, or get hired by the publisher? Where will I start? Bill Gates himself doesn't have the cash to pay for all the bug fixes we need. I contribute actively to two projects, and I don't have enough time to do a decent job on those. And I'm happy with the job I have. In any case, each of those would only allow an incremental change to one application.

What I feel is the best thing to do is to help everyone to do better. I don't feel that enough attention is paid to quality in open source. That's why I write articles like these and and this and this one..

To some extent the poor quality of many open source packages works out OK because so many of the users are committed because of their ideology. But what about people who have no such emotional commitment, who want to get work done and get on with their lives? People who don't know how to program, are frightened by talkback, and find bugzilla unfathomable? Are you suggesting that their answer is they shouldn't bother, and use proprietary software like Internet Explorer?

27 Nov 2002 (updated 27 Nov 2002 at 14:47 UTC) »

Updated: Please Understand That This is Not a Troll

Despite the fact that it got modded as off-topic, Please Understand Why My Wife Can't Stand Mozilla was meant in all seriousness to help the Mozilla developers succeed in their effort to win the browser wars.

I'd like to suggest that anyone writing software meant for regular users (as opposed to developers) would benefit from reading it.

(Updated yet again: filed this in bugzilla as #182221.

Embedded Systems Programming

... is certainly challenging. Recall my earlier diary entry where I said that despite kroah's warning I thought it would be really cool:

Personally, I would be really thrilled if I could get paid for six months in order to get a beeper not to warble. I've been in that position before, and I loved it. I miss it terribly.

Well, now I'm spending six months trying to get a beeper not to warble. Sometimes it's hard to remember that I'm supposed to be thrilled.

I proposed my current project to my client. I came up with the idea myself, but he has the hardware my idea would need and is in a position to market it.

He was skeptical whether it could be gotten to work, and suggested that before he committed to funding the product, that I write a "proof of concept" that would demonstrate the core features of the product without actually going to the effort of completing a finished product.

"Sure," I said, "no problem. That should take me about a week." So he agreed to pay the amount I'd normally want for a week for the proof of concept to be demonstrated.

Well, I'm going on week six now for my "proof of concept". There have been many late nights. My wife has been very distressed at it all, seeing economic disaster on the horizon. I have to say I've been pretty distressed too, but I've mostly I kept that to myself.

I had thought I could deliver it two nights ago, but once I got the software "feature complete" (for just the features required in this early version) the performance was so slow as to make the product completely unusable. I had planned to leave any optimization for the next phase of development, but I feared that showing what I had to my client would make him decide to cancel the project.

I stayed up round the clock up until late last night hand-optimizing my code, and although I got it to go much faster, it is still too slow to even be presentable, let alone to make a product we're going to charge money for.

Late last night I thought of a different approach I could take to my optimization, and I'm working on that now. I'm pretty sure I can get it to the point that my client will believe in its potential when he sees it. But there will be significant challenges remaining before it can be brought to market.

One thing I have to say, I'm much better at embedded systems programming than I was to start with. That's been part of my problem, not having too much of a clue how to debug complex problems when the target system doesn't have enough ram to support a debugger.

I had spent some time studying how to do embedded programming before I got actual work in it, but I found long ago that I learn most effectively (at least about programming) when I'm being paid to do what I'm trying to learn.

For one thing, I can afford to put real time into the study. An importantly, the sense of desperation that ensues when I don't understand how to do something that I will only get paid if I accomplish it helps me to develop a mental focus that I'm not normally able to accomplish. If I have to understand something difficult in order to continue eating, I'll do whatever it takes to come to that understanding. It's hard to develop that kind of motivation when I'm just studying something out of idle curiousity.

But it's been really painful. I had two individual bugs that took me a week apiece to fix, and one bug that took me two weeks. I had to spend a lot of time reassuring my panicked wife that my business wasn't going to fail, we weren't about to become homeless, because fixing those bugs was "right around the corner" - even when I wasn't so sure myself.

And getting everything done that I had planned to do for my "proof of concept", and getting it all working reliably, only to find it so horribly slow, was simply devastating.

Part of what keeps me going is the simple fact that I just don't have anything better to do right now. The alternative would be to just take some time off and do nothing. I am in discussions with several potental new clients, but setting up new work takes time. So despite all the unpleasantness, I might as well continue the work and do the best I can. At the very least, I'm learning a lot, and there is still the potential that it might pay off well in the end.

I'm sorry to be so oblique about what my product actually is. I know it's not at all in the Free Software spirit, but I want to get the product to market before anyone can start working on a competing product. I will be sure to announce it here once my client announces it on his website, and I'll put a link to his announcement. My current development plans call for the product to be announced in about two more weeks :-/

29 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!