Older blog entries for MichaelCrawford (starting at number 40)

My wife has scored a contract with a second client for her web design business. She is quite stoked.

Her first clients are a folk-rock band, and at the release party for their recently completed CD, the band members publicly thanked Bonita for doing their site. This prompted one of the other women at the party to approach Bonita to do a site for her store downtown.

Word of mouth really is the most effective advertising. The challenge is in getting it to happen.

I'm proud that I got Bonita into using cascading stylesheets. Now she knows a lot more about CSS than I do.

I'm probably going to do a little web application programming to help Bonita out on this job, writing a small program to allow the store owner to easily list her inventory on her site without having to know how to code HTML. Very likely there is some open source application that does this.

Brother, Can You Spare a Clue?

I have made considerable progress at improving the performance of my embedded application. Using a better algorithm (what I believe now is considered the fastest algorithm) in C got me a factor of ten improvement. Re-coding that in assembler got me another factor of three improvement.

The problem is, it's still not fast enough to meet the client's requirements. I need to increase the performance by another factor of three. I believe I can do this by tweaking the assembly, but it's not at all obvious to me how.

That's why I submitted an "Ask Kuro5hin" on the topic of ARM Assembly Code Optimization?

I felt the discussion would be more useful to others, and likely to result in more ideas that would inspire me to solve my problem, if I encouraged people to post performance tips for any architecture.

I mention in the article that I don't have the budget to buy all the books that have been recommended to me. One reason for that is that I rather dramatically underestimated the time it would take me to deliver the proof of concept, and I had agreed to do that work for a fixed price (being under the impression it would be so easy!). But the happy news is that I asked my client if he would be willing to help me out with the cost of a couple of books that people recommended to help me speed up my assembly, and he agreed.

After a couple hours of calling around (and being disappointed that online bookstores won't ship on Saturdays) I got a bookstore a couple hours from my house to put on hold for me:

My friend Dave Lyons, who recommended Hacker's Delight to me, said that he was quite suprised by what it showed you can get a two's-complement execution unit to do.

Embedded Systems Programming

I worked through the night, night before last, to speed up my code, and hit what I believed at the time was a brick wall. The kind of brick wall that would make it impossible to get my product fast enough to bring to market.

But after I finished up for the day and spent a few hours just relaxing I thought of a possible solution. I don't know if it will work, but it could work, and maybe if I can get my solution working at all some further work will make it fast enough.

I was eager to implement my idea yesterday, but I hadn't slept and was exhausted. It was very hard to bring myself to go to bed. I managed to, but I woke up at 2:30 am today and got up to go to work. Even though I had gone to bed in the late afternoon, it's not like me to wake up so early and go straight to work. That's how badly I want to succeed.

This embedded project has been a Trial by Fire. But it still has the potential to pay off well, and the painful experience I'm gaining should make future embedded projects a lot easier.

Writing

Zach Frey very helpfully wrote to me about my last diary entry where I discussed the tension between my struggle to make a living and my desire to do things that I felt were more important like writing and political activism.

He referred me to a speech C.S. Lewis gave to some British University students at the outbreak of World War II, in 1939. The speech is called Learning in War-time.

A Google search for "Learning in War-time" doesn't find the complete text online but there are a number of quotes available that I have found helpful:

The war creates no absolutely new situation: it simply aggravates the permanent human situation so that we can no longer ignore it,..

Human life has always been lived on the edge of a precipice. Human culture has always had to exist under the shadow of something infinitely more important than itself. If men had postponed the search for knowledge and beauty until they were secure, the search would never have begun...

We are mistaken when we compare war with 'normal life'. Life has never been normal.

I'm going to go look for the full text of the speech in the library and a couple bookstores some time in the next few days.

Zach also wrote about the importance of what I do in the way of making a living, that this advances the cause of Freedom - working independently as a consultant, and making what contributions I can to Free Software.

My long-sufferring wife understands why what I do is so important to me, and she supports and encourages me as best she can, but the way my other activities sometimes threaten to bring us closer to the precipice often frighten her.

I wish I didn't cause her so much pain when I take off to do something that I'm not getting paid for. I wish the way I do make my living were much more secure, so that when I am working billable hours, there would be more certainty in our lives. Usually I make enough money from my work that I can afford to take take off now and then to write, but it's never certain at the time I do it that my consulting is going to work out well.

Very proud to have done my small part

So I was hanging out on Slashdot for quite a while during the discussion of Because only terrorists can use 802.11 because I felt it was an important opportunity to get the message out about working together to oppose the terrible course the U.S. has been taking since September 11.

But my wife was distressed because things are very difficult financially and I need to get a bunch of hard work done in order to get ourselves out of this jam.

The more upset she got that I should be working instead of raising people's consciousness on the Internet, the more upset I got that she was trying to prevent me from doing something that I felt was of signficantly more importance than spending the night hacking some code.

We had a long talk about this eventually. It cleared the air quite a bit.

My wife is a very apolitical person. She prefers not to read the news because it just gets her down. She doesn't feel that the individual has much power to make a real difference. And she is very frightened about how we are going to pay all these bills that are overdue.

I, on the other hand, read the news pretty obsessively and am very frightened about what's going to happen a year or two down the road if Bush, Cheney, Ashcroft & Company succeed in what they are up to.

I worry that saying stuff like this could get me imprisoned someday - or worse.

Many of you might feel that that's an absurd thing to say, but I say that it's only relatively recently in history that many people at all have been able to speak their minds without fear of reprisal, and many, many people even today live in places where speaking out can mean a sudden disappearance.

I feel the only thing that I can really do to prevent that from happening here is to continue to speak out.

After reading some of the comments posted in reaction to posting the essay at K5, I feel that it's even more important that I continue to speak out. A couple of people raised the point that the Left is rather lacking in people able to speak eloquently. Or even coherently.

I feel that my ability to reason with people through my writing is a gift that I should be using for the common good.

The main reason I am a Green and no longer a Democrat is that the Democratic Party has shown itself to be completely spineless as far as mounting a credible opposition.

If my writing is able to convince even one person to stop what they're doing, to speak out, and to take action, then I think it is very important that I write. I think writing "Is This the America I Love?" is one of the best things I have ever done. I feel I need to do more of it.

But then there's the problem of all those bills. I need to keep food on the table and a roof over our heads. If I don't keep producing as a programmer, I won't have to worry about bootheels at our door coming to arrest me, but about the bank coming to repossess our home.

But I told my wife that if I had to give up doing the things that I felt were the most important to me just to ensure our survival, that there was just no point to even continuing, I might as well give up.

Our life has been a constant struggle the last several years. But if the only thing I could do was to struggle just to survive, and not to pursue the finer things, then it would break my spirit.

To give up my writing, and my activism, would be to lose the struggle. It would be worse than to be hungry or homeless. It would be to have no reason to live anymore.

There has to be something better than just struggling to get by. There has to be a reason for what I do beyond mere survival.

For me, that reason is to write.

But I wish it weren't so hard on poor Bonita. I really do. It breaks my heart to see her so frightened when I take a night or sometimes a few days off to write. No one pays me to write. Writing doesn't put any food on our table and it doesn't pay the mortgage - but I think it makes the struggle to get food on the table worthwhile.

Thank you for your attention.

I have gotten my embedded product working quite a bit faster than it was. Recall that the demo I gave my client was way too slow and it took some convincing to get him to continue funding the product, but he did eventually decide to continue paying me for development.

Unfortunately, it is still too slow to be marketable. However, it has become suprisingly usable even for the performance that it does have. Before it wasn't usable at all - I could demonstrate that it worked correctly, and measure its miserable speed, but that's about it.

I can't claim that much credit for the speedup. It turned out that there was a more efficient algorithm for what I'm trying to do, it gets the same results but is much faster. However there was the challenge of getting it to actually work in my firmware, which took a couple of days. I was very relieved when I got the new algorithm functioning and found that a significant improvement had happened (a speedup of about 20 times from what I had).

But I need to speed it up by another factor of 10 before my client will consider marketing it. And at this point he's only committed to paying me to improve the speed, more work will be required to bring the product to market and I have to hit a minimum performance threshhold for that to happen.

I have always been a big fan of tweaking code for performance. I have yet to find an optimizing compiler that I cannot beat by writing better C. And I am always able to beat my best C performance by writing in assembler.

Everyone says that you should only try to improve performance by selecting a better algorithm, but what if you are already using the best known algorithm and it's not fast enough. That's when you need to tweak your code.

Also some algorithms are only more efficient when they are used on large data sets, because they are expensive to set up. If you have many independent and small data sets, what would ordinarily be considered the best algorithm may really be a poor choice. In that case an algorithm that scales poorly but can be coded tightly may be a better choice.

I'm afraid that because I considered performance not to be a real concern for the proof-of-concept demo, I didn't even try to look for better algorithms when I wrote my demo, because it simply didn't occur to me that the straightforward and obvious way could have had such disastrous results. I've done some research now and I'm pretty sure that the algorithm I have now is the best available.

I don't think it will be too hard to write it in assembly and there are a number of ways that I can see that this will help. I should be able to get some performance gain easily. Getting enough may be difficult but I still have several days left of the time I have funded. I could take more time if I needed it but it would make things unpleasant financially again.

Again I'm sorry I won't say what exactly it is I'm working on but hopefully it will only be about a week and a half until the product is announced.

What Kuro5hin members are saying about my essay Is This the America I Love?:

Alright, who let this intelligent, well written, reasonable person into the Left?
-- wiredog

Michael better be careful, or people are going to start thinking that Greens are capable of rational thought.
-- wiredog

very well written and articulated, profound and thought provoking. Plenty of information and examples backing your opinion.
-- theforlorneone

Thank you for your attention.

5 Dec 2002 (updated 5 Dec 2002 at 07:09 UTC) »

Updated: softkid, yes, implementing the performance-critical part of my product in assembly is one of the things I'm doing. But it's not completely obvious what the best assembly is. So one of my tasks this week is to experiment with different alternatives in assembly code, and understand the performance of each that I get.

One problem is that I haven't done assembly in a long time, although I used to do a lot. Another is that I haven't ever programmed the assembly for this instruction set architecture before. I've done well at learning it so far, but of course I don't know all the clever tricks yet. I used to write some really odd 68000 assembly code, stuff that no one could figure out just by reading it, but that was significantly faster than the best I could ever get in C.

There are some tricks to what I'm doing that turn out to speed it up quite a bit, that I was unaware of when I wrote the proof of concept demo. I had known that there were some well-known optimizations, but didn't look into them because I didn't realize they would be necessary even for a proof of concept. I had no idea that the straightforward, obvious way to implement this would be so slow. But I am aware of some of the optimizations now, and am researching others.

I'm working on a new implementation in C that should be a lot faster even that what I had in my first attempt at assembly code. When I get that working well, I will have another go at assembly and probably should be able to improve a little on the C.

Again, I'm sorry to be so oblique about what my product is. But it's something I think you will all be happy is available. It won't be expensive, and maybe some of you will want to get one. Maybe I can even ship it in time for Christmas.

Happy News!

My client has decided to fund continued development of the project I'm working on for him. As I previously reported he was testing my proof of concept demo before deciding whether to go further, but the performance was unsatisfactory, not fast enough to make it a marketable product..

I spent several days explaining to my client how I intended to speed it up, but some of those ways will be quite difficult and would take some time, and things are so tight financially that I could not afford to just implement those changes and prove it to him by showing him working code. He based his decision on his faith that my plans would work out.

Ordinarily I enjoy doing optimization. I think it is the best part of programming. I have some tricks for optimization that some people think are quite twisted, but have worked well in the past.

In this coming week I face one of the most significant challenges of my entire career. I have to make my code go so much faster than it does now that I really honestly don't know if I can succeed. There are some tricks I could use if I were running on a desktop PC but I am programming an embedded device with very little RAM or ROM.

This week will get funded, but whether funding for the product continues to bring it to market depends on the results I am able to achieve.

It would be fun if there weren't so much riding on it.

Sometimes I am able to forget about the peril we are in and just enjoy coding. The whole experience has been very hard on my poor wife, who asks how I can be in such good spirits during a time of such turmoil. I tell her that is is like programming with the Sword of Damocles hanging over my head, and my only hope for survival is to write good code. But I cannot write good code while I am frightened, so I do whatever is necessary to put it out of my mind.

Embedded Systems Programming

My client is testing the test version of the firmware I delivered for him as I write this. I'm very anxious to hear his reaction, as he will base the decision of whether to fund bringing the product to market on how his testing of this proof of concept goes.

He had some trouble with his testing at first, but I was able to help him get the tests working after talking to him on the phone for a little bit. But I had to let him go work on his own because I had written up a complicated test procedure to go with some test tools that I gave him tonight, that would prove to him that the product actually did what it was intended to. I needed to let him study my test procedure and try it out.

It would really be a problem for me if he doesn't want to continue with the development. I have other work in the pipeline, but it would be a while before I could see a paycheck from any of the other projects.

29 Nov 2002 (updated 29 Nov 2002 at 22:09 UTC) »
Embedded Systems Programming

Recall my previous diary entry where I said how difficult my project has been, and how I have been facing some severe performance problems.

It took me quite a lot of work, but I have implemented the optimization I was contemplating. It was really hard to get the code working correctly with this optimization in place.

However, it seems to be working correctly now.

It meant working Thanksgiving night. I ended up staying up all night and worked through the day today. I had a bad bug today.

I'm running a long test to check the performance. It looks faster, but not as fast as I'd hoped. I still have some debugging code compiled in, which would slow it down some.

I told my client just now that I would run some more tests on a release build once I was sure I had it working correctly, then send him whatever I had. I'm not going to try to optimize it anymore for this delivery. It doesn't have to be fast - it just has to be fast enough to make my client believe I can get it fast enough in the final product. It at least had to be usable, even if unpleasantly slow. I'm just afraid that not I'm sure if it's usable yet.

Posted again so it stays in recentlog a little while:

29 Nov 2002 (updated 29 Nov 2002 at 14:18 UTC) »

Updated: Why I Write

New at GoingWare's Bag of Programming Tricks:

How to Promote Your Business on the Internet

Getting a Google search for software consultant to list my resume on its first page of search results didn't require me to pay a lot of money to a search engine placement service. I didn't have to do anything dishonest, like build a link farm. It didn't cost a cent. It did take time and a lot of hard work, but my method is so simple that anyone could do it - and if you did, the Web would be a better place.

Does Advogato save text that's being edited?

I had a crash in Mozilla while the above text was in a diary entry window. The crash was while emailing the above announcement to some friends. When I went to re-enter my diary entry, which I had previewed but hadn't posted, I found that my text was restored. Neat. Or is that a Mozilla feature?

A feature I have always wanted in a browser is a confirmation dialog: "Post form contents before closing window?"

31 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!