Rethinking Embedded Systems Development
I've been doing embedded systems development for a little over a year now. There has also been a little OS X and Windows driver development, to handle the host interfacing to the widgets I was working with.
It's been mostly FireWire and SCSI protocol and driver programming for storage devices (the packet-level part of the SCSI protocol is widely used these days over a wide range of physical cables; on FireWire one transports SCSI via the Serial Bus Protocol 2.)
I'm starting to wonder whether this has all been such a good idea. I need something to be different about my work, or I'm going to reach the point where I can't do it anymore.
I was getting to the point up until a couple days ago where I was getting ready to post a diary entry that was going to say "I'm ready to do any work at all that doesn't involve computers". But now I'm thinking maybe writing code would still be OK, I just need to be doing something different than what I'm doing.
Early in my career I quit my job doing tech support, and started applying for any job I could find that didn't involve computers. I applied to work on a loading dock and as a boat builder.
During an interview to be a mechanic at a bicycle shop, the shop owner told me that what I really should do is go back into computers and pay my dues. He felt that if I did so I would have a much more rewarding career than I would as a bicycle mechanic. I was really disappointed, but in the end, that's what I did, and I did have a rewarding career, that has lasted sixteen years so far. But now I'm back in the position where I think it would be really neat to be a manual laborer if it meant I didn't have to write code anymore.
I've been completely dead in the water for about two months now. For some time I've been doing "fixed-bid" contract work, meaning that I only get paid for completed work. For small jobs, I just get paid when I'm done. For big jobs, I get paid in increments when I reach certain milestones.
I completed a big project just before I moved, and got my last paycheck for it. Since then I've got nothing done, and haven't earned any paychecks either. I'm fortunate to have some savings otherwise I'd really be in a fix. It's probably for the best that I'm doing fixed-bid, because if I was working hourly I would certainly have gotten fired by now. Even so, my client really does need me to get this work done soon.
It's gotten to the point where trying to work feels like trying to push water uphill. On a good night, I spend a lot of time at Slashdot, Advogato and Kuro5hin, maybe listen to a little Radio Paradise. On a bad night, I sit staring at an empty source file window, with a feeling that starts out as a vague uneasiness but eventually moves into outright panic.
For a while now Bonita and I have been discussing how I need to get out of programming. Not many people work as actual coders for as long as I have. I'm pretty sure I'm much older than most of you (I'm 38). It's pretty rare to see an employment ad seeking a senior software engineer that wants more than seven years experience. Only once in my life have I ever seen an employment ad seeking a programmer with fifteen years experience.
Most coders will move into other computer industry jobs such as marketing, management, project management or sales by their mid-30's. I've never wanted to do anything but code - anything in the computer industry at least - because none of those jobs appeal to me. I don't expect I would be competent at them either.
Sometimes I think about going back to school. At first I was thinking about getting my PhD in physics. That was what I was aiming for when I was studying at Caltech twenty years ago. I wanted to become a professor so I could teach and do research. I actually was in grad school for a little while ten years ago. All the other grad students thought I was out of my mind to want to give up a career as a software engineer to become a physicist, but I was very happy to be in grad school.
I like to write, and I'm good at it. I've written here before about how I might like to write professionally. Sure, it's hard to make it as a writer, but I expect that if I persisted I could do it.
For a month now I've been playing piano again. I used to compose for the piano, back when I played regularly, but it's been ten years since I've really played enough that I could do that. But I've been steadily practicing two hours a day now, and getting good again, and have started improvising again. The way I'm able to compose is that I just improvise for a while each day, and every now and then a new song just jumps out at me.
So, as if physics and writing aren't unlikely enough careers, I've started to entertain the idea of being a musician and a composer. I think I could, if I studied long and hard enough, but even if the music I could write were to become popular, it would easily take ten years of study to get good enough that that would be a possibility. Even so, lately I've been thinking that I would prefer that to software engineering.
Did you know that Philip Glass didn't major in music when he was an undergraduate? Yet he is one of the most creative and forward-thinking composers of our time. His bachelor's degree is in mathematics and philosophy!
Last night Bonita noticed the despair that I was experiencing and spent some time discussing it with me, trying to get at the root of my problem. I didn't know what was wrong, just that I had been feeling sad all day. I didn't think I was sad about anything in particular, I just had spent hours thinking about all sorts of sad things. I spent a lot of time thinking about something sad that I'm going to post here in a future diary entry.
Somehow Bonita managed to worm out of me that I was contemplating a career in music, and hadn't quite managed to break the news to her. This caused her quite some concern.
I went on to remind her that when I left Caltech in 1984 I planned to become an artist, and intended to major in art when I transferred to UC Santa Cruz in 1985, but for some reason that is still not clear to me, I lost courage and continued to study physics. I eventually got my physics degree, but it took me from 1985 to 1993 to complete the last two years of study. I also took up the piano right when I dropped out of Caltech, but gave it up shortly after I left grad school in 1994.
Yes, she was distressed, but not as much as you might think. Bonita has a degree in marine biology from Memorial University of Newfoundland, and is just a few courses short of a second bachelor's degree in agricultural chemisty from Nova Scotia Agricultural College. But for several years now she has been studying art. She has
some of her work online. In January she's going to start taking some classes at
NSCAD University, formerly the Nova Scotia College of Art and Design.
I had encouraged her to quit her job at a biotech firm in California, and pursue her love of art, in part because I saw how talented she was, and how much she loved art, and in part because by doing so I could give her the chance I never had. I promised that I would provide for her during her study and until she could get her art career established. I fully expected that would take a long time, and that there was the risk she might never be a success.
So what Bonita told me was that if I wanted to become a musician, she would support that. She's known for quite some time now that my days as a programmer are numbered and has been trying to help me find a way to change careers. But she's not yet able to provide for the two of us, and I expect it would take even longer for me to become a professional musician than for her to become a professional artist.
What she asked me to do was to make a plan. I needed to find some way that I could eventually get what I want (whatever it might be, my ambition seems to change almost daily) in such a way that we are able to continue providing for ourselves.
She felt that if I had some credible plan in mind, and was working steadily to carry it out, then I would be able to tolerate continuing to write code for the time it took to pay off our debts, save up money and learn to play and compose for the piano like a pro.
Well, that made sense.
She suggested a compromise to what I've been doing lately. Recently I seem to always be able to find the time to practice my piano, but have a much harder time getting my work done. She suggested I alternate each day. One day I would devote myself to my music, the next to my work. I am a good enough programmer, and we live modestly enough that if I were able to work productively during the time I was at my desk, we would be able to get by with me working every other day.
I thought that would work but honestly all I could think of was how I would get to spend all day playing the piano without having to even sit at my desk.
But I decided that today, the first day of my plan, that I would work at my programming. I did that mostly to set Bonita's mind at ease. The last couple months of me pushing water uphill have not been easy on her nerves.
I'd been trying without any success to write a small Windows application that included a device driver. I decided I'd give up on the driver for now, because I wasn't getting anywhere on it, and just work on the user interface for a while.
And I'd been sitting at my desk hacking away at some C++ GUI code, actually getting a lot done, feeling
flow for the first time since before I left Maine, when I said to myself:
Hey, this is pretty cool!
And it occurred to me, maybe it was a mistake that I ever went into embedded systems development. I still want to find something else to do, but I think if I have to work as a programmer, I'd rather be doing graphical user interface programming.
GUI is the main work I've done since 1990. It's only in the last year that I've done embedded. I did a little operating systems work in the mid-90's, but that was Mac OS system software, so what I was really doing was working on the core of a GUI system.
I have a problem though. I had a lot of good reasons to move from GUI to embedded, and all those reasons still apply.
Primary among them are my age and my style of programming. It is very uncommon to find GUI developers with fifteen years experience. On the other hand, age and experience are valued very highly by the sort of people who hire embedded developers. I started to think about embedded around the time I started getting told I was "overqualified" by potential employers, but then had an interview at an embedded systems manufacturer whose youngest programmer had twenty years experience.
My style of programming was a constant source of friction between my clients and myself the whole time I've been a consultant. People who pay to have GUI applications written want the work done quickly and cheaply, and are willing to tolerate poor performance and buggy code. If you don't believe this, consider for a moment the performance and quality of the products published by the market leader in GUI applications.
I write good code, but one thing I don't do is write it quickly. I take the time to write it well. My code is reliable and performant. I've worked hard throughout my career to improve the quality of my code. Unfortunately, I've never been one to implement a lot of functionality quickly or cheaply.
I've actually tried to cut corners and churn out crap code quickly, but somehow I'm not able to work that way. It's not like I can just type faster if I stop taking care to watch for mistakes. I'm not able to work at all if I'm not paying attention to detail.
So the other reason I went into embedded is that reliability and performance are valued by people who pay embedded programmers in a way that they aren't valued by those who pay GUI programmers. At best, poor quality will make an embedded product fail in the market. At worst, poor embedded software can get people killed.
Performance is more important in embedded work because the people who pay to have the software written are also paying to manufacture the computers the software runs on. If Microsoft had to sell you a brand new computer every time you bought a new word processor or spreadsheet, you can be sure that performance would be microsoft's top concern. It's cheaper to manufacture slow hardware, and to use less memory. Such hardware also consumes less power, yielding longer battery life and generating less heat. It's quite common these days for embedded programmers to write code for 50 Mhz processors with sixteen bit busses. There are still lots of applications for eight bit processors.
It has been my experience since I've been doing embedded work that my clients are happier with me. To a large extent that's because they support me in taking the time to get it right.
I had a hard time learning to do embedded work. There was a lot to learn. One target I programmed had 1700 BYTES of RAM. But I've been doing it for a year and to the extent I can focus effectively on my work I do well at it now.
But I think I've realized that I just don't like it. I was thinking for quite some time that I didn't like any sort of programming. But I don't think that's the case, I just don't like doing embedded work, or writing device drivers. I still get a thrill out of doing graphical user interface development.
So I'm in quite a quandary right now. Bonita wasn't too happy when I shared this with her. This last year was the most difficult of my entire career, in large part because I had a really hard time figuring out how to get things to work right. It's been my worst year financially since I've been working as a consultant. But she stood by me because she agreed with my reasoning as to why I should become an embedded systems developer.
When I said to her that maybe it's all been a big mistake, maybe I should instead have told her that I was having an affair, or had just enjoyed my first hit of crack cocaine. She was very upset about it.
I was shocked at her reaction. I had expected her to be overjoyed that I finally found a way I could work productively. But Bonita remembered very well all the reasons I chose embedded, and she remembered even better all the clients who made my life so miserable when I was writing GUI applications.
Despite the difficulties of the last year, one remarkable difference has been that all the clients I have had have been very good to me and have been happy with my work.
I'm not real sure what to do. I expect that if I tried, I could get GUI contracts again, but could I find clients who were willing to pay a premium for reliable, performant code?
The one solution I am able to think of is that one can do GUI and embedded programming at the same time. The market for that is actually booming right now, with new handheld computers appearing every day. Cell phones are becoming a significant platform for GUI applications.
That's actually more like what I had in mind when I decided to go into embedded. I have a good friend who does Palm programming, and he just loves it. The reason I ended up doing FireWire is that the first embedded contract I ever got was for a FireWire storage vendor, and once I had that work on my resume it was easy to get more FireWire work.
It's probably not best to have a particular career because one stumbled into it. I do think FireWire is pretty cool, and have enjoyed learning all about it. But maybe it's not really for me.
I appreciate any advice you may have for me.
Bonita wants me to wrap up for tonight. I'm very excited though. You know why? Because tomorrow I get to play piano all day. There have been some things I've been wanting to try, but have never been able to get to, because I play dozens of finger exercises every day, and by the time I get that done I'm out of time to do much else with my piano. I've been persisting with the exercises because I can tell they're helping me learn to play better.
In January I'm going to start taking piano lessons again.
But I'm not going to get very far is a pianist if I can't work as a programmer. It's hard to find anywhere to keep your piano when you're homeless.
Thank you for your attention.