Secret Lives of the Programmer

Posted 28 Apr 2003 at 02:12 UTC by garym Share This

This is a "No one understands me" in the Advogato whiners category, but it's also a study and a muse in the in my ways of working because I am curious if the way I work, a cloistered midnight-do-not-disturb and eccentric-zombie-in-the-grocery-story mode, I wonder if this is cultural, generational, or if maybe it's deeply necessary and secretly the life of all the other programmers ...

Sam's Blog comments on Diego in a thread on the strange life of being a programmer. Says Sam,

I find that the best time for me to get work done has been between 10 at night and 3 in the morning or from 6 in the morning till about 10am. These have almost always been my most productive hours of the day.

I wonder if it's because our job demands intense visualization and concentration. You have several thousand tokens strung together like tinkertoys, all suspended in your mind as you rotate it from different angles, modify a few struts and then prop open a delicately balanced section while you try some inserts ... or like a Rubix Cube, only you have to be able to roll-back a dozen rotations if this next spin doesn't solve the puzzle ... and then the phone rings, or your collegue thrusts a newspaper article at you, or there's a canvasser at the door or ... and it all comes flying apart. After the interruption, you spend the next next hour getting your mental model back to that point (so you've now lost half an hour) and find there's not enough time left to finish the concept because it takes (by several testimonies including my own) about 3 hours minimum to get any useful amount of work done.

I found it prohibitively frustrating in the cubicell world. I could only really do brainless work during the daylight (like email or easy debugging or the endless design focus-group negotiations) so as to be ready for the interruptions, and then had to work unpaid overtime at night to actually get the work done; it looks like you make a lot of money until you tally up the unpaid hours, and then you find out you'd have been off fflipping burgers for Macdonalds.

We also appear to be magnets for interruptions. Sales and mangers are always writing or talking (mostly the latter, it's their job) and lab-coats are always jumping about with clipboards and parts and apparatus, but us, we're just sitting there, seemingly idle, not typing, not looking at the screen (not really) just kind of in a daydream as we try to spin that incredible tinkertoy Rubix cube three turns back and 5 to the left, no make that 6, no, 4 back and 7 no ...

No other occupation is like this. Maybe writers of murder mysteries. No one else understands how we can turn daydreams into the very fabric of our modern world. They all want instant lines of code and the conforting tap tap tap of wasted syntax robotically dumped to disk.

And this job, the dreaming, it takes a long, long time. It's there at the beginning: Coding is easy once you have the tinkertoy spun just right, code almost writes itself (limited by typing speed of course, never forget to factor your typing speed into your estimates), but just getting to that domino-tumbling point can take every waking moment for weeks. And the dreamtime is there at the end, in the really tricky blackbox-probing debugging, visualizing how your tinkertoy castle fits into the Apache group's castle, and why it now spins the Rubix rows that particular way instead of how it should.

Other than in the Zen-time of the pure tap tap tapping, the dreamtime is ubiquitous, it's not a process, it's an environment. Sometimes you can't speak because it's cluttering your verbal lobes. Sometimes, it's just there in the background like a daemon process, like a socket-listener waiting, waiting, waiting for the design to gell and click snap another tinkertoy into place. It's done in movie theatres, waiting in line at the bank, in the bath ...

Maybe it's different for the other master programmers trained in the art of coding. Maybe they just sit down like a carpenter with a woodpile and a sketch and just proceed and it's done, just like they thought, right on time, right on the mark. It's sure not that way for me. If I charged any client for the full measure of essential dreamtime, no one could afford my work. No one.

It's not that I don't see the value of collegues. I do. I depend on lifelines in email and instant messenger, I "practice armed defense daily", but I can't do this non-stop intravenous-IRC XP thing, it drives me bananas, it's worse than trying to code in the middle of a daycare. Maybe eXtreme Programming's dynamic-duo hives of Vonnegut twins is a generation-gap thing I will never quite understand; my way of doing this job is closer to the Zen proverb, "The planning is with the many, the decision is done alone," and for me, the decision is the design. I accept inputs only to a point, and I accept revisions during the same phase in many iterative cycles afterward, but to get that magic click where the coding falls into place, that final mental picture needs one and only one author.

I don't know about others (because I've been cloistered off away in the woods for nearly a decade) but I think that's why many of us prefer to be hidden away in time and/or space, left free of interruptions, where we choose our own best times suited to the specifics of each task, and where we will not have to suffer that social awkardness of being seen lost in deeply abstract contemplation for most of our day.

One interpretation of how programmers think., posted 28 Apr 2003 at 13:11 UTC by davb » (Apprentice)

Have you seen Reciprocality? Its an interesting look at how everyone thinks. The initial ideas were developed by looking at software engineers.

Reciprocality ..., posted 28 Apr 2003 at 16:21 UTC by garym » (Master)

It's going to take some digesting to get through this one (where did I put my I.Eng CogSci hat?), but so far, I like it. I like it a lot ...

Ritual junkies don't know they are junkies and so have no opportunity to exercise their will in the face of addictive behaviour. They get nasty when deprived of ritual or subjected to novelty. It's why programming, which should always be about solving new problems every time, degrades into ritualised bullying to "follow procedures" with empty outcomes in commercial settings.

The ritual is more important than the result, and most people don't have the cognitive capacity to be fluent with problem domains and understand programs holistically, as non-trivial systems must be understood. This is why the commercial software sector hasn't produced anything new since 1993 while freeware is heading for a Vingian Singularity. This website is powered by Linux.

flow, posted 29 Apr 2003 at 03:52 UTC by dalke » (Journeyer)

After the interruption, you spend the next next hour getting your mental model back to that point

That's a well-known phenomonon called 'flow.' Do a search for that and "psychology" and "programming." You might also stick in "Csikszentmihalyi" as he's written about the effect of creativity on flow. Similarly, read Rapid Development for references to studies on the effect of the work environment on software development productivity.

No other occupation is like this.... No one else understands how we can turn daydreams into the very fabric of our modern world.

Egocentric. As you pointed out, writers do this. As do other artists. Physicists. Mathematicians. Consider how little effect most code has on the world compared to, say, a presidential speech writer, or even a writer for Saturday Night Live, where a good turn of phrase can cover the world for years. Engineers have also turned daydreams into something which changed the world.

Programmers are not that unique.

If I charged any client for the full measure of essential dreamtime, no one could afford my work. No one.

Huh? Well, if the results of your essential dreamtime is worth it, then yes they could.

Maybe eXtreme Programming's dynamic-duo hives of Vonnegut twins is a generation-gap thing I will never quite understand;

Such biased words. There's an XP FAQ.

Won't pair programming interfere with "flow", which requires solitude and silence?

Pair programming appears to access another kind of flow, which is shared between the two people pairing. There's some talking, but not a lot of idle chitchat, and and it's easier to return to the state if someone interrupts you.

It asks for time, posted 29 Apr 2003 at 08:50 UTC by gilbou » (Observer)

When I start programming it is very slow at first. On the first hour, I have to concentrate a lot, then after two hours it is like I'm inside of it and working faster. After three hours and after, I am totally concentrated and working faster. Sometimes too I have to write down a kind of "algorithm" for the program I want to write and once done it just doesn't work sometimes. I go away and do something else, then when I come back to it, I write the routine at once and it works perfectly. It's like there is some needed time between the algorithm and writing it in C code (or Ruby since I do C && Ruby). I should spend more time writing down things before programming I guess.

I'm calmer at night, posted 29 Apr 2003 at 12:27 UTC by MichaelCrawford » (Master)

I'm posting this at 7:30 am, not because I'm up early, but because I'm up late.

I can't concentrate during the day. I'm much calmer at night. I always get my best work done between midnight and 6 am. Unfortunately I don't seem to be able to control whether it happens at all, so sometimes I just sit up all night wasting time, but if it's going to happen, that's when it's going to happen.

I talk more about this in On Creativity.

In Living with Schizoaffective Disorder I talk about how my sleeping patterns make it so that software consulting is the only way I could hope to make a living. I don't think I could hold down a real job if I tried.

Unfortunately being so inconsistent makes my consulting work a lot harder than I think it has to be. I've spent several years trying to figure out how to get myself to be able to focus more regularly. I've not had much luck at finding any answers.

Maybe it's no accident that my other favorite pastime is writing.

I don't think this is just programmers, posted 30 Apr 2003 at 10:50 UTC by chalst » (Master)

As MichaelCrawford hints, I think this is common across the field of creative composition. George Orwell has an amusing portrait of the sterotypical hack journalist, who is given a book review to write by his editor, spends all of the next few days in self-hating avoidance of the task until finally writing the damned thing over the night before it has to be handed in (can't remember the title, it's in his collected essays).

Not everyone follows this pattern: my wife (a linguist) can fit all the sitting-in-front-of-a-desk bit to daylight hours, plus occassional hasty scribbling down of ideas at odd times of the day when inspiration hits.

I find myself between the patterns, both for coding and research work: sometimes creative work goes well according to the normal work day, but sometimes I find myself blocked in a way that doesn't come unblocked until I hammer away at the problem in the early hours. From the creative point of view that's just fine, but I do think this pattern is rather destructive of balance in one's life, so I try to minimise this.

It can be done, posted 1 May 2003 at 18:06 UTC by mx » (Journeyer)

I used to need that 11pm -> early am to find dev focus, but I've been able to change that. These days, I can be focused any time of the day ... BUT I had to learn to tell people to piss off.

Interruptions are a focus killer for me though, so I usually log off of IRC, shut down my mail client, and dnd my phone. And then when people interrupt me in person, I tell them to go away. I am easily 10x more productive when I'm an asshole (which is how I justify it to my PHBs).

The other thing I was able to toss was caffiene, for the most part. Some years ago I was under the impression that caffiene made me more productive, but for me it was a farce. The effect of sugar and caffiene are short lived, and I've found a solid diet and more sensible rehyrdration go much further. The high/low cycles of sugar/caffeine whack productivity, so don't be fooled by that.

There's one other thing that I've found affects my motivation, and it's related to job satisfaction (which has many contributing factors). I'll guess that every person has different need in terms of finding a job that fits them, but I'll wager that if you hate your job that productivity will be NULL.

Us is not just us, posted 2 May 2003 at 18:29 UTC by garym » (Master)

Sorry for the confusion, I didn't mean to say that I thought this trait peculiar to programmers, but at best, as Michael points out, it's an occupation where there seems a high concentration, and where, until recently, there was even a demand. By 'us', I'm meaning what Reciprocality is calling "The Mappers" and even here I recognize in myself that I am a mapper in some domains, a packer in others, so it is a matter of degree and balance.

What seems apparent, though, is that it is difficult to shift from mapper to packer mode when the market demands change. It may even be impossible. Lord knows I've had my share of paying the dues gigs, but it's painful. I'm also not saying that I necessarily arrive at any genius solution which could not be acheived by careful packing strategy, I'm only saying that <u>I</u> couldn't get there by the packer road.

Tinkertoys, posted 3 May 2003 at 19:26 UTC by nymia » (Master)

That was a nice description of what goes on inside our vast mental space. I really liked the way how tinkertoys get constructed and modified in thinking-space, though. Yes, it is true that I find myself mostly sitting down looking like some sort in a trance. Not the real trance were one is totally spaced-out, not that one. But a coding trance where mental gears go super high and patterns of code hopping up and down, running along the vast mental landscape.

Regarding time, I noticed time has somewhat become a non-issue when I started writing down some of them, somehow the act of writing made them permanently stuck in my mind's eye. The important was being able to identify patterns of code, placing them in some sort of repository like a journal or database.

And this is probably why I will always be a programmer, it is so addictive that I crave the moment somewhat termed as trance. I tried looking for this kind of mental state in other work and it somehow felt like hell.

telepathy, posted 4 May 2003 at 06:34 UTC by badvogato » (Master)

Yo, reading The Go Modem Protocol and Go Text Protocol . Ease your breath, it's all about telepathy that in your breath connects immortal with a mortal fool.

us against them, posted 13 May 2003 at 22:38 UTC by carol » (Journeyer)

there was so much very very good information here, there is no way to digest it all. i would like to share with you something i have been noticing occuring here where i live. there is a lot of medication among the wealthy and the poor. the middle class and a few that can live in the extremes and escape medication are being bullied and battered and are disappearing very quickly.

i suggest that you give up your meds and become a problem again. carol

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

Share this page