Older blog entries for chromatic (starting at number 149)

GOAL!:

I'm on holiday this week, visiting my parents. Part of being on holiday means not doing my day job. (Those of you who've seen me work or who have seen the output of my work know that this is pretty rare.) I do have most of my work resources with me -- my laptop, my phone, a notepad, and an Internet connection, but this week is devoted to something besides writing and editing.

One of my goals this week was to do more work on Everything. Though the company's long since out of business, it's still viable software. There are still plenty of improvements to make, and I'd like to use it for a couple of projects in the future.

To aid in my goals, I decided to write yet another standalone web server. Being a smart (read lazy) programmer, I used HTTP::Daemon. I could install Apache and mod_perl on my laptop and work through the configuration there, but this allows other people to debug Everything on their laptops and will help with a future goal.

HTTP::Daemon and CGI.pm have never played well together. With the GET request method, it's not difficult to populate %ENV appropriately. It's harder with POSTed parameters. One solution is to tie *STDIN. I've done it before. Another idea is to write your own CGI parser. Good luck. (I've done that too.)

This time, I took a two-pronged approach. Since Gisle had already explained the wonders of headers_only last July (thanks, Gisle!), I didn't have to work around HTTP::Daemon having already read the entire request. First, I modified Everything slightly so as to pass an initializer filehandle to the CGI constructor. Then, I overloaded the iterator (readline, basically) in HTTP::Daemon::ClientConn. With a few lines of code to read a line at a time (respecting buffers and reading from sockets), CGI now gets something it expects.

GET works, POST works, images work (and can be generalized to regular files easily), and cookies work. I'm pretty sure uploads work, but I haven't tested them. All of this in a hundred and thirty lines of readable code: EVServer.

That was a big boost to my mood. As it turns out, though, I was coming down with a nasty, nasty head cold. That explains why I've been wanting to sleep twelve hours a day for the last week.

Moods:

Warning, tangentially technical.

I'm starting to think there's a tendency towards depression or melancholia that's associated with some of the other traits that make gifted system administrators and programmers. The correlation to musical or linguistic ability is pretty evident. Likewise, there are trends toward problem solving, logical thinking, and medium to strong introversion.

It's tempting to simplify, but I have a hard time picking one of the options. Is this personality type drawn very strongly to this industry? Does the state of the industry magnify latent traits? Is adolescence prolonged in the types of people drawn to computers and technology? Does participating in these activities prolong adolescence?

For example, patching systems that must have perfect uptime during business hours means working after hours or on weekends. The creative energy it takes to solve a thorny programming problem can strike anytime: during a walk, a shower, or in the middle of a dream. Sometimes it takes wrestling with a bit of software or an API for several hours before it starts to make sense. A great deal of free software is written on non-bank hours.

Let's also not forget the sense of play -- online debates and flame wars -- that take up so much time and energy. Granted, I see part of that as normal masculinity (in the C. S. Lewis sense), but there are lots of "debates" that lack some of the simpler social skills.

I bring this up because I worry sometimes about my ability to relate to other people. I suffer from insomnia and the lack of a set schedule takes its toll on me pretty quickly. I've done a lot of travelling in the past couple of years, had a few really big life changes, and live in a land of overcast, rainy winters. I've been really edgy lately, and have lost a lot of motivation.

Granted, worrying if I'm crazy is a pretty good sign that I'm not... but sometimes I care a great deal about what kind of a world I'm helping to build. I want to make a place of peace, a place of learning, a place of healing, and a place that helps people to become better and more complete. It's a huge goal, and the best way I know of is to be a good example.

I don't feel like the best example right now... but I'm trying. Here's my list of goals for the next little while.

Eat more fruit and vegetables. Drink lots of water. Go to bed early. Get up early and go for a walk. Clean out your inbox. Call your parents. Tell the people you love and appreciate one more reason you feel the way you do. Teach someone else to do these things.

If you're like me and haven't been feeling fully-charged lately, play along at home.

Back to Java:

When I returned to computer programming during my senior year of college, I chose Java. The Internet was the Big Thing back then, and applet programming looked pretty interesting. I'd dabbled in C++, but it was a lot more than I wanted back then.

In my subsequent job as a printing and imaging guru (color laser printers), I wrote a couple of in-house apps in Java. This was where and how I really got into Linux -- my brother was a fan and showed me how much easier development was than on a Windows (especially 3.11) machine.

I realized that Java wasn't meeting all of my goals when one of my projects could have been implemented in a 10-line shell script instead of several times that much Java. This was back in the 1.1 days, before the 1.2 beta, and part of my dissatisfaction was also in the awkwardness of AWT. Of course, it was also before the language was retargeted at desktop development, then server-side development, and before Linux had "official" support.

After several years of Perl, I'm brushing up on Java again. Why? Secret work project.

I don't know that I'd do a big project in Java, but it's always good to exercise the ol' computer science brain by exploring some of the Other Ways To Do It.

Perl 6:

Having been nominated as a Perl 6 Testing Guy, I spent some time trying to get new Perl 6 tests to work. In the process, I edited some documentation and fixed some bugs in the debugging framework. My tests are still not working, but at least what the documentation promises is now available. Tomorrow, I'll have a better chance of tracking down what's wrong.

The moral of the story: developers cannot rise above the quality of their tools.

More Parrot Hacking:

With a little more research today, a little more hacking, and a lot of debugging help from my roommate ("wait, is that a void pointer?"), I now have two animated smiley faces bouncing across the screen. Here's the relevant Parrot assembler:

    new P0, .SDLAnim
    new P1, .SDLAnim
    set I0, 0
    set P0["filename"], "smile.bmp"
    set P0["blankfile"], "blank.bmp"
    set P1["filename"], "smile.bmp"
    set P1["blankfile"], "blank.bmp"
    set I1, 320
LOOP:
    eq I0, 320, LAST
    inc I0
    dec I1
    set P0["x"], I0
    set P0["y"], I0
    set P1["x"], I0
    set P1["y"], I1
    branch LOOP

LAST: end

I have several enhancements to make before I release the code. First, I need to separate the window creation from the actual image objects. That's easy -- just create another PMC and make a way to associate the images with the screen. I have a pointer reserved just for that purpose. Second, I'd like to figure out how to add a new function to the PMC. Currently, the PMC redraws the image on each coordinate assignment. That's not so good for speed. I think this has something to do with the 'invoke' opcode, but I'm not sure.

Finally, I'd like to figure out how to make my new PMCs work without having to be core PMCs. Yep, I'm running a hacked version of Parrot. If I can figure out how to load a PMC dynamically, I'm good to go.

This has been a good learning experience (and I'm very glad to have gdb at my disposal). It's also been a little frustrating, as my C is rusty when it's not ignorant. Besides that, I have a feeling I'm at the very fringe of Parrot development here.

It's nice to do something no one else has ever done, though, and to get it to work.

More Testing:

Moving sucks. Five weeks later, there are still boxes and books lying all over my office. At least my housemates aren't any better. It's really cut into my motivation and coding abilities, though. Let's not talk about non-day-job writings, either!

This was a productive week, though. I've published and obtained several excellent articles. I've written an article about building online communities to be published on Monday, and think it's pretty good stuff. (It's very similar to chapter 8 of the book, which was quite fun to write.)

Development-wise, I've started on testing the hard bits of CPANPLUS. Autrijus is extremely smart; he just wanted a couple of examples and he understands the technique I'm using to make it easier. In return, he helped me get Module::Signature working correctly. (Or I helped him, either way. :)

I've also added several more tests for Everything. With new features coming in, now's the time to get people used to testing. Stopping development until we had complete coverage didn't work so well, as Zenon and I both seemed to burn out. Adding tests for new features and bugfixes will hopefully work better. My current task is to test the base SQL interface so I can simplify and improve it -- mostly done.

The other coup was having my first patch accepted into Parrot. Okay, it was a documentation patch, but I can't seem to pass a document that needs editing without fighting the urge to do some editing. It was productive. Nothing wrong with that...

A little introspection, if you please:

One of my last acts at my current job is writing a server that speaks SOAP and/or XML-RPC. Another developer is writing the client, in Python. I've used XML-RPC before, but hadn't done more than read through the docs for SOAP::Lite (and reviewed a couple of chapters for the upcoming Programming Web Services with Perl book). It's quite impressive, though.

For performance reasons during development, we switched our internal protocol to XML-RPC. Writing a server to handle that was also very quick and easy with SOAP::Lite. Writing a forking server is not difficult, either. Where the Python client is multi-threaded, I'm using multi processes as Unix intends!

Since several users can use the application at the same time, we're implementing access controls. Different user accounts exist, with different permissions. Some accounts only have read access, others can update, and still others can delete.

I like the way SOAP::Lite automatically dispatches requests to a specified package, but building in access controls to every method is too much work.

I created a proxy object (and not in the Matt Sergeant sense) to which SOAP::Lite autodispatches. The real work is done in the server object. The proxy object actually implements the access controls, creating user sessions, checking magic cookies, and testing if the user has the proper credentials to perform each operation.

The remaining issue was to identify the accessable methods in the server object and to mark them with the permissions they need. Maintaining a separate list somewhere struck me as too much work -- I'd have to keep updating code in two places. That's insufficiently lazy.

What I did was to use Attribute::Handlers to create a 'Publish' attribute. Now any method I want to be publicly available needs that attribute. Publish also takes an argument, the necessary permissions. The attribute handler stores the name of the method, a reference to the method, and the permission string. The proxy object calls get_methods(), receiving this data structure. It installs wrappers around each of these methods in the proxy object itself. That way, they're visible to the SOAP::Lite dispatcher, and I only have to resume a user session and check permissions once.

Through this all, I'm pleased to note that my decompisition skills are improving. The Once and Only Once principle really pays off -- if you get in the habit of factoring out common code, it's amazingly easy to add new features. Nearly everything that is reusable is reused.

Finally, a legitimate reason to use attributes and typeglobs in the same program.

QA:

I'll quickly admit to having tasted the Software Development Kool-Aid. Having worked for big companies and small companies, as well as for myself and with other free software developers, I've never even seen a project approach the levels of organization, documentation, and reliability that standard "best practices" prescribe. (Here, I'm thinking of just about everything that isn't Agile. ISO-9001 made busy work for two or three people out of 60 in the manufacturing group, and everyone else just ignored it until the auditor came around. Why would that be different with programmers?)

It is, of course, hubris to think that books, articles, and example code will make the entire culture of developers pick up better habits. I believe in peer pressure, and I believe most people do want to do things well, but I don't buy the Manifest Destiny undertones of some of the "open source" discussions. I do have several metaphors (relating the education of my particular subgroup of programmers to software testing -- my focus this past year) which seem to be writing themselves. I do plan to continue to write (oh, the plans!), because that is having some positive effects.

Looking at the state of programming in general, I see a couple of trends. There are people who program as a job, and people who program as a passion. Neither class is intrinsically immune from panic mode, where things spiral out of control. There are also people who've had training (whether formal computer science, vocational mentoring, or classes) and people who've trained themselves. I've seen mediocre code and shoddy development practices along both axes, so I won't credit formal study any more than necessary. It's not as if you have to swear to use proper encapsulation to pass your midterm.

Looking further, there are people who ought to know better and should be reminded that Quality Counts, and there are people who've never heard that things can be improved and need to be evangelized. I want to reach both groups.

This is one of those jobs that no one really wants to do, though, even if they know how. Of course, there are plenty of people who want to contribute back to their community, but don't necessarily know how. It's a good match.

The trick, as I see it, for Perl and QA is this. How do we:

  • Make mistakes and take the arrows once and only once
  • Learn the principles of good QA from those mistakes
  • Induce those principles into the minds of the community
  • Develop good tools to make everyone else much more productive
  • Spread the wisdom to other communities

One of Schwern's goals for Perl 5 QA was to work out our bugs with Perl 5 so we'd have a better idea how to handle Perl 6. I think it's time to start in on Perl 6 with vigor. Hopefully, we'll get a warm reception. (No, this entire, poorly-organized personal journal was not all just an excuse for that photo. It's a nice benefit, though.)

Yes, I have scary plans... stay tuned.

Dear Customer, Write It Down:
p>I'm an XP practitioner. This means that, in my world, customers devise specifications by writing stories. Developers break those stories into the tasks required to implement them and base their time estimates upon those (adjusting for the actual time the previous estimates have taken). The customers order the tasks based on business value and risk. Software is delivered every n weeks, where n is a small number (two or three are fairly popular).

This has several advantages. First, it makes the actual work for each feature apparent to the customer. Second, it pushes the responsibility for technical risk on to the programmers. Third, it pushes the responsibility for scheduling and business risk to the customers.

Until tonight, I would have argued that another benefit is that it forces the customers to think about their requirements. After all, if something's not written on a story card, it won't be included in a task, will not be estimated, and will not be programmed. It effectively does not exist.

My current project has been moderately high on the risk scale, with the potential for big payoffs. I wasn't keen on it from a scheduling point of view, because there was also potential for failure. (The schedule was way too tight, and the business people have the nasty habit of claiming that nearly everything is of "highest priority".) We took it on anyway, because the reward would be amazing good will within an entire industry.

The schedule was and is tight, so that the first iteration doubled in size (already a bad sign), and the lowest-priority "emergency" feature will be delayed to the next iteration. Yeah, we're not even in the second iteration. We did hit the first iteration deadline just fine, and the customers have had the software for a while. Of the bug reports I've received, they've all been setup errors, easily fixed.

The problem is, I was celebrating the christening of the new wgz.org web server tonight, when I got a call at 8:30. Apparently, the customer's failure to give us an accurate specification several weeks ago has escalated into a crisis that means I "must" go into the office tomorrow to add a feature that never appeared on a story card. I do have plans for tomorrow.

My current plan is to go into work (and they are being "generous" in "allowing" me to come in at 9 am instead of 8:15, as is my usual habit). Because this feature was not on the story card, I feel perfectly justified in saying, "This is not an emergency. This is not my responsibility." I'm a nice guy, though, so I'll make the (small) database and code changes to store the data they want to collect. They'll have to wait until the next iteration (one week) to run reports.

To all of the customers reading this: If you ask me to build the software equivalent of a bicycle path, do not come up to me at the ribbon cutting and expect me to work overtime because you just remembered that elephants wander across it twice a year.

Real World Misbehavior:

I've released a new version of Test::MockObject. One of the nice things about documenting your code (or, I suppose, writing an article about it) is that if you find something you really don't want to explain, or cannot explain simply, it's usually a good sign that you need to rethink the bits in question.

Besides that, Ovid and his company have been using the module for testing their wacky deadline POS project. They've found some bugs too (a couple of them just after I did), and he has suggested some improvements that made their way into version 0.07. I'm pretty happy with the idea of next_call(), as it'll make some of my test code even shorter and easier to write.

It's also nice to have other people using my code. I like knowing that, for example, the printable display on a Slash-based site runs on code I wrote. As I told one of my students last night, "I like the idea that I've earned the right to use the free software I've been using."

140 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!