Two interesting interviews.
How BioWare Makes Game Communities Work
An Interview with Scott Bakker
The Lone Software Developer's Methodology
Entertaining and interesting rant (with explanation).
When unit testers fail...
At some point over the last week, my nose unit tests for twill started
flailing. Yes, not just failing, but flailing. There
was so much output that it was tough to figure out exactly what was
going on. I spent a few minutes across a few days trying to figure
out what had changed; I'm pretty sure it was due to a nose version upgrade,
now, but I could never actually figure out what version of nose recognized
all my tests *and* still worked.
Whatever the proximal cause, it turns out I designed my unit tests
incorrectly. The tests are in modules (separated in individual
files), with 'setup', 'test', and 'teardown' functions in each module.
The latest version(s) of nose was discovering the 'setup' and 'test'
functions and running (as near as I can tell) 'setup' twice, 'test'
once, and 'teardown' zero times.
I'm still not sure if this is a bug; I couldn't figure out what the
behavior of nose should be, because it's not explicitly documented
on the nose page. (Maybe it's on the py.test page?)
Finally, I discovered 'setup_module' and 'teardown_module' and renamed
setup and teardown in all my tests. That solved my problems.
I also learned (from the nose documentation) that you could specify a
test collector in setup.py when using setuptools. So now 'python
setup.py test' will run all the tests.
The shocker for me in all of this was how uncomfortable I felt even
thinking about modifying twill without the unit test framework.
Even though I could execute the tests individually, I knew that I
wouldn't run them nearly as frequently as I do when 'nose' is working.
Automated tests are a really important security blanket now...
What users want, and what they'll get.
So, it appears that an increasingly common use case for twill is Web browsing.
When I started developing twill, I intended to develop a replacement for PBP.
PBP was aimed at Web testing, and that's what I needed myself.
Roughly, this meant:
- a simple scripting language, with an interactive shell;
- Functional cookies/formfilling/etc, so that interacting with a Web
site actually worked;
- a fair amount of assert stuff, like "code" (to make assertions about return codes) and "find" (to make assertions about presence of specific text/regexps).
Then I made a few design choices: to use python and python's Cmd for
the interactive shell, and to use a metaclass to automagically import
all twill.commands functions into the interactive shell.
This meant that all of the functions accessible from twill were also
directly accessible via Python, and the twill language docs functioned
equally well as Python language docs.
Thus, twill could be used from Python code to do most Web browsing
tasks that didn't involve JavaScript.
I was happy with this result, but it was largely unintended. I mostly
use twill via the scripting language.
However, the early press on twill was from people like Grig and
Michele, who talked glowingly about the ability to use twill from
Python. This has led to people wanting more functionality:
especially, sensible return values from twill commands. This, in
turn, has led to a bit of caviling on my part about this direction,
because I haven't really thought it through.
Anyway, the upshot is that I have to rethink my priorities for twill a
bit. I was going to focus on recording functionality and extension
management for the next release, but it seems like I should also work
on simplifying and documenting the browser interaction code. Given
the one-to-one mapping between twill.commands and the
scripting language, I don't want to add things like return values and
Python-specific documentation to the commands; perhaps I can satisfy
people with a browser-object interface...
The big surprise for me -- and it really shouldn't have been a
surprise -- is that people really seem to want to do Web browsing via
a trivially simple interface: go, follow, submit, and
get_page contain 90% of the desired functionality. mechanize
and mechanoid are serious overkill for this level of Web browsing, and
the fact that twill puts a high priority on "just working" (albeit at
the expense of customizability) probably helps contribute to users'
interest.
The simplest route for me to go is probably to work on two
applications I've been planning: twill-crawler, and mailman-api.
(Presumably the names give away the function ;).) Then I'll have some
idea of what people need for Web browsing; right now I'm feeling a bit
underplanned, so to speak.
--titus