It's been awhile...
I punted the lexer glue and am busily grinding through a rewrite of the macro expander. The goal here is not to forget about the tokenization of the original macro, but preserve as much of it as possible. This will dramatically reduce the amount of data that has to be copied around, reexamined, etc.
So far, object like macros work, and I'm starting on function like macros. [These are the terms the standard uses. It's like this:
#define foo bar /* object like macro */ #define neg(a) (-(a)) /* function like macro */
Function like macros take more work, because you have to substitute arguments. In the example above, a might be replaced by a big hairy expression.
I took some time out and stomped on about a hundred compiler warnings. We're now sure that string constants are always treated as, well, constant. I've also got as far as the Valley of the Dead in Nethack, which has never happened before (still only about halfway through the game, though).