Advogato blog for dan

Musical Interludes

Wed, 24 Feb 2010 17:10:39 GMT

Part II of the financial thingy is on its way, along with a subjective evaluation of the merits of rspec (I quite like it), but in the meantime have a look at http://github.com/telent/pacioli which is where the work in "progress" is going. I say "progress" because that's a daytime job and, well, something else came up at work.

In the meantime, please feel free to wander across to http://telent.posterous.com/ which is where I'm describing my attempts to put an embedded "Mini2440" Linux system in the Firebrox pedalled sound system

The Programmer's Guide to Financial Book-keeping, Part I

Tue, 9 Feb 2010 15:11:35 GMT

Once upon a time I knew enough about bookkeeping to implement a a rudimentary accounting system for the consulting business I was running at the time. Then I got a real job, and after that I forgot most of it. Recently I've had to relearn it all, and as the accountancy/bookkeeping web pages that I've found on the Internet are decidedly mixed (an honourable mention here for the Gnucash manual, which is actually quite good), this time I'm writing it down.

The intended audience for this is chiefly me and people like me: computer programmer types who have to make their systems talk to accounts departments and accountants. If you are looking for more information on bookkeeping or accountancy from a professional perspective, it is less likely to be useful.

It should not be necessary - though it probably is - to state that I hold no professional qualifications and have had no training in the field, and if you want proper advice you'll have to pay for it from someone entitled to give it. This information is offered as-is, and no warranties as to its correctness, usefulness or completeness are offered.

Feedback welcome - see the page footer for details.

Definition

Let us define bookkeeping as: the collection and processing of financial records for an entity, with the object that interested parties can learn (1) as of a specified time, how much money (and other valuable stuff) it owns, against how much it owes to other entities; (2) over a specified period of time, how much has come in and how much has gone out. Bookkeeping deals not just with money but with all kinds of valuable stuff: cash, shares, financial instruments, land, saleable equipment, stock in trade, etc etc - in the rest of this post I'll be lumping it all together as "value".

End results

In the UK, the end processes of bookkeeping/accountancy for a company or other trading entity are usually produced annually -

the Balance Sheet - a document of type (1), which lists the assets (stuff we've got) and liabilities (stuff we owe) broken down by category, at the end of the trading year. We start with assets, listed in order from most liquid (e.g. money at hand or in the bank) to least liquid (things we own that would be complicated to sell), then we subtract liabilities (usually ordered from short-term to long-term), then the bottom line is what we're worth. This is often referred to as the Accounting Equation: Assets - Liabilities = Equity although other people will say that Equity is really what the company owes its owners (e.g. the shareholders) so the equity will appear as a liability account and the equation is "Assets = Liabilities". Mathematically it makes no difference.

the Profit and Loss account, or P&L - also known in the US as the Income Statement. This is a document of type (2) which lists what's come in and what's gone out over the course of the year.

We probably also want quarterly reports for VAT (that's "Sales Tax" in other countries), and ad-hoc reports for credit control (we need to know who owes us money so we can chase them) and management accounting.

Derivation

Obviously, if your trading entity is you and you alone and there's no regulatory requirement on you to show anyone else the figures, you can choose any categories you like. But for most of us, there are accepted rules about the breakdown that people want to see, what you're allowed to assign to which categories, and what you'd actually want to assign to which categories (which might be a question with different answers depending on whether you're trying e.g. to maximise profit for the investors or minimise it for the taxman). This kind of decision is what you have an accountant for: keeping the numbers is what you have a bookkeeper for. So, look on the difference between those two roles as a policy/mechanism distinction (and a big difference in hourly rate: don't pay an accountant to do a bookkeeper's job)

Accounts and transactions

So, with the aid of an accountant we can establish how we need to categorise our assets and liabilities for the reports we need to produce. Each category (or sub-category, or sub-sub-category) is an account: each transfer of value from one account to another is a transaction. A transaction is usually associated with a source document (for example, a purchase order, or an invoice, or a receipt) - the so-called paper trail is not necessarily kept on actual carbon laminate these days, but it's still important. In essence, what we do is record the transactions.

Credit and Debit

We record each financial transaction as a flow of value from one (or several) accounts into another (or several others). Historically, bookkeepers don't get on with the concept of negative numbers - this is possibly because it can be confusing to have your "Income" account get steadily more negative as the year goes on (we'll come back to why this happens), or maybe just because the principles of double-entry bookkeeping were invented in a time and place (Renaissance Italy) that hadn't really yet heard of negative numbers. Whatever. But the upshot is that they made their own words up instead: the account that loses value is said to be "credited"; the account which gains value is "debited".

This is, of course, completely bass-ackwards from the perspective of normal people, though it has been claimed that the problem is that we're backwards. When the bank send you your statement of account it's printed from their point of view, not yours. So, if you deposit £50 in the Royal NatMid, in their eyes that creates a liability to you (after all, it's money they have but you own). The more money you give them the more they can transfer (debit) to Assets/BranchSafe or Assets/Vault or Assets/SubPrimeMortgages, but they have to credit that transaction to Liabilties/AP/YourNameHere. So the effect is that we perceive being in credit with the bank as a good thing: they see it as a bad thing. It's just a matter of perspective.

(I am slightly suspicious of this explanation. "Credit" and "Debit" are both from Latin roots: /creditum/: "a loan, thing entrusted to another", and /debilitum/: "thing owed," neut pp. of /debere/ "to owe". In the end they're just words, but it's still confusing enough to be just more fuel for my scepticism towards the claim that negative numbers are avoided because they cause confusion. Maybe that's just me.)

Double-entry

The principle of double-entry accounting is that the value credited in a transaction must equal the value debited - value cannot be created or destroyed. The name comes from paper-based systems: if we have two accounts affected by a transaction, we must enter the transaction details into both. Using a computer, of course, we can enter it once and it will appear in both, but that's not the point. We are interested in the principle of "conservation of value", not so much in the mechanism of how we achieved that in the old days.

A simple example: our shop runs out of float in the till, so on Monday we must visit the bank and get some more cash. We record this as a transfer from the bank account (which is credited) to the till account (which is debited).

Here we're moving value between two asset accounts: they both represent monies that we own - just in different places. So it's pretty easy to see that "conservation of value" holds true. But the principle of double-entry bookkeeping is that the debits and credits in any transaction must always balance, so the astute reader will now be wondering how we do that for a transaction that actually makes the company money. For example, if we do some work and get paid, then the value in the transaction is clearly going into the bank account, but where is it coming from?

Income and expenses

The answer is that we create "Income" accounts which serve as a proxy for the outside world as it affects our company. So, if we get $200 for configuring Joe's web server, our bank account is debited $200 and the outside world, as represented by our Income (or Income/Sales, or whatever subcategorisation we want to use) account, is credited $200. Expenses accounts serve a similar but opposite role: we pay for stuff (like stationery, utilities, salaries) that makes us poorer (our assets are credited) and the outside world richer (our expenses are debited).

Income and Expense accounts are key to the P&L statement that we will produce at the end of the year, because they act as summaries of our interactions with the world - which is what P&L is all about. The USAnian name for them "Income statement" hints as much.

Accruals

Another key concept is accruals. In most businesses there is a delay between when we provide something of value (e.g. do some work) and when we actually get paid: there is also often a delay between when we receive something of value and when we have to pay for it. In a cash accounting system there's nothing we can do about that, but in an accruals system we can create "accounts receivable" and "accounts payable" for these sums which are "in the post". This allows our accounts to say that we are worth $4000 because we have that amount expected to come in from Michael next week, even though we haven't got it in the bank yet. So, this makes payment a two-stage process: first we send an invoice and transfer $4000 from Income/Sales to Assets/AR/Michael, then when he pays it four weeks later (or perhaps four months later if he's a public sector body) we transfer $4000 from Assets/AR/Michael to Assets/Bank. We haven't actually made any new money in that second transaction, but at least it now exists in the bank and not just on paper.

Accounts Payable is similar but opposite. We order office furniture on account, it gets sent with an invoice, and we log that transaction as a transfer from Liabilities/AP/IKEA to Assets/Furniture. When the invoice is due (or three weeks later if you have really good credit control) we send them a cheque and we do another transaction from (crediting) Assets/Bank to (debiting) Liabilities/AP/IKEA, which hopefully reduces the balance of the latter account to zero.

Most of the examples later in this post ignore accruals in much the same way and for the same reason as Kernighan and Ritchie ignore error checking: it slightly obscures the pedagogical point, but that doesn't mean you won't do it for real.

Sale of goods

If you're selling services, the transaction is Income/Sales->Assets/Bank. That's simple. If you're selling goods, though, (1) you have to buy them first

£6 cr. Assets/Bank = dr. Assets/Inventory/Widgets

and then (2) when you sell them you are selling at a different price.

£10 cr. Income/Sales = dr. Assets/Bank
£6 cr. Assets/Inventory/Widgets = dr. Expenses/Cost of sales

The net effect is to increase Income by a tenner and Expenses by an unwell cephalopod (that's "sick quid" to you. Sorry). Thus both effects of the transaction will be represented on the appropriate P&L rows.

VAT / Sales Tax

VAT in the UK is not really ever money we have earnt, it's just money we are collecting on behalf of the nice people at HMRC. So, if we are registered for VAT we must collect it on each sale into a holding account which we send them later, but it's not "ours" and doesn't show in Sales.

£20 cr. Income/Sales + 3.50 cr. Liabilities/VAT = 23.50 dr. Assets/Bank

Watch out for the credit/debits in that transaction. We should end up with cash in the bank (a debit), some of which is owed to the VAT man (credit). If they don't sum to zero, you've done something wrong.

Similarly we can also claim back VAT on purchases from our VAT-registered suppliers

11.75 cr. Assets/Bank = 10.00 dr. Assets/Inventory + 1.75 dr. Assets/Input_VAT

At the end of the quarter, we pay HMRC what we owe them, less what they owe us

1.75 cr. Assets/Input_VAT + 1.75 cr. Assets/Bank = 3.50 dr. Liabilities/VAT

Note that this is not reflected in any Expense account - it shouldn't be, because it wasn't in an Income account to start with

Year end

We've already talked about producing the Balance Sheet and P&L. The other action we take at end of year is to close the accounts: in the case of Income and Expenses, we will want to start the following year with a clean sheet. How to do this: after producing the end-of-year reports, move the entire contents of Income and Expenses accounts into a summary "Retained Earnings" account, debiting and crediting as appropriate.

Contingent concepts

We have not talked about: journals, day books, cash books, general ledgers, T accounts, and trial balances. Most of these are historical practices that are necessary in manual systems either because the latency of entering everything directly in double-entry form is high (so transactions are initially recorded elsewhere instead), or because there is no automatic checking that the accounts are in balance, or because obtaining summaries of groups of accounts (answering queries like "what's the total AP for all suppliers") isn't a trivial bit of SQL.

Where next?

This is Part I of a two-part series. In the second part I'm going to write about my experience implementing all this in Ruby, but that will have to wait until I've done the actual implementation.

Hopefully though, this post should provide you with a view of the principles such that you can google for anything else you see and you have a framework to hang it on.

Oh. Ouch, Sorry

Sat, 23 Jan 2010 00:10:32 GMT

I've only just spotted this host didn't come back cleanly after last night's power-cut-induced outage. Well, here it is now

In the meantime, I have found the stunningly attractive Ruby FFI gem and written some rather gross code to play music through PortAudio with it. Unfortunately it turns out that PortAudio's Pa_OpenDefaultStream function is buggy (either the code or the spec) and it defaults to OSS anyway. Still, a step closer

Changing OSS in mid-stream

Tue, 19 Jan 2010 23:11:55 GMT

It was always a given that the Oliver OSS (that's "/dev/dsp" to you) interface would eventually need to be swapped out for something else, just because it doesn't work on many machines other than mine.

Now Oliver has reached the point that it functions as a rudimentary music player for general use (i.e. it presents me a list of my music on the left that I can drag into the playqueue on the right), I realise that "eventually" is sooner than I was expecting, simply because it opens the dsp exclusively and forces all other audio programs (e.g. youtube vids) to fail. In short, it doesn't work acceptably well on my machine either. Fail. I'd forgotten how miserable Linux audio used to be.

In other news

webkit doesn't support javascript 1.7, which is a grave disappointment as this is the one with proper lexical scope (via let )

minor edit to previous post for comprehensibility, as re-reading it I realised it made more sense in my head than on the page.

as tweeted: "there is a fine line between genius and idiot, and I'm still not sure on which side of it to place the jquery api". Selecting document elements using CSS syntax, yes, great idea. Returning those elements in an object sufficiently similar to an array that Firebug prints it as square brackets, but then making it respond to map in an entirely different way to the real Array - no, stupid idea. Hours of my life I won't get back

Testes on testing

Sat, 16 Jan 2010 01:12:49 GMT

Lest the reader assume from my previous post that I'm against automated testing: no, I'm not. In fact, Oliver hacking time over the last couple of days has been all about writing tests for the playqueue and turning the hacked together OSS interface into something that can pass them.

I offer for your consideration, though, that the benefits of writing a test suite are not so much about "having an executable specification" or even catching regressions as they are about making the software easier to test, by (1) decoupling interfaces so that units may be tested without a plethora of complicated test doubles (stubs, mocks) which may themselves contain bugs, and (2) reducing their dependence on complicated state. The easiest code to test is the purely functional, and happily this is also the easiest code to statically reason about (thus reducing our need to write tests in the first place).

There are other considerations: I object to the false confidence of "our change passes tests, so we can feel good because it must be correct" - though I suspect I'm fighting a straw man there anyway - and I am not sure that even attempting to write tests for some classes of bug (say, race conditions) is a more productive use of ones time than, say, Thinking Very Hard at the program to be tested. And there's the whole issue of whether the executable specification (probably written by a computer programmer) is a fair reflection of the business requirement (if written by the "customer", probably much less precise and probably not even consistent), but by that point we just have to accept that, well, TDD won't fix world hunger either. Chiefly my message is that writing (and more importantly, maintaining) tests is Not Free nor axiomatically Good, and therefore is only to be commended when there is some expected benefit from it.

I leave you with two final thoughts that are vaguely related but don't fit into the overall argument anywhere else...

First: correct me if I'm wrong here, but

Test-Driven Development is what, back in the XP days, they called "test-first programming": you write the unit tests before the code that they test. In Ruby, Test::Unit is/was the tool for running unit tests

Behaviour-Driven Development is an exercise in redefining concepts, to change the emphasis from unit tests (which typically operate on a particular class) to functional tests (which may span several classes, and address user stories or whole-system behaviour). In Ruby, RSpec was developed with this end.

Missing the point, therefore, must be the practice of editing all your Test::Unit cases into RSpec syntax to pretend more convincingly to the new orthodoxy. They don't magically turn into functional tests just because test_barf_when_value_negative is now written it "should barf when value negative". Er? Do they?

Second: a powerful driver for a good test suite, from my experience with SBCL, is that of checking the program is not broken by environmental changes (new os, new cpu architecture, whatever). When working on that project I saw test suite breakages far more often in those circumstances than I ever did from hacking the code they were supposed to protect.

Streaming XHR with Ruby and Mongrel

Fri, 15 Jan 2010 18:07:28 GMT

An entirely pun-free post title today, and still it sounds like something you'd see a vet about. Hey ho.

Suppose you are writing a web app in which the server needs to update the client when things change, and you don't want to do it by polling. It turns out there is a technique for this that is probably more than two years old: you make the client do an XMLHttpRequest (aside: that name is almost as bad as its capitalization) to the server, and then the server sends its response v e r y s l o w l y. The clients XmLhTtPReQuEst object will get an onreadystatechange event every time a new packet arrives, and just has to pull the new data out of xhr.responseText and decide what to do with it.

Well, that's the theory. There are a variety of more-or-less-documented bugs and pitfalls to do with browser compatibility, as there always are (google "Comet", there are lots of resources and none of them I have the personal experience to recommend), but the new wrinkle I observed when doing this yesterday was a quarter-of-a-second lag between the server sending and the client receiving. Odd. Ruby can't be that slow, can it?

Wel, no, it's not. After monkeying with wget and netcat and wireshark and mostly failing to find out what was going on, I did strace -e setsockopt ruby server.rb and connected to it, and lo, what should I find but that something in Ruby or something in Mongrel was setting the TCP_CORK socket option

setsockopt(5, SOL_TCP, TCP_CORK, [1], 4) = 0

TCP_CORK (since Linux 2.2)

Now I don't know where it's being set - a cursory grep of the mongrel sources says probably not there, but it's simple enough to unset again. So, here's one I made earlier. Note that you can't use the response.start method (or at least, I don't see how) - you have to reach a little deeper into Mongrel::HttpResponse

class StatusHandler < Mongrel::HttpHandler
  def process(request, response)
    fh=nil
    response.status=200
    response.send_status(nil)
    response.header['Content-Type'] = "application/x-www-form-urlencoded"
    response.send_header
    # something inside Ruby or inside Mongrel is setting TCP_CORK,
    # which is bad for latency.  I suspect Ruby C code, because 
    # the interpreter complains there is no definition for Socket::TCP_CORK
    # :#define TCP_CORK 3 /* Never send partially complete segments */
    response.socket.setsockopt(Socket::SOL_TCP, 3, 0)
    response.socket.setsockopt(Socket::SOL_TCP, Socket::TCP_NODELAY, 1)
    response.write("# gubbins for webkit bug "+("." * 256)+ "\n");
    response.write("# stuff follows\n");

    300.times.each do
      response.write("stuff\n")
      response.socket.flush
      sleep 30
    end   
    response.done
  end  
end

We limit the response to 300 lines in case of browser timeout or connection interruption or just to stop the client-side memory going up unboundedly as responseText grows without let or limit. It's simple for the javascript to kick off another handler when this one dies.

For completeness, here's some client-side code to go with it

// XXX we made the_req global just so that we can look at 
// what's going on in firebug.  It's not required in normal use
var the_req;
function json_watch_stream(url,callback) {
    var req =new XMLHttpRequest();
    the_req = req;
    req.open("GET",url,true);
    req.last_seen=0;
    req.onreadystatechange = function() {
	if(req.responseText) {
	    callback(req.readyState,req.status,
		     req.responseText.substr(req.last_seen))
	    req.last_seen=req.responseText.length;
	}
    };
    req.send(null);
}
function json_start_status_receiver (){
    json_watch_stream
	('/status',
	 function(ready,status,text) {
	    if(ready==4) {
		// server response concluded, need to start again
		json_start_status_receiver ();
	    }
	    text.split("\n").map(function(line) {
		    if(!line) { return; };
		    var data=line.substr(1);
		    switch(line[0]) {
		    case '#': break;
		    case 'O': update_track_timer(data); break;
		    case 'P': update_track_number(data); break;
		    case 'S': stop_track_timer(); break;
		    default: 
			console.log("status stream: unrecognised flag ",
				    line[0],data);
		    }
		});
	});
}

OSSes for Courses

Fri, 15 Jan 2010 18:07:28 GMT

I'd like to talk today about the state of audio output APIs that are implemented on Linux and accessible to Ruby, but there aren't any.

That's an oversimplification, I know. Oliver is now quite happily streaming PCM data to my headphones, so it's not even a good oversimplification. But whereas any of portaudio, pulseaudio, jack or alsa are the conventionally accepted routes to getting the speakers to squeak, what did I end up using? OSS. Yes, that's right, the Evil! Bad! Wrong! Deprecated! API invented by proprietary coders and left unmaintained for years that doesn't even support AC3-encoded 7.1 HDMI audio with reverse-phased channels and unlimited software mixing. Because ... well, because it Just Works. Open a file, send it a few ioctls for sample format and rate, and for the all-important "we don't care about latency that much" switch that lets interpreted code keep up without stuttering, then just chuck PCM data at it until we're out of PCM data. And it's quite well documented, and because it's all file-based it plays nice with Ruby's green threads: I just set the dsp stream to non-blocking mode and the audio keeps on going even while it's serving HTML and doing JSON calls.

Let me know when the massive ALSA DLL - or any of the plethora of other Modern audio libraries - becomes that simple to use (and preferably becomes usable without resorting to C), and I might give them another go. Yes, I am aware that under the hood the OSS API is implemented using ALSA drivers: how ironic.

Oh well. This isn't supposed to be the computer contrarian's blog, but some days it just feels like it.

http://www.googlefight.com/index.php?lang=en_GB&word1=alsa+audio&word2=oss+audio

Back after the break

Fri, 15 Jan 2010 18:07:28 GMT

If anyone reading this is presently as disillusioned with computing/hacking/coding as I was a few months ago, I cannot recommend Peter Seibel's "Coders at Work" highly enough.

Why my discontent? I don't trust anyone who elevates a programming practice (or indeed, almost anything else) to the status of a religion, and it seemed to me that "Test Driven Development" shares the important characteristics of UML, J2EE, pair programming, and "goto considered harmful" in that it promotes a high-ceremony dogma that is on the face of it really rather unlikely to save you time, on the professed basis mostly that it will make you a Better Person. And, presumably, get your reward in heaven.

But herein lies the problem. If you want to write stuff for a Unixoid platform right now, after discounting the really tedious heavyweight stuff like C and its variants your choice is basically Ruby. Or, at a pinch, Javascript. Common Lisp is - for all that I spent the last decade or so hacking on SBCL (more about that soon) - a platform not just a language, and a platform that nobody really wants to stand on; Perl is basically moribund; Python is dull and patronises its users (ESR likes it, which fits because he is also dull and patronising); the Schemes are fragmented and don't have momentum; and Java and C# - well, honestly they're out of the realms of everyday religion and straight into the flagellation: not my kink at all.

(Did I miss anyone?)

Anyway, Ruby is actually quite OK apart from the syntax (as a Lisper, I don't actually care what the syntax is, that it has one is the problem), it'll have acceptable performance in 1.9, and the lack of macros/programs-as-data is mostly - I don't say entirely, but mostly - made up for by a lexically succint notation for blocks, which do most of what you might want to do with them. (It's actually a minor source of amusement to me that Ruby folk are so mad keen on DSLs when they have to jury-rig them all into method syntax, but that's a story for another time). Anyway, if I can still remember my original point, I think it was that Ruby is currently the nearest thing around to a programming language that is both interesting and useful, but what about the TDD weenies?

So, back to Coders at Work. I bought it after it seeing that JWZ's interview had generated a certain amount of controversy with the unit testing dweebs, and reading it over the space of a couple of days I was immensely cheered to find that none of the other interviewees - interesting people, great hackers and luminaries among them - were blind adherents of that particular orthodoxy either. And, bizarre and pathetic as it sounds, that gave me the innoculation against TDD mind control that I thought I'd need in a group of Ruby programmers, so I have started (1) making some kind of an effort to get to the London Ruby Users Group meetings, (2) making a good-faith attempt to actually learn the language.

LRUG? Well, the first meeting was all about reinforcing the stereotype: a "Coding Dojo", or "what do you get if you try to scale pair programming". Answer is, you get mob programming: one active pair and ten to fifteen other onlookers. Which doesn't mean it wasn't thought-provoking , of course

The second meeting I turned up principally for the functional programming talk, which seemed to create quite a buzz in the pub (that's a good thing) and left me quite reassured that at least some part of the Ruby community "get it".

And learning the language? I'm writing a program. It has bits of Ruby, bits of Javascript, bits of audio coding (fairly noddy audio coding, as getting Ruby to run fast enough for anything more interesting looks like a challenge) and will eventually embed webkit. Yes, I'm writing an mp3 player/library manager, just on the basis that the downgrade from Amarok 1.4 to 2 in Debian has left me without a working music player that will export playlists to my phone, but that I already have a Twitter client. More on that as it progresses.

I'm never quite sure what "normal" is around here, but hacking-and-blogging service has been resumed.

[ posted a day later. But look what else I found since - Eleanor McHugh's On Agility rant. This is not exactly what I wanted to say about TDD and BDD, but it has themes I'm going to pick up on ]

Meet the new box (same as the old box)

Fri, 15 Jan 2010 18:07:28 GMT

My two-year-old Conmpaq NC2400 laptop came to an untimely end last month, as a result of being dunked in a rucksack with a pool of GT85 (long story). The screen developed slightly psychedelic oil bubbles which moved slowly around, the disk stopped working (or at least, stopped booting), and I really didn't fancy the task of stripping it down to rinse out all the electronics and still not know for definite that I could resuscitate it.

Ypou would think, wouldn't you, that after two years there'd be something better available at around the same price point (~ £600)? No. I took a long look at netbooks, which are all variously too small (1024x600 pixels? Don't be daft), too slow (Atom CPUs were never really the performance demon, even compared to my elderly U2500), and/or fail of VT (Atom again) and decided that the best thing to do was buy exactly the same computer again. Which I did, from Ebay, for £170

I still felt the need to do something to make it seem like an upgrade, though. As reviewers mostly seemed to agree that the bottleneck in the NC2400 was the 1.8" disk, I added a new 2.5" disk in the optical bay - never really used the DVD anyway - and migrated everything to it.

To do this you will need an optical bay caddy, such as the ones from Newmode US whose praises I will gladly sing. After having ordered the wrong caddy for my machine (in part due to an arguably misleading web page on their site, now corrected) I emailed them to explain and they sent out the correct one for the $8 price difference, without asking for the old one back or for the international postage to be paid again. This is service far in excess of what I was expecting.

A nice fast 2.5" pata disk, like the Samsung HM160HC also helps, that being the point of the exercise

The hardware assembly is fairly straightforward, when you have the right sized caddy. Watch out for setting the jumper on the drive before installing it: it shares an IDE channel with the original disk, so needs to be jumpered to "slave".

The software installation obviously varies according to what you previously had. When I put Debian on this machine it gave me an encrypted disk with LVM on top of it, so http://www.debian-administration.org/article/How_To_Migrate_to_a_full_encrypted_LVM_system was the key to working out what it had done and how.

The result according to hdparm:

/dev/hda: Timing buffered disk reads: 64 MB in 3.06 seconds = 20.91 MB/sec /dev/hdb: Timing buffered disk reads: 184 MB in 3.01 seconds = 61.15 MB/sec

The result according to my subjective experience: far faster and less laggy. Power consumption does for some reason seem to have jumped from 10W to 13W with concomitant decrease in battery life but I strongly suspect that's unrelated

Gnus over ssh

Fri, 15 Jan 2010 18:07:28 GMT

Suppose you had a broadband account with a static IP address (or small network) and an NNTP service, and that you wanted to read news from places other than the broadband network. Suppose further that your newsreader of choice is Gnus.

The intersection of all these assumptions is that you'd probably be me. In which case you already have incoming ssh access to the machine(s) on the broadband and all you need do is add the emacs magic to connect back from wherever in The Cloud(sic) you are through your home net to your news service

(defun nntp-open-via-ssh-stream (buffer)
  "Open a nntp connection by ssh to coruskate then then nc to zen"
  (let ((command '("ssh" "me.homeip.example.com" "nc" "news.myisp.example.com" "119"))
	proc)
    (setq proc (apply 'start-process "nntpd" buffer command))
    (save-excursion
      (set-buffer buffer)
      (nntp-wait-for-string "^\r*20[01]")
      (beginning-of-line)
      (delete-region (point-min) (point))
      proc)))


(setq gnus-select-method '(nntp "news"
				(nntp-open-connection-function 
				 nntp-open-via-ssh-stream)))

I was a bit surprised there's aparently no canned function for this. I crafted this by cut and paste from nntp-open-telnet-stream, which seemed simpler than trying to bend it to my will by editing the variables therein

Zen Internet come highly recommended as usual