Older blog entries for cdent (starting at number 486)

There is a whiff of nostalgia in this sort of warning, along with an undeniable truth: that in the pursuit of knowledge, slower can be better. Exploring the crowded stacks of musty libraries has its own rewards. Reading — even browsing — an old book can yield sustenance denied by a database search. Patience is a virtue. Gluttony a sin.

In every case, Bennett found, heat dissipation occurs only when information is erased. Erasure is the irreversible logical operation. When the head on a Turing machine erases one square of the tape, or when an electronic computer clears a capacitor, a bit is lost, and then heat must be dissipated. In Szilárd's thought experiment, the demon does not incur an entropy cost when it observes of chooses a molecule. The payback comes at the moment of clearing the record, when the demon erases one observation to make room for the next.

Forgetting takes work.

The telegraph could, of course, save many keystrokes — infinitely many, in the long run — by simply sending the messge "π." But this is a cheat. It presumes knowledge previously shared by the sender and the receiver. The sense has to recognize this special sequence to begin with, and then the receiver has to know what π is, and how to look up its decimal expansion, or else how to compute it. In effect, they have to share a code book.

For Kolmogorov, these ideas belonged not only to probability theory but also to physics. To measure complexity of an orderly crystal or a helter-skelter box of gas, one could measure the shortest algorithm needed to describe the state of the crystal or gas. Once again entropy was the key.

A simple object can be generated — or computed, or described — with just a few bits. A complex object requires an algorithm of many bits. Put this way, it seemed obvious. But until now it had not been understood mathematically.

But why do we say π is not random? Chaitin proposed a clear answer: number is not random if it is computable — if a definable computer program will generate it. Thus computability is a measure of randomness.

This pair of questions — how random and how much information — turn out to be one and the same. They have a single answer.

Ideas have "spreading power" he noted — "infectivity, as it were" — and some more than others. An example of an infectious idea might be a religious ideology that gains sway over a large group of people. The American neurophysiologist Roger Sperry had put forward a similar notion several years earlier, arguing that ideas are "just as real" as the neurons they inhabit.

She devised a process, a set of rules, a sequence of operations. In another century this would be called an algorithm, later a computer program, but for now the concept demanded painstaking explanation. The trickiest point was that her algorithm was recursive. It ran in a loop. The result of one iteration became food for the next.

Unfortunately the written word stands still. It is stable and immobile. Plato's qualms were mostly set aside in the succeeding millennia as the culture of literacy developed its many gifts: history and the law; the sciences and philosophy; the reflective explication of art and literature itself. None of that could have emerged from pure orality.