The Emperor's New Mind

Recently a conversation on the subject of free will prodded me into reading Roger Penrose's The Emperor's New Mind, dealing with consciousness, its relationship to physics, and its computability, in a sense. The thrust is, as I understand it, that the human mind is in some sense not algorithmic, and that the strong AI position is untenable: no algorithmic system is capable of true consciousness. In pushing for this conclusion Penrose treats the nature of consciousness, its connection to basic physics, and such things as the nature of time itself, which is closely connected to my conception of the free will question. Having read the book, I can say that it is well-researched, and presents its mathematical and physical background lucidly; particularly with regards to entropy. Penrose's explanation of what entropy is and how it works makes a great deal more sense to me than the one I learned as an undergraduate (though given that it was taught me by physicists and Penrose is a mathematician, this should not surprise; physicists usually can't explain things even to themselves). However, I do not think Penrose made an entirely convincing case for his philosophical theses.

I would put my own positions thus:

1. I favour the strong AI position.

2. I don't believe in time.

As a consequence of #2, I have doubts about the existence of free will. Qualified doubts. I'll attempt to explain what I mean by 'I don't believe in time' in some more detail later, but for now, I'll stick to Penrose and his anti-AI arguments.

First, his speculations on possible quantum effects in the brain, and quantum gravity, and the collapse of the wavefunction are, as he himself takes pains to stress, speculations. Much like the 'many worlds' interpretation of quantum mechanics that Penrose admits to disliking. There are other possibilities. Which of them you buy is mostly a matter of personal preference: you're free to tentatively back whichever theory that fits the currently-known facts you find most aesthetically pleasing. When more data and more robust theories enter the picture, then we can settle this. 'When' in this case may be up to several decades from now. One need not argue with Penrose on these speculations, because he admits himself there's no evidence backing him up.

There are some things I can argue with. Early on Penrose cites John Searle's Chinese Room thought experiment. Imagine if you will that you are sealed away in a closed room, with supplies for an indefinite period; all you get from the outside world are strings of Chinese symbols, which you cannot read or understand at all. You have with you a great big fat book, in English (assuming English is your native language, which is very Anglocentric of me), containing very specific instructions, which you follow without error, for manipulating the symbols sent in to you, and extracting from any given string of Chinese symbols another string of Chinese symbols, algorithmically, which you then pop into another slot and feed back to the outside world. It just so happens that the symbols you're being fed are perfectly reasonable and grammatically correct questions in Chinese, and the symbol strings you're algorithmically producing are the appropriate replies. In essence, you, the room, the slots, and your rulebook form a computer, for answering questions in Chinese. Now, Searle asks, this system is capable of responding sensibly to certain inputs; can it be said that this computer understands what it's saying, even though you, yourself, haven't the foggiest idea? The strong AI position must lead one to say that it does, which Searle and Penrose both hope you will find absurd. I don't. If you don't, then there's nothing about the Chinese Room to damage your position. It's only an effective argument if you allow yourself to be intimidated by its silliness. If you think that consciousness is a process or pattern independent of its specific hardware, then yes, this Chinese Room is a sort of mind, while you're chugging through your rulebook. I think it's a reasonable position. To think otherwise is to make consciousness into a very priviliged thingy, which seems terribly anthropocentric to me, verging on the mystical. The idea of consciousness as a 'pattern' may or may not be more general than the idea of consciousness as an algorithmic process, which Penrose argues against but is unable to refute decisively. Or it may not. It depends on what I mean by 'pattern'; I am not sure yet.

Penrose also cites Godel's Incompleteness Theorem to argue against the algorithmic nature of consciousness. Any formal system contains a statement which is not provable within the framework of the system, but which, nevertheless, a human mathematician can recognise as a true statement. Therefore, it seems, no formal system or algorithm can describe the mathematical abilities of a human mathematician. This is, I think, misleading. Firstly, it is not at all obvious that strong AI requires consciousness to be algorithmic in the sense that Penrose claims. Mathematicians, after all, have been known to be wrong. (Graduate school would be much, much easier if this were not the case.) A consistent formal system cannot make mistakes. This would seem to imply, then, as Marvin Minsky pointed out, that mathematicians are not consistent formal systems. Since we are prone to error, our mathematical abilities cannot be described in terms of a formal system to which Godel's theorem applies: we are, like everyone's favourite rogue AI, Heuristic ALgorithmic systems, not merely algorithmic.There are other arguments against this, too. Some of them appear here, in a piece by Hans Moravec. For example, suppose mathematicians are algorithmic, but require about a hojillion axioms, more than our conscious minds can store and work with. We wouldn't be able to understand our Godel statement, or even recognise it as our Godel statement, because we couldn't grasp our own axioms; we'd have to call on some more powerful mathematical force (like a big hojillion-squared-axiom computer) to tell us what it was. My favourite counter-argument, though, goes like this...Godel guarantees us that, if our system is consistent--that is, if it is never the case that both a statement and its negation can both be proven true within the system--we have a statement P in our formal system which can neither be proven nor disproven. In a clever and coded way, the statement P is, in spirit, 'P is not provable.' Therefore, in any formal system, either the system is inconsistent, or it contains such an unprovable statement P. Therefore, if we assume we're consistent, we can prove within our system the statement 'P is not provable.' This is logically equivalent to the original Godel statement P. (For more details, I invite you to consult the appropriate Wikipedia article.) There are so many holes in this argument, that Godel somehow forbids AI, that I'm frankly surprised that a man of Penrose's intelligence would use it.

It's possible that Penrose might have been drawn to the Godel argument by his fairly extreme Platonism. There's nothing wrong with Platonism; I'm a mathematical Platonist, too. I'm just...not quite as much of one as Penrose is. He seems to assert in his book that the Platonic world of pure mathematics really truly exists, and that mathematicians make contact with it somehow when they get a really good idea¹. Godel's results would live in this Platonic realm. The physical world is a sort of echo of this mathematical world, imperfectly: so one can almost convince oneself that Godel's Incompleteness Theorems should find themselves echoed in the physical world along with all that crap about differential equations, and so one might be tempted to apply them to physical things like mathematicians, forgetting, alas, that mathematicians are not mathematics...

Penrose displays great erudition in his book, and he's an extremely intelligent man, but his position is painfully weak. He's like a man arguing theology: he takes the position he does not because he has solid, objective evidence that it is the correct position, but because he believes in it so firmly. He has no compelling arguments for his own position, and he cannot offer any compelling arguments against the opposition, like strong AI or the 'many worlds' hypothesis, beyond his own credibility as an expert, which ought not to get anyone anywhere in the physical sciences.

I will spare you the obligatory closing pun on the book's title. I'm sure you can make one up on your own.

***

1. Penrose, Roger. The Emperor's New Mind. Oxford: Oxford University Press, 1989. Page 428.

Posted by aloysius at June 23, 2003 08:57 PM | TrackBack |