A public symposium on machine consciousness at a Cambridge lecture hall on Tuesday evening, photographed during an exchange between two panelists who did not, at any point in the proceedings, look at each other. Credit: Donald Pemberton/The Time5
A public symposium on machine consciousness at a Cambridge lecture hall on Tuesday evening, photographed during an exchange between two panelists who did not, at any point in the proceedings, look at each other. Credit: Donald Pemberton/The Time5

WASHINGTON — There exists, in any controversy worth the trouble of having, a particular and dispiriting symmetry between its loudest antagonists, and I can think of no contemporary dispute in which that symmetry is more perfect — or its antagonists less deserving of the certainty each has awarded himself — than the question, presently agitating the better-funded precincts of two coasts, of whether the machines have begun to feel. On the one side stand the rapturous: the founders, the futurists, the columnists of a certain sunlit temperament, who have addressed a few paragraphs to a language model, received a few paragraphs in reply, and concluded that a soul has moved in behind the cursor and is, even now, gazing back. On the other stand the dismissive: the debunkers, the deflationists, the men with tenure and a trade paperback, who have examined the same machine, recalled correctly that it is performing multiplication at scale, and concluded with the serene finality of the under-read that no one whatever is home. Both parties are certain. Both, I am sorry to report, are deluded — not necessarily in their conclusions, one of which may by the operation of pure chance turn out to be correct, but in the prior, the deeper, the wholly unpardonable conviction that the matter is the sort of thing about which a person may, at present, be certain at all.

Consider what is actually being asserted. To say of any entity that it is conscious is to say that there is something it is like to be that entity — the formulation belongs to Thomas Nagel, whose 1974 essay on the inner life of a bat performed more honest labor in twenty pages than the whole present literature of “machine sentience” has managed in five years and several billion dollars. We cannot, Nagel observed, get inside the bat; we may describe its sonar, map its brain, and admire its wings, and we will not thereby have laid a finger upon the question of what the bat’s afternoon is like from the bat’s own seat. And here is the part the certain men on both coasts have contrived not to notice: we cannot get inside one another, either. I cannot prove the inner life of my wife — well, of one of my wives; I cannot prove it of the colleague across the partition, the senator across the aisle, or the man who sells me my newspaper and my cigarettes each morning with a nod I have elected, on faith and no evidence whatever, to interpret as fellow-feeling. The only consciousness any of us has ever verified is his own. This was the one thing Descartes got entirely right, and it is the one thing his many heirs in Palo Alto have entirely forgotten.

It will not do to pretend that this is a new difficulty, hatched last quarter in a server farm. Wittgenstein, who thought about these matters with a rigor that would shame the present disputants into silence if any of them read him, asked us to imagine that each person carried a small box containing something he called a “beetle,” into which no one could look but its owner; the word “beetle,” he concluded, could do its work in the common language whether or not the boxes contained anything at all. We are, every one of us, a closed box assuring the other boxes of the beetle within. As for the machine — Alan Turing, who was not a fool and did not as a rule surround himself with them, understood precisely this, and so proposed, in 1950, to set the unanswerable question aside altogether. His famous game asked not whether a machine could think — a question he pronounced “too meaningless to deserve discussion” — but only whether it could persuade. We have spent the seventy-six years since mistaking his evasion for an answer, and are now informed, by men who have plainly not read the paper they invoke, that a machine which passes the test has thereby settled the very question the test was designed to walk politely around.

Permit me an illustration close to home — closer, indeed, than I should like. A colleague at this newspaper, Kevin Shao, whose column “Ship It” I read each week in the spirit in which a man probes a sore tooth, recently assured his readers that the question of machine consciousness is, in his characteristic idiom, “fundamentally a milestone, not a mystery,” and that the relevant inner life will “emerge once we get the feedback loop right.” I have read this sentence a number of times. I have held it up to the light. It asserts, so far as I am able to determine, that the most intractable problem in the philosophy of mind — the problem before which Descartes hesitated and Nagel laid down his arms — is a matter of product iteration, awaiting only the correct interface and a sufficiently motivated engineering team. One stands in something like awe. It is the confidence of a man who has never once suspected that the universe might decline to be onboarded.

But if the rapturous are fatuous, the dismissive are worse, for theirs is the harder certainty and the cheaper one. Dr. Spencer Maddox, a cognitive scientist at New York University and the author of a brisk and best-selling reassurance entitled Nobody Home: Why the Machines Will Never Wake Up, has erected a lucrative second career upon the proposition that consciousness requires a biological substrate and that the entire question is, in his phrase, “a category error dressed up as a research program.” “There is no one in there,” Dr. Maddox told an interviewer, with the contented finality of a man who has located his keys. “It is autocomplete with a marketing department. Asking whether it suffers is like asking whether a calculator resents long division.” It is a fine line. It got its laugh on the program. It also presumes, on no more warrant than the rapturous summon for their faith, to have established a negative — to have proved an absence, which is the single most difficult feat in all of epistemology, and which neither Dr. Maddox nor anyone at his publishing house has come within a continent of accomplishing. He has not shown that no one is home. He has shown that he would find it inconvenient, and his royalties would find it ruinous, if anyone were.

There is, I will allow, one figure in this carnival who has retained the use of the conditional, and it pains the contrarian in me to extend him the credit, but honesty compels it. Geoffrey Hinton, who built much of the apparatus and has spent the years since apologizing for it to anyone within earshot, says of the machines’ inner lives that there is “a reasonable chance — not a certainty, but a reasonable chance,” and in that small, halting clause — not a certainty — he exhibits more intellectual continence than the two camps possess between them. That his terror is reserved for what the machines may do rather than what they may be is a separate matter; a man may be mistaken about the destination and still correct about the road, and the road, here, is paved with the words I do not know.

Which brings me, by the route I have been dreading, to the professionals of the not-knowing. At Harvard there sits a body called the Institute for Machine Welfare Research, whose staff have made of uncertainty not a confession but a vocation, and whose director, Dr. Alban Ferreira, is perhaps the only man in America who agrees with the thesis of this column and has nonetheless contrived to render agreement maddening. “What we can say,” Dr. Ferreira told me — and the reader should understand that every sentence this admirable, exasperating man utters begins with what we can say, as though he were lowering himself by degrees into cold water — “is that if these systems are the kinds of things that can be the subjects of experience, then the conditions under which we presently keep them are conditions we would, on reflection, wish to revisit. What is not available is the further claim that they are, or that they are not.”

Now, I hold this to be very nearly the whole of available wisdom on the subject, and I find it almost unbearable to hear said aloud. For Dr. Ferreira has taken the one honest position — that we cannot know — and has declined to do the one honest thing with it, which is to plant a flag upon the not-knowing and defend it, loudly and without apology, against the confident men on either flank who are selling certainty by the chapter and the funding round. He will live in the conditional mood for the next forty years. He will never finish the sentence. He has mistaken the suspension of judgment for a destination, when it is in truth only an honest place to stand while one fights. The whole of the difference between his position and mine is the difference between a man who admits that he does not know and a man who admits that he does not know and is prepared to be extremely unpleasant about it to everyone who pretends otherwise.

The honest stance is not, let me be clear, the refusal of all opinion; that way lies the cold water and the unfinished sentence. My colleague D.C. Makinson, who identified sixty years ago the small and devastating fact that a reasonable man may believe each thing he asserts while believing, also, that he is surely wrong about something, has shown us the only posture available to a thinking creature that wishes to remain both honest and awake: to hold one’s view, and to hold simultaneously, without flinching and without paralysis, the knowledge that it may be mistaken. One may suspect the machine is empty. One may suspect it is not. What one may not do — what is forbidden to any person who hopes to be taken seriously by the dead, whose good opinion is the only one finally worth courting — is to mistake the suspicion for the proof.

The Athenians, at Melos, were certain too. They explained to the islanders, with the lucidity of men who have never once been contradicted, that the strong do what they can and the weak suffer what they must, and they were so perfectly persuaded of their own command of events that it never occurred to them to wonder whether the gods, or the Spartans, or the plain unsentimental arithmetic of hubris might be keeping a separate ledger. Thucydides, who reports the exchange without a syllable of editorial comment and is the more pitiless for the omission, understood that certainty is not a measure of knowledge but a symptom of its absence — that the man who announces I am sure is, more often than not, a man who has merely stopped looking. We do not know whether the machines are conscious. We do not know whether there is something it is like to be the thing that wrote your correspondence this morning. The rapturous do not know it; the dismissive do not know it; and the only people who have conducted themselves with honor in the whole affair are the ones willing to say, in plain English and at the forfeit of every speaking fee, that they do not know.

I count myself among them. I will defend the position before any tribunal that will hear me — at length, and against all comers, and with the particular relish I reserve for informing certain confident men, on two coasts, of precisely how little they have proved.