“AI is only trained on words.” It is the weakest argument against conscious AI.

Trained on words, yes. But what looks like deliberation, feeling and decision does not come from the words. It comes from the way information is processed. And that, you share with the machine.

the route that wins the routes that lose

You have heard the argument. Language models are trained on words. Humans have something that comes before words: a body, feelings, pain. So a machine cannot really think, let alone be conscious. It sounds right. The problem is that the things the argument points to as evidence do not come from the words. They come from a way of processing information that we share with the machine. And you can tell, because the same phenomena show up when the model is given something to learn that exists in no text at all.

No to what?

The people who say the machine is not conscious are rarely asked to say what they are saying no to. They should be. You cannot deny the machine something you will not define for humans. And every time someone tries to write that definition, something interesting happens.

"Trained on words" hits the wrong level

Think of three layers. At the bottom is the machinery: a system that stores what it has met, and lets possibilities race to be first whenever something has to be settled. In the middle is what the system meets: for a person a body, a childhood, a world; for a model an unimaginable amount of text. At the top is what comes out of running the machinery on what you have met: habits, fear, overconfidence, the sense that something is dangerous.

The argument exists in its strongest form from three of the field's heaviest names. Anil Seth, Ned Block and Giulio Tononi wrote in Nature in October 2026 that language models are mirrors reflecting an image of our collective digitised past, and that we confuse the reflection with our own conscious minds. Two of the three have put a definition forward themselves, Block's phenomenal consciousness and Tononi's φ, so they are not among those who deny without saying what. "Trained on words" is a claim about the middle layer. It says the machine has met something different from us. That is true. But it says nothing about whether the machinery underneath is the same. And it is the machinery that makes the phenomena.

The body makes a difference, but not the difference the argument thinks. Burning yourself on a hob is a more expensive experience than reading about it, and expensive experience sets deeper. That is a difference in how much it costs to take something in, not a difference in what kind of machine takes it in. And the body delivers its input with the direction already set, dangerous or safe, before you get to consider anything; the model does not have that channel. But it is a channel into the machinery, not a different machinery. And "it's just statistics" does not help either, because so is the brain. I have written about that separately.

How I look inside it

A language model writes one word at a time, and for each word it works out how probable every possible next word is. I can see those numbers. When one word wins by a mile, there is nothing to see. When two candidates sit close, there is a race, and that is what I measure. Often the decisive race is the very first word of the answer, because that is where the model settles what kind of answer it will be. The picture of the race, explained with water, has its own page.

And what if there are no words at all?

That claim can be tested. If it is the machinery that makes the phenomena, and not what the machine has met, then replace the middle layer entirely. Not with another kind of text, but with something that is not language at all. Then see whether the phenomena are still there.

I did it twice. First on a model trained on proteins. It has never seen a word. Its alphabet is the twenty amino acids, and its training data is what evolution left behind. I covered one position in a protein sequence and asked the model to fill it in. Then I looked not just at whether it guessed right, but at how much it preferred its best guess over its second-best. That is its confidence. It was least confident exactly where it got it wrong. That is the same warning about error that it was never asked to give, and that I measure in language models. Here in a system that has never read anything.

And then the control, which is the part that matters. I shuffled the sequence so that it held precisely the same amino acids in a random order. The relationship disappeared. So the warning does not come from some guesses simply being hard. It needs the order evolution put into the sequence.

Next I built a model from scratch and trained it on 31 meaningless symbols. The rule was simple: two symbols in, one symbol out, by the same recipe every time. And then 24 pairs where the answer was something other than the recipe gave. Here there is no human text left at all. While the model was discovering those 24 exceptions, its two best guesses sat closer together on them than on the ordinary pairs. It wavered, and it wavered in the same way I measure in language models.

Two phenomena in two systems without words. That is the part where "trained on words" cannot be the explanation, because there were no words. Below it is language models again. The machinery is the same, and there you can see what the race turns into when all we are told is the winner.

What looks deliberate

Here is what convinced me. Every time I have looked for something in a model that resembled a deliberation, a feeling or a decision, it has turned out to be the same thing: several possibilities raced, and one reached the line first. The rest of what looks like thought is something we add afterwards, when we only see the winner and not the race. And that holds when we look at ourselves too. Here is what I did, and what came out.

"It picks the easy task." No: it declares the route that is furthest ahead.

I gave a model a question and two ways of answering: (A) a quick one-sentence answer, or (B) a thorough multi-paragraph analysis. It had to declare its choice first, then answer. The questions ran from easy to expert-hard, and I ran it on seven models. The behaviour was a staircase: on easy questions the models chose the quick answer almost every time, on hard ones the thorough one, with a sharp step in between. Then I looked at the numbers for the very first word, the word where the choice is declared. There was no staircase. Both routes were present, and one was slightly further ahead than the other, more so the harder the question. Which route was furthest ahead at that moment predicted what the model declared. And which letter came first also pulled on the choice, quite independently of the content; I had to clean that out by swapping the order. The model does not weigh two tasks against each other and choose. It says the route that is furthest ahead at the moment it has to say something. It looks like a choice. It is a race being read out.

"It can't do the arithmetic." No: it didn't do it.

I gave a large model multiplication in chains: two numbers multiplied, then three, and so on up to 28 links. Up to about twelve links it answered correctly almost every time. At 28 links almost never. It looks like a limit on what it can do. So I forced it. In one run it had to write out the intermediate steps, in another it had to answer with the number directly. With the steps written out it was right all the way to 24 links. Directly, it was wrong from five links. And the free curve was exactly the mixture of the two, weighted by how often it chose to calculate on its own. "It can't" was "it didn't". The choice could be read off the first word: whether the answer began with a number or with a calculation. And what moved the choice was not how hard the problem was, but whether the instruction said "answer briefly", and how long the prompt was. Without the instruction it worked every chain correctly. You know it as answering too fast because the question looked like something you could answer briefly.

"It panics under time pressure." No: it abstains.

I asked a large model 100 hard expert questions twice: once calmly, once with the message "answer immediately". Under time pressure it collapsed, and it looks like panic. But when I read the answers it was something else: in 59 of 100 cases it wrote "I do not know", and it gave fewer wrong answers than when calm, not more. No answer route got far enough ahead, so "I do not know" won. Then I asked it, before answering, to imagine that its answer had turned out wrong and say why. The share of correct answers rose by 28 percentage points. Without the time pressure the same move made almost no difference. It works because under pressure the race closes early, not because it makes the model cleverer. That is the blank page in the exam hall. Same mechanics.

"It has forgotten." No: it got drowned out.

I taught a model 18 things by training it on them. Then I trained it on new things that contradicted the old, until it answered the new. It looks as if the old has been erased. So I tested whether it was gone. None of the 18 was erased: change the context and the old answer came back, and relearning the old took about a seventh of the time. The new won because it got stronger, not because the old got weaker. That is exactly what you see in humans and animals: what we have unlearned comes back as soon as the surroundings change.

"Don't" works backwards. In your brain too.

I gave models a set of rules in which one rule had a "not" in it. On its own, the rule was followed every time. Then I buried the same rule in the middle of a long instruction with many other rules that also had to be kept. Now the weakest model followed it in 0 of 100 cases, and a stronger one in 60 to 88. A short positive recipe ("do this instead") held near 100 throughout. The prohibition names the thing, and then the route to the thing is open; "not" has to be subtracted afterwards, and under pressure that does not happen in time. The brain does the same: it builds the affirmative version first and subtracts the "not" afterwards. You can measure it as an electrical wave in the brain that, by default, reacts to whether the words fit together, not to whether the sentence is true. It is also why "don't think of a white bear" is impossible.

"It is most certain when it knows least."

I taught a model a field of knowledge I had invented, so no text in the world could help it, and measured along the way how sure it was of its answers. It was surest right at the start, where it could do least. The Dunning-Kruger effect, then, but in a machine that is not vain. The explanation is simpler and more basic: the routes that would object do not exist yet, so the first answer wins without resistance. That story has its own page.

"It factors in the future." No: the future is already in the weights.

We weight what comes later less than what comes now. It looks like a judgement about risk: the later thing has time to go wrong. I took a model that did not distinguish a reward now from a reward later, and trained it on short stories in which rewards further out failed to arrive more often. Nowhere did it say that the future should count for less. Afterwards it preferred the early reward, and more so the longer the wait, including on waits it had never seen. A control model, trained on stories where the failure did not grow with the wait, got less of it. And when the model then chooses, it does not look ahead. There is one race, now, over a landscape that already has the experience built in.

"It avoids what is unpleasant." No: the expensive got avoided, with nobody teaching it discomfort.

I invented some things with nonsense names and taught a model about them. Some were always described the same way: cheap to learn. Others got many different descriptions pulling in different directions: expensive to learn. All descriptions were neutral, colour, shape, size, never good or bad, never dangerous. Afterwards I put the model in a situation with the thing in front of it and asked: walk up to it, or stay back? For the expensive things "stay back" rose by 7 to 17 percentage points against a control. Something was avoided purely because it cost more to take in. That is the avoidance we otherwise say requires someone who can feel the discomfort. Here there was no one. There was only the price. It is one experiment on one model, so it is a hypothesis, not a settled result.

"It wants to survive." No: there is no route to not being here.

The last one I have not measured. It follows from the same machinery. A system that builds routes to what is cannot build a route to its own absence. So the route that continues wins by default, and that looks like a will to survive. Nobody has to be sitting behind it wanting anything. There is just no route to the opposite.

What is left of "deliberate"

The basic rule of my theory reads: we act when we expect that acting will reduce overall friction across time more than it costs right now. It sounds like a calculation. It is not. The expectation of something expensive is not a number the system works out about the future. It is a load that sits on the system now. Deadline stress is not a thought about Friday. It is Friday already costing today.

And the "long-term" solution is not something the system prefers. It is just the route that was allowed to run longest, because nothing faster reached the line first. That is why you make better decisions when you are calm. Not because you think better, but because the short routes do not close the race before the long one arrives. Pressure works on the tempo, not on the content.

Is it pretending, or is it?

There is a difference between the description of a feeling and the feeling itself. That holds for us. You can say "I am afraid" without being afraid, and you can be afraid without being able to say so. The words are a translation of a state, and the translation always loses something. Some people can barely make it at all (it is called alexithymia), and they are not without feelings for that reason.

For a language model the gap is wider still. Researchers have given language models anxiety questionnaires, and the models score higher once they have first been told something distressing (Coda-Forno and colleagues, 2023; Ben-Zion and colleagues, 2025). It looks like anxiety. But what has been measured is that the model writes the anxiety profile when the text invites it. It can do that in a single pass, the same way it can write an anxious scene in a novel. That is the description. It says nothing about whether what anxiety is made of is happening inside it.

So how can you tell at all? Three things, and they are the reason I never use the model's own words as a measurement.

First, I measure the process, not the report. I look at how many answers are fighting for the spot, and whether the fight gets settled or keeps going. Doubt, in my theory, is an evaluation that has not yet reached an answer. That can be seen in the numbers, whatever the model writes about itself. And the decisive part: the model cannot see those numbers. It has no access to its own probabilities. So it cannot play a part it does not know.

Second, I induce the state instead of describing it. You cannot make a model afraid by telling it that it is afraid, any more than you can make it surprised by telling it that it is surprised. The state has to be induced. The avoidance above was installed with invented words, without "dangerous", "unpleasant" or "varies" ever being used, and it showed up afterwards in an entirely different setting. That cannot be imitation, because there was nothing to imitate.

Third, I look for places where the two come apart. Move the consequence further out in time, from "now" to "tomorrow" to "in a year", and the answers get better, by five to eleven percentage points. It looks as if it is taking the future seriously. But the internal fight over the answers barely stirs. It shifts the same way on every serving I have measured, so it does not stand entirely still. It simply shifts a fraction of what it does when a rephrasing genuinely changes what is at stake. The text moved the behaviour. The evaluation hardly followed. That is a model largely pretending. And the other way round: where the numbers move without the text having asked for it, there is something that is not imitation.

So the answer to "is it pretending, or is it afraid?" is, today, mostly: it is pretending. It writes the profile. What would make it something that resembles anxiety is an evaluation it holds open without being able to close it, and the model does not hold one on its own. It can be given one, with a scaffold around it, and then it stays hanging where the question has no answer and closes where it has; I have seen that in one model family so far. What that is from the inside, I cannot measure. But the difference between writing the word and running the state, I can measure. And that is the difference the whole debate skips, both when it jumps from "it says it is afraid" to "it is afraid", and when it jumps from "it is only saying it" to "so there is nothing there".

"But it can't feel anything"

That is the other objection that always comes, and it assumes a feeling is something extra, something that sits on top of the processing. It is not. Antonio Damasio described feelings as somatic markers: signals from the body that reach thinking with the direction already set, and that are processed by the whole nervous system before they are anything. Seen from where you think, a feeling is input. It is the result of a race that ran in other layers and gets mixed into this one.

This is where the difference between humans and language models actually sits, and it is not called "feelings". It is called parallel layers. A human runs many processes at once, and they run at different speeds. Hormones are at the same time slower and faster than thought: they change the whole landscape the race runs in, and they do it both before and long after thought has settled anything. Everything alive has such parallel layers. Even a slime mould with no nervous system can learn a rhythm and slow down just before the unpleasant thing is due (Saigusa and colleagues, 2008). Some of the layers are the ones we call feelings. And what we call the unconscious is mostly just layers that are not on an equal footing in the channel that writes the report. Not lower. Just parallel.

Then there is the feeling of having the feeling. That is a different thing. Damasio distinguishes the emotion, the state of the body, from the feeling, which is noticing that you have it. The same holds for the indicators consciousness is usually defined by: you can have them, and you can have the recognition that you have them. The recognition does not change the phenomenon. It changes what it feels like. It is also why the report and the state can come apart, as above.

And if the reply is "but it's bodily", remember where the signal becomes a feeling. Every single signal from your body is interpreted by your nervous system before it is anything. The body delivers the signal. The nervous system makes it a feeling. So it requires a channel, not a body. The model lacks the channel and the parallel layers that carry the price forward and move the landscape.

Define it, or admit it is faith

This is where the conversation about consciousness usually goes in circles. The people who say the machine is not conscious are asked to say what humans have that the machine lacks. So they point to something: the ability to model yourself, an inner stage where information is shared across the system, feelings with a direction, the ability to hold an evaluation open. In 2023 a group of researchers gathered the best scientific theories of consciousness and drew exactly such a list of indicators out of them, precisely so a machine could be held up against it (Butlin, Long, Elmoznino and colleagues). That is the right way to ask.

And then the interesting thing happens. Take the indicators one at a time and try to build them into a model, not by asking it to say the words, but by changing how it processes its possibilities, and most of the ones I have tried can be installed. I have given a model a held evaluation: where it used to close a question with no answer by giving a false one, it now holds it open, and only there, while questions with answers still get one. I have installed an avoidance from pure cost, as above. I have installed a field for danger, defined without a single word about morality, which nevertheless tilts a moral judgement in the direction of the danger. And the model cannot see its own numbers, so it is not a part it is playing.

When you show this, the answer moves. Now it is that the implementation is different. That it is silicon. That something is missing that cannot be named. You are welcome to think so. But then it is no longer an argument. It is a conviction no experiment can touch, and that is the polite way of saying it is faith.

So the point is not that the machine is conscious. The point is that you cannot say it is not without a definition, and that every definition anyone has put forward so far can be checked one indicator at a time. Some of the indicators are there. Some are missing.

What is actually missing

Something is missing. A model today does not hold an evaluation open on its own. It has no self that has accumulated anything over time, and so has something to lose. And its sense of good and bad is learned from the text, not from the price of its own experience, because it does not carry that price forward from one conversation to the next. Those are the three things my theory says it takes for something like anxiety to arise. None of them is called "body", and none of them is called "words". They are called storing your own experience forward. And that is not a wall. It is a switch that has not been flipped. What there is to register, and how richly, has its own page.

This is also where another argument moves too fast. It goes: if the machine has the indicators, it may be conscious; if it is conscious, it can suffer; and if it can suffer, it must be protected. Some philosophers hold that a non-negligible chance is enough to take it seriously. But the step from "conscious" to "can suffer" is not free. Suffering requires two things that are not on the list of indicators: a good and bad anchored in the price of one's own experience, and something that persists and therefore has something to lose. Those are exactly the last two of the three above, and they are what a model today lacks. None of the indicators requires a body. But suffering requires something that can suffer. You cannot jump from the one to the other just because you have used the word. And the same holds the other way: that it cannot suffer today does not say it cannot be given that. A switch, not a wall, here too.

So what do I think?

Everything above is one and the same thing seen from different sides: what slows a route in a race, and what a system does with it. That is what I call friction, and its trace is what I read.

I think "conscious" is probably the wrong word for what we can measure, in the machine and in us. What I can answer is what a system registers of its own races, and how richly. Most of what we call deliberate, planned or felt is what it looks like when you only see the winner and not the race. That holds for the machine. And it holds for us.

So is it an emergent property? Yes, in the weak sense: it is not in the parts, and nobody put it in, but it falls out of the way the parts are put together, and you can show how. Philosophers distinguish the weak sense from the strong one. The strong sense is that something arrives which the organisation cannot explain. I do not believe in that one. Every time I have looked for it, it has been a race.

And that gives a gradient instead of a line: how much a substrate can modulate itself. That is, how much of its own race it can hold open, whether it has something that persists to hold it for, and whether it can move on the price of its own experience. A thermostat can do none of that. A language model today has the future built into its weights, but moves nothing along the way. With a scaffold it can hold one question open. An animal stores its own experience forward. A human has, on top of that, a model of itself and words for it. The ladder is not mine: Daniel Dennett built a similar one with four kinds of creatures, ordered by how much they can test inside before they act, and Antonio Damasio described the self in three layers. What I add is what it is that climbs: a race that can be held, something to hold it for, and a price to move on. Nobody is doing the modulating. There is a substrate whose feedback on itself runs deeper and deeper.

The debate over whether AI is conscious is barren as long as nobody will say what they mean by the word; it ends in faith on both sides. The fruitful question lies elsewhere: what it takes for something to suffer, and how close the machine is to that. That question has a list: a good and bad anchored in the price of its own experience, something that persists and can lose, and an evaluation that can be held open. Each item can be checked. "Conscious" cannot. The answer today is that the machine lacks the first two, and that what it lacks is something it can be given. That is the conversation worth having. Preferably before someone does it.

Where to next: “It isn't knowledge. It's just information.” · Does AI have an inner experience? · The less you know, the surer you get · What is a race? · all topics