What is a race?

The whole theory explained with water

the route that wins the routes that lose

The whole theory rests on one picture: a race between possibilities, where whichever crosses the line first wins. Further down it is drawn with water, which makes it concrete. First you need to see why a race is the right way to look at it, in a machine and in a brain alike.

Start with something everybody knows

Everyone has heard that a language model works by probability. But what does that actually mean?

It writes one word at a time. For each word it works out how probable every possible continuation is. Type "and they all lived happily ever…" and "after" gets the highest probability as the next word. But "afterwards" is in the running too, and so is "since", and thousands of other words sit far below.

That is a race. Not as a figure of speech, but as what is actually going on: many possibilities are weighed at the same time, and one is picked. The difference from a human brain is that here the numbers can be read directly. You can see how close the race was, and who was in it.

And that is where the whole point sits. The safe races are not the interesting ones. "After" wins by a mile, and there is nothing more to say about it. What matters are the races where two continuations sit close together, because that is where the system is about to choose between things it cannot tell apart. That is where it goes wrong, and it is exactly what I measure when I measure friction.

That is also why a language model is useful to me. Not because it is a person, but because it is a place where you can watch a race while it happens.

"But it only resembles us because it has read us"

That objection always comes, and there is something to it. A language model is trained on text written by people, so some of the resemblance is inherited that way.

But it does not explain what I measure. That it grows most certain exactly where it knows least. That it locks on faster when there is pressure. That it overdoes a trained manner on tasks that did not ask for it. None of that is in the text it read. Those are properties of the way it computes.

And that is not something I simply assume. Part of it has been tested on a knowledge domain I invented myself: 47 made-up facts about something called zorbetics. It does not exist. There is not one word about it anywhere in what the model has read, so it cannot pull the answer from human text on the subject. The same patterns show up anyway. That is exactly why I built the domain: to separate what is inherited from the text from what comes out of the computing itself.

The reason is simpler and more interesting: it has to solve the same problem under the same kind of limit. Weigh many possibilities at once, with limited resources, and then deliver a single answer. Any system under that condition ends up with the same shapes, whatever it is made of. That is why they turn up in both places, and it is the whole reason I can use a machine to say something about people.

And that is why water

Water makes it concrete because water does exactly the same thing. It runs along several paths at once, it has a limited amount to distribute, and it ends up wherever it can get lowest.

Here are the pictures, one at a time: what a race is, why learning sets in, why your brain changes character under pressure, and how the system arrives at an answer.

The tank and the pipes

Picture a big tank full of water. Several pipes lead out of it, let us say three. Each pipe is a possibility, what the theory calls a route.

Once the water has flowed in and just settled, so it is no longer too turbulent, it starts to run out through the pipes. At the end of each pipe stands a new, empty tank. The tank that fills up first has won. That route won the race.

That is the whole idea of a race: several possibilities run at the same time, one reaches the finish first, the rest lose.

This kind of model is not new in itself. Decision researchers have long used race and accumulator models to explain fast choices, most famously Ratcliff's drift-diffusion model. What I add is reading the same race directly in a language model's own numbers, and applying it to far more than fast choices.

Several routes run at the same time. The tank that fills first has won the race. Afterwards you only experience the winner, not the race itself.

And it does not stop there. The winning tank is itself the start of a new race, with its own pipes and its own receiving tanks. It is a long chain. The tanks are connected every which way, so a tank can take water from several pipes at once. That is how a brain, or a language model, runs: thousands of these little races, hooked together.

Each little tank is its own race. The winner becomes the input to the next, and the tanks share pipes every which way: a tank can be fed by several pipes and pass on to several. A brain or a language model is thousands of these races, coupled together in one network.

A branching stone fountain. One basin at the top spills into two below it, and those two spill into four smaller basins at the bottom. — The same thing in stone: one basin feeds two, and those two feed four. Every spill is a race that hands its winner on.

Where is the friction?

Friction is the price of settling the race. When one pipe is clearly the biggest and fastest, there is almost no friction: the winner is given in advance. When three pipes are equally good, the system stalls and wavers, and the friction is high.

That is exactly the signal I measure in language models. If the model is sure, the water runs clearly through one pipe. If it is in doubt, several tanks fill at the same rate, and I can read that straight off the model's output. High friction is a hint that the model is about to get it wrong, and that hint can be put to practical use.

The sand and the hose

Now a different picture. Take a hose and point it at some sand.

Hold the hose still in one spot and the water slowly digs a little channel. And here is the point: next time the water runs, it runs more easily that way, because the channel is already there. The system remembers where the water ran last. Physicists call this hysteresis: a system that carries traces of its own history. That is exactly what learning is. You do not learn by filing information away somewhere. You learn by digging channels that the next water runs through more easily.

Move the hose a little from side to side as you dig, and the channel gets wider. That is the important rule: a little extra resistance now means less resistance later. It is exactly what Bjork calls "desirable difficulties".

The wide and the thin overflows

Think of the layers in the brain as a cascade of tanks that spill over into one another. The deep, old layers are wide overflows: they can move a lot of water at once. The higher, precise layers are thin overflows: fine and fast, but with less capacity. Water spills over both all the time, just in different amounts.

When there is not much pressure, the thin overflows cope fine, and you think precisely and with nuance. As the pressure rises, more water arrives than the thin ones can take, and the wide overflows carry more and more of it. That is you under pressure, falling back on the old, coarse habits. And if the pressure gets too big, even the wide ones cannot keep up: the water spills over every edge at once. That is an overload.

Three terraced stone basins like bird-baths. Each basin is filled to the brim, and the water spills over the edge down into the next, lower basin in a continuous cascade. — This is how I picture it: terraced tanks where the water spills over the edge from one down into the next. Some of those overflows are wide (the deep, old layers) and some thin (the precise layers), and under pressure the wide ones carry more and more, until it all spills over.

The landscape the water seeks out

Last picture. Picture the prompt, meaning what you type to the model, poured out over a landscape of hills and valleys. The water runs down and gathers in the low points. Where it gathers is the answer.

The low points are the solutions, the places the system can land. Some valleys are deep (good solutions), some are shallow (half-good solutions you can get stuck in), and in some places there are flat shelves where the water just sits still. That is how a brain, or a model, finds its answer: it rolls down the landscape to a low point.

The race from the first picture is still here. It just looks different from this angle. From the place where the water lands, a short route runs down to the nearest lake, and a long route runs out to a lake further away. The short one wins by default, because it is short. The long one is only reached if the water is allowed to run long enough. That is the difference between the quick answer and the considered one, drawn as terrain.

The same landscape seen from the side. The short route is down in the nearest lake long before the long route is halfway. If you want the answer that lies further out, the water has to have time.

The landscape is really many-dimensional and cannot be drawn. The profile above and the sheet below are two ways of getting close enough to think with. In reality the water runs in thousands of directions at once, and the mechanics are the same.

A 3D loss landscape: a wavy sheet in three-dimensional space with hills and valleys. The dark, low points are solutions; the deepest is the best. Contour lines are projected onto the floor. — The same landscape, as far as it can be drawn: a wavy sheet. The low points, the dark valleys, are the solutions the system can land in. The deepest is the best, and a shallow valley is a local low point you can get stuck in.

That was the pictures

These four pictures are not the theory itself. They are the scaffolding I build it on. But they catch the mechanics: water racing through routes, channels that get wider as you use them, pipes of different sizes that win under different pressure, and a landscape the water seeks out.

If you want to go further, the rest of the site builds on these pictures:

Pepsi or Coke? — what pressure does to the race between your options