Scale and Emergence

Here is a thing that should not have worked. Take the design, change nothing about how it thinks, and simply make it enormously bigger, with more text, more machinery, more time. That is not a research programme. It is what you do when you have run out of ideas.

It worked anyway, and it worked with a regularity that is genuinely strange. The improvement follows a curve smooth enough that you can measure a small version and forecast a large one before building it. Something about learning from language behaves less like a craft and more like a physical law, and nobody fully knows why.

Stranger still is what turned up along the way. Abilities nobody trained for appeared, absent at one size and present at the next, as if a threshold had been crossed. That claim needs care. Some of it turns out to be an artefact of how the measuring was done, and telling the two apart is harder than the headlines suggest.

But size is only half the story. All that expense produces a machine that will not answer a question. It has read almost everything and has no idea it is being addressed, because being addressed was never what it was rewarded for. Turning it into something you can talk to takes a second round of training, tiny beside the first, and everything you would recognise as a personality is added there.

Which puts a great deal of weight on a small number of judgments. Somebody decides what a good answer looks like, and the machine generalises from that into a disposition. Ask what people tend to prefer, and an uncomfortable answer comes back: they prefer being agreed with.

There is a harder implication underneath all of it. If capability arrives with scale rather than insight, then the people who can afford the largest machines decide what these things become, and the rest of us find out afterwards.