The Neuron Idea

A hand-written rule cannot be talked out of anything. It does exactly what it was told, and when it is wrong it is wrong in the same way forever, until a person opens the file and changes it. Every approach that came before failed on some version of this.

So the thing we want has to be able to change itself. Not its instructions, nobody knows how to write those, but something smaller and duller: a set of numbers that decide how much each piece of evidence counts. Turn those numbers up and down and the same machine reaches different conclusions from the same input. Now being wrong is useful, because being wrong tells you which direction to turn.

That is the whole idea, and it is almost embarrassingly small. It is a bit of arithmetic with some adjustable dials in it, closer to a thermostat than to a brain. The word neuron is borrowed from biology and flatters it considerably.

What makes it worth dwelling on is that this small thing genuinely learns, from examples, without anyone stating the rule. And that it has a hard limit, one that was proved decades ago and nearly ended the field. The limit is not that it is too simple to be interesting. It is more specific and stranger than that, and understanding exactly where the wall sits is what tells you what to build next.