Loading slide

Module 12 Chapter 2

The pattern it cannot learn

The machine from the last chapter worked. Eight bottles went in, four numbers came out, and those four numbers held a rule about wine that nobody had written down.

This chapter breaks it.

Not with a trick, and not with a harder sum. With an ordinary case of wine, of a kind any merchant would recognise, that the machine you just built cannot price. Not badly. At all. More bottles will not help, more passes will not help, and a better learning rate will not help.

You will watch it try, and you will watch it give up on its own.

Then you will find out what is missing, add it, and watch the familiar wine machine calculate its first learned hidden signals. That addition creates a new problem: the bottle gives us a correct price, but no correct answers for the hidden units inside the machine.

You will build the missing calculation one visible step at a time. Only after its route makes sense will we give it its technical name.

In this chapter

  • A pattern one layer cannot learnwhy four wines make the weights cancel each other out
  • Adding a layer in the middlehow hidden units can recognise combinations
  • What the hand-built fix leaves outwhy supplied weights are not learning
  • Sending the mistake backwardshow the final error reaches every earlier weight
  • From effect to changehow small steps update weights and biases
  • What more layers make possiblewhy the same learning process scales beyond wine

Loading contents...

[░░░░░░░░░░░░░░░░░░][░░░░░░░░░░░░░░░░░░░░░░░░░░░░]0 / 38
<back>next