Loading slide

Loading contents...

[░░░░░░░░░░░░░░░░░░][░░░░░░░░░░░░░░░░░░░░░░░░░░░░]0 / 21
<back>next

Module 7 Chapter 1

The Problem with Sequences

Language comes in ordered pieces. A sentence might contain one word or hundreds, and a word near the end may depend on a clue near the beginning.

How can a machine keep reading for as long as needed without losing those earlier clues?

One old answer passed a changing list of numbers from one word to the next. This was called a recurrent neural network. It handled sentences of different lengths, but its fixed-size list and its step-by-step reading created new problems.

In this chapter

  • Order matterswhy the same words can describe different events
  • No fixed lengthwhy a sentence needs room to keep growing
  • Carrying numbers forwardhow an RNN passes earlier information forward
  • A note that cannot growwhy old clues may weaken or disappear
  • The long pathwhy distant words struggle to reach each other
  • From translation to attentionhow a fixed sentence summary became direct access to earlier words
# citations