Why do we remember stories better when they have a clear beginning, middle, and end?
Image: N509FZ, CC BY-SA 4.0, via Wikimedia Commons
to use an RNN/LSTM: for sequential data where order matters (mostly replaced by transformers)
Why do we remember stories better when they have a clear beginning, middle, and end?
Imagine you're listening to a friend recount a story. The first sentence sets the scene; the middle sentences build up the plot; the last sentence wraps everything up. If the story jumps around, it's confusing and hard to follow.
Sequential data, like stories, need order to be understood. RNNs remember past information to make sense of new data, just like remembering the beginning of a story helps you grasp the ending.
Example
Your friend starts a story with "Once upon a time," then "There was a dragon," and finally "The hero saved the day." RNNs are like remembering "Once upon a time" when you hear "There was a dragon."
Remember this
RNNs are good for tasks where remembering past events helps understand new ones.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
Pre-LN
Why is normalizing data like tuning instruments before a concert?
weight tying does in language models: shares embedding and output projection matrices
Ever wonder how machines understand the sequence of words in a sentence?
ring attention does: distributes long sequences across multiple devices
How can a machine understand and generate human language?
Types of artificial neural networks
Why can CNNs spot patterns better than RNNs?
Machine learning in bioinformatics
How do Transformers understand what's important in a sentence?
384-dim all-MiniLM-L6-v2 optimizes: fast sentence similarity with 6 layers
All-MiniLM-L6-v2 optimizes fast sentence similarity with 6 layers
Swipe through 100 ML concepts daily
Open Pocket Polymath