Definition
RNNs update a hidden state as they read each token or timestep, making them natural for sequences. Vanilla RNNs struggle with long-range dependencies due to vanishing gradients.
LSTMs and GRUs improved memory; transformers largely replaced RNNs for large-scale NLP.
In simple terms
An RNN is reading a book while jotting a running note, each new sentence updates the note before you continue.
Where you see it
- Early machine translation and language models.
- Time-series forecasting and sensor streams.
- Legacy speech systems before transformer ASR.
How it works
1.Read one step
Combine input with previous hidden state.
2.Update memory
Produce a new hidden state.
3.Emit output
Optional prediction at each step or at the end.
Why it matters
- RNNs taught the field how to model sequences, essential context for why transformers won.
Often confused
Nobody uses RNNs anymore.
They still appear in constrained devices, some speech models, and teaching curricula.