RNN & LSTM Networks

Question 1

Recurrent Neural Networks (RNNs) are designed for…

Accepted Answer

Sequential data with a hidden state over time — RNNs process sequences one step at a time, carrying a hidden state that summarizes prior inputs — giving them memory.

Question 2

The 'hidden state' in an RNN functions as…

Accepted Answer

Memory summarizing prior steps in the sequence — At each step the hidden state is updated from the current input and the previous state, encoding sequence history.

Question 3

Vanilla RNNs struggle with LONG sequences because of the…

Accepted Answer

Vanishing gradient across many time steps — Gradients shrink exponentially as they propagate back through many time steps, preventing the network from learning which early inputs matter — the vanishing gradient problem. (Exploding gradients also occur but are controlled separately by gradient clipping.)

Question 4

LSTMs address the vanishing gradient problem using…

Accepted Answer

Gates that provide a gradient-friendly path across time — The LSTM's gated cell state provides a near-linear path for information across time, preserving long-term dependencies.

Question 5

The three classic gates in an LSTM are…

Accepted Answer

Forget, input, and output gates that control state — The forget gate drops old info, the input gate writes new info, and the output gate decides what to expose — all learned.

Question 6

The LSTM's CELL STATE acts as…

Accepted Answer

A long-term memory conveyor carrying information forward — Information flows along the cell state with minimal interruption (additive updates), which is why gradients survive over long sequences.

Question 7

A GRU (Gated Recurrent Unit) differs from an LSTM by…

Accepted Answer

Using fewer gates and merging the cell state — GRUs merge gates and state, giving a lighter model with similar performance and fewer parameters than LSTMs.

Question 8

RNNs are trained using…

Accepted Answer

Backpropagation through time, unrolling the recurrence across steps — BPTT unfolds the recurrence into a deep feedforward graph over the sequence and applies backprop, summing gradients for shared weights.

Question 9

Weights in an RNN are…

Accepted Answer

Shared across every time step in the sequence — The same recurrent weights apply at every time step, letting RNNs handle variable-length sequences with a fixed parameter set.

Question 10

A bidirectional RNN processes the sequence…

Accepted Answer

Processed in both forward and backward directions — Two RNNs (forward and backward) are combined so each output sees the whole sequence — useful when the full input is available (not for live generation).

RNN & LSTM Networks Visualizer

Network Architecture

Playback Settings

Current State

Current Input

Hidden State

Gate Values

What does each LSTM gate do?

Test your knowledge

Recurrent Neural Networks (RNNs) are designed for…

Learn these first

Interview Questions

References & Further Reading

Love this visualization?