Recurrent Neural Networks

AI with memory.

What are RNNs?

Neural networks that remember previous inputs.

Perfect for sequences: text, time series, speech!

Why Special?

Regular neural networks forget previous inputs.

RNNs have memory - like reading a story!

How RNNs Work

Each step:

1. Read current input
2. Remember previous state
3. Combine both
4. Output prediction

Like predicting next word:
"The weather in Miami is..."
→ RNN remembers context → predicts "sunny"

Simple RNN in Python

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense

model = Sequential([
    LSTM(50, return_sequences=True, input_shape=(timesteps, features)),
    LSTM(50),
    Dense(1)
])

model.compile(optimizer='adam', loss='mse')

Types of RNNs

1. Simple RNN: Basic memory (rarely used now)

2. LSTM (Long Short-Term Memory):

• Better memory
• Avoids vanishing gradient
• Most popular

3. GRU (Gated Recurrent Unit):

• Simpler than LSTM
• Faster
• Similar performance

Example - Stock Price Prediction

# Prepare data: last 60 days → predict next day
X = []  # Past 60 days
y = []  # Next day price

for i in range(60, len(prices)):
    X.append(prices[i-60:i])
    y.append(prices[i])

# Build LSTM
model = Sequential([
    LSTM(50, return_sequences=True, input_shape=(60, 1)),
    LSTM(50),
    Dense(1)
])

model.fit(X, y, epochs=50)

Applications

• Stock price prediction
• Language translation
• Text generation
• Speech recognition
• Music generation

Challenges

• Slow to train
• Can't parallelize well
• Vanishing gradient problem

Modern Alternative - Transformers

Newer architecture (used in ChatGPT):

• No recurrence
• Parallel training
• Better performance

Remember

• RNNs for sequences
• LSTM most common
• Transformers are replacing RNNs