Introduction to Deep Learning

Traditional ML hits a ceiling with complex data like images, speech, and text. Deep Learning breaks through that ceiling.

What is Deep Learning?

Deep Learning = Neural networks with many layers.

"Deep" refers to the depth (number of layers), not complexity. These layers automatically learn hierarchical features from raw data.

Traditional ML:

Raw Data → Hand-crafted Features → Model → Prediction

Deep Learning:

Raw Data → Neural Network → Prediction
(Features learned automatically)

The Neuron

Everything starts with a single neuron:

import numpy as np

def neuron(inputs, weights, bias):
    # Weighted sum
    z = np.dot(inputs, weights) + bias
    # Activation function
    output = 1 / (1 + np.exp(-z))  # Sigmoid
    return output

# Example
inputs = np.array([0.5, 0.3, 0.2])
weights = np.array([0.4, 0.6, 0.8])
bias = 0.1

result = neuron(inputs, weights, bias)
print(f"Neuron output: {result:.4f}")

Building a Simple Neural Network

import tensorflow as tf
from tensorflow import keras

# Simple neural network
model = keras.Sequential([
    keras.layers.Dense(128, activation='relu', input_shape=(784,)),
    keras.layers.Dense(64, activation='relu'),
    keras.layers.Dense(10, activation='softmax')
])

# Compile
model.compile(
    optimizer='adam',
    loss='sparse_categorical_crossentropy',
    metrics=['accuracy']
)

# Train
model.fit(X_train, y_train, epochs=10, batch_size=32, validation_split=0.2)

Key Components

Component	Purpose
Layers	Transform data step by step
Weights	Learnable parameters
Activation	Adds non-linearity
Loss	Measures prediction error
Optimizer	Updates weights to reduce loss

Activation Functions

# ReLU - Most common for hidden layers
def relu(x):
    return np.maximum(0, x)

# Sigmoid - For binary classification output
def sigmoid(x):
    return 1 / (1 + np.exp(-x))

# Softmax - For multi-class output
def softmax(x):
    exp_x = np.exp(x - np.max(x))
    return exp_x / exp_x.sum()

When to Use Deep Learning

Use Deep Learning:

• Images, video, audio, text
• Very large datasets (millions of samples)
• When feature engineering is hard
• State-of-the-art performance needed

Stick with Traditional ML:

• Tabular data (often)
• Small datasets (<10K samples)
• When interpretability matters
• Limited compute resources

Key Takeaway

Deep Learning automates feature learning through layered neural networks. It excels at unstructured data (images, text, audio) but requires lots of data and compute. Start with simple architectures, understand the basics, then explore specialized architectures like CNNs and RNNs for specific tasks.