Introduction to Deep Learning

Traditional ML hits a ceiling with complex data like images, speech, and text. Deep Learning breaks through that ceiling.

What is Deep Learning?

Deep Learning = Neural networks with many layers.

"Deep" refers to the depth (number of layers), not complexity. These layers automatically learn hierarchical features from raw data.

**Traditional ML:** ``` Raw Data → Hand-crafted Features → Model → Prediction ```

**Deep Learning:** ``` Raw Data → Neural Network → Prediction (Features learned automatically) ```

The Neuron

Everything starts with a single neuron:

```python import numpy as np

def neuron(inputs, weights, bias): # Weighted sum z = np.dot(inputs, weights) + bias # Activation function output = 1 / (1 + np.exp(-z)) # Sigmoid return output

Example inputs = np.array([0.5, 0.3, 0.2]) weights = np.array([0.4, 0.6, 0.8]) bias = 0.1

result = neuron(inputs, weights, bias) print(f"Neuron output: {result:.4f}") ```

Building a Simple Neural Network

```python import tensorflow as tf from tensorflow import keras

Simple neural network model = keras.Sequential([ keras.layers.Dense(128, activation='relu', input_shape=(784,)), keras.layers.Dense(64, activation='relu'), keras.layers.Dense(10, activation='softmax') ])

Compile model.compile( optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'] )

Train model.fit(X_train, y_train, epochs=10, batch_size=32, validation_split=0.2) ```

Key Components

| Component | Purpose | |-----------|---------| | Layers | Transform data step by step | | Weights | Learnable parameters | | Activation | Adds non-linearity | | Loss | Measures prediction error | | Optimizer | Updates weights to reduce loss |

Activation Functions

```python # ReLU - Most common for hidden layers def relu(x): return np.maximum(0, x)

Sigmoid - For binary classification output def sigmoid(x): return 1 / (1 + np.exp(-x))

Softmax - For multi-class output def softmax(x): exp_x = np.exp(x - np.max(x)) return exp_x / exp_x.sum() ```

When to Use Deep Learning

**Use Deep Learning:** - Images, video, audio, text - Very large datasets (millions of samples) - When feature engineering is hard - State-of-the-art performance needed

**Stick with Traditional ML:** - Tabular data (often) - Small datasets (<10K samples) - When interpretability matters - Limited compute resources

Key Takeaway

Deep Learning automates feature learning through layered neural networks. It excels at unstructured data (images, text, audio) but requires lots of data and compute. Start with simple architectures, understand the basics, then explore specialized architectures like CNNs and RNNs for specific tasks.