Support Vector Machines

Draw the best dividing line.

What is SVM?

Finds the best line/plane to separate categories.

Like drawing a fence between cats and dogs!

Simple Concept

Imagine separating red and blue dots on paper:

Goal: Draw line with maximum space on both sides

SVM finds the line with biggest "margin" (gap).

How It Works

1. Find possible separating lines
2. Choose line with maximum margin
3. Use support vectors (closest points)

Python Example

from sklearn.svm import SVC

# Customer data: [age, purchases]
X = [[22, 2], [25, 3], [45, 10], [50, 12]]
y = [0, 0, 1, 1]  # 0=Regular, 1=Premium

model = SVC(kernel='linear')
model.fit(X, y)

# Predict
customer = [[35, 7]]
result = model.predict(customer)
print("Premium" if result[0] == 1 else "Regular")

Kernels

Linear: Straight line separation
RBF: Curved boundaries (most popular)
Polynomial: Complex curves

When to Use

• Clear separation between classes
• Medium-sized datasets
• High-dimensional data

Advantages

• Effective in high dimensions
• Memory efficient
• Versatile (different kernels)

Disadvantages

• Slow for large datasets
• Hard to interpret
• Sensitive to feature scaling

Real Applications

• Text classification
• Image recognition
• Bioinformatics

Remember

• Scale features first!
• Start with RBF kernel
• Great for clear separations