Decision Trees

Tree-based decision making.

What are Decision Trees?

Like a flowchart that makes decisions!

Each branch asks a question, leading to final decision.

Example Decision Process

Goal: Should I go to the beach in Miami?

Is it sunny?
├─ Yes → Is temperature > 70°F?
│         ├─ Yes → Go to beach!
│         └─ No → Stay home
└─ No → Stay home

How It Works

1. Find best question to split data
2. Create branches
3. Repeat for each branch
4. Stop when data is pure or max depth reached

Python Example

from sklearn.tree import DecisionTreeClassifier

# Weather data: [temperature, sunny]
X = [[85, 1], [70, 1], [60, 0], [50, 0]]
y = ['beach', 'beach', 'home', 'home']

model = DecisionTreeClassifier(max_depth=2)
model.fit(X, y)

# Predict
prediction = model.predict([[75, 1]])
print(prediction)  # 'beach'

Advantages

• Easy to understand
• No data scaling needed
• Works with numbers and categories
• Visual representation

Disadvantages

• Can overfit easily
• Unstable (small data change = different tree)
• Not great for very complex data

Hyperparameters

max_depth: Maximum tree depth
min_samples_split: Minimum samples to split
min_samples_leaf: Minimum samples in leaf

Remember

• Very intuitive algorithm
• Prone to overfitting
• Use Random Forest for better results